Dead sounding low E string on Baja Tele?

I'm wondering if anyone else has had a similar problem, or has any solutions for fixing this:

The low E String on my Baja Tele sounds dead throughout the range of the entire string (which makes me assume it isn't a nut problem).

It sounds almost like that string is a flatwound compared to the others which sound normal. It isn't a terrible sound, but I'm just wondering what the cause is.

I tried different saddles and I even tried filing a groove in one of the saddles, but it seems the same which makes me think it is also not the saddles.

Could it be the neck relief? If so, I'm guessing I would need to add more relief? The string isn't buzzing at all though and it just sounds a bit muted.

I also tried raising and lowering the saddle. Lowering it seemed to help a tiny bit, but going too low caused buzzing. It is at a comfortable height that matches the rest of the strings so I'm not sure why only the low E has problems.

The only other thing I can think of is adding a shim, though I'm not sure what that would help either.