I’m on the T3i, and I run into this problem all the time when I use my 50mm 1.8. I also get some weird lighter, hazier spots in the middle of the frame when I don’t strictly control the light entering the lens. My hunch is that you’re running into the same problem.
I’m using an old FD 50mm from 1985 with an adapter. (Don’t laugh, it was free!) I’ve found that the lack of modern coatings on the glass, plus adding two more layers of glass in the adapter brings the picture quality down quite a bit from my kit lenses. Add an ND into the equation, and things just get muddy. That’s why I only use my 50mm for very specific shots where I need something very stylized.