I’ve had the same problem with my SDI Sony monitor and without, I know exactly what you are seeing and I was never able to fix it either, what I did do is only use Livetype and Boris Red for titling, any FCP title or photoshop file would look jaggy after rendering but perfect before, very frustrating. I have a Dual G-5 with aja IO