I was seeing an issue like this as well, but on a PC. The fix for me was to open the DeckLink control panel and turn off “Remove field jitter when video is paused”
This seemed counter-intuitive to me but it worked like a charm for previewing in AfterEffects. Now things are crystal-clear and “full-res” looking again. I musth have flipped this and forgot about it while exploring the control panel.