I had the same problem as you, with my GeForce 7600GT! The problem was, that OpenGL didn’t work. And without the openGL acceleration, it can’t be played back in realtime, not even if it’s just plain video. I have to admit i don’t know what the AJA card is XD so i hope i haven’t misunderstood the issue
But the trick that did it for me was completely reinstall my graphics driver. The thing is, you have to uninstall your current driver, before installing the new one off nVidia’s website. I used a tool called DriverCleaner i think. Reinstalling it completely should fix the openGL driver, so after effects can find it, and thus enable openGL acceleration and everything will be much faster!
Try it even if you do have the newest driver, that’s what did it for me!