If your footage is HDV its likely the format that’s causing problems in After Effects.
HDV is an MPEG2 format which means it does not contain whole frames of information and has to be transcoded before After Effects can play it back.
After Effects isn’t good at dealing with footage that needs on-the-fly transcoding.
As Dave suggested, you’re better off rendering the files out of Premiere in an AE friendly format (uncompressed, ProRes, etc) and using these in After Effects.
Premiere can deal with a variety of footage types far better than After Effects can, particularly with the Mercury Playback Engine which uses the CUDA on your Quadro card for the heavy lifting of transcoding. After Effects doesn’t use the Mercury Playback Engine.
As the inimitable Mr. LaRonde says:
Dave’s Stock Answer #1:
If the footage you imported into AE is any kind of the following — footage in an HDV acquisition codec, MPEG1, MPEG2, AVCHD, mp4, mts, m2t, H.261 or H.264 — you need to convert it to a different codec.
These kinds of footage use temporal, or interframe compression. They have keyframes at regular intervals, containing complete frame information. However, the frames in between do NOT have complete information. Interframe codecs toss out duplicated information.
In order to maintain peak rendering efficiency, AE needs complete information for each and every frame. But because these kinds of footage contain only partial information, AE freaks out, resulting in a wide variety of problems.