If you export a Quicktime reference movie, which you then take in to Compressor, aren’t you then re-compressing the render files? The nice thing about sending direct to compressor is that, I believe, it starts from scratch and makes a higher quality video.
I’m also having this problem with a sequence with hundreds of stills. It’s going to be hard to isolate which is the culprit.