I’m actually getting the exact same thing you are. “CompressorTranscoder” and all four of my “CompressorTranscoderX” (one for each instance, and I’m running 4 instances on my 8-core Mac Pro) show up in the Activity Monitor as “Not Responding”. This also goes for “CompressorJobController”, “ContentAgent”, and “ContentController”.
However, that said, during an encoding process to my virtual cluster, each “CompressorTranscoderX” is using the CPU as normal. So it seems to be running just fine otherwise. To confirm this, I did a test on my own… Encoding normally using “This Computer” in Compressor, and encoding using my virtual cluster. When using “This Computer” (not my virtual cluster), the encode took 3:29. Using the same settings with my virtual cluster, it took only 2:40. Obviously, my virtual cluster is running, and it is speeding up the encode process… But still, this issue that we both have is very weird.
Also, as a sidenote, I got another weird Qmaster issue earlier today. I tried encoding a movie with my virtual cluster in Compressor, and Compressor popped up with an error message saying it had an “internal error” and “Apple Qmaster File Agent not found”. It wouldn’t let me encode at all with my virtual cluster. So, I deleted my preference files, “Reset Services” in the Apple Qmaster preference pane, and restarted, and now everything works as I stated above.
It is an odd issue, though… Compressor is accepting job processes to my virtual cluster, yet those “Not Responding” messages pop up in the Activity Monitor. If anyone has advice and/or a solution, I’m sure we’d both appreciate it.