Sorry you are having issues. We are purchasing several of the PMW 320s. I understand the issue is with the stock lens. We demoed a PMW 320 and I do not remember it having these issues- at least not as bad as I have seen it on our PMW EX1Rs. With the EX1Rs we began using a remote zoom control that seemed to help the issue a little.
As far as the 320s go, we were needing a longer lens for two of the cameras so upgrading those will help. We decided to wait until we have cameras in hand until we decide if we need to upgrade all the lenses.
How are the lenses on your EX3s. I do not know for sure but there may be an adapter that will allow you to use those lenses on your 320s.
Let me know what you find. It seems I am about to be in the same boat.
Thanks
Glenn