I have been having the same exact problem. I hesitate to call it “grainy” since it is not anything like either the desirable graininess one might add to a shot, or the graininess that comes from poorly lit situations, etc. It was almost a bunch of tiny black dots that would appear for several seconds in the middle of a camera move, then disappear suddenly, and the video was clear again.
The depth of field setting correction listed above fixed the problem immediately. I’ve been struggling with this for some time now, especially since the problem only occurred sporadically, and am thrilled to find the solution.