the help says a lot but in the Tips and Tricks for GI there is this
GI and Animation
GI and animations are technically challenging. If a single image takes a long time to render using GI, then rendering an animation with a series of these images will require very long render times. In addition, Irradiance Cache tends to flicker if its settings are set too low.
You should adhere to the following when creating animations with GI:
When using IR+IR/IR+QMC: if flickering occurs on surfaces, the Samples value probably too low (four-digit figures may even be required). If, on the other hand, flickering occurs in small regions around an object, the Record Density value is too low. In this case, decreasing the Maximum Spacing value should help. If this does not do the trick, set the Record Density to a higher setting.
Flickering can even occur at optimized IR+LM settings if the base light map is of poor quality (see ,Optimize Light Mapping’ below).
QMC+QMC will not flicker (the render times will increase accordingly; these will be at a maximum).
long story short, animation of both camera and objects is going to take time to kill the flicker