I’m dealing with the same issue – cloning on to particles.
If I go to my cloner and hit the C key to bake the clones (and turn them into un-animated, non-dynamic objects, without a cache), they render quickly, so it’s definitely not a geometry loading problem.