Someone told me I should use motion sketch – I can’t believe I didn’t know about this! I redid it and it’s much better – still needs some tweaking on the timing.
I tried to use a Particle system but don’t have Trapcode Particular and couldn’t figure out how to make a custom particle within the particle effects already in AE. Would be good to know if it’s possible:)