To do this, I usually place an arrow on each layer and then duplicate the layer and adjust its location in the timeline randomly. But, this depends how many arrows you want. If you want a lot of arrows, then the procedure will differ.
Aanarav Sareen, ACE
Adobe Certified Expert, Premiere Pro