Sometimes, it’s helpful to know why this is happening. H.264 is a fantastic codec for final delivery because of it’s ability to compress videos to such a small size while still retaining decent quality. The issue is that the way h.264 compresses uses a lot of math to take one frame and using motion prediction formulas to create the next and/or previous few or many frames without saving all those frames individually. It’s way more complicated than that and I don’t know much more than that, but that’s the general idea. You may already know that, but the important thing to take away from that is that when you’re playing it, it requires the CPU to do a lot more calculations in order to give you a picture than a larger file where the limiting speed is just the transfer speed from the hard drive. When you’re just playing it (like in VLC player), the CPU can keep up just fine, but when you’re editing, that takes more oomph and it’s difficult to get that even with good CPUs. If you bring in a clip that doesn’t start with one of those keyframes, it’s gonna have to do a lot of extra math to display those couple frames.
It’s more complicated than that, but it’s the general idea and hopefully helps you understand why it’s so difficult. It’s not a fault of Adobe or GoPro, just the nature of the products.
Kinda unnecessary information, but hopefully it’s interesting at least.