I ended up digging around more on the forums and found some other threads. I just changed my sequence to ProRes LT and the Render options to “Render in High Precision YUV” It made the finished product look way better.
I get what I should do, I still just want to know why the H.264 looked crappy, but now that I know how to fix that situation I guess it doesn’t matter!
Thanks for all the help Brian!