I seemed to have fixed it, but I have no idea how this worked.
What I did:
1) In Premiere, duplicated the graphics clip and put it on another track.
2) Disabled the duplicate track.
3) On the original clip, added a Set Matte effect. Set the matte layer to the duplicate clip.
4) Also on the Set Matte effect, made sure Composite Matte with Original was checked.
Any idea why this worked?