I have several shots in this sequence and I ended up using the Keylight layer as a matte for the same footage and then using a spill suppressor on that layer. It works but I have never seen Keylight do that to some footage so I was a little confused. Thanks for your help guys.
Just solved my own problem. I created a Premiere Pro project with the assets organized and when I want to import the assets organized, I import the project file. Works pretty well.