[Jordan Service] ” But whenever I have a layer with Key light on it, and i use a mask, it gives me back this 1 pixel line through image.”
whenever you have 2 mask’s edges from two layers that are supposed to to line up perfectly and you see the seam – you can use the Alpha Add mode on the upper layer and make it go away. this is because the mask edge is anti aliased and has 50% opacity. 50+50 in opacity is not 100 but 75… the meyers explain it better then me: https://www.provideocoalition.com/after-effects-hidden-gems-weekly-alpha-add-mode/
[Jordan Service] “Also, if the mask is not square it squares off the edges. “
you are probably trying to use the screen matte or combined matte mode while using the mask to create your luma matte. this effect does not allow you to do that but what you can do is set the effect to intermediate matte or result and add a fill effect set to white (to make the alpha white) and if you don’t want it to be on transparent background use a solid composite effect set to black (to have a black background)
Before:

After:

BTW I usually don’t bother with luma mattes with keying and just use the alpha matte. if I want to see how the luma looks I just use the ALT+4 shortcut to view just the Alpha channel in my composition. I would recommend this advanced keying course by mark christiansen especially when dealing with dividing a matte when keying: https://www.lynda.com/After-Effects-tutorials/After-Effects-Compositing-04-Color-Keying/114914-2.html
Roei Tzoref
After Effects Artist & Instructor
♫ Ae Blues Tutorials