Ok, at the risk of elevating the ire of the community and lengthening this topic….
I hadn’t actually watched that youtube tutorial in the initial post until now (since you mentioned it again).
Where do I start…. I haven’t yelled at TV for a while but that gave my lungs a good workout!
I never like being negative in these forums because it doesn’t help anyone, and because there’s never one right way to do something. And often, when investigating even dubious techniques, even us seasoned pros can learn a trick or two.
Buuuutttt…. There is just so much questionable information in that tutorial. So much so that the instructor doesn’t seem to know how to use AE in some areas. At one point he decides to use another layer as a matte. Since forever we have this thing called track matte that lets you pass the layer below through the layer above in either alpha or luma mode. While his technique is not wrong , it sure is a long way round a simple every day occurrence, and his technique is one we haven’t used since AE was called COSA. So that makes me wonder where the rest of the info has come from and his level of expertise.
Anyway getting beyond that little nit to pick.
The first issue could be classed a preference, but using full-red to check a key – any full chroma will overload your eyeballs and your monitor. You will not be able to see subtle issues in the matte edge with your cones screaming to turn down the volume and your monitor being pushed to the limits of its gammut. In addition, any two full-chroma primaries next to each other (red/Blue, Red/Green, Green/Blue) can vibrate in the eye (especially red/Blue – I can explain why if you like but its not important here)
His method for recoloring the screen to make it easier for Keylight to see…. Anytime you add a color correction or level trim or compression, or even change the color space, you are losing or transposing information. You can’t make more of what was there you can only crush it or invent it. Its like yelling at a person with a hearing aid. It kind of overloads a delicate system put in place to detect subtle things and you end up warping or replacing the information.
When we do film transfers we go to great lengths (enormous lengths, these companies spend month getting LUTS just right to tease the most out of the footage) to ensure that as much of the color of the screen is preserved, because even in a $1500 an hour grading suite you are loosing or shifting information and every time that happens the key gets worse.
There are subtle little roll offs between the green of the screen and the person in front of it. The keyers are designed to pick up those subtle differences and work in magical ways to determine what is screen and what is human; pushing and pulling on feathered edges and gentle blurs, even keeping but desaturating greenscreen pixels and letting them play as foreground pixels. By changing the color of the screen you are forcing colors into those subtle areas in ways that are more heavy handed than a keyer would and without control of that roll off. A keyer doesn’t care whether it sees a particular color, it only cares about the chroma separation between that specific color and things that are not. You can use keylight with green or blue or red, or beige or skybluepink if you want, but use it with the color that came on the virgin footage. You are also killing the grain structure when you stomp on the screen color – this is great for reducing the noise, but noise is detail.
Will his replace color technique produce a sharp key with less issues? You bet! Will it be the most accurate key and keep all the little feathery wispy details and show you his outline right to the last pixel of him? Not on your life. If you used that technique with something like water being poured into a glass (all on green screen) you would end up with a posterized mess.
Then there’s the whole matte math done with, among other transfer functions, the Lighten. Instead of describing how lighten works across a range of uses, you just have to use it for a while in compositing to realize it has a very narrow niche. Try using it instead of screen or overlay or a few of the other blend modes that are used daily. It creates harsh roll offs which look sort of posterized. When applied to mattes or the product of mattes its not going to be the best and will surely wreck those subtle rollsoffs the keyers were designed to deal with. Then he adds a blur to overcome the harshness all his monkeying added. You could have probably added the blur to the initial keylight matte and then crushed it and come out the same. But again, its not going to preserve all the little edges that make a comp look real vs pasted on top. If you have harshened an edge and then softened it, you have changed that edge irrevocably, an no CSI type software is going to be able to resolve the original details again. ( I have a friend in Forensics who says CSI is is favorite comedy show, I’m starting to feel the same about this youtube video).
I’m sure there are situations where some of those techniques might be useful but to put them forwards as “here’s how you pull a key”, is doing a disservice to all the things keying software can do when used properly. And I think this shows in his final comps. Especially the guy with the glasses at the end. I don’t have any clients that would accept that boiling bubbling key where the stem of his eyeglasses disappears.
Notice I’m not posting this rant to his youtube page. I only spouted because you are having troubles, and this video you brought forward is going to steer you wrong.