A Deeper Technical Understanding

[Vishesh Arora]

Dale

Check this tutorial about the Technology of Chromakey. You may find this helpful.

Vishesh Arora
3D and Motion Graphics Artist
Films Rajendra

Blog:
https://digieffects.wordpress.com

2011 3D Demo Reel:
https://www.youtube.com/watch?v=lHPgIJU_BR8

[Dale Paquette]

Thanks both. I agree completely with your assessment. What I’m actually trying to do is better understand the actual internal math that allows keying, spill suppression, and compositing to work. Steve Wright’s book Digital Compositing for Film and Video does go into some significant detail and suggests that if your keyer of choice (Keylight in my case) runs into a problem there are alternatives if the math is known. I suspect, there is still another level of detail beneath what Steve describes and I suspect I may indeed have to get into the science of image processing to wring out any more.

[Tudor “ted” jelescu]

My advice is to look for a company in your area that does professional chroma key studio setup for tv stations, virtual studios and so on and take out the lead tech guy for a beer. I learned more technical details this way than from any book I’ve read and it was more fun as well.

Tudor “Ted” Jelescu
Senior VFX Artist

[Todd Kopriva]

I learned a lot about compositing from this classic:
https://www.amazon.com/Art-Science-Digital-Compositing-Second/dp/0123706386

———————————————————————————————————
Todd Kopriva, Adobe Systems Incorporated
After Effects quality engineering
After Effects team blog
———————————————————————————————————

[Darby Edelen]

Todd’s suggested book is a good one.

I recently had a lot of keying to do and had the opportunity to try a few different keyers.

I’m most familiar with and comfortable with Keylight, which is a color difference keyer.

Ultimate is also a color difference keyer and works on the same fundamental principal as Keylight(although with a somewhat different approach).

There’s also Primatte which takes an entirely different approach to keying in terms of what’s under the hood. Conceptually it uses 3 different polyhedrons that contain color values which correspond to the solid, transparent and spill colors. Most of the adjustments you make to key internally deform these polyhedrons. I was very impressed with Primatte initially but found it difficult to massage the key into something I was satisfied with.

I also tried out the Image Based Keyer (IBK) in Nuke which uses a clean plate to key. I assume this works somewhat similar to a difference key, but the results are pretty amazing. You can pull some fantastic edges even on a really messy green or blue screen. It seems that you really need to rely on another keyer to generate a core matte though.

That was a long winded post 🙂 I’d encourage you to look for training materials for some of these keyers (some of the better ones cover the theory to some degree).

Darby Edelen

[Dale Paquette]

Thanks for your thoughts. I have looked at a few other keying packages but since I don’t have them, I’ve not gotten too far. Where I am starting gain some deeper insights is in study of scholarly articles on computer graphics and research papers from several universities. Of course, it’s easy to get bogged down in the math, but there is a lot going on that I suspect will seriously impact our industry one day. I’m especially impressed at some of the progress being made in keying based on depth mattes which will separate an arbitrary subject from an arbitrary background (the holy grail if you will). In its current state, this requires a laser scan along with the photography, but there are some statistical approaches showing promise. In the meanwhile, I am headed towards reverse engineering if you will the Keylight algorithms, at least close enough that I can conceive of variations and alternatives to help the process along when the going gets rough.

I hope others are interested in the details and this thread keeps going.

[Darby Edelen]

[Dale Paquette] “In its current state, this requires a laser scan along with the photography, but there are some statistical approaches showing promise”

While I’m sure a laser scan would be more accurate you can also achieve this with a stereo camera setup or specialized cameras that use infrared sensors. Software is of course required to generate the depth from the stereo image.

[Dale Paquette] “In the meanwhile, I am headed towards reverse engineering if you will the Keylight algorithms, at least close enough that I can conceive of variations and alternatives to help the process along when the going gets rough.”

There are lots of processes involved in Keylight, which I’m sure you’re aware of. The Keylight documentation should give you a good basis for what’s happening in general terms. It is, at its core, a color difference keyer so the basic keying algorithm involves generating the alpha based on the relative strength of the key color against the other color components. In the case of a green screen: more green than either red or blue = transparent or semi-transparent pixel.

Darby Edelen