Point-Cloud Camera Tracker and LED Wall Helps Satore Blend Eight CG Sets into Production That Fools You Until the Very End
In the last few years, virtual production (VP) has become something of a buzzword for visual effects professionals and amateurs alike. Fueled by the impact of COVID-19 and productions like The Mandalorian and The Lion King, VP has come to denote an emerging tech, something that’s not quite ready for prime time unless you have the resources of a company like Disney. But for organizations like Satore Studio, VP is something else: a chance to level the playing field.
Long before The Mandalorian highlighted what LED screens can do, Satore was experimenting with VP workflows. Working out of multiple, decentralized offices in London, Cardiff, Lisbon and New York, the multi-disciplinary studio immediately saw the potential of VP, and how it could give them the same sort of creative immediacy they found in their immersive work. Then COVID-19 hit and everything changed. Productions needed anything that could help them stay alive without a serious dip in quality or reams of staff onsite. That gave Satore an idea.
Over the summer, as the first wave of COVID restrictions briefly began to ease, Satore’s creative team partnered with MBS Equipment to demonstrate the power of VP by creating a demo that looked so real, even a trained professional wouldn’t know it was fake until the very end.
Produced for Quite Brilliant, the short was built on the idea that with VP, location is relative. But for most of its 90-second runtime, you wouldn’t know it as the locations behind the actor are fairly normal – street corner, nightclub, forest, car. It’s only at the end where the camera pulls back, revealing a crew and LED wall, that you realize you’ve been had. It’s a moment of pure awe and one of the best selling points for VP realized so far.
While a handful of real props and furniture pieces fleshed out a few of the scenes, the CG backgrounds effectively do all the heavy lifting. The result is a photorealistic series of shots that depict a performer in multiple locations – but in reality were all shot on Rebellion Productions’ stage over the course of two days.
“Everything you see in the background of the demo was final pixel VFX, which is simply incredible,” said Tupac Martir, creative director and founder of Satore Studio. “After the demo was done, I was watching it and I couldn’t remember seeing a market stall on set. It turns out even I was fooled!”
To set the stage – so to speak – Satore created two CG background scenes using a combination of tools including Maya, Disguise, Houdini, ZBrush, Substance and more. Six other scenes previously created by an archviz company were then optimized for use in the demo and rendered using Renderman, Octane and Arnold. The environments were housed on Universal Pixels servers running Unreal Engine. Each virtual background was created to run in 25fps, using a proprietary workflow developed by Satore.
Tying it all together was an Ncam tracking system, which blended all of the CG and video data coming from the Arri STX into a single in-camera mix. By using a natural marker-based point cloud as a tracking tool, the creative team was able to easily track the performer in relation to the lighting, background and props, providing a real-time view into how everything was lining up. With everything in-camera, the director and DP could quickly see if a take was working, making it easier to call out changes in the moment – sometimes to amusing effect. At one point, the photorealistic mix had the director so fooled he asked Satore to move a digital lamp, only to discover the lamp was actually one of the production’s on-set props.
In addition to live reviews, the data could also be streamed to remote locations, giving others the ability to contribute to the scene without needing to be present on site. For a studio with offices in multiple cities, this is a convenient and efficient method of creating content. For a world still under the threat of COVID-19, it is potentially the new norm.
For this demo, the majority of the work was created using previz, with only 20% done on set. Fewer than 30 people set foot on stage over the two days, with an average cast and crew of nine individuals.
While reaching the levels of photorealistic productions seen in The Mandalorian still require vast resources, Satore’s demo offers a closer look at the potential on a more grounded level. Not all studios need to create epic sci-fi locations for a space opera, and having the ability to change between environments in as little as 20 minutes – as was the case with the demo – means that anyone from major productions to indie studios could potentially film an entire movie, series or advertisement all in one location.
“Satore is a small indie company, and works in a very indie way,” said Martir. “There are still some hurdles slowing virtual production from becoming mainstream, but if you’re clever, you can use these techniques to create something truly incredible, and do it safely and with ease.”
VP is still in its infancy, and as with most tech, it will continue to improve. Costs will go down, and by using existing technologies like Unreal, more and more assets will become available to create new shortcuts and resources. But as the need for social distancing remains ever-present, and as the convenience offered by VP continues to offer new benefits, Satore Studio and organizations like it will continue to push the boundaries.