3D camera tracking in a dark room

[Mike Sevigny]

We’ve successfully shot some stuff in a very dark space with a handheld camera using ‘throwies’. They’re cheap and can be painted out relatively easily (over black). Might be useful for some of your setups.
https://www.instructables.com/id/LED-Throwies/
Keep in mind that the light can bleed (slightly) into the edge of your subject if they cross in front of them. This can usually be handled in post but if it happens dozens of times throughout the video you might be kicking yourself.

Good luck,
Mike Sevigny

[Pascal Reinhardt]

thanks for the feedback everyone!

Yes, i’d rather spend two weeks fixing everything in pre production and testing instead of trying to fix it in post. Because i think the chances of it looking cheap are pretty high;)

I’m now trying to put alot of markers with neon green tape on the black backdrop, and use mic stands with pink tape on it next to the singer so that the tracker has a lot of points in 3d space. And of course i will test it first and make sure it works.

Just one more question about the 3d tracker in after effects. Can i give it information about the distance between tracking points? because i thought of haveng a few points in a grid with a fixed 1 meter distance and have random markers in between. Anyone have experience with this?

Thanks!

[Mike Sevigny]

Although AE’s 3D Tracker is very powerful, it has very few settings. I wouldn’t count on it for something like this, you’ll need an actual Matchmoving package like PFTrack, Boujou, Syntheyes or 3D Equalizer. Any of these will let you identify the distance between markers among other things.

Mike Sevigny

[Pascal Reinhardt]

Ok so i don’t have to have exact distances if i can’t tell it to after effects anyway?
I only have after effects and Mocha and not quite the budget to buy something like Boujou;) Even though i’d love to!

[Roei Tzoref]

Geez Ted! thanks for that, I was not aware of that. let me return a courtesy – there is an even shorter route. you can reply your own or any other message, and you will enter a slight different “readpost” thread. here you will be able to see the edit button underneath your original message 🙂

now all we need is a way to copy paste images right in the window and we are almost a 20th century message board 🙂

[Pascal Reinhardt]

Hey everyone,

good news, and a little problem.

Good news: the tracking now works. I fixed the screen and tracking points and found someone in my city to track it with boujou.

This is what the setup looks like now. A few markes in a 1m grid, and the rest radnom positions + mic stands for better 3d tracking.

So i have the boujou tracking data and the original footage in after effects and technically it works fine. BUT the whole scene is shifted.
So when i put in a text layer and want to simply put it back in z-space, i have to put it back on the z-axis and the x-axis because the whole scene and camera are shifted to the left.
This is the top-view in my after effects window:

it’s quite a time killer to move and rotate everything to fit the camera. I don’t know if that happened in Boujou or in AE. And i don’t know if i’m missing a very simple trick to fix it.
Can i just select everything and rotate it so it’s in the right angle?!

Thanks!

[Kalleheikki Kannisto]

In most cases you can fix things like this by creating a 3D null, connecting everything to it, rotating and scaling the null until you have things aligned how you want and then disconnecting everything from the null.

Kalleheikki Kannisto
Senior Graphic Designer

[Greg Gesch]

I suspect that you have too few planes which may be confusing the tracker – totally my instinct, I will add. I suspect the tracker is seeing only 2 or 3 planes – red flags, right wall and/or left wall – and only the red flags are crossing over anything behind them and giving clear parallax. It might be worth trying a setup where you add some free standing green markers away from the walls so that as the camera moves there is clear crossing over of markers at different distances from the camera for better position and depth information?

[Pascal Reinhardt]

Now that the video is done i wanna post a little summary:

The tracking on the black screen with green markers and pink planes in front worked great with a few restrictions. If one of the pink planes was out of frame it, of course, lost the 3d tracking points. So all the important planes for 3d space should ALWAYS be visible.
And another thing i could’ve done better is to remove the green markers behind/close to the person. Because there were enough other markers and the one that sometimes show from behind the person were impossible to remove, so i had to remove them by hand frame by frame. Keylight 1.2 couldn’t get rid of them perfectly ,it always left a nasty frame around them.

Also, shooting on a black screen seemed good in theory, but because in post so many things changed and different ideas came to mind that it would’ve been better to have a greenscreen to completely remove. And shooting a black screen in a corner wasn’t necessary, because it may have confused the tracking program and it only made sense if in the end, the final shot was also placed in a corner. Even placing just text(for a lyric video) over the invisible walls in post looked kind of weird if there weren’t any walls to give the right spatial feeling.

I also shot on a greenscreen for the same video and just had some green and pink markers in the corners which i could just mask out. This footage proved to be way better to use. The person/talent was mostly to well lit because of the bright greenscreen behind it but it did work and was way easier to handle than the black screen. I shot with 25 frames and had some issues with motion blur over the greenscreen so i would recommend trying a higher framerate.

One more advice i would give for those 3d tracking shots. Film in 4k if possible and rather keep more trackers and planes in the frame. If you go to 1080 in post you can track more markers in the 4k footage and then just crop it down to the actual image you need.
And the most important thing. KEEP TESTING IN PRE PRODUCTION AS MUCH AS YOU CAN! I was on an impossibly tight schedule so i had to do some learning by doing and come up with new ideas because stuff just didn’t work as planned(as usual). But all those mistakes won’t happen the next time;)