-
Facetracking issues trying to pin a 3d object to 2d footage
Hi. I am having a particularly difficult time with a head track. I need to pin an e3d object, a helmet, to his face. I then mask around his fingers on a higher layer to have in front of the helmet. (I thought THAT would be the tedious part)
The footage is, essentially, green screen 720p, so no issues with other elements. It is the 3d coordinates I am struggling with.
Facetracker works great with detailed tracking but has no 3d coordinates so it isn’t enough. I use facetracker to nulls script but it really only works for corner pinning and 2d siutations. I THINK) Perhaps it would be enough if I were better able to keyframe the 3d rotation, but I am not.
Mocha works great with corner pinning but, again, I need 3d data(i think) to perfectly pin the helmet.
Boujou and Pftrack both are having difficulty due to his fingers in the foreground. (I THINK that is the issue. I cannot seem to get enough tracks to stick perfectly to get a flawless camera solve. I have done well over a dozen solves but the export comes out way jittery in both AE and 3dsmax But I am not an expert on any of this stuff so it is possible I am doing something wrong. :/ (Although I believe I have configured all parameters correctly) But I am pretty sure that the issues are with the track, not the export
I try not to ask for help cause I like to figure stuff out on my own but I am kinda really stuck. If it were just this clip, it would be fine, I could manage the matching, myself, but it is actually four minutes of it, and at different distances from the camera, all with similar sh%t in the face foreground, fingers and hands jacking up the track.
Edit- as I was typing I was thinking maybe I just break each shot down into even more parts… Should I track this one three times? One for the center facing, one for each the left and right rotations? I was hoping for an easier, less time consuming way but maybe the footage won’t allow for it… I apppreciate the help, any help
Here is a clip https://youtu.be/RtOmEPYnHTw
Thanks.
———————————————–
Shifted Leering, Any Perspective
Shifting Literally All Perceptions
SLAPBlood type O negative. I bleed zeros and ones
-
9 Replies
-
You’ve a bit of a conundrum there. It’s not so much that your have the hands obscuring the face as you still have pretty clean sections of both sides of the head and forehead to work with. The problem comes from having nothing to derive any perspective in order to get a 3D cam solve. I was going to ask what the source footage looks like w/o the greenscreen removal (if there’s texture there to track), but even so it’s a locked off camera so there’s still no perspective change.
Ideally, you want to find a method of deriving a 3D null from your tracked head points that can be used to make a nodal point null of the head for mostly rotational info. I’m not sure how one goes about that but perhaps there is a way. Seems like a common issue. You might pose this question on the Imagineering Systems forums. See if you can pique Mary or Martins’ interest to help.
https://www.imagineersystems.com/
What about going the other way; bring this footage into your 3D app & animate the helmet to the actor? Basically, Z position is just a guess as there’s no Z translation, and scaling to what you see is going to be the determining factor. I think that ultimately might be the easier route. A lot of keyframing but…
-
Well dernit… Thank you for confirming my suspicion that there is a big problem with the footage. I should be clear and say that that IS the original footage. White screen, not green. just meant that there is nothing for the tracker to pick up on.
I will try Mary but all of the tuts I have seen her do have to do with corner pinning 2d objects, I haven’t found any regarding 3d pinning. About taking it into the 3d app, I am using e3d, so no option there, right? I only tried the export to 3dsmax cause it was so bad in AE.
Thanks for the help, i will check out the links now.
-
Oh my.
Well, there is one:
but it’s using bg areas to track & get the camera solve, so doesn’t help in your case.
I tried fiddling with this in Lightwave and while doable, a keyframing nightmare.
I had a similar project a few months ago where I had to change the eyeline in a scene. I didn’t need to deal with 3D space though; was able to do all in 2D. Even so, it was still problematic with sunglasses and fast, blurry head movement.
Well, if I think of some other option I’ll post.
-
Thanks, I will check the tut out now. It is only one little extra d, I don’t know what the problem is, lol
-
Hi, this sort of thing can be done in the free 3d software called Blender. There is a tutorial on almost exactly what you are asking! The problem is, you’d have to understand Blender first. I think, though, that the 3d programs are all pretty similar, just different buttons and layouts to do the same things. So if you’re adventurous, and it sounds like you are, you can maybe watch the Blender tutorial and then try to do the same thing in your 3d program of choice. Have fun!
https://www.youtube.com/watch?v=7K4TTNm9b2I&t=176s
-
You can take 2D tracked points on the face and convert the, to 3D rotation values with the technique in the link below.
The tutorial shows how to achieve this with the 2D points generated with the facetrack in AE but it will work with any stabe 2D tracked data.
Good Luckhttps://www.vdodna.com/tutorials/iron-man-hud/
Liran Tabib
-
… except if you are still using CS6.
-
Hey that is great, thank you. I DON’T really understand blender, yet. I use it to render my mp4 files from the jpg or png sequences i render from AE. But I will learn more. I actually went back to the Mocha route and just tried to do the orientation matching by hand. Came out pretty solid. Almost no drift. But almost doesn’t really count so I am game for anything that will work. Much appreciated. That is how I will spend the rest of my 4th.. On blender.. Never thought I would say that.
-
WHAAAAAAAAAAAAAAAAAAAAAAT?!?!?!?!?! Thank you so much, mane, this is what Im talkin bout.
Reply to this Discussion! Login or Sign Up