Stabilize phone screen and keep flat in shot

[Brendon Murphy]

Without seeing the shot, I’ll just guess that you need to freeze frame the hand and track it back into the shot with x and y position data only (to match the camera movement). Paint out or cover up the original hand. Roto the shoulder/head so they overlap the hand/arm properly. Re-grain the hand so it doesn’t look like a still.

Another option is to do the same steps, but stabilize the hand in mocha instead of using a freeze frame. However, this could look a bit funny if there’s much movement on the hand.

I’ll point out that with shots like this, it’s generally ok if the hand has a little bit of natural movement – you just use something like mocha to do a planar track and then parent your screen graphics.

[Michael Szalapski]

You could just track the screen and use that data to stabilize it. You can use data from Mocha AE or AE’s own trackers. Depending on how it moves, you may need to get fancy. Mocha AE (the version of Mocha that’s included in AE) doesn’t include stabilization, but you can use the data to do stabilization anyway. More info here.

[Daniel Boyd]

Ok! So I tracked the motion using the two top corners of the screen, then stabilized the result, and that actually worked really well. The phone keeps the same width even when tilted to the side. There is a short period when the user’s thumb crosses over one of the tracking points, but I found if I deleted those specific points that were incorrect, After Effects filled in the blanks really well. The only issue I’m having now is that, when the phone gets tilted down, the screen gets longer, so not sure if I can track multiple points, for example, tracking the two top corners of the screen and then the bottom parts of the screen in relation to the top portion.

The only other issue is there is some noticeable jitter left in the video, but I should theoretically be able to export as an AVI, then re-import and use the 3D tracking stabilization.

Fortunately, the user’s left hand holding the phone stays pretty stable, so it’s just the right hand that tends to cross over the bottom on the screen making it more difficult to track that lower screen portion

[Brendon Murphy]

That’s the issue with stabilizing the screen – the rest of the image inherits the movement, resulting in your jitter. Most of the time, you do not want to stabilize a shot around a phone screen – you want the phone to move naturally, and your graphics to move with it. You can do this by just tracking the graphics to it, or by stabilizing the screen and then reverse-stabilizing it to restore the natural movement. If you need everything perfectly static for some reason (camera and hand) why not use a freeze-frame?

To answer your question about the lower screen portion, just manually keyframe those bottom corners. I like to use mesh warp – change it to 1X1 and quality to 10, and you basically have a corner pin with bezier handles.

[Daniel Boyd]

Beautiful, mesh warp worked really well!

Yea, apologies if I wasn’t clear of the first post, I wanted to keep the hand movements, just keep the phone screen itself static and let everything else warp and move as needed 🙂

I do need to look into Mocha as well. I’ve read in a couple threads here that motion tracking seems to work much better so maybe that will help the 1-2 pixel jitter I’m experiencing

[Brendon Murphy]

Yes, Mocha is a planar tracker. Think about a clipboard with a pattern painted on it – Mocha tracks how the pattern distorts as the clipboard is moved around (as opposed to tracking points like the AE tracker). Mocha lite is actually included in all versions of after effects CC, and will do everything you need here. Just select your footage then go to Animation>Track in Mocha AE. That will open your footage in Mocha!

A couple more tips – you can do your tracking on a de-grained version of your footage to make the track smoother. You can also export a hipass version to bring out more detail in the footage (google hipass after effects for tutorials).