[Blaise Douros]

Wow. You really showed me.

Look, sorry if your feelings are hurt by my above annoyance. But it’s not helpful to OP if you (a) don’t take a second to read and comprehend his whole post, (b) do the same thing when reading my response, and then (c) miss the point of OP’s post entirely by focusing on responding to an insignificant detail of my response because of (a) and (b).

If you’re just going to nitpick someone for pointing OP in the general direction he has asked for, then don’t expect to be treated as anything but a troll.

But no, I guess I’M the one who needs to take a hike out of YOUR teeny tiny corner of the internet.

[Blaise Douros]

That’s pretty much doing what you want. You might choose “frame blending” as your interpolation method if you want the movement to be a little bit smoother.

[Blaise Douros]

Okay Alan, then how about one of the other proxy formats that Premiere provides? Shall I draw your attention to the words “something” and “like,” which are arranged in a particular order in the sentence above in such a way that they indicate the meaning “a format similar, but not necessarily identical to, ProRes LT, which is given as an example?”

Or, you could have read in OP’s actual post that they have tried ProRes flavors, but rejected them as not working well due to their higher bitrate when working in 4K, thus implying that they might be one of the many people who have installed Quicktime and its codecs on their Windows machines, allowing them to use ProRes.

I can’t believe that I actually had to explain any of this.

[Blaise Douros]

Seriously, though, you need to renegotiate if they are dead set on this. If you gave them a scope-of-work document at the beginning, that is a valid reference. If they’re going outside the scope of what you, and they, outlined in that brief, then you are completely within your contractual rights to let them know that this request is going to cost them more money.

If you’re pulling a chroma key from this (and Dave’s right, their DP should be put out of their misery for this setup), it might be wise to do that first before attempting the color changes they want. Have you tried the rotobrush tool?

[Blaise Douros]

Why not proxy your footage in 1080p with something like ProRes LT, and then do your final render with your full-res media? That’s what a lot of folks do when they can’t edit at full-res 4K due to high data rates. The same principle applies to media with heavy processor load due to compression.

To answer your question–yeah, it’s kind of dumb. Sorry ☺ Most modern NLEs have a workflow that allows you to create proxy media for exactly the purpose you need.

[Blaise Douros]

My advice? Charge them hourly, and tell them how many hours it’s going to take you to get this right.

If they aren’t paying you, or if it’s a per-project fee, tell them to either live with the colors they shot on set and presumably had plenty of time to choose in planning and preproduction, or reshoot it.

[Blaise Douros]

Hey Roei,
Great questions. Answers follow:

1. No, I didn’t prep the footage; the one thing that I did differently was track from the beginning of the clip until the camera pans over to the hallway, to give the 3D tracker more movement data to work with. When it solved the camera, it gave me “Error, could not solve camera for these frames” towards the end, but that didn’t matter to me–all I wanted was the window data, so I trimmed the clip on which the tracker was applied so those error frames wouldn’t appear. I ended up putting a duplicate clip (with no tracker applied) above the exterior plate to use for the rotoscoping, but left the tracked copy in for reference.

2. I’ve found that the 3D Camera Tracker is somewhat good at guessing AOV, but whenever I have any information on the angle of view, I specify it. I’ve done a lot of testing, and it seems to make the tracker more accurate–and that makes sense, given that After Effects can now estimate the expected depth and movement of the scene based on the lens width. At the very least, it results in a virtual camera whose focal length (and thus, characteristics when interacting with the virtual objects in the scene) match the taking lens. As for how I guessed the angle…it was just a guess based on what I could see in the scene. I shoot as well as edit and work in After Effects, so my eye is somewhat accustomed to estimating that kind of thing.

3. Yes, that’s true, but it’s nice to be able to do it so quickly based on the tracking information in the scene, and as you say, without leaving AE. But I take your point. Mocha is probably a better and more accurate choice for tracking the window frame and building the foreground matte. I like the 3D camera tracker for a lot of other applications (I do a lot of titles that are tagged into the scene’s 3D space), so I’ve gotten pretty comfortable with it–it’s an easy solution for a lot of problems. I need to get more fluent in Mocha, though. Great conversation!

[Blaise Douros]

Here, take a look at this AEP file. I ran a 3D camera tracker on the footage (track data would have been cleaner if I knew the camera used and the FOV of the lens), a quick Roto Brush on the window, and used the camera track data to place a Null on the brick wall outside, which I then used to place an image into the scene.

10874_windowreplacement.aep.zip

You’ll need to relink your footage and find an image to replace the one I placed in the scene.

This method allows the exterior image to move with the camera, and not appear to be tagged flat onto the window frame. There are cleaner ways to mask out the window, and Mocha might even be a good choice for that. But for realistic image placement within the scene, this is a great way to go.

[Blaise Douros]

Holy crap this is a complicated way to go about it. Why not just do a 3D camera track, mask the window out, and place the footage way back in 3D space? That way you get some parallax motion between the room and the background.

[Blaise Douros]

Dave is terse, but correct. There is NO solution that allows AE to automagically identify an image as being similar or identical to an onscreen object, and correcting the position, scaling, etc to match. That’s a job for advanced computer vision and machine learning specialists working with huge supercomputers on custom software, and After Effects just can’t do it.

So, as Dave says, either prepare to do some manual work (and get used to it, there’s a lot of that in this business!) or perhaps if you can explain what exactly you’re trying to accomplish, we can find a better way to help you.

Personally, I don’t really understand why you’re trying to replace a still object with a 2D image of the same still object.