Looking to subtract between two images & add the difference to another

[Walter Soyka]

[Michael Morgenstern] “I have some footage that has a strange banding effect — wide bands of light and dark very slowly moving up the image. It’s regular and consistent, and there is a part of the image that doesn’t move at all, so I would like to freeze frame that part, grab the difference between the freeze frame and the current frame, and then apply that difference, inverted, back to the image. I’m playing with blend modes and precomps, but the issue is that things like the subtract mode clip values and don’t act perfectly. Is there a plugin that lets me do arithmetic blending of images?”

Subtract mode will not clip values in 32bpc (they will go negative), but inversion by definition always expects input in a 0-1 range — invert(a) = (1-a) doesn’t make sense when a is less than 0 or greater than 1 — so you’ll run into trouble there.

Can you build an adjustment layer manually that corrects the bands, then animate it to match the footage?

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog – What I’m thinking when my workstation’s thinking
Creative Cow Forum Host: Live & Stage Events

[Michael Morgenstern]

I’ve tried to do that a few times and it hasn’t been great. The shift is so subtle that I don’t really have something to go off of, and since I have the adjustment layer in there already I’d much rather figure out how to make that one work.

I was also thinking of using some plugin that allows me to directly do arithmetic on the layers: for example, mapping the difference values to center around 0.5.

[Darby Edelen]

Invert shouldn’t clip values if the project is set to 32bpc.

I’ve removed my previous answer as it was posted before I could view the images. For the record many corporate environments block Dropbox and the COW does have its own photo embedding feature 🙂

Now that I’ve looked at the shot I’m still concerned that you may not be able to solve it with the method you’re suggesting but if you’d like to give it a whirl anyway you should definitely set the project depth to 32bpc. Then you’ll essentially do what you’re doing now:

The clean area’s current frame gets subtracted from the clean frame, that value is spread out horizontally to cover the frame and then added back into the whole frame.

The concerns I have with this are:

1) The blur required to remove high frequency noise is going to cause trouble along the edges, likely solvable with the right radius blur

2) the band is likely raising/lowering color values in a non-uniform manner across the frame. My guess is that this may have been caused by one of the lights being a fluorescent that was out of phase with the camera. In that case the dark areas are areas during the exposure of the frame in which the light was “off.” Since lights are not uniformly bright and also cast shadows I’d be concerned that the variation in values in the bands are not uniform. That said: I may be over thinking this and everything might work out just fine with the described method 🙂

I’ve used the temporal filtering feature from this plug-in in the past to repair issues like the one you describe:

https://revisionfx.com/products/denoise/

In addition, there’s this one from the same folks:

https://revisionfx.com/products/deflicker/

I haven’t used it. It’s advertised primarily for high frame rate or time lapse footage but the problem it’s solving is essentially the same.

Darby Edelen

[Michael Morgenstern]

Perhaps I explained this incompletely. What I want to do is take a small segment of the image, which doesn’t move, and getting the difference between that and the freeze frame. Then I will blur that difference, expand it horizontally so that it covers the whole image, and invert it / subtract it from the video footage.

Yes, the small section of the image this diff layer was taken from will be adjusted to be close to the image, but with the blur, the adjustment will not be perfect — it will just be a broad brighten/darken to even out the tones of the image. And the rest of the frame would be affected too.

So:
F = a pixel from the freeze frame
A = a pixel from the video
X = stretch(blur(F-A))

And I’m looking to get A+X
or A+stretch(blur(F-A))

[Darby Edelen]

I think I was editing my last response when you responded. Hopefully that one is more helpful now 🙂

Darby Edelen

[Michael Morgenstern]

Thanks Darby! Even if there is a bit of a color shift, I know (hope) I can get rid of the greyscale shift… Also, as you can see the difference layer is colored so I’m hoping that will affect the cast.

I’ll check out your links! Question on one thing you said, though: “do essentially do what you’re doing now”

What am I doing now? I’ve researched Subtract, Difference, etc, and it’s not clear to me exactly what operations I should be doing. Subtract (F-A) -> Blur&Stretch -> Invert -> ?? to the original source?

Thank you!

[Darby Edelen]

No need to invert depending on your order of operations. If the still area on the clean frame is A, the same area on the current frame is B and the entire current frame is C:

C + stretch(A – B)

If you need to blur I’d do it where you suggest.

Again, this will need to be done in 32bpc.

Darby Edelen

[Chris Wright]

if it was stationary objects or even slight camera movement, it would be hard but not impossible, but…

i don’t think a temporal filter or timeremap/difference matte will work because there’s actors in the scene and they will move about and each frame will be completely different than the previous frame. (i.e. its impossible to create a clean matte)

I just don’t think this is savable. at, least quality-wise.

saying all that, here comes the partial solution:
solution 1:

your only hope is a motion tracked mask of the actual bars moving around, then they themselves can be deflickered or color corrected back into the scene with nice opacity and feathers.

solution 2:

solution 1 has failed and you are out of options. precomp your video, timewarp 50% the precomp, then timestretch the same amount. The video will have the same duration but it will create intermediate frames essentially slowing down the moving boxes to such a crawl, that they become unnoticable.

if you have 15 frames you wouldn’t mind uploading, I’d be more than happy to take a stab at it.