Optical Flow

[Mads Nybo jørgensen]

Hey Steve,

Admittedly, I asked Gemini, as I was curious – and too busy to go to the source:

The author’s intuition in the second half is spot on, though the math in the first part has a minor technical discrepancy. Premiere Pro will not detect or strip out duplicate frames on its own.

1. The Math Check (Premise 1)

• Duration & Count: Slowing down to 10% speed expands the clip’s duration by a factor of 10. If a 1-second 24 fps clip (24 frames) becomes 10 seconds (240 frames), Premiere needs 10 frames for every original frame. That means it inserts 9 synthetic frames between each pair of originals (1 original + 9 generated = 10 total), not 10 inserted.

• Effective Cadence: The reasoning on perceived time is conceptually sound: each frame advances the captured action by 1/240th or 1/300th of a second, producing continuous, fluid slow motion provided the algorithm doesn’t artifact.

2. What Happens with Baked Duplicates (Premise 2)

Premiere Pro treats any imported video file as raw raster data. It has no automated cadence detection to realize that a baked file has repeated frames ($A, A, B, B$). It strictly evaluates sequential frames as they exist in that source file.

If a clip with baked duplicate frames is stretched with Optical Flow, here is what the engine calculates:

• Between Frame 1 ($A$) and Frame 2 ($A$):

  The pixel values between the two frames are identical. The motion vector field calculates zero velocity ($\Delta x = 0, \Delta y = 0$). Optical Flow synthesizes intermediate frames that are exact clones of $A$. The action completely freezes.

• Between Frame 2 ($A$) and Frame 3 ($B$):

  The engine suddenly detects a large displacement because all the movement from two frame intervals has occurred across a single frame step. It calculates vectors from $A$ to $B$ and interpolates motion across this window.

• Between Frame 3 ($B$) and Frame 4 ($B$):

  Zero motion again. It generates identical clones of $B$.

The Resulting Artifacts

Feeding baked duplicates into Optical Flow causes two distinct problems:

• Pulsing / Hitching Motion: Instead of steady slow motion, the clip will play with an uneven, stuttering cadence: freeze → morph → freeze → morph.

• Severe Warping and Tearing: Because the motion between $A$ and $B$ is twice as large as it would be in native footage, the displacement often exceeds the algorithm’s vector search window. This causes tearing, rubbery distortion, and boundary smearing.

How to Fix It Before Applying Optical Flow

If working with a file that already has baked duplicated frames (e.g., a 24 fps file doubled to 48 fps with duplicate frames, or a 25 fps file padded to 50 fps):

1. Interpret Footage: If the file’s container frame rate was doubled to accommodate the copies, right-click the clip in the Project Panel → Modify > Interpret Footage, and conform it back to the true base capture rate.

2. Remove the Duplicates on a Sequence: If the file has duplicates at standard timeline frame rates, drop it into a sequence matching its real capture rate, set speed to 200% with Frame Sampling (which drops every second frame), export a clean master, and then bring that master back in to apply 10% speed with Optical Flow.

Hope this helps?

It would be interesting to compare this with FCPX, Davinci and AVID + RevisionFX Twixtor, for how the different “optical flow” or similar processes works.

Atb
Mads