Twixtor Problem

[Pierre Jasmin]

That indicates a frame rate issue somehow

Normally this happens when the source FPS (nested comp or media) is not the same as the Input FPS slider in Twixtor. For example someone types 24 or 23.96 instead of 23.976…

Are you sure that’s not it?

Pierre

[Anton Björkegren]

Yes. The Twixtor input framerate is the same as the original clip. In this case it’s 29,97.

[Peter Litwinowicz]

Does the original clip have repeated frames within it (for example, does it have some type of pulldown in it)?

Often you can’t tell that from playing the clip, but you should step through the original clip (before Twixtor) and see if there are repeated frames. If so, you need to remove those repeated frames before applying Twixtor).

Also, you might have Input:Fields set to Upper Field First and Lower Field First when the original clip does not have interlacing. You might try setting that to None.
Pete

[Alejandro Petrakovsky]

hello pierre, I was wondering if you could help me with this problem i am having on twixtor using it in finaul cut.
i have some snowboard tricks shooted on canon 7d at 60 fps, but when I use twixtor i have a problem with motin blur like if the image repeats, i ve tried with many possiblities , touching de motion blur compensator for exmaple but I can get a good use of the effetc
will apreciat your help
thank you
ale

[Peter Litwinowicz]

When you say you are having a problem with “motion blur like if the image repeats”?

Are you have a problem with motion blur, or are you having a problem that the Twixtored results start and stop because of repeated frames in the source footage?

Pete

[Alejandro Petrakovsky]

no, the image does not stop, it slows down as I want but it those like a ghost in the frames when the rider is making the jump , specialy in the part of the board and arms….
maybe it happens beacuse the trick is quick? or is does not have anything to do?

[Peter Litwinowicz]

Ah, if you have a lot of motion (or objects crossing each other with a fair amount of motion) then you will see this problem.

You can reduce the “gloopiness” by watching this help tutorial. https://library.creativecow.net/freitag_lori/Twixtor-Fixing-Unwanted-Warping/1

However, I suspect there’s not much you can do to get the ghosting to disappear (even after watching that tutorial), except to shoot next time using a higher frame rate. Our Pro version allows you to roto the skier (frame by frame animation) and then use that to tell Twixtor that the skier is a separate object to be tracked, but even that takes a bit of trial and error with shooting properly to allow that to work.

You probably want to read about “problematic footage” from our manual that I reproduce below.
pete

PROBLEMATIC FOOTAGE

If you have not used Twixtor yet, this section might prove to be a little bit esoteric. This information will be of more use to you as you become more familiar with Twixtor and want to understand how to generate the best possible results.

It might happen that Twixtor might do poorly in specific instances. The idea here is not to discourage you and tell you it does not work but, rather, to give truth in advertising. As you become an expert user, you learn to predict the kind of material that can cause problems.

1. “Picket Fence”: A very regular pattern in motion (for example someone wearing a t-shirt with fine stripes) with an object moving in front of it (e.g. the same person hand for instance) might confuse the motion vector calculation. Any very structured pattern rotating can cause “blind spots” for the analyzer.

2. “Transparency”: Overlay of semi-opaque surfaces might create unexpected results. Some cameras for example will streak under fast motion and that can create disappointing tracking results.

3. “Short Interval Defects” : Sudden global illumination change (e.g., a flash), strobing, dust, … can create unexpected / undesired results. Also, if there is a piece of hair or a scratch on the scanned film for a frame this would influence the tracking so you really should try to clean such defects before processing. You might try using the Mark Segments feature to mark these single frame defects as a separate clip to avoid the defect from spreading it’s influence into other frames.

4. “Fields”: We provide some basic field handling. Always remember that by definition, even and odd scan lines in material with fields are considered half a frame apart timewise. Because fields are spatially a line offset from each other, regions where there is no motion and horizontal lines might produce a slight up and down motion in phase with the source frame rate, for that reason some people prefer to first deinterlace their footage with a tool like FieldsKit

5. “Duplicated Frames”: Twixtor does not provide automatic duplicated frame detection support. As such, you will need to remove duplicated frames before the application of Twixtor. Also, you should be aware that if your material has 3:2 pulldown you should remove it beforehand. The same applies to animations on 2’s. If you leave the duplicate frames (or fields, in the case of 3:2 pulldown) the freeze-frame, inherent in the duplication of the frames, will be stretched (or sped up, as appropriate).

6. “Alternate Motions”: When motions going in different direction are layered it is possible that the dominant motion affects (spills into) the background motion. The PRO version manual explains ways to handle such issues.

7. “Specular Highlights”: If you have moving lights, e.g., a shiny object that reflects the light as it moves, it might cause problems because when the motion estimator attempts to match two images, as the motion estimator will tend to follow the highlight as if it was an object. (of course, sometimes this is what you want).

8. “Ultra-Fast Structured Motion”: We are very particularly perceptive to human actions. We have sometimes seen that certain complex rapid motion such as someone doing a frenetic dance creates interframe displacements that are just too big for our motion estimator to resolve satisfactorily. When planning a shoot for an effect that involves Twixtor in the pipeline, consider that for Twixtor fast articulated motion should be easier for front facing subject then sideway views as there will be less pixels traveled per frame on the screen, which is really the only thing that Twixtor cares about.

9. “Occlusions”: Problems caused by object motions tend to be one frame problems and localized in an area of the frame which is called an occlusion, which is some pixels that you see on one frame but are not visible on the other as a result of camera and/or that object motion.

10. “Limited Reach”: As a rule of thumb considers that Twixtor will be most accurate for pixel displacements that are a maximum of 5% of your image resolution (for 720×486, this comes to a maximum horizontal displacement of 35 pixels or so). With displacements larger than that Twixtor will start to become less precise as it tries to separate motions from one another in an image sequence.

11. “Compression Artifacts”: Certain video coding techniques such as DV compressors use 8 by 8 pixels blocks based compression. What this means for you is that if for instance you have a sharp edge that moves, it will switch of 8 by 8 block and therefore locally it’s neighbor values will be all different (substantially not like 2-3 values over 255 but sometimes 40 off near an edge). This is why green screen like setups perform badly with DV… Without smoothing the source, this can sometimes certainly create tracking problems.

MORE AESTHETIC / COSMETIC CONSIDERATIONS:
Gee! It breaks on these 3 frames what do I do?

Now that we have discouraged you, let’s help you to be creative in repairing problems. We know that most of the time, there is nothing we can do to prevent problems at the source because the material already exists, and you need to fix it. So let us share a bit our experience. Experience we acquired playing with this kind of process in production, in trying to make a product that works, and in supporting our users. There is an astronomic quantity of possible images and it’s easy to develop assumptions that prove not true if you always work on the same few clips. So, so you know we fully appreciate short movies (5 frames is usually enough) where Twixtor fails somehow, this is how we learn, and can hope to make an always better product.

1. 1 or 2 frame defects on source material: Sometimes you receive a movie with one or two frame defects (could be a flash for instance). Take the good frames on both ends of the bad ones and slice them together into another comp, create a short Twixtor clip of the proper duration and reinsert it back over the bad frames. Naturally if you also want to apply Twixtor to the whole movie, perform the repair before applying Twixtor to the whole sequence.

2. Splotches: There are times when you will still get some splotches (unwanted warping artifacts). You can try one of two things:

In applications that support the animation of pop up menus, you can animate the Motion Vector Quality menu so that it is set to Best for the non-problematic areas and set to None for the problematic areas. For host applications that do not support the animation of pop-up menus (like FCP), you can run Twixtor twice (once with Motion Vector Quality set to Best and once with Motion Vector Quality set to No Motion Vectors, for example) and simply replace the problematic images of the Best setting with the images made with the None setting. If your problem only occurs in a small part of the image, you can hand-create a matte to relace the bad portions of the “Best” version with the corresponding parts of the “No Motion Vectors” version, using the compositing functions of your application. You might want to do a soft matte for nice blending. And naturally if you have the Twixtor Pro version, you can provide Foreground/Background separation mattes to Twixtor, which you should do probably in a manner that respects the shape of the object in the foreground. Additionally, in the Twixtor Pro version you can provide guidance to the tracking (see the Twixtor Pro manual for more info).
Try turning down the Motion Sensitivity setting for the problematic portions of the image sequence.
3. Ghosts: Since errors such as ghosting are often due to occlusions (pixels that were not visible in the previous frame, so don’t have correspondence in the other frame due to motion), they are usually easy to paint out so when the result is played in motion no one will see it. Some camera motion such as driving on a bridge tend to create a multi-frame ghost which usually is not too problematic to paint out in apps like Commotion or Combustion that have clone offset paint tools. You might also prefer to not apply motion blur at this stage, paint/fix and then use ReelSmart Motion Blur to add some motion blur.

4. Using existing mattes: Sometimes, rotoing out a fast moving foreground object will make the background very stable and produce tracking data without the foreground motion influence. If you have the luxury of having mattes for a layer, and don’t have the PRO version, you might want still to try to apply Twixor to the foreground and background separately (so that the motion tracking influences on the layers go away or are reduced by Twixtoring them separately). Note in some cases you would need to fill the foreground hole with some form of clean plate image so it has the general motion of it’s surrounding area. This process is somewhat what the FG motion masking services of the PRO version would do for you automatically.

5. Shakes…: Although by definition we deal with camera shakes… note that when you slowmo something you will also slowmo the camera shakes and vice-versa, so you might consider to pre-stabilize the sequence on which you want to apply Twixtor.

6. Noise, compression artifacts: Our algorithms should not be too sensitive to small grain noise at the analysis level, however it might look weird to extrapolate a lot of frames per frame on material that contains a lot of noise (as it essentially slows down the noise also) so you are advised to remove some noise in the color source, perhaps running a form of smoothing or noise-reduction filter before. With material (such as some “really damaged” MPEG 1 material), lower accuracy motion estimation might even produce more desirable results. Also with DV-like sources, avoid nearest Frame Interp mode. The smoothing of blending actually reduces noise.

7. Very Large SlowMo: The amount of clean frames you can generate is often more a factor of the action in the scene as there is probably no answer to the question of how many frames can be inbetweened with a process like this, as it depends… As a rule of thumb, if the image is moved 10% or more between two input frames frame, consider yourself lucky if it works. (Again there are no absolute answers). Perhaps also you should consider that there is a limit to how much you slow down things as even perfect inbetweening with a computer animation systems can look weird as it won’t have any local dynamics, that is it becomes similar to doing global shape key framing, everything will sort of move at the same rate.

8. Softening: Another related thing to watch out is that you will notice that inbetweens will “soften” the texture details. The thinking is that it’s sort of OK as you don’t need as much detail on moving elements (you assume you will have some motion blur) and the image where it does not move will remain very clear. If unwanted softening occurs, and the Weighted Blend is not enough, you can consider little post tricks including slightly sharpening (maybe simply an unsharp filter, such as one found in our SmoothKit plugin set) and adding a bit of noise… Depending on what you are doing this might be more or less realistic.

9 Handling Multiple Motions: A “rough” garbage matte roto that isolates the foreground object might be necessary on the few frames where the problem occasionally occurs. After applying Twixtor to the frames with the garbage matte (on both foreground and background, with inverse mattes of course!), the inbetweened frames can be recomposited back together. When you make a garbage matte for this purpose, you can typically be a bit more “more fat” than the usual roto matte for compositing. In general since Layer selection is not animated it might be simpler sometimes to run the whole thing once and then animate something like the Motion Vector Quality or Motion Sensitivity and then cut back the relevant parts of the two comps together.

Of course, remember that Twixtor Pro has many advanced features specifically designed to handle crossing objects and multiple motions.

10. Fast Action: So, with fast action content, you will probably get better results with Twixtor from material captured with a 60i FPS camera than with a DV camera that does 12 of 15 real FPS, because the same action will move farther, per frame, in the 12fps footage than in the 30fps footage. If you have the option of shooting progressive or interlaced video, a basic rule of thumb might be if you have fast action (or because you are running with a hand-held camera) shoot interlaced (because we construct 60fps from the 60 fields, which gives us more frames to deal with, which reduces the motion per frame), otherwise shoot progressive (because we don’t have to reconstruct a frame from a field, you will get better quality results).

11. “Film Look” and Other Fields Related Problems: If Fields cause you problems, remember that we offer also a full FieldsKit which allows to flexibly organize your workflow to possibly create a better deinterlaced source for Twixtor. Note that some people want to simulate a telecine look by going down to 24 fps and then adding a 3:2 pulldown. This is certainly possible and FieldsKit will eliminate the need to create an intermediary render. But, it is our experience that you obtain something closer to a film look by simply Deinterlacing and applying ReelSmart Motion Blur and in that case maybe you don’t need Twixtor. For more on our products and “film look” issues, please read our film look FAQ (located here).

12. Flickering: Similarly weird flickering as sometimes happen with problematic film capture can have an impact. Note Twixtor mostly uses the black and white information and is by default hardwired to video brightness, which largely discounts blue. Sometimes just doing an RGB invert of the color source and using it as the tracking source will help. So particularly for shots that have a lot of blue, it might be worth to supply an alternate track which remaps colors and pushes up certain values. This also applies with certain dark shots (low-light level), it might be worth to expand their range and lift them up (making sure not to clamp values that would make entire regions “look the same” to Twixtor’s motion estimator).

[Alejandro Petrakovsky]

look at this video from 00:45 https://vimeo.com/25945429
for example we can say that the movement of the skate is so quick that warping is going to happen?

[Pierre Debroux]

so i had the same problem as you .i m working in vegas. when i extend the movie ( make it slower like in the tutorial for vegas show on creatvie cows) the movie stopped every seconds. if i don’t do it but instead of that , i make my subclip shorter (means i increase the speed of the footage ) and than i use twixtor than i don’t have the problem. sorry for my bad english.

[Pierre Jasmin]

It’s probably a project settings issue (mismatch of frame rate – media versus project or if the video has fields), that will generate what you are describing. If you have a short clip and .veg project you can send to techsupport at revisionfx dot com we can look at it.

Pierre