Marco Solorio reviews: RE:Vision Effects, ReelSmart Twixtor with FieldsKit Deinterlacer

Originally published December, 2002

Not even Einstein could have made a time machine as good as this one!

Looking at traditional methods of time-remapping, frame-rate conversions and other “spatial continuum” methods, we’ll see how the popular Twixtor plugin compares to the task at hand. The results are interesting with some surprising results. We’ll also look at the ways Twixtor works well with its sister plugin, “ReelSmart FieldsKit”. Those looking into film-look conversion techniques will especially benefit from this review.

Developer Spotlight:

It’s safe to say that Pete Litwinowicz and Pierre Jasmin, the creators of the ReelSmart plugins, are a couple of pretty smart guys. They’ve developed a process in which their plugins look at pixel information and interpolates new data based on morphing algorithms as opposed to traditional frame-blending methods. And best of all, the new data that is created is formulated from areas of the image that need it, leaving the areas that don’t need it untouched, thus preserving the integrity of the shot with as less interpolation “loss” as possible. Now that’s “Reel Smart”!

About Twixtor

Let’s first look at the basics surrounding the Twixtor plugin. You’ll be hard pressed to find an application that won’t run Twixtor. Supported applications include Apple Final Cut Pro, Apple Shake, Adobe After Effects, Adobe Premiere, Discreet Combustion, Discreet Flame, Avid, Pinnacle Commotion, Quantel generationQ and Eyeon Digital Fusion. Supported operating systems include Mac OS 7.5 to OSX, Windows 95 to XP and Irix 6.5 and up. Now that’s compatibility! For the full low-down on product support, visit the RE:vision compatibility page. The process of this review will be using Twixtor in Adobe After Effects 5.5 on OSX.

Twixtor comes in two flavors: a standard version ($329.00 USD) and a Pro version ($495.00 USD). Note that some applications (e.g., Shake and Avid) only offer a Pro version with slight price reconfiguration. Some applications also support Render Node support ($165.00 to 198.00 USD). There are also upgrade offers from the Standard version to the Pro version ($165.00) for applications that support the Standard version.

The Pro version adds 16-bit rendering support (for applications that support it, like Adobe After Effects), which to me is the biggest reason to go for the Pro version since I work with nothing but 10-bit uncompressed files in 16-bit After Effects projects in “Trillions mode”. Also in the Pro version is the ability for Twixtor to assign the foreground/background setting with a matte so Twixtor knows which portion of the image comprise a foreground object(s) and which areas comprise a background object(s) that move in dissimilar directions (including the possibility of one object(s) staying static). This function can prove vital for those really tough clips with intricate motion circumstances.

The “prime cut” to this beefy plugin is it’s ability to change the speed of a clip (either constant or keyframed) and to change frame-rate speeds. As noted from the RE:Vision Effects website:

ReelSmart Twixtor enables you to speed up, slow down or frame rate convert your image sequences with visually stunning results. In order to achieve its unparalleled image quality, Twixtor synthesizes unique new frames by warping and interpolating frames of the original sequence… employing RE:Vision’s proprietary tracking technology that calculates motion for each individual pixel.

This is very important to understand because Twixtor doesn’t merely frame-blend the motion together like traditional methods (although the option to do that is there).

Getting started with the plugin

The first thing you’ll encounter when using Twixtor for the first time is its inherently high learning curve. Even seasoned compositing pros might find the plugin a little daunting at first. But rest assured that given the time and patience, you’ll soon grasp the concepts, functions and power of Twixtor. It’s really a matter of getting familiar with the plugin terms and function relationships/differences. Download all the Twixtor tutorials, study and fiddle around with them. I found this the best way to get accustomed to the plugin as my initial ideas of controlling the parameters were way off target. Don’t waste time trying to figure it out on your own like I first did… download the tutorials, read the manual and you’ll be much happier in the end!

The biggest thing to understand when using Twixtor is how to initially treat the footage you import in. For example, when you import a clip into After Effects, the default settings for the clip will most likely be set to either “Upper Field First” or “Lower Fields First”. You must change the Separate Fields setting to “Off” for Twixtor so it can fully use both fields of information from the clip. My initial lack of desire to read the manual and play with the tutorials (but over ambitious desire and excitement to just start “playing with the plugin”) transcended into results that I referred to as, “ewww… why does this look so bad?!!” In fact, failure in going through the manuals and tutorials may make you utter words like, “man, this plugin sucks!” In reality, proper education will result in words like, “man, this plugin rocks my world!”

Next on the “to-do” list of items is deinterlacing the footage. Even if you’re bringing interlaced footage in and spit it back out to the same exact frame-rate with interlacing, Twixtor still deinterlaces the footage in order for it to properly work its magic. This can be done in two ways. The easy way is to have Twixtor deinterlace the footage using its own simple deinterlacing function. The best way (and the only way I do it) is to use RE:Vision Effects ReelSmart FieldsKit. This will add to the already long render time you’ll encounter, but it will be worth your efforts as the quality is bar none.

When placing a clip in the After Effects timeline, it’s good practice to use layer solids as the layer in which to use Twixtor and use the imported clip as a Color Source target (the default otherwise targets itself). Although this isn’t necessary in many cases, it is required should you want to exceed the clip’s duration past its native length. So for example, if you want to stretch your 5-second clip to 10 seconds, you cannot use the Twixtor plugin on the imported clip itself because the clip can’t “physically” fill 10 second of time in the AE timeline. By using a solid, you can then make the solid’s length as long as the comp itself, target Twixtor’s Color Source to the imported clip and then stretch the duration of the solid to your heart’s content. The red arrows point to the link between the solid and the source clip in the AE window (other Twixtor options have been cropped out in the image).

Also make certain that your solid’s frame size matches your source clip’s frame size or you’ll get a red solid with no color information. If you move the source clip from left to right in time in the AE Timeline window, it will affect the solid layer’s target information as well. You can also see that the source clip doesn’t need to be visible (eye icon) and the quality switch can be set to low since neither of these settings affects the solid layer. Another important note is that you cannot add any filters (or masks) to the source clip because the targeting solid wont recognize them. This is a limitation of After Effects and not Twixtor. Other plugins that use solid’s targeting layer color information succumb to the same problems. You’ll need to precomp the source clip with its effects or add the effects on the solid layer after the Twixtor plugin. You also cannot add masking to the solid layer and must be precomped or assigned a track matte for any masking procedures.

Deinterlacing: More than a trendy effect

As I mentioned earlier, deinterlacing your footage is required in order to work with the Twixtor plugin. This process occurs even if your input and output settings are set to Lower or Upper Field First in Twixtor’s settings. If you’re using Twixtor to simply alter the clip’s speed, then you should match your current clip’s attributes to the other footage based in your overall project. With NTSC footage, this means setting the Input and Output Frame Rate to 29.97 with Upper or Lower Field set appropriately on both. Changing the “Output: Fields” setting to None will output your effect with no interlacing (as shown in the image). Some people like this non-interlaced flicker effect as it reminds them of a film-look process (more on this later in the article).

For the most accurate deinterlacing methods, let’s look at RE:Vision Effects’ ReelSmart FieldsKit plugin set. FieldsKit is composed of three distinct plugins; Deinterlacer, Pulldown and Reinterlacer. We won’t be using the Pulldown or Reinterlacer plugins in this review, but we will be using the Deinterlacer plugin in association with Twixtor from this point on in the article. The two plugins together are a perfect marriage for optimum quality and I personally wouldn’t do it any other way. So let’s get a closer look at Deinterlacer.

The screenshot of the FieldsKit Deinterlacer plugin shows the default settings. Here’s our goal: we want to deinterlace the footage in a way that will maximize the available amount of information that is currently in our source clip’s 29.97 interlaced footage without throwing out the little information we have while also not introducing unneeded interpolation. The best way to perform this is to take each even and odd field group and turn them into their own frames. In other words, we want the upper field to create a full new frame and the lower field to create a full new frame and so on. We can do this in two ways… doubling the clip’s duration, or doubling the clip’s frame rate. Opting for the latter method simplifies things so we don’t have deal with duration conversions. So let’s get to work.

Import your footage. Like the earlier Twixtor example, control-click or right click the clip in the AE Project Window, select “Interpret Footage”, click “Main…” and change the Separate Fields setting to “Off”. Create a new comp and make the Frame Rate of that comp 59.94 FPS. Place your imported clip in the AE Timeline. Apply the Deinterlacer plugin to the clip. Let the dog out to go pee. Oops! Thinking out loud here!

Now we can start tweaking with the Deinterlacer settings. Change the Timing Mode setting to “2X FPS, 1 Frame Per Field”. Since we doubled the comp’s frame rate to 59.94, we’ll need to use the 2X FPS option. Select the proper Field Order in accordance with the type of clip you are using. For me, I’m using an AJA Kona SD for this review so my setting is Lower First. My Aurora Igniter system on the other hand uses the Upper First setting (note to self: call up all the manufacturers and beg for a common standard).

So now we’ve created a 60p comp (59.94 progressive). This is incredibly valuable when dealing with any kind of time remapping because we essentially have an overcranked “clip” (precomp) to work with. So for example we want to create a 400% duration increase, we’ll have a smoother result based on the extra frames of the 60p sequence. If we worked from a 30p sequence, we’d be creating frames based on further interpolation values, which in turn will result in a more “interpolated look.” Think of this 60p technique as a higher “bit-rate” for time expansion! There are times when going from 60i to 60p isn’t totally necessary and going from 60i to 30p is good enough. You’ll get a better grasp of this with experience.

The next ten settings, from Fill Method to Sharpen are what you the user tweak to work the magic of the Deinterlacer plugin. Each type of clip will require different settings altogether. There is no “general settings guide” because each setting does something different and has advantages/disadvantages against the other settings. A clip of a head shot will require different settings than a clip with two trucks crossing their paths with text on the sides of them. This is a whole other article on its own! But for the sake of demystification, I sometimes have the following settings:

• Fill Method : Blend or Best 5 Neighbors (sometimes 3)
• Detect Motion: 3 Frames Compare
• Detect Method: Large Area
• Motion Tolerance: .35 (I’ve never gone lower than .30 or higher than .40)
• Mask Suppress: 0 or 1, typically
• Mask Grow: 0 to 5
• Mask Feather: 0 to 5
• Smooth Vertically: 0 but if it’s called for, between .5 and 1
• Sharpen: 0

Remember, these are general settings that get tweaked around on a clip-by-clip basis. Simply plugging these values into Deinterlacer on all your clips wont allow Deinterlacer to work at its full potential and may even make things worse. And USE the “View Motion Mask” option to see what areas of the image Deinterlacer is affecting. This is a very cool and powerful tool to use and lessens the guesswork in all of this. Play around with the settings until you get it right!

So how does Deinterlacer compare to other deinterlacing methods? Glad you asked. Let’s just say the other methods don’t compare in the least! Most deinterlacing plugins simply blend the to fields together resulting in loss in vertical resolution (although Deinterlacer does have this option should you opt for it). Below is a dynamic image comparison example of different deinterlacing methods. This is an impractical image in real world applications, but what the test elements do is challenge the deinterlacing plugins to some impossible tasks. The most impossible task is preserving 1-pixel lines. Why? Because the deinterlacing method takes 1-pixel lines (thinking they’re interlaced fields) and blends them together. Because of this phenomenon, detail in sharp edges and text can also get tossed out the window through traditional methods. Another problem with traditional deinterlacing tricks is that a single object can double in appearance due to frame blending. An example of this would be a moving, talking head and instead of two eyes looking at you, you get four eyes trying to blend together… scary!

You’ll need JavaScript active to make the comparison function operate.

Barend Onneweer, one of our esteemed Creative Cow leaders has a deinterlacing article (COW Admins: coming soon!) here at the Cow that covers the different methods of deinterlacing video with After Effects. He too concludes that FieldsKit Deinterlacer is the best way to go.

Okay, enough on deinterlacing. Let’s start really playing with Twixtor!

Frame rate conversions: 60i to 24p

One of Twixtor’s powerful features is its ability to convert frame rates. The most popular method for this of course is converting standard 60i video footage to 24p. If you’re new to the video world, the reason for doing this is to convert NTSC 29.97 FPS interlaced video to a 23.976 FPS progressive format, known as 24p. This 24p conversion process is part of the technique to make your video look like it was telecined from a film source since film is commonly shot at 24 FPS.The basic process of converting 60i to 24p is three-fold.

1. Use FieldsKit Deinterlacer to convert 60i (29.97 FPS interlaced) to 60p or 30p. Precomp or sub-sequence the clip.
2. Bring the precomp into a new comp and apply Twixtor to the precomp and change the settings to input 60p or 30p and output 23.976.
3. Render your comp with 3:2 pulldown back to 60i (29.97 FPS interlaced). Those with 24p NLE support need not perform this step.So let’s dissect this three-step process.

Deinterlacing

Use the techniques mentioned earlier in the article to deinterlace your footage. I personally prefer to always convert to 60p as opposed to 30p since this will be beneficial to other areas of concern. You now MUST precomp this layer. If not, Twixtor will merely look at the image information, minus the deinterlacing process. So precomp this layer and place it in a new comp. Apply Twixtor to the precomp layer. Add 2 cups of sugar and follow as directed.

60p/30p to 24p conversion

This must be done correctly in order for you to achieve the best results. Remember how I mentioned the Deinterlacer plugin has different settings to achieve optimal results in different situations? Well Twixtor is the same way. You have different ways to perform interpolation, motion vectors, motion blur and motion sensitivity. You really need to tweak the settings to get the best results… no “general settings” here either.

The one thing that will always be the same is the way in which you convert the frame rate. Looking at the image, we can see I told Twixtor the incoming source is progressive (marked as “None”) with a frame-rate of 59.94 FPS. I then told Twixtor that I need to keep it progressive on output but at a new frame-rate of 23.976 (as displayed by 23.98).

The thing that’s tricky about Twixtor is avoiding a goopy mess when interpolating frames. Through serious tweaking of the settings, you can avoid this. Additionally, using the Pro version can really help avoid this by using a matte to specifically tell Twixtor which portions of the image need interpolating and which don’t.

Below is another dynamic image comparison example of the finished 60i to 60p/30p to 24p conversion process by means of different methods.

Going back to 60i and adding 3:2 Pulldown

There are several instances when going back to 60i with 3:2 pulldown isn’t needed. They include finishing off to progressive DVD, CGI production, PAL conforming to 25 FPS or printing to film. Otherwise you’ll need to go back to 60i video and introduce 3:2 pulldown. And if you don’t perform the 3:2 pulldown, you’re motion will have an ugly fast-slow judder to it. Some systems (like my AJA Kona SD and Aurora Igniter Film) can fully edit in native 24p, so rendering 3:2 pulldown isn’t necessary (this puts a VERY large smile on my face!). These 24p compatible systems add the 3:2 pulldown on the fly on output to your monitor and deck. Totally trick.

The image below shows part of the After Effects Render Settings window to add 3:2 pulldown to 60i from a 24p timeline.

The process in whole looks something like the image below. Keep in mind this is a very basic diagram of the process and that other methods can be introduced (like conforming instead of interpolating).

What a freakin’ mess! Obviously if you have means to a 24p camera… use it!!! Note also that the 30p conversion step can also be a 60p conversion process (my preference ni most cases) but is shown as 30p for simplicity’s sake.

A little off topic: Conforming
Obviously the best way to get a 24p look is to shoot with a 24p camera. But if that’s not possible, then shoot with a PAL camera running at 25 FPS. Then convert that 50i footage to 25p with FieldsKit Deinterlacer and conform that 25p conversion to 24p. By conforming, you’re keeping all the frames in tact and only decreasing the speed by 96%… hardly noticeable by any account.

You can also conform your 60i to 30p conversion to 24p, which will reduce the speed by 80%, which is noticeable. This is great when audio is not part of the footage and you want a slight overcranking effect. The process slightly slows down the footage enough to give it a smoother appearance in motion.

Time remapping

It would only make sense that Twixtor would also be able to perform speed-change functions as well. Whether your speed is constant or keyframed for a “Matrix” type look, the results can be awesome. There are three “Stretching Methods” as Twixtor calls it; Constant Stretch, Keyframed Stretch and Speed. The “Stretch/Speed” value setting acts as they should. A setting of 2.000 in Constant Stretch mode will double the duration of the clip (don’t forget about the solid layer rule in the beginning of this article). To keyframe a time remapping sequence, I really like using the Keyframed Stretch option. Make sure you also check your Motion Blur Amount setting… to little and it might look a bit jittery and too much may induce too much motion blur and possible “goop”. This function also works in accordance with the Motion Sensitivity value. After you get used to these three different styles of time manipulation, you’ll soon see the advantages it has over AE’s time remapping system. And keep in mind too that you will not need to use any of AE’s frame blending functions to make all this work. Twixtor and FieldsKit work their magic independently under their own superior algorithm systems.

In some strange instances however, I found that using AE’s time remapping function with frame blending to be better suited… but this is on a rare occasion. I can’t really explain it other than you need to try both methods out first before settling on a final result. More than likely however, you’ll stick with Twixtor. I felt I needed to add this info because in my experience, Twixtor isn’t the answer 100% of the time. And if time is a serious issue and you can sacrifice some quality (when Twixtor does show superior results) then using AE’s time remapper with frame blending might be a better solution because Twixtor (especially in conjunction with FieldsKit Deinterlacer) is a serious render hog. More on that later in the article.

Despite the serious render times and monotonous tweaking one may have to succumb themselves to, the results can be truly amazing to say the least. I’ve taken the duration of clips up to 800% (that’s a reduction in speed from 100% to 12.5%!) with KILLER results. They looked like they were overcranked at 192 FPS! The more organic and natural the shot is, the more realistic the in-between frames will be. Additionally, if you’re strictly staying in an interlaced environment (60i for example), Twixtor performs an incredibly beautiful job of outputting extended durations with proper interlaced fields… something After Effects can only dream of. Even my above test image with an 800% increase in duration maintains detail in the blue jeans. Likewise, the bubbles on the left margin look natural and absolutely beautiful in every organic sense. Very cool stuff.

From spatial effects to special effects

You can get some really cool effects with Twixtor too. Using the Motion Blur Amount and Motion Sensitivity values can produce some neat results. Careful though… setting the Motion Blur Amount setting to high values can really tax your CPU. Below are examples of using these motion tools in a creative way.

The dark side of the mooooo

Pink Floyd aside, there is a dark, ugly and evil side to this beast. Fact is, Twixtor is a tough cookie to crack. I think part of it is due to the user interface. I was confused at first how the speed options worked. Maybe adding a percentage system would help. But in the long run, it’s just a matter of getting used to the UI and with time, you’ll get accustomed to it. I would really recommend downloading the demo and getting a feel for it. If it’s just too much for you to grasp, then maybe sticking with the methods you already know may be best. And for goodness sake, download all the tutorials and manuals! There’s even further help as found in the RE:Vision Effects CreativeCow.net forum with help from Pete Litwinowicz himself (the creator of the plugin). As complicated Twixtor might be at first, there’s no excuse for not finding help with the forum and online tutorials. Seek and ye shall find.

The Jekyll to this Hyde of a plugin is it’s a pig of a render hog. Render times will increase when you increase the Motion Blur Amount value, the Motion Vector Quality setting and a few other settings. The price of beauty takes its toll! Below are some render time comparisons using my PowerMac G4 DP 800. Note that all the render times for Twixtor include the processing time it takes to deinterlace the footage using FieldsKit Deinterlacer, so it’s two-fold.

Closing Arguments

Yeah Twixtor and even FieldsKit Deinterlacer might require a degree in brain surgery to learn the nuances of the plugin, but the time to fully understand them will reward you ten-fold. I would say I use Twixtor about 95% of the time over AE’s built-in time remapping functions… even with the high render time overhead. It’s just so beautiful! And quite honestly, using Twixtor without FieldsKit is brainless. Buy both, you’ll be happier. But like I said earlier, first download the demo versions to make sure you can grasp the concepts of these powerful plugins. Maybe in time we’ll see a user interface update and render time improvements. Until then, it’s still the best damn way to play time machine.

I can’t give Twixtor a perfect 5 rating because the learning curve is high, it may require a lot of tweaking and the render times suck. Besides that however, the accuracy and fluidity of the final piece can have the potential of sheer amazement. I highly recommend this plugin to those seeking absolute superiority in time frame manipulation.

Pros: Very accurate. No image doubling. Vector based morphing algorithms. Could pass for overcranked film.	Cons: Slowwww rendering times. Intricacy can be daunting at first. Usually requires a lot of tweaking.
	I give it 4 COWs

Price as tested:

ReelSmart Twixtor Pro – Web Delivery  $495.00 USD

ReelSmart FieldsKit – Web Delivery  $89.95 USD

Links: RE:Vision Effects – www.revisionfx.com

Marco Solorio is a multi-award winning digital media producer in the San Francisco bay area. He owns and operates OneRiver Media, which focuses on producing animated content for broadcast, as well as serving production needs for content developers. And of course, SpongeBob SquarePants is his third most favorite cartoon in the world.

::: Page design, layout and image dynamics by Marco Solorio of OneRiver Media
::: Article and images © 2002, Marco Solorio

Enjoying the news? Sign up for the Creative COW Newsletter!

Sign up for the Creative COW newsletter and get weekly updates on industry news, forum highlights, jobs, inspirational tutorials, tips, burning questions, and more! Receive bulletins from the largest, longest-running community dedicated to supporting professionals working in film, video, and audio.

Enter your email address, and your first and last name below!