Some compositing systems (AE, toxik, combustion, flame) are internally straight while others (Shake, Fusion, Nuke like FCP) are internally premultiplied by the alpha. None is right or better, you just have to be consistent, that is, as an operator/filter on the pixels you either expect one or the other. Much like for image IO, you usually set an incoming image sequence to the proper host basis if different (i.e. a premultiply or unmultiply button). If you select the wrong basis, then later in your project you will have issues.
Of course what follows (LONG) only apply if the Alpha is not full-on.
All this means is a semi-transparent pink tuna like color is as R,G,B,A
not-premultiplied: 1,0.8,0.6,0.5
pre-multiplied: 0.5,0.4,0.3,0.5
The not-premulted application (eg AE) will do a premult prior to display result (as part of the display transform itself). The already premulted one will just draw RGB since A is supposed to be already applied to the color values (or composite it on a checkerboard background). It’s the same color (but not the same numbers) and in floating point a lossless operation to convert one to the other back and forth.
Certain operations you want to do in one basis and others in the other basis, which assumes you know you are in a particular compositing basis, which is I think what Pete is complaining about, some FCP tools are apparently not premulted and another operator does not know that (as FxPlugs plugins we are told to expect premultiplied images).
For example gamma conversion and most colorspace type work as well as a sequence of alpha channel manipulation, you probably want to do in so-called straight space,
while most processing that composite / adds / average pixels you probably typically want to do in premult space.
For the particular case of keying, it’s not unseen in premulted system to have the option not to premult the result out so one can continue manipulating the colors and alpha with a tool after it — prior to applying the alpha premultiplication necessary for the color to make sense at the end. In systems that allow that, then the user is expected to apply a separate premult filter after that (or post-multiply) before a compositing operation (where the two inputs/layers are merged).
Now understandly FCP tries to maintain that workflow complexity out of your sight as it does not claim to be an effects system and Apple leaves that sort of process more exposed to EG Shake users… However there is certainly a bug in FCP keying tools if you can’t apply a chromakey followed by a gaussian blur without getting back background pixels your prior key is supposed to have removed. Of course if one removes complexity and does not provide workarounds then the user response is simply, well I’ll do that work in AE…
It would appear that this could maybe just be an internal compositing state tag (premulted, unmulted) issue or something like that not handled prior to passing back the image(s) to the next operator/filter.
I say that not knowing the code from the other side but recognizing that bug when I see it as is common to do such mistake when you have image processing libraries that have to run on either basis as plugin developers for multiple host applications must do. I imagine FCP has to do some minimal book keeping of that nature since for example an FxPlug expects premulted images and an AE plugins straight pixels… to say, since the bug is seen with many FCP video filters, the bug might be at another level of the system where compositing state is handled / managed ?
Bug has been filed.
Pierre
http://www.revisionfx.com