Creative Communities of the World Forums

The peer to peer support community for media production professionals.

Activity Forums Adobe After Effects How to Render Alpha But Not Apply As Matte In Comp!!!?

  • How to Render Alpha But Not Apply As Matte In Comp!!!?

    Posted by Ross Newton on July 14, 2010 at 6:29 pm

    I’m an advanced After Effects user and also use Nuke. I finally had a situation where I needed to render an Alpha channel in an AE comp but I needed it to NOT matte the comp through that Alpha channel to the background. It seems AE does this by default. I tried lots of channel effects and render settings and couldn’t get it to do this. Here’s an example:

    I have a CG building over a sky I created with a solid and gradient effects. How do I render this out of AE with the building and sky as the RGB and the building’s alpha channel as the alpha? When I turn on my sky layer it fills in the alpha of the CG building. If I set the sky layer to have the CG building’s alpha, then it mattes the sky out of the comp. Even if I subcomp the whole thing and apply the alpha that way, it still mattes to the background. I can never use the building’s alpha and have it render the RGB of the sky. It seems to always “matte” the comp. I need to “unpremultiply” this matting effect.

    In Nuke, I can just pipe in an alpha channel and it doesn’t necessarily matte it.

    The only work-around is to duplicate the comp, turn off my solid-sky, and render that as “Alpha-Only” and also render my original comp as “RGB”.

    What am I missing?

    Jonathan Kempe replied 15 years, 8 months ago 2 Members · 6 Replies
  • 6 Replies
  • Ross Newton

    July 15, 2010 at 12:28 am

    Hi Dave, thanks for the response.

    I think you answered my question, that unless there’s transparency in the comp, then there’s no alpha. That’s what I’m trying to basically do, add an alpha channel but not use it for transparency upon rendering out.

    I know I can duplicate the comp and just render an alpha out of the building. I was trying to avoid that just to save time and see if this could be done, like other compositing programs.

    It’s basically like a shortcut for the compositor, even though we want the sky in the RGB of the shot, I wanted to include the building’s alpha channel as a way of isolating the building, and to do this quickly in AE.

  • Ross Newton

    July 15, 2010 at 5:16 pm

    Thanks again for the response Dave.

    Actually, I think you’re misunderstanding what I’m asking. I know how to render an alpha channel that represents the transparency of my comp using the appropriate render settings.

    What I’m trying to do is render an alpha channel with my comp, that DOESN’T represent any transparency and therefor doesn’t get matted in my comp and in the render. I want to “override” the alpha channel with what I choose.

    In the above example, I have a CG building and “solid” sky in my comp, so in AE’s world I have no alpha channel because it’s completley blocked by the full-frame “solid-based” sky. HOWEVER, I want the CG building’s alpha to “override” the alpha for my comp when I render it out. The PROBLEM is that, whenever I do a set channels or set matte on my comp, it always MATTES that alpha channel thus killing my sky layer and matting it to the background of the comp.

    What I want is a FULL RGB render of my building and sky WITH the alpha channel set as the building’s alpha channel… all in ONE render (yes, I know I can do it seperately). The reason I posted this is to see if I can do it in one render. Other compositing packages can do this with no problem, they don’t automatically matte your comps based on your alpha channel.

    Now, you’re probably wondering, why would I want to do this if I’m already in the comping process. In this case, I’m actually building sub-comp passes for a senior compositor who wants to bring in the building, sky, and then use the CG building’s alpha as a color-correction matte for the building. It just makes it easier if I can do all that in one pass.

    That’s the crux of my questions… can AE do this?

  • Ross Newton

    July 15, 2010 at 6:14 pm

    Yes, I want to be able to “select” an alpha channel.. And there’s layer effects to do this (“Set Channels, Set Matte, etc.) But you realize this wouldn’t be at the codec or render level… you would set it up in the comp and then just render as “RGB + Alpha” to whatever filetype you want, same as any other render and filetype that can handle the extra alpha channel.

    It’s just like any other render with an alpha, it’s just this alpha wouldn’t equal transparency. The alpha channel is just another channel, it doesn’t have to equal transparency, but it seems like AE forces this at the comp level.

    I even played around with some of the channel effects like “Set Channels” and “Set Matte”. No matter what, it always mattes the alpha through to the background color set for the comp. I was hoping there was a setting to turn this off. You see if it didn’t do this, then I’d have my building, my sky, and my alpha channel. As it is now, the alpha knocks out my sky and renders through to the AE background color, which is set to black.

  • Jonathan Kempe

    August 26, 2010 at 8:17 pm

    I am also having this problem!!

    I have lots of rendered tiff sequences separated into passes from Maya. It looks like this. First a Beauty pass with the main object and a real alpha channel, then on top of that a Glow pass, Spec pass, SSS pass etc. Those passes doesn’t come with a real alpha channel because they do not need a alpha since they are just black with lighter areas. And the lighter areas are all you want. SO, you use the multiply layer effect in the composition, the ADD (or Color dodge, screen etc). Voila! looks perfect against a background!

    But it only works against a background, like a solid, a filmclipp or anything. BUT, if i want to export the Maya files with a alpha channel and not the background, its seems very hard. I tried to change to checkerboard background but I cant get rid of the Background. I really need to render just the 3D to make the workflow easier. (I have 186 layers in the comp right now…)

  • Jonathan Kempe

    August 26, 2010 at 9:08 pm

    Thanks, I have tried that but it does not work at all unfortionally…

    Since some of the multiplied layers like GLOW creates glow outside of the Beauty-pass with alpha I cant use the Beauty as a mask because the glow will not show since it is masked away. I have also tried to do the Beauty white and lighten the glow, also to make the spec pass lighter and try to mask with the luma-matte, but that is not possible since it destroys the gradients and everything looks really bad.

    So, I dont really know how to solve this… I think it should be possible since After Effect for sure creates some kind of alpha-channel information to make it work when there is a background.

  • Jonathan Kempe

    August 27, 2010 at 10:42 am

    THANKS A LOT!

    This is exactly what I was looking for. I installed the full Knoll Light package, then just added the unmult effect to the separate layers (the multiplied layers with ADD, Screen etc. ) Then changed to checkerboard background and you can see the actual alpha.

    Then… render.. then… success!

    😉

We use anonymous cookies to give you the best experience we can.
Our Privacy policy | GDPR Policy