• Adobe After Effects Expressions

  Using sampleImage to drive CG render passes

  Posted by Joel Gerlach on March 31, 2016 at 12:38 am

  Hey everyone!

  So we’re building a CG theater for use in VR, and instead of using the movie screen to light the geometry in the scene (requiring us to render out every single frame of a movie to get interactive lighting on the CG assets) I had an idea to use something like sampleImage to drive the different render passes (diffuse, glossy, etc), which means all we’d have to do is render out the theater one, and then let the movie itself drive the lighting in AE.

  I found a post here: https://markos.co.nz/2012/05/after-effects-expression-use-the-luminance-of-one-layer-to-drive-another-layer/ which uses the expression pasted below. But it sounds like sampleImage might take a long time to process for each frame of the film, or may be too robust for our more simple simulation.

  So I’m wondering… do you all know of a good expression set I could use to sample the brightness & color of the movie file and then use it to drive the brightness & color of the combined render passes? Or perhaps expressions aren’t needed at all?

  Thanks so much for your thoughts! Appreciate you guys.

  exposure = effect("Exposure")(3);
multiplier = 2;

  driverLayer = thisComp.layer("MuzzleFlash");
  samplePoint = [0,0];
  sampleSize = [1280,720];

  lightnessSample = driverLayer.sampleImage(samplePoint,sampleSize);
  driverLightness = rgbToHsl(lightnessSample)[2];

  exposure = driverLightness * multiplier;

  Kevin Camp replied 10 years, 5 months ago 2 Members · 5 Replies
• 5 Replies

• 

  Kevin Camp

  March 31, 2016 at 4:44 pm

  it seems like it should work for you… I don’t think sampleImage() will slow you down too much (it does disable multiprocessing, if you are using cc2014 and earlier)

  if you have many expressions using sampleImage() that are all sampling the same layer, and you feel that it is slowing your renders down. you could apply a single sampleImage() expression to a color expression control (effects>expression control>color control) to sample the color, then use that sampled color in the other expressions.

  i’d apply the color control to the movie layer and add this expression to the color control to sample each from oaf the movie:
  sampleImage(anchorPoint,[width,height])
  if you step through the timeline, you should see that color chip in the color control change color.

  then you exposure expression could look something like this:
  driverLayer = thisComp.layer("My Movie Layer");
multiplier = 2;

  lightnessSample = driverLayer.effect("Color Control")("Color");
driverLightness = rgbToHsl(lightnessSample)[2];
driverLightness * multiplier;
  but unless you have many expressions sampling the same layer, I don’t know that you’ll see a big change in performance.

  Kevin Camp
  Art Director
  KCPQ, KZJO & KRCW
• 

  Joel Gerlach

  March 31, 2016 at 5:45 pm

  Kevin you’re a genius. Let me give it a go and see how it works out!
• 

  Joel Gerlach

  March 31, 2016 at 11:23 pm

  Kevin, quick question. I was thinking I could use an exposure effect and set the channels to individual so I could use sampleImage to drive exposure on a R,G,B basis and instead of transforming the color I could drive individual exposure channels after the color neutral CG theater pass.

  About that, though, with the section
  driverLightness = rgbToHsl(lightnessSample)[2];

  That’s creating a saturation result, right? If I understand rgbToHsl correctly. If I was going to modify this expression to drive individual color channels would this still be the best approach? Final question: if the end result is R,G,B exposure values, I wouldn’t want to run this expression under each exposure value, should I modify the original sampleImage to give out those RGB values and then reference them in the exposure channels? Not sure the best way to approach that.

  Thanks for your help. I owe you a big one. 🙂
• 

  Joel Gerlach

  April 1, 2016 at 8:19 pm

  For anyone interested, I figured out my own answer by removing the rgbToHsl expression and replacing it with the individual RGB vector appropriate for each exposure. So I went into the Exposure settings, changed it to “individual channels” and then used the following expression on each channel’s exposure properties:

  driverLayer = thisComp.layer("My sampleImage layer");
multiplier = 9;

  lightnessSample = driverLayer.effect("Color Control")("Color");
driverLightness = lightnessSample[0];
driverLightness * multiplier;

  Where lightnessSample[0] is set for Red, 1 for Green, and 2 for Blue. In order to mimic the higher contrast of the screen, I brought the gamma of the diffuse pass down to .38 (for each channel) and .65 for the glossy pass (as glossy surfaces are more visible than matte due to reflectivity). The end result is perfect, and will save a lot of render time. And, at least in AE CC2015 the sampleImage script runs incredibly smooth.

  Thanks Kevin for your help!
• 

  Kevin Camp

  April 1, 2016 at 8:30 pm

  rgbToHsl()[2] returns the lightness component (the 3rd value of the HSL array).

  rgbToHsl()[1] would return the saturation and rgbToHsl()[0] would return the hue… i’ve made that mistake more than a few times…

  if you were going to drive color channels, then you probably don’t need to convert the RGB to HSL, you could just use the RGB values, unless you are using an effect that requires different values (most of ae uses RGB).

  so if you reworked the expression, it would simply be:

  driverLightness = lightnessSample;

  at which point you might want to rename the variables 😉

  but that’s up to you.

  Kevin Camp
  Art Director
  KCPQ, KZJO & KRCW

Reply to this Discussion! Login or Sign Up