AE – Optimizing Lip-Sync or Keyframe-Handling

[Bogdan Brak]

one more concrete idea how it should work (without knowing how to implement this)

let’s say we have three diffrent states:
1. no one speaking (if music value <10)
The mouths’ height = 100%

2. someone’s talking (if music >= 10 && < 20)
The mouths’ height = 130%

3. someone’s talking loud/screaming(if music >= 20)
The mouths’ height = 160%

The mouths’ height is ALWAYS trying to get down to 100%.
Maybe this way:
for(i = 100; i > 100; i–)

F.e. If the height is at 140% because it’s in the third state it should count down to 100%.

That would also mean, that after effects should NOT check the state every single frame. It must check it maybe every 500milliseccond.

I hope you can follow me…
I try to sum it up:

The audiospectrum must be devided and mapped into three states.
– no sound
– sound
– loud sound

after effects must check the state every 500ms and not every frame.
the mouths height should always try to close and animate the mouth if its value is bigger than 100%

and… the whole thing should get an Easing.

Any ideas how to implements this in AE?
cheers

and again…thanks in advance

[Bogdan Brak]

ok…now I think I know how to explain it in a few words.
The whole thing shoul act like a smooth VU-meter. It should not check the peak 30 times a secon. Maybe only 4 times. And after every check the animation/mouth is trying to come down to 100%.

[Xavier Gomez]

What you are describing (reading audio amp values at periodical time intervals) doesnt look too good, not very convincing motion. It can be written like this:

var sc = transform.scale // the keyframe value of the scale
var T = 0.50 // the period at which the audio amp is read
var t = time-(time%T);
var audioAmp = thisComp.layer(“Audioamplitude 2”).effect(“Beide Kanäle”)(“Schieberegler”).valueAtTime(t);

var extra;
if (audioAmp>=20) extra = 60;
else if (audioAmp>=10) extra = 30;
else extra = 0;
sc +[0,easeOut(time-t, 0,T, extra, 0)];

————

I would like to suggest another expression but for some reason it won’t copy in the browser; part of the code is being chunked out, even if i copy it in the “code” box. Sorry.

Xavier.

[Xavier Gomez]

Finally got it to paste it correctly:

// the parameters to be tweaked;
var thresh1=10; var extra1 = 30;
var thresh2=20; var extra2 = 60;
var dampling = 0.5;

var sc = transform.scale // the keyframe value of the scale
var audioAmp = thisComp.layer(“Audioamplitude 2”).effect(“Beide Kanäle”)(“Schieberegler”);
var dt = thisComp.frameDuration;

var t=time;
var val= audioAmp.valueAtTime(t);
var val_old = audioAmp.valueAtTime(t-dt);

while(val_old>val && t>0){
t -= dt;
val=val_old;
val_old=audioAmp.valueAtTime(t-dt);
}
var extra;
if (val>thresh2) extra = extra2;
else if (val>thresh1) extra = extra1;
else extra = 0;

sc + [0, extra * Math.exp(-dampling*(time-t))];

[Bogdan Brak]

Hey great. Thanks. Got to play around a little bit with it.

By the way. Yesterday I found a way.

simply that code:
temp = thisComp.layer("Audioamplitude 2").effect("Beide Kanäle")("Schieberegler");
if(temp < 8){
temp = 0;
}
[100 ,temp*20 +140]

than I made the curve of the amp-values smoother. (There’s a function for this in AE / Window -> Smoothen)

finally I added a little bit of the CC Motion blur effect.

Acutally… it looks pretty good.

But still got to check your code.