One problem you might be having is that there will be discontinuities at 1 and 2 seconds, where the amount of delay changes abruptly. You might want to ease through those changes with something like this (not tested at all):
easeTime = .25;
delay = thisComp.layer("xp").effect("d")("Slider");
t=time;
d = delay*thisComp.frameDuration*(index - 1);
if (t < 2){
dly = ease(t,1-easeTime,1,-d,0);
}else{
dly = ease(t,2,2+easeTime,0,d);
}
thisComp.layer(6).scale.valueAtTime(time + dly)
Dan