[Gavin Impett]

I’m assuming no Forge or Noise Reduction, or Ed’s plug-in(which I think is probably your best bet), so I’d recommend starting by throwing on a paragraphic eq, or as many as it takes (more is better, as Gene Simons would say) on the female track and see if you can knock out his voice that way. Then render to a new track and see if that kills the flanging. You didn’t say if you tried adjusting the pitch, but it might do something.

Think multiple passes/renders to get this right.

Consider sub-clipping early and often. You might be able to key-frame the audio/distance change, but you’ll probably have better success by cross-fading clip to clip.

Consider adding a chorus to the male voice (If you can’t beat them, join them, as Freddie Mercury once said).

There are better audio folks around, but yes I’ve had that bit of fun.