Tell it to try really hard(its an option)
Try not splitting the audio file
Try not telling it they are in chronological order
Try splitting the video files into a couple segments instead of audio
my guess is that your audio files plays slightly slower than your video files and the ‘drift back is confusing PE.
FYI it has been my experience that audio feeds run a frame or 2 longer per 30minutes than my video capture but I only know that to be true for the cameras and gear I use