If you open the Info panel and watch that when you tap the spacebar, you’ll see the issue, After Effects has to cache audio each time it previews. So the first 10 seconds is it compiliing (probably not the right word) the audio file so that it can play it back alongside the cached video.
I don’t think there’s anything you can do about it, although WAV files are not compressed as MP3, so I think they take less time for preview playback.