Hey Oliver!
I was having very similar issues, and my audio guy and I were going back and forth, pulling our hair out trying to figure out what the heck was going on. We even had a spreadsheet going of different combinations of preferences in X2Pro checked, to weed out any variables.
Anyway, I’m not sure if this has been mentioned yet or not, but I stumbled on this forum post on X2Pro’s website that completely fixed our problem. It was sample rates.
https://x2pro.net/help/support-forum/12-x2pro-professional/1253-sound-is-out-of-sync-in-aaf-file.html
Make sure Pro Tools (or I assume any DAW) DOES NOT apply SRC (sample rate conversion) when importing.
I hope this helps. Please forgive redundancy if this has already been raised.