Yes, use either pan/crop or track motion (the latter would probably be easiest).
And yes, I’d say use only two keyframes, and make sure motion type is “Smooth”.
If it’s still not slow enough, you can use a tool like Fusion to do the same thing, but it’ll also give you control handles to exaggerate the slow in/slow out from its default setting.