I would say, do the tracks for different parts of the footage.
Create a null object layer and apply all of the different segments to the null object layer. Then any effects or masks made (on different layers) there after can be parented to the null. There is actually some expressions in a tutorial featured on this sight called “a walk in the park” by Eran Stern. That seems to be to keep a track happening while the subject is leaving the frame. Hope this has been some help. cheers from jules!