I suggest that you start from scratch then do the following –
1) track the foreground object (man) with a loose tracking spline. Keyframe the tracking spline as often as needed to ensure that the foreground object is always enclosed within the tracking spline.
2) create a new tracking spline for the background. you have lots of co-planar areas to track. therefore, use multiple tracking splines when tracking the background – you should use the Add X-spline (or B-spline) tool to add a second or subsequent tracking spline onto the same layer. Ensure that your tracking splines here are sufficiently close to the area you want to track – this is to ensure that you do not force/ask mocha to track superfluous movement in the background.
3) track the background layer once the tracking splines are in place. it is always a good idea to begin your track at the frame which gives mocha the most pixel information. When tracking the background, ensure that the foreground layer sits at the very top of the Layer Controls Panel – and ensure that it’s Process switch (cog icon) is off, to disable tracking it again.
You shouldn’t require Manual Tracking unless your foreground object fully covers the background tracking area. Even then you will only require a few frames of Manual Tracking when auto tracking is impossible to achieve.
What you should try to do at the end of your tracking procedures is to use mocha’s Adjust Track module/tab/feature to tweak your results – if required.
HTH
RoRK
Intensive mocha & AE Training in Singapore and Other Dangerous Locations
Imagineer Systems (mocha) Certified Instructor
& Adobe After Effects CS6 ACE