It would help for us to see the clip in order to analyze the problem.
If you are trying to run the camera tracker of a football game where players are running around in the shot, you are not going to get a good track since it gets confused by all the motion. You would need to mask out the moving players.
If the camera is mounted and zooming you have even less of a chance of getting a track. The camera tracker needs parallax to determine the dimensionality of the shot.
But that just a wild guess without seeing what the shot looks like.
Tom Daigon
PrP / After Effects Editor
http://www.hdshotsandcuts.com
https://www.youtube.com/watch?v=DxPrG3WUyz8
(Best viewed at 1080P and full screen)
HP Z820 Dual 2687
64GB ram
Dulce DQg2 16TB raid