Hi Ben,
to quote you “you need depth (foreground objects, background objects) in order to get a good 3D solution. And your trackers need to be spread out on the back wall. (You probably already know all this, but it’s worth pointing out for other people who might stumble across this post.)”
That’s exactly the problem i realized, i didn’t have any depth to track so my trackers fail, which i knew they would. But as far as camera movement is concerned. in the longer shots, its dollying in-and-out as well as panning left-to-right. Which equates to a quite a bit a of parallax.
but i agree with u that I might be out of luck.