It looks like you are successfully working through your tracking concerns and learning that it is important to test PRIOR to actual production. This eliminates the “fix it in post” thinking that can drive up costs and efforts if you don’t test.
Some other thoughts: When AE tracker wants to track the wrong object, you can usually mask out that object, track, then delete the mask for the final render.
You can, of course, shoot in brighter lighting and then make the “darkened room” look in post. This would also simplify some of the tracking concerns. Or, as Dave suggested, make your tracking points part of the set. Maybe your set could be partially virtual in the sense that it is hard to know what was a practical and what was created in post.
Best of luck on this project.
Jim