You are trying to track an area that’s way too big and complex for one shape. Usually one would track and then roto one piece at a time eg. the thigh and the leg would be separate shapes.
“Yes, I technically could just realign/adjust the points manually, but that is not only extremely tedious, but horribly time consuming.”
There’s no magic bullet for roto, there are just things that might save you some time and trouble.