you could always use the method where you space out a number of duplicate layers close together in z-space. If there is a lot of motion, the motion blur may cover up the fact that it is not a true 3D object.
You can apply a blur to the edge only by creating a new adjustment layer with a Gaussian blur of 4. Then set the transfer mode to colour and you have a slightly softened edge before the key. Ive used this method countless times on DV footage and its allowed me to get a much cleaner edge than without it.
Im not sure about the settings as I havnt tested with your footage yet but you may get a better key if you make a tighter junk matte.
Have a look at Aharon Rabinowitz’s ‘super tight junk matte’ tutorial:
try moving the anchor point of the layer into the middle of the ellipse. To do so, click on the ‘pan behind tool’ (left of the ellipse tool) and drag the anchor point into the middle.