Lighting is everything.
That does not look well.
Also, .h264 codecs are a poor choice for keying.
You want 422 color or better.
Advice;
rotoscope. Take it into AE and make a tight mask around him, key frame it for the entire clip. Then key out what little is left. Feather.
Chris