This is one idea that may or may not work (because I’ve never had this situation before). You can try cutting a matte out of a solid layer that is the same color as the blue screen. The cut would have to be exactly the same size as the shirt area, and it would have to be placed directly underneath the video layer that will have the key applied. This should add the blue back into his shirt when you apply the key effect to the video layer. Then you can motion track the entire sequence using a part of the shirt for the reference. Apply the tracking data to the solid and preview. As the long as you’re tracking for position, rotation, and scale, this could work.
The better idea is to get the guy back for another shoot…