I also found that selecting all your keyframes and changing their interpolation to linear got rid of the boomerang. Just select your keyframes, right-click on one, click Keyframe Interpolation, and change the Spatial Interpolation to Linear. This would probably have undesired effects on more complicated animations, but for pulling a camera straight back, it should do fine.