Dave –
I use Red 4.2 and Vegas 8. It may be considered a pain to you, but if you use the stand-alone application (I’m pretty sure that FX has that) you can key in that and save it uncompressed and bring it in to Vegas.
I’ve found the Boris Red keyer to be the best that I have used (including Adobe Ultra and the Vegas keyer). It is generally a one click key. Another trick though, is to create multiple mattes, starting with a garbage matte and working your way down closer to the keyed subject. You can then use whatever keyer works best for each stage as it gets closer to the final edge.
Don’t give up on Boris yet. I can’t find another plug-in as powerful and stable as Boris has been (and easy to use). After using the program for the last year, I finally bought the Red training CD from BorisFX. Big help. It just pulls a lot of issues together quickly that you can’t always get from the manual or just pushing buttons on the interface.
One last thing about using Boris in Vegas. One limitation (and in the documentation) is that motion effects generally don’t work in Vegas and must be accomplished in the stand-alone application. A keyer would be a motion effect vs a filter that is applied generally. Peculiarities in Vegas only allow Boris to pull a single frame from the program when it opens the plug-in, resulting in motion effects not working properly. One great thing about Boris and Vegas is that you don’t have to render anything to see the effect when you close the Boris window. Keep working at it and I think you’ll see the power in Boris.