Just shot an uncompressed 10-bit 422 test video. This time around FCPX does recognize the footage (because it is 10-bit now instead of 8-bit). However, after applying the built-in “keyer” effect, the footage becomes blurred. Placing background image underneath the keyed footage doesn’t cause the previous issues (degradation and color dots). But the footage looks blurry, even after rending the composite out as Prores 422.
Do the same thing in AE (using Keylight), and the result is beautiful, like before. Slightly worse in Premier Pro (using Ultra Key), possibly due to the keyer.
My preliminary conclusion – in terms of chroma keying and compositing, FCPX 10.2.1 probably isn’t the best tool to use.