Data per frame is also affected by frame rate 720p24 would have more data per frame than 720p60. Of course the motion vs the frame rate might impact the key too (motion blur for example at lower frame rate).
-3db can yield lower noise than 0db but it does depend on the shoot circumstances.
Since “well lit” isn’t very technical it’s hard to really determine if your key was lit properly. It takes a lot more than throwing lots of light on things to get a good key.
Since you’re not describing what you mean by “noise” I can’t help you there either. If it’s properly lit with 0 or -3 gain you shouldn’t get noise.
One might use color smoothing techniques if the issue revolves around 4:2:0 color space but that’s “noise.” I’ve seen keys pulled from the EX1 4:2:0 that look fine since the files are such high resolution (pixels close together help limit the edge nasties).
One might convert the source to Uncompressed or Apple Pro Res or some 4:2:2 format when doing critical compositing but again your description doesn’t tell me enough to suggest what to do.
If you’re in a “studio” situation one might consider taking the HD-SDI out of the camera and go to uncompressed or Apple Pro Res directly.
You really need to post frame grabs unless you can otherwise describe things in better and more accurate detail.