As far as I can tell, the 3 codecs that retain alpha in a quicktime are animation, PNG, and prores 4444. But they all do a terrible job of compressing. You lose the ability to clamp bit rate, and only animation gives you a quality slider that doesn’t do a very good of compression (even with it cranked down to zero) while looking terrible.
It sounds like it’s just impossible to get great compression with the an alpha channel in a quicktime.