From what i can see from your screenshot, the issue is with how you set up your secondary windows. Right now you have inverted them. To subtract them, you actually want to select the mask icon, next to the inversion one (see the attached screenshot). Once you do that, the result of the masking should be visible as you connect the alpha of the node to the alpha output of the node tree.
As for the OFX, that depends on which one you are trying to use. Some already have a built-in checkbox that lets you decide if the mask should affect them or not (usually at the top in the inspector. Others might not have that and will have to be delt with in a different way. An example would be to add a layer node and have the mask inverted affecting that layer so that the OFX only shows where the mask is. If you could specify which OFX you are trying to use, I could be more specific.
Hope this helps.