Dropdowns in Ae use index numbers to represent values. For the Text Selector Mode, the first item, Add, is 1. The second item, Subtract, is 2, etc.
Here’s the code snippet you’d want, with comments on two lines explaining the changes:
var Animator1 = layer.Text.Animators.addProperty("ADBE Text Animator");
var A1opacity = Animator1.property("ADBE Text Animator Properties").addProperty("ADBE Text Opacity");
var A1Selector = Animator1.property("ADBE Text Selectors").addProperty("ADBE Text Selector")
var EndA1 = A1Selector.property("ADBE Text Percent Start");
var ModeA1 = A1Selector.property("ADBE Text Range Advanced").property("ADBE Text Selector Mode"); // no "subtract" here, just get the object
ModeA1.setValue(2); // set the Text Selector Mode to the second item in the list, "Subtract"
EndA1.setValue(36);