Yep, you can control it. Just select the text layer and hit “U”. This reveals any keyframes in the layer. Drag the furthest keyframe toward the beginning to quicken the Color Wipe or drag it toward the end to lengthen the color wipe. You can even stop the animation at the end of a phrase if you like, just go to that spot, check the “keys” checkbox, that sets a keyframe there then change the keyframe to a hold keyframe. Frozen. Go down the line check again, to get a newa new keyframe (probably a another hold), change it to Linear (select, Animation>Keyframe Interpolation and you get a box to decide they type of keyframe you want).
Or if you want to animate different phrases at different rates, from the beginning, set keys again at the spots you want rate changes, then just drag the keys around to get the speed of change you want, closer = faster, farther apart = slow, so you could have the first phrase slow (closer frames), second fast (father apart) and thrid slow again, if you want.