I’ve found the solution!
Using the setValuesAtTimes method is much much quicker. Now the script loops through and creates two arrays. One array of composition times and one array of values. This method applies all the keyframes with one method call to after effects.
The 40 minutes to run the script has now been cut down to under 1 minute!
I’ve now used this approach on all my other scripts with large amounts of keyframes and the results are the same.
Lesson learned. If you have to set many keyframes(Tens of thousands) us the setValuesAtTimes method.
P.S. Turning on undo makes things worse. After Effects dies trying to keep undo history for every keyframe. Perhaps it will be different with this new approach though. I haven’t tried that.
Thanks for all the suggestions!