-
Extendscript: Image sequence array concatenation – need optimization advice!
I have a script I’ve been using/developing for some time now which greatly simplifies importing multiple image sequences (mainly for CG renders with render passes). Everything is working great but there’s one specific area that runs really slowly when you start getting into thousands of images and I’m trying to figure out how to speed it up.
What I am trying to optimize is going through an array of potentially thousands of image files, and concatenating it down to just the first image in the sequence. For example my array might be [img.001.png, img.002.png, img.003.png, img.004.png, img.005.png] and what I want to do is remove every instance of “img.###.png” except for the first one. So after my function I would have just: [img.001.png].
Here’s how I do it right now:
1) I take array item A, regex out the number sequence
2) then array item B, regex out the number sequence
3) and compare the two to see if they are identical. If they are I splice out item BThis works 100% of the time which is awesome, but is quite slow when you get above a thousand or two images… What I’m trying to do is find a way to do this “pruning” step more quickly. A friend of mine told me regex is pretty slow so maybe I need to do it without regex? I also found several native JavaScript array methods that might help such as forEach() every() and filter() but I don’t see how these are going to be faster because I still need to do 2 regex evaluations, and I still need to compare the items to each other.
Any help would be immensely appreciated!
– Spencer
Here’s a snippet of my code:
currentFolder = new Folder ("//12.34.5.67/my folder on the network/")
var folderChildren = currentFolder.getFiles().sort();var searcher = new RegExp("\\d{3,5}[.]");
for (var i = 0; i < folderChildren.length; i++) {
// Go through the array and strip out all elements that are the same once their numbers have been removed with a regex
if (i > 0) {
currentResult = searcher.exec(folderChildren[i].name); //check if a sequence
if (currentResult) { // it is a sequence
// first parse out the comparison strings - current item and item before
var testNameBefore = folderChildren[i-1].name;//if we have a sequence before our current item, we need to delete the numbers.
var beforeNum = searcher.exec(testNameBefore);
if (beforeNum) {
testNameBefore = testNameBefore.substring(0, testNameBefore.length-8);
}var testNameCurrent = folderChildren[i].name;
testNameCurrent = folderChildren[i].name.substring(0, testNameCurrent.length-8);//compare to the element before it and delete if the same!!
if (testNameBefore == testNameCurrent) {
folderChildren.splice(i, 1);
i--;
}
}
}
}
-
4 Replies
-
I haven’t profiled it, but I’d guess that .splice() might be slow, too, especially with large datasets. What about creating a new array and pushing the select values you need into it?
Walter Soyka
Designer & Mad Scientist at Keen Live [link]
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
@keenlive | RenderBreak [blog] | Profile [LinkedIn] -
Thanks for the tip, there are definitely other ways of doing that step so I’ll check it out.
On the topic of profiling, what do you use? I know the extendscript toolkit has a built in profiler but I found it quite confusing, and I use sublime (can’t stand the toolkit). It probably doesn’t help that this is all wrapped within a recursive function… I tried out $.hirestimer per the manual but I found some funky stuff – for example if you run the same script back-to-back you will get a much quicker timestamp on the second iteration, which I assume is due to the javascript buffer but $.gc didn’t help.
– Spencer
-
[Spencer Tweed] “On the topic of profiling, what do you use? I know the extendscript toolkit has a built in profiler but I found it quite confusing, and I use sublime (can’t stand the toolkit).”
I honestly haven’t done a lot of profiling, but I have used the ExtendScript Toolkit profiler a few times out of curiosity. Is there something in particular you’re hung up on?
Walter Soyka
Designer & Mad Scientist at Keen Live [link]
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
@keenlive | RenderBreak [blog] | Profile [LinkedIn] -
I don’t remember exactly what I ran into, but I got confused because of my recursive loops and whatnot. I’m sure if I took the time I could figure out the profiler, which I’ll do when I have a moment.
I got an interesting reply on Stackoverflow which doesn’t use splice: Link
Only problem is I tried running it in AE and I got an error on the array.replace() method – and in doing some research looks like Extendscript doesn’t support higher-level array methods… Reference. However per that post you can add them with Polyfill, but I haven’t had a chance to try it out yet.
Anyway, basically just some more testing to do and no time to do it!
I’ve uploaded my script if you want to have a go, or just use it for free. I use it non-stop throughout the day, I don’t even import files the old way anymore unless they’re single images or video clips (hence why I’m trying to optimize). I continue to develop it every week or two, but in it’s current stage it is totally stable so far as I know. That said i’m still on CS6 for the majority of my professional work, so you might run into a thing or two in CC…
10494_stimportpanelv1.2.jsx.zip
– Spencer
Reply to this Discussion! Login or Sign Up