-
Is the a more performant way to find the biggest value in a 2D-array
Hello,
i am working on a graph that is generated of a .CSV-File with about 16 columns and a variable count of rows. To generate the graph within a diagramm i need the max value between some defined rows.
While the script i wrote is working and returns the maximum value in between the rows, it will only run smoothly as long as the row-count stays a small number. When i use the maximum number of rows (about 150) the script gets really slow which brings up the render time.
Is there a way to get the maximum value with less performance requirement than my two used “for”-loops? Here is my code:
maxVal = []; //empty array
endRow = Math.round(effect("END-Zeile_verstellbar")("Schieberegler")); //Last row in array to compare
startRow = Math.round(effect("START-Zeile_verstellbar")("Schieberegler")); // Start Row in array to compare
for (j=1;j<=16;j++){ //counts through columns
for(i=startRow;i<=endRow;i++){ //counts through rows
maxVal.push(footage("dw_allebl_7tagesinzidenz_zeitreihe_wide.csv").dataValue([j,i]));; //pushes the values in an array
}
}
Math.max.apply(null, maxVal) //returns the max. value -
10 Replies
-
Hi Jonas,
it’s the dataValue() part that’s very slow.
If the CSV doesn’t have any gotchas like escaped commas in text-columns then a naive split into linebreaks and comma’s will give a perfect array.
Manipulating that is way faster in Javascript then waiting for After effects to read a DataValue() and you can use all the modern javascript array methods like reduce, map, foreach, flat,…for example find the max of all columns:
data = footage("MOCK_DATA (1).csv").sourceText.split(/\n/).map( (ln) => ln.split(",") );
maxValue = Math.max.apply(-Number.MAX_VALUE,data.slice(start_row,end_row+1).flat());
-
thx for the help, but i ran into some issues . Since i am really new with scripting, i get the error message “< does not have a value” and i am not sure what causes this.
I implemented your code the way you can see in code below. Do i miss something (maybe a variable that need to be declared)? Or maybe it is something in my CSV-File that is not correctly formated.
i added a screensshot of my example CSV-File. It has 18 columns and a variable number of rows, but i just need to check the columns 2-17 (or when started from 0, the columns 1-16), the both date columns can be ignored.Here is the screenshot of the CSV.
end_row = Math.round(effect("END-Zeile_verstellbar")("Schieberegler")); //Last row in array to compare
start_row = Math.round(effect("START-Zeile_verstellbar")("Schieberegler")); // Start Row in array to compare
data = footage("dw_allebl_7tagesinzidenz_zeitreihe_wide.csv").sourceText.split(/\n/).map( (ln) => ln.split(",") );
maxValue = Math.max.apply(-Number.MAX_VALUE,data.slice(start_row,end_row+1).flat());
-
Hi Jonas,
You would need two slices: one for the rows, one for the columns so the dates are skipped.
You can do that with filter() or map().
so we slice the array to get just the rows we need.
Then we remap (replace) every row to a slice of the row with just the columns we need.
The end gets flattened into a single list and fed into max.Something like this:
start_row = 1;
end_row = 10;
start_column = 1;
end_column = 16data=footage("germandata.csv").sourceText.split(/\r\n?|\n/).map( (ln) => ln.split(",") );
Math.max.apply(Number.NEGATIVE_INFINITY,
data
.slice(start_row, end_row+1)
.map( row => row.slice( start_column , end_column+1) )
.flat()
);
I also changed the line-split so it would handle different file formats / linebreaks better.
-
Appreciate your help, but tried to get your code running… But still AfterEffects isn´t working with it.
Here is a screenshot:
I really new to scripting and just got most things to work, but your code is just hard to understand for me. Sorry to bother you, but would be happy to get this solved.
Just one thought, how i maybe could get the project to be more performant with my old code – is there a way to stop the both “for-“loops after the last value is found/ or after a specific time frame. I tried to put the for-Loops into an if(time<1) …. “do maxValue calculation” else “displaymaxValue”;
The start worked out but as soon as it finished the first secound, it set the maxValue back to 0. So i thought the last value of the for-loops would have overwritten the start value, but it seems it isn´t existing outside the for-loops – if i understand this correctly?
Maybe i need to read more about how the variables are working.
Edit: Maybe asked more shortly and simply: How do i get the maxValue i calculated via “for”-loops in the first second to exist, when i exit/stop my for-loops? -
I think there’s a problem with the csv file.
Maybe one of the values is a string like “< 5”. Although then the error should be a few lines lower.Try adding this after flat():
.flat().map( n => parseFloat(n))
This forces all the values in the array to become a number. If that doesn’t work, can you share the csv?
About your second question: expressions either calculate or they don’t. Expressions do not remember what was calculated in a previous frame.
So you can’t do a complex calculation once, and then store the result.Scripts can do that, run once and store the result.
-
Still doesn´t seem to work, i think it has problem with the CSV for sure. The first and the last columns have dates and times – but both can be ignored (they are not needed and will change to a german date-from in future – if need to know this! So dates like 2020-07-22 will in future be 22.07.2020).
But for my calculations if the max. value i only need the numbers in between.Here is the link:
14245_dwallebl7tagesinzidenzzeitreihewide.csv.zip
Really big THANKS for your great help and time spent! -
Ah, should have mentioned this earlier:
Go to Project Settings -> Expressions -> Expressions Engine.
And make sure that is set to “Javascript”, not “Legacy Extendscript” -
ok, thanks, how could i miss this. I thought i had changed this before, but maybe because i took an old project, it changed back to the legacy engine. This now means a lot of more work needs to be done, since this makes a lot of scripts not functioning anymore. First have to try to get it back working, will report back later if everything worked out.
-
Here’s a way that’s compatible with legacy extendScript:
start_row = Math.round(effect("START-Zeile_verstellbar")("Slider")); // Start Row in array to compare
end_row = Math.round(effect("END-Zeile_verstellbar")("Slider")); //Last row in array to comparedata = footage("14245_dwallebl7tagesinzidenzzeitreihewide.csv").sourceText.split(/\n/);
samples=[];
for (i=start_row; i < end_row; i++) {
data[i] = data[i].split(",");
for (j=1; j < 17; j++) {
samples.push(parseFloat(data[i][j]));
}
}
Math.max.apply(Number.NEGATIVE_INFINITY, samples);
-
Really big thanks to you Filip…
After changing the project to the java-engine, I messed a bit around with the code you send me and found the error. Forget to add +1 to the row counter! Now it works much faster. REALY big thanks for your great help 😉
Reply to this Discussion! Login or Sign Up