-
Consolidating ArriRAW files before conform/grading
There were these threads some months ago on consolidating ArriRAW files so one need not have to connect multiple source file drives/RAIDs to Resolve to do a conform or grade…
https://forums.creativecow.net/thread/277/21620
https://forums.creativecow.net/thread/277/21497I’m revisiting the idea as I had a project recently that I was consulting on, for exactly the same task.
How does one copy out only used ArriRAW media, referenced in 8 reels for a feature of about 2 hours. Media spread over hundreds of folders residing in 5 RAIDs and 4 drives totaling 40 TB of ArriRAW? The RAIDs were all Mac HFS and grading was to happen on a system that couldn’t connect all the drives simultaneously. So copying only referenced selects to another drive was the only option.
I came across a utility called ‘EDL Spy’ and after trying it out, bought it. It’s here…
https://edlspy.felixhuesken.deThis is a JScript, so it will run on any Mac or PC too. It basically reads an EDL or XML and extracts filenames (or folder names in case of ArriRAW). It then lets you remove duplicates, assign a target folder, and writes a batch script that you can run and the copy takes place.
If your data is spread over many drives and is large, then it can take time. I’ve had it take up to two hours. There’s no status display so you aren’t sure if its working or given up. So I keep the Activity Manager open and check CPU usage for EDL Spy that tells me its still going. But after a while it does come up with the script. And the script runs in Terminal serially so you can continue doing stuff on your system.
I had earlier developed the same logic using Numbers. My method consisted on taking a snapshot of all drives that contained ArriRAW files, then extracting clip names from EDLs, and making Excel or Numbers do a lookup to find the exact path. Then make a set of commands to do the actual copy.
But I found EDL Spy easier for someone who couldn’t write formulae in Excel or Numbers.
And Felix is very knowledgeable. You can write to him and he can give you a fix in case you’re stuck.
———————————–
Neil Sadwelkar
neilsadwelkar.blogspot.com
twitter: fcpguru
FCP Editor, Edit systems consultant
Mumbai India -
5 Replies
-
Seems really great tool!
—
Margus
https://iconstudios.eu
https://vimeo.com/iconstudioseu/videosDaVinci 9, OSX 10.7.4
MacPro 5.1 2×2,93 24GB
GTX 470 / Quadro 4000
Multibridge 2 Pro -
Does this actually extract just the portions of each clip that are in use (optionally with handles), or does it always copy the whole clip?
—
Joakim Ziegler – Postproduction Supervisor -
It copies the entire folder containing the referenced ArriRAW files. Meaning, the full clip.
To be able to copy only the referenced ‘in’ to ‘out’ point duration, would need parsing the ‘source in’ and ‘source out’ timecodes and reading filenames off it.That’s not very tough, but with motion effects, the ‘source out’ needs to be calculated from the EDL, which, I believe is not very easy.
But, given a directory listing of sorted full takes, and an XML of the sequence referencing them, maybe I could write a simple script to move the unreferenced files to a safety folder so you can check the conform to ensure some wrong files didn’t get lost before deleting unnecessary files en masse.
———————————–
Neil Sadwelkar
neilsadwelkar.blogspot.com
twitter: fcpguru
FCP Editor, Edit systems consultant
Mumbai India -
Hi Neil
Thanks for working this out and sharing this. The script is a great idea. I’d be interested in it!
David Davidson
Managing Director
solventdreams, llc
http://www.solventdreams.com
4222 santa monica blvd
los angeles, ca 90028
323 906 9700
info@solventdreams.com -
I did this using my own formulae, but then gave up further development. For two reasons. There’s now an App called ‘Resolve Collect’ which does exactly this, and I bought and used it for an entire feature. And two, Resolve 10 or maybe 11 can do the same thing too,
———————————–
Neil Sadwelkar
neilsadwelkar.blogspot.com
twitter: fcpguru
FCP Editor, Edit systems consultant
Mumbai India
Reply to this Discussion! Login or Sign Up