Codec choice for long term SAN storage?

[Bob Zelin]

A shared storage envoirment is not for archiving. Archiving is done with LTO Tape, or with cheap individual SATA drives. People edit from their shared storage enviorment, they do not “archive” on it.

All the modern SAN’s or hi speed NAS’s on the market today can handle any compressed HD resolution. FCP users use everything from PhotoJPEG to ProRes4444, and everything in between – including DVCProHD. AVID users (for systems that work with AVID) use any DNx resolution, up to DNxHD220 (which is equivalent to Apple ProRes422HQ – both 220Mb/sec or about 30MB/sec). They all work.

So, to simplify my reply – Apple users typically use ProRes422 or ProRes422HQ (many use lower resolutions, especially if they are working with MultiClip), and AVID users typically use DNxHD145 or DNxHD220 (and use DNxHD36 for off line work or MultiCam).

There is no standard. Everyone has their own requirements. All the modern SAN systems can do it all (but only the best most expensive ones like Facilis can do uncompressed HD). Newer 10Gig systems from Maxx Digital and Small Tree can do uncompressed HD as well with direct connect 10gig ethernet, but these systems have not gained wide popularity, as most people are more than happy with compressed HD codecs, like ProRes422.

Bob Zelin

[Mark Suszko]

Bob, I really appreciate the time you took for a detailed answer. The theoretical we’re looking at is, what do we do if we go to card-based cameras, the footage must be kept, but the cards must be turned around rapidly for re-use, pretty much every day. The footage needs to be kept forever, it may need to be accessed by three or four editing rooms, on very short notice, hen again, footage might stay there for years until needed again. So: SAN, NAS SOTWAN? (Some Other Thing With a Nickname) I gather you don’t think we should bother converting footage, just store the native files, whatever flavor they are.

[Andrew Richards]

[Mark Suszko] “The footage needs to be kept forever, it may need to be accessed by three or four editing rooms, on very short notice, hen again, footage might stay there for years until needed again. So: SAN, NAS SOTWAN? (Some Other Thing With a Nickname) I gather you don’t think we should bother converting footage, just store the native files, whatever flavor they are.”

[Bob Zelin] “A shared storage envoirment is not for archiving. Archiving is done with LTO Tape, or with cheap individual SATA drives. People edit from their shared storage enviorment, they do not “archive” on it. “

Like Bob said, LTO is the archiving champ. There are good solutions out there from Cache-A, Tolis, StorageDNA, Archiware, and Atempo. SAN and NAS are for what you are working on now, archive products like the ones listed are the best way to store data long term.

Best,
Andy

[Eric Hansen]

hey Mark

it sounds like you’re asking 2 different questions and Bob and Andrew addressed the archive question. i agree with them 100%. you should transfer your camera raw, whatever that is, to something like LTO right away. always save your camera’s raw format, no matter if you transcode for your editing workflow or not.

but to answer your question about codec. you have a few different options. i find it easiest to get everything into a common codec. this is because of FCP7’s need to render anything that doesn’t perfectly match the timeline codec. to keep my editors working and not rendering, i transcode everything to either ProResHQ or ProRes Proxy, all 1920×1080 23.98fps.

we use PR Proxy as an offline codec for very large projects. we just edited a movie using Proxy because it would have required about 200TB of SAN space to be edited at PR HQ, and we only have 56TB. then we swapped to the PR HQ files during the online.

your other option is to deal with the rendering from different codecs, or use plug-ins like those from Calibrated Q.

the ideal is getting to the point where the edit system is codec-agnostic and will do whatever hardware rendering is required on-the-fly. Adobe is close with their Mercury Engine.

i understand there are cases where your footage will live on your SAN forever. for things like stock footage or projects that you constantly revisit. i typically keep the final exports of every project on our SAN, so they can be watched quickly when new clients come through. but SANs are not designed for archive and it’s very expensive for that use. use LTO tapes or something similar for your archive. make sure everything that’s on your SAN, is also backed up somewhere else.

e

Eric Hansen – https://www.erichansen.tv