-
Standard Archiving Practices
I recently became the digital post-production coordinator at a new media company who is churning out quite a bit of content each week.
I’ve been trying to wrap my head around the best workflow, and I’m pretty happy with where we’ve settled.
My only real question depends on archiving. Right now, we burn the finished render to DVD and shelve it. But I’d like to keep all the source footage, project files, etc on a hard drive somewhere in case we need them in the future.
Right now, we’re running multiple external hard drives that the previous coordinator would just slap a project onto without labels or a database and hope for the best, they have around 25 750 GB Externals.
My first question would be, do we need to keep using those externals or is there a better way?
We’ve toyed with the idea of setting up a 2 TB RAID 1 (Set to be 1 TB) setup that would host the various projects.. but I feel those would fill up fast (each project is anywhere from 25 GB to 50 GB, so we’d have 20-40 projects per 1 TB, and we produce roughly 15 per month. So we’d have to constantly be buying new drives, and in the case of business.. putting in a new purchase order each month could get annoying.
But what do we do when that drive is filled? Put it in an anti-static bag, shelve it and then go back to it if need be? But that could be a pain if I needed to find something on that drive, I’d have to unplug the current RAID setup, reload that setup, find the file, unhook it, resetup the current system.. all the while no editor could grab anything from the current drive because it would be offline.. so there has to be a better way.
So, my second question is, what is the best way to archive finished projects?
Lastly, folder structure.. we’ve settled on something like this but any suggestions or comments are welcome. Every project gets a folder that looks like this, so every project is uniformed.
PROJECT
> ASSETS
>> GRAPHICS
>> VIDEO
>> MUSIC
>> PHOTOS
> REFERENCE
> SOURCE
>> CAM A
>> CAM B
>> SCRATCH
> PROJECTS
> SFX
> EXPORTS
> ENCODES -
5 Replies
-
how long would this sit before you went back to it? If you let a hard drive sit the head settles and can stick to the disk, when you go to fire up the drive the head breaks and your data is toast. I suppose longer storage options might include LTO4. I’ll let others chime in on this as I’m not the best archivist person, I just know that a bunch of hard drives is not the way to go for long term data storage, if powered off. (long term = 6months or more on a shelf powered off without touching it)
-jw
-
hey tyler
for my clients, i suggest one of 2 systems. either sets of bare hard drives that are rotated addressing Jordan’s concern about them sitting around, or LTO4.
if you’re putting a lot to archive, say 20TB+ a year, then i would suggest a tape-based system such as LTO4. tapes can be had for $40/800GB tape. the expense is the LTO deck. i recommend having 2 because if one deck goes down, you lose access to all of your archive. this is one reason i like archiving to multiple sets of hard drives because if one hard drive becomes inaccessible, you don’t lose access to all of your archive. i used to have a SDLT deck and it always went down at the worst times.
for hard drives, i suggest you have 3 complete sets that are rotated and checked for integrity every 6 months. keep 2 sets in house and one set in controlled storage.
i can give a much more complete answer, but i’ve answered this question a few times on the Cow in a few different forums. search my posts to get more info.
e
Eric Hansen – The Audio Visual Plumber – http://www.avplumber.com
-
Hi Tyler,
At our facility we do backups like this:
Files are stored online on a raid5 system until the project is done.
We copy the files to a mirrored set (caldigit S2VR Duo) with a program that does checksum (ex. copy cloner)
We take one disk home and leave the other at office. We do this to comply with insurance requirements.
To keep a record of all offline data we use DiskCatalogMaker (comes bundled with Toast).
You can also export a pdf. of the info as documentation for production staff.
We rotate our disks every 12 months, as we see this to be sufficient. We have not yet had problems. But as a note, we use enterprise disks.
It works and is inexpensive.
Good luck!Marc Bostrom
-| just another PRO FCP user |- -
i use AutoCat for cataloging the archives. it makes Finder aliases, so you can search in Finder/Spotlight and not have to use a separate cataloging program to read the database.
e
Eric Hansen – The Audio Visual Plumber – http://www.avplumber.com
-
We’re using a Sony Petasite (tape), with CatDV to create offline previews of stored media. We’re hoping to soon add a NAS server and ditch those filthy external drives. Not practical for anything but a transitionary solution. It’s not a matter of IF they’ll fail, only WHEN.
At this point, your best bet is to, quickly as possible, establish best practices for file names and folder structures. If you haven’t done so already, start now, today, don’t wait. Delay only creates more work.
A good database program is a must as well. While you might be able to cobble together some kind of XL based way of doing it, you’re better off getting something designed for managing this type of data. Can’t say enough good things about CatDV. Works and plays well with Mac’s, PC’s, FCP, Premiere and many more. Can handle just about any media you throw at it, though it requires 3rd party help for formats like XDCAM, it is still the most bang for your buck out there.
Welcome to file based media. It’s a blessing, and a curse.
Marvin Holdman
Production Manager
Tourist Network
8317 Front Beach Rd, Suite 23
Panama City Beach, Fl
phone 850-234-2773 ext. 128
cell 850-585-9667
skype username – vidmarv
Reply to this Discussion! Login or Sign Up