-
A Somewhat Unique Situation
Greetings,
I have a gig editing, encoding, and managing educational content for an online company. In the past, I’ve handled this work by editing and storing the company’s content on an internal RAID 0 that I backed up using firewire drives stored offsite. This is a terrible workflow, and the company has expanded to the point where the amount of footage being produced this year has more than tripled, so my 6 TB internal RAID 0 will no longer hold a year’s worth of footage.
I am estimating that they will produce roughly 15 TB this year, including all associated project files. It’s all standard def, usually delivered tapeless, and the editing is very rudimentary, a few edits per hour of footage. Most of the job is encoding, making DVD’s, uploading, etc. Once more, the footage is only “good” for a year, so long term archiving beyond that is not important.
I’m looking for a better workflow. These files get shipped to me daily in 100-200 gig batches. 95% of the time, I edit, encode, upload .mp4’s to a server, make DVD’s, and I’m done with those particular files forever. Once in a while they might want to do some re-editing a month later, or an encode gets corrupted, and I need to re-output, but these are rare occasions.
At 15 TB, it seems like a high performance RAID solution holding all of those files would be especially expensive and unnecessary. I don’t need constant access to all of these files at once, since 95% will never be revisited after DVD’s have been burned and web files had been uploaded.
Are there lower cost “backup” RAID solutions that offer lower performance without sacrificing security ? Are they cheap enough that I could purchase two, using one as a backup that I move offsite?
Are there other backup solutions that could be paired with some sort of RAID?
Sorry if I’m rambling, but this job is an odd combination for me: extremely modest editing/performance requirements coupled with extremely high storage requirements.
I’d like to spend no more than $5000-$6000, but that may prove unrealistic.
Thanks for your input,
Ed
-
3 Replies
-
Ed,
1. Ramping up your hardware to make double copies of the DVDs might be part of your next solution.
2. Your RAID 0 you are using now already gives you no protection from a big data crash. A bigger RAID 0 would lead to a bigger crash when the time comes. How about planning on a relatively small RAID 1 or 5 you can assemble yourself (buy the insides but use a dead computer`s case for the box).
3. This topic hosted by Bob Zelin`s name, and a lot of exchanges with John Heagy`s name, will net you plenty of answers awaiting pencil and paper. The exchanges though, are spread over almost a year`s time.
4. I don`t know how fast this backup works, but you might plan for and ask -here- if buying a few DAT24 scsi backup tape outfits, then buy “DAT24” (60 minutes = 8 GB, or 90 minutes = 12 GB) tapes from http://www.tapeonline.com, seen right above where you are reading or typing right now, is the backup portion`s answer. I haven`t seen any previous discussions of DAT24. -
Ed, the concept you’re describing is called tiered storage: online, nearline, and offline.
Online material lives on your high-performance RAID. It’s immediately accessible. Nearline is a compromise between online and offline storage; it’s a bit slower and not suitable for live production work, but still live. This might be a Drobo or an automated tape library. Offline storage is not immediately accessible. These could be manual tape libraries, DVDs or Blu-rays, or stacks of SATA drives sitting on a shelf.
[Ed McCarthy] “Are there lower cost “backup” RAID solutions that offer lower performance without sacrificing security ? Are they cheap enough that I could purchase two, using one as a backup that I move offsite?”
This might be Drobo. It’s cheap, and it’s way too slow for production use, but it’s not bad for big, low-cost nearline storage.
Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog – What I’m thinking when my workstation’s thinking
Creative Cow Forum Host: Live & Stage Events -
Hey Ed, I’d agree with Walter — we use the DroboPro for various backups. Not fast enough to edit from, but plenty fast to pull a few files and re-burn a DVD or re-render a master. You can get one loaded with 16TB of cheap drives for $2,000ish (B&H is running a special right now).
I think you’d be smart to build up “tiers” of storage, too. Have a nice RAID-5 drive for your fast editing, and a Drobo or something similar for slow (but safe) backups.
Good luck!
Reply to this Discussion! Login or Sign Up