RAID 5 Advice for both enclosure and HD drives

[David Roth weiss]

I understand your desire to economize James, but I think you’re cutting the wrong corners in your effort as outlined.

RAID 5 via FireWire 800 isn’t a great idea for several reasons. Software RAID 5s are not nearly as reliable as hardware RAID 5s to begin with, and since the entire purpose of a protected RAID configuration is reliability, I think there are far better solutions for you.

A client of mine just bought one of the new Promise Thunderbolt 6-bay enclosures, with 12Tb running in a RAID 5 config. It cost $2K, runs like a top, and is supposedly very reliable. I think that’s the way to go on an iMac now, unless you just want to stick with simple FW800 drives.

I hope this helps…

David Roth Weiss
Director/Editor/Colorist
David Weiss Productions, Inc.
Los Angeles
https://www.drwfilms.com

Don’t miss my new Creative Cow Podcast: Bringing “The Whale” to the Big Screen:
https://library.creativecow.net/weiss_roth_david/Podcast-Series-2-MikeParfitandSuzanneChisholm/1

POST-PRODUCTION WITHOUT THE USUAL INSANITY ™

Creative COW contributing editor and a forum host of the Business & Marketing and Apple Final Cut Pro forums.

[James Bayliss-smith]

Thanks for your advice David. Can you clarify the difference between software and hardware configurations. Obviously hardware is when you have hard drives built into a tower and then set up as a raid array. My set up would be using a special enclosure to connect the drives that, as far as I know, does not need any software to run it just switches on the back. Also my main reason for wanting raid 5 is so that everything is backed up with minimum redundancy. 6gb out of 8gb worth of drives. Also when you talk about reliability are you referring to disk failure. Are you saying that disks are more likely to fail in my proposed set up? I thought the whole benefit of raid 5 (or any raid for that matter) is the ease of replacing a disc should one fail. I have no thunderbolt on my iMac or MacBook pro that I will be editing on. Thanks, j

[David Roth weiss]

James,

Below is a quote from a Cow review I wrote about their HDPro2 RAID. I recommend you read the entire review at the following link, which will give you a pretty good education on the subject of protected RAID configurations. https://library.creativecow.net/articles/weiss_roth_david/caldigit-hdpro2.php

HOW DOES IT WORK?

Inside the HDPro2 is a sophisticated RAID controller with 512MB of RAM cache onboard, upgradeable to 2GB. The controller does all the heavy lifting utilizing a built-in Intel XScale Core Processor to accelerate and optimize RAID performance, processing, power consumption, and data protection.

The HDPro2 also features a direct connection to the PCI Express bus, linking the internal RAID controller directly to the computer’s internal bridge chip and memory system. This direct connection eliminates the major data bottleneck of the earlier interfaces such as fibre channel and SCSI, generating bandwidth that’s nearly five times faster than traditional fibre solutions.

The bottom line is, with older MBPs and older iMacs, you will be very limited in terms of the variety RAID subsystems that can be used. FW800 in a RAID 5 configuration is not very reliable and has traditionally not used very often, but your mileage may vary.

David Roth Weiss
Director/Editor/Colorist
David Weiss Productions, Inc.
Los Angeles
https://www.drwfilms.com

Don’t miss my new Creative Cow Podcast: Bringing “The Whale” to the Big Screen:
https://library.creativecow.net/weiss_roth_david/Podcast-Series-2-MikeParfitandSuzanneChisholm/1

POST-PRODUCTION WITHOUT THE USUAL INSANITY ™

Creative COW contributing editor and a forum host of the Business & Marketing and Apple Final Cut Pro forums.

[James Bayliss-smith]

Hi David thanks for the info. I totally understand that there are better, faster products more reliable products out there. I am really seeking advice on my specific ultra budget solution. If I don’t get a cheap one I don’t get one. Simple as. So I’m wanting to find out if my specific solution will work or not.

You speak about reliability. Do you refer to disk failure or efficiency of user experience. I’ll only be editing DVCPro for the foreseeable future and it is can be used mainly as a back up. I could easily get away with not even using it for editing. I edit short news pieces so I can just edit them right here on my laptop (MacBook Pro 2009) and backup the footage up onto the RAID. I chose RAID 5 because it has the least redundancy so I get more back-up for my buck so to speak.

If using RAD 5 means a few dropped frames here and there I can live with that. But if I fundamentally should not be using a RAID 5 disk array over fire wire then that is a different situation. Is there the same issue with a RAID 1 set up? It would be a pity to loose those two TB but I could live with that.

I’m really keen to do this inexpensively or my work will just tell me to back up re-useable archive to tape then re-capture as I need it. (we shoot on P2) I’m keen to build a digital archive then use FCPX to manage this archive and use it for editing. RAID 5 (in a stand alone unit) seemed the way to go with my limited budget. Perhaps RAID 1?

Any more thoughts much appreciated,

James

[David Roth weiss]

[James Bayliss-Smith] “I am really seeking advice on my specific ultra budget solution. If I don’t get a cheap one I don’t get one.”

I completely understand what you’re hoping for James, but what I’m trying to tell you is that FW800 simply may not give you the safety factor you’re hoping to gain with RAID 5 “protected mode.” If a hard drive fails, a FW800 RAID simply may or may not rebuild itself properly.

Cobbling together a FW800 RAID 5 hard disk subsystem is a bit like buying a discount parachute. Would you jump out of a plane with just one chute if you built it yourself from parts available a local store selling discounted parts meant for dropping equipment from planes, but not people? Honestly, that’s perfectly analogous to what you’re wanting to do… And, if it should fail, you’re going to be completely hosed, and your investment won’t seem like such a bargain.

I know this is not what you want to hear, but it is reality.

David Roth Weiss
Director/Editor/Colorist
David Weiss Productions, Inc.
Los Angeles
https://www.drwfilms.com

Don’t miss my new Creative Cow Podcast: Bringing “The Whale” to the Big Screen:
https://library.creativecow.net/weiss_roth_david/Podcast-Series-2-MikeParfitandSuzanneChisholm/1

POST-PRODUCTION WITHOUT THE USUAL INSANITY ™

Creative COW contributing editor and a forum host of the Business & Marketing and Apple Final Cut Pro forums.

[James Bayliss-smith]

[David Roth Weiss] “If a hard drive fails, a FW800 RAID simply may or may not rebuild itself properly.”

Hi thanks David, that is the sort of info that makes me stand up and listen! I appreciate your advice. Is this the case for RAID 1 over firewire 800. I could live with 4TB of space with the same set up.

If RAID 1 is ok, is it still prudent to buy an extra drive when I buy all the others so that I can replace any potential failed drive with an exact copy or is this not so significant with RAID 1?

Cheers in advance,

James

[James Bayliss-smith]

Hi David (or anyone out there!) I’ve been doing a little more research prompted by your excellent advice and I came to the realization that I have a ExpressCard/34 slot on my MacBook Pro at work. The RAID enclosure I have been looking at has a eSata connection so does that now make my proposed RAID 5 set up viable?

If so I will need to buy a eSata to ExpressCard/34 slot converter. Any recommendations? (I’ll look myself but of the top of your head)

Newbie question: can I daisy chain firewire 800 through eSata connected drives?

All the best

James

[James Bayliss-smith]

So I’ve done some more research and I thought I had decided on this card from Sonnet: TEMPO SATA PRO

https://www.sonnettech.com/product/temposataproexpress34.html

but then I saw this advice from you David,

Get your express card from Firmtek. There are many expres cards on the market, but only a few that work reliably on a Macbook/Macbook Pro. https://firmtek.stores.yahoo.net/seritek2sm2e.html

Is this still valid it was from 3 years ago?

Cheers

[Jeff Meyer]

I think you’re too worried about equipment. If you’re cutting HDV and DV25 a Firewire400 connected solution should be sufficient. You seem attached to the idea of needing RAID also – which may or may not be a good solution.

If the budget is there look at the G-Speed Q products. Another option would be the Drobo products, which can grow with you by simply adding drives. I would like to add a word of caution against using proprietary technologies such as Drobo. If they go out of business tomorrow all the BeyondRAID technology in the world won’t matter if you can’t get tech support when you need it – which brings me to my next point. RAID-5 and RAID-1 are band-aid solutions. They don’t replace a tape on the shelf. They’re still dependent on something spinning thousands of times a minute, with extreme manufacturing tolerances that will fail. The real question is how things will work when a hard drive fails. If you want to cross your fingers (and risk your client’s money) with only RAID-5 feel free.

My advice at this stage is to get a 3TB FW800 drive for your media to live on. Get a second 3TB drive (FW or USB) and manually keep them synced. You’ll know what projects you’ve worked on day-to-day, so it shouldn’t be difficult. Right-click on a Finder window, pick Show View Options and check the “Calculate all sizes” box. This feature and a side-by-side comparison can help you make sure you aren’t missing an entire reel or something crazy like that. Once every three to six months wipe the backup drive and copy/paste everything back over just to be certain everything is current. It sounds like a lot of oversight, but the Calculate all sizes and the fact that media doesn’t change makes it relatively trivial for the peace of mind it offers. When I was in undergrad this very method save my tail after a drive failure.
Keep your projects on your system drive with a Time Machine backup so that it is always up to date – or a Skydrive, Dropbox, Sugarsync style service to keep things current in the cloud. This will be much easier than having to sync the drives daily. Source media (the video clips) don’t change too often, but project files change hourly.
You can sell this solution internally by saying the startup cost is lower, but also inform them that you will need additional funding to purchase another couple of drive in a year or however long you believe filling 2.5TB may take. Leave some extra space on your drives. You’ll eventually use it, and it keeps them performing faster.

The advantage of this method is there are two discreet copies which can (should) be in two different places. With RAID they have to travel together, which makes power surges or natural disasters more of a factor. You said MacBook somewhere in this thread, which generally means drives are moving, which dramatically increases the failure rate – a very good reason to avoid RAID-5.

Now, to get to the last question you asked, eSATA, Firewire800, and USB are all 100% discreet. You can not start at eSATA and jump out to a Firewire or USB drive. You can daisy-chain through Firewire, and you can get hubs for both USB and Firewire, but remember, for a hub you’ll be limited by the throughput of the single connection from the hub to the computer.

I wouldn’t be too worried about specs like areal density. For HDV (ProRes, ProResLT) or DV25 a single drive connected over Firewire will get it done. What you need to do is figure out how to get cutting, and how to get assets and projects backed up.

[David Roth weiss]

[James Bayliss-Smith] “I came to the realization that I have a ExpressCard/34 slot on my MacBook Pro at work. The RAID enclosure I have been looking at has a eSata connection so does that now make my proposed RAID 5 set up viable?”

Yes, but you won’t be able to move it over to to your iMac and you’ll be stuck working on your MBP with your best hard drive setup.

The problem you’ve got James is that there is no RAID 5 solution that will really be ideal for you. Were I in your position, I would go with RAID 0 and simply do manual backups for now. Save your ammo for another day when you can purchase a solution that will do all of the things you require with complete reliability.

RAID 1 is also not a great solution for video editing BTW. It’s really meant to provide redundant backup for documents and still photos, etc., but does not ultimately have the throughput for video editing without dropping frames.

David Roth Weiss
Director/Editor/Colorist
David Weiss Productions, Inc.
Los Angeles
https://www.drwfilms.com

Don’t miss my new Creative Cow Podcast: Bringing “The Whale” to the Big Screen:
https://library.creativecow.net/weiss_roth_david/Podcast-Series-2-MikeParfitandSuzanneChisholm/1

POST-PRODUCTION WITHOUT THE USUAL INSANITY ™

Creative COW contributing editor and a forum host of the Business & Marketing and Apple Final Cut Pro forums.