Raid 10

[Eric Newbauer]

I can’t wait to see what’s in the can of worms you just opened! 😉

If you look at it from just an academic perspective it’s hard to argue against RAID 10 being the optimal way to go for video editing… if your most important two factors are reliability and speed. (Similarly, if your most important factor is only speed, then the underappreciated RAID 0 is the optimal way to go.) Unfortunately, most companies don’t have the luxury of making a buying decision solely on just those two factors, and of course, the balanced solution of cost, speed, and reliability (in that order, until things go wrong) wins out, which is why RAID 5 is still the dominant level.

There’s one statement in that article I believe *is* inarguable: “RAID is not backup.”

Eric Newbauer
Studio Network Solutions (SNS)
https://www.studionetworksolutions.com

[Herb Sevush]

[Eric Newbauer] ” the balanced solution of cost, speed, and reliability (in that order, until things go wrong) wins out, which is why RAID 5 is still the dominant level.”

The author seemed to imply that Raid 10 didn’t need a controller card, because it was much simpler to implement. If that’s true, wouldn’t that reduced cost pretty much offset the price of the additional drives?

Herb Sevush
Zebra Productions
—————————
nothin’ attached to nothin’
“Deciding the spine is the process of editing” F. Bieberkopf

[Eric Newbauer]

Hmmm… I think you might be referring to this statement FTA:

“In practice, RAID-Z exists only as a piece of software, not as a plug-in card, and it may outperform the sort of cheap RAID-5 that’s often advertised as a feature of desktop computer motherboards simply through better software engineering.”

In fact, it is possible and common to setup systems where the RAID creation/processing of any given level – RAID 0, 5, 6, 10, whatever – is done entirely in software. If the implication is to use RAID 10 w/o a controller and leverage that cost savings to compensate with more drives, then that math would only make sense up to a very small number of drives.

Eric Newbauer
Studio Network Solutions (SNS)
https://www.studionetworksolutions.com

[Bob Zelin]

most of us want support from hardware manufacturers, so we can “point the finger” and get assistance when things go wrong. In the Mac world, we are at the mercy of ATTO, Areca and Highpoint. Add the PC and Linux list, and we add LSI Logic (and ATTO, Areca and Highpoint all support PC’s and Linux too).

In these companies, we trust. And when “they” say to go RAID5, or RAID6, that’s what we do. (I use RAID 6 almost exclusively these days, because of the crap quality of drives currently being made by WD and their associated companies).

If SSD’s didn’t cost so much, we would be using them. But they are cost ineffective for the Terabytes required by most customers these days.

As you know, things just don’t work. And just like you need support, WE need support. And we all know very well, that even the “core manufacturers” that make these components make mistakes, and have constant driver and firmware updates.

This ain’t magic. S#$% happens, and when it does, you need to be able to refer to a person to complain to. In case I was not clear in my post, there is not one person on any of these threads, nor is there one single company that advertises on Creative Cow, that is a core manufacturer.

Bob Zelin

Bob Zelin
Rescue 1, Inc.
maxavid@cfl.rr.com

[Herb Sevush]

[Bob Zelin] “In these companies, we trust. And when “they” say to go RAID5, or RAID6, that’s what we do. (I use RAID 6 almost exclusively these days, because of the crap quality of drives currently being made by WD and their associated companies). “

And when you say use Raid 5/6 that’s what I do, which is why the article I pointed to surprised me because it made it seem like Raid 10 was a no brainer, and yet I didn’t know of anyone using a Raid 10 system. Just curious is all.

Herb Sevush
Zebra Productions
—————————
nothin’ attached to nothin’
“Deciding the spine is the process of editing” F. Bieberkopf

[Alex Gerulaitis]

[Herb Sevush] “And when you say use Raid 5/6 that’s what I do, which is why the article I pointed to surprised me because it made it seem like Raid 10 was a no brainer, and yet I didn’t know of anyone using a Raid 10 system. Just curious is all.”

There’s a number of people in the IT community (spiceworks.com specifically) that believe, with near-religious zealotry, that RAID10 is the best RAID level on Earth. I think this is why you see that view propagated in articles around the web.

It’s certainly not a no-brainer, as it has significant drawbacks in efficiency and resiliency. 10 is best used in places that need a lot of small random I/Os, whereas most desktop media arrays need sequentials I/Os where RAID5 and 6 are more efficient.

I am with Eric and Bob: in media (and many other) arrays, RAID5 and 6 are dominant, for a good reason. RAID10 is a good way to address RAID5 performance and resiliency problems – at the cost of being less efficient. It’s far from a clear winner when compared to a well-implemented RAID6.

Alex Gerulaitis
Systems Engineer
DV411 – Los Angeles, CA

[Chris Gordon]

“Which RAID is better” needs to be bounded by two factors. First, what is your work load. Second, what is your budget.

The typical work load in the video world is dominated by very large sequential reads and writes. These IOs are going to quickly blow through any cache and limit you to what the disks can do. To make things faster, you throw more spindles at the problem.

Budget means you have a set number of disks for whatever RAID configuration you choose. In RAID10, you get essentially half the disks contributing to your IO. In RAID5 you get N-1 disks (RAID6 is N-2) to handle the IO.

In our case (video editing) you’re limited by the throughput of your disks. The more disks, the more throughput. RAID5/6 win in this case.

So what are the “bad” things with RAID5/6 (parity RAID)?
– Parity Calculations. You have to calculate parity for any write. This is work done by some CPU (on a controller card, on some other CPU in the system, etc). The penalty in performing this calculation is negated by the additional disks to handle IO. For large sequential IOs and any half decent hardware/software setup, the additional spindles in a parity RAID more than make up for the parity calculations.
– Updates Require More IO. When you update a file in a parity RAID, you have to read all of the file and recalculate parity in order to make the update even when the update is only a small portion of the file. This just doesn’t appear in the typical work load in a significant way, so it’s not really a problem.
– Recovery from failed disks. Parity RAID is going to have a much more significant impact on performance when recovering from a failed disk and that hit is often going to be much longer than in RAID10. That’s the price you pay. If that hit isn’t acceptable, then you need more money.

[Alex Gerulaitis]

Cris – great post, much better written than mine.

[Chris Gordon] “When you update a file in a parity RAID, you have to read all of the file and recalculate parity in order to make the update even when the update is only a small portion of the file.”

Did you mean “the stripe” on which the updated bits are, rather than “the whole file”? The OS (unless it’s OS-based RAID) doesn’t know anything about the RAID, and the RAID doesn’t know anything about files, it only deals with sectors, blocks and stripes, AFAIK. The RAID gets a write request and updates the stripe where that write request is sent, knowing nothing about the file.

[Chris Gordon] “Recovery from failed disks. Parity RAID is going to have a much more significant impact on performance when recovering from a failed disk and that hit is often going to be much longer than in RAID10.”

True but perhaps there is a cure (in the future): RAID write bitmap. It’s awesome for those “non-failure” events that are quite common where a healthy disk is offlined or popped out temporarily. With complete failures, I can also see a local spare being pre-configured for that duty on schedule where rebuilding a RAID can take minutes rather than days or hours. RAID bitmap is not available on common desktop RAID controllers from LSI, ATTO, Areca (AFAIK) but hopefully will eventually make it there.

[Chris Gordon]

[Alex Gerulaitis] “Cris – great post, much better written than mine.”

Thank you.

[Alex Gerulaitis] “Did you mean “the stripe” on which the updated bits are, rather than “the whole file”? “

Yes. I was just trying to keep the discussion simple and high level. There are probably a couple of other places where I took similar liberties.

[Herb Sevush]

Alex, thanks for taking the time to explain things, I can now quietly go back to work on my Raid 5s without thinking I’m sitting on a time bomb of my own devising. There’s nothing like reading just a little bit of information to scare yourself silly.

Herb Sevush
Zebra Productions
—————————
nothin’ attached to nothin’
“Deciding the spine is the process of editing” F. Bieberkopf