Whats the deal with RAID?

[Darby Edelen]

[matt cue] “He says that a striping RAID puts strain on the processor and will slow it down. Even more than a mirror RAID does. This might be true but is it a significant amount on a decent quad core, 4GB RAM, 32Bit XP machine?”

I’ll weigh in on this part of the discussion.

In both RAID 0 and RAID 1 you are writing data to two separate physical disks. In RAID 0 you are writing alternating blocks of data to the disks which should speed up access to the logical volume. In RAID 1 you are writing the same blocks of data to both disks (hence ‘mirrored’).

I would be surprised if RAID 0 puts any more strain on a processor than RAID 1, furthermore the ‘strain’ on a processor that any software RAID causes should be very minimal. Accessing data on your non-striped RAID is much more of a bottleneck than could possibly be caused by a decent striped software RAID.

The only thing I would say you really need to consider when choosing between RAID 0 and RAID 1 is whether you need the speed of 0 or the redundancy of 1.

Darby Edelen

[Dave Johnson]

[add an LTO tape drive … spend six grand on this … doesn’t solve any back up solutions for anyone else]

And, I’ll weigh in on this part.

Different editors obviously have different subjective preferences for storage and backups, but most editors I know would advise you to step away from the tape drive solution. My experience has been that they’re a hassle to work with and I can’t think of a reason to go that route nowadays since hard drives are so cheap. I have several good friends who are Video Engineers so this is no disrespect to Engineers, but I think it’s important to note that an Engineer’s perspective and priorities are very different. Their job is to set up equipment that they are familiar with and can, therefore, support and keep working reliably … that is an important and understandable priority, but it doesn’t always equate to the best workflow and functionality for the people who actually have to use the equipment. Video Engineers are not always computer technicians too so they may not think of or, in some cases, shy away from entirely computer-based solutions.

You didn’t mention the amount of storage you have/need so it’s hard to be precise, but you did mention that you have multiple workstations. For about the same money you’d spend on a tape backup system and constantly buying tapes (and the mandatory headache medicine), you could set up a good external RAID for shared storage and/or backup. You could also buy fast external TB drives for backup purposes … Maxx Entertainment Digital (which has its own forum here on the COW now) sells triple interface (FW400, FW800 & e-SATA) TB drives pretty cheaply and its a great company to deal with.

Perhaps the best way to make my point is to describe our setup …
With the help of Bob Zelin who is a master at this subject and can be found here on the COW, we recently replaced a dead (and severely overpriced) Apple X-Serve RAID with a Dulce RAID as our primary storage. As our storage needs grow and/or for backups, we can add a second RAID (third, fourth, etc.) by using a PCI-e port expander. For budget purposes right now, we use internal drives striped to a RAID and a set of the triple-interface Maxx drives I mentioned, along with Tri-Backup software, for a second and third level of backups and, if need be, storage overflow. The end result is:
[1] all the storage I could possibly need and expandability
[2] the data protection the primary RAID alone provides
[3] a second level of data protection via the internal RAID’s dual-purpose or interchangeable-purposes of backup and/or storage overflow
[4] a third level of data protection via the external triple-interface drives’ dual-purpose or interchangeable-purposes of backup and/or storage overflow
[5] the mobility of the external triple-interface Maxx drives
[6] the users don’t have to do a thing to accomplish any of it … no keeping track of or loading tapes, etc.

Zero worries about data loss or lack of adequate storage … what a great way for an editor to focus on the craft instead of the technical mumbo jumbo! I hope my two cents is helpful.

[Matt Waryga]

Thanks for your response,
I thought I was correct in thinking that a RAID 0 striping speeds up performance where RAID 1 is for protection, and RAID 5 both. I just needed someone to tell me I wasn’t crazy. Our engineer told me different and given the fact he is an engineer I respected his opinion enough to check it out. After a little research I discovered that a software RAID 0’s strain on the CPU is “negligible”. It only becomes and issue in parity-based RAID’s.

[Matt Waryga]

LOL, you hit the nail on the head. As I had mentioned this guys new here and is very very old school. You’re probably correct in the fact that he offers solutions that he is familiar with and not what is the best solution for the company. Which is why I think he doesn’t want me to have a RAID b/c he is unfamiliar with these. Obviously he is if he thinks a RAID 0 will slow down my performance. What would be the point in spending extra money for more drives to setup a RAID 0 if there was no benefit? It wouldn’t exist if it made it easier to loose data and slowed down your bandwidth.

We have been talking about a RAID storage \ transfer solution and I believe are in the works of getting one. Which is why I suggested buying a 1TB external drive and putting my archive data on there until the RAID storage was implemented. Thus, saving us a lotta money. But now we have this tape drive to live with. And you’re right for the money spent on this it would have been better spent on a viable solution for every workstation not just one. But what can you do except post on forums and bitch.

follow up question:
what do you know about a unity? These are AVID network storage devices for multiple editing suites to access the same archive of video’s correct? Are these a good solution for networking all the graphics workstations and AVID suites or are the AVID specific? My suggestion was not for a unity but a RAID like you previously suggested.

thanks >_<

[Ron Lindeboom]

[matt cue] “I think he doesn’t want me to have a RAID b/c he is unfamiliar with these. Obviously he is if he thinks a RAID 0 will slow down my performance.”

[matt cue] “He says that a striping RAID puts strain on the processor and will slow it down.”

He is completely wrong that RAID 0 will slow down your computer’s performance and that it will put strain on the CPU. The strain is on the BUS not the CPU. The cycles needed to load RAID 0 files into a system are negligible as the whole point of RAID 0 is to turn off all error correction and verification processes and just basically tell the CPU to just “load the files without bothering to look at them please.”

In the old days of striping, you had to know how to turn off these processes with what we called mode switches. By turning off all the mode switches you were telling the CPU that you did not care if there was corruption in the file, just load it. The normal CPU process of verification that the file grabbed and in the process of being loaded was indeed the *exact* file as it was in its original form — verification — was ignored.

That is the essence of striping in RAID 0.

To say that this requires large CPU processes is nuts. We used to do this on 40 Megahertz machines and when we got up to 80 and 100 megahertz we thought we had died and gone to heaven.

It worked even with such insanely limited processors.

With today’s incredibly powerful multi-core systems with Gigahertz to burn, saying that you need to worry about cycles due to RAID 0 makes me wonder if this guy knows anything about this field whatsoever.

I sure as hell wouldn’t work with him.

Best regards,

Ron Lindeboom

Creativity is a type of learning process where the teacher and pupil are located in the same individual.

[Dave Johnson]

I now use FCP for editing and it’s been many years now since I worked on Avid systems so I can’t comment on the Avid Unity.

[guys new here and is very very old school … solutions that he is familiar with and not what is the best for the company]

Just to clarify, I didn’t mean to imply the Engineer offered solutions based solely one his familiarity regardless of the company’s needs … rather, that as staff Engineer, his job is to keep all equipment functional so, for reasons you may be unaware of, he may have determined that tape backup is the better solution for that facility. For example, if the Engineering staff can fully support a tape system, but there are no staff computer techs to fully support multiple shared RAID systems, all things considered, tape may very well be the better solution.

And, my opinion is that a smart video editor considers an Engineer among his favorite people on the planet since no professional video facility can survive without them. I myself am also very, very “old school”, but was also “new school” once so my advice would be to never forget that, without the old school people and practices that built our industry, neither of us would have a job today. Cheers!

[Matt Waryga]

Hey no doubt. I have much respect for the old days. Heck, I learned a lot taking apart and putting back together my 486, and I’m sure there’s people who would say that’s not that old school . . But, I just have a growing suspicion that this guy doesn’t know what he’s talking about for many reasons too long to list. However, I meant no disrespect towards anyone b/c of their age, era, job title, or obsession with DOS. ez pz lmn sqz

[Ron Lindeboom]

[Dave Johnson] “And, my opinion is that a smart video editor considers an Engineer among his favorite people on the planet since no professional video facility can survive without them.”

While this is true, it is only true if the engineer is familiar with video and is someone like Bob Zelin, an engineer with the kind of experience that makes him invaluable to many clients he serves.

But if an engineer came in talking with quotes such as I read in this thread, I wouldn’t hire him for free.

Your mileage may vary,

Ron Lindeboom

[David Johnson]

[matt cue] “I meant no disrespect towards anyone b/c of their age, era, job title, or obsession with DOS.”

Obsession with DOS!? lmao No offense taken and I didn’t mean to sound defensive since I’m not that old after all! lol Just making the point that it’s generally a mistake to assume age and/or years in the business has an inverse relationship with knowledge and/or skill and that, usually, just the opposite is true. Hmmm … that was a pretty confusing way to say that wasn’t it!? lol

[Ron Lindeboom] “But if an engineer came in talking with quotes such as I read in this thread, I wouldn’t hire him for free.”

Point taken. At the same time, I’m in corporate now so there are no staff Engineers, but the last station I worked at had several “old school” Engineers who practically built the place from the ground up, yet I wouldn’t be surprised if they didn’t even know what a RAID is since there were separate computer technicians who wouldn’t know a deck, camera or routing system if it bit them. That’s a fairly common scenario in some contexts these days, but at the same time, the Engineers I know wouldn’t go spreading misinformation about things they don’t know either so, again, point taken.

[Brendan Coots]

RAID 5 is a mix between 0 and 1 in the sense that you get both redundancy and performance.

The problem all around is that a 2-drive RAID is fine for RAID 1, but is pretty pointless with 0 or 5 – you won’t notice much improvement with RAID 0 and RAID 5 requires at least three drives.

The ideal setup without an external box would probably be three or four drives in RAID 0, while using an automated backup software solution to do nightly backups of the drive. If your guy is willing to drop 6K on a DLT drive, getting him to buy a hardware RAID card for $100 and a few extra drives shouldn’t be hard.

Brendan Coots
Splitvision Digital
http://www.splitvisiondigital.com