Check specs to build a new PC…

[Marlen Mathers]

okayyyyyy…..this RAID thing is getting interesting. I originally set up the RAID 0 in the BIOS without formating the discs. However, I couldn’t see the drives in Explorer.

I then as Dave suggested use DISK Manager to initialize the 1.8 GB (combo of two 1 TB drives) as DISK Manger did see the drive. I’m getting confused as to what my RAID controller VS the software angle is. I didn’t set up any partitions on the RAID – just left it wide open. Is this right for what I am attempting to do – which is use the RAID 0 to hold project media while I render back to swap file and finalized file on my boot drive. And then eventually store finalized assets on a G Raid external. Hoping for the best speed in rendering possible?

[John Rofrano]

[Marlen Mathers] ” Is this right for what I am attempting to do – which is use the RAID 0 to hold project media while I render back to swap file and finalized file on my boot drive. And then eventually store finalized assets on a G Raid external. Hoping for the best speed in rendering possible?”

What you did is correct and is exactly how I have my editing workstation set up. You create the RAID in the BIOS and initialize/format the drive in Windows as one big partition.

I was just asking Dave to clarify some of his statements because they were not what I expected to hear.

~jr

http://www.johnrofrano.com
http://www.vasst.com

[Marlen Mathers]

What you did is correct and is exactly how I have my editing workstation set up. You create the RAID in the BIOS and initialize/format the drive in Windows as one big partition.

I was just asking Dave to clarify some of his statements because they were not what I expected to hear.

Good deal Thanks for clarifying..

[Dave Haynie]

There is generally no such thing as “hardware RAID” on a motherboard. This is very often misunderstood.

RAID is very much a software protocol. When people say “hardware RAID”, they usually mean there’s software running on some other thing than the main PC’s CPU(s)… your extra hardware is a SPARC/MIPS/ARM chip running the RAID software on your RAID card, or perhaps on the CPU in an external box like my Drobo.

RAID on a motherboard is usually still running on your host CPU, just at the BIOS level, rather than at the filesystem level. “Firmware” and “software” are the same thing… we engineers tend to call it “Firmware” if it’s in a ROM somewhere. But that just means it’s always available on your MB… it’s no different in practice than software loaded from hard disk into RAM (well, technically it is, but not in a good way… Flash ROM access is hundreds of times slower than DRAM access, but the BIOS driver is only used at boot time, once you’re in Windows, there’s a Windows driver in RAM doing the same job).

So yeah, your motherboard has a Flash ROM, and that may contain a BIOS-level RAID driver, but the code from that ROM is still executed by your host CPU, just as any other code. And you probably have to load up a RAID driver under Windows to access this, which is just another chunk of driver-level code doing the RAID thing, rather than the Windows file system doing the RAID thing.

For example, a common chip on i7 motherboards is Intel’s ICH10R, which “everyone knows” does RAID.. even RAID 5. Your BIOS will set it up, and in Windows, you have to install the Intel Matrix Storage Driver to access it. But all that RAID code is running on your PC’s CPU (well, at least one of them), not offloaded to some other CPU. It’s “software RAID”, or what Linux people call “fakeRAID”.

Is it any better than using Microsoft’s NTFS RAID? Maybe. In older systems, probably not. You used to have a separate SATA or PATA “RAID” controller, which lived on the PCI bus, versus SATA build into the host chipset. So your RAID actually ran much slower and used more system resources than the built-in SATA.

These days, yeah, probably, you’re better off with the low-level driver. The SATA controller used for RAID is either in the main system I/O chip, on a private high speed chip to chip bus, or on ePCI… either way, it’s just as good as any other SATA in your system. Software-wise, even though it’s running on your host CPU, it may have small optimizations that improve performance. Intel’s driver knows very specific things about Intel’s SATA controller chip. So it may be more efficient than Microsoft’s file system approach.

Why does everyone think this is “hardware” RAID? Largely because it’s built-in to the BIOS, and in particularly, because they really don’t understand RAID very well. There is virtually no extra overhead for RAID 0 or RAID 1. And these days… benchmark a single SATA drive sometime, and see what impact that has on CPU performance. Nothing much, if it’s even still measurable (I stopped bothering, back in the PATA-UDMA days, once HDD overhead dropped below 5% CPU). Doubling that isn’t noticable. RAID 5 will have more overhead… but consider, back in the old dark days of PIO PATA, people were perfectly happy seeing 20-50% overhead from hard drive access. If Intel’s RAID 5 pushes it even as high as 5%, no one’s likely to complain.

-Dave

[John Rofrano]

[Dave Haynie] “For example, a common chip on i7 motherboards is Intel’s ICH10R, which “everyone knows” does RAID.. even RAID 5. Your BIOS will set it up, and in Windows, you have to install the Intel Matrix Storage Driver to access it. “

I’m not seeing this. I can access my motherboard RAID even in Linux where there is no Intel Matrix Storage Driver and on my dual boot XP as well as Windows 7 64-bit.

The advantage of defining the RAID at the BOIS level is that it’s OS independent. Every OS on your box (Linux, Windows 7, XP, etc.) will see it as one large drive. If you define the RAID in Windows 7, Linux will only see two unformatted drives instead of one large drive, and XP might not see it correctly either.

This is why I always recommend that you define your RAID at the BIOS level. With the horrible backward compatibility of Windows, dual boot is mandatory for a while when upgrading. The last thing you want is all your data to disappear because you upgraded your OS. I would never recommend letting Windows manage your RAID, but that’s just me.

~jr

http://www.johnrofrano.com
http://www.vasst.com

[Dave Haynie]

RAID at the BIOS level would be just dandy, and fully transparent, to the OS in question, with no additional drivers needed… as long as that OS used the BIOS. However, the BIOS is basically a 16-bit real mode program… modern 32-bit and 64-bit OSs don’t access it after booting.

Linux has had built-in support for the Intel Matrix RAID since at least 2005, maybe earlier. It doesn’t let you make any changes, but it does support it, via MD-RAID and DM-RAID. FreeBSD also supports it out of the box.

Windows has had built-in support of Intel’s RAID since XP. Like much of what you find built into Windows, the drivers originate somewhere else (in this case, Intel).

And of course, MS-DOS and Windows 3.1, assuming other things don’t stop them, would also support this, since they actually do use the BIOS.

I prefer the BIOS/driver-level support anyway, because, as you say, it’s not specific to Windows, and it’s also not specific to NTFS. So I could use FAT32 (not sure why I’d want to) or XFS or ext4 on Linux, and read them in Windows, whoops… can’t do that, there are only ext2 drivers for Windows. Ok, maybe Linux and MacOS (not sure why I’d want that, either).

The other good reason is disc recovery. When your filesystem gets corrupted, repair/recover tools need to go below the file system, to the low-level driver, to fix things (I wrote one of the best disc repair/recovery tools for the AmigaOS… have not been interested in doing anything like that again for other people’s OSs). If you have an NTFS raid, that repair tool needs special knowledge of Windows RAIDs. If you’re using the driver-level RAID, it looks just like any other hard drive… so you’ll inherently have a better selection of repair tools.

So really, I have no problems with BIOS/driver-level RAID, I just like everyone to understand how things really work. True “hardware”
RAID does the full abstraction… my Drobo looks like any other gigantic Firewire or USB drive. This works with any OS that can handle partitions larger than 2TB; only the custom Drobo software knows the difference. Same would be true of any of the RAID cards that use a separate processor to run the RAID code.

What you give up is generality. A Drobo drive only works in a Drobo. If my hardware fails, I need to buy a new Drobo to get at the data. If you have a Windows NTFS RAID, any old Windows computer (same or later OS revision, at least) would be able to access that partition. If you’re using BIOS RAID, you probably need the same kind of motherboard controller. There could be standards I suppose (there weren’t, the last time I used BIOS/driver RAID), but it’s a safe bet that a drive stack on one Intel ICHxR controller will work on another.

-Dave

[John Rofrano]

[Dave Haynie] “So really, I have no problems with BIOS/driver-level RAID, I just like everyone to understand how things really work. True “hardware” RAID does the full abstraction…”

I appreciate the clarity. I have an AMS Venus T5 enclosure that supports JBOD and RAID levels 0, 1 and 10. Unfortunately, I want to use RAID 5 but didn’t read the fine print before I bought it so I’m just using it as JBOD for now. 🙁

How do you like the Drobo? I’m still looking for an external RAID 5 enclosure for video archive and I was turned off by the Drobo because it was proprietary technology and sounded like it was the same as RAID 5 anyway.

What are your thoughts? Why did you go with Drobo instead of another RAID enclosure?

~jr

http://www.johnrofrano.com
http://www.vasst.com

[Dave Haynie]

I love the Drobo.

About the only issue with my particular unit is the interface… this is USB 2.0 and Firewire 800. No shock — knew that when I bought it, and spent the prerequisite several evening tricking Windows 7 into actually supporting Firewire 800 speeds. There are several newer models that support GigE, eSATA, and/or USB 3.0 which would be an overall better idea. I get about 45MB/s reads over FW800, about 25MB/s on USB 2.0… so this isn’t for speed versus single SATA drive, but reliability.

And one nice thing about the Drobo — if I bought one of the upgraded units, I simply pop out my drives, move ’em to the new unit, and I’m good-to-go.

Yeah, it’s proprietary. Though I’m not sure how interchangeable drives are between different model RAID units anyway. The way it works is pretty cool, too; it works with 2-4 drives (newer models at 5 and 8), the drives don’t have to be the same make/model/size, and it scales. That’s actually a big win — I currently do have four WD “Green” 2TB drives in the unit, but I bought at different times, from different suppliers… so there’s somewhat less chance of two failing at the same time than if I had started with four identical drives from the same lot.

So with two drives, it’s more like a RAID0, with four drives, more like a RAID5. Pop out any single drive any time you like, drop a new on in, and within about a day, it’s reformed and redundant again.

Along with this is the virtual partition: I created a 16TB partition over 8TB of storage. Should I start running low (I’m currently using 3.14TB, 2.28TB free), I can replace 2TB drives largely models (they claim to support up to 4TB). Newer models support either single or dual drive redundancy, and allow you to dynamically switch from dual to single.

The company’s been very good, too. I had some issues… over two years ago, and they were very responsive. Not a problem since. I think they run a bit more pricey than a basic RAID/JBOD box, but it’s worth it.

-Dave

[John Rofrano]

Awesome… I forgot to ask… How quiet are they?

The fans on my AMS box sounds like a 747 jet and I have to turn it off to create video tutorials because my microphone picks it up. I’m looking for something that is fairly quiet and allows me to do voiceovers at the PC with it on.

~jr

http://www.johnrofrano.com
http://www.vasst.com

[Dave Haynie]

Mine is really quiet. Like, you can’t hear it. It’s certainly possible the 5 or 8 bay models get louder, or perhaps with hotter drives (I have 2TB WB “Green” drives, which run pretty cool).

My main PC has a new loudness since the MB upgrade… not sure it’s the CPU cooler or a case fan, but I’ve been doing audio lately on my field recorder (recently upgraded from Tascam DR-1 to Zoom H4n)… also because my good audio interface (Tascam US-122) is questionable under Windows 7. But one of those things that needs addressing. Not so bad doing music, but for something like voiceover, this would be a real issue.

-Dave