RAIDS and Expanders (Expando Chassis?)

[Bob Zelin]

you write –
Please forgive my ignorance. I come here seeking knowledge. I don’t want to get bashed for not knowing this subject, I want to learn.

REPLY – but it’s Saturday night 8:05pm, and I am about to take out my wife and her friend. Can’t I bash you a little bit ?

you write –
Assuming you have a Mac Pro (or I suppose a PC too) that has a place (an open slot) for a RAID card/controller. Could this RAID card/controller be an “Expander” that had additional slots for more drives, enclosures or what?

REPLY – I am not reading your entire post before I respond, becuase I have to go soon. Expando is a marketing term. There are several disk drive host cards (like the ATTO R380 and the Areca 1680x) that have the ability to control 128 disk drives. Most drive chassis are 4 drives, 8 drives, 12 drives, or 16 drives. Well, what if you buy an 8 drive chassis, and run out of room? It says that you can run 128 disk drives with some of these cards. So how do you do it? You buy a disk drive array with a SAS/SATA expansion port, that allows you to daisy chain disk drive arrays together. Companies like Maxx Digital, Sonnet, JMR, AIC, and others make these disk drive chassis.
They have SAS/SATA expansion chipsets in them from companies like Vitesse, PMC, and LSI Logic, that allow for SAS/SATA expansion, so you can hookup 128 disk drives to one host adaptor card.

you write –
Someone interested in building one of these raids asked if this controller could be used with it:
https://www.3ware.com/products/sas-9690SA.asp
because “it can handle 128 max disk drives via SAS expanders.”

REPLY – host controller cards from companies like ATTO and Areca can run 128 disk drives. You buy one of these cards, and get a disk drive chassis that has SAS/SATA expanders in it, and you have accomplished your mission. Let me assure you that only certain host cards work with only certain SAS expanders. And you will find that certain disk drives don’t work in all of these drive boxes.

My basic questions, then, ARE:
Where is the expansion actually taking place?

REPLY – in the chassis. The disk drive chassis plugs into the SAS/SATA host controller card, and has to have the ability to daisy chain to the next chassis. The disk drive enclosure company has to put the SAS/SATA expander in their box. This is what Maxx Digital does (for example – there are other companies that do this too).

Do you need more chassis for the drives?
REPLY – of course you do. If you have an 8 bay chassis, that holds 8 drives, and you want to add 8 more drives, where the hell do they go, on the floor? They got to get plugged into something !

Are the enclosures daisy-chained?
REPLY – yes

The raid controller in the “build this RAID” blog isn’t expandable, as is, right?
REPLY – the Raid contoller (the RAID 5 hardware card like the ATTO R380 card) goes in your comptuer. As you add drive arrays (more boxes with more drives in them) the same RAID 5 card controls the drives, and their RAID ability (you create new RAID groups with the new boxes, using the same comptuer, and the same SAS/SATA host controller card – you don’t buy more ATTO cards, for example).

Would it be if the 3ware or another expandable controller were used?
REPLY – I am not keeping up with what 3ware and the Sidecar products are doing these days. Not all expander chassis are compatible. So you just can’t say “hey, I will go out and get a 3ware package, and then get a Proavio later, because it’s cheaper”. Not everything is compatible.

Thank you very much for talking the time to read this and (in advance) for any advice given!

REPLY – this is my advice. Don’t try a do it yourself solution. We suffer all the time, trying new products – what works, and what doesn’t work. We spend hours on the phone screaming at manufacturers (we say they are idiots, they yell back and tell us that we are idiots). Eventually we find out what works, and what doesnt’ work. And then something new comes out, and people ask me “well, what about that new product (like the 3ware) – how does that work” – well, I DONT KNOW, I haven’t tried it. When you see people say “hey , I want to buy my own cheap Samsung SATA drives and put them into a CalDigit HD Pro – will it work” – I dont’ know, and Cal Digit won’t let you. They put A LOT OF EFFORT into finding out what does, and what does not work. Everyone thinks that they are going to “roll their own” and save a bunch of money. sometimes you get lucky, sometimes you dont’. And when you don’t, you get screwed financially.

You either buy a working recognized product, or you take your chances. The companies you see advertised on Creative Cow are basically very good companies. There are basically no companies on Creative Cow that advertise, that make “junk”. The ones that make junk are afraid to have user forums on Cow, because they will be bashed.

I didn’t bash you too badly – did I?

bob Zelin

[Eric Harnden]

Amazing response(s) Bob. Thanks so much!

I feel properly “bashed” (and educated)! I really appreciate you taking time this evening to answer this post.

Hope you have/had a good time out tonight.

[Bob Zelin]

Eric writes –

Someone interested in building one of these raids asked if this controller could be used with it:
https://www.3ware.com/products/sas-9690SA.asp
because “it can handle 128 max disk drives via SAS expanders.”

Hi Eric –
I have just read thru this website on the 3ware. If you are using Final Cut Pro for your editing systems, it will not work –

from this URL –
https://www.3ware.com/products/sas-9690SA.asp
Operating System Support
Microsoft® Windows® Vista/2003/XP/2000, Red Hat Linux, SuSE Linux, Fedora Linux, 2.4 Linux kernel, 2.6 Linux kernel, FreeBSD; 32/64-bit driver support for Windows, Linux and FreeBSD. For complete OS listing, Go to: https://www.3ware.com/support/OS-Support.asp

There is no MAC support. The 3 SAS cards that can handle expansion chassis come from ATTO, Areca, and Highpoint. To my knowlege, no one else is making it (LSI might, and they certainly make the SAS Expansion chip set, but I don’t know if they have a SAS/SATA host controller that has MAC drivers).

Bob Zelin

[Eric Harnden]

Thanks for the in-depth investigation!

[Jon Hunwick]

Hi,

I was disappointing when i saw MAC OSX was not supported so i did some further investigation…

there is a link on the above page that says: click here for a full listing of suported OS’s

https://www.3ware.com/support/OS-Support.asp

The 3ware 7000 series Parallel ATA RAID controller, 8000/9500S series Serial ATA RAID controller, 3ware 9550SX(U) series Serial ATA II RAID, 3ware 9650SE series RAID 6 PCIe Serial ATA II controllers, and 9690SA PCIe SAS/SATA controllers provide support for the following operating systems:

Windows | Linux(sorted by controller card) | Linux(sorted by Linux distribution) | FreeBSD(sorted by controller card) | FreeBSD(sorted by FreeBSD version) | Mac OS X | OpenSolaris | VMware |

further down:
9690SA Series

* Mac OS X 10.4.8 (or higher) and 10.5.0 (or higher) for Mac Pro systems

So this should mean I can use it with MAC OSX. I am going to get in touch with 3ware and find out what expander chassis + drive combos they recommend. I saw their both at NAB and they had a few working setups with both Avid and FCP.

I’ve also been looking at the ATTO, CalDigit and Rocket raid cards…

I intend on building a solution in the near future… but it has to be with the right Card, chassis and drive combo.

thanks for you info so far though!

Cheers

Jon

[Archie Cruz]

Will this work ?
https://lifezero.typepad.com/blog/2008/10/super-fast-4tb-raid-for-under-2k.html

[Bob Zelin]

Hi Archie,
I have copied this quote from the article you sent the link for –

“Hey John,

Great post – I saw it via a link at RedUser. There’s a great article from Bob Zelin about turning the CalDigit HDPro into a “poor man’s Avid Unity” shared storage – I bet it would work with your config as well. (Not for 2K, however: only uncompressed SD, and ProRes422HQ and DVCProHD across the network.)”

REPLY –
this is an ASSUMPTION that I made when we started doing this – that any RAID array, and any SATA host adaptor will work. What we found is that if you are doing 30 minute – 60 minute shows, SATA host cards (like the Highpoint mentioned in this article) have latency issues. What does this mean in real life ? It means that you play out your show, and everything is working great, and then 28 minutes into your show, you get DROP FRAME ERROR – you check the drive array, and it checks out fine, but the damn show just stops. When you choose a SAS/SATA host adaptor like the ATTO R380 card (that costs 3 times as much as the Highpoint mentioned in this article), you don’t get these problems.

This is what everyone always forgets about these “build it yourself” systems. You build it, it works, and then you say “wow, this is great, I just saved a ton of money”. And then it fails when you actually start to use it. This is why most turnkey solutions are better for many people – because a company stands behind the product. Believe me, I wanted cheap too – and I SUFFERED thru experimenting, and failing. I went thru this with the cheaper host cards, and I went thru this with the Seagate 1.5 Terabyte drives. No one could convince me that they were crap, until mine started to die out. They worked great for 3 months. Sometimes you need a kick in the ass to see what works, and what doesn’t work.

Bob Zelin

[Archie Cruz]

Yes Bob.Your point is well made- borne out in this follow up article in DV “The Ultimate DIY RAID – UPDATED”
By Ned Soltz, January 23, 2009- somewhat ‘fresher’ than the LifeZero article (which mysteriously went dead dead last Winter)
https://www.dv.com/article/45378
Note especially the change in bus preference (is that the correct term?) from SATA to SAS.
Ned Soltz echoes your words BUT that still doesn’t stop him from publishing the article and no doubt- there must be SOME DIY That duplicates the so called ” Bullet Proof” CRM that CalDigit offers? Surely?