Mac Pro 2008 / Internal Raid

[Alex Gerulaitis]

[Pat Ostman] “2. I would really like a raid 6 configuration that is at least fast enough to break 500MB read / write. Is that possible internally? I would like to have about 6TB of useable storage, is that sensible with just 4 drives? Which drives should install?”

RAID6 with just four drives is not very practical, and while you may get peak read speeds of 500MB/s (when drives are empty), write speeds will be lower – unless you use SSDs. For arrays where performance matters (not just redundancy), it’s recommended to have at least 6-8 drives in a RAID6 set.

Four HGST 7K4000 3TB drives in RAID6 will give you 6TB usable capacity; about 400-550MB/s peak read speeds with ATTO R680 (not Areca 1882 – it has issues), and I am guessing 250-350MB/s peak write speeds.

Speeds will drop down to about 50% of peak values when drives get full or fragmented.

To sustain 500MB/s r/w speeds throughout the surface of a RAID6 array, it has to be at least 12 high performance 7200rpm drives. Why I think that: using 8 drives in RAID6 on R680 gives about 700MB/s peak write speeds (read speeds are 1GB/s and up); divide this number by 2 to arrive at sustainable write speeds of about 350MB/s; you would need at least another 3-4 drives to bring it up to 500MB/s.

Alex Gerulaitis
Systems Engineer
DV411 – Los Angeles, CA

[Pat Ostman]

Alex,

Thanks for your comments. I am curious about the Areca issues. Are they long standing or related to the recent release of Mountain Lion?

Would an internal raid 6 be a reasonable interim solution until I build out into an external enclosure? I am trying to determine a quick ramp up starting point.

Broader questions based on your input.

Is it best to get a card with multiple SAS external connectors or can the drive enclosures be daisy chained off 1 port. I read somewhere that and 8 bay enclosure would require 2 SAS cables back to the Mac Pro. But it is unclear if this is a card requirement or an enclosure req.

[Alex Gerulaitis]

Areca issues were reported on this forum, summarized here. Something to do with the firmware, I think – the hardware (LSI brain) is fairly standard.

[Pat Ostman] “Would an internal raid 6 be a reasonable interim solution until I build out into an external enclosure? I am trying to determine a quick ramp up starting point. “

Not with desired performance in mind; otherwise yes.

[Pat Ostman] “Is it best to get a card with multiple SAS external connectors or can the drive enclosures be daisy chained off 1 port. I read somewhere that and 8 bay enclosure would require 2 SAS cables back to the Mac Pro. But it is unclear if this is a card requirement or an enclosure req.”

The simple answer: it’s the enclosure; some require a MiniSAS port for each four drives; some don’t.

The longer answer: (please bear with me, English is not my first language :))

There are two types of external SAS boxes: “dumb” and expander. “Dumb” boxes need a SAS lane for each drive; each MiniSAS connection has four such lanes; thus to connect an 8-bay “dumb” box, you’d need a controller with two MiniSAS connections. That’s probably what you’ve heard.

“Expander” boxes are daisy-chainable, and run off a single MiniSAS connection. Each normally has two “expander” MiniSAS ports to which you can daisy-chain additional enclosures. There is a limit on how many, but it’s pretty high: e.g. up to 8 total enclosures, up to 122 total drives. You probably want to stick to the same manufacturer and even type of enclosure when daisy-chaining; they are not necessarily all compatible or interchangeable.

Most “dumb” boxes are 8 bays or less; thus whenever you look at a 12+ bay SAS enclosure, it’s likely the “expander” type.

To answer your question… If we assume that you need sustained 500MB/s in RAID6 and based on all of the above, the best bet is a 12- or 16-bay “expander” enclosure (dual 8-bays also possible). That enclosure will run off of a single MiniSAS connection.

Prices:
– 8-bay “dumb” MiniSAS 6G box such as CR-T08: $600-900
– 8-bay “expander” MiniSAS 6G box such as CR-T08E – $1500
– 16-bay “expander” MiniSAS 6G box such as CR-16E: $2400.

Not including drives or a controller of course.

Disclaimer: Areca (and ATTO) only support enterprise class drives in their enclosures and with their controllers.

Unless there is an extremely tight budget, it’s recommended to buy the whole thing from one place: the array will get initialized and benchmarked, drives tested. (Disk array performance issues aren’t easy to trouble-shoot on-site without spare parts, and drives do fail, even enterprise class, even DOA.)

Did I cover everything?

HTH.

(If you are looking for a single place to buy such a thing, I’d be email you a quote, shoot me an email, ag at dv411 dot com with your contact info.)

Alex Gerulaitis
Systems Engineer
DV411 – Los Angeles, CA

[Alex Gerulaitis]

[Alex Gerulaitis] “The simple answer: it’s the enclosure; some require a MiniSAS port for each four drives; some don’t.”

Well in most case it’s true but not in all; some SAS RAID controllers don’t support SAS expanders, notably Areca 1600 series; 1800 do, and so dooes ATTO.

Alex Gerulaitis
Systems Engineer
DV411 – Los Angeles, CA

[Pat Ostman]

Alex,

Thank-you again for your insight.

Unless I misunderstand your explanation of how the enclosures work, I can get a card with an internal and external mini-SAS port and lose little long term flexibility provided I assure I buy an enclosure that acts as a SAS-Expander vs just a “dumb” enclosure.

So my plan – today – is:

1. Buy a card and 4 3TB disks for use internally. Accepting this will not give me the best performance over the long term, but it will improve the < 45mb/s I am now seeing.

2. Buy an 8 or 12 bay external unit, or 2 8’s chained together for use as working storage set probably with 2TB drives. Once this step is complete, I’ll use the internal set as time-machine and bits and pieces storage.

3. Use a Synology 4 or 5 bay unit for off-site (building) backup.

I’m split between the Areca 1882lp and the ATTO 644. The Areca seems to be better value, has more cache, etc. Reading posts here and other groups show both have strong and not so strong points.

[Alex Gerulaitis]

Pleasure!

[Pat Ostman] “I’m split between the Areca 1882lp and the ATTO 644. The Areca seems to be better value, has more cache, etc. Reading posts here and other groups show both have strong and not so strong points.”

Areca confirmed the issue (of slow read speeds) is with the firmware: they were able to find a previous version that doesn’t have the issue, now they just need to figure what they changed… 🙂

[Pat Ostman] “Unless I misunderstand your explanation of how the enclosures work, I can get a card with an internal and external mini-SAS port and lose little long term flexibility provided I assure I buy an enclosure that acts as a SAS-Expander vs just a “dumb” enclosure. “

Perfect.

Alex Gerulaitis
Systems Engineer
DV411 – Los Angeles, CA