Raid for on-set use

[Alex Gerulaitis]

[Gautam Pinto] “I have no clue as to which chassis to select. I want an 8-12 bay chassis for 3.5″ drives 6G SAS and 4TB drive support. I need small, fast, quiet and not too expensive.”

Will a tower work? CineRAID CR-T12E is quiet, fairly inexpensive as far as SAS expanders go.

Alex Gerulaitis
Systems Engineer
DV411 – Los Angeles, CA

[Bob Zelin]

what a long post !

Areca 1882x and Atto R680.
REPLY – both good cards

My Mac is crowded… I’ve got GPU cards, Red Rockets, QIO, Black Magic video IO, PCIe Expansion card, ATTO HBA, and more… If I can simplify and use the SAS Raid controller to connect to individual disks via Esata and simultaneously connect to the RAID that would be a huge benefit to me. Mac pro’s are slot limited.

REPLY – this is where you will die. You can’t use your server for Red Transcoding, QIO transfers, editing and grading with the Blackmagic cards WHILE you are using this computer as a server for your other editing stations. It’s not going to happen. This means, that since you need all these functions (editing, grading, data transfer, RED transcoding, QIO transfers, etc.) you need A SECOND MAC PRO, and not an expansion chassis for more slots. Plenty of companies on this forum will show you their wonderful Win 7 and/or Linux servers with built in drive arrays, that can fit into 2 RU, that can do everything in one box – but these are not Mac’s – they are dedicated. And if you insist on a Mac, you need a dedicated Mac. You ain’t doing RedCineX on this Mac while people are editing using this as a server.

System resources are limited

In terms of reliability and performance, both cards seem equal. Atto seems to have better compatibility with LTO 5 devices ( a plus for the future ), but since it does not have an active cooling system there are some reports of heat issues ( a definite bad thing for on-set use ).

REPLY – both cards work. You can’t have 10 pounds of S#$% in a 5 pound Mac Pro. You want to do everything in one Mac, because of space, weight, setup time, and expense. Sorry, it’s not going to happen. Can you use a Mac Mini – sure, but with an expansion chassis and an external drive array, and your speeds will not be what you can achieve with a Mac Pro.

Silence is key for set use

REPLY – no silence if you have a switch in your system. An 8 bay drive array is quiet (like a Mac Pro) but none of the good switches are quiet. If you use DIRECT CONNECT (multiport card in the server computer, out to 10gig or 1gig ports in your client computers) then you will have “silence” – but not with one of the fancy schmancy 10 gig switches on the market. No miracles yet.

I plan to mount the RAID controller in a PCI express expansion chassis that has 2x GPU’s inside. The fans in the chassis are silent and dont circulate heat as efficiently as say a Mac Pro with fans on full blast.

REPLY – like I said before, this will fail. You will get a Cubix or Magma expansion chassis, in hopes that one computer will do everything without issue. It won’t.

Heat is a potential issue, especially while shooting in hot environments.

REPLY – you will suffer. Heat is the #1 killer of drive arrays. If you were shooting outside in Florida, with the heat and humidity (and bad power), you will kill any big drive array. And if the inside of your Mac Pro has got an R680 and a Red Rocket, and a Blackmagic Decklink HD Extreme+ (and 10gig cards, etc, etc) you will just blow up.

I also need a sas expansion chassis for the disks, I currently have 10x4TB hitachi SATA III drives to use with the array. I plan on using 8-12 drives on-set, and later can use the array and expand it in a post studio for color correction work. I work with 4K Red material, Alexa, and other systems frequently.

REPLY – you want a full facility rig, all in one computer. Sorry, not going to happen.

Performance is a must

REPLY – easy – 10 gig Ethernet with 10gig cards or boxes on each client computer. You won’t get multi computer 10gig performance with a Mac Mini. So if you go with direct connect (you did not say how many client computers you will have) – you need to either avoid the switch, and go direct connect, or go with a switch, and bond multiple 10gig ports to a switch. And the switch makes noise. And all these cards cost money, and create heat – especially with all the other crap you have in your current Mac Pro.

I am on a cart based system so when everything is added up, including two monitors, the mac pro, various rack hardware, and a gigantic UPS the system becomes very heavy. Weight is definitely a consideration for a mobile setup.

REPLY – in a word – this ain’t gonna happen. You want an entire facility to live on a single cart, that can do Red transcoding, editing, color grading, data transfer, and act as a shared storage server (with storage) all in one little lightweight cart. Not going to happen. You will see plenty of people respond to your post with their under $8000 boxes. You will find that none of these will give you the speed you want, or the expandability that you want for # of clients, or the amount of storage that you want.

Weight is a key factor

REPLY – I know. Sorry, you can’t have what you want.

I have seen SAS various expander chassis from Proavio, Sans Digital, Enhance, iStorage Pro, PC-Pit Stop, Areca 4036, Cineraid, and others. The size of the chassis, the noise and the weight, are all important factors. Since I am an independent freelancer, the cost of the chassis is a factor also, so to avoid weight, heat, noise, and high cost I’d rather stay away from redundant power and redundant controller chassis options. From what I can see, most of the options in the market seem to be LSI – Areca expanders built into these boxes. I don’t know if this has an impact on performance and if the Areca or Atto Raid controllers are superior. Many benchmarks I have seen show that the ATTO actually outperforms the Areca even though it’s an older card. But for the enclosures many of the spec sheets do not publish noise and weight numbers, and some dont list updated compatibility with 4TB drives.

REPLY – you are accurate in your assessment on this. All good companies (who is PC Pitstop?) – the good stuff costs money. Big drive arrays are heavy. A server array will make noise, so will a professional switch. Both the ATTO and Areca are NEWER cards (April 2011), and both have similar performance. They both work perfectly with 4TB SATA drives.

I have no clue as to which chassis to select. I want an 8-12 bay chassis for 3.5″ drives 6G SAS and 4TB drive support. I need small, fast, quiet and not too expensive. Now before I get a good BOB Z. bashing (which I am ready for) 🙂 I am willing to talk to a VAR, I am willing to spend money, and I am willing to read manuals, do research, and purchase a tested system. I also want the maximum capacity I can get my hands on, but once you get up to 16bay chassis things get hot, loud, heavy, and expensive. This tray less chassis seems to be a good option as it can be expanded and can be used without all bays being populated: https://www.pc-pitstop.com/sas_cables_enclosures/scsas156gt.asp however this unit is heavy, and does not seem to have the build quality of some of the other products in the same price range. It’s basically a tower with a bunch of iStar tray less sata enclosures in it and an Areca expander.

REPLY – I want your business, and so do all the other companies that will respond here. But I for one cannot give you what you want – this entire super fast expandable 10gig system, that is lightweight, silent, inexpensive, and can run ALL THE OTHER CARDS on the same computer at the same time. I can’t do it. When you look at other good solutions from the people that will respond to this post, these are STAND ALONE solutions that are either Linux or Win 7 based computers with drives inside the chassis. None of them will run your software, or contain your Red Rocket, Blackmagic cards.

Compatibility with a wide array of SAS enclosures and overall Cost are both important to me.

REPLY – there is no compatibility. You can’t mix and match a Promise array with a iStorage Pro chassis. Unless you have matching brands of SAS expanders, you are betting for trouble. Actually, the entire system you are asking for is just begging for trouble.

Other benefits for me for both the chassis and controller are the following:
– A raid controller that can play nice with LTO 5 drives (on second channel)
REPLY – the ATTO card is the right card for this, not the Areca. But you can buy a Cache-A, and just plug it into your network switch (or isolated network port).

– A raid controller that can scale up to larger chassis / daisy chained arrays
REPLY – many of the companies you listed offer SAS expanders in their products. You can’t mix and match these.

– A reliable controller that will not suffer from heat issues inside a crowded enclosure
REPLY – your choice is the R680 and the ARC-1882x. End of story.

– Speed and performance for large file video workflows (usually I am grading and offloading data at the same time)
REPLY – you aint’ doing this while other people are doing shared storage on this computer.

Unfortunately in terms of all the chassis I have seen, 8 bays seems like its not enough capacity, and 12 -16 bays seem like they are too heavy, loud, and expensive.

REPLY – no kidding. Welcome to “state of the art”.

So Cow friends, what chassis and controller combo is the best for me to get? I know it’s quite a bit to ask for, and I am realistic in terms of what can be done. I am not expecting any miracles just some advice if anyone has worked with something like what I am looking for.

REPLY – no you are not being realistic. You sound like the guy that want’s to build his entire RED Rig for $15,000 complete, and carry it in his backpack. You want an entire facility of equipment that can run all on one Mac, all at the same time. And you want it to be inexpensive, silent, lightweight, and super fast.

Sorry – no miracles for you.

Bob Zelin

I’d also be willing to look at a box that has a “Built in Raid” but need to know that it will work flawlessly with my ATTO H680 HBA, and that I will get similar performance to an R680 or 1882x. I’ve seen posts on the web about the H680 having issues with the Areca built in raid systems.

Bob Zelin
Rescue 1, Inc.
maxavid@cfl.rr.com

[Gautam Pinto]

Alex, looks like a good option. What is the MSRP? Also do you know if the weight is really 25lbs without drives? Does this chassis work the same with the Atto and the Areca? Have you been able to stress test with both cards, or benchmark with both cards?

Is there an 8bay option?

The Areca has a BBU, but the Atto cache assure looks like a more elegant solution. Care to comment?

Gautam

[Eric Hansen]

@Bob – i don’t believe the OP ever mentioned wanting to share media over ethernet. It sounds like he’s just putting together a DIT cart.

@Gautam – I use the R680 and have never used the Areca card, but I hear it’s great. I’ve heard that only ATTO allows LTO and hard drives on the same card. I don’t think Areca cards can do this. Go LTO6 and direct attach it so you get the speeds. Cache-A units are too slow for what you’re trying to do. as far as the eSATA fan out cable, it will work on the R680, although I’ve found it to be flaky. if you run the fan out cable or LTO, they have to be on their own port. you can’t connect a tape device to a chain with hard drives on it. I’ve not tried this with the fan out cable, but i don’t think it will work on a chain. Heat will definitely be your biggest concern, especially if you put all this in a rolling rack that will sit outside. something like this will want to be in a controlled environment otherwise components will wear out or fail completely.

Eric Hansen
Production Workflow Designer / Consultant / Colorist / DIT
https://www.erichansen.tv

[Gautam Pinto]

Bob, thanks for your valued feedback, however I’m sorry I think you misunderstood.

I’m not looking for a shared storage environment. I have several systems to do that with If I need to. I am looking for a mobile single user storage Raid that can handle color grading a single stream of digital files while simultaneously offloading data at about 100MB / sec. I need something quiet, and not too heavy since I have to move the system on a wheeled cart setup. Currently I am using Resolve so unfortunately I need the PCIe expansion chassis for GPU cards. Since I have no slots left I am forced to use the chassis if I want to implement a SAS raid, or alternatively use a box that has a built in raid controller with my Atto H680 HBA. I just was looking for some advice for a quiet system that is not designed for a server room, but for a quieter environment. I just dont know if the H680 works well with the systems that have built in Raid controllers. I am currently using an older Areca Raid controller in the PCIe chassis along with 2x red rockets, 2x GPU’s and a Black Magic card, H680, GPU and PCIe expander card in the tower. Every works perfectly, and I have absolutely no issues. I use this system for months on end 24/7 on film and TV shows. I have backups for every component, but since everything is on a clean power UPS, I have not had any power issues at all. I want to just replace an older 4bay raid, with a newer 8-12 bay raid for larger capacity and higher performance.

Gautam

[Alex Gerulaitis]

[Gautam Pinto] “Alex, looks like a good option. What is the MSRP? Also do you know if the weight is really 25lbs without drives? Does this chassis work the same with the Atto and the Areca? Have you been able to stress test with both cards, or benchmark with both cards?

Is there an 8bay option?”

MSRP $1899. You can order it from me unless you already have a reseller you work with, who sells Areca/CineRAID products.

Not sure what the exact weight is. The 8-bay SAS expander (CR-T08E) is also 25 lbs according to specs – but not as quiet as the 12-bay one.

Yes works the same with Areca and ATTO. I haven’t personally stress-tested them – my clients have. One of them reported getting around 1GB/s on 12 4TB WD enterprise drives in RAID6 with Areca 1882x.

Alex Gerulaitis
Systems Engineer
DV411 – Los Angeles, CA

[Alex Gerulaitis]

[Gautam Pinto] “One thing I don’t know is if either of those cards can use one channel for Raid and the other for standard SAS to E-sata fan-out cables to mount single drives in OSX.”

Technically it’s possible – the controllers support JBOD (individual drives), and support SATA. Not sure about removable devices (a must with eSATA). I’d be nervous connecting eSATA shuttles to a controller which has my precious RAID6 humming. I know space and money are tight, but like sensei BZ says, this is asking for trouble. I’d look at a separate el-cheapo machine just for that purpose.

[Eric Hansen]

it might not be a bad idea to get 2 Mac Pros and tie them together with 10Gb Ethernet. A Cubix costs as much as a Mac Pro anyway. one MP with Red Rocket, Titan, BMD card and 10GbE card, the other with 10GbE card, SAS card and Caldigit eSATA/USB3 card.

as far as GPU power, i think a single Titan will be fine for dailies work. upgrade to the Rocket-X when it’s released so you can get realtime on 5k content.

e

Eric Hansen
Production Workflow Designer / Consultant / Colorist / DIT
https://www.erichansen.tv

[Rainer Wirth]

Hey, make it simple,
Alex has given you the best advice. A 12 bay set up. Both cards and brands are nearly the same.
Best ideas are simple!

cheers

Rainer

factstory
Rainer Wirth
phone_0049-177-2156086
Mac pro 8core
Adobe,FCP,Avid
several raid systems

[Gautam Pinto]

Interesting idea. While I like the idea it would also introduce the following challenges.

– Mac pro weighs over 50lbs with drives and takes up 8.2″ width and 20″ height.
– I would still need massive fast storage (another 4u enclosure)
– slower renders for ProRes with single Titan compared to 3x GTX 580’s
– larger increase in component failure probability

Interesting option. Here are the benefits…

I would not need a KVM as I could remote screen share to the other tower for offloading only.
Since the other tower is dedicated to offloading it could be the least expensive mac tower refurb quad core.
10Gbe file sharing from mac to mac would offer around 800MB / sec of theoretical throughput?
Avoid the cost and noise of a switch.

Eric, this does sound like good option. Would I be able to get 110FPS with a few nodes in resolve for Alexa 444 with LUT encoding to ProRes LT with a single Titan and a GUI card? Wait a sec is Titan dual width? What about power? Would I be able to squeeze a two cards in the tower? Or will I need Titan as GUI and GPU.