affordable RAID and possible media share recommendations?

[Walter Biscardi]

[Jim Bruce] “1) Any suggestions for about 6 TB of RAID 5 (or 6?) storage? Affordable and reliable? CalDigit vs. G-Tech? Other Brands? “

Maxx Digital EVO Series. We’re running two year old older models and a brand new EVO 16TB system in a shared configuration.

[Jim Bruce] “I have read Bob Zelkin’s description of a cheap network with Small Tree cards and a dedicated server and for me it’s just not worth it for my particular situation of only wanting 2 work stations. “

Actually it is. We’re running three FCP systems AND three iMacs all tied into the Final Share system from Maxx Digital which is the ethernet based shared storage system. All 6 of our systems are tied together via ethernet and all 6 of them can edit 1080i ProRes HD all day.

You only need two workstations now, but you can add in any number of other computers moving forward. Does the Producer have a Mac Laptop? they can connect to the server and look at raw clips while you edit.

Here’s my article on my SAN. https://library.creativecow.net/articles/biscardi_walter/media_san.php

[Jim Bruce] “3) Since I’m not going to be a 24/7 post-house type of user is there any reason to go for anything beyond the HDElement? “

We’re not a 24/7 post house either, pretty much 8 or 10 hour days. We’ve been running the SAS/SATA type of arrays here over 2 years now all in RAID 5 and they’ve been plenty fast and very reliable.

[Jim Bruce] “4) Is Raid 6 worth considering? “

Not sure why. We run RAID 5 and knock on wood, we’ve never had a failure. Of course we have spare drives just waiting in the cabinet so that’s probably one reason why….

Walter Biscardi, Jr.
Biscardi Creative Media
HD and SD Production for Broadcast and Independent Productions.

Read my Blog!

STOP STARING AND START GRADING WITH APPLE COLOR Apple Color Training DVD available now!

[Neil Sadwelkar]

Walter is right about getting storage that’s made for more than the 2 stations you have at the moment. If you make a ‘for the moment’ solution, then that will also cost something. To make that shared may add even more of an expense.

That being said, if you’re absolutely sure this will be your need for a while, then you can consider getting some really fast RAID 5 storage for a MacPro. Then connect both its GigE ports to a small GigE switch. And connect an iMac to the same switch. You can edit/capture on the RAID 5 from the iMac by mounting it on the iMac.

So this MacPro will be an edit station as well as a server. You may have trouble if you capture using the iMac to the RAID 5 while the MacPro is also editing off the same. But this solution will not cost you anything extra. So you can check if it works, and if its troublesome, then look for something like what Walter has suggested.

Neil Sadwelkar

FCP Editor, Mumbai, India.
Completely PAL.

[Mark Raudonis]

Jim,

If you’re 100% confident of needing JUST two systems, then you can simply connect the two computers with an ethernet cable and you’re done. No switches needed. Just cable the two ethernet ports on the computer together. Google “Sharing two computers” over ethernet for the details.

Of course, everyone’s needs change, and your “two computers” today will become “several” tomorrow. In that case, perhaps you should spring for a switch. But… if you just want to get through this project , then
one cable is all you need for just two computers.

Mark

[Bob Zelin]

Hi Jim,
I am only responding to your stupid post, because the esteemed Mark Raudonis has wasted his time answering you. I have the actual answer for you. I am responding only because I want to drill into Mark Raudonis’ head that NOTHING IS CHEAP ENOUGH. No matter how cheap it is, if it is not FREE, it’s not cheap enough (can’t I get someone to PAY ME to use shared storage !).

OK Jim, here is your answer, as I have already written another loser in the SAN Networks forum of Creative Cow. If you don’t like being insulted like this, I dont’ care – this is free advice that actually works, so tough noogies. You will take my insults.
–

REPLY – ok, here we go. Connect a regular boring CAT 5e ethernet cable between MAC Pro Ethernet Port # 2, and the iMAC. Nothing else can be using ethernet port 2. If you have your network setup on these computers, just use ethernet port 1 for your regular network, until we finish this demo – ok ?

OK, so now you have a single ethernet cable between the MAC Pro ethernet port 2 and iMac ethernet port.

On the MAC Pro (our server), go to System Preferences>Network>Ethernet Port 2, and MANUALLY configure this port. Assign a manual static IP address of 192.168.2.3, and a subnet mask of 255.255.255.0. Noththing else. Hit APPLY. Click on ADVANCED. Click on the ethernet tab. Change from Automatic to MANUAL, and say 1000baseT, flow-control, full-duplex, and
MTU CUSTOM:8100 (you have to enter the number 8100). Click Apply, OK.

On your iMAC, go to System Preferences>Network>Ethernet Port 1, and MANUALLY configure this port. Enter a manual IP address of 192.168.2.4, and a subnet mask of 255.255.255.0. Nothing else. Hit APPLY. Click on Advanced. Click on the ethernet tab. Change from Automatic to MANUAL, and say 1000baseT, flow-control, full-duplex, and set the MTU size to custom – 8100 (manually type in the number 8100) . click Apply, ok, until you are out of this menu.

On your iMAC, go to System Preferences>SHARING, and on the top, you will see the name of the computer. Make it something nice and simple, like “edit1”. Click on System Preferences>Accounts, and make sure that the same name appears here, like “edit1”. Make sure that this computer has a password, and if it doesn’t, give it a password, like “JIM”.

Go back to your new “server” computer – the MAC Pro. Go into the System Preferences menu>SHARING, and make sure this computer has a nice name, like “edit2”, or “server”, or wahtever you like. Then go into System Preferences>Accounts, and make sure that it has the same name (like “edit2”), and make sure it has a password (like “JIM”).

OK, now, on the server computer – the MAC Pro, click on the System PReferences>SHARING Icon, and you will see on the left hand side – FILE SHARING. Click this on. All of a sudden, 3 columns open up. The first column is for drive volumes that you are going to share. Click the “+” key, and add your RAID array (or your 1 terabyte external drive, or anything else) to be your shared drive volume. In the next column is USERS. Click on the “+” sign in this column, and add a NEW USER. The new user box will open, and you will enter the NAME AND PASSWORD of the MAC G5 Quad – like “edit1” for the name, and “JIM” for the password. Once you click on this, you will now make “edit1” able to share the XServe RAID on the MAC Pro. In the third column, it now says “READ ONLY” next to “edit1”. Make that say “READ AND WRITE”.

OK, we are ready.

Walk over to your iMAC computer. Click on GO>Connect To Server. Enter this number – 192.168.2.3 (this is the address of ethernet port 2 of the MAC Pro). It will now ask you for a name and password. Your name is “edit1”, and your password is “JIM”. Click CONNECT. WOW, there is your RAID appearing on your iMAC.

Click on the drive. Open up a Quicktime file, or FCP file. HOLY CRAP RAY – you are now playing the compressed HD movie from the MAC Pro on your MAC G5 QUAD computer.

YOU DID NOT SPEND ONE PENNY, and now you have shared storage. Wow, Ray, how did this happen ?

Now, if you want to have true shared storage, all you need is a dedicated MAC Pro, and about $2000 in special ethernet equipment, and THAT IS ALL. Pretty amazing – huh ?

See JIM – it DOES work – stop asking dealers whose only concern is to get $50,000 from you.

Bob Zelin

[Jim Bruce]

Hi Bob, Mark, Walter and Neil,

Thank you all for weighing in. I would have replied sooner but I apparently forgot to check the box that emails me when someone posts…

Walter: Thanks for jumping in. I’ve read about your setup and wish that it applied to me (which it would if I had cash to hire 2 other fulltime editors!) but the cost of even the dedicated server computer makes it not necessarily something I’m unable to pay for, but something that doesn’t make sense for my project – which is all I’m going to be doing for about the next year so (and no plans to start a post house thereafter).

Neil, Mark and Bob:

Thanks very much for the tips (and the abuse Bob!), but my key question is will simple ethernet file sharing run the risk of file corruption if both computers are using the same media files at the same time? I’m very comfortable with the process involved (and appreciate Bob’s detailed approach) but I posted here in the hopes of getting a specific take on whether I run the risk of media corruption over time, which to me would be a major reason not to do it.

Has anyone edited for a considerable period of time (6-10 months) using this type of sharing?

Is there a software program that I could use to serve a traffic cop and mitigate this risk?

I ultimately need (over the course of the next year) about 1.5 stations. If a “proper” share like Walter’s is going to cost 4 or 5 grand, I’d rather set up a duplicate system using Firewires for 1.5k (in storage cost since I have the IMAC already) and have the pleasure of one totally redundant storage setup in case my primary RAID goes fuzzy – this would kill the backup and second station birds with one stone.

So if anyone has thoughts on corruption from long-term file sharing that would be great.

And in terms of the actual primary RAID purchase, if anyone’s willing to chime in on the CalDigit Element vs. HD One vs. HD Pro vs. G-RAID Fibre Channel vs. EVO 2K Expando vs. ??? debate that would be a huge help

Thoughts?

Any feedback on best brands of 2TB firewires? G-Raid 3? Cal Digit VR? Other brands? In terms of buying the best and most reliable I am happy to pay up here.

Thanks all for your help!

And Bob, thanks for calling me a loser! If that’s the price of your great advice I’m happy to pay it…

Cheers,

Jim

[Neil Sadwelkar]

We’ve done exactly what you refer to, since Dec 2004. Not one single media file has ever been corrupted. Just because someone opened it over the network.

Just to clarify, we mount each others drives and share media over GigE between 4 FCPs. All with attached storage. So, an editor could be editing without knowing that someone elsewhere is actually running media files that are on his disk.

What we don’t do is to capture to each others drives. But that is not our requirement. Ultimately you have to realise that if you’re editing media that’s not on your disk, all you’re doing is actually just playing those files. FCP doesn’t alter media files while editing them. Except if you change reel name, file name, or timecode.

Neil

FCP Editor, Mumbai, India.
Completely PAL.

[Bob Zelin]

1) how do I get Neil to participate on the Creative Cow SAN Networks forums. You obviously have lots of experience doing this, and know the right (and wrong) things to do – to avoid problems.

2) I will respond to your questions below –
you write –

will simple ethernet file sharing run the risk of file corruption if both computers are using the same media files at the same time?

REPLY – you can read the same file at the same time. Just don’t try to WRITE to that file ! Open up a duplicate project name (like GamesA and GamesB) and you will have 2 similar projects reading the same media at the same time with different metadata. You will not have file corruption. Managing your shared storage volumes (knowing what things are called, and where you are putting them ) is critical in making a system like this work. No messy workflow – it’s too easy to screw up someone elses files.

you write –
I posted here in the hopes of getting a specific take on whether I run the risk of media corruption over time, which to me would be a major reason not to do it.
REPLY – you will be fine. However, over time, drives fail, and RAID 0 arrays can become corrupt. Lots of backups, or a RAID 5 group will save your ass over time.

you write –
Has anyone edited for a considerable period of time (6-10 months) using this type of sharing?
REPLY – yes, it works.

you write –
Is there a software program that I could use to serve a traffic cop and mitigate this risk?
REPLY – not that I am aware of. Well, that is a lie. Bigger SAN systems have all kinds of file permissions management (like Apple XSAN), but you have no money, so just behave yourself, and you will be fine. No sloppy workflow and file management.

I ultimately need (over the course of the next year) about 1.5 stations. If a “proper” share like Walter’s is going to cost 4 or 5 grand, I’d rather set up a duplicate system using Firewires for 1.5k (in storage cost since I have the IMAC already) and have the pleasure of one totally redundant storage setup in case my primary RAID goes fuzzy – this would kill the backup and second station birds with one stone.
REPLY – Firewire drives will die. What are you using – Lacie – this is not a “backup” drive or a redundant drive. This is an accident waiting to happen. Buy quality disk drive arrays.

And in terms of the actual primary RAID purchase, if anyone’s willing to chime in on the CalDigit Element vs. HD One vs. HD Pro vs. G-RAID Fibre Channel vs. EVO 2K Expando vs. ??? debate that would be a huge help

REPLY – Cal Digit – great. G-Tech, very good. Maxx EVO – great.
Lacie – PIECE OF GARBAGE. There are other good brands too like Sonnet and Dulce Systems. Anyone that considers going to a place like Best Buy to buy Lacie FW drives for their professional projects (including student films and wedding videos) is a moron, and should not be allowed to work in a professional industry (even if you are doing a student film). If you value your work, your art, your creativity, you must use GOOD PRODUCTS, not the cheapest convenient crap you can get your hands on.

Any feedback on best brands of 2TB firewires? G-Raid 3? Cal Digit VR? Other brands? In terms of buying the best and most reliable I am happy to pay up here.

REPLY – there are no 2TB drives. The Western Digital 2TB is 5400 RPM and currently too slow. 2TB products are acutally two 1 terabyte drives in a box. You just mentioned two good brands. Most of the brands advertised on Creative Cow are very good companies.

Bob Zelin

[Jim Bruce]

Hi Bob,

Thanks again for another great reply…

And I couldn’t agree with you more about LACIE generally – a mistake made many years ago which I am trying to avoid by talking brands/details with you now…

A couple more questions:

I will be buying bus-powered FW’s. I know we both consider LACIE a scourge from hell but I have a LACIE Rugged that I have had for several years with zero problems and it has taken a lot of jostling.

1) Have you had particular problems with that model? Would you recommend a more durable/droppable brand of bus-powered FW?

2) The CalDigit Element has a big price advantage. Any reason to go more pricey for 6TB of RAID 5?

My own trauma-inducing 2005 LACIE experience (it sounds like you had one as well) involved their early multiple-drives-inside-one-enclosure 1TB models.

3) What is the largest “single” power-supplied FW drive available? 1 TB? 1.5TB? Is having just 1 drive vs. 2 in the box a safer proposition? Less safe?

4) How about going RAID 1 with a Cal Digit VR 2 or 3 TB?

5) Is there a specific brand/model of FW in the 1 TB range that you feel is most reliable?

I am looking for the best possible (vs. cheapest in the case of my media share) option and would love to get your opinion on that.

I do not want to go cheap on firewires and am doing everything I can to research the most reliable drive, and I will still make sure I have redundancy until I have them on the RAID 5, at which point I will burn camera masters to BluRay and keep 1 set of firewires for maximum disaster insurance.

Your thoughts and advice (and occasional abuse!) are much appreciated!

Jim

[Jim Bruce]

Hi Neil,

Thanks very much for your advice. I think I’m gunshy about corruption from early years with AVID sharing and unexplainable weirdness coming about even with a dedicated server and program managing access to the media.

So I’m very glad to hear you’ve had success with your approach which is most certainly what I’ll try.

Just a few questions because I’m an editor, not a networker (if it’s not obvious already):

Any particular specs to have or not have with the switch?

Is this the type of thing you’re referring to?

https://www.netgear.com/Products/Switches/DesktopSwitches/GS608.aspx

Do you plug your internet router ethernet cable into the switch as well or keep that separate?

Do you mount and unmount the drives using Disk Utility or some other program?

If, for example, the Mac Pro is connected to the RAID, does the Mac Pro have to be on for the IMAC to see the RAID or since the RAID is on would it see it even if the Mac Pro is asleep or off?

If the Mac Pro needs to be on, will the IMAC being used keep the Mac Pro from switching in to “sleep” mode? Would the IMAC have all its media thrown offline in that event the Mac Pro does go to “sleep”?

Are there any other do’s or don’ts besides don’t write to files (I.e. File Name, Reel # etc) while both computers are in FCP?)

Do you have any tricks/things to do or NOT to do for sharing FCP projects back and forth?

I apologize for what may seem absurd or simplistic questions. My background is either working on big projects with AVID Unity where I don’t have to deal with the nitty gritties or much smaller projects with firewires and sneaker sharing and no networking at all.

So I have a lot to learn.

Your help is much appreciated, even more so knowing you’re halfway round the world.

Cheers,

Jim

[Ryan Sarnowski]

Bob – I am looking for a similar solution when I came across this. I am wondering what sort of ethernet hardware or software would be needed to make it possible for two iMac’s to be connected to the MacPro and RAIDs?