Ethernet SAN success story

[Bob Zelin]

I read your article. Everyone thinks that I know a lot about all of this SAN stuff, but I don’t. After dealing with Small Tree, I realized that setting up shared storage was cheap and easy. It also proves once again that SUPPORT from a great company like Small Tree is EVERYTHING. These guys that say “you can get a Netgear switch cheaper” think that saving a few hundred bucks on the switch, and then suffering for 2 – 3 weeks getting it setup correctly is a savings.

I am glad it worked for you. We just got 5 FCP systems working for Electronic Arts in Maitland Florida using this exact configuration.
No one can believe how cheap it is to do this !

I would like to ask you a question off line. If you could email me privately, I would appreciate it.

Bob Zelin
maxavid@cfl.rr.com

[Chris Blair]

Bob Zelin: We just got 5 FCP systems working for Electronic Arts in Maitland Florida using this exact configuration. No one can believe how cheap it is to do this !

I came to the conclusion while I researched systems last year that the SAN/NAS industry operates like car repair or internet companies. They’re whole gig (pardon the pun) is based on making their products and setup sound as complex and technical as possible.

What we found is that it’s neither. Certainly you have to know and understand networking AND the requirements of video to build a working SAN or NAS setup. But if you have the time, intellect, and patience, it can be done. I was convinced I could’ve built a working system, but as a facility owner and a working director and editor, I just didn’t have the time to do the research on the components nor the configuration.

Our friend Bob has done a lot of that for us! All hail Bob!

Chris Blair
Magnetic Image, Inc.
Evansville, IN
http://www.videomi.com

[Eric Hansen]

ditto on Small Tree. i was having trouble updating the firmware on the EdgeCore switch so i could enable Flow Control, and Chris Duffy from Small Tree tech support spent an hour and a half with me over screen sharing in iChat, showing me how to telnet into the switch from Terminal and run all the updates and commands. that guy’s a ninja. i foresee always using Small Tree, just so i dont have to learn a whole new language with these switches. and hour and a half with Chris saved me probably days of learning the correct commands and such. since fibre switches for this size installation start at $5k, my client didnt even bat an eye at the $1800 for the ethernet switch. they are very happy so far.

i have a client whose 4 seat Xsan is approaching 3.5 years old. there’s not a lot of money to put towards an update, so i’m going to push a combo of fibre and ethernet. basically just have 2 computers connected to the Xsan – the online edit system and a main ethernet server serving all the edit systems. i have heard about PCIe switches that work the same as as fibre, but much faster. i have a feeling this is the direction that medium-size post houses will go as high quality video compression gets better. super high speed shared storage for one or 2 computers, and the main ethernet server. the editors work in DVCPRO HD or ProRes, then the online editor and maybe a GFX/VFX computer can work at uncompressed.

e

Eric Hansen, The Audio Visual Plumber – http://www.avplumber.com

[Bob Zelin]

Eric writes –
so i could enable Flow Control

REPLY –
what made you think about enabling flow control? Were you having bandwidth issues ? Are you now going to normally enable flow control in future installations like this ?

Bob zelin

[Eric Hansen]

i was having intermittent drop frames during capture, but no playback problems. so bandwidth wasnt a problem, but latency was. Small Tree recommends turning on Flow Control on the switch as standard procedure. their ethernet cards have flow control on by default. the issue with the switch is that you can’t do it with the shipping firmware, and after you update the firmware to the current version, you can’t enable FC in the GUI. you have to use telnet. thus is why i had to spend 1.5 hours with Chris to get it going. right now Small Tree modifies the setup of every EdgeCore switch they ship to make it easier to log into it the first time via IP address to get to the GUI (the switches can’t do this by default). the next time i order a switch i will ask them if they can enable FC before shipping. the drop frames went away after that, so i consider this required for these switches.

another interesting note. if the client systems are on Tiger, Small Tree has a list of configuration settings to get the max throughput for video over ethernet. if they are on Leopard, then these settings have already been made by Apple in the OS. all the client systems in this install are Tiger. so if i have future speed or drop frame issues, the next step is to update to Leopard. i’m not going to waste my time tweaking settings files when the client has already agreed to pay for the Leopard upgrade.

e

Eric Hansen, The Audio Visual Plumber – http://www.avplumber.com

[Shane Sokolosky]

It’s good to see people providing solutions like this for the smaller setups, some places really have the need to share and don’t need a super high bandwidth SAN like you said. Way to go guys!

P.S. You guys ever try to do this same thing without metaLAN? It’s possible with just a server (Mac or PC)… I’m sure, so what are the benefits of having MetaLAN compared to just a server with NFS, SMB, or even AFP running? Have you guys had a chance to test it both ways?
(I once setup a 4 seat NAS for editing HDV, 2 ethernet ports truncated from the server (OS X server, Tiger at the time) to a netgear switch and one cable to each of the 4 stations, each station got 40MB/s, enough to edit HDV all day long and that using AFP on OS X server (which is probably slower than the alternatives).

I haven’t had much time to see what difference MetaLAN makes, any ideas?

Shane Sokolosky

Consultant / Systems Engineer

XSAN for Video Apple Certified Technician
Apple Consultants Network – Storage Area Networks
Apple Developer Connection

SANtech.TV
Office: 714-639-3767
Mobile: 714-599-1611

shanesky@santech.tv
https://www.SANtech.tv

[Eric Hansen]

hey shane

i should have mentioned in the article, but this install is all based on AFP. no MetaLAN. a little while back a COW member posted that their Ethernet SAN was running slower under MetaLAN than just AFP. so i contacted MetaLAN and asked them about the install i was about to do. they said it was too small to benefit from MetaLAN. this is a guess and i could be totally wrong, but i think MetaLAN dynamically changes the priority tags on ethernet packets based on if your capturing (highest priority), editing (medium priority), or transferring files (low priority). basically balancing the bandwidth on the system and giving it to people who need it. it makes sense that you would need something like this in larger installations, but not in smaller ones. when i spoke with Apace systems about their all-in-one solution, they also had a similar application that had to be installed on every system that would be used for capturing.

i don’t know what the magic number is for how big your system needs to be to need MetaLAN. maybe when your total bandwidth needs exceed your server’s available bandwidth? with the installation i just did, i think they would have to triple their number of edit systems before the server would even start to sweat.

e

Eric Hansen, The Audio Visual Plumber – http://www.avplumber.com

[Bob Zelin]

I am doing exactly what Eric Hansen is doing, and exactly what you are suggesting. Just Apple File Sharing, using AFP. Nothing else.
It’s easy, it works, there is no “education process”. Just a couple of minor “tweeks” from Small Tree that are done in the Terminal.

Bob Zelin

[Eric Hansen]

i ended up upgrading all the client systems to Leopard. everything now is super stable. with Tiger and Flowcontrol off, i was getting drop frames, errors with Final Cut writing quicktime files, and random system crashes. with flowcontrol on, and the clients on Leopard – to the user, everything feels like Xsan. very responsive, stable and fast. when i left the facility last night, i had one system copying from its direct-attached Xserve RAID to the ethernet SAN, 2 systems capturing DVCPRO HD to the SAN, and the fourth system backing up via Time Machine to a Firewire drive attached to the SAN server. the server was seeing an average of 250MB/s over ethernet and everything was hunky dory.

e

Eric Hansen, The Audio Visual Plumber – http://www.avplumber.com

[Sebastien Gariel]

Hi there,

First of all, a great great thanks to Bob Zelin from a french independant IT worker.
I discovered that my networking and video knowledge associated to your wonderful idea could be useful to small postproduction teams.
We are now on day 2 running and testing this home made SAN and it works fine. I spent time carefully choosing the parts. And it appeared that you could find good reliable pieces for very low price. Really.
I think i’ll keep MetaLAN soft as i plan to setup 2 AVID workstations editing at the same time, updating the same project.
We’ll have them running by tomorrow.
I’ll tell you how it works.

Thanks again
Sebastien