Not exactly an affordable solution

[Chris Blair]

Alec,

We own an Apace vStor so I’m not sure I’m a good person to ask about the advantages of MetaLan vs. Apace.

We added a new switch when we installed the vStor so if you add the cost of the 48 port GigE managed switch ($800) plus installation and cabling by our IT company ($1500), plus new ethernet cabling ($250), the cost of the vStor was more in the $11,750 range. That doesn’t take into account the cost of 2 LaCie NAS drives we installed for project storage and backup at a cost of about $3000. There were also things like having to get a new rack to hold the vStor (it’s huge and very long) and sound dampening foam to reduce noise (it sounds like a 747), which added about $250 more. So we spent at least $15,000 on upgrading to shared storage after it was all said and done.

When we were pricing hardware to couple with MetaLan, we talked to Aberdeen Systems and Winchester Systems. Tiger Technology recommended another company, Midwest Communications or something like that. We could never get anyone from there to call us back or respond to emails.

The NAS hardware from Aberdeen and Winchester was in the $4500 range. It came with the Linux OS and 4 GigE ports. This was basically the same hardware as what Apace sells minus all the little software tweaks and utilities they provide. Had we added MetaLan software, I beleive it was going to add another $2250 for our 4 edit stations. We were also going to have to add a dedicated server at a cost of about $1000 to handle the Metadata that Tiger’s software requires to keep track of volumes, users and basically act as a traffic cop for all the data coming in and out. That all adds up to $7750…about $1500 less than the vStor. So had we gone with a DIY NAS box and MetaLan, it would’ve cost us about $13,500, because we still would’ve needed the new switch and the backup LaCie drives.

For me it wasn’t worth saving $1500 to build a DIY system. The guys at Tiger are great, but unless there was a performance advantage or a bigger cost savings, I just couldn’t justify the added time I would’ve spent installing, configuring and learning their way of doing things.

With Apace…for us at least, installation was relatively easy with help from them every step of the way. It worked exactly as advertised after installation. And in 6 months of constant use has been rock-solid so far.

Maybe as you go up in size in terms of storage there are bigger cost savings, but at the 4TB level, it just didn’t make sense to us to try to save that amount of money. Apace also guaranteed in writing that their box would work with our editing system (Leitch VelocityQ). If it didn’t, they’d take it back and refund us everything but shipping.

Hope that helps.

Chris Blair
Magnetic Image, Inc.
Evansville, IN
http://www.videomi.com

[Alec Gitelman]

Thanks Chris,

this very much answers my question. as long as there is no considerable cost or performance benefit to a diy system i would prefer to go with a turnkey solution, considering very limited IT knowledge in our office. guys at Apace even offered to build us a custom 8Tb v2100 for a few $K more than the standard 6Tb. moreover, your answer is a good guide as to how we should approach the SAN installation.

one more question, what made you decide to go with lacie network drives rather than Apace eStor? also, what would be your advice on what to do with a few dozen external drives that we’ll have sitting around after the installation? can some of those be set up for backup or is it not possible?

Alec.

[Alec Gitelman]

to avoid confusion, i mean setting up a firewire array on one of our systems to run daily/weekly backups. i’ll have a hard time explaining to my bosses the need for extra investment in network drives.

[Chris Blair]

We basically bought them due to budget limits.

If I had to do it again I wouldn’t choose the LaCie’s for project storage, but they’re fine for backup.

We bought the rackmount 2TB and 4TB NAS appliances. I read good reviews of them on several IT sites, but after buying and installing, we found that we really should’ve just put our project files on a volume on the vStor. That’s what the guys at Apace recommended, but we were trying to keep project files separate from video and audio, thinking this would ensure better performance for capturing & playback.

After editing with it for 6 months, it would’ve worked fine both in terms of space and speed (we’ve tested it) just using the vStor for projects. In fact, I think you have to use the vStor for project storage with some editing systems…like Final Cut and Avid. I could be wrong on that though so I’d check.

The LaCie’s we bought are just spanned drives. They’re not configured in a raid at all. So you get NO speed bump using them. They also have very basic GigE cards, which DO NOT support jumbo frames, which can give you quite a significant bump in speed and reliability if available.

We do get about 22-25MB/sec sustained read/write, which is pretty good across even a GigE network for a product at their price point. And the 4TB works fine for backing up, which is done overnight each night using Second Copy software.

But the weak point in our system is the LaCie 2TB drive that’s used for shared project storage. It works fine, but we get occasionally pauses while trimming or cutting clips that we THINK is caused by the drive. It doesn’t ever affect timeline playback though…just some random slowdowns doing basic stuff in the timeline.

So if I had to do it again, I’d just forgo the 2TB LaCie and use the vStor for project, audio and video storage. Then just back it all up on the 4TB LaCie. Remember, the vStor is raid5, so if 1 drive fails, it will rebuild the data. So we have that as protection, as well as our nightly backup onto our 4TB LaCie.

As for the direct attached external drives, we have a bunch of them, most still attached to the systems. We don’t use them at all. We considered using them for backup, but none was big enough to back up any of the volumes we created on the vStor. We’re keeping them for now as a “just in case” something happens to the vStor. But in reality, I cannot see using them for much of anything except maybe a graphics/compositing station or something like that.

We’ll probably end up selling some of them on Ebay.

Hope that helps.

Chris Blair
Magnetic Image, Inc.
Evansville, IN
http://www.videomi.com

[Chris Blair]

Well I’d tell your bosses:

Do you want me to spend $2500 now for a backup solution that will work, is reliable, has a warranty, and will allow for us to be back up and editing in a half-day?

Or do you want me to take pieces and parts, try to make them work, spend a couple hours a week fussing with backups having to offload, archive files all the time etc. Then when the system crashes, spend 2 days finding and restoring files from a dozen drives and offloaded DVDs or portable drives?

Your backups are critical. And I’d recommend at the very least new drives. You can get 4-6 TB NAS arrays pretty cheap nowadays. I wish we’d gotten a backup drive that was Raid5 so it also had the added benefit of being able to rebuild data if one drive failed…but I was too stupid to know the difference 10-12 months ago.

I wouldn’t skimp when it comes to backup though.

Chris Blair
Magnetic Image, Inc.
Evansville, IN
http://www.videomi.com

[Gary Gowman]

why not share the info with all the readers of the forum?

[Tyler Jones]

Hey Bob… If you’re still reading this, I have everything setup now, but I’m only getting about 23 MB/sec transfer, instead of 70. Our setup is like this:

We’ve got a Windows XP machine acting as the shared storage. It has 2 raid boxes each with 4 1TB drives in them. The two boxes are all configured together as one big 7TB drive in a raid 5 configuration. We have an intel 4 port gigabit card and the ports are teamed together acting as 1 4GB/sec port. This teamed port is connected to a netgear 24 port managed gigabit switch with the link aggregation setup on the 4 ports. My osX machine is connected directly to the netgear switch with a cat 5e cable and I’ve got jumbo packets (9000) turned on and a static IP address. Also, I’ve made sure jumbo packet support is on in both the switch and the Windows XP machine. What am I leaving out? Why am I not getting the speeds I’m supposed to be getting?

[Bob Zelin]

Tyler –
you are the PERFECT ad for buying a turnkey solution (like Apace Systems).

I told you exactly what to purchase – the exact model # of Small Tree ethernet card, the exact # of the Small Tree managed switch. I am very familiar with these products, and know exactly what to do. I know the “bug” in the MAC OS-X 10.5 that needs to be set with a SUDO command on the “server” to enable jumbo frames.

In contrast, I know NOTHING about PC servers. I know nothing about the Netgear switch (I could never get anyone on the phone from Netgear). I know nothing about using a PC server with MAC clients. I cannot assist you, and I cannot help you troubleshoot. With the Small Tree switch, there are multiple menus – I know exactly what menus to go into, to check if things are wrong. I know exactly what commands to type in to check for speeds. With the Netgear, and with a PC server (running Win XP) – I can’t even guess what to do.

You are now proving that a “do it yourself” solution limits you to suffering if something goes wrong. There is no one to call. Is your managed Netgear switch setup correctly – you are at the mercy of your IT people – do they know how to do this correctly ? I can tell you that I have ONE CLIENT that I told about this system, and they had an IT expert, and he did not need my assistance. It was on an Apple Xserve, and he “knew it all”. And nothing worked. They finally called me in, and this guy had setup the Small Tree switch link aggregation incorrectly. So, even if I was there with you right now, I DO NOT KNOW the correct settings for the Netgear switch. I certainly do not know how to link aggregate the Intel 4 port ethernet card, and create a bond0 aggregate port. And I don’t know how to enable JUMBO frames on a Win XP PC server.

If you decide to do this with the simple inexpensive equipment that I mentioned to you (a MAC Pro running OS-X 10.5, and the Small Tree hardware), I can help you with this. But with the Intel 4 port card, Netgear switch, and WinXP server (in a MAC client enviornment) – you are on your own, or at the mercy of your IT department.

This is why turnkey solutions (like Apace) are much better for some people.

Bob Zelin

[Tyler Jones]

Hey Bob! Thanks for responding! If I had been the one purchasing the equipment, I definitely would have gone with only suggested hardware. However, I just work here, so I have to try and make due with what I’m given. So, being in the situation I’m in, can you tell me a few things about how you configure the setup you’re familiar with? My specific questions are:

1. With the Small Tree switch, do Jumbo Frames have to enabled on every single port on the switch? Or just the ports connected to the client machines and the aggregated ports don’t need it?
2. Same thing with the ports on the bond0 aggregate port on the host computer (which we have successfully setup). Do each of the ports need to have jumbo frames enabled? Or just the big aggregate port.
3. I don’t know how to test the speed of the transfer rate of the raid drive from inside the host computer itself. Could it be possible that the transfer rate of this drive is the bottleneck I’m experiencing?

Like I’ve said everything “works”, I’m just not getting the 70MB/sec speed we’re wanting… Thanks in advance for any help you can give. AND GOD BLESS YOU BOB ZELIN! GOD BLESS YOU!

[Bob Zelin]

Tyler writes –

If I had been the one purchasing the equipment, I definitely would have gone with only suggested hardware. However, I just work here, so I have to try and make due with what I’m given.

REPLY –
based on the equipment that has been purchased, I will make several assumptions. You are in a “PC company” with “PC IT guys” that know Windows XP, and have Microsoft Network certification. I will assume that they think they know everything. NO ONE KNOWS EVERYTHING. Without any further background, I bet you money that your IT staff has no clue of how to properly link aggregate the Netgear switch, or link the 4 port Intel Ethernet card, and just “poked around and figured it out”. I bet you money that NO ONE ever called Netgear to get the correct way of doing this. To these guys (my assumption), they think that if they clunk around on the “Trunking” Menu, they can get it to work. Without support, you get NOTHING, and SOMEONE has to call Netgear. I will discuss Small Tree further, but without support, you get nothing. No one just “figures this out” by poking around.

you write –

1. With the Small Tree switch, do Jumbo Frames have to enabled on every single port on the switch? Or just the ports connected to the client machines and the aggregated ports don’t need it?

REPLY – The Small Tree Switch and the Netgear switch (GSM7324) both support jumbo frames, but it does not have to be enabled. On MAC systems, on the client end, you enable jumbo frames, by going into System Preferences/Network/Advanced Settings, click on the last tab (in OS-X 10.5) and set the frame size to CUSTOM (MTU 9000). This is jumbo frames. It may not “stick”, so you check your jumbo frame size by going into the Terminal, and typing IFCONFIG, and you will see the MTU size for EN0 and EN1 (the 2 ethernet ports), and one (EN1 if you are using ethernet port 2) will report back MTU9000. How do you do this on a PC – I DON’T KNOW.
On the server, once the link aggregate is created with the 4 port card (bond0), you CANNOT set Jumbo frames, because there is a bug in the OS-X 10.5 operating system for the MAC shell. However it does work, but it has to be done in UNIX in the Terminal. Its a simple single line SUDO command that sets the aggregated port to JUMBO frames (MTU9000). You also check this (once you set it) with the IFCONFIG command, which will report back bond0 MTU9000.
How do you do this with the Intel card on a PC with Windows XP – I DONT KNOW.

you write –

2. Same thing with the ports on the bond0 aggregate port on the host computer (which we have successfully setup). Do each of the ports need to have jumbo frames enabled? Or just the big aggregate port.

REPLY – the bond0 aggregate port must have the SAME MTU as the client (MTU9000). It cannot be a different size. On the MAC, this is done thru a UNIX terminal command. You then must disable the 4 individual ports on the card, so the computer only “sees” the bond0 link aggregate. I have NO IDEA of how to do this on a PC with Windows XP and the Intel card. Who at Intel did you speak to, to get this information ?

you write –

3. I don’t know how to test the speed of the transfer rate of the raid drive from inside the host computer itself. Could it be possible that the transfer rate of this drive is the bottleneck I’m experiencing?

Reply – for MACs, everyone uses AJA Kona System Test. For PC’s, I use Steel Bytes HDTest. I have NO IDEA of how to setup the Netgear switch, Intel 4 port card, or your server XP computer. I bet that you or no one on your staff ever even spoke to Netgear or Intel on the proper configuration of this.

you write –
Like I’ve said everything “works”, I’m just not getting the 70MB/sec speed we’re wanting

REPLY – with even an improper setup system, you will still be able to get some thruput, but not enough for uncompressed SD, or ProRes422HQ. I have no idea of how the MAC’s can see an NTFS formated drive array in your server. I have no idea of what format you are using to communicate (SMB ? – you sure are not using AFP) !

IN SUMMARY, this is WAY too complicated, and detaled for most humans (including me). It should be SIMPLE AND EASY. This is why you either use a simple MAC solution (like I outlined), or you buy a turnkey solution from a company like Apace systems, WHO CAN HELP YOU.

Without support, you have NOTHING. Until you tell me WHO you spoke to at Netgear to confirm your proper configuration, I have nothing else to say. If NO ONE at your company spoke to Netgear about the link aggregation setup, then you guys are just all crazy, and wasting your bosses money and time.

Bob Zelin