Working over a SAN

[Sean Oneil]

[Anne Leaf] “There are two computers networked – One is the host directly connected to the raid via ethernet, the other is connected to the host via ethernet.”

I have to ask, why on earth do you have it set up like that??? The purpose of a SAN is so all machines are connected directly to the AoE storage and you use MetaSAN or a similar app to allow sharing of block-level storage. With the setup you just described it is utterly pointless to use AoE storage since it’s not even a SAN. Your AoE storage is being used as a direct attached RAID and you’re sharing it over regular file sharing. If that’s all you need you could just connect an eSATA or SAS RAID to the host and it will be a lot faster and a lot cheaper. But if you want to keep the AoE storage you should Google MetaSAN and call those guys asap!

[Anne Leaf] “My question is where should the Final Cut project live? Does each editor have her own project file locally on their machine, or is there one master project file that they both work from that lives on the server? This later one doesn’t seem like it could possibly be right, but how do they share sequences if it’s the former.”

You can’t have multiple editors working in the same project at the same time. Avid Unity can, but Final Cut doesn’t work like that unfortunately. You can keep all your projects on shared storage for convenience purposes if you like and either editor can access it – they just can’t work with it together simultaneously and only one can save the project (the other can do a “Save As”).

Sean

[Chris Blair]

Whoa…she said right off she’s a newbie. Lighten up a little. I have a colleague that uses shared storage with several Final Cut workstations and when they have multiple editors working on the same project they just give each project a letter or number designation at the end of the project name to identify each segment.

So if they’re editing on a project called “Jazz Festival.” The main editor puts a _1 at the end of his project name. The graphics/compositing guy puts a _2 at the end of his file name (Jazz Festival_2) and so on.

When they want to unify everything, they just import (or whatever you do on Final Cut) all the relevant timelines. Since they use shared storage, it easy to move projects back and forth.

As for how you have your system setup….if you’re using a GigE switch, the setup as described could work, since the host machine could theoretically be looked at as the server, and the 2nd machine a client. We use an Apace vStor, and it’s basically a server (with an embedded Linux OS), with a SATA II, 8 disk raid. It’s connected to the switch, and our 4 workstations are cabled to the switch, then into the 4 GigE ports on the back of the vStor (which are teamed). We get nearly 100MB/sustained with this setup across all 4 client workstations…which is actually higher throughput than we got with direct attached, 15K four and six drive SCSI raids.

So depending on how she has everything configured, her description could actually work.

Chris Blair
Magnetic Image, Inc.
Evansville, IN
http://www.videomi.com

Chris Blair
Magnetic Image, Inc.
Evansville, IN
http://www.videomi.com

[Sean Oneil]

Wasn’t trying to be harsh, just conveying the urgency of the situation. I wasn’t suggesting Anne should somehow magically know better herself. But whoever set this up or told her to set it up like that should.

[Chris Blair] “As for how you have your system setup….if you’re using a GigE switch, the setup as described could work, since the host machine could theoretically be looked at as the server, and the 2nd machine a client. We use an Apace vStor, and it’s basically a server (with an embedded Linux OS), with a SATA II, 8 disk raid. It’s connected to the switch, and our 4 workstations are cabled to the switch, then into the 4 GigE ports on the back of the vStor (which are teamed). We get nearly 100MB/sustained with this setup across all 4 client workstations…which is actually higher throughput than we got with direct attached, 15K four and six drive SCSI raids.”

That’s not what she described. With your system the back-end is SATA. With hers the back-end is AoE. Imagine taking your Vstor and connecting it to only one computer as a direct attached device. You then have all the other computers connect through that host computer rather than the Vstor itself. It would be a completely pointless extra layer in there. Your Vstor would be serving no purpose whatsoever – you could simply connect the SATA disk directly to the the host computer instead and it would achieve the same exact thing. But that’s not what you’re doing. Instead, all the editing machines speak directly to the vStor. Whether through a switch or not doesn’t matter, your vStor is a shared storage device that all machines access. Anne’s setup is totally different.

Her setup is what a non-video IT person might set up. It is the typical non-video way to set up AoE or iSCSI. Think of a server hosting a web-site or a database or something. The purpose of SAN storage in this case is so that different servers, maybe at different locations, can access it at different times using IP routing or switching. Imagine the server for creativecow.net is in one city and it goes down suddenly. But all the forum messages are stored on an AoE storage device, not the actual server. So when that server goes down, a backup server takes over and it can connect to that same AoE storage automatically and take over like nothing happened. The advantage in this situation is that the storage can be in a different location than the server and the backup server – and that each backup server doesn’t have to have duplicate copies of all the data (like our messages here).

Very complex stuff, and it is not how you use a SAN for video sharing. A SAN in a traditional IT environment is for one computer and one computer only to access the storage at a given time. For a video SAN you need all computers accessing the storage at the same time. Hence the entire purpose of it. And to do wthis with AoE storage you need MetaSAN. Otherwise using AoE is pointless. If all you want to do is share direct-attached storage over AFP file sharing, you could achieve the same thing with a LaCie disk.

Sean

[Ohannis Dikramanjian]

Hi Anne, 1st what type of project are you trying to work with? DV, HDV, HD?. Keep in mind that NAS has limitation of what it can deliver bandwidth wise.
To answer your question, make sure to have the render folders directed to your local drive, as far as the project file goes you have leave it on the NAS drive, but yet again the footage type is that determines the performance…. Hope I was helpful.

-Ohannis D.
NLESystems Inc.
http://www.nlesystems.com
818-242-4996 x112

[Chris Blair]

I would suspect that Anne’s description of how they have it set up isn’t exactly correct. I assumed from reading her post that the host was connected to the raid via SATA…not via ethernet….I missed that little detail.

So you’re right that her description doesn’t sound correct. But her question was about how to set up projects. So I was assuming the hardware/software setup is actually working and that they have whatever SAN software they need in place to handle the file sharing issues.

Anne…care to enlighten us? Does your setup work ok in terms of being able to edit on both machines?

Chris Blair
Magnetic Image, Inc.
Evansville, IN
http://www.videomi.com

[Matt Geier]

Hey all,

I wanted to pop in here and enlighten folks with some information. about this solution. When I read all of these posts, you’re all correct with your thinking and implementation ideas, somewhat. There’s more then one way to skin this cat, after all.

Anne’s configuration is simply a couple of Mac Pro systems. Each of them is a Final Cut workstation, however, one is also “acting” as the server.

Why an AOE RAID and not some other “Direct Attached Drive” per each system you ask? Simply because Anne wanted “Centralized Shared Storage” for her Final Cut users. FireWire drives are limited with bandwidth, and storage capacity. When you have more then one person beating on a firewire drive over a network, the bandwidth becomes really poor.

Firewire 800 will allow you 786.432 Mbit/s @ full-duplex
(98MB/sec Total Avail Bandwidth @ Top Perfomance)

Small Tree’s 4TB AOE solution was her choice to use to address that concern. With this solution, Small Tree was able to give 200MB/sec of Total Network Bandwidth (RAID Limit / 2 Ethernet Ports).

The alternative to hooking this up to the Client/Server, would be hooking the AOE RAID to a Ethernet Switch (like a traditional Fibre Channel environment (only Ethernet)), issue a couple of AoE Client Licenses, and then make some storage volumes to mount at each workstation.

One downfall of AoE is that only (Volume A) can be mounted by just one other location at one time. Hence, the reason for the server, because now, you have that one location, but by accessing the volume via the server ALL of your editing systems coming into the server can see the volume eliminating the need for multiple client licenses and multiple mounting points etc.

For purposes of which system the storage attaches to; typically it would be an Xserve or something just playing and acting in the server role. However, in Anne’s case, buying a server on top of storage wasn’t an option due to budget. So we beefed up the Mac Pro she already had with some additional memory, hooked in a Dual Port Gigabit card, set up Link Aggregation and Jumbo Frames, and AFP Shared the storage. Now she has a Shared Storage Solution capable of 200MB/sec (Dual Port Ethernet) of bandwidth which both of her clients can access at the same time. If she adds additional clients, they just need hooked to the network like normal anyway.

With regard to using MetaSAN/MetaLan or other “SAN” software, you only need to use that software in an environment where more then one user will be working on the same bits of project that other users are also at that time. It’s playing traffic cop, making sure were not reading and writing over each other, which would ultimately corrupt the data.

In this solution, if user A is working on project A, and user B is working on project B, those projects can both live on the same part of the RAID and not be affected (other then by bandwidth back and forth on the network). – NO SOFTWARE REQUIRED.

In the alternative solution, if user A is working on project A, and user B wants to access project A at that time, then yes. – SAN SOFTWARE NEEDED.

I hope that helps,

Matt.

Matt G.

[Anne Martin]

Thank you everyone for all the responses. It is enlightening.

To answer the easiest question first, our footage is primarily HDV with a little bit of DVCPro HD.

Matt’s description of the set up is far better than anything I could probably write. We were looking for a very low budget solution (under 5k) for two editors to share footage and work on the same project. We’ve had some problems with the set up – mostly due to my lack of experience with networking and some bugs in 10.5.6 – but now that the whole thing is set up properly it does seem to be working well, for our needs. I have done things just with file sharing to access firewire drives on shared computers and I do not think it is as fast.

I think for our purposes, two project files will work and we can just pull sequences into a master project when we’re ready to combine scenes.

Another question though – If both editors want to do something with the same clip… say both end up editing parts of that clip into their sequence… would that cause problems?

[Matt Geier]

[Anne Barliant] “Another question though – If both editors want to do something with the same clip… say both end up editing parts of that clip into their sequence… would that cause problems?”

Yes, that’s when you want to work on the same bits of the same project on the same part of the disk. In order to make that work sucsessfully, you need some traffic cop software. Be it Dave (Thursby SMB SAN Software), FibreJet, MetaLAN, etc ….

I’m not sure if you’re talking about working on different sections of the same project and what affect that would have…..

I don’t know specifics about those software packages and their support for AoE Targets/Devices. We (Small Tree) are looking at offering a solution in the future for purposes just like you describe.

Thanks,

Matt G.