SAN for VFX systems

[Jordan Woods]

I’m not to sure about that setup… but I can tell you, when you are trying to step through frames that are 12MB+ in size across gigE, that will not be fun. I used to do a lot of rotoscoping back in the day, and doing that cheaply was not fun at all.

I believe what you mean by “editing SAN” is an actual fibre SAN. There is not really a difference between editing SANs and VFX SANs. I’m not sure what you think are small IO packets are when you are talking about 2k or 4k frames… small IO packets to me would mean that you need to access a lot of Word documents or text files, at which time a “database SAN” would work great.

Bigger question- If you are working in 2k or 4k uncompressed why would you want to do an Ethernet based solution? just go with a standard fibre solution. You still need to pass IO fairly quickly. You may not need to stream it at 24fps, but at least when you step the frames you aren’t waiting a minute for the update.

-jw

[Neil Sadwelkar]

Actually what I referred to as ‘editing SAN’ is the kind of SAN over GigE that the link referred to. Not XSan or some such FC SAN. I know about XSan and have used it very successfully in an editing share situation.

Anyway, I’ll take this up with you over at the XSan forum as well.

———————————–
Neil Sadwelkar
neilsadwelkar.blogspot.com
twitter: fcpguru
FCP Editor, Edit systems consultant
Mumbai India

[Bob Zelin]

Hi Neil –
my cheapo solution will not work. Not only can it not handle large file formats (like uncompressed HD, 2K and 4K), my system bogs down with large render files – even with small file formats like ProRes422HQ or DVCProHD.

The way we “fixed” the render problem that we had (which I hate doing, because it makes our simple system more complicated) – is to dedicate two of the link aggregate ethernet ports to the one client that is doing heavy Adobe After Effects rendering. This way, it does not bog down the entire system. We actually create 2 link agg bonds (we have done one system creating 4 link agg bonds, using the 6 port ethernet card from Small Tree, and the 2 internal ethernet ports on the MAC Pro). Then you assign each user to a specific bond IP address. Anyone can share, but it dedicates more bandwidth to a single user that is doing heavy rendering.

Is this a bulls#$% workaround ? – you bet it is. I said this type of solution is inexpensive, not great. (but it works). For the typical shop that has 2 – 6 FCP systems, doing ProRes422HQ, it’s a perfect solution. I always tell people now to render locally (heck – even Mark Raudonis tells his XSAN shop to render locally), and drag your rendered files to the shared volume, so you don’t bog down the server in a 4 hour render. But the “workaround” is to create dedicated bonds with just a couple of ports for the one user that is doing heavy rendering.

With that said, this cheapo system IS NOT GOING TO WORK for your application. You need an “expensive” Fibre solution. Even if you were willing to try 10Gig ethernet (which is still expensive, especially the 10Gig switch), you will only get 180Mb/sec max bandwidth using Apple AFP.

And remember, these bare bones systems have no “metadata server” that can control or save the different .msm mediadatabase files required by AVID systems – so no AVID’s on these inexpensive systems.

Bob Zelin

[Neil Sadwelkar]

The local storage idea is a good one and I can attest it works.

For one 24-seat VFX group of MacPros, we found when they all hit the SAN over GigE (its a Bright SAN) they all fall down. So we devised a workflow around giving all systems 3x1Tb internal drives. They copy relevant frames from SAN locally. Work like mad on Shake. Render out locally too. Then copy back to the SAN.

This creates copies, of course. But copies means crucial data has a back up. And copying in two bursts, rather than zillions of small hits means the network is freed up from ‘small arms fire’.

———————————–
Neil Sadwelkar
neilsadwelkar.blogspot.com
twitter: fcpguru
FCP Editor, Edit systems consultant
Mumbai India

[Christopher Tay]

Hey Neil the man…

Do you have the GigE Expander for your Bright ? But even if you do, it’s only a 4 port GigE connection to your 24 VFX Macs but will bottleneck the pipe. We still pretty much recommends working locally as there are no network bottlenecks to contend with, coz it’s really hard to track who is doing what. And once you throw in the renderfarm, which is most likely to be there, it gets worse.

Running these VFX on FC will be too costly…stick to local drives. Proven and least $$$…but you know that already 🙂

-chrispy

[Matt Geier]

Neil my friend,

I don’t like to bore people with technical jargon so I’ll keep this as simple as I can.

Furthermore, you’ve said it, yourself in the fact that it’s not “real time” which means you don’t really have to follow some of the rules you need to follow in order to get that kind of result you’d be looking for in a Real Time Video Editing environment.

Some additional things to know ..

Backups in their true form, are often run offline, or aside from the daily usage. Back when I was at SGI, we had a storage attach rate that was outstanding. Most of that storage was for backing up daily usage, offline. (high latency intensive… as apposed to low latency intensive for faster disk responses and faster raid controllers, back planes, front planes, etc..)

In an environment where you people doing mission critical work for your business, you just don’t want to have hiccups like bursts of backups occurring during “critical time”. The bottom line being, you’ll often find that its suggested to have your main storage (central storage) separate from your backup storage (offline storage).

Gigabit Ethernet is able to contain 100MB(MegaBytes) of data per second. Using Ethernet or not can be independent from the kind of storage you’ll pick. So establishing what kind of transfer rates and numbers you are looking for, need to be discussed first and foremost to decide where you want to start. Do you need to go 500MB/s or are you okay backing up your information at 90MB/sec .. or 160MB/sec … does it need to take an hour .. or can it take 5…. etc … you have to decide what you want to expect.

Ask yourself questions like what you have now, and what you expect it to be when it’s “improved” in your mind. Set your own expectation before someone else sets one for you and fails because it’s not what you thought it was.

Your idea of doing backup on Gigabit or 10Gb is completely viable. If you can deal with the speed in either case, and/or cost associated differences of Gigabit or 10Gb and then compare to a Fiber Channel NAS/SAN solution.

The likely hood of a Storage RAID being able to use iSCSI or AoE in your case, or even FCoE in the future (FCoE = Fiber Channel over Ethernet) is very possible. I’m trying to tell you that it’s possible to do your backups on Gigabit and/or 10Gb as long as you’ve designed a network that’s able to handle the kind of performance you are looking for.

When you don’t have “real-time” requirements, your most mercy needs to lie in what’s available to be used for bandwidth. At this point, it’s not about how “fast” the data can pass, it’s about how much you can move in a single transfer and do you have the needed disk space etc. The reality is, you still want to create a network that is fast, but can also handle a large amount of information being passed along the wire.

What you’re asking to do is not unreasonable.

You would be benefiting all of us if you layed out a design of your entire network as a whole, and HOW much data you want to back up and over what period of time you’d like it done.

If what you want in a solution can be proven, and accomplished, would you say that the solution has worked, and would you buy it?

Thanks,

Matt G.

[Neil Sadwelkar]

Hey Matt,

How are you doing? Great to hear from you. We’ve corresponded, we’ve spoken on the phone. We’ll probably meet one day if we haven’t already.

Your explanations are always so honest yet so informative. I appreciate that.

Times are not what they used to be. We now go over every expense with a loupe, and then mull it over for a bit. Which is not altogether a bad thing. I guess we are actually getting more value for money now. I wish we’d been so careful when times were a rockin’. Maybe we wouldn’t have gotten into this mess. But that’s another story.

I hear your suggestion of laying down expectations first and then figuring out how to meet them. I plan on studying the problem carefully before laying down requirements. Then, yes sure, if there’s a solution that one can be reasonably assured will work, I’m sure we can go in for it.

———————————–
Neil Sadwelkar
neilsadwelkar.blogspot.com
twitter: fcpguru
FCP Editor, Edit systems consultant
Mumbai India

[Matt Geier]

Neil,

Thanks for the kudos. I’ll consider that the start of our “beer tab” 🙂

I understand these hard times like the rest of the world too.

I think maybe you’ll want to end up giving me a call at some point and I can talk to you about some solutions I know of that are Ethernet based.

I believe you have my contact info already.

Matt G
651-209-6509 x 1