[David Gagne]

One of our integrators recommended them (Midwest Media Group).

We’ve had some challenges with it, and the annual support is ridiculously expensive, and of course necessary. I think in hindsight I maybe would have gone with a traditional SAN… but it was a tough call. Many of the awesome advantages of Isilon we do not take advantage of enough to make it worth it.

Isilons strengths really show up when you start to add more nodes to your cluster. No need to migrate data etc, really fast Infiniband backbone, etc. No need for fiber or metadata controllers is nice as well.

Downside is that you have to mount via NFS mountscripts and permissions get wacky if you’re not careful.

[David Gagne]

In more detail:

XSAN stayed home where it’s safe.

We brought:

Isilon 6000x Cluster:
(4) nodes, each with 2 gigE ports. 8 gigE connections total on Isilon.

(4) XServes with Kona LHi. These are our normal ingest machines. We did put in GT120s which seemed to smooth the playout issues we were having.

(4) Intel Mac Pros with some old Kona LHe we pulled from our broken GVS equipment.

(2) Intel Mac Pros for editors to work on location.

I distributed the clients to connect to differing IPs so that it wouldn’t overload any of the nodes/ports.

Node 1 Port 1: XServe1
Node 1 Port 2: XServe2
Node 2 Port 1: XServe3
Node 2 Port 2: XServe4
Node 3 Port 1: Mac Pro 1,2
Node 3 Port 2: Mac Pro 3,4
Node 4 Port 1: Editor 1
Node 4 Port 2: Editor 2

When monitoring the traffic it was something like:
Node 1: 200Mb/s (25MB/s)
Node 2: 200Mb/s (25MB/s)
Node 3: 400Mb/s (25MB/s)
Node 4: Varying

It was peaking at around 1Gb/s (250MB/s) total throughput.

I know that’s not super high for a system like this, but with no dropped frames over GigE over the course of 4 days and around 10 hours per day, I was pretty happy.

Being a NAS does cause some limitations of course, no live editing, weird permissions, etc… but over all it got the job done.

[David Gagne]

For AFP sharing it’s easy, just use a mac pro to share it out — or you can continue to use XSAN and use StorNext controllers (hp + redhat).

[David Gagne]

Yeah, this leaves me with a bunch of questions…

1. Will they add dual psu? LOM?
2. What does this mean for Mac OS X Server’s future? I hear devs haven’t received Lion Server yet…
3. What does that mean for XSAN’s future? Will we all be on StorNext soon?
4. Maybe they’ll add fiber channel to mac mini?
5. Will vendors like Active Storage create XServe replacements like they did with XRAID?

[David Gagne]

Bob: Yachts. Yachts are allowed to cost a lot of money.

[David Gagne]

I can dig up some stuff I have on them — but really I wouldn’t recommend using them for anything worthwhile. We’ve had 4 out of 5 of ours fail (raid controllers, power supplies, both non-redundant).

I have nothing positive to say about GVS… it’s a frankenstein of a machine…

[David Gagne]

1) Most importantly, am I going to have an issue using a pretty standard PC as server to my more expensive Mac editing stations? I can’t afford to buy a dedicated Mac Pro as a server.

It doesn’t sound like an ideal setup to me… But it also depends on the network — what is your network infrastructure like?

2) Do I need to purchase management software like MetaLAN or something or can I forego that expense? And if I do need some sort of management software, what’s good for both PC and Macs sharing the same storage?

You don’t *have to* depending on what you’re trying to do… You can just reshare it via NFS or SMB but then you’ll take performance hits if your PC is slow. I’d definitely invest a little extra to make sure that PC has enough cpu/memory to handle the load. This setup is more of a NAS, not really a SAN…

3) I’ve already read Bob’s article on a low budget do it yourself SAN solution which is very good. He says there you need the management software. But I’ve also read on this forum that it can work without. What gives?

If I understand right, you’re only directly attaching to one machine — you won’t need management. (Correct me if I’m wrong).

4) Since we’re shooting on the Sony EX, I’m used to using the XDCam codec. Is that codec going to make it over the network? If not, and again this is a newbie question, if I have to use the Apple Pro Res format which I haven’t really used before, how bad is the hit in quality and how hard is it to up rez back to the original if needed?

I’m not familiar with the XDCam codec, but you definitely don’t want to waste time/quality by converting if you don’t have to…Which XDCam codec is it? XDCam EX?

5) Semi-unrelated bonus question… Can I use the PCs in a render farm with my Macs or am I just a lunatic?

Don’t think so… I don’t think QMaster/Compressor is cross platform.

—

Side note —
Even though you are a non-profit, it’s not excuse to have a sloppy growth plan. I work for a non-profit as well, and in the past running things sloppy has hurt us bad (lost footage, major down-times, wasted money, etc). Start putting solid building blocks in place and have a plan for growth. It might be worth looking into picking a platform instead of having a mix of both PCs and Macs — it will save you a lot of headache. Also, think about your bandwidth and storage needs and how they will grow. Can you do something now to prepare for growth? Also, think about disaster preparedness. What happens if your PC crashes (hardware or software). Do you have any redundancy in place? Sure, it costs more, but is it really worth the risk?

Don’t build your house on sand. Get something redundant power, redundant controllers, or you will regret it.

[David Gagne]

1) I know 28MB/s isn’t enough for edit. Our XSAN currently does 90MB/s easy even though it’s 90% full (crazy editors!). I’m pretty sure our performance will be much better when we don’t have full luns.
2) 10GigE may be out, but most main-line solutions do not support it yet. Isilon says it will be out sometime this winter for them. I don’t have as much trouble with the price really (we’re ready to spend six figures), but my priority right now is capacity, not bandwidth. Eventually I may want to upgrade to 10GigE however, and I’d like to be forward compatible…

Again, our situation is that we need capacity, and our bandwidth is fine on our existing XSAN — I’m more curious to see how Isilon performs to see if it can eventually be a replacement for XSAN or not.

[David Gagne]

Thanks for the input. For bandwidth, we may continue to have a smaller footprint SAN unless Isilon can meet that need. I know that as your storage grows, generally so does your bandwidth, as each node adds 2 more gigabit connections to your storage. I believe it has some sort of round-robin scheme for spreading clients out over these connections. Still you won’t get the same speeds that you would get over fiber on a single machine until 10GigE comes out.
I know if you have more clients than nodes and you are running into congestion you can add more performance accelerator nodes if you need more bandwidth.

One thing I’ve also considered is as we grow we may want to ditch our fiber infrastructure for the flexibility of ethernet… in the short term I think keeping our 20TB XSAN is smart.

[David Gagne]

Kevin,

I like yourself work for a ministry doing a lot of video production. I am the IT administrator that cares for the storage solutions. I can give a few thoughts that might be helpful for you.

First: Be prepared to spend $$ to send your IT guy to training if he isn’t an expert on some of this stuff. Depending on the solution you pick, he may need Final Cut Server class, and/or Snow Leopard Server classes, etc. These things are pretty complex, and if your guys don’t understand user permissions, server/client configurations, etc etc you can find yourself in bad situations even with good equipment. We are planning to go take some training now so we can fix things that didn’t get set up right the first time. Classes from Apple run around $2k per class.

Second: Be careful what solutions you pick. Our systems integrator has steered us wrong before, so make sure to get a few opinions before you pull the trigger. Promise Vtrak has worked for us pretty well, but I think at this point Active Storage is better (assuming you need that kind of bandwidth). I’d stay away from Drobo as edit storage, but possibly use it for backup or archive storage.

Third: You don’t *need* Final Cut Server. If you have some disciplined people who have a defined workflow that’s simple enough, you can move/tag the files yourself. We’ve been doing this for a few years now, and still haven’t needed FCS — although we’re getting close to needing it as our workflow gets more complex.

Fourth: Team Management and Systems Management is critical. Having a well defined workflow, someone who keeps people following “the rules,” etc., rather than just letting people do whatever they want with your expensive equipment. Also, if you don’t have someone monitoring and taking care of your systems, you will likely have costly down times. Again, if you’re going to spend the money on the equipment, make sure it’s being utilized properly, otherwise you are wasting your money.

Our Example:
We’re quite a bit bigger than you, dealing with 60-100 hours of live footage a week, and also creating promos, DVDs, etc., constantly.
We have:
20TB XSAN (Promise vTrak) – This is our edit storage, very fast, where live projects live.
MDC1,MDC2 — Apple XServers that handle metadata moving back and forth to the SAN. You can get by with a Mac Pro, but you definitely want a second Mac Pro as a live backup MDC in case one crashes.
20TB Isilon Storage (4 Nodes) – This is our ingest storage, but will be moved to be archive storage soon. It’s Ethernet based, so it’s slower… Drobo could also do ok for archive storage probably, and is a lot cheaper. We will probably use XSAN for ingest in the future (we do live capture of 1080i).
We have 7 Mac Pro clients that do all our editing, connected to the SAN via QLogic fiber switch.

We also have around 50TB of footage sitting around on bare SATA drives that go into external firewire chassis. I do not recommend this — this was a result of having way too much footage and not enough storage, and a tight budget. Get Drobo or something else, even if it’s not super great, just don’t do the bare drive thing, as there is no safety net.