Success: 8 CAM DVC PRO HD Ingest on Isilon 6000X

[Bob Zelin]

I have no idea of what you have accomplished here, and I am very interested – please explain –
you make it sound like there is no SAN system – just an Apple XServe, with a simple Gig Etherent network, and an Isilon fibre array hooked up to the XServe. You have 8 discreet sources (8 MAC’s with AJA Kona cards), and the ethernet cables going into the Gig E switch, and you were able to capture all 8 sources at once to the Isilon array (using NFS as your network protocol) –

is that what you are claiming (because I don’t believe you, or I dont’ believe it works with a simple system like this) – please explain further. Yes, I am VERY interested in what you have accomplished.

bob Zelin

[Bob Zelin]

David –
I am so facinated by your post, that I searched for your other posts about your system, and found this –

David writes –
“We have:
20TB XSAN (Promise vTrak) – This is our edit storage, very fast, where live projects live.
MDC1,MDC2 — Apple XServers that handle metadata moving back and forth to the SAN. You can get by with a Mac Pro, but you definitely want a second Mac Pro as a live backup MDC in case one crashes.
20TB Isilon Storage (4 Nodes) – This is our ingest storage, but will be moved to be archive storage soon. It’s Ethernet based, so it’s slower… Drobo could also do ok for archive storage probably, and is a lot cheaper. We will probably use XSAN for ingest in the future (we do live capture of 1080i).
We have 7 Mac Pro clients that do all our editing, connected to the SAN via QLogic fiber switch.”

REPLY –
So you have a big XSAN system. I am glad that your system is working, but you have just posted this. Yes, you can capture a 15MB/sec single stream from 8 MAC’s at one time on almost any modern SAN system. As long as your drive arrays can do over 120MB/sec (and yours can, because you have Promise and Isilon arrays),
15 x 8 = 120, so this is not an unusual accomplshment. I was about to give you a “hard time” because I thought this was on a single dedicated system (like a single MAC) – and if you have a dedicated product like Telestream Pipeline, this single $10,000 box can only bring in 2 streams at once, but if you have a large Fibre Based XSAN system with 8 dedicated MAC workstations, then yes, this is no issue.
Again, I am very glad this is working for you.

Bob Zelin

[David Gagne]

In more detail:

XSAN stayed home where it’s safe.

We brought:

Isilon 6000x Cluster:
(4) nodes, each with 2 gigE ports. 8 gigE connections total on Isilon.

(4) XServes with Kona LHi. These are our normal ingest machines. We did put in GT120s which seemed to smooth the playout issues we were having.

(4) Intel Mac Pros with some old Kona LHe we pulled from our broken GVS equipment.

(2) Intel Mac Pros for editors to work on location.

I distributed the clients to connect to differing IPs so that it wouldn’t overload any of the nodes/ports.

Node 1 Port 1: XServe1
Node 1 Port 2: XServe2
Node 2 Port 1: XServe3
Node 2 Port 2: XServe4
Node 3 Port 1: Mac Pro 1,2
Node 3 Port 2: Mac Pro 3,4
Node 4 Port 1: Editor 1
Node 4 Port 2: Editor 2

When monitoring the traffic it was something like:
Node 1: 200Mb/s (25MB/s)
Node 2: 200Mb/s (25MB/s)
Node 3: 400Mb/s (25MB/s)
Node 4: Varying

It was peaking at around 1Gb/s (250MB/s) total throughput.

I know that’s not super high for a system like this, but with no dropped frames over GigE over the course of 4 days and around 10 hours per day, I was pretty happy.

Being a NAS does cause some limitations of course, no live editing, weird permissions, etc… but over all it got the job done.

[Bob Zelin]

you can edit over a NAS – it works just fine, especially if you have dedicated ports (as you have). This is how Final Share, Granite Stor, EditShare, and Apace work – there is no reason why you could not have done live editing.

I am very impressed – you should be proud of yourself. What made you look into Isilon in the first place ?

Bob Zelin

[David Gagne]

One of our integrators recommended them (Midwest Media Group).

We’ve had some challenges with it, and the annual support is ridiculously expensive, and of course necessary. I think in hindsight I maybe would have gone with a traditional SAN… but it was a tough call. Many of the awesome advantages of Isilon we do not take advantage of enough to make it worth it.

Isilons strengths really show up when you start to add more nodes to your cluster. No need to migrate data etc, really fast Infiniband backbone, etc. No need for fiber or metadata controllers is nice as well.

Downside is that you have to mount via NFS mountscripts and permissions get wacky if you’re not careful.