[Steve Modica]

Hi James,
I think you or the customer has talked to Matt a few times over here at Small Tree about this issue.

For those of you perusing this thread or in a situation similar to James, I suggest you talk to us first.

Our entire business model is to implement high end network services on Macs (FCoE, Link aggregation, iSCSI, 10Gb etc), and then sell that. Obviously people use it for networking storage and editing, so over time we got hit with so many storage issues, we started to carry switches and storage. I got sick of people blaming their Gb ports when we could see their switches couldn’t handle the traffic (no flow control) or their storage had way too much latency. The vendors would always shrug their shoulders and say “but it goes 800MB/sec.” (That doesn’t make a difference when you’re talking frame rates)

We supported realtime stuff for NASA, for the Airforce, for the Army, and for the big animation houses like Disney and Dreamworks while we were at SGI. Not only can we write drivers, we know how the unix kernel works and how IO and networking interact.

I can promise you are are the best value out there in a working solution. The prices are low because we have very little overhead. We exhaustively test everything and we’ve actually contributed a ton of code to the open source world just to get the right utilities working for Mac so we could generate the loads we need (no one else is doing load generation like us.)

I don’t mind so much that people try DIY and buy stuff from Newegg. I can promise one of two things will happen. It won’t work and you will learn to live with it, or you won’t be able to live with it and you’ll eventually call us looking for something that works. That’s the most common call we get. 🙁

So you can buy from guys that don’t understand drivers and the kernel at all, or you can buy from computer engineers that supported top 500 Supercomputers (including the Big Mac at VT and the *bigger* Mac at Colsa!)

One last bonus to buying from Small Tree. We make all this work on a mac. So when you upgrade your server or repurpose the storage for the next project, that Mac can go on someone’s desk. If you build something with linux or windows or buy someone’s linux black box, that’s a throw away system. Everything I sell you will work for a long time and you won’t be wasting a lot of money.

Steve

[Steve Modica]

If you are using more than 10drives, you should definitely be raid6.

There are two things to look for in a raid vendor:

1. Do they talk to you in guaranteed stream count or MB/sec? If it’s the latter, they probably don’t understand the difference between average performance and realtime frame delivery. Who knows what else they don’t understand? That’s not FUD. People will build something and claim it works because the plugs fit together. 🙁 We get these calls all the time.

2. Do they understand the Mac kernel? If not, how can they support you when something fails? Can they instrument the kernel and profile the IO activity? Do they know how to spot a “hot lock” that might be slowing you down? No.. They don’t. We write out own drivers and deploy our own firmware on the hardware we sell. We absolutely understand this stuff.

I’ll wager that outside of Apple, only Small Tree can do this, because we supported realtime stuff like this for SGI for the military and NASA. Even better, our stuff is less expensive than most of the other stuff out there. We don’t have to build anything fancy, we just know how it works!

Be especially careful as you wade into 6Gb SAS and 10Gb ethernet. These are new technologies and I haven’t seen a single piece on my desk that worked correctly out of the box. So there’s a lot of caveats to putting something like that together.

Steve

[Steve Modica]

Hi Tony,
Sorry about coming late to the party. Bob sent a link to this thread and I read it over today.
Small Tree tests all of this stuff and we’re currently working on all the new stuff (SSD and 6Gb SAS/SATA along with 10Gb Ethernet). We are pretty versed in what is required to make the video work without stuttering 🙂 (We are all former SGI realtime/networking guys).
From what I gathered, the material you were playing was actually getting dragged over to the local machines, or was playing off the local machines which caused the problems.

The hardest part of all this isn’t the networking. That’s usually the easy part. The storage is the hard part. A local drive can handle 40MB/sec, but that’s an average over time. It cannot handle 40MB/sec continuous in a realtime fashion. The RAIDS we sell (as well as Maxx) are designed to handle the workload in realtime. So the overall bandwidth can actually be lower, but they are able to satisfy each IO request within the designated window. For the most part, storage that is optimized for the highest bandwidth will not work well in realtime.
Steve Modica

[Steve Modica]

The mac mini is still running a core 2 duo chipset. The i3-7 stuff is using the broadcom. I’m sure when the mac mini hits i5 it’ll be a non-jumbo chip too 🙂
I did my best to let someone within apple know the benefits of jumbos. Hopefully we get them in future hardware.

[Steve Modica]

They specifically call out jumbos on the mac pro and they don’t on the imac, so I think the imac still does not support them.

[Steve Modica]

Sorry to be late in responding. I think there’s a lot of confusion over the issue of jumbo frames, segmentation offload, and the new i7 performance.

The new imac (and the new Mac Book Pros) use the Broadcom 5764 chipset and it does not support jumbo frames. It supports a feature called TSO (transmit segmentation offload) that is meant to provide jumbo frame performance without the requirement of switch and partner support. Large packets get handed to the card, and it chops them up and applies headers. This saves the OS the trouble. That’s all good and true enough.

On the receive side, you still receive 1500 byte packets. The receiver has to strip the headers and reassemble. This takes a lot of CPU. It generates more interrupts and more context switches. It causes more work for the server. One solution to this is receive side coalescing, but no gigabit chips today support that. (The Intel 82599 10Gb chip on our newest 10Gb card does). So for the present, when an imac is reading in video, there’s 0 jumbo frame “help” so to speak. It’s full on tiny packets.

As for performance, I don’t have a good quantitative answer for what you lose using an i7 vs an older core 2 duo. Walter made a qualitative judgment that the older system does a better job and doesn’t drop. He has some fairly complex timelines. In house, I find my older core 2 duo laptop was about on par with the new imac. I would not recommend either for serious editing. (When I say serious, I mean at least 2 active Pro Res HQ streams coming over the wire at 60MB/sec. If you are doing DV editing or something lower bandwidth than pro res, I’m sure the imac is fine)

The USB dongle sited here was tested by Bob Z and it didn’t support jumbo frames. Since USB is limited to 480MBits, it’s also clear it won’t get us to 2 Pro Res HQ streams. So it was a non-starter for me. Even if it did jumbo frames, I would not expect to see more than 60MB/sec over the wire and with overhead, that won’t guarantee two Pro Res HQ streams (although in practical use, it might be OK for short things).

So all this being said, I think if you are a pro res editor, and each stream is a reasonably large fraction of your gigabit bandwidth, you need a mac pro. If you are doing something else, imac, mac book pro, mac book, mac mini are all potential candidates for you. Just be fore-warned that they won’t compare to the capability of the larger system.

One last note, I prefer it when we can get one of our gigabit cards in the client. We have tools that let us dump stats and see the state of the chips. Apple doesn’t have anything like that, so when something hangs, it’s very hard to debug. We’ve had problems like this recently. With our PCIE cards, I can dump all sorts of good stuff to see why it hung.

[Steve Modica]

I would suggest you look at the cpu meter during playback. I’d be interested to know if one cpu is pegged.

[Steve Modica]

What kind of system is the client?

[Steve Modica]

For the record, 10Gb is being supported on cat6A. (it’ll run over other qualities, but the distance will be shorter. I think it’s 55m for cat6).
Anyhow, if you are plumbing the building with this stuff, you should consider putting in cat6a so you can plug in those new mac pros with the 10Gb Lan on Motherboard that people are spreading rumors about 🙂

[Steve Modica]

One more comment:
MetaSan/MetaLan are really meant to act as scalable replacements for Xsan/AFP. The idea is that you could create a SAN (with metaSAN) and then export than SAN with multiple servers using MetaLAN. It’s good for big environments where they want a redundant, clustered type server. In my experience (which isn’t that much), metaLan is not faster than AFP for a single system setup.