[Steve Modica]

[Shane Ross] “What transcoding? When you drop a still into a timeline, and then render, FCP renders a media file that matches the sequence settings. Then you have a media file that matches all the other media files. It is, in essence, no different than those other media files. Did you RENDER before you played?

Actually no. When I’m testing performance, I don’t usually render. I let the App transcode. We do this on purpose to measure the IO and latency and see if we’re keeping up.

You’re absolutely correct that rendering is something people should do, but sometimes they don’t want to, or FCP shows a green line and when they render, that section is missed.

Testing a rendered timeline would be pointless for me 🙂 I know one stream works.

Steve

Steve Modica
CTO, Small Tree Communications

[Steve Modica]

I’ve seen stills cause drops. I don’t think it’s the size of the still as much as the transcoding that FCP has to go through to play it as a video stream.
When we set up customers to edit over gigabit, an imac (without jumbo frames) will reliably drop on a still image. The size and complexity don’t seem to matter. If we render that section it’s fine. Mac Pros seem to ride through it just fine. It seems related to CPU utilization.

Steve Modica
CTO, Small Tree Communications

[Steve Modica]

[Bruce Little] “I have put together an ethernet based SAN already for another post house that cost about
20k that works just fine, but wondering if there are any better solutions available for this price.”

Give us a call at Small Tree. We can do an awful lot for $20k. You need to tell us the codecs, storage space requirements and we can design something for you.

In general, with 3Gb SATA and Gb, our stuff runs $500-1000 per TB. I think that’s pretty good compared to most of the ethernet based vendors out there. Plus we know what all the sysctl parameters do since we worked with BSD unix kernels at SGI.

Steve Modica
CTO, Small Tree Communications

[Steve Modica]

[Bob Zelin] ” Remember, that the only expensive part of this process is a drive array that is fast enough to serve out the files that you need.”

I’m going to shamelessly plagiarize wikipedia (note the bit about servers):

Real-time computing is sometimes misunderstood to be high-performance computing, but this is not always the case. For example, a massive supercomputer executing a scientific simulation may offer impressive performance, yet it is not executing a real-time computation. Conversely, once the hardware and software for an anti-lock braking system has been designed to meet its required deadlines, no further performance gains are necessary. Furthermore, if a network server is highly loaded with network traffic, its response time may be slower but will (in most cases) still succeed. Hence, such a network server would not be considered a real-time system: temporal failures (delays, time-outs, etc.) are typically small and compartmentalized (limited in effect) but are not catastrophic failures. In a real-time system, such as the FTSE 100 Index, a slow-down beyond limits would often be considered catastrophic in its application context. Therefore, the most important requirement of a real-time system is predictability and not performance.

Steve Modica
CTO, Small Tree Communications

[Steve Modica]

I agree with Bob’s post 100%.

“Real fast” doesn’t mean “Real time” and the loads you want to push are realtime. When you crank up those 10Gb ports across all those clients, the OS is going to “burp” a lot. When it does, everyone is going to drop frames. (To make matters worse, it won’t burp immediately either. It’ll wait a few hours til the page cache is fully populated)

10Gb to Gigabit isn’t trivial. 10Gb ports don’t talk Gigabit to gigabit ports. They hammer them with 10Gb bursts. Imagine the normal flow control issues with gigabit times 10. I can promise you, for the purposes of video editing, it won’t work right. Either the vendor won’t support xoff transmit from their 10Gb ports (thanks Cisco and Extreme) or they won’t have their water marks correct to be useful (broadcom).
If you buy a 10GbaseT switch, it probably won’t negotiate gigabit flow control correctly either.

You may find that 3 1Gb ports are enough to kick the 10Gb ports butt.

I don’t want to be the guy that goes around spreading FUD all the time, but we seriously sit around banging our heads on this crap everyday. Chris and I are looking at the new Samba, how the OS processes IO buffers and what happens when 10Gb ethernet is dragging data off of 6Gb storage (hint: it’s not good for the OS). Keep in mind, Small Tree sells the fastest PCIE cards you can put on the Apple PCIE bus. So you might expect we’re out there from time to time letting them test our stuff.

What I’m trying to avoid is this upset customer scenario where you call us saying “it doesn’t work” and we basically demonstrate to you that the cards go fast, but your storage and server won’t keep up. Sorry… Now what? You hate me because I won’t spend my engineering resources to fix the stuff you bought somewhere else. 🙁

Steve Modica
CTO, Small Tree Communications

[Steve Modica]

[Jimmy Brunger] “Thanks for the reply. I wish I understood all that you just said, but I’m afraid I don’t! Is there a less techie way of explaining that and any way to solve it?”

I’ve come to learn that it’s a hierarchy of little things all related. So you learn about 5 things (disks, raid controllers, switches, network cards), and then you learn 5 things about each of those 5 things (rotational vibration sensors, aerial density, URE rates, etc) and before you know it, you have this laundry list of 250 items that need to be right for everything to work.

Steve Modica
CTO, Small Tree Communications

[Steve Modica]

This is the kind of thing where you have to understand what premier is doing. Quicktime and FCP both use AIO (asynchronous IO) which does the IOs in parallel from within the kernel. The App basically tells the kernel what to read and it starts several threads to do it. (10.6 makes AFP work in parallel as well which is good for multiclip).
Premier may not be doing that. It may be using a different IO type like write/read and it may be using a poor IO size. (Some apps choose their IO method based on the type of filesystem underneath, some do not).
FCP uses 4MB IOs when it can. There are smaller IOs for the audio.
There’s also the issue of frame size and how busy the OS is doing the IO. If you are using standard frames, it’s likely the OS is very busy parsing all the packet headers.

Steve Modica
CTO, Small Tree Communications

[Steve Modica]

I have a feeling this is a hardware issue rather than a disk corruption issue. Keep in mind that the two go hand in hand since drives “dropping out” often lead to corruption. The more it happens, the worse it gets.

You certainly can’t run disk warrior if the drive is dropping off the bus every ten minutes.

My first thought is heat since it goes away and comes back later. Have you tried running with the top off and fanning it? Another trick it to put it in the freezer for a while.

While you have the top off, make sure all the cables on snug.

Does your firewire cable look kinked or messed up?

Once you have the drive staying online for a long time, then consider disk warrior or verifying the disk with disk utility.

Steve

Steve Modica
CTO, Small Tree Communications

[Steve Modica]

Small Tree sells a gigabit ethernet card for that slot. The peg34m. It works *and* supports jumbo frames.

Steve Modica
CTO, Small Tree Communications

[Steve Modica]

someone mentions below you are going over a gigabit link. Is the raid array on a server? Have you tried playing the video in quicktime or something other than PP? First thought is you’re just dropping frames due to storage latency (or network latency).

Steve Modica
CTO, Small Tree Communications