-
FCPX NFS tuning
Hi everyone!
First: I’ve been a reader of this forum since quite a while and would like to thank everyone for this nice resource of insight!
Now I have a question you guys might be able to help:
The current setup I administrate (in Education…) is using a MacMini as fileserver with 2 Promise Pegasus attached (via thunderbolt, obviously), 1GbE and multiple MacPros as editing stations. Original media is stored centrally on the fileserver via AFP. Everything is running on latest 10.9. We’re mainly editing FullHD ProRes 422 with Multicam content using up to 6 cameras.
Originally we used the sparsebundle-workflow with the sparsebundles hosted on the fileserver to make libraries available on all editing stations, but after a while, this turned out to be insufficient. Editing a multicam-project becomes slow as soon as FCPX has to render some content (due to any effect or whatever) and doesn’t access the original media anymore. So we switched over to place the libraries locally on the editing stations.
I only learned recently that NFS is actually supported to host libraries in a network storage, so right now I’m trying to get it running. It basically works, but also doesn’t perform well. As long as I’m playing back original media (multiclip but no effects) which are accessed over afp it plays back smoothly, but as soon as FCPX renders content (due to color correction or whatever) and has to access those files from the library (this time over NFS) there are lags of 1-2 seconds and frame dropouts of sometimes >50%.
The Promises still have lots of “headroom”, the fileserver doesn’t max on cpu or ram, the ethernet network should also handle the traffic (copying files both over NFS and AFP are very close to 120MBytes/s, also activity monitor never maxes the ethernet connection while editing).
Do you have any tips on how to improve performance in this setup? I really don’t see what’s the bottleneck in this setup and it’s getting kind of frustrating ;-).
We plan to upgrade the uplink fileserver -> switch to 10GbE, but with a single client in the testing environment this shouldn’t be necessary yet, right?
Thank you for any tips!
Benni Graf.
-
24 Replies
-
Sorry to tell you Benni, but what you have is all you’ll get for now. Apple has not built-in the proper networking protocol (TCIP) for Thunderbolt that’s necessary for shared storage. Without that, you will experience latency issues, and there is no fix for that at this time.
David Roth Weiss
ProMax Systems
Burbank
DRW@ProMax.comSales | Integration | Support
David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.
-
Thanks for your reply,
but are those very heavy lags (1-2 and sometimes more seconds when starting Playback) common?
Does thunderbolt cause them?Oh my, well, I guess we’ll stay with local editing then… Thanks!
Best, Benni.
-
[Benjamin Graf] ”
but are those very heavy lags (1-2 and sometimes more seconds when starting Playback) common?
Does thunderbolt cause them?”Yes sir!!! That’s exactly what you get when you don’t have the proper network protocols built-in, it’s almost identical to what many encounter when they attempt to build a home-brewed NAS for shared video storage.
As all of the storage industry professionals here will reiterate, if shared storage for video was easy, companies like ProMAX and the many others you see advertising here on the Cow would simply not be in business.
David Roth Weiss
ProMax Systems
Burbank
DRW@ProMax.comSales | Integration | Support
David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.
-
Obviously makes sense! Thanks for pointing this out!
-
Benni,
See this post in another conversation: https://forums.creativecow.net/readpost/197/860241
David Roth Weiss
ProMax Systems
Burbank
DRW@ProMax.comSales | Integration | Support
David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.
-
Just out of curiosity, but how is TCIP for Thunderbolt connected to OP question?
As I understood Mac Mini is connected to RAID via Thunderbolt and than shared via Gigabit Ethernet.
Practically Thunderbolt is used only for storage expansion, and known Thunderbolt TCIP issues you have linked doesn’t have any impact in this scenario (he is not trying to use Thunderbolt networking).
Or am I completely wrong? 🙂https://twitter.com/#!/disample_dcc
-
Even I didn’t get the Thunderbolt TCP-IP connection. The OP basically has a NAS.
I think the problem lies in the sharing. Maybe the switch. It may improve if the Macmini were to connect with the switch over 10GigE going into a 10GigE port on the switch if it has one.
———————————–
Neil Sadwelkar
neilsadwelkar.blogspot.com
twitter: fcpguru
FCP Editor, Edit systems consultant
Mumbai India -
[Neil Sadwelkar] “I think the problem lies in the sharing. Maybe the switch. It may improve if the Macmini were to connect with the switch over 10GigE going into a 10GigE port on the switch if it has one.”
Hi Neil,
Ethernet speed is NOT the issue, he’ll still have latency issues using T-Bolt drives, even over 10GbE, it’ll just be a more expensive failure.
David Roth Weiss
ProMax Systems
Burbank
DRW@ProMax.comSales | Integration | Support
David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.
-
[Ivan Radovanovic] “Just out of curiosity, but how is TCIP for Thunderbolt connected to OP question?
As I understood Mac Mini is connected to RAID via Thunderbolt and than shared via Gigabit Ethernet.
Practically Thunderbolt is used only for storage expansion, and known Thunderbolt TCIP issues you have linked doesn’t have any impact in this scenario (he is not trying to use Thunderbolt networking).
Or am I completely wrong? :)”Sorry Ivan, but you are wrong in this case – I understand it may seem illogical, but there are TCIP issues involved.
FYI, our Platform Studio servers have Thunderbolt-2 connectivity, however we are simply unable to get full streaming performance adequate for editing in a shared storage environment off a connected Thunderbolt array, though we can off connected SAS, eSATA, and even Firewire drives. We can use the T-bolt arrays for tier-2 backup and archive, but NOT for editing. Trust me, many have tried and failed, this a well known issue.
David Roth Weiss
ProMax Systems
Burbank
DRW@ProMax.comSales | Integration | Support
David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.
-
Hi –
sorry, but I just had to respond. I fully understand the desire for Thunderbolt networking to work – one reason – IT COSTS NO MONEY if you have a new Mac Pro with it’s 6 Thunderbolt ports.1) you still need a big fat disk drive array that doesn’t cost 500 bucks
2) since you are not working with your other editors and graphics artists at the same desk, 6 feet away from them, if Thunderbolt Networking DID work, you would need the expensive Corning Optical Cables to connect your Mac Mini, etc. to the Mac Pro. So 4 of these cables is EXPENSIVE.
so your costs at the end of the day would be similar to a 10 G (or 1G) system that you can find commercially available right now, when you factor in the cost of the drive array, and the Corning cables.
The bottom line here, is that all of these stupid posts come down to one thing – “how can I have shared storage for free”. There is a simple answer – YOU CANT.
And the next knucklehead will respond “but what if I buy a $1500 Drobo”.
What is the LONG TERM plan for “free” shared storage ? One day (when both David and I am retired) the internet will have wonderful amazing connection speeds, and you will in fact be able to use a cloud site as your shared storage, so that you can use your FCP-X “share to iCloud”, and it will actually allow you to edit from iCloud your Blackmagic 4K media. However TODAY IS NOT THAT DAY (and neither is tomorrow) and by the time this becomes a reality, both David and I will be retired, and you won’t see these “angry” posts on Creative Cow.
Bob Zelin
Bob Zelin
Rescue 1, Inc.
bobzelin@icloud.com
Reply to this Discussion! Login or Sign Up