-
RAID6 via 10GbE – Challenges with protocols, reliability and performance
It has been a tough and long road finding a video editing solution to fit our needs – we’re not there yet – so any advice is much appreciated!!
Goal:
– connect 3 to 5 Adobe PP and AE (occasionally FCPX) client workstations (2010 MacPro 12-core with ATTO FastFrame NT12 NICs)
– to RAID6 with minimum of 8 drives
– and backup to DLT tapes & individual drivesConcept 1:
NAS (Linux Software RAID6)
QNAP with 10GbE card installed
TVS-871-I5-8G 8BAY 3.0 GHZconnected to:
10GbE Switch
Netgear ProSAFE Plus XS708Ewhich is what the clients connect to.
Problems we ran into:
Protocols:
Although RAID6 performance was fantastic via the network (ca. 700-800 GB/s), we had serious problems with AFP and SMB3 on the QNAP.
In AFP, during long data transfers onto the QNAP, we got random general i/o errors (using Finder or ChronoSync).
In SMB3, it was not possible to copy to FCPX Projects to QNAP because SMB apparently can’t resolve symlinks which are common in FCPX projects.Reliability:
Also, my understanding from research is that a software RAID6 is slow and unreliable to rebuild after a drive failure, where as an Areca hardware RAID “will absolutely keep on trucking with zero downtime during a drive failure” (Chris Marriott on imore).Concept 2:
Hardware RAID6
Areca 8050T2 (aka Maxx Digital ThunderRAID2, apparently)connected to:
MacMini & 10GbE NIC
MacMini i5 2,6 GHz TB2 with ATTO ThunderLink NT 2102connected to:
10GbE Switch
Netgear ProSAFE Plus XS708Ewhich is what the clients connect to.
Problems we ran into:
Performance:
Although RAID6 performance was in the 800-1000 GB/s range as expected via Thunderbolt2 on the MacMini, it dropped to ca. 450 MB/s for a single client on the network, about half the speed of the QNAP. But what is amazing is that using ChronoSync, copy speeds via AFP to the RAID drop down to ca. 4Kb / sec… inexplicable. A 200MB mp4 movie took over 6 hours to transfer. This happened today on our first day of testing the system.
Also, I took SMB3 out as an option right now, as there seems to be shocking performance drops in 10.11.5 and the FCPx incompatibility remains.Any advice to solve any of these issues is greatly appreciated!
Thanks!
Daniel -
8 Replies
-
You’ll want to use NFS exports for FCPX. Either system can do this, but may need some tuning to get the best performance.
Unfortunately 10.11.5 has performance issues with SMB. There is a post here about that:
https://forums.creativecow.net/thread/197/860869..You’ll find info elsewhere as well. Apple have a habit of messing this up.
As for the QNAP w/AFP thats kinda weird – you shouldn’t get such problems. I don’t know what version of netatalk QNAP use, but I expect the issue is not the server. Try with a clean share that has not been accessed using SMB…
RE software RAID reliability don’t write off products that use filesystems like ZFS or BTRFS. A hardware controller won’t necessarily be the answer to all of your troubles. ZFS is my choice, you can import a pool just about anywhere using fairly basic hardware. Its very powerful!
PS. You probably mean MB/s not GB/s, 😉
alex@indiestor.com
Open Source Shared Storage -
Hi –
see, it’s not so easy to do this yourself, is it ?
As Alex said, use NFS for FCP-X. Not getting great performance ? You need NFS tuning. No, I am not giving that to you.
Raid 6 rebuild is not slow and unreliable. Who told you that ? The QNAP rebuild works fine. The Areca hardware rebuild works just fine (both ARC-8050T2 and the ARC-1883x PCIe card with an external mini sas array). Do a huge file transfer (multi terabytes) while you are trying to have multiple editing, and you will suffer, and your transfer will suffer. And this not only applies to QNAP and home made Mac server systems, but systems like AVID ISIS 5500 as well.The Mac Mini is a terrible server. It has one thunderbolt buss, so you are splitting a single x4 lane to both your ARC-8050T2 and the 10G thunderbolt box. Use a $3000 Mac Pro and use separate thunderbolt busses. Or use an old Mac Pro with a hardware areca and external minisas raid array. And the Netgear XS708E is a huge disappointment – which is why for small systems I always use the XS712T. Today, you would have purchased the brand new XS708T for the same price you paid for the XS708E.
Your QNAP will outperform the Mac Mini, no matter what you do. You should have bought 16 G of RAM for the QNAP.
If you are doing FCP X, use NFS. And dont’t transfer 48 TB of media while you are editing – no matter whose system you own.Bob Zelin
Bob Zelin
Rescue 1, Inc.
bobzelin@icloud.com -
Thanks for the NFS tip, will try that in due time.
The AFP problem was an issue I isolated down to the QNAP. When backing up 1-2 TB over night, I would get random i/o errors. Not sure about the netatalk, but we had updated to the latest firmware.
And yes… I meant MB/s not GB/s 🙂
Thanks!
-
see, it’s not so easy to do this yourself, is it ? #
Never thought it would be, but I didn’t think it would be impossible…I understand all your points – at the moment I am checking the very basics and there are issues already.
I have taken the entire 10GbE network out of the equation and am checking the Areca ARC-8050T2 and MacMini (late 2014 with TB2) ONLY.
– I can play 4 HD streams in ProRes with only a few jittery hiccups during playback in this most basic configuration. Surely playback should be smooth – but it isn’t
– when I simultaneously duplicate a folder on the RAID using the finder (ca. 20Gb), HD playback performance plummets to unusable, sometimes coming to a standstill before “catching up” by playing the stream 4x-5x speed
Any clues? Or should I try a QNAP TVS-1282T?
Thanks!
-
you cant take the 10G out of the equation. The 10G is the ONLY reason why any of these systems work.
A 1G network (a 1G based server) CANNOT provide reliable playback.
I don’t care if it’s QNAP, a Mac Mini, or a SNS EVO, ProMax, Small Tree, Facilis, AVID, Edit Share – without the 10G backbone, NOTHING is going to work. The QNAP 871T is just fine, as long as you use the 10G port to a 10G switch.
Same with the Mac Mini.Either way – you can have everything in your system 10G – don’t do a multi terabyte file transfer while you are editing.
End of story.Bob Zelin
Bob Zelin
Rescue 1, Inc.
bobzelin@icloud.com -
-
The choppy playback, etc I was describing is happening using a direct Thunderbolt connection – NOT 1G. I am not testing 1G vs 10G. I wanted to see how the Areca performs with the mac mini before the mac mini acts as a server. And currently it performs poorly. I was asking in the forum about ideas why that is the case.
Thanks,
DanielPS: I don’t know what you mean by “Interstellar”, but I am not from there in any case.
-
Hi Dan –
ok, sorry for the confusion. You are telling me that you have a Mac Mini, connected to the ARC-8050T2, and there are no other devices in the thunderbolt chain (no 10G adaptor, no second computer on the thunderbolt buss) – and you are getting choppy playback ? The ARC-8050T2 is a wonderful, very fast RAID array – as fast as anything from any other manufacturer. So I apologize for still not having a completely clear picture (because you are not contacting me) – but I must ask again – in this “isolated” system – do you have any Blackmagic or AJA products also on the thunderbolt buss ?When you run AJA System Test or Blackmagic Speed Test on the Mac Mini, testing the ARC-8050T2, what speeds do you get ? You should be getting around 900 MB/sec.
Bob Zelin
Bob Zelin
Rescue 1, Inc.
bobzelin@icloud.com
Reply to this Discussion! Login or Sign Up