-
10GB networking / storage / thunderbolt questions
Dear Fellows
We have installed a quick and dirty 10G ethernet setup in our company that partially works and consists of the following:
– A 2010 Mac Pro workstation with 10G pci card
– A Mac Mini i7 with a 10G ethernet card in a Sonnet thunderbolt chassis. With attached tunderbolt RAID. We use the Mini as storage server, LTO Backup station, Davinci Resolve Database Server
– A Mac Pro 2013, which is our color grading workstation.
– A small Netgear switch with two 10G ports and 1G ports.The old Mac Pro and the Mac Mini are connected over 10G ethernet via the switch. The new Mac Pro is connected to the Mac Mini via thunderbolt networking.
Even if it looked like an elegant and cheap solution we found that thunderbolt networking does not work. We have sometimes good transfer speeds but very unreliable. After reading on this and other forums it looks like it’s the consensus that thunderbolt networking just does not work properly. Has anyone here managed to get a thunderbolt networking setup between two Macs to work reliably?
The transfer speeds between the old Mac Pro and the Mac Mini over 10G are generally good. But strangely read speed from Pro to Mini maxes out at around 500MB/sec while write speed goes up to 800MB/sec. (The TB RAID can do over 1000GB/sec). Is that what can be expected or are there any performance tweaks that I don’t know of? SMB signing is off and jumbo frames are on.
As the current setup does not work for the new Mac Pro we will upgrade to a dedicated storage box very soon.
The most elegant solution looks like a combined 10G / thunderbolt box from Qnap (TVS-1582TU or similar). We would connect either the Mac Mini or the new Mac Pro to the Qnap via thunderbolt and the rest via 10G ethernet. This would spare us from getting additional pieces like a 10G switch and more TB>10G boxes.
As this setup would require the Qnap to act as a switch / bridge from TB to 10G I would like to ask what the experiences with such a setup here on the forum. Does TB networking via the Qnap show similar problems like between two Macs or does that work well?Another solution would be to get a 10G only box from Qnap or another vendor & more TB>10G boxes & a switch with more ports and connect everything over 10G ethernet. Is there anything that speaks against the mixed TB / 10G setup and for the 10G only?
Any tips, hints and recommendations are much appreciated.
-
5 Replies
-
replies below –
We have installed a quick and dirty 10G ethernet setup in our company that partially works and consists of the following:
reply – you thought this would be easy, right ?
– A 2010 Mac Pro workstation with 10G pci card
– A Mac Mini i7 with a 10G ethernet card in a Sonnet thunderbolt chassis. With attached tunderbolt RAID. We use the Mini as storage server, LTO Backup station, Davinci Resolve Database Serverreply – the Mac Mini is not a suitable server. You are sharing a x4 lane single thunderbolt buss between the thunderbolt RAID (you have not said what model this is), and a Sonnet Twin 10G product for connection to your Netgear switch. You will get crappy performance. And if you are running macOS 10.13 Server – forget it, this software no longer works (everything was fine until 10.13).
– A Mac Pro 2013, which is our color grading workstation.
– A small Netgear switch with two 10G ports and 1G ports.reply – so you have a 10G backplane, but your connection to your clients is 1G, which means you get a max of
100 MB/sec per client – barely enough for 4K or 6K work – and on a Mac Mini server – forget it. And you did not say how much RAM you had in the Mac Mini (do you have 16 Gig) – and you did not say what your thunderbolt RAID is –
is it a Promise Pegasus R8, or G-Tech Studio XL with 8 drives – because if you are using a 4 bay or 6 bay, its never going to give you reliable playback – even if you had a Mac Pro cylinder as the server (which you don’t – you have the Mac Mini).The old Mac Pro and the Mac Mini are connected over 10G ethernet via the switch. The new Mac Pro is connected to the Mac Mini via thunderbolt networking.
REPLY – you will get terrible performance from the “new Mac Pro” via Thunderbolt 2. Thunderbolt bridging network connection is absolutely useless.
Even if it looked like an elegant and cheap solution we found that thunderbolt networking does not work. We have sometimes good transfer speeds but very unreliable.
reply – that is right. If you simply do AJA System Test or Blackmagic Speed Test on a thunderbolt bridge connection – you will see wildly varying speeds – you get 800 MB/sec, then you get 56 MB/sec – up and down. It’s not like Ethernet where you get consistant speeds. when your transmission speed drops down, you get stuttering playback. But that’s ok – you are doing everything wrong anyway – you probably don’t have an 8 bay RAID array, you don’t have 16 Gig of RAM in the server, the server is not powerful enough to handle this task, and you are using thunderbolt 2 bridging for connection to one of your computers. So you are doing everything wrong.
After reading on this and other forums it looks like it’s the consensus that thunderbolt networking just does not work properly. Has anyone here managed to get a thunderbolt networking setup between two Macs to work reliably?
reply – no. Thunderbolt 3 is better, but it’s still not as good as a simple thunderbolt to 10G adaptor from Promise or Sonnet (or the new iMac Pro). See – you should have kept reading and just bought a QNAP or Synology with 8 drives and a 10G port. You would have saved a lot of time and aggravation (and money).
The transfer speeds between the old Mac Pro and the Mac Mini over 10G are generally good. But strangely read speed from Pro to Mini maxes out at around 500MB/sec while write speed goes up to 800MB/sec. (The TB RAID can do over 1000GB/sec). Is that what can be expected or are there any performance tweaks that I don’t know of? SMB signing is off and jumbo frames are on.
reply – as I stated above, you have too much of a load on your Mac Mini on the single x4 lane thunderbolt 2 buss. You will get one computer to work – add in the second computer, and BOOM – everything will get sucked down, because you don’t have the bandwidth.
I have to assume that you had a lot of this stuff lying around. Had you gotten an antique Mac Pro 4,1 from 2009, and put an external RAID on it, with a PCIe 10G card (yes, you still need 8 drives) – you would have gotten a working system.As the current setup does not work for the new Mac Pro we will upgrade to a dedicated storage box very soon.
The most elegant solution looks like a combined 10G / thunderbolt box from Qnap (TVS-1582TU or similar). We would connect either the Mac Mini or the new Mac Pro to the Qnap via thunderbolt and the rest via 10G ethernet. This would spare us from getting additional pieces like a 10G switch and more TB>10G boxes.
As this setup would require the Qnap to act as a switch / bridge from TB to 10G I would like to ask what the experiences with such a setup here on the forum. Does TB networking via the Qnap show similar problems like between two Macs or does that work well?REPLY – once again, you are wrong. You should not connect your computers to the QNAP via thunderbolt. Connect via 10G Ethernet. Don’t worry – there are lots of NEW cheap 10G switches coming out (some are out) in 2018 that will allow you to have full 10G connectivity to the QNAP. And you are picking the wrong QNAP. You have a limited budget – you can get a better system from them for less money. And cheaper thunderbolt 3 to 10G adaptors are about to come out. Same with cheaper PCIe 10G cards. We are approaching NAB 2018, you know !
Another solution would be to get a 10G only box from Qnap or another vendor & more TB>10G boxes & a switch with more ports and connect everything over 10G ethernet. Is there anything that speaks against the mixed TB / 10G setup and for the 10G only?
REPLY – this is the correct setup. All computers get a 10G PCIe card, or thunderbolt to 10G adaptor. All these ports go into a 10G switch. The switch goes into the 10G port of the QNAP or Synology (remember, 8 drives, 16 gig of RAM minimum). And then IF you know how to set all this up, everything will work.
Hey – I made tons of mistakes when I started doing this. And guess what – I still make tons of mistakes. What you will learn is that just because someone releases a new product – GUESS WHAT – they don’t always work as advertised. This is what learning is all about – suffering. And when you eventually figure all of this out – then you can say “I know how to do this”. And I still make mistakes.
Bob Zelin
Bob Zelin
Rescue 1, Inc.
bobzelin@icloud.com -
Hi Bob
Thank you! I was really hoping you would reply to my post.
I know I did everything wrong. I even posted in the wrong forum. My question should have probably gone to the “NAS …” section ☺
Your answers confirmed what we experienced with our quick setup.
– Thunderbolt networking sounds promising, but doesn’t work.
– Also very good to know that the way to go is the regular 10GB ethernet and not a hybrid of Thunderbolt networking and 10GWhat we did was really just a quick attempt to setup shared storage with what we have lying around. Actually after what you write I’m rather surprised that the setup works well at least for the old Mac Pro, which is properly connected over 10G. The mixed 10G / 1G switch doesn’t seem to slow anything down. Of course Mac Pro and Mac Mini connect to the 10G ports. The mini is a 16GB model on OSX 10.12 and the Raid an 8 drive Areca model. Just yesterday we rendered a full feature film in Resolve on the old Mac Pro with Arriraw source footage on the “shared storage” and it pushed trough an average of 30fps.
We will upgrade to a proper 10G NAS setup in the next days and keep that mac Mini as an archive server / backup station / Resolve database server.
Shopping list:
8 Disk 10G NAS
Sonnet twin 10G Thunderbolt Box
5 or 8 port 10GB switch like Netgear XS508M or XS505MFor the NAS we are looking at the rather affordable 8bay models like the Synology DS 1817, QNAP TS-873U-8G (or similar) or Netgear ReadyNAS 628X.
In some reviews the Netgear performs best. Though the popularity on this forum seems to clearly go towards QNAP and Synology. Do you have any experience with the Netgear?
Our main interest is high transfer speeds for a low number of clients. What would be your recommendation to achieve this?
As the 10G cards and the NAS each have two 10G ports: Is link aggregation something that works and could help to improve speed?Thanks again for your help, Mr. Zelin.
-
Hi –
the QNAP TVS-873e is a brand new product. I have setup literally ONE of these, and this will be for demonstration at the NAB 2018 show at the Maxx Digital booth. In my initial tests, I get about 500 MB/sec write and about 1000 MB/sec read. This is with a single Mac Pro 6,1 computer using a Sonnet Twin 10G. That is all the testing I have done with this unit.I cannot tell you how well it performs with multiple systems. I am surprised at it’s low price point – it’s about
$1300 without drives for the 8G model. This of course requires the optional 10G card – the LAN-10G2T-X550.
More expensive QNAP models like the TVS-1282T and the TS-1685 come for free with the dual port 10G card, and 16
Gig of RAM. I can assure you that the TVS-1282T and the TS-1685 work great for video editing. SO – will the
TVS-873e work great – I don’t know. It’s my first time doing the tests on this new unit.I have setup several Synology systems, and they are excellent too. Please remember that with any system, you really need eight SATA drives and 10G ports for reliable playback. Getting a 4 bay from any manufacturer will do nothing for you. And without the 10G ports, you are wasting your time.
While I use Netgear 10G switches on literally every installation, I have never tried a Netgear ReadyNAS.
To my knowledge, Netgear only makes one series of rack mounted 12 bays with 10G ports, and they are about
$4000 without drives. Their smaller models to my knowledge, don’t have 8 drives, and without 8 drives, you are wasting your time.Both QNAP and Synology do heavy marketing and research for video applications. IT was at NAB that I saw the QNAP, and decided to test it. Unfortunately, Netgear does not participate in any video trade show, and has no exposure for the video market. Is it a great product – I just don’t know. Where did you read the ReadyNAS was better than the QNAP. IF someone is reviewing a product, and is not referencing the type of video work that we do – with Adobe, AVID, Davinci Resolve and FCP X – then it means nothing to me. I don’t care how great a product is with accounting data. That means nothing for our industry.
Bob Zelin
Bob Zelin
Rescue 1, Inc.
bobzelin@icloud.com -
Hi Bob
Thanks again for your reply.
The QNAP TVS-873e sounds interesting.
We already ordered the Netgear unmanaged 8 port 10G switch and an additional Sonnet twin 10G.
I think We will just get a Netgear Readynas and test it, and report back here.
Those units are in stock here and we can return them within 10 days if we run into problems.
The 628X is a unit with 8 drives and dual 10G ports. Stocked with total 80TB drives the cost will be less than 5000€, which is really a steal.
The Netgear UI looks much more nerdy and complicated than Qnap and Synology. But I could imagine that they have an advantage when it comes to raw data throughput. Also the Xeon D processors and 8GB ECC ram sound promising.There are not so many reviews about the Readynas 628x. One on storagereview suggests that the Netgear has a bit of an advantage in throughput against similarly priced Synology and Qnap NAS. Also even the benchmarks published by the manufacturers are a bit lower for the Qnap than for the Netgear.
As written I think we will just get one next week and do some real world testing.
-
I look forward to hearing how your user experience is with the Netgear ReadyNAS
Bob Zelin
Bob Zelin
Rescue 1, Inc.
bobzelin@icloud.com
Reply to this Discussion! Login or Sign Up