Creative Communities of the World Forums

The peer to peer support community for media production professionals.

Forums Storage & Archiving Two new Mac Pros, two Thunderbolt 2 RAIDs, one Thunderbolt Bridge ….

  • Two new Mac Pros, two Thunderbolt 2 RAIDs, one Thunderbolt Bridge ….

  • Neil Smith

    January 6, 2014 at 9:46 am

    Apologies guys, just seen that Bob Z referenced the same Ars Technica article in an earlier thread below … should have read that before I posted.

    Have sent an email to Iljitsch asking him if he’d care to join in our discussion.

    Neil

    Neil Smith
    CEO
    LumaForge LLC
    faster storage faster finishing
    323-850-3550
    http://www.lumaforge.com

  • Neil Smith

    January 6, 2014 at 11:40 am

    Just got a reply from Iljitsch … it would appear that’s he’s having some trouble signing up to the Cow.

    Below is a snippet from his email reply … as soon as he gets signed up he’ll join in the thread … agree with his comment on the linked to blog about the need for 10GbE port on the back of the nMP … was thinking the same thing yesterday while plugging in a gig E cable for Xsan config.

    I’m waiting for a Tbolt2 to PCIe expansion box to come in next week … when it does, I’ll put a 10GbE Myricom card in it and connect to a 5,1 Mac Pro and Win 7 PC and see how that goes.

    From Iljitsch email:

    ” …. I tried to sign up for an account but I didn’t get an email.

    What I wanted to add to the discussion is seeing if the TSO can be turned off (using ifconfig?) so many small packets are used rather than fewer big ones. Perhaps this will help. I don’t think CPU utilization is an issue here as even on those < 3 GHz dual core machines I was able to get impressive speeds some of the time.

    And I’m interested to hear what you guys think about this blog post:

    https://www.muada.com/2014/01-03-the-mac-pro-needs-10-gigabit-ethernet.html …”

    Neil

    Neil Smith
    CEO
    LumaForge LLC
    faster storage faster finishing
    323-850-3550
    http://www.lumaforge.com

  • Iljitsch van Beijnum

    January 6, 2014 at 11:44 am

    Thanks for the invite!

    It’s very disappointing that this issue persists with the latest and greatest hardware. I was hoping it was caused by an immature Thunderbolt implementation in my 2011 MacBook Air, but apparently not.

    One thing I wanted to try was disable TCP segmentation offloading (TSO) to see if that would help but unfortunately the ifconfig en2 -mediaopt tso command doesn’t work.

    I don’t think CPU usage is an issue here, as even between my sub-3 GHz laptops with only dual core CPUs the speeds can be good, they’re just not consistent.

    It’s too bad that this doesn’t work as it should, as other ways to do > 1 Gbps networking on the Mac Pro are cumbersome and expensive, as I wrote in a blog post the other day:

    https://www.muada.com/2014/01-03-the-mac-pro-needs-10-gigabit-ethernet.html

  • Chris Murphy

    January 6, 2014 at 5:27 pm

    TSO is typically done in hardware, hence “offloading”, meaning the CPU doesn’t do it. In this Thunderbolt IP bridge arrangement, there’s no hardware to do the TSO in this case, it’s done by the CPU no matter what. So it’d only be a choice of where the segmentation is being done, and it stands to reason they may have more efficient segmentation code at a lower level, so they simply don’t allow it being disabled.

  • Iljitsch van Beijnum

    January 6, 2014 at 5:34 pm

    Right. I just wanted to see if it made a difference just in case there was some bug in that code or some kind of unexpected interaction that is triggered by the very large segments, because those don’t happen in “normal” networks.

  • Alex Gerulaitis

    January 6, 2014 at 5:35 pm

    Would artificially setting the packet size below MTU size help figure if it’s TSO that’s the problem? E.g. doing a disk or iperf benchmark with block size at 1K?

    Just wild guessing that this might help limit or eliminate segmentation and if we see less choppy bandwidth (although perhaps much lower due to high latencies) – that will pinpoint probable cause?

  • Iljitsch van Beijnum

    January 6, 2014 at 7:08 pm

    Probably not. The way TSO works in network interface cards is that the software hands over a big packet and then the NIC slices it into MTU-sized chunks. In that case, lowering the MTU makes no difference at all to the sending TCP stack, which is in charge of setting the transmit rate.

    But I suspect what’s happening with Thunderbolt networking is that the segmentation step is simply skipped. In that case the MTU value is completely meaningless.

  • Chris Murphy

    January 6, 2014 at 7:12 pm

    Does this virtual NIC support jumbo frames? Maybe I missed that part.

    We know that Thunderbolt 2 can produce a sustained high transfer rate greater than 1000MB/s. And we know a hardware 10gigE NIC can produce a sustained high transfer rate also. So the problem seems to be somewhere in the virtual NIC implementation which is in the kernel.

    Linux has been doing TSO in paravirtualized NICs for a while, and getting well more bandwidth between virtual machines than is needed for the use case in this thread. So it seems it could work here too. But it’s also a matter of optimization.

  • Alex Gerulaitis

    January 6, 2014 at 7:26 pm

    [Iljitsch van Beijnum] ” lowering the MTU makes no difference at all to the sending TCP stack, which is in charge of setting the transmit rate”

    I meant lowering the block size (below MTU) before it gets sent to TCP. Eliminate TSO overhead.

  • Chris Murphy

    January 6, 2014 at 7:33 pm

    Umm, pretty sure MTU still applies no matter what. This isn’t a case where we can skip physical and link layers by bonding two network layers together and just share large blocks. That the physical layer is virtualized doesn’t mean we can break the laws of ethernet and share 64KB packet sizes at that layer. I don’t think.

Viewing 11 - 20 of 35 posts

Log in to reply.

We use anonymous cookies to give you the best experience we can.
Our Privacy policy | GDPR Policy