Slightly OT – Curious about effective link aggregation (trunking) in OSX

[Eric Hansen]

not off-topic at all.

Steve Modica can chime in with the technical explanations, but basically, OS X can’t benefit from link aggregation on the client side. it can only support a single connection to a server, in this case, your NAS.

doing a link agg from your NAS to your switch has benefits however, as it doubles the number of links available to outside clients using the same IP address (it doesn’t go faster, it just makes more connections available)

as far as seeing double the speed, and high bursts, I believe this is the OS sensing more than there really is. If you setup a RAID-1 in disk utility on 2 internal drives, and then copy a file to it and watch it in Activity Monitor, you will see a read speed of 80MB/s (for example) and 160MB/s write because the system is writing to the 2 drives in the RAID-1 and counting that as 2 streams. i think if you link agg on the client side, you will see a variation of this, and that’s why it still takes 1:41 to copy the file.

hopefully steve chimes in with a more technical answer

e

Eric Hansen
Production Workflow Designer / Consultant / Colorist / DIT
https://www.erichansen.tv

[Ed Murphy]

Thank you, Eric.

I suspected it was something like this, and Petros Kolyvas just gave me the “one train can’t ride on two tracks but two trains can” analogy in another forum’s old thread I had revived asking the same question.

So my follow up question is: If I easily have the means to do so since I’ll be upgrading switches anyways, is it totally pointless to set up link agg on the four MacPro clients I have? Or is it possible that would be helpful in scenarios other that a single file transfer?

It sounds like it would possibly help in situations where one workstation was sending or receiving data to multiple NASs or other workstations in terms of a file transfer or similar. Is it also possible though, that the OS/network/NLE (PPro in this case) would be smart enough to use those multiple links when working with multiple sets of media? Perhaps only when that media is in different locations?

Thank you again for any thoughts.

Ed Murphy
Senior Editor / Technical Director
David Lynch Foundation Television

[Bob Zelin]

https://forums.creativecow.net/thread/197/858151

Bob Zelin

[Bob Zelin]

from the Small Tree site

Q: CAN YOU TRUNK WITH LINK AGGREGATION FROM THE SERVER TO THE CLIENT AND GO TWICE AS FAST?

A: No. This is a common misconception. Many inexperienced Apple resellers will make this mistake, which is why you want to talk to a networking expert before installing any serious infrastructure.

Link aggregation works by load-balancing “sockets” (referred to in the IEEE 802.3ad spec as “conversations”) over all of the available ports. On a server, this works wonderfully since a server might have 10 clients connecting. Each of these 10 sockets can be routed over a different port, providing an evenly-loaded network with a higher aggregate bandwidth.

Clients generally connect to one server. Hence, if you run link aggregation on a client, this one socket (conversation) will run over one of the ports. You might see some traffic on the second port if you’re also checking e-mail or viewing web pages, but you won’t see 2X the bandwidth to the server. Some people implement link aggregation on clients, but it’s primarily for redundancy (fail-over capability).

[Bob Zelin]

so, with all that said –
I have actually casually discussed this with Steve Modica at one point, but never got into it, or understood it, BUT –
I am right this second working with an AVID ISIS 5000 at Disney, and I can take two ethernet cables from the two ports of the Mac Pro, assign static IP addresses to them, and shove them into the Dell Force 10 S25 switch, and get about 170MB/sec bandwidth from a Mac Pro WITHOUT using LACP. I have NO IDEA of how this works, but AVID said it would work, and they were right. I wish I knew how they did this.

Bob Zelin

[Eric Hansen]

Bob wrote: “I wish I knew how they did this.”

by charging a lot of money 🙂

e

Eric Hansen
Production Workflow Designer / Consultant / Colorist / DIT
https://www.erichansen.tv

[James Mckenna]

“by charging a lot of money :)”

That’s right. The client software you’re paying for has the ability to break up the requests along a dual paths. Each port has it’s own IP address. I was out of Avid way before ISIS became a shipping product, but I’ve heard that this is the way it’s done, and that makes sense to me. Personally I think 10GB-baseT over copper 6a is the more elegant solution, and optical is even better (then FC becomes an option).

In the end it doesn’t really matter for that product, even if you can push 190MB/sec on a single client, you can do that on all of TWO clients before your ISIS 5000 drops to it’s knees.

Jim McKenna
Facilis Technology
978-562-7022 x101

[Ed Murphy]

Thank you for the great information, Bob. I read the thread you linked and appreciate having a better understanding.

It seems most points suggesting it wont help performance are citing the scenario of many clients talking to one server, and you said:
“Clients generally connect to one server. Hence, if you run link aggregation on a client, this one socket (conversation) will run over one of the ports. You might see some traffic on the second port if you’re also checking e-mail or viewing web pages, but you won’t see 2X the bandwidth to the server. Some people implement link aggregation on clients, but it’s primarily for redundancy (fail-over capability).”

However, in our environment we have several clients connecting to 2 or 3 servers, (NASs) and sometimes to each other. If my clients were link aggregated, would that offer the occasional performance increase when dealing with multiple connections, just as it does on the NAS side with multiple clients wanting to connect?
I totally understand now that I’m not doubling my throughput over any one connection, but it seems like this would still be a minor performance boost in our environment.

If nothing else, there’s really no down side to doing it if I have the ports and the coper anyways, right? I mean aside from my time spent of course, but then, heck we’d have fail-over too…

Thanks again for everyone’s input!

I also have a related question.
Could I uplink via 10GbE to one of my NASs to at least improve performance to/from that node? I originally thought uplinks were only meant for other switches.
What I imagine in asking this question:
Our largest NAS is the Qnap 879 Pro, which has the option to install a 10GbE NIC for about $300. I was previously looking at 24 port HP 1810 series switches, however those don’t appear to have 10GbE uplink options. If I were to instead go with one that does appear to, such as the Cisco 2960-S line, or something from Small Tree, could I theoretically uplink via 10GbE to our 879 Pro for greater performance of at least that media storage to the rest of the network?
Forgive me if this is an ignorant question.

Thank you again for your thoughts.

Ed Murphy
Senior Editor / Technical Director
David Lynch Foundation Television

[Matt Geier]

Hi Ed,

Link Aggregation on the client won’t work out of the operating system the way you’d expect.
For you to link aggregate those client ports and pass data down both at the same time, requires the software application to support this, much like the reference to AVID Isis.

If you set this up out of the os without something special from the software on the client side, you’ll see data move over 1 of the links. The other will be connected and appear to be doing something at the same time, but if you watch the io you’ll see what I mean.

Link Aggregation on the server will be what you look to do, even if you have more then one server, you just set them all up with link aggregation to your managed switch.
This is where most folks will implement link aggregation, even for six ports of 10GbE in some cases, not just 1GbE networking.

Smart Switches are a one step up from Unmanaged Switches. They have some things in common. Most of them cannot be set up to properly support Dynamic Link Aggregation which is what Apple O/S uses by default. Dynamic is the best for cases of shared storage most of the time. These kind of switches also don’t have high packet forwarding rates and lack in overall processing power to manage the data properly, leaving you at the mercy of the switch performance and what it does do.

They should not be used for video or traffic/data intensive networks. It’s not the right choice.

Perhaps calling some of the companies and speaking to the vendors will help you better understand this and get some clear cut answers to your questions.

Feel free to call Small Tree or shoot over to our creative cow forum for more consult on the subject. I represent their West Sales region of the US and Canada. Maybe connecting to get some user references and other resources is what you need too.

Let us all know what else we can do to help further.

Best regards, and good luck,

Matt Geier
(Video Networking Solutions Expert)
(Creative Design Workflow Consultant)
(Social Media Networks Consultant)
(Technical Video Industry Sales Consultant)

[Jon Thomasberg]

Channel bonding is certainly possible to use more than 1 link between: host and NAS (which is technically just another host acting as a iSCSI target, CIFS, AFP, or NFS server); host and a capable enterprise-class switch (we are talking Arista, Cisco Nexus, etc. not some cheap thing from your local big box retailer). This is accomplished either via specialized software (as already cited) or by configuring Channel Bonding in ‘balance-rr’ mode (a.k.a mode 0). Be aware that TCP sessions will have packets arrive out of order and require significant retransmits, reducing the aggregate bandwidth to less than the sum of the links, but still significantly more than a single link or attempting to combine the throughput of two links using LACP, as many of you all have tried and realized that it is no faster than single link, this is due to the fact that LACP uses a per-destination flow over only 1 link, regardless how many links you aggregate in the LACP. This is done because traffic is segmented in chunks (size defined by maximum transmission unit, or MTU) and each segment is given a sequence number so that the segments arrive in order for reassembly on the other end.

So back to ‘balance-rr’ (load balancing, round-robin) mode: All the links in the bonded virtual interface must be of the same speed, duplex, MTU, buffer size, etc. or it won’t work. This works in Linux and in Mac. I am not certain how to accomplish this in Windows, or if it is even possible.

If you have linux, you can read up on the topic here:

/usr/src/linux/Documentation/networking/bonding.txt

…pay special attention to section 12.

This is the beginning piece. However, if you are savvy in networking and you feel comfortable editing .conf files in linux CLI, you should be able to accomplish this quite simply. On the other hand, if what you have read above sounds like Greek to you, you can: A) start reading up on it, learn it and test (on a sandbox, non-production system); or, B) hire someone knowledgeable to do this.