Real aggregate using built-in Mac Pro ethernet?

[Bob Zelin]

A. Are the built-in ethernet ports capable of a real 2Gbps aggregated link for a single program (FCP), or is this effectively just dual channel gigabit, helping with multiple transfers?

REPLY – no, who told you that? Why do you assume that. You can’t link aggregate the client end, just the server end to the managed switch. You are not going to get over 90MB/sec per client, if you put 2 or 4 or 6 ports in the client – you are stuck with the bandwidth of the ethernet protocol, and only jumbo frames will help you.

B. Will a dual port card like Small Tree’s offer a real benefit if the internal ports cannot?

REPLY – not on the client end .

C. Will a software solution like metaLAN help with either hardware approach? Their site mentions support for increased bandwidth via 802.11ad and sounds promising

REPLY – no, MetaLAN, and every other product follows the same rules. There are no magic tricks. You want faster than 90MB/sec (peaking over 100Mb/sec) with standard Gig E – move to another protocol, like FCoE, fibre, 10Gig ethernet, or Infiniband. And iSCSI is not faster either.

Bob Zelin

[Shane Winter]

Brilliant…that’s the definitive I needed.

To Confirm: Bandwidth is capped per Client-server connection. Server sees >90Mbps only by connection to several machines

So then…the only real benefit I have for my aggregated workstation links is fail-over

[Bob Zelin]

confirmed.

Bob Zelin

[Steve Modica]

Link aggregation is socket balancing. It’s 802.3ad (there are both static and dynamic variants and apple is dynamic only. Cheap switches are often static only).

Clients only use a single socket to connect to the server. The spec requires that each “conversation” (aka socket) must stay on a single port for its lifetime unless there’s a failover event. So when you connect to the server, the socket is going to use one port.
The socket coming back from the server may use the other port. So in some instances when you have bidirectional traffic (like during a render with source and destination on the server) you *may* see traffic going on both ports. however I can’t guarantee this since Apple has a frustratingly random 802.3ad assignment algorithm. if they offered more flexibility there, we could lock it down and use this effect.

Under “normal” circumstances, you might see traffic on one port going out and the acks (lots of tiny packets) coming back on the second port.

I haven’t spent any time looking at the 10.6 802.3ad driver. Perhaps I can hack that to offer a way to lock in unidirectional traffic on each port with our edge-core switches. I’ll check.

Jumbo frames are obviously a good thing to use.
Also, with our new 10Gb cards we have RSC supported. So we can aggregate incoming frames to the server. That’s a nice feature that helps performance (especially with the imacs that don’t do jumbo frames)

Steve