[Steve Modica]

10Gb itself goes line rate. You can fire up a benchmark like iperf and show that pretty easily. The problem lies in pulling data off of disk, segmenting it and getting it out to the network. There’s significant overhead in that (and reversing the process on the other side).

Our 10Gb products are doing segmentation offload now and receive side coalescing, so that helps. I have not run new benchmarks to see if things have gotten a lot better.

I think the main code improvements need to be in the Samba/AFP code. This isn’t an apple issue either. They all have these limitations.

10Gb running a block protocol like FCoE or even iSCSI goes pretty close to line rate.

Steve Modica
CTO, Small Tree Communications

[Steve Modica]

In the iSCSI or FCoE world, we would tell using:

sar -n DEV 2 100

This samples 2 seconds at a time (100 times) and outputs the per second average IO over each port. (You run this in a terminal)

This won’t work for FC stuff since they aren’t ethernet (They are EtherNOT!!!)

If your FC drivers came with utilities, you might find some way using one of those.

Steve Modica
CTO, Small Tree Communications

[Steve Modica]

The opensource mediaharmony code is used by at least one enterprise vendor I’m aware of.
We’ve been spending a lot of time with that code lately.

Steve Modica
CTO, Small Tree Communications

[Steve Modica]

In answer to my own question:
The 2 RAID0 striped drives in the destination stations will be the bottleneck. If they are SATA devices I think you’ll be topping out at 140MB/sec best case.

Steve Modica
CTO, Small Tree Communications

[Steve Modica]

Jumbo frames could help, although that will mostly be cpu reduction.

10Gb can probably help with a couple provisos:

1. Using a normal copy, there’s only one cpu thread running to push the data across. This one thread will top out at around 300MB/sec. This is a function of how fast a single cpu can turn the TCP crank. Faster cores could make this a little faster.

(FCP and quicktime use AIO routines which work in parallel. Servers get the advantage of having many open sockets, so more cores are brought to bear)

2. You might not get 300MB/sec if the storage on both sides can’t handle that bandwidth. You won’t go faster than the slowest element.

Steve

Steve Modica
CTO, Small Tree Communications

[Steve Modica]

The links are presented as a single logical channel. That is true.
The 802.3ad spec requires that a “conversation” must exist on one port. (This is to maintain TCP ordering. TCP stacks cannot deal with lots of out of order packets. That’s an exception condition).

So what happens is a socket opens and gets assigned to a port. The only time it will ever hit another port is if the first port fails.

Many people think LACP acts like a striping utility with the packets. This can’t work because packets 1 2 3 and 4 would end up on different ports and would arrive “out of order”. The stack would go bonkers.

At SGI, we did this experiment. We actually wrote a driver to do this. It took 3 CPUS to handle 2 striped gigabit ports and if we added a 3rd port, it couldn’t go any faster (the re-ordering caused a bigger slowdown than the new ports additional bandwidth).

This is a problem people have wanted to solve forever. SGI’s NUMA and several other protocols that use “scheduled transfers” and RDMA like buffer splitting were created. However they were all etherNOT and etherNOT never wins. It required special hardware and rewritten stacks etc.

Steve Modica
CTO, Small Tree Communications

[Steve Modica]

Link aggregation won’t work in a scenario like this. It’s basically a socket balancing thing. When clients connect to servers, they open a socket. That socket gets assigned to a port. As more clients come in, they get randomly assigned and are load balanced.
On the client side, there’s only ever one socket, so you won’t see any additional bandwidth. Does that make sense?

Steve Modica
CTO, Small Tree Communications

[Steve Modica]

First thing would be to try and ping them. (Look at their ip addresses in the pref panel and open a terminal to type “ping “. If that doesn’t work, then bonjour clearly won’t. If it does work, then it may be that sharing is disabled or that the preferences on the finder has sharing off. to ping, you have to broadcast the arp request. So if you can ping, broadcast works.

Steve

Steve Modica
CTO, Small Tree Communications

[Steve Modica]

You should download the bonjour browser and run it. It will show what’s out there broadcasting that it can share. You should see all your macs. If they are not in the same broadcast domain, you won’t see them.
Steve

Steve Modica
CTO, Small Tree Communications

[Steve Modica]

We’re going to try Vbrick.

Steve Modica
CTO, Small Tree Communications