Globalsan iSCSI initiator vs Synology DS1511+ multipath I/O

[Glenn Venghaus]

p.s. If after the SNS and synology guys who are currently working on the problem can not figure it out and get it working to give me close to 200MB/s to a single workstation, i will start agreeing with you and might even spring for a macpro (as soon as a new model comes out)with xtarget.
But if it starts working even a newer macpro will not give me anything extra with 2x1Gb iscsi connections and i will happily stick with my inferior setup that gives me exactly what i need.

p.s. off topic but ssd’s rock !! Can’t have enough of those.

[Alex Mitchell]

@Glenn Venghaus

Hi Glenn, did you get Multi-Path working for you?

I have been testing out a HP N40L server solution as an iSCSI target using Global San iSCSI and Open Media Vault as the target OS.

I am currently trialling 3 x 3TB hdd’s as RAID 5 (so not great write but ok read) and I have been able to get reads of 150MB/s over dual ports. (Not maxing out connection due to only using 3 drives- (but they are expensive!)).

What was an important consideration was making sure that the NICS are on different subnets, GlobalSAN is great for MPIO as you simply ‘add’ another connection and point it to your 2nd Target NIC.

Let me know if this has helped or if I am on another planet with your current question.

However this sort of ‘aspect’ of computers is only going to get more important.

Currently my HP N40L has cost a bit more than I expected… quite a bit more… but I am happy with what it is pushing out with only 3 drives… will but crazy with 2 more!

YMMV

[Glenn Venghaus]

Hi Alex,

Thanks for your feedback man. Yes in the end i got it to work, but it required a new beta driver from Globalsan as it was the combination of globalsan/osx10.7.4/synology that did not work as expected.

Performance is ok considering the hardware, but still tweeking to get more out of it. Just hacking with some network settings (internal on the synology) can already make a big difference of tens of megabytes/s.

[Alex Mitchell]

Remember the performance of Software Raid (which is what I believe your box to be) is 100% dependent on the amount of drives. (more so than Hardware- as Hardware is fast with minimal drives).

I think that 7 drives is the suggested amount for SW Raid. And FAST arrays use 10’s of drives.

I have found reports of users pulling 260MB/s out of only 5 drives so don’t despair.

MPIO is totally worth it in my opinion.

Cheers,
Alex

[Paul Muller]

Sorry for re-opening the wound, but I am having similar challenges and wonder if/how you solved the problem.

After months of using a FreeNAS SAN and struggling with general flakiness and the feeling that I was running without a safety net, I switched over to a DS3612xs which features 4x 1GbE ports as well as a slot for a 10GbE NIC (which I have populated with an Intel x540T2 paired to a (wonderful) SmallTree dual-10GbE NIC on my 2009 12 core Mac Pro) and the Globalsan iSCSI initiator.

FreeNAS worked great with this setup and iSCSI MPIO (albeit I was never able to get it working faster than about 250MB/sec), but I can’t get it working on Synology DSM 4.3 (most recent updates installed).

Enabling multiple sessions on the DS3612xs results in a duplicate drives on being created – one for each instance.

What am I doing wrong?

[Glenn Venghaus]

You are doing nothing wrong. It is a known issue with Synology that the way they implemented iscsi and multiple network connections somehow does not work nicely together with MacOSX and multipath supporting iSCSI initiators. Did a lot of investigation on this together with SNS.
Its not only with the Globalsan driver that it gives issues. I was offered to test a SmallTrees initiator , which also did not work.
My guy feeling was after lots of test that it was down to 2 issue on the synology side.
1. bad isolation between the networks internaly , causing the 2nd network path to be reachable and advertised over the 1st path. Playing with the inbuild firewall and OS lavel hacking the synology gave mixed results on this.
2. The way a iscsi device was advertised by the target (maybe in combo with OSX as heared better stories on windows)), makes it somehow difficult for the initiator side to see that 2 devices are actualy the same device over 2 paths.

In the end the very friendly SNS guys worked with me for a long time and we got to a stable beta verions that worked for my unique combination, but this did not make it into the main release stream. For understandable reasons as they had to hack / bypass synologys issues for a very small group of their target users (look at the lack of comments on this thread or google search for mpio and you know enough) , that even could negatively effect other more widely used combinations.

For me this meant i froze my config at OSX 10.7.4 synology at DSM4.0 and Globalsan at an unreleased beta.
But its getting me stable 135-140MB/s over 2x1GB.

For my small 4 seat setup it is fast enough and I will make it till december (Mac Pro christmas time 😉

So unless now lots of synology users start using multipathing , which i do not really see happening, you are out of luck on the mpio front.

Maybe when 10G multi cards (like smalltrees) get more mainstream (and a lot cheape (maybe pushed by proper competition of thunderbold based storage networks in the future) , the whole mpio stuff gets the attention it deserves on more fronts. Its a magic bullet if used properly.

[Steve Modica]

I was playing with this not long ago. I think the issue is configuration on the target side. (this is freeBSD I was playing with). I saw similar problems.
FreeBSD 10 builds the iSCSI target code into the kernel, so everything changes anyhow.

Steve Modica
CTO, Small Tree Communications

[Paul Muller]

Thanks Glenn for confirming that I am not losing my marbles (at least in this case), if I were a betting man Synology’s approach to iSCSI probably works great with Windows and vmWare which is the bulk of the market anyway) and unless that’s broken there’s little commercial pressure to address it.

Steve – as usual you are a source of great help and knowledge. My 10GbE FreeNAS iSCIS implementation was never able to get any faster than about 2.5Gb/sec (R/W) even with 16 striped HDDs (or 4 striped SATA III SSDs) on the fastest controllers I could find. The STRANGE thing is that AFP/SMB with the same box easily hit 10Gb/sec R/W speeds.

So clearly my SmallTree NICs are NOT the problem (whew).

So, what is it about iSCSI that is such a pig? When I saw you at NAB you demonstrated your amazing FreeBSD based appliance that was using a sparse disk image with FCPX to get 10Gb/s speeds, what is the iSCSI performance like?

(to be clear, I would have bought one of those instead of a Synology were it not for the fact I needed more spindles than would fit in your great device!)

p.s. it’s perversely comforting to know that despite Apple’s success OSX remains a poorly supported operating system, somehow demonstrating we’re still safely on the fringes of respectable society 🙂

[Steve Modica]

Hi Paul

You’re not going to get screaming performance out of a stock FreeNAS kernel for things like AFP and SMB (especially not for video editing). They are setup as stock fileservers.

Steve Modica
CTO, Small Tree Communications

[Jordy Richir]

Hi Glenn,

I’ve read this topic and was wondering you still use this setup?

How does it performs now and did you tweak it more?

Is it possible to send me your beta driver so i can test it on my setup?

Thanks