Infortrend SAS RAID6 Running very slow

[Chris Murphy]

Spinning rust drives get slower as they get full, assuming the file system allocates sectors from outside to inside of the platter, which is usually how it works. And the difference is pretty significant. Some companies include the min and max in the spec, most don’t. I’ve seen the difference be upwards of 40%. What you’re reporting is a lot more than that though, more like a 70% drop in performance.

So that makes me wonder if one or more drives is having problems writing data and it’s having to retry a lot. I’d like to think that the RAID software has some testing options, maybe even a way to issue SMART commands to each drive, and get some idea what’s going on. If not then it means breaking the array and doing both read and write tests to individual drives, and that is a destructive test.

[Vadim Carter]

Jason, I concur with Chris – you have one or more drives that are going bad and the RAID controller keeps on retrying whatever operation is failing (most likely a write). Now would be a great time to back this whole thing to your LTO tape drive. ASAP. It is your insurance policy. After that you can either upgrade the drives to 2TB ones, I recommend Hitachi Enterprise drives, or pull your Seagate drives out and test them one by one by connecting them directly to a PC and running the SeaTools Hard Drive diagnostics utility on each drive. You should be able to isolate the bad drive this way. You can then put the remaining good drives back in your array and it should come up in a degraded state. Get a replacement drive, rebuild the RAID Volume and you’ll have the performance back.

Good Luck!

Vadim

Lucid Technology, Inc. / 801 West Bay Dr. Suite 465 / Largo, FL 33770
“Enterprise Data Storage for Everyone!”
Ph.: 727-487-2430
https://www.lucidti.com

[Jason Huxley]

Yes I think you’re right. I’m going to run the backup this weekend. Haven’t had time to get back to it yet. See if I can work out what’s exactly going on. I’ve been looking at the 7k400’s and they seem to have the best lifetime specs. It has taken me ages to get the software to be able to talk to the RAID. Infortrend have been super helpful via their online web chat. I’ll let you know how I go.

[Jason Huxley]

Hmmm. After further testing I’ve found today that it seems to be more complex than I thought. On my OLD Mac Pro with a LSI SAS3801E card I get upwards of 400 MB/s.. On my iMac, no more than 135MB/s. On my iMac I’m using an ATTO ThunderLink SH 1068. I’ve put on the latest drivers and Firmware and am still getting those speeds.. So it looks like my RAID is fine and it’s something to do with the Thunderlink. As far as I can tell they both have similar specs. The ATTO is meant to be up to 800MB/s throughput. Both the LSI and the ATTO have 2 x4 external SFF-8088 mini-SAS connectors.. So now I’m not sure where to start..

Any ideas anyone? Has anyone out there got an ATTO Thunderlink? I’m wondering if I’ve missed something basic..

[Jason Huxley]

Hi everyone. For those who are interested finally got this solved. It ended up going to ATTO as a bug and they took their sweet time solving it. Here’s the fix.

——

After some extensive replication and testing of your configuration ATTO have confirmed that there are some settings which should increase the performance of the 1068.

It is advisable to limit the transfer size that the 1068 driver uses. By default, the 1068 driver transfer size is unlimited. This unlimited transfer size seems to not work well with the Infortrend interleave (stripe size). Therefore, by limiting the transfer size in our driver, you are lining up the transfer sizes with the stripe size. I found that limiting transfer size to 256k or 512k was best in my setup. I suggest you try values between 256k and 2M on your setup to see what works best. To limit the transfer size, you need to issue the following command in terminal:

sudo nvram atto-args-tlsashba=”-i 0x00080000″

Then reboot. (NOTE: You must enter the command exactly as seen above).

The above example is for a 512k transfer size. The following values are what you should try:

256k = 0x00040000
512k = 0x00080000
1M = 0x00100000
2M = 0x00200000

You need to reboot the Mac after issuing the command each time. Therefore, issue the “sudo nvram” command above for one of the suggested values, reboot, then try and test. Then repeat for each value until you find the one that works best in your environment. 512 will usually give the best results for both read and write and 1M gives the best results for reads.

—

This solved my problems instantly and my RAID now runs as expected!! Took almost 4 months for the fix which was a simple terminal command! Hopefully they add these options to the driver at some stage so you don’t need to use terminal to change them.

Jason.