Fileserver for VFX Compositing and 3D.

[Murat Karslioglu]

Hi Roberto,

The cheapest and fastest improvement you could make would be getting Intel X520-T2 copper 10G cards and keep using the existing setup until the next upgrade.

This is the practical limit for SMB2 on 1G network, you need to switch to 10G network, iSCSI or FC.
Using the same hardware you can get 100MB/s between two linux systems, so in this case SMB is your bottleneck.

Before Windows Vista it was even slower (max 30-50MB/s), thanks to the SMB2. SMB2 improved performance with large file transfers over 50%.

Point to point is completely different story on Windows systems. 10Gb p2p is disappointing on Windows too, you are lucky if you get around 600MB/s on 10G p2p connection over SMB. Of course you would need a fast local storage on your file server, otherwise your limit is the storage right now. If you need to keep copper connection Intel 10G NICs are ok. (If looking for SPF+, Emulex and Myricom are much better)

What is the RAID controller you are using? 600mb seq/read is also kind of slow for 3G controllers and enterprise drive performance. It should be around 800MB/s with 8 drives configured RAID5.

If you are looking for a “fast” storage for a production system such as real-time 4k editing, etc. with a 6G JBOD (16 SAS drives) using 6G LSI or ATTO controllers I get around 2400MB/s seq/read and 1900 seq/write (it is fast).

But if you are looking for a shared storage, you should consider getting a 8G FC SAN or 10 iSCSI SAN device or converting your existing server into one (let me know i you need more info how to do that).

As you may know FC requires expansive cabling and FC HBAs, but it is reliable and you can get around 600-800MB/s from a decent FC SAN solution.

iSCSI is easy, since it does not require special cabling and uses regular network, but with 1G iSCSI you cannot get higher than 90-105MB/s from a single connection either, you may team the ports and double or quadruple it, but it is not suggested.

10G has the same fibre cabling requirement as FC SAN, 10G NICs are easier and cheaper to get.

Let me know if you need more info or detailed specs, test result. It is what i do every day 🙂

Thanks.

Murat Karslioglu | Co-Founder and CTO
RAIDUNDANT LLC
112 W. 9th St. Suite 226
Los Angeles, CA 90015
Tel: 213-489-9041
http://www.raidundant.com

[Roberto Esperata]

Hi Murat!

Thanks you very much for your answer!!! It helps me a lot.

This machine will only serve sequence files for 6/8 compositing seats. Our ONLINE suites work independently with their own network and local RAIDs. We did some tests using 10G for “server to Switch” connection, but for our use and RAID speed it seems to be the same as 4 Giga connections with Teaming.

Our most importante question right now is: Is it normal to copy at 90MB/s when grabbing a 100 file sequence (8MB each file) from any Windows 7 but then find 45MB/S limit when doing a ram preview from Fusion or After Effects from the same sequence? You mention Samba as 30/50mb/s and Samba2 as higher. Do you think samba is the limit? Could ISCSI do better?

Thanks for all your help once again!
Roberto.

Roberto!

[Roberto Esperata]

Murat I’m a newby and my emails take long to appear, so just to make it faster another question: Do you think going to Win 2008 Server might help in any way?

Regards and thanks again!
Roberto

Roberto!

[Roberto Esperata]

Murat,

I was confusing SMB and Samba. We had Samba3 with SMB1, but now with SMB2 nothing changed.

Do you think iSCSI would help or we even worst?

Thanks in advanced!
Rob.

Roberto!