Update on our SAN – looks like os X.3.9 on G4’s are no go.

[Francois Stark]

Our G4 was running on os X.3.9:

All 8 LU’s in the system profile but only 2 in the disk utility.

And here I tried it in Tiger X.4.1:

In Tiger also all 8 LU’s in the system profile, but only 2 in disk util.

So here I did a raw installation of X.3:

And suddenly I could see all the LU’s in the disk util.

Finaly I went up to os X.3.4 and this is what I saw:

All the LU’s in disk util! I installed sanMP and could mount and read and write to the shared fibre arrays.

Conclusion: Os X.3.9 and X.4.1 are not able to see LU’s properly on a ADTX array on a G4 mirrordoor. Os X.3.8 works in G5 machines.

At the moment I’m getting 6 streams SD from the 10 drive raid 0 array (two 5 drive raid 0’s striped by apple disk util). I can play 6 streams on one G5 in FCP real time, or two streams each on three machines (two G5’s and one G4). This is quite reliable.

However I have had three crashes on our G5 while capturing SD PAL to the SANMP fibre raid. FCP stops with disk IO error, and then finder also does not see the SanMP disk anymore. Everything freezes and I have to do a hard reboot. SanMP recovers after it’s 15 second delay because I did not unmount properly.

So there is still a reliability problem when capturing – more tests to follow.

Regards
Francois

[Bart Harrison]

Hi Francois.

I wish I had time for a quick trip to SA. I’d love to get to the bottom of this problem you’re having. I’ve got many ADTX/SanMP installations all over North America some running perfectly for more than eighteen months. For me it’s been the real winning combination. Other than the fact that I refuse to use the Apple/LSI HBA’s I don’t see that you’re doing anything different. Why don’t you see if someone can loan you an Astera Rhino 3000 or an Atto Celerity 22XH to test out. FYI: the Astera card uses it’s own proprietary set of drivers while the Atto card uses parts of the Apple FCAL library. Either way it’d be a good test. By-the-way which version of the Apple cards do you have ? Rev 2 has a /B in the part number and Rev 4 has a /D in the part number. The /D cards were a nightmare of unexplainable problems for me, LUNS showing up somtimes and not showing up at other times (sound familier). I switched to the Astera cards and now everything works.

Bart

– – – – – – – – – – – – –
Bart Harrison
MPA – The HD Suite

America’s VAR
TurnKey Editing Systems, Storage Area Networks
HD Consulting, Production & Post, Exhibition & Distribution
http://www.hdsuite.com
954-894-1221

[Francois Stark]

Here I re-did the post because the previous links worked for our internal LAN only:

Our G4 was running on os X.3.9:

All 8 LU’s in the system profile but only 2 in the disk utility.

And here I tried it in Tiger X.4.1:

In Tiger also all 8 LU’s in the system profile, but only 2 in disk util.

So here I did a raw installation of X.3:

And suddenly I could see all the LU’s in the disk util.

Finaly I went up to os X.3.4 and this is what I saw:

All the LU’s in disk util! I installed sanMP and could mount and read and write to the shared fibre arrays.

Conclusion: Os X.3.9 and X.4.1 are not able to see LU’s properly on a ADTX array on a G4 mirrordoor. Os X.3.8 works in G5 machines.

At the moment I’m getting 6 streams SD from the 10 drive raid 0 array (two 5 drive raid 0’s striped by apple disk util). I can play 6 streams on one G5 in FCP real time, or two streams each on three machines (two G5’s and one G4). This is quite reliable – I looped these six playing streams for about 1 hour.

However I have had three crashes on our G5 while capturing SD PAL to the SANMP fibre raid. FCP stops with disk IO error, and then finder also does not see the SanMP disk anymore. Everything freezes and I have to do a hard reboot. SanMP recovers after it’s 15 second delay because I did not unmount properly.

So there is still a reliability problem when capturing – more tests to follow.

Regards
Francois

[Francois Stark]

Hi Bart

Thanks for the encouragement – I seem to be sorting the problems out one by one. Sad about having to go back to an earlier version of os X, but someone said apple changed fibre port discovery from os X.3.7. So it seems I can go back to X.3.6.

I don’t know what’s going to happen with tiger – I know SanMP is not ready for tiger yet, but that cannot be far off. And if this problem is not solved, I’ll be stuck in Panther on the G4’s. And FCP 5 does not run on X.3.6; it needs at least X.3.9…

Q: Did you notice any difference between G4 and G5 machines on the Astera cards? What version of os X are you running?

I’m at home now, so I’m not sure about the rev of apple fibre card. Will check tomorrow.

On the G5 that froze during capture: I’m running the Apple fibre card (LSI) in the top slot (PCI-X 133MHz by itself), and the decklink extreme with the Atto UL4D in the second and third slots (sharing PCI-X 100MHz). I copied some files off the Huge 320S array to the fibre array and got 135MByte/s read from Huge, and 135MB/s write to the Fibre array. No crashing.

The only time it froze was during capture. I’m going to try the decklink card in the top slot (to isolate it from other cards), and the LU4D and Fibre cards in the second and third slots.

Hopefully, if I can get this fibre array working very reliably, I can sell the UL4D and Huge array – which means I will only have the fibre card and Decklink installed.

struggling on…
Francois

[Bart Harrison]

[Francois Stark] “Q: Did you notice any difference between G4 and G5 machines on the Astera cards? What version of os X are you running?”

Sorry, we retired/repurposed the G4’s at all my SAN installation so far.

[Francois Stark] “Hopefully, if I can get this fibre array working very reliably, I can sell the UL4D and Huge array”

What I’ve done at a number of SAN installations is to set up one of the old G4’s and the Huge array as a support server tied to all the editing/graphic systems via an inexpensive Gbit Ethernet switch. I set them up as an NFS volume which appears on each of the editing systems as an “internal” folder on their system drive. At one site this server is hosting 700MB of music (about six music libraries) plus all of their still graphics assets. The music and graphics play right off of the support server without consuming any of the valuable SAN resources.

Bart

P.S. When running a dual controller ADTX, a QLogic 5200 switch, and a dual channel Astera 3000 (with dual 2Gbit fibre) we usually see well over 300MB/sec read and write at every workstation. If you’re only seeing 150MB/sec then something is definately wrong !

– – – – – – – – – – – – –
Bart Harrison
MPA – The HD Suite

America’s VAR
TurnKey Editing Systems, Storage Area Networks
HD Consulting, Production & Post, Exhibition & Distribution
http://www.hdsuite.com
954-894-1221

[Bart Harrison]

[Francois Stark] “What version of os X are you running?”

I have versons of OSX ranging from 10.3.4 to 10.3.7. I’ll be upgrading the 10.3.4 installation (my oldest) to FCP 5 next week. We might try 10.4.1 just as a test although we’ll probably go with 10.3.9 for the time being. I’ll keep you posted.

Bart

– – – – – – – – – – – – –
Bart Harrison
MPA – The HD Suite

America’s VAR
TurnKey Editing Systems, Storage Area Networks
HD Consulting, Production & Post, Exhibition & Distribution
http://www.hdsuite.com
954-894-1221

[Francois Stark]

Hi Bart

I found where Jeff Bernstein talks about the change from os X.3.7:
——————————————————————

Name: Jeff Bernstein
Date: May 27, 2005 at 04:44 gmt
Subject: Re: Not solved yet – seems G5 is better.

Okay, this is starting to sound a little familiar. I have a feeling you are running OS X 10.3.7 or greater. This could mean you are not running the latest firmware on the ADTX in which half the LUNs would not mount due to the way OS X changed the behavior in LUN discovery.

Another thing to note is that there is a nasty bug on the ADTX when used with SANmp. If your ADTX needs to rebuild, you not be able to recover the data. We just discovered this the hard way. ADTX is working on it and I welcome you to apply some pressure for I would hate for this to happen to you.

Let us know how it goes.

Can you tell us which OS and which rev of the firmware on the ADTX?

Jeff

Jeff Bernstein

Digital Desktop Consulting
Apple Pro Video VAR
XSAN Certified

323-653-7611

[Bart Harrison] “set them up as an NFS volume”

Why NFS? Maybe NTFS? for PC graphics machines? (we all make typo’s sometimes…) or is it really NFS?

[Bart Harrison] “we usually see well over 300MB/sec read and write at every workstation” Today I got about 180MB/s on the decklink disk speed meter. Remember, only 10 drives (two 5 drive raid 0’s striped using apple disk util), not a fully populated ADTX yet.

On your ADTX settings: What is your setting for cache pre-read streams? Mine is set to 32 at the moment, default is 8. If I remember right it can go up to 128? I’m not interested in HD streams (yet), I need as many SD (20MB/s)uncompressed streams as possible.

Thanks
Francois

[Bart Harrison]

[Francois Stark] Jeff: “half the LUNs would not mount due to the way OS X changed the behavior in LUN discovery.”

With the Astera 3000 it’s the card/driver that does the LUN discovery not OS X. When using the Apple/LSI cards you’re subject to the whims of Apple who as you know are continuously changing things (driver revs, OS revs, hardware revs). This is why I’ve always preferred the Astera card. Unfortunately I’m not sure if they’re going to be making them any more.

[Francois Stark] “Why NFS? Maybe NTFS? for PC graphics machines? (we all make typo’s sometimes…) or is it really NFS?”

Yes, NFS (the Unix Network File System). Because of it’s Unix core NFS networking is already built into OS X. Of course if their are a lot of XP systems I’d go with SMB, but in a primarily OS X or Unix/Linux environment nothing beats NFS for convenience. Please be mindful this has nothing to do with the underlying disk format. You can share a Mac OS Extended, FAT 32, or NTFS volume via any networking topology you have available. If you excuse the expression, “they are apples and oranges”.

[Francois Stark] “Remember, only 10 drives (two 5 drive raid 0’s striped using apple disk util), not a fully populated ADTX yet.”

I have never recommended any 10 drive array in a shared-storage SAN environment. Every SAN installation I have is set up with 7 drives on one controller (Raid 5) and 7 drives on the other. I create 12 LUNS on each side and stripe them together to create 12 OS X volumes. The 15th drive is used as a hot spare.

[Francois Stark] “On your ADTX settings: What is your setting for cache pre-read streams? Mine is set to 32 at the moment, default is 8. If I remember right it can go up to 128?”

I set the ADTX with Cache’ Mirroring off, Write Cache Enabled, and the Look Ahead Table to 128 Entries. I arrived at these settings through extensive performance testing and consultation with ADTX and Astera engineeing. (Of course, your’re milage may vary.)

Hope this helps !

Bart

– – – – – – – – – – – – –
Bart Harrison
MPA – The HD Suite

America’s VAR
TurnKey Editing Systems, Storage Area Networks
HD Consulting, Production & Post, Exhibition & Distribution
http://www.hdsuite.com
954-894-1221

[Francois Stark]

Hi Bart
thanks for this discussion.

[Bart Harrison] “You can share a Mac OS Extended, FAT 32, or NTFS volume via any networking topology you have available.”

When sharing over SMB to PC’s we can only access the user;desktop etc. We can not see any non-boot drives. So we’d have to get os X server software.

[Bart Harrison] “7 drives”

Working towards that.

[Bart Harrison] “Look Ahead Table to 128 Entries”

I’ll try that today.

Regards
Francois

[Christopher Tay]

Hey Bart,

Since Astera is no longer making the Rhino FC cards, which alternative will you be looking at ?

Have you tried the ATTO Celerity series ?

-chrispy