Creative Communities of the World Forums

The peer to peer support community for media production professionals.

Forums Storage & Archiving Update on our SAN – looks like os X.3.9 on G4’s are no go.

  • Update on our SAN – looks like os X.3.9 on G4’s are no go.

  • Francois Stark

    May 28, 2005 at 4:39 pm

    For those of you following my SAN installation testing from the XSAN and OS X forums:

    Yesterday we figured out that the secondary controller in the ADTX was faulty.

    Last night I realised that when we had bypassed the switch and connected straight to port 3 to test it(on controller 2), we only had a LU 0 mapped on controller 1. Controller 2’s LU’s started at 4. So the macs could not see any LU 0, and thus would not see ANY LU.

    So today I remapped the LU’s on the secondary controller to also include LU 0 and suddenly it started working. I’m still not sure if the macs need to see LU 0 on every port, or only every controller. So I mapped the LU’s to have LU 0 on every port, in such a way that every distinct LU is only seen on one port.

    Fine. Both G5’s worked (Os X.3.8). I reformatted the SANMP volumes and stripe sets and did some speed testing on the G5’s: getting about 200 MB/s write nd 150MB/s read. OK, I guess. On two 5 drive raid 0 arrays. Getting 3 streams SD on one machine and 4 streams on the other G5. If I add an 8th stream everything starts stuttering.

    Checking the data streams on the qlogic 5200 fibre switch, I could see that the dual link machines (G5’s) are splitting their data streams over both links. Good.

    BUT both the G4’s (os X.3.9) would still not mount most LU’s. I’ll post some pictures when back at work. I could see all 8 LU’s in the system profiler, but only 2 LU’s would show up in the disk utility.

    I tried Tiger X.4.1 on the G4 – and it also saw all 8 LU’s in system profile, but it saw three other LU’s in Disk Util. Then I tried a raw os X.3 installation, and saw all the LU’s in the Disk util! That’s when my time ran out.

    I’ll update you when I’m back at the office – for now the plan is to go to X.3.5 on the G4 and see if Disk util and SANMP works ok.

    Regards
    Francois

  • Francois Stark

    May 29, 2005 at 2:18 pm

    Our G4 was running on os X.3.9:

    All 8 LU’s in the system profile but only 2 in the disk utility.

    And here I tried it in Tiger X.4.1:

    In Tiger also all 8 LU’s in the system profile, but only 2 in disk util.

    So here I did a raw installation of X.3:

    And suddenly I could see all the LU’s in the disk util.

    Finaly I went up to os X.3.4 and this is what I saw:

    All the LU’s in disk util! I installed sanMP and could mount and read and write to the shared fibre arrays.

    Conclusion: Os X.3.9 and X.4.1 are not able to see LU’s properly on a ADTX array on a G4 mirrordoor. Os X.3.8 works in G5 machines.

    At the moment I’m getting 6 streams SD from the 10 drive raid 0 array (two 5 drive raid 0’s striped by apple disk util). I can play 6 streams on one G5 in FCP real time, or two streams each on three machines (two G5’s and one G4). This is quite reliable.

    However I have had three crashes on our G5 while capturing SD PAL to the SANMP fibre raid. FCP stops with disk IO error, and then finder also does not see the SanMP disk anymore. Everything freezes and I have to do a hard reboot. SanMP recovers after it’s 15 second delay because I did not unmount properly.

    So there is still a reliability problem when capturing – more tests to follow.

    Regards
    Francois

  • Bart Harrison

    May 29, 2005 at 3:07 pm

    Hi Francois.

    I wish I had time for a quick trip to SA. I’d love to get to the bottom of this problem you’re having. I’ve got many ADTX/SanMP installations all over North America some running perfectly for more than eighteen months. For me it’s been the real winning combination. Other than the fact that I refuse to use the Apple/LSI HBA’s I don’t see that you’re doing anything different. Why don’t you see if someone can loan you an Astera Rhino 3000 or an Atto Celerity 22XH to test out. FYI: the Astera card uses it’s own proprietary set of drivers while the Atto card uses parts of the Apple FCAL library. Either way it’d be a good test. By-the-way which version of the Apple cards do you have ? Rev 2 has a /B in the part number and Rev 4 has a /D in the part number. The /D cards were a nightmare of unexplainable problems for me, LUNS showing up somtimes and not showing up at other times (sound familier). I switched to the Astera cards and now everything works.

    Bart

    – – – – – – – – – – – – –
    Bart Harrison
    MPA – The HD Suite

    America’s VAR
    TurnKey Editing Systems, Storage Area Networks
    HD Consulting, Production & Post, Exhibition & Distribution
    http://www.hdsuite.com
    954-894-1221

  • Francois Stark

    May 29, 2005 at 4:42 pm

    Here I re-did the post because the previous links worked for our internal LAN only:

    Our G4 was running on os X.3.9:

    All 8 LU’s in the system profile but only 2 in the disk utility.

    And here I tried it in Tiger X.4.1:

    In Tiger also all 8 LU’s in the system profile, but only 2 in disk util.

    So here I did a raw installation of X.3:

    And suddenly I could see all the LU’s in the disk util.

    Finaly I went up to os X.3.4 and this is what I saw:

    All the LU’s in disk util! I installed sanMP and could mount and read and write to the shared fibre arrays.

    Conclusion: Os X.3.9 and X.4.1 are not able to see LU’s properly on a ADTX array on a G4 mirrordoor. Os X.3.8 works in G5 machines.

    At the moment I’m getting 6 streams SD from the 10 drive raid 0 array (two 5 drive raid 0’s striped by apple disk util). I can play 6 streams on one G5 in FCP real time, or two streams each on three machines (two G5’s and one G4). This is quite reliable – I looped these six playing streams for about 1 hour.

    However I have had three crashes on our G5 while capturing SD PAL to the SANMP fibre raid. FCP stops with disk IO error, and then finder also does not see the SanMP disk anymore. Everything freezes and I have to do a hard reboot. SanMP recovers after it’s 15 second delay because I did not unmount properly.

    So there is still a reliability problem when capturing – more tests to follow.

    Regards
    Francois

  • Francois Stark

    May 29, 2005 at 5:13 pm

    Hi Bart

    Thanks for the encouragement – I seem to be sorting the problems out one by one. Sad about having to go back to an earlier version of os X, but someone said apple changed fibre port discovery from os X.3.7. So it seems I can go back to X.3.6.

    I don’t know what’s going to happen with tiger – I know SanMP is not ready for tiger yet, but that cannot be far off. And if this problem is not solved, I’ll be stuck in Panther on the G4’s. And FCP 5 does not run on X.3.6; it needs at least X.3.9…

    Q: Did you notice any difference between G4 and G5 machines on the Astera cards? What version of os X are you running?

    I’m at home now, so I’m not sure about the rev of apple fibre card. Will check tomorrow.

    On the G5 that froze during capture: I’m running the Apple fibre card (LSI) in the top slot (PCI-X 133MHz by itself), and the decklink extreme with the Atto UL4D in the second and third slots (sharing PCI-X 100MHz). I copied some files off the Huge 320S array to the fibre array and got 135MByte/s read from Huge, and 135MB/s write to the Fibre array. No crashing.

    The only time it froze was during capture. I’m going to try the decklink card in the top slot (to isolate it from other cards), and the LU4D and Fibre cards in the second and third slots.

    Hopefully, if I can get this fibre array working very reliably, I can sell the UL4D and Huge array – which means I will only have the fibre card and Decklink installed.

    struggling on…
    Francois

  • Bart Harrison

    May 29, 2005 at 6:05 pm

    [Francois Stark] “Q: Did you notice any difference between G4 and G5 machines on the Astera cards? What version of os X are you running?”

    Sorry, we retired/repurposed the G4’s at all my SAN installation so far.

    [Francois Stark] “Hopefully, if I can get this fibre array working very reliably, I can sell the UL4D and Huge array”

    What I’ve done at a number of SAN installations is to set up one of the old G4’s and the Huge array as a support server tied to all the editing/graphic systems via an inexpensive Gbit Ethernet switch. I set them up as an NFS volume which appears on each of the editing systems as an “internal” folder on their system drive. At one site this server is hosting 700MB of music (about six music libraries) plus all of their still graphics assets. The music and graphics play right off of the support server without consuming any of the valuable SAN resources.

    Bart

    P.S. When running a dual controller ADTX, a QLogic 5200 switch, and a dual channel Astera 3000 (with dual 2Gbit fibre) we usually see well over 300MB/sec read and write at every workstation. If you’re only seeing 150MB/sec then something is definately wrong !

    – – – – – – – – – – – – –
    Bart Harrison
    MPA – The HD Suite

    America’s VAR
    TurnKey Editing Systems, Storage Area Networks
    HD Consulting, Production & Post, Exhibition & Distribution
    http://www.hdsuite.com
    954-894-1221

  • Bart Harrison

    May 29, 2005 at 6:11 pm

    [Francois Stark] “What version of os X are you running?”

    I have versons of OSX ranging from 10.3.4 to 10.3.7. I’ll be upgrading the 10.3.4 installation (my oldest) to FCP 5 next week. We might try 10.4.1 just as a test although we’ll probably go with 10.3.9 for the time being. I’ll keep you posted.

    Bart

    – – – – – – – – – – – – –
    Bart Harrison
    MPA – The HD Suite

    America’s VAR
    TurnKey Editing Systems, Storage Area Networks
    HD Consulting, Production & Post, Exhibition & Distribution
    http://www.hdsuite.com
    954-894-1221

  • Francois Stark

    May 29, 2005 at 6:21 pm

    Hi Bart

    I found where Jeff Bernstein talks about the change from os X.3.7:
    ——————————————————————

    Name: Jeff Bernstein
    Date: May 27, 2005 at 04:44 gmt
    Subject: Re: Not solved yet – seems G5 is better.

    Okay, this is starting to sound a little familiar. I have a feeling you are running OS X 10.3.7 or greater. This could mean you are not running the latest firmware on the ADTX in which half the LUNs would not mount due to the way OS X changed the behavior in LUN discovery.

    Another thing to note is that there is a nasty bug on the ADTX when used with SANmp. If your ADTX needs to rebuild, you not be able to recover the data. We just discovered this the hard way. ADTX is working on it and I welcome you to apply some pressure for I would hate for this to happen to you.

    Let us know how it goes.

    Can you tell us which OS and which rev of the firmware on the ADTX?

    Jeff

    Jeff Bernstein

    Digital Desktop Consulting
    Apple Pro Video VAR
    XSAN Certified

    323-653-7611

    [Bart Harrison] “set them up as an NFS volume”

    Why NFS? Maybe NTFS? for PC graphics machines? (we all make typo’s sometimes…) or is it really NFS?

    [Bart Harrison] “we usually see well over 300MB/sec read and write at every workstation” Today I got about 180MB/s on the decklink disk speed meter. Remember, only 10 drives (two 5 drive raid 0’s striped using apple disk util), not a fully populated ADTX yet.

    On your ADTX settings: What is your setting for cache pre-read streams? Mine is set to 32 at the moment, default is 8. If I remember right it can go up to 128? I’m not interested in HD streams (yet), I need as many SD (20MB/s)uncompressed streams as possible.

    Thanks
    Francois

  • Bart Harrison

    May 29, 2005 at 7:16 pm

    [Francois Stark] Jeff: “half the LUNs would not mount due to the way OS X changed the behavior in LUN discovery.”

    With the Astera 3000 it’s the card/driver that does the LUN discovery not OS X. When using the Apple/LSI cards you’re subject to the whims of Apple who as you know are continuously changing things (driver revs, OS revs, hardware revs). This is why I’ve always preferred the Astera card. Unfortunately I’m not sure if they’re going to be making them any more.

    [Francois Stark] “Why NFS? Maybe NTFS? for PC graphics machines? (we all make typo’s sometimes…) or is it really NFS?”

    Yes, NFS (the Unix Network File System). Because of it’s Unix core NFS networking is already built into OS X. Of course if their are a lot of XP systems I’d go with SMB, but in a primarily OS X or Unix/Linux environment nothing beats NFS for convenience. Please be mindful this has nothing to do with the underlying disk format. You can share a Mac OS Extended, FAT 32, or NTFS volume via any networking topology you have available. If you excuse the expression, “they are apples and oranges”.

    [Francois Stark] “Remember, only 10 drives (two 5 drive raid 0’s striped using apple disk util), not a fully populated ADTX yet.”

    I have never recommended any 10 drive array in a shared-storage SAN environment. Every SAN installation I have is set up with 7 drives on one controller (Raid 5) and 7 drives on the other. I create 12 LUNS on each side and stripe them together to create 12 OS X volumes. The 15th drive is used as a hot spare.

    [Francois Stark] “On your ADTX settings: What is your setting for cache pre-read streams? Mine is set to 32 at the moment, default is 8. If I remember right it can go up to 128?”

    I set the ADTX with Cache’ Mirroring off, Write Cache Enabled, and the Look Ahead Table to 128 Entries. I arrived at these settings through extensive performance testing and consultation with ADTX and Astera engineeing. (Of course, your’re milage may vary.)

    Hope this helps !

    Bart

    – – – – – – – – – – – – –
    Bart Harrison
    MPA – The HD Suite

    America’s VAR
    TurnKey Editing Systems, Storage Area Networks
    HD Consulting, Production & Post, Exhibition & Distribution
    http://www.hdsuite.com
    954-894-1221

  • Francois Stark

    May 30, 2005 at 4:53 am

    Hi Bart
    thanks for this discussion.

    [Bart Harrison] “You can share a Mac OS Extended, FAT 32, or NTFS volume via any networking topology you have available.”

    When sharing over SMB to PC’s we can only access the user;desktop etc. We can not see any non-boot drives. So we’d have to get os X server software.

    [Bart Harrison] “7 drives”

    Working towards that.

    [Bart Harrison] “Look Ahead Table to 128 Entries”

    I’ll try that today.

    Regards
    Francois

Viewing 1 - 10 of 18 posts

Log in to reply.

We use anonymous cookies to give you the best experience we can.
Our Privacy policy | GDPR Policy