• Storage & Archiving

  All SAN volumes disconnect

  Posted by Eric Hansen on December 24, 2009 at 12:16 am

  hey guys

  i had an editor working when all of a sudden the system locked up and the beachball started spinning. about 30 seconds later, a small dialog box came up saying the 4 SAN volumes have gone offline and theres a Disconnect button at the bottom of the box. we restarted the editing computer (but didnt do anything with the server) and i checked the Console on the edit system. the first message listed at the time of the crash is this:

  12/23/09 4:34:22 PM kernel ASP_TCP Disconnect: triggering reconnect by bumping reconnTrigger from curr value 0 on so 0x16efe660
  12/23/09 4:34:22 PM KernelEventAgent[68] tid 00000000 received VQ_NOTRESP event (1)

  and then the next 2 messages repeat for all of the 4 SAN volumes:

  12/23/09 4:34:22 PM KernelEventAgent[68] tid 00000000 type ‘afpfs’, mounted on ‘/Volumes/SAN 1’, from ‘afp_4nmyOs0009zB0000oM0000VU-1.2d000004’, not responding
  12/23/09 4:34:22 PM kernel AFP_VFS afpfs_DoReconnect started /Volumes/SAN 1 prevTrigger 0 currTrigger 1

  and then:

  12/23/09 4:34:22 PM KernelEventAgent[68] tid 00000000 found 4 filesystem(s) with problem(s)
  12/23/09 4:34:22 PM kernel AFP_VFS afpfs_DoReconnect: connect to the server /Volumes/SAN 1
  12/23/09 4:34:22 PM loginwindow[67] 4 servers now unresponsive

  then this line repeats (for the 4 volumes) until i shut down the editing system:

  12/23/09 4:34:22 PM kernel AFP_VFS afpfs_DoReconnect: reconnect already in progress for /Volumes/SAN 3, going back to wait

  the Console log on the server is clean, and the other editors kept working on their systems without interruption.

  server system:

  Mac Pro 8core 2.26 16GB RAM
  ATTO R380
  2 Enhance Tech JS enclosures with 16 1TB Hitachi drives each, divided out into 4 volumes
  6 port Small Tree ethernet card

  i googled around the first Console message and a few forums with Time Machine issues came up. TM is not running on any of the edit systems or the server. also, Spotlight has been turned off for the 4 SAN volumes on all systems connected.

  thanks

  e

  Eric Hansen – The Audio Visual Plumber – http://www.avplumber.com

  Matt Geier replied 16 years, 4 months ago 4 Members · 8 Replies
• 8 Replies

• 

  Bob Zelin

  December 24, 2009 at 5:05 am

  are the clients Nehalem MAC’s, or are they old MAC’s. If they are old MAC’s, it’s just one of those things.

  If it’s NEW MAC’s (the Nehalem), I know the answer.

  Let me know.

  bob zelin
• 

  Eric Hansen

  December 24, 2009 at 4:04 pm

  yes Bob, they are Nehalems.

  thanks

  e

  Eric Hansen – The Audio Visual Plumber – http://www.avplumber.com
• 

  Bob Zelin

  December 24, 2009 at 6:53 pm

  easy fix.
  Get Small Tree PEG1D cards for each Nehalem computer. The network chip in the Nehalem (Intel Hartwell) disconnects when you exceed 60Mb/sec. Put in the cheap single port ethernet card, and your problem will be over.

  Bob Zelin
• 

  Eric Hansen

  December 24, 2009 at 7:11 pm

  interesting, thanks Bob. looks like i’ll have an extra port for those Euphonix controllers after all.

  so this is just on the Nehalems? the older Mac Pros and G5s we’re going to add will be fine?

  thanks as always and Merry Christmas!

  e

  Eric Hansen – The Audio Visual Plumber – http://www.avplumber.com
• 

  Jordan Woods

  December 24, 2009 at 7:17 pm

  What is the setup for this NAS? You have 4 direct attached volumes on a single server pushing out over AFP to your client stations?
• 

  Eric Hansen

  December 24, 2009 at 7:38 pm

  Jordan writes: “What is the setup for this NAS? You have 4 direct attached volumes on a single server pushing out over AFP to your client stations?”

  yup. its the Bob Zelin-style ethernet SAN setup. theres 4 6.3TB volumes on the 2 Enhance Technology SAS enclosures (aka ProAVIO), hooked up to an ATTO R380 card. we chose 4 volumes because there’s 4 main edit systems that each have their own working volume (for the most part). then the 6 port Small Tree ethernet card is link aggregated to a 24port EdgeCore switch from Small Tree. the system has 4 full edit bays with Nehalem Mac Pros, 2 Mac Minis as logging stations, and a laptop here and there.

  i’m gonna order the ethernet cards that Bob suggested on Monday. hopefully that solves the issue. but Jordan, if you have any thoughts, please let me know.

  thanks

  e

  Eric Hansen – The Audio Visual Plumber – http://www.avplumber.com
• 

  Bob Zelin

  December 24, 2009 at 8:51 pm

  we discovered this working on a 100% new system (all Nehalem), getting the disconnects, but the one older Mac (with 10.5.8) never disconnected. It was not 10.6 – it was the Nehalem. After 60Mb/sec – say BYE BYE. Add another non Hartwell ethernet port, and no more problems.

  Bob Zelin
• 

  Matt Geier

  December 25, 2009 at 12:14 am

  Hi Eric,

  You should call me on Monday.

  I’ll send you an email.

  Thanks,

  Matt

Reply to this Discussion! Login or Sign Up