-
Gspeed ES Pro RAID keeps disappearing from Mac, have to re-initialize!
MAC PRO KEEPS LOSING CONNECTION TO RAID DRIVE.
After 9 months of organizing a documentary film edit on an external RAID drive (12TB, Raid 0. Yes, I have 2 layers of backup), I purchased the PNY Nvidia Quadro 4000 video card for the primary edit. Soon after, I began having boot up issues. The screen would freeze at various points during the startup process, requiring manual power-downs.
Over the past two months, my computer has lost communication with the RAID 3 times, forcing me to rebuild it from my backup drives (10TB). One tech at Diskwarrior said my computer lost the partition, and that the 1s and 0s had been largely replaced with gibberish.
The Raid becomes unrecognized by my computer in one of two ways:1- After two days of copying data back over to the raid (rebuilding it), I get an error code -36, saying a file cannot be read or written, then the Raid partition disappears from disk utility and the next time I start up the computer says it doesn’t recognize my drive. The ATTO configuration tool (the card that attaches my raid to the computer) sees the drive, but my computer does not. I have to re-initialize it and start over again.
2- The Raid will go from being recognized to not being recognized when I reboot one day. Again, I have to re-initialize and rebuild.
I removed the PNY Graphics card and put in the stock ATI Radeon HD 5770 it came with. On day 2 of rebuilding the RAID, I got the error -36 and the Raid disappeared from disk utility again. When I opened the raid folder, which seconds before had 10 TB of data on it, it read as zero items. So the PNY Nvidia card was NOT the problem.
Apple has worked on the computer 3 times, running extensive hardware tests, but the same problem persists. At one point they said my blackmagic card was the problem, and more recently they said I probably have bad ram. I tried separate ram (the old 6GB my computer came with), and my computer seemed to begin booting up okay, but one day later I had the -36 error, the RAID couldn’t be recognized again and I lost the data during another rebuild. Apple even replaced the Ram Tray, but the same problem persists.
G technology says my RAID’s hardware is fine, and ATTO, who makes the Raid card, says it looks fine to them too. This does not mean they are 100% not the problem though.
Yesterday as I was attempting to rebuild the RAID again, I saw the -36 error come up, and files stopped copying. I immediately made a screenshot and noted the time.
Today I spoke with a really cool tech at Apple named Ron who helped me investigate the Kernel log, and this is what it said (the RAID is called FadingWest):
Dec 6 17:05:36 Jesses-Mac-Pro kernel[0]: hfs_swap_BTNode: invalid forward link (0xbb739311 >= 0x0000df00)
Dec 6 17:05:36 Jesses-Mac-Pro kernel[0]: hfs: node=2717 fileID=4 volume=FadingWest device=/dev/disk3s2
Dec 6 17:05:36 Jesses-Mac-Pro kernel[0]: hfs: Runtime corruption detected on FadingWest, fsck will be forced on next mount.Ron believes my problem is with the RAID card. He thinks it’s dropping the connection. So I’ve got an RMA going with ATTO for a new one.
Here are the final things I can troubleshoot at this point:
• New RAID card (ATTO)
• New RAID drive (G-Technology)
• Motherboard / PCI lanes (Apple).
• NEW COMPUTER (Apple)Any ideas are appreciated. Here are the items in question:
• Mac Pro mid 2010 2×2.66 Ghz 6 core Intel Xeon OS X Lion 10.7.5
• G speed ES Pro (G Technology) – 12TB @ RAID 0
• ATTO R680 RAID card -
75 Replies
-
Reply from Bob Zelin –
I will respond below to multiple points, but the first and most important thing, is that this is not “voodoo”. Your drive issues have NOTHING to do with “bad RAM” or the NVidia Quadro 4000 card. Your NVidia Quadro 4000 could catch on fire inside the computer, and it would not affect your RAID array.replies below –
MAC PRO KEEPS LOSING CONNECTION TO RAID DRIVE.
After 9 months of organizing a documentary film edit on an external RAID drive (12TB, Raid 0. Yes, I have 2 layers of backup), I purchased the PNY Nvidia Quadro 4000 video card for the primary edit. Soon after, I began having boot up issues. The screen would freeze at various points during the startup process, requiring manual power-downs.
REPLY – when you power up, and your computer freezes, step 1 is to disconnect the drive array. Reboot your computer (with your ATTO R680 card in it). If it boots up quickly, then your problem is your drive array. The installation of the Quadro 4000 and your problems is merely a coincicence.
Over the past two months, my computer has lost communication with the RAID 3 times, forcing me to rebuild it from my backup drives (10TB). One tech at Diskwarrior said my computer lost the partition, and that the 1s and 0s had been largely replaced with gibberish.
The Raid becomes unrecognized by my computer in one of two ways:REPLY – your components in question are the ATTO R680 card, the miniSAS cable to the R680, the GSpeed eS enclosure and the drives.
When you say “it lost the partition” – does that mean that when you go into the ATTO Configuration Tool that it shows that there is no RAID Group ?1- After two days of copying data back over to the raid (rebuilding it), I get an error code -36, saying a file cannot be read or written, then the Raid partition disappears from disk utility and the next time I start up the computer says it doesn’t recognize my drive. The ATTO configuration tool (the card that attaches my raid to the computer) sees the drive, but my computer does not. I have to re-initialize it and start over again.
REPLY – this is important. So you are telling me that the ATTO Configuration Tool SEES the RAID Group, you see all drives are there, the RAID group in the ATTO Config tool says ONLINE, but the operating system will not mount the drive array on the desktop ?
Is this what you are saying ?Because if it shows that the RAID group is ONLINE, but it does not appear on the Mac desktop, you go to ATTO RAID MANAGEMENT on the top of your screen, and select AUTOMAP, and this will force your drive to mount on your desktop.
2- The Raid will go from being recognized to not being recognized when I reboot one day. Again, I have to re-initialize and rebuild.
REPLY – again, you need to be specific, as per the ATTO Configuration Tool. When you say “not being recognized” – do you mean that the ATTO Configuration Tool shows the RAID Group to be online, but the array does NOT mount on the Apple desktop – or do you mean that there is no RAID group in the ATTO Configuration Tool ?
I removed the PNY Graphics card and put in the stock ATI Radeon HD 5770 it came with. On day 2 of rebuilding the RAID, I got the error -36 and the Raid disappeared from disk utility again. When I opened the raid folder, which seconds before had 10 TB of data on it, it read as zero items. So the PNY Nvidia card was NOT the problem.
REPLY – yes, the NVIDIA has nothing to do with this. Neither does your RAM. And stop going to Apple to ask them about this.
Apple has worked on the computer 3 times, running extensive hardware tests, but the same problem persists. At one point they said my blackmagic card was the problem, and more recently they said I probably have bad ram. I tried separate ram (the old 6GB my computer came with), and my computer seemed to begin booting up okay, but one day later I had the -36 error, the RAID couldn’t be recognized again and I lost the data during another rebuild. Apple even replaced the Ram Tray, but the same problem persists.
REPLY – again, the NVidia, and the RAM have nothing to do with this issue. Your only tech support should come from ATTO, or G-Tech (or this forum).
G technology says my RAID’s hardware is fine, and ATTO, who makes the Raid card, says it looks fine to them too. This does not mean they are 100% not the problem though.
REPLY – how does G-Tech know that your RAID hardware is fine – have they looked at your miniSAS cable, have they looked at your drive chassis ? Have you send your logs to ATTO for examination ? Has ATTO requested these ?
1) have you replaced your miniSAS cable ?
2) what Mac OS are you running ?
3) are you running ATTO Driver V3.50 ?
4) are you running ATTO Firmware Sept 27th, 2012 ?
while running V4.09 ATTO Config Tool, click on help, and run the diagnostic, so you can get a set of logs from the ATTO card, and SEND THEM TO ATTO.
5) exactly what has G-Tech done for you in this situation ?Yesterday as I was attempting to rebuild the RAID again, I saw the -36 error come up, and files stopped copying. I immediately made a screenshot and noted the time.
REPLY – for all I know, you have a flakey drive. But I need more details as to what the ATTO Configruation Tool is showing you.
Today I spoke with a really cool tech at Apple named Ron who helped me investigate the Kernel log, and this is what it said (the RAID is called FadingWest):
REPLY – your Apple Tech knows nothing about the ATTO card or the G-Tech drive enclosure. You do not have an Apple computer problem. Stop talking to Apple about any of this.
Ron believes my problem is with the RAID card. He thinks it’s dropping the connection. So I’ve got an RMA going with ATTO for a new one.
REPLY – a new ATTO card will do nothing for you, if you don’t have the latest driver and firmware loaded.
Let me know some answers to my questions above.
Bob Zelin
-
Wow, thanks for all the help Bob! Answers below in CAPS:
REPLY – your components in question are the ATTO R680 card, the miniSAS cable to the R680, the GSpeed eS enclosure and the drives.
When you say “it lost the partition” – does that mean that when you go into the ATTO Configuration Tool that it shows that there is no RAID Group ?ATTO CONFIG TOOL SHOWS THE RAID GROUP. MAC OS DOES NOT. THE RAID’S PARTITION IN DISK UTILITY LITERALLY VANISHES.
REPLY – this is important. So you are telling me that the ATTO Configuration Tool SEES the RAID Group, you see all drives are there, the RAID group in the ATTO Config tool says ONLINE, but the operating system will not mount the drive array on the desktop ?
Is this what you are saying ?YES
Because if it shows that the RAID group is ONLINE, but it does not appear on the Mac desktop, you go to ATTO RAID MANAGEMENT on the top of your screen, and select AUTOMAP, and this will force your drive to mount on your desktop.
I NEVER DID THIS BECAUSE IT WAS NEVER SUGGESTED IN MY TECH SUPPORT CALLS WITH ATTO OR G TECH. THEY BOTH TOLD ME I HAD TO RE-INITIALIZE THE RAID AFTER VARIOUS ATTEMPTS TO GET MY OS TO RECOGNIZE IT.
REPLY – how does G-Tech know that your RAID hardware is fine – have they looked at your miniSAS cable, have they looked at your drive chassis ?
THEY HAD ME SEND OVER A REPORT (CAN’T REMEMBER HOW, LOST IN SEA OF TECH CALLS), AND SAID FROM THEIR END THE RAID ITSELF APPEARED TO BE FINE. THE CABLE WAS NOT ADDRESSED.
Have you send your logs to ATTO for examination ? Has ATTO requested these ?
YES. THEY’RE SUPPOSED TO RESPOND TODAY. I SENT THEM THE OS KERNEL LOGS, THE ATTO LOGS, AND THE EXACT TIME & DATE OF THE LAST TIME THE RAID DISAPPEARED. ON THAT NOTE, APPLE TOOK ME THROUGH THE LOGS (WHY DID IT TAKE 2 MONTHS FOR THESE TO FINALLY BE CONSIDERED I WONDER?) AND CONCLUDED FROM THEM THAT MY MOTHERBOARD WAS NOT DROPPING THE CONNECTION, RATHER THAT THE RAID CARD OR RAID WAS.
1) have you replaced your miniSAS cable ?
NOT YET. IF THE NEW RAID CARD DOESN’T WORK, G TECH IS SENDING ME A NEW RAID & CABLE SO I’LL KNOW THEN FOR SURE.
2) what Mac OS are you running ?
10.7.5
3) are you running ATTO Driver V3.50 ?
I’M ON MY LAPTOP BUT KNOW 100% SURE MY ATTO SOFTWARE IS UP TO DATE. DOUBLE CHECKED WITH ATTO TECHS WHEN THIS ALL STARTED.
4) are you running ATTO Firmware Sept 27th, 2012 ?
YEP.
while running V4.09 ATTO Config Tool, click on help, and run the diagnostic, so you can get a set of logs from the ATTO card, and SEND THEM TO ATTO.
DONE. SEE ABOVE.
5) exactly what has G-Tech done for you in this situation ?
THE HEAD TECH HAS BEEN SUPER COOL. HE’S GOING TO SEND ME A NEW DRIVE (THANKFULLY I’M ONLY 2 HRS AWAY) IF I SAY THE WORD.
REPLY – for all I know, you have a flakey drive. But I need more details as to what the ATTO Configruation Tool is showing you.
SEE ABOVE
REPLY – your Apple Tech knows nothing about the ATTO card or the G-Tech drive enclosure. You do not have an Apple computer problem. Stop talking to Apple about any of this.
HA! I KNOW WHAT YOU’RE SAYING BUT SINCE NOBODY… I MEAN NOBODY HAS FIGURED OUT THIS ISSUE’S CAUSE BETWEEN DISK WARRIOR, ATTO, G TECH AND APPLE, ALL I CAN DO IS ACCUMULATE INFO AND TAKE THE NEXT STEP. HENCE THIS FORUM.
REPLY – a new ATTO card will do nothing for you, if you don’t have the latest driver and firmware loaded.
YOU DON’T THINK THE CARD ITSELF CAN BE A DUD? WHY NOT?
THANKS AGAIN BOB. ANY FURTHER INFO APPRECIATED!
JESSE -
ATTO CONFIG TOOL SHOWS THE RAID GROUP. MAC OS DOES NOT. THE RAID’S PARTITION IN DISK UTILITY LITERALLY VANISHES.
REPLY – highlight your RAID group in ATTO Config Tool (it should say “online” next to it). Now, click on ATTO RAID Management on the top of your screen, and select AUTOMAP. Does the drive mount ?
Have you send your logs to ATTO for examination ? Has ATTO requested these ?
YES. THEY’RE SUPPOSED TO RESPOND TODAY. I SENT THEM THE OS KERNEL LOGS, THE ATTO LOGS, AND THE EXACT TIME & DATE OF THE LAST TIME THE RAID DISAPPEARED. ON THAT NOTE, APPLE TOOK ME THROUGH THE LOGS (WHY DID IT TAKE 2 MONTHS FOR THESE TO FINALLY BE CONSIDERED I WONDER?) AND CONCLUDED FROM THEM THAT MY MOTHERBOARD WAS NOT DROPPING THE CONNECTION, RATHER THAT THE RAID CARD OR RAID WAS.
REPLY – Apple knows nothing about the ATTO cards (or anyone elses cards). Forget Apple for support on this. Let me know what ATTO says, and let me know if Automap works.
YOU DON’T THINK THE CARD ITSELF CAN BE A DUD? WHY NOT?
REPLY – it is SO unlikely that you have a bad ATTO card (anything is possible), but it is VERY likely that you have a cable that is bad, because cables ALWAYS go bad.
It’s nice that G-Tech is willing to send you another chassis.
Try Automap, and let me know what ATTO says about the logs – make sure that you follow up with them – don’t wait for them to call you – YOU CALL THEM, and force them to give you an answer to what they see in the logs.Bob Zelin
-
HELLO AGAIN BOB. I JUST PUT THE NEW ATTO CARD IN MY TOWER AND FIRED IT UP WITH THE RAID ATTACHED. LAST TIME I HAD THE RAID ATTACHED WAS WHEN I LOST IT DURING MOST RECEND 10TB REBUILD (-36 ERROR).
REPLY – highlight your RAID group in ATTO Config Tool (it should say “online” next to it). Now, click on ATTO RAID Management on the top of your screen, and select AUTOMAP. Does the drive mount ?
THE ATTO CONFIG TOOL SEES THE RAID AND CONFIRMS IT IS ONLINE, BUT MY COMPUTER DID THE USUAL ‘CAN’T RECOGNIZE THE DISK” SO I HAD TO INITIALIZE IT WITH DISK UTILITY.
Have you send your logs to ATTO for examination ? Has ATTO requested these ?
I EMAILED ATTO ABOUT THESE TODAY AND “ENGINEERS ARE LOOKING AT THEM”. I’M ABOUT TO CRACK THE WHIP AGIAN.
REPLY – it is SO unlikely that you have a bad ATTO card (anything is possible), but it is VERY likely that you have a cable that is bad, because cables ALWAYS go bad.
I COULDN’T FIND A MINI SAS IN MY AREA SO I ORDERED ONE ONLINE THIS MORNING. WILL BE HERE THURSDAY SUPPOSEDLY.
Try Automap, and let me know what ATTO says about the logs – make sure that you follow up with them – don’t wait for them to call you – YOU CALL THEM, and force them to give you an answer to what they see in the logs.
HA! ON IT… THANKS AGAIN BOB. UNTIL NEXT TIME.
JESSE
-
SORRY BOB, I FORGOT TO MENTION – I TRIED AUTOMAP AND IT DIDN’T MOUNT THE DRIVE.
-
whow, Bob is really looking into it. It is very likely your Gspeed Raid (hardware) or the cable (we have never had any problems with SAS cables)
Definitely not ram (your computer would react totally weird and not loosing just the drive)
You can try Tech tool pro, a very good hardware check software.
My Instinct tends towards your Raidsystem.Another Test:
I think you are on Raid 5 or 6. When your Gspeed is ready, pull one drive and replace it. It should rebuildt. If the Gspeed has a hardware error the rebuildt function should fail.good luck
Rainer
-
thanks Rainer.
I’m on RAID 0 because it’s the only way to get all my data on the raid. I have 2 layers of backup.
jesse
-
ATTO called and said their tech couldn’t find anything conclusive from the diagnostic. Does that even make sense?
-
Once again,
the raid’s your problem. Either the internal controller or (raid0) one of the Harddiscs in the raid. What hard discs are in the raid?Rainer
-
Is there one HD going red?
If not, it’s the internal raid controller.Rainer
Reply to this Discussion! Login or Sign Up