Creative Communities of the World Forums

The peer to peer support community for media production professionals.

Activity Forums Storage & Archiving 2 drive failure on RAID-5

  • 2 drive failure on RAID-5

    Posted by Eric Hansen on March 3, 2010 at 9:00 pm

    So i just experienced something that I thought would (almost) never happen – 2 drives failing one right after the other on a RAID-5 volume. when the first drive failed, i replaced it with a spare and started the rebuild process. when the process reached about 60% complete, another drive in the 8 drive volume failed. all the other drives in the 2 enclosures are fine.

    according to the logs, it looks like it was a heat issue. i’ve brought some fans into the server room (the ambient temp is now high 60s) and i have the HVAC guys coming in to re-analyze the setup. none of the other drives are reporting heat issues.

    here’s my question: do i keep it RAID-5, switch to RAID-5 with a hot spare (i don’t think RAID-5 with a hot spare would have helped in this situation though), or go RAID-6? i always thought RAID-6 was overkill if you’re right there to swap a drive when one fails. but i guess i experienced the one time a RAID-6 would be better than a RAID-5.

    thanks

    e

    Eric Hansen – http://www.erichansen.tv

    Eric Hansen replied 16 years, 1 month ago 5 Members · 6 Replies
  • 6 Replies
  • Bob Zelin

    March 4, 2010 at 4:35 am

    And dare I ask what drive array you are using ? And what SAS Controller card are you using ?

    When I used Seagate drives, I had this exact same thing happen to me. No more Seagates for me.

    Bob Zelin

  • Eric Hansen

    March 5, 2010 at 3:49 pm

    hey bob

    no, i learned my lesson with Seagate. this RAID is a ProAVIO/Enhance Tech enclosure, Hitachi Saturn drives, and an ATTO R380 card.

    ATTO Tech support helped me get the volume back up so i could copy all the media off. those guys rock! i replaced the 2 drives that failed and turned that particular volume into RAID-6. so the volume is now 5.46TB instead of 6.37TB, but the RAID-6 helps me sleep better at night. as we finish projects on the other volumes, i’m going to change them to RAID-6 also. we have 4 volumes currently. i might even merge them into 2. our plan was for 4 edit bays initially, thus the 4 volumes. but now 2 of the bays have turned into a Pro Tools suite, and a finishing/color correction/transcoding suite. so maybe we’re better off with 2 larger volumes instead of 4 smaller ones.

    e

    Eric Hansen – http://www.erichansen.tv

  • Olivier Jean

    March 9, 2010 at 12:03 pm

    Hi Bob,
    I’m surprised to see that you do not trust Seagate drives anymore,
    dare i ask if you used enterprise or desktop models?

    We been using enterprise Seagate 1TB with 99% of the raid we’ve sold (infortrend),
    and the failure rate has been almost nil.

    I have to admit that Seagate had some firmware issues about 8 months ago and we had to update a lot of them.

    I have to say we didn’t venture into the 1.5TB.

    I remember asking most manufacturer @NAB last year and most of them used Seagate or Itachi.

    Regards
    Olivier Jean
    Video Sales Consultant
    Apple Certified Trainer Final Cut Pro 7
    Powermedia Systems
    Sydney Australia

  • Dan Leery

    March 16, 2010 at 11:30 pm

    Personally I’d go with raid-6. This does indeed sound like a heat issue, as the failure rate goes up dramatically as heat increases. Raid-6 is only one more drive than raid-5, and although you may incur a slight performance hit over raid-5 for writes, random read performance may improve slightly. At any rate, the single extra drive is worth the cost of keeping your data safe imho.
    Dan
    https://www.storageplatter.com

  • Neil Sadwelkar

    March 20, 2010 at 11:27 am

    There’s this article I remember reading sometime about how, when RAIDs get past a certain size (in Tb) the chances of more than one drive failing simultaneously get higher.

    I too have experienced this with old RAIDs – like 5 years old. We invest in RAID 5 with the impression that data is safe from drive failure.

    But, in old RAIDs when drives get old, the chances of them failing simultaneously seems to get higher. I guess one needs set a life for a RAID as soon as one gets it, and after that time, consider every additional day a bonus.

    ———————————–
    Neil Sadwelkar
    neilsadwelkar.blogspot.com
    twitter: fcpguru
    FCP Editor, Edit systems consultant
    Mumbai India

  • Eric Hansen

    March 20, 2010 at 4:40 pm

    Neil says: I guess one needs set a life for a RAID as soon as one gets it, and after that time, consider every additional day a bonus.

    sage advice. i personally operate under the assumption that all drives will fail – are you prepared if a random drive in your facility failed RIGHT NOW? i have all my non-RAID systems backed up to a Crash Plan server, and extra drives sitting next to the RAID ready for swapping. at first i thought the manufacturer’s warranty period was a good indicator of a drive’s life. but my experience is that it’s not. i have a G4 tower running on its original factory drive, and i’ve had a few server-level hard drives fail within a month.

    through all these installations, i’ve noticed that very few take environmental factors into consideration. a small production facility would never think about a “server room”. i’ve never been able to install a system in an ideal environment. the closest one is an Xsan that’s in a basement. the Xsan pretty much takes care of the heating for the facility in the winter. during that installation we budgeted for a rack refrigerator (very cool unit that runs about $5k – looks like a Coke cooler). but the basement stays cool enough that the fridge wasn’t needed.

    during the construction of the building for my most recent SAN, i went over the needs for the server room a few times with the contractors, and they just never seemed to get it. their solution was to create a closet with a 1 foot vent at the top, and just allow convection to take over. this is part of the problem being out in the boonies where a “server room” is place where waitresses hang out. now they’re back in there (and can finally feel the heat this system puts out) and talking $3k+ for a dedicated A/C system.

    e

    Eric Hansen – http://www.erichansen.tv

We use anonymous cookies to give you the best experience we can.
Our Privacy policy | GDPR Policy