RAID 5 Drive failure

[Alex Gerulaitis]

The very 1st thing I’d do is attempt to clone the volume, e.g. using Casper or some other block-level process, and keep a log of the process to see if you get any read errors, and how many.

The fact that you can read some files gives hope.

The problem can be caused by file system corruption (not RAID related), RAID engine failure, and/or drive failure. That’d be the 2nd thing to figure out, and I am afraid, you will need expert help on that.

[Rex Polanis]

I fixed it.

This is what I did:
During the boot up process I pressed F10
this brought up the built in Nvidia RAID control utilitiy
Once there the screen showed 2 drives in the RAID
one drive bad and another drive degraded
I selected the bad drive and pressed enter for information
the utility showed the bad drive was on esata port 2.1
I went back and checked the degraded drives (meaning healthy but not active) and the nvidia utility showed the degraded drives where on esata ports 3.0 and 3.1

with this information I was able to figure out which drive was bad.
I powered down, removed the bad drive and replaced it with a new drive of the same make, model, and size.

I rebooted and hit F10 again
the Nvidia utility now saw one RAID drive with the option to Rebuild.
I hit “r” to rebuild the selected RAID and left the computer on, in that utility, over night.

The next day I exited the Nvidia utility and rebooted straight to windows 7 and all my information was restored!

Hooray!

One man with courage makes a majority.

Canon 7D
Adobe CC Master Suite
Digital Juice
Video Copilot

[Ericbowen]

I would run a parity check now or move data off and rebuild the volume. Had the Raid volume info been clean, you should have seen your data with just 1 bad drive. I would not trust that raid volume data yet.

Eric-ADK
Tech Manager
support@adkvideoediting.com

[Chris Murphy]

I agree. Scrub (parity check) is a minimum requirement. It’s a needle in a hay stack, but it’s typical for drives to spit garbage as they die and neither RAID nor the file system have any means of disputing the garbage. That garbage just causes confusion and strange OS behavior, including file system problems. The other thing that’s possible is one or more of the surviving drives has bad sectors resulting in transient read failure. That causes delays as data is being rebuilt from parity (on the fly) in the array’s degraded state. Upon rebooting to the RAID firmware interface, initiating the rebuild on a replacement drive, the remaining drives are permitted to take quite a while (30 seconds, maybe more) to make multiple attempts at reading these transient sectors. The thing is, without an explicit read failure those transient sectors aren’t fixed (or replaced). And if they did produce a read error, that’d mean their data isn’t returned which means a collapsed RAID 5 array. So… yeah. I wouldn’t trust it.

And that’s why buying cheap drives not designed for use with RAID is shooting yourself in the foot. They’re explicitly designed to have long error recovery times, instead of producing an error quickly, thereby causing the RAID to fix that sector, so they don’t accumulate. Which they do with the wrong kind of drives.