Archiving Strategy

[Stephen Mann]

Ummm. No.

FAT hasn’t been used for a long time. NTFS has a much more robust indexing mechanism that is a lot less at-risk of losing a piece of the chain as FAT was. (When did you last have to run chkdsk /f to try to recover a lost chain?)

The drive hardware does have error correction built in. No magnetic recording medium is error free and error correction is used to hide those from normal operation. Further, the act of reading and writing (copying) adds another level of error detection. If there are bad sectors on the hard disk, your copy will fail.

I’ve been archiving on hard disks for ten years, and I haven’t had one failure yet. (I do have a bunch of 10Gb drives to recycle.)

Steve Mann
MannMade Digital Video
http://www.mmdv.com

[Cory Petkovsek]

Steve
Ummm. No.
FAT hasn’t been used for a long time.

“File allocation table” is a concept. Yes FAT12/FAT16/FAT32 are three implementations, one widely used today (FAT32 is often used on flash disks). Since you mentioned NTFS, that also uses the same concept. NTFS uses a relational database called the Master File Table (MFT) to store it’s file allocations of the data store. Like FAT it has two copies and utilities like chkdsk run many consistency checks within the database and between the two copies, among other things. So FAT, MFT, same concept, but these details aren’t really necessary to answer the op’s question.

The drive hardware does have error correction built in. No magnetic recording medium is error free and error correction is used to hide those from normal operation. Further, the act of reading and writing (copying) adds another level of error detection. If there are bad sectors on the hard disk, your copy will fail.

In the IT industry, “error correction” is a term often used to describe something that verifies either with the actual source or with a checksum of some sort. Consumer level memory does not have error correction for instance. I checked my own knowledge and found that you are indeed correct. Consumer harddrives do have ECC built into them. For the Op, ECC will only verify during reading or writing, but won’t help when the spindle fails.

As I mentioned to the Op, hard drive archival is a viable option, but with it’s own caveats.

Cory

—
Cory Petkovsek
Corporate Video
http://www.CorporateVideoSD.com

[Walter Soyka]

[Cory Petkovsek] “Finally, with moving parts, harddrives eventually will fail when the electronic motors give out. Thus plan for it failing eventually and you won’t be disappointed. If you only use the drive once in a while it may last 50 years; who knows? If the motors give out, there is a high likelihood that the data is still readable on the platters of the disk. However you have to take it to a specialty shop for recovery and will spend $1500-2000 for a data dump, with no guarantee.”

This is why I’m leary of disk drive-based backups. They are an integrated package: the controller, the motor, and the read/write mechanism are all bundled with the actual storage medium. A failure of any one of them means data loss, and as you noted, repair is not trivial.

Tape separates the storage medium from the control and access mechanism, so it’s more robust over a longer period of time. If the tape drive dies, you replace the tape drive, and your tape library will remain accessible.

Cory, I completely agree with you that duplication is necessary for archiving. But if you’re going to consider a hard drive-based archive, I think that Steve’s systematic and frequent refreshing of the archive is also critical.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog – What I’m thinking when my workstation’s thinking
Creative Cow Forum Host: Live & Stage Events

[James Fuller]

Thanks to all for the feedback. As usual it seems there is no one correct answer – but all of these thoughts are helpful as I determine the most appropriate strategy.