SAN Backup

[Matt Geier]

Hey Nick,

Have you looked into EMC’s Retrospect?

https://www.retrospect.com/products/software/retroformac/

According to a PR I read from January 09 (https://www.retrospect.com/press/news/?id=310) this is in your price range when it was announced by EMC……

Retrospect 8.0 for Mac will ship during the first quarter of 2009 in three English language editions, (all prices listed are U.S. suggested retail pricing):

Retrospect 8.0 Desktop 3-User, $129 new, $59 upgrade from Desktop edition

Protects a single, non-server Mac and up to two additional networked Mac, Linux, or Windows desktops and laptops; additional desktop/laptop client licenses and support and maintenance will be available separately;

Retrospect 8.0 Single Server, $809 new, $539 upgrade from Workgroup edition

Protects a single Mac OS X Server and an unlimited number of networked Mac, Linux, and Windows desktops and laptops; includes 1 year support and maintenance; additional server client licenses will be available separately;

Retrospect 8.0 Multi Server, $1,669 new, $939 upgrade from Server edition

This is the one I hear good things about from others that use it.

Good luck!

Matt G

[Bob Zelin]

Hi Nick –
you are not going to backup an entire 16TB array on a regular basis. You backup individual projects. A Cache-A LTO4A is $7995 (and about 60 bucks for a 800 Gig LTO tape). And it’s slow. And what is 800 Gigs going to do for your 16 TB Cal Digit array ? Nothing.

There are many options, some are expensive. Small Tree sells a wonderful NEAR LINE ethernet back up system, but I know you – you won’t want to spend the money. So you buy inexpensive SATA drives, and you back up individual projects. Products like BRU and Retrospect were used with Quantum backup tape storage, but Cache-A outdated all of this, and they provide their own web (safari) based GUI over ethernet – but it’s slow (and it’s $7995). Believe me, you don’t want an outdated Quantum DLT (which will require an LSI Logic SATA host adaptor in one of your MAC’s).

Get some SATA drives, and backup individual projects. There are many wonderful brands out there – you know who they are.

Bob Zelin

[Nick Hasson]

Hi bob,

Let me tell you about what just happened. I lost an entire 16TB volume. Poof, gone. Very rare, but it can happen. So Yes i do want to backup an entire 16tb raid. this way if that raid dies, I do not lose anything. I can not go thru what i just went thru again. I want to walk out every night knowing that backup software will run and take care of it. It does not need to be super fast. 120MB a second would be perfect, even 80MB. The first night that i create allot of media, the backup might take long, but each night after that, it would only backup whats new.

5-7k doesn’t seem like i’m not trying to spend money here. The Online storage was only 7k. Nearline storage for the same price does not seem crazy. I don’t want to just go buy cheap drives.

My idea was to buy at least a 16tb raid from a company with support (maxx digital, cal digit, duce, etc, and have backup software mirror it every night. The cost of the raid should be cheaper than my online storage. But still a raid 5.

Nick Hasson
EDIT – Smoke artist
http://www.niceedits.com

[Bob Zelin]

I know that you have a Cal Digit HD Pro. This is an excellent product – DO YOU HAVE A UPS on your drives ?

Obviously, I can recommend the Maxx Digital line, that uses the ATTO R380 card with Hitachi enterprise drives, but Nick – with no UPS, not even this drive array will save you, if you take a major power hit.

Bob Zelin

[Nick Hasson]

I do have a ups. Power hits can ruin your whole setup in a second.

Nick Hasson
EDIT – Smoke artist
http://www.niceedits.com

[Dave Klee]

Hey Nick, normally I would try to talk you out of backing up your whole RAID unless it has nothing but archival material on it — for example, tapeless files, original images or other original content that cannot be replaced. I would argue that on any RAID that’s used in video production, there are pieces of it worth keeping and pieces not worth the time, effort and expense to backup. Is it worth backing up terabytes of render files that could be re-rendered if lost? Is it worth backing up capture scratch that could be re-captured off tape (if properly logged)? Unless you’re in a high-throughput environment where you need close to zero recovery time, probably not.

But, you know your life and your world a lot better than I do, so I’m not going to try and talk you out of anything.

As you and others have said, backup is both a software and hardware issue. Some vendors do both, but often it’s separate.

For Mac backup software, Retrospect is an industry standard. Personally, I don’t enjoy it. If you’re a Retrospect person and have been using older versions, I can’t think of a good reason to ditch it. But, if you’re looking for something new to do simple backups, I think it is way more in-depth than you need, and not as user-friendly as it should be. Plus it’s more than you really need to spend.

My current favorite backup program for Mac is ChronoSync: https://www.econtechnologies.com/pages/cs/chrono_overview.html I don’t work for them, don’t care if you use it, but it’s made my life easier and I think it’s worth the $40. There are a few other mid-tier backup programs out there in that price range that I would recommend over Retrospect for most people.

For hardware, I feel like you’re aiming low for a good 16TB RAID for $5-7K. Honestly, like you said, you just need the thing to be reliable — speed isn’t that big a deal as long as you can meet an overnight backup window for all the changes made that day, and maybe a weekend window for an occasional full backup (in most environments).

While nothing like the top-notch stuff advertised here, I think the DroboPro makes a great backup solution. If you’re really nervous, you can put it into RAID6 (or RAID5) with e-mail notifications. Speed is not fantastic, but so far for me it has proven extremely reliable for very minimal cost. You can get a 16TB kit for somewhere around $2,600. And, if that’s not enough space when RAIDed, you can buy two and still meet your budget.

Besides cost, there are two key objectives to keep in mind for backup solutions: Recovery Point and Recovery Time. Recovery Point is the point you want to get back to after a failure: i.e. what is acceptable to lose if something dies. Recovery Time is how long it takes you to get back up and running. Getting both of those close to zero (zero loss and zero downtime) can be expensive and complicated in a video environment where you can’t afford the cost or performance hit to mirror everything all the time in real time. But just know it’s a trade off — zero loss generally means more down time, and less down time generally means higher loss (assuming you are keeping costs low).

Making smart decisions about what is REALLY important to you now — about your Recovery Point Objective — will keep both your costs and recovery time lower in the long run.

Good luck with everything!

[Chris Blair]

While Retrospect seems very popular, I’ll second that it “sucks!” We’ve used it for nearly a decade and I’ve tested probably 25 other backup programs during that time trying to replace it….but alas, few were any better.

A couple that we have moved to and completely ditched Retrospect are Second Copy, which isn’t a true back up program in that it’s designed to copy your files to a backup location (hence the name). Although it does offer compression, incremental backup and lots of other features similar to regular backup programs. It has NEVER failed and we’ve never had it corrupt a backed up file in over 4 years of use. Best of all it’s like $39 per license.

Another one we recently found is from a well respected IT company that a lot of IT pros use called GFI. They have a backup program that does traditional backup, it’s incredibly easy to use and in our testing seems reliable. And get this…it’s FREE. There are a couple small bugs in it’s interface, such as clicking on a restore icon will gray it out as if it’s not working. All you do is arrow off it from the keyboard then back onto it and it restores the functionality.

We’ve moved to using these two programs not because they’re cheap but because they work and they’re simple for non IT people to setup and use. As for back-up hardware, Apace makes near-line storage are pretty affordable prices, although I don’t think you’ll get 16TB for $7000. I agree with Bob, get SATA drives. You could even build a raid 5 or 6 with SATAs and get added speed and redudancy if one or two of them fail. But I bet you’d spend at least $5000 building a 16TB SATA backup in a simple RAID case.

Chris Blair
Magnetic Image, Inc.
Evansville, IN
http://www.videomi.com

[Eric Hansen]

hey nick

that sucks man. i’m sorry you had to go through that. i agree with everyone’s suggestions here. i’ve never tried Drobo though. i’ve heard not so great things about the standard Drobos mostly related to their speed, but the Drobo Pro as an iSCSI device could be really nice. the fact a 16TB one is only $2800 currently, is pretty cool. and i will “third” that Retrospect sucks. a facility i work for went with BRU PE with LTO-4 and i couldnt be happier. we deal with way too much media to consider hard drives as an archive solution. this is the system that we’ve set up to deal with your scenario of all volumes disappearing:

we have a backup server with CrashPlan software. it fully backs up all the boot drives on all the edit systems. it also backs up certain target folders on the SAN, such as project files and gfx. it’s an incremental backup system, so it will save many versions of a file. we don’t use this on media though.

as footage comes into the facility; field drives, cards, tapes, etc; everything is placed in a holding tank on the SAN. footage is then converted into its necessary format. we work with a lot of formats such as Cineon that can’t be edited directly. so we transcode everything to ProResHQ 1080p23.98. then, the original files and the transcoded files are copied over to LTO-4 tape (2 sets). then the ProResHQ files are placed on the SAN volume that the editor is using. once the footage is on the LTO-4 tapes, it is deleted from the field drives (we always make sure there’s 2 copies of all media, so field drives don’t get deleted until that footage is both on the SAN and on LTO-4)

so we’re doing 2 things at once here: we’re copying all our footage to LTO-4 for archive, but also indirectly creating a backup of the SAN. anything that’s not included in the original footage, and isnt a render file (such as GFX or sound) is backed up by CrashPlan.

i would suggest something like this instead of a full RAID backup because of cost, and it forces you to be organized with your footage. it also uses less energy because you don’t have to power all those RAIDs. however, i could see setting up a dual RAID system if you’re working with mostly source footage that’s not yours (you wouldn’t be archiving it anyway). then i would suggest a 16TB Drobo Pro because it seems to do what you need for a very nice price.

e

Eric Hansen – http://www.erichansen.tv

[Walter Soyka]

Hi Nick,

I’m facing a problem similar to yours; I’ve currently got a nearline NAS for immediate live project backup only and I’m thinking about adding an LTO-4 for archival. I’d be curious to hear about what you’ve done and how it’s working out.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog – What I’m thinking when my workstation’s thinking
Creative Cow Forum Host: Live & Stage Events

[Marcus Lyall]

Hi there,

I’ve got a similar setup.
Having got an LTO4 drive, we went through hell getting it to work.
We’re just about there now. Tried to do it on the cheap. Don’t.

First we bought Bru Server. Paid a lot of money. Spent ages configuring it. Lots of having to use Unix scripts to optimise buffering. Basically, it sucks. Boffin interface. Buggy code. Support who treat you like an idiot if you can’t run Unix shell scripts.

Then we tried the new Retrospect. That sucked too. As ever.

Then we got Archiware’s Presstore. Our IT guy recommended it. We like it. It works. It does max write speed without needing configuring. It backs up. Paid the IT guy to make the problem go away.

One other thing. Don’t buy a SCSI LTO4 drive like we did. The SAS version is way quicker.

Most of our projects come back on a yearly cycle, so we keep a live backup of everything. Each major project (normally 4-6 tb) we buy a Raid 5 4-bay enclosure. And run a Chronosync backup off the server every night. When the project is more or, the drive goes on a shelf and gets spun up every few months.

And we also back up to tape. To eventually take home. Costly, but we hardly ever use video tapes, so if you lose it, it’s gone.

Spend the money on getting the backup right. You will sleep at night.