CatDV Re-archive footage with Cache-A

[Lindsay Simpson]

Okay, so I realize I’ve been told on a few occasions now by different sources that file/media type shouldn’t matter, but I decided to do a bit of testing anyways.

I took a single clip from each of our media types and converted each of them into 8 different formats with a QT wrapper. The formats ranged from ‘Animation’ codec to basic DV NTSC. I also threw an AVI and H264 example in the mix as well. I then imported these files,which are sitting in an isolated location on our raid drive, into a blank CatDV catalog. Next I fired up the CacheA, re-initialized a tape I have been testing on, configured the CatDV plug-in to point to the correct VTAPE, highlighted a portion of the clips no more than 20 to 30 GB at a time, and started the archive using the CatDV plug-in. Not a single one of these files transferred without error ‘Resource temporarily unavailable’.

I moved a few test files over to our C-drive to try an archive from there and eliminate the Raid as a factor, but had the same error filled transfer result.

At this point I feel there has to be something funky going on with my CatDV software/plug-in. Rolf, CatDV, can you guys chime in? I’ll try you through your formal support contact as well.

Thanks,

Lindsay

[Rolf Howarth]

Sorry to hear about the problems people here are experiencing with the archive plug-in.

It’s true that spanned volumes are not supported.

What Tom says about restoring manually is not quite true however, as CatDV 9 will ask the Cache-A to restore files from tape to vtape. For this to work you need a reasonably up to date version of both CatDV and Cache-A firmware (within the past 6 months say), and need to enter the hostname of the Cache-A device, not just the vtape share location. We have recently learned that this may not always work realiably however: it seems that the request to restore files might sometimes not return as quickly as we thought, so we may need to increase an internal timeout value to prevent errors when issuing the restore command.

We are also aware of an issue specific to the Mac that can cause problems, both when archiving and restoring. If the Cache-A volume goes offline while using CatDV that CatDV might in some situations then create a folder on your local drive called /Volumes/Cache-A , which means that when the Cache-A is next mounted it will get the name /Volumes/Cache-A-1. To see if this has happened to you, use Finder > Go > Go To Folder and type in /Volumes. If you unmount the Cache-A and still have a folder by that name, move that folder to the trash (first checking to see if it contains any “archived” files!), then remount the Cache-A and reselect the vtape folder in the CatDV archive config settings. CatDV is supposed to protect against that happening so we’re trying to find out under precisely which circumstances one of these phantom folders can appear.

We are investigating both these issues and will issue an updated version of CatDV shortly with any changes as necessary.

Regarding not being able to archive to a tape, eject it, load it again, and add more files to it I’m not aware of anything in CatDV that would stop that working, so that may be a Cache-A issue. Can you add more files to an existing tape if you use drag and drop from the Finder (or Explorer)?

-Rolf

[Jeff Schaap]

[Rolf Howarth] “it seems that the request to restore files might sometimes not return as quickly as we thought, so we may need to increase an internal timeout value to prevent errors when issuing the restore command.”

YES. This happened to us the yesterday. We called for a restore and Cache-A restored everything to the vtape perfectly but after we got a timeout error message for CatDV and nothing was restored to our SAN. We repeated the request with the plugin and then CatDV pulled everything from the vtape without issue.

[Rolf Howarth] “. If the Cache-A volume goes offline while using CatDV that CatDV might in some situations then create a folder on your local drive called /Volumes/Cache-A , which means that when the Cache-A is next mounted it will get the name /Volumes/Cache-A-1.”

Yes again. We just found out this happened to us. We’ve had one station that was using the CatDV plugin fine and the all of a sudden would not archive files any longer. Every time we tried it would say “Unable to create directory [target directory]”. We just went and checked and sure enough the phantom folder was there. I think Nathan was getting this same error.

[Rolf Howarth] “Regarding not being able to archive to a tape, eject it, load it again, and add more files to it I’m not aware of anything in CatDV that would stop that working, so that may be a Cache-A issue. Can you add more files to an existing tape if you use drag and drop from the Finder (or Explorer)?”

We haven’t up to this point. When dropping stuff directly into the Cache-A (without the CatDV plugin) we have only done it in single sessions. But I will try a multi-session (ejecting the tape in between write sessions) and see what I get.

Rolf, is there a difference between a software update from Cache-A and the firmware update you are talking about? Tom, how do we know if we have the latest Cache-A firmware (we have software version 2.1.15)? How do we request a firmware update if we need it? We bought our Cache-A in July 2010.

Rolf and Tom thank you so much for dialoging with us about these issues.

Jeff

[Rolf Howarth]

[Jeff Schaap] “Rolf, is there a difference between a software update from Cache-A and the firmware update you are talking about? Tom, how do we know if we have the latest Cache-A firmware (we have software version 2.1.15)? How do we request a firmware update if we need it? We bought our Cache-A in July 2010.”

Sorry yes, I meant a Cache-A software update. I believe 2.1.15 is the current version.

[Jeff Schaap]

Hey All,

We did more testing this past week. Rolf, we do seem plagued by the time out problem you mentioned. Here’s what we did:

1) We wrote 650 files, about a terabyte’s worth, to LTO through CatDV without any issue.

2) We ejected the tape and deleted the vtape then reinserted it. We called for a restore of all 650 files. We got a time out error (after about 2 minutes). We tried to restore one file and that came back fine. Left for the weekend.

3) Monday, we call for a restore of the 650 files. After about a ten minute pause it took off with the restore without issue. However, it only restored 251 of the 650 files and CatDV said it was “unable to restore” the rest. They were also not on the vtape.

4) We call for another restore. It skips the ones it already has done and recalls the rest from tape without issue. So it works, sorta.

There seems to be some sort of issue with CatDV timing out when there is a large amount of files to be restored. Rolf, what exactly is happening at this stage that is causing the system to pause so long? Is it building and transmitting a list of files to restore to the Cache-A?

We are performing the multiple write to the same tape tests right now (without using CatDV) and we’ll let you know what happens there.

Jeff

[Tom Goldberg]

Right 2.1.15 is the latest and greatest. You can always check for the status of all our releases on our support page and just click on the SW update request button to get an update scheduled for your serial number.

Warranty or not, we’ve always provided software updates for free.

That said… a heads up, we are going to v3.0 soon and that will be a paid upgrade, pricing TBD. For 3.0 we’ll be moving to CentOS6.2 64 bit and it will be distributed on a flash drive. Keep an eye on our support page for this announcement and details.

Tom Goldberg
Cache-A Corporation
602 Park Point Drive
Golden, CO 80401
mailto:tom.goldberg@cache-a.com
https://cache-a.com

[Rolf Howarth]

[Jeff Schaap] “There seems to be some sort of issue with CatDV timing out when there is a large amount of files to be restored. Rolf, what exactly is happening at this stage that is causing the system to pause so long? Is it building and transmitting a list of files to restore to the Cache-A?”

Using the Cache-A API, CatDV queues a list of files to be restored one by one. Normally adding individual files returns very quickly, so there’s a timeout of 5sec, but it seems that occasionally the Cache-A doesn’t respond in time so it times out. We’ll increase the timeout in the next release of CatDV but for now you should be able to wait a little while then repeat the restore command until it successfully restores all the files. Apologies for the inconvenience.

[Jeff Schaap]

[Rolf Howarth] “Normally adding individual files returns very quickly, so there’s a timeout of 5sec”

So, if there is small delay on the transmission of each file then it wouldn’t be abnormal for it to take five or more minutes to transmit 650 “file restore requests” right? We thought maybe something was wrong when we were getting such long pauses. It would be great to have a progress bar or something to show that it’s working.

Can you say when the next version will be available?

Thanks for your help,
Jeff

[Jeff Schaap]

Hello All,

I have the results from our attempts to do multiple write sessions to the same tape using just the Cache-A interface. Here’s what happened:

1) We copied a few files to the staging folder and then dropped them into the vtape. Ejected the tape, clearing the vtape and then re-insterted the tape. Wrote two more sessions (3 total) repeating this procedure of write, eject, erase vtape, reinsert tape.

2) We called for a restore all file from the tape to the vtape. Cache-A went into a pending status and never did anything. Eventually we had to restart the Cache-A.

3) After a restart the Cache-A said the vtape and and tape were not a match. It only offered three options and selected “eject the tape.” Then we re-inserted the tape and called for a restore of all files and they all came back without error.

So, Rolf, with some caveats, it does appear that the system can recall files from a multiple session write without issue. So I am left to wonder why this isn’t working when we do it with CatDV. Have any thoughts on it?

Thanks,
Jeff

[Tom Goldberg]

Jeff,

Just a note, not sure why the Cache-A would have hung at pending, but if that kind of thing happens, restarting is undesirable and slow – you are better off going to System Tools page > Utilities tab > Restart Tape Manager button – much quicker and safer.

Tom Goldberg
Cache-A Corporation
602 Park Point Drive
Golden, CO 80401
mailto:tom.goldberg@cache-a.com
https://cache-a.com