Archiving EX footage to XDCAM disks

[Michael Slowe]

In answer to an earlier comment following my earlier post, I am concerned with archiving finished productions rather than camera media. It’s really the same question though – tape, disc (moving), drive (moving) or static card. To archive back to, say, an S X S card in the camera (possible through SDI?) would be an expensive option at current card prices but might be the safest option, what do others think about this?

Michael Slowe

[Craig Seeman]

Basically your options are Blu-ray or XDCAM disc (as data). The above discussion should give you pause for thought as both have long term pitfalls. If the master is small there’s always DL-DVD.

There’s LTO which we didn’t bring up but that has its on benefits and detractions (as long as there are readers, you’re ok).

We didn’t discus the issue of codec. You can go back to XDCAM MP4 quite easily in FCP. Some would use EX .MOV others would go to Apple ProRes. MP4 can go back to MOV or MXF or used as MP4 in the future so it’s sort of the “universal.” At this point Apple makes the ProRes decoder freely available crossplatform so would not be needed but QuickTime would be. There are third party plugins (Calibrated Software) for EX .MOV but if Sony/Apple’s EX FCP codec and FCP itself go away that can become a challenge to resurrect.

Personally I think the codecs will endure. It’s the media and the players that will be a problem as time marches on.

Solid state isn’t there yet. I’ve heard SDHC manufactures say data retention is at around 10 years. That’ will change but right now any of the optical discs mentioned above (if of good archival quality) would outlast that.

[Mark Raudonis]

[Michael Slowe] “I am concerned with archiving finished productions rather than camera media.”

Go to video tape. Specifically: HDcam or HDCAM sr. This will give you the widest range of professional options for future use. Sony has a very good record of making future formats play nice with old legacy formats. For example, the Digibeta decks that we have can play back just about every form of Beta ever made. Our HDCAMsr decks can playback the lower quality HDCAM format. For masters, I say you want to be on tape.

Makr

[Lucian Evans]

It took me a year before i bought an EX3 as i was mulling over the archive conundrum. I looked at all the archive options and found i would be spending more time dumping footage onto discs than i really wanted with most options. Finally i decided on LTO4 which so far seems to be very good value with an easy workflow. With LTO4 the tapes are 800GB and can be set up to automatically backup my entire edit or i can just drag and drop.

1 x 50gb xdcam disc = £45

1x 800gb LTO4 tape = £30

[Marvin Holdman]

I would have to agree with Craig on this one. Devices come and go very rapidly. The facility that I’m currently working at recognized this and bought into the Sony Petasite, which is a hybrid based tape storage solution with hard disk on the front end to expedite access. The thought behind this being lesser proprietary file based archival device is preferred.

Some will say that the this is a potentially obsolete device as well, but here’s the catch…

When the time comes to find a new device, it becomes a matter of data migration, which is simpler for a single point access device. In other words, we won’t have to hire someone to endlessly put a huge stack of BlueRay disc into a player to migrate our archives.

I think this discussion is fragmenting into two different subjects.

In house archival and deliverables archival (I’m not talking about finished masters here, JUST raw footage and perhaps the edit project).

In this respect, there is little practical choice for deliverable archives these days but a blueray solution (whether common Blueray DVD or proprietary Sony Blueray). Either of these usually represent short term requirements, usually to appease a hiring entity requirement that all media from a project be provided at the end of the job. In this case, I think the client drives that decision (what are THEY asking for?) and it is up to us, as producers, to provide this. Matters little what the best solution might be, only what is required to fulfill the contract.

Of course, you COULD go to tape with all of your raw footage, but who wants to do that?

Marvin Holdman
Production Manager
Tourist Network
8317 Front Beach Rd, Suite 23
Panama City Beach, Fl
phone 850-234-2773 ext. 128
cell 850-585-9667
skype username – vidmarv

[Craig Seeman]

Lucian, what is (or would be) your workflow to “reconstitute” an offline project?
One thing optical discs afford is the “random access” and fast copy times the LTO doesn’t have.
On the other hand LTO has very low cost of use for large amounts of data and its apparently a format that will remain in use (or at least backwards compatible) as advancements are made.

[Tim Allison]

This has been a good discussion, honestly, a LOT more than I expected. In our particular shop, we’ve already committed to the large-form XDCAM-HD format. We have the cameras and decks that handle those nifty plastic-case enclosed XDCAM disks. As I’m typing this, I have a Mac Book Pro sitting in front of me. It has a SxS card reader on the left side, and I have a PDW-U1 hooked up to the USB port on the right side. It just seems like it would be SO EASY to plug in a SxS card on one side, and transfer those shots over to a disk in the U1. Then I can put the disk on a shelf, to access whenever I need those shots.

Craig, you did bring up a scary thought which had previously escaped me….what if software support and development of the PDW-U1 stops, but the computer OS continues to the point that the U1 will no longer “talk” to the computer? Crap….I haven’t even thought of that….a drive that still functions perfectly, but at some point, the computer and drive may not communicate with each other.

I guess it’s just a gamble. While Blu-Ray presently looks promising for long term storage, there really is no guaranty that Blu-Ray will out last XDCAM, or vice-versa.

[Craig Seeman]

Good arguments Marvin.
Delineating between in house and deliverable archive is important. I’d venture to say that in the longer term (you mentioned shorter term) there is less of a difference.

In the case of the 2″ preservation project, the original entity once had many 2″ machines and eventually had NONE nearly 20 years later so the process of finding a facility with a working 2″ machine was expensive I’m sure. They wen’t with D2 as the format to archive to so I must imagine that process must have happened again.

Mark made mention of the PDW-U1 having few moving parts. Devices can still fail at a point where finding new parts or a replacement device difficult. I’ve seen this with Syquest, Jazz, Zip drives although none of those are really like an XDCAM device. I have seen CD and DVD readers whose lasers age and fail over time. The ability to focus declines. I am concerned that a laser based XDCAM disc reader may eventually suffer from declining or failed lasers as well. Certainly there’s the risk that the appropriate extension/drivers may not be maintained as Operating Systems update. Look at Sony’s handling (lack of) for the .kext and Snow Leopard on Mac.

I can certainly envision a day where one may be faced with failing laser, lack of driver for recent operating systems, becoming a major road block. This is why I again reiterate we must look at both media and player availability into the future when we think of long term archival.

Marvin brings up the important point of the ease of moving the files to a newer media as part of the process.

[Craig Seeman]

[Tim Allison] “Craig, you did bring up a scary thought which had previously escaped me….what if software support and development of the PDW-U1 stops, but the computer OS continues to the point that the U1 will no longer “talk” to the computer? Crap….I haven’t even thought of that….a drive that still functions perfectly, but at some point, the computer and drive may not communicate with each other.
“

It seems Snow Leopard broke PDW-U1 compatibility. All new Macs are Snow Leopard. When I upgraded my MacPro to Snow Leopard the OS reported that the PDW-U1 extension is not compatible. Sony is apparently MONTHS from fixing this. I certainly can NOT TRUST Sony to diligently maintain OS compatibility. Yes it is quite possible that there may come a time a perfectly good PDW-U1 will not function under a current OS and you may not be able to install the old OS with old components/extensions/drivers. Sony’s handling of SxS driver, the PDW-U1 extension, the XDCAM Transfer software does NOT give me “consumer” or “professional” confidence in Sony’s long term support. Sure it’s my very subjective opinion but the “breaking” of the above are quite real. Only the SxS driver has been fixed to date.

On the other hand it would be fairly easy to hook up an external Blu-ray burner/reader with USB, Firewire, eSata and get access to the data on the disc. There are several manufacturers that make such devices and if one drops from the market I can look to another manufacturer.

I am not a fan of proprietary devices especially when the come from a single manufacturer. Granted 2″, 1″, D2 had support from the “major players” and all those machines are not easy to come by in working order. Syquest and Iomega had drives/readers that were proprietary but quite popular. Many computers had Zip drives built in. So even wide use doesn’t mean continued manufacturing support.

It’s just a hunch but I suspect solid state (SD and CF) will be as standard and backward compatible as CD and DVD. Readers will certainly advance (Blu-ray can read and burn CD and DVD) but there is interest from the manufacturers in maintaining backward compatibility and “instant” support as OSs advance. That’s why I think solid state will be the future.

Again subjective but I am NOT yet confident in the long term support for XDCAM disc. I don’t think Beta is quite direct comparison in part because Beta tape formats endured and dominated for a long time yet I think I do not think XDCAM disc will (certainly not 10 years for example).

I have mixed feelings about Blu-ray since it is not “built in” to computers like DVD is but there are many manufacturers making burner/readers though.

Sorry if any if this is redundant but I keep clarifying my thoughts as I go along.

[Mark Raudonis]

[Craig Seeman] “Again subjective but I am NOT yet confident in the long term support for XDCAM disc.”

Geez Craig, I think you’re a “glass is half empty” kind of guy!

Nothing is for certain in this life. That is for certain.

We can argue semantics, but I’d say that Xdcam is a good bet for the long term. Sony has proved throughout my career to consistently have the technology and corporate will to create formats that
last.

Mark