Dan Montgomery
Forum Replies Created
-
Thanks Tim for reminding me why I don’t normally post in forums (and won’t comment beyond this).
Your opinion (and livelihood) apparently depend upon a proprietary system somehow being more reliable than LTFS. Sorry, I don’t buy it. There are many petabytes stored using this open standard and it’s growing, not declining.
Heck, there’s even an article in this month’s TV Technology about OpenAXF using LTFS…”With LTFS, the industry has a data tape interchange standard that support file system access.”…”Using LTFS is the key to having interchangeability between third-party data tape systems.”..XenData
Yes, there have been many ‘flavors’ of LTFS but as you know that’s to match the manufacturer’s equipment. It’s no different than seeing continually updated ATTO drivers so the SAS card will work with the latest OS version so you can mount YOUR tape drive.
And BTW I was just at Discovery and they made no mention of any problems mounting tapes. Ever.
So bottom line Sam: Don’t be afraid of LTFS. It will be around long after we’re all gone.
Offload with Confidence…
-
Sam,
We introduced PreRollPost at NAB 2012. Our company has been providing workflow solutions for the video industry since 1991–including many cataloging, backup, transcoding and library solutions on various platforms. (The 2002 Winter Olympics in Salt Lake used our Windows library system for their archives.)
Just a couple points to consider…LFTS is an open standard. That means there are *many* solutions out there supporting it, now and in the future. Tapes are interchangeable between them, and you can always get files off the tapes even without a given manufacturer’s software.
When we were repeated asked to provide a LTFS solution it became immediately apparent that the big pitfall for software only solutions (where we’re not providing the turnkey box as a pre-loaded appliance) was the driver installations. So PRP does all that for you with a single install click.
PRP has built-in tape spanning and tape duplication features. For spanning, we know video and break backup files logically so there is no tape dependencies (e.g. a given camera created volume will not be split between two tapes).
For best practices you should always make at least two copies of archives and store them in different locations. (If the sprinkler goes off it doesn’t matter how the tapes are formatted they’re going to be ruined.)
PreRollPost can control two or more tape drives at the same time to write identical tapes for you. Or you can mount and duplicate a tape-to-tape later at any time.
As for the power interruption scenario, first of all something as essential as a backup station should be on a UPS if for no other reason than power conditioning. But if a write session is stopped by power outage LTFS does have recovery options such as mounting the tape in ‘Read Only’ mode or deep data recovery tools. These are built into PRP to assist you.
Let me be clear, you can damage any tape by power off during a write session–Not during a Read.
LTFS also is improving. It also has built-in self healing by reading bytes after they’re written and automatically re-writing them if need be.
The current version of PRP supports all 3 Discovery tape types and has been certified by them. Also, we’ve included Discovery Metadata Editor to create the required metadata XML files. There’s also a Discovery Tape Verifier feature, so you can check your tapes against Discovery’s rules before delivering–regardless of how you make the tape–.
I’m a little surprised by your recovery speed issue. One of the advantages of LTO-6 is improved retrieve speed. However, to get the best performance the data needs to be organized and called up in an efficient manner. Perhaps you were simply using Finder to dump files onto the tape and trying to bring them back in your initial tests?
Writing, searching and retrieving are the main reasons for using a software application or turnkey appliances when working with LTFS (or any proprietary format for that manner).
One of the things not mentioned by others in this thread is that PreRollPost is the only solution that also offers visual indices. Besides automatically extracting video metadata, it also generates thumbs for your content. You can also add proxy generation and instant playback of video content in native or proxy formats. If you choose to make proxies, these can be done ahead of the backup session or during it.
Good luck in your search and tests Sam. If you have any questions feel free to ask via our website. Dan
Offload with Confidence…
-
Thanks for the error log. It shows the checksum failure is on SPOTLIGHT files.
In Preferences, turn off copying Hidden Files/Folders and this will not happen.
Video logging is just the beginning…
-
Just wanted to let you know the problems detailed in this thread with ShotPutPro v5 have all been addressed and available in the V5.0.3 patch. Including: drag & drop destination paths, MD5 failures on zero byte and empty files & folders, eject problems with certain card types and readers (e.g. TB connections) and more…
The comment about an added VOLUMES folder in the path was previously corrected in v5.0.2 — that had to do with combining volumes into a single offload.
With V5, copy settings are inside Presets instead of global. This gives added flexibility when dealing with multiple cam shoots and where you offload to. However, just like in the older versions, the attributes of a particular offload are set at the moment you drop anything into the queue. So you must select these items before adding volumes to copy.
To see the full list of changes we’ve added, click the Versions link under the Support menu on our website.
Video logging is just the beginning…
-
Johnny,
I read something recently on the REDUSER forum about Al3xa data manager corrupting files.. You may want to check over there for tips.
(What I read sounded ugly.)
Video logging is just the beginning…
-
John,
Glad to hear you got it working!
I’m jumping in here to subscribe to any posts to this thread as there are still people struggling.Doesn’t appear to be anything we can do with ShotPut Pro to alleviate the pain of driver conflicts, etc.
Video logging is just the beginning…
-
HD-VU is designed as an ‘all-in-one’ player of virtually any camera original media. In that sense, it’s a general purpose viewer, but doesn’t rely upon QT codecs for most formats. That said, it’s not overpriced and is becoming quite popular 😛
The problem with your desired workflow Chris is most readers (of AVCHD cards and other types) can’t be used for reliable, smooth playback due to the connection to the computer. I suspect this is the root of your comment about the software seeming ‘buggy’.
Perhaps as Thunderbolt or faster connection readers become more prevalent a workflow of playing from the source to select items for offload may be more practical than today. (Mac’s don’t even support USB 3 ports today!).
The purpose of HD-VU to to watch offload destinations (already on a hard disk) and allow quick checking of the clips before erasing the cards.
To allow individual clip selection (ala Log & Transfer) begs the question what format are you coming from and going to. With a NLE you’re generally ingesting into that editors format (e.g. QT ProRes) at the time of selection.
To pick a few clips and make an all new, say, AVCHD volume means you have to edit the folders, files and metadata to create a new volume that’s recognized by any editor as if it came directly from the camera.
Changing originals opens the door to people accidentally not saving all the clips they recorded/wanted. In our terminology means you’re asking for a LOGGER not an Offloader, (because you’re changing things).
I’m sure there’s a subset of intelligent folks that want that, but the vast majority using ShotPut Pro are simply in a hurry to make 2-3 pristine, verified copies of the entire originals. That alone is a pretty important, significant task given the wide variety of formats, equipment and operating systems.
I guess what we’re seeing is the clip selection process has shifted from a logging/pre-post-production phase to the ingest step into the editor. A driving force behind that too is the declining cost of hard disk space, even if used temporarily.
Video logging is just the beginning…
-
John,
As we discussed, ProxyMill is a non-destructive application. It only reads from a source, it does not modify the files there.
It is possible of course to set the output location of PM to anywhere you wish, but I don’t get the sense that you were attempting to write back to the card.
More likely the issue is with Spotlight. Any time you read media with a Mac you run the risk of Spotlight indexing the contents and writing a small index file back onto the card/disk etc. These are only a few kb, but if the card is entirely full one or more of your original source files may be damaged.
I’ve seen this repeatedly with P2 users who fail to physically lock the card before attaching them to a Mac. With P2, it was always a large file (the video MXF file) that was shortened, thus corrupting the last few frames of the clip.
In your case, if this is indeed what’s going on, it’s probably one of the .thm files getting changed that’s causing objections from FCP. The bad news is CF cards don’t have a physical lock, and it’s very difficult to turn Spotlight off.
As other respondents suggest, one should always work from copies, not the source material, just for reasons such as this. ProxyMill is designed to work in conjunction with ShotPut Pro so you can point it at one of your output locations and process the files as they become available on the copy disk.
As for the timecode, keep in mind the numbers are simply labels. They do not affect the number of frames in the clip. Many non-broadcasters do in fact use Non-Drop timecode, though TC itself is becoming less and less consequential. We will consider adding an additional choice in the TC preferences into ProxyMill for non-timecoded media.
Video logging is just the beginning…
-
ShotPut Pro does not modify files or contents of folders. It does however compare that the copy exactly matches the original.
The only other explanation is the copy was stopped midstream.
Check the offload reports Josh. You should be saving them with the offloads to refer back to, particularly if you’re having issues.
Might want to switch to byte-byte verification too. With File Size comparison we rely on what the OS tells us about the final files, instead of comparing each byte as it’s created.
(BTW, I’m assuming you’re using the latest v3.1.2 software.)
Video logging is just the beginning…
-
Josh,
Since the problem as you’ve described relates to spanned cards, not clips, I’m guessing FCP is looking for the ‘lastclip.txt’ file and not finding it on the copies you made with ShotPut Pro.
This can happen if you drag just the CONTENTS folder into the ShotPut Pro processing area–you’re telling it you only want to copy what’s inside that folder.
Be sure to drag the card from the Attached Media list (lower left), or the card itself from the desktop into the processing list. Doing it either of these ways will ensure you copy both the lastclip.txt and the CONTENTS folder.
Video logging is just the beginning…