-
Purchase decisions between BRU Producer Edition or Image Products PreRoll Post
-
I’m still evaluating the two apps. Only when one uses each after a good period, then its strength and weaknesses will be realized. Still only have a single LTO-6 drive for testing purposes. LTO-6 is not fast and it takes time especially backing up raw camera footage files.
On Pre Roll Post, will that PRP LTFS created backup be readable in BRU? I don’t really mind using LTFS. But the drivers needed is a concern. Because I use LTO-6 strictly for very long-term archival and not backup, my need is to be able to restore that data many years from now and not having to dig up obsolete older version of LTFS driver just to be able to restore that data.
I used to do tape backups in the late 90s w/ DDS-3. Data capacity scale was in the 18-50 Gb range. Hdd was a premium. Used to remember a 4 Tb Seagate external SCSI-2 drive costed about $1K. Nowhere close to the TB range as seen today at 1/10 the cost. DDS-2 was quite unreliable for media archives and literally abandoned it for almost 20 years. DLT was hot during the early 2000s. But didn’t really bothered to get into it because it was in the $4K range. Good thing I did not because DLT won’t be able to archive Tb of data economically. Fast forward to now I’m out of touch w/ the latest in backup apps & software. I prefer not to use turnkey systems. Prefer the software and tape drive solution only. But whatever it’s one thing for certain is that hdd will go bad on you at anytime – especially when it’s sitting on the shelf for years w/ out powering it. About 8 of my 150 3 Tb hdds (only after 3 years) are all experiencing sector read errors. I don’t know about LTO-6 media gone bad years even after full verify during the archive process. This hdd read sector error is unacceptable for my needs and LTO-6 is the only viable solution that is appealing in cost and overall flexibility.
-
Kevin – while you have run into some of the GUI issues that we’ve suffered with our UI through the versions, one thing that we’re very proud of is that at no time has a BRU backup left you with a tape or disk based archive from which you could not restore. This cannot be said for any LTFS implementation.
For a perfect test example – select a few 10’s of GB of data and start writing them to tape with BRU. Before the job finishes, shut the tape drive off (simulating a power loss. hard tape failure or system failure). BRU PE will complain and the job will abort. Now, perform the same steps with a mounted LTFS tape – drag and drop the same files to the LTFS tape and before the copy completes, power off the drive.
From this situation, everything that BRU had written onto the tape can be easily restored. However, you’ll find that you can’t even remount the LTFS volume. Now, imagine that you were actually 2TB into a 2.5TB job. Where would you rather be with that data?
Just saying’ …
Tim
—
Tim Jones
CTO – TOLIS Group, Inc.
https://www.productionbackup.com
BRU … because it’s the RESTORE that matters! -
[Sam Lee] “will that PRP LTFS created backup be readable in BRU?”
As long as the PRP formatting used a unique serial number for the tape as it was created, BRU PE will import the tape’s ToC and provide it in the Restore panel just like a BRU PE archive so that you can search it with the LTFS volume is unmounted. We distinguish LTFS tapes by their serial numbers (generated when the tape is formatted).
Tim
—
Tim Jones
CTO – TOLIS Group, Inc.
https://www.productionbackup.com
BRU … because it’s the RESTORE that matters! -
Sam,
We introduced PreRollPost at NAB 2012. Our company has been providing workflow solutions for the video industry since 1991–including many cataloging, backup, transcoding and library solutions on various platforms. (The 2002 Winter Olympics in Salt Lake used our Windows library system for their archives.)
Just a couple points to consider…LFTS is an open standard. That means there are *many* solutions out there supporting it, now and in the future. Tapes are interchangeable between them, and you can always get files off the tapes even without a given manufacturer’s software.
When we were repeated asked to provide a LTFS solution it became immediately apparent that the big pitfall for software only solutions (where we’re not providing the turnkey box as a pre-loaded appliance) was the driver installations. So PRP does all that for you with a single install click.
PRP has built-in tape spanning and tape duplication features. For spanning, we know video and break backup files logically so there is no tape dependencies (e.g. a given camera created volume will not be split between two tapes).
For best practices you should always make at least two copies of archives and store them in different locations. (If the sprinkler goes off it doesn’t matter how the tapes are formatted they’re going to be ruined.)
PreRollPost can control two or more tape drives at the same time to write identical tapes for you. Or you can mount and duplicate a tape-to-tape later at any time.
As for the power interruption scenario, first of all something as essential as a backup station should be on a UPS if for no other reason than power conditioning. But if a write session is stopped by power outage LTFS does have recovery options such as mounting the tape in ‘Read Only’ mode or deep data recovery tools. These are built into PRP to assist you.
Let me be clear, you can damage any tape by power off during a write session–Not during a Read.
LTFS also is improving. It also has built-in self healing by reading bytes after they’re written and automatically re-writing them if need be.
The current version of PRP supports all 3 Discovery tape types and has been certified by them. Also, we’ve included Discovery Metadata Editor to create the required metadata XML files. There’s also a Discovery Tape Verifier feature, so you can check your tapes against Discovery’s rules before delivering–regardless of how you make the tape–.
I’m a little surprised by your recovery speed issue. One of the advantages of LTO-6 is improved retrieve speed. However, to get the best performance the data needs to be organized and called up in an efficient manner. Perhaps you were simply using Finder to dump files onto the tape and trying to bring them back in your initial tests?
Writing, searching and retrieving are the main reasons for using a software application or turnkey appliances when working with LTFS (or any proprietary format for that manner).
One of the things not mentioned by others in this thread is that PreRollPost is the only solution that also offers visual indices. Besides automatically extracting video metadata, it also generates thumbs for your content. You can also add proxy generation and instant playback of video content in native or proxy formats. If you choose to make proxies, these can be done ahead of the backup session or during it.
Good luck in your search and tests Sam. If you have any questions feel free to ask via our website. Dan
Offload with Confidence…
-
Let me start by saying that I agree that PreRoll Post has brought a lot of goodness to the LTFS format for a far more reasonable price than other solutions. However, PRP is NOT the LTFS format. And, PRP’s ability to write to tape is at the whim of the engineers writing the LTFS specification unless you spin off your own code tree that your team controls. But then, you’re adding to the already existing 21 known LTFS implementations currently in the wild.
[Dan Montgomery] “Let me be clear, you can damage any tape by power off during a write session–Not during a Read.
Dan – that is not true. You can’t damage the tape through a power loss event. With BRU’s format (or tar or cpio for that matter), powering off the tape drive does not affect any of the data that has been written onto the tape up to the point of the failure. And even with the –deep-recovery option to LTFSCK, you still run into far more unrecoverable tapes than you should – this is a weakness related to the dual partition nature of the format.
[Dan Montgomery] “LTFS also is improving. It also has built-in self healing by reading bytes after they’re written and automatically re-writing them if need be.”
This statement is misleading and I’m not sure what LTFS code you’re looking at, but LTFS’ read after write verification is totally dependent on the tape drive’s algorithms, not something special in LTFS. Any software that writes to a tape device has gotten this functionality for free as far back as the QIC drives of the 1980s – it’s not an LTFS thing and has no relationship to LTFS. However, this functionality is highly affected by the GIGO effect – Garbage In, Garbage Out – in that the drive is only able to compare the data that it received in it’s input buffers. We have a white paper that describes the way this works on our site: Reliable Tape-based Verification. To perform software controlled Read After Write verification as you imply would slow the format’s write speed dramatically.
[Dan Montgomery] “There’s also a Discovery Tape Verifier feature, so you can check your tapes against Discovery’s rules before delivering–regardless of how you make the tape–.”
The problems related in my discussion were not in the formatting of the data in the strict Discovery layout, but rather that inability of the tapes to be read at all. This is an LTFS theme that continues to crop up time and again – the LTFS tape that you created today, can’t be mounted and read next week.
Additionally, we’ve (the tape industry members including Exabyte, Archive, HP, and a few no longer existing firms) demonstrated at public shows such as COMDEX, CeBIT, IBC and InterOp (plus many smaller regional shows) that modern tape media is much more resilient than nay-sayers promote. We froze tapes, soaked them in coffee, and even went so far as passing consumer-grade magnets over them and were still quite able to restore the data that existed on them after drying and warming the media to normal operating temperatures and humidity levels. So while I definitely agree that two copies at different locations is better than one copy in a desk drawer, the concern over getting an LTO tape wet and losing data is not a valid scenario. In fact, with LTO media, it takes physical destruction of the cartridge to really create a scenario where data cannot be retrieved from a tape.
Oh, and BRU PE has offered Doubler Mode (writing the same data to two tapes simultaneously) since 2006. We even sell Desktop, Rackmount, and library hardware packages that offer the Doubler Mode solution in a very easy to use, plug and play package.
One additional comment about your site – PreRoll Post is LTFS compatible, not LTFS certified or compliant. The compliance testing suite is still being defined and currently only applies to writers of the LTFS low-level driver tools. User-level applications are not currently being certified for compliance. The current list of compliant solutions is offered on the LTO.org site here (scroll to the bottom) – LTFS Compliant Solutions
Tim
—
Tim Jones
CTO – TOLIS Group, Inc.
https://www.productionbackup.com
BRU … because it’s the RESTORE that matters! -
Thanks Tim for reminding me why I don’t normally post in forums (and won’t comment beyond this).
Your opinion (and livelihood) apparently depend upon a proprietary system somehow being more reliable than LTFS. Sorry, I don’t buy it. There are many petabytes stored using this open standard and it’s growing, not declining.
Heck, there’s even an article in this month’s TV Technology about OpenAXF using LTFS…”With LTFS, the industry has a data tape interchange standard that support file system access.”…”Using LTFS is the key to having interchangeability between third-party data tape systems.”..XenData
Yes, there have been many ‘flavors’ of LTFS but as you know that’s to match the manufacturer’s equipment. It’s no different than seeing continually updated ATTO drivers so the SAS card will work with the latest OS version so you can mount YOUR tape drive.
And BTW I was just at Discovery and they made no mention of any problems mounting tapes. Ever.
So bottom line Sam: Don’t be afraid of LTFS. It will be around long after we’re all gone.
Offload with Confidence…
-
To others – I apologize ahead of time for this, but I can’t sit back while such a hit and run is posted. If you’re not interested in the deep facts that are being discussed, it’s probably best to ignore this posting.
[Dan Montgomery] “Thanks Tim for reminding me why I don’t normally post in forums (and won’t comment beyond this).”
So, because someone disagrees with you and points out invalid statements in your claims you shut down the discourse. Ignoring something is definitely one way to make it go away.
Nothing that I have presented is false or non-attributable. In fact, our LTFS Caveats document has stood the test of time for more than three years with a standing offer to update or even remove it if even one of the claims made can be truly refuted.
As for published articles, anyone with enough money can get an article published that promotes the topic of the vendor in question. We have been offered many so called “editorial opportunities” by pubs and sites if we were willing to purchase large ad blocks – it’s just the way that things work. Of course, the Xendata folks would make such a statement because they, like you, depend on LTFS to access any type of tape device. If neither of you had LTFS, where would you be from a tape standpoint? Our BRU products support both BRU format AND LTFS and we provide tools that make it possible to read and write tar and cpio tapes on OS X as well as reading MTF/MTIO tapes from the Windows realm on OS X.
Next, you don’t MOUNT our tape drive as tapes shouldn’t be mounted – tape is not disk. The ATTO drivers are included in OS X and Windows out of the box. While it’s a good idea to update to their latest versions, it’s not required and the drivers from the HBA manufacturers don’t change the format that is written to the tape itself. And, the LTFS layer STILL depends on those same ATTO (or other) device drivers for proper operation. Unlike updating an HBA driver, changing an LTFS driver can mean the difference between a drive working or not working. It can also mean the difference between an old LTFS tape being readable under the new drivers or not.
Additionally, BRU supports far more vendors’ tape devices than LTFS. LTO-1 through LTO-6, DDS-1 through DAT320, AIT, SAIT, DLT, SDLT, T10000A through D, and even antiquated QIC and Pertec devices, Additionaly, we support cloud storage standards such as SFTP and WebDAV in addition to local disk and even optical media. And, the archival container format is the same across ALL of these device infrastructures, so you could backup to a disk and transfer the archive directly to a tape and BRU can restore the contents with no further work on your part. You can then readily copy the contents of an LTO-4 tape to an LTO-8 tape in 2020 and BRU would still be quite happy restoring the data from the new technology.
Finally. while you were at Discovery, did you ask specifically about LTFS failure rates? Not the Discovery specific format issues, but the failure rates of mounting volumes at all? I’m sure that they wouldn’t simply volunteer such data without specific probing.
I’m sorry to be so verbal about all of this, but so many vendors have jumped on the LTFS bandwagon without truly understanding what’s “behind the curtain” that I can’t just sit by and let them all poo-poo BRU with our success rate and 29 years of proven reliability when they really don’t have a true, device level understanding of how this all works. My team does know what’s in LTFS and have even made code-level patch submissions to the project.
Tim
—
Tim Jones
CTO – TOLIS Group, Inc.
https://www.productionbackup.com
BRU … because it’s the RESTORE that matters! -
Sam –
Sorry to have turned this into such a hornet’s nest :).
Tim
—
Tim Jones
CTO – TOLIS Group, Inc.
https://www.productionbackup.com
BRU … because it’s the RESTORE that matters! -
Right away, the OSXFuse bundled w/ the HP StoreOpen installer is a RED FLAG. I’m looking decades from now to be able to restore those LTO-6 media and I doubt OSXFuse will exist in 2030. Well, at least 5 years from now I’ll migrate to LTO-7 or 8. The LTO-6 drive and media will likely be sold to some one else. If LTO-8 can read LTO-6 media (which LTO format claims to have at least 2 generations of read compatibility), then I won’t have to migrate the massive amount of LTO-6 to LTO-8 as much. Ultimately, I’ll have no choice but to move to LTO-8. The massive 4K raw footage and countless other media sources are pushing the limits of LTO-6 now. It’s taking way too much physical tape archive storage space. This technical disclosure helps a lot to have a deeper understanding of LTFS & the LTO-6 format in general. The LTFS drivers compatibility headache is not something I want to experience w/ archive within years from now.
>>Next, you don’t MOUNT our tape drive as tapes shouldn’t be mounted – tape is not disk. The ATTO drivers are included in OS X and Windows out of the box. While it’s a good idea to update to their latest versions, it’s not required and the drivers from the HBA manufacturers don’t change the format that is written to the tape itself. And, the LTFS layer STILL depends on those same ATTO (or other) device drivers for proper operation. Unlike updating an HBA driver, changing an LTFS driver can mean the difference between a drive working or not working. It can also mean the difference between an old LTFS tape being readable under the new drivers or not.<<
-
Sorry – just checking in on this one recently.
The anecdotal information about StorageDNA pricing is incorrect.
Pricing of StorageDNA LTFS based solutions start under $8K and are turnkey, meaning they INCLUDE hardware, software, connectivity etc. and include many workflow features far and above just moving content from disk to tape.
Features like AAF/XML/EDL-based archive and restore jobs, conforming from LTO, camera master and NLE metadata search including Clip Names, metadata tagging, incremental archives and restores, lo-res proxy viewing, integration with CatDV, Avid Interplay Production and many others, ability to centrally manage more multiple libraries across an organization, Discovery Networks delivery formatting, import of any LTFS tape – to name just a few.
So, when you talk about pricing, please be sure to do a thorough comparison of what you are getting for $499.
Doug
Reply to this Discussion! Login or Sign Up