Purchase decisions between BRU Producer Edition or Image Products PreRoll Post

[Tim Jones]

Hi Doug,

The price I mentioned was based on the number that one of our customers was quoted. It included a 2U Library and LTO-6. But as you mention, what your products provide is far and beyond the free LTFS that is open source. This is the point that I’m trying to get across in this discussion. LTFS does NOT bring all of that needed functionality to the game, the companies that provide the wrappers around LTFS such as both StorageDNA and Image Products do.

And, to the best of my knowledge, neither StorageDNA, Crossroads, Cache-A nor PreRoll Post products are open source. Thus my point that the discussion of “proprietary” totally depends on which part of the solution that you are looking at.

Tim
—
Tim Jones
CTO – TOLIS Group, Inc.
https://www.productionbackup.com
BRU … because it’s the RESTORE that matters!

[Doug Hynes]

Tim –

When it comes down to it, many customers are actually looking for a more functional, open and workflow based solution – one that fits into their file-based pipelines and actually helps solve more of their challenges – not simply moving content from one place to another. If these features are not important to a customer, we (and they) walk away – as they are not our target audience.

As far as LTFS is concerned, your portrayal of it as an unreliable and unusable technology is just not true, as evidenced by the hundreds of customers we have using it every day in very critical environments.

One should also realize that Discovery’s inability to read LTFS tapes submitted to them has more to do with their very specific naming, placement of files (data and metadata) requirements, rather than the LTFS format itself. We’ve had to make several tweeks to our Discovery features based on a customers’ rejection from Discovery and not once has it had to do with the actual formatting of the tape. Once again, for this very specialized use case – many users are finding that it is worth paying for a good piece of software to be able to match those requirements – which DNA Evolution (and others) can do.

Even you have to admit – there must have been a reason why one of your own customers went as far as receiving a quote for a DNA Evolution system

– just sayin’

Regards,

Doug

[Tim Jones]

Doug,

You’re correct, they got an SDNA quote (and a Cache-A quote, and a Quantum quote, and a SpectraLogic quote) and then bought our solution.

You (and others) have chosen to take the LTFS path as your primary tape format. This is a good thing because you have all brought many of the features that LTFS should have had built in to your users. We have chosen to support LTFS in a secondary roll with an understanding of its weaknesses and strengths. My point is that TANSTAAFL – LTFS is what it is, but you pay extra – very much extra in some cases – to make LTFS truly useable.

We sell a lot of BRU solutions to users that started with an LTFS option and became frustrated with the issues. The purpose of a backup / archival solution is to make sure that you can get the data back later. This is our focus – the reliability of the data written to the archival device regardless of it’s type / technology. This is also why we don’t write a MAM/DAM and instead partner with others to provide that part of the solution.

Tim
—
Tim Jones
CTO – TOLIS Group, Inc.
https://www.productionbackup.com
BRU … because it’s the RESTORE that matters!

[Doug Hynes]

Look, I think we all get “your point” and like Dan, this will be my last comment on this topic, as this horse was already dead before I felt compelled to correct the inaccurate StorageDNA pricing statement.

[Tim Jones]

To summarize for everyone else being entertained by this thread – my only points are:

• LTFS is a simple format, like HFS+ or NTFS for tape
• LTFS doesn’t offer features that should be included at the low level such as verification, spanning, or robust protection from I/O failures.
• There are documented failures of LTFS mount compatibility issues
• You can add forms of these missing features and protections by purchasing other, wrapper solutions – which are as proprietary as any other tape solution.
• The cost for these wrappers is not included in the LTFS layer and can cost many thousands of dollars (with PRP being the notable exception at $499).

The other vendors have felt it necessary to argue their point that LTFS is great but have provided nothing to refute any of these points.

Doug, my SDNA pricing statement wasn’t inaccurate, so don’t try to paint me in a bad light here. It just wasn’t your cheapest solution. You then went on to try and belittle the $499 that we and Image Products charge for our software-only options by implying that you get less and that they are bad options.

Plus, if one person reads through this thread and questions the marketing claims that are being made – even if they decide to go with an LTFS-based solution – this discussion is not a dead horse.

Tim
—
Tim Jones
CTO – TOLIS Group, Inc.
https://www.productionbackup.com
BRU … because it’s the RESTORE that matters!

[Raul Zeummes]

I’m writing for anyone who is going through this decision process now or in the future, because I haven’t found as much information as I would have liked about it.

My situation is as follows : I’m looking to use an hp ultrium 6250 drive to make quality backups of video footage during filming, and also to make long-term backups of general company files in a manner that is not particularly time-sensitive.

My preference was for a pc-based workflow, and a very affordable solution. (this eliminates all the $6000+ solutions)
I first tried the xendata ltfs software, and the people over there were extremely nice, available, and helpful.
A first issue where the software would just crash explorer without giving an error was tracked down to the length of filenames. Once I fixed all the file names, I still had random crashes of explorer (ie would crash on a given data set the first time, but not on the same data set on a second try)
In any case, there is no possibility of tape spanning, so you need to manually select combinations of files and folders that are 2.07 TB, which is not fun.

I started looking at mac based solutions, and quickly narrowed things down to PreRoll Post (prp) and Bru PE.

PRP post is LTFS based, whereas Bru has their own proprietary system.

Above we have a conversation between Dan Montgomery, president of Imagine products, the makers of PRP, and Tim Jones, the president of the Tolis group, makers of Bru.

In my reading of it, Tim seems to have a much deeper mastery of all the detailed technical aspects, and you can tell he has learned a lot from being in this field for so long. He has answers for all of Dan’s points. Some of Dan’s statements seem to have been intended to gloss over or mislead on certain points.

Here are some advantages I see to using BRU :

It seems to have the highest reliability. Lots of people who need high reliability use it.
It’s been around for quite a while.
Their president really knows his stuff.
A windows version is announced
Does tape spanning (so does PRP)

Disadvantages to using BRU :

I’d be keeping my files in a proprietary format. There’s only this one company that’s responsible for it, and who knows where they’ll be in 30 years.
You can’t backup to a tape and a disk in one operation. (which you can in PRP) If they did add this functionality I would assume the files would be in bru format on the disk, which seems undesirable.
The interface is much less straightforward, even confusing at times. Why does it say LTO5 everywhere when I’m using LTO6 ?
Many comments on the web about BRU being frustrating to use.

Here are some advantages I see to using PRP :

The interface is really excellent
You can very easily make a backup to a disk and to a tape in one operation as your media cards are coming out of the camera.
Does tape spanning (so does BRU)

Disadvantages to using PRP :
The company making it is younger
The ltfs format seems to be inherently less safe than the bru format, which is an essential point for backups.
Their president’s points online seem to lack clarity, and possibly forthrightness

The question for me comes down to how unreliable is ltfs really going to turn out to be ?
The bru folks have a lot to say about that, but not all of it is pertinent to PRP.

– “open source means no support” : PRP is supported by imagine
– “corrupts the backup if the system hibernates” : I’ll turn off hibernation
– “do not use ltfs as a live filesystem” : I’m not intending to, that would be silly
– “use finder/explorer replacements sparingly” : don’t need to, I’ll be going through the backup database
– “multiple simultaneous access not recommended” : don’t need it
– “do not access the tape between writes if writing multiple segments” : don’t intend to
– “additional write access will run more slowly than first write” : this is no problem for big data sets, but seems like it would be a drag during a shoot. I wonder how much more slowly, considering I’m not intending to unmount / remount.
– “updating files does not release space” : I won’t be updating
– “do not use to playback music or video” : we get it, it’s not meant to be used as a live filesystem
– “ltfs doesn’t offer a verification mechanism” : but as far as I can tell, PRP has saved an MD5 of every single file. it compared the original hashes to what was written after the backup.
– “ltfs tapes can only be used with ltfs” : I’m going to be going all bru or all ltfs anyway

Outstanding questions in my mind :
– why is bru adding ltfs functionality if the format itself is so bad ? the fact that they are doing that gives me the impression that ltfs is here to stay, and that it can’t be all that insecure if properly implemented.
– If the power cuts out during a backup (which I’m going to be doing all I can to avoid), I’m going to assume I’ve lost the data on the tape, and start the backup from scratch anyway.
– is there a way to lose data during a read-only operation on ltfs due to a power loss ? (this would be much more worrysome)
– when I came back to PRP after an overnight backup test (small, under 300gb), I had a mac system message that the software had crashed. But I also still had a window showing the details of the backup having been completed, and all the md5s having been completely verified. how often is it going to be crashing ?
– from what I can tell so far, in PRP you can search your backup database for a file, and get the result, but if the tape containing that file is offline, you can’t get the file’s path. (using the print function) because you can’t put an offline file into the retrieval section
– how will PRP do on a backup test spanning multiple tapes ?
– how will BRU do on the backup tests which I’ve started today ?
– I still have to look into PRP to see if there’s a way to compare a live file to a backup reference file in order to check if the live file has been corrupted for example
– apart from the power being cut during a backup, what are the other risks of using the PRP implementation of ltfs ?

If anybody has anything to add to the comparison process I’ve outlined above, I would be most grateful.
If not, I hope it can help somebody going through this process at some point

[Tim Jones]

[Raul Zeummes] “- why is bru adding ltfs functionality if the format itself is so bad ? the fact that they are doing that gives me the impression that ltfs is here to stay, and that it can’t be all that insecure if properly implemented.

We added support for LTFS for 3 reasons –
1 – When we added support to BRU PE for LTFS, we were simplifying the arcane command line mechanisms required to format, mount, check, and rollback LTFS tapes on OS X. Remember, when we launched less than 3 months after LTFS was announced, command line operations were the only option on OS X.
2 – We have customers receiving tapes in this format and wanted to allow them to use the same search mechanism for their LTFS tapes as for their BRU tapes.
3 – We wanted to provide a mechanism that would create properly formatted tapes every time in the instance that one of our customers had to use LTFS.

– If the power cuts out during a backup (which I’m going to be doing all I can to avoid), I’m going to assume I’ve lost the data on the tape, and start the backup from scratch anyway.

Not required with BRU archives. Even if the power fails, the data that BRU has written to the tape is completely recoverable up to the point of failure. What if the failure also took out your disks? In that case, you couldn’t re-run the backup.

– is there a way to lose data during a read-only operation on ltfs due to a power loss ? (this would be much more worrisome)

Yes – because the filesystem is mounted, it can become corrupted just like a disk even if you’re only reading. There really isn’t a way to mount an LTFS volume as read only.

– when I came back to PRP after an overnight backup test (small, under 300gb), I had a mac system message that the software had crashed. But I also still had a window showing the details of the backup having been completed, and all the md5s having been completely verified. how often is it going to be crashing ?

I won’t be touching that one except to say that 300GB takes around 30minutes with BRU…

– from what I can tell so far, in PRP you can search your backup database for a file, and get the result, but if the tape containing that file is offline, you can’t get the file’s path. (using the print function) because you can’t put an offline file into the retrieval section

BRU can locate files on your media (BRU and/or LTFS) and provide the full path information of the selected files. If you then select those files for restore, you wil be told which tapes are required for the restore process. Also, the tapes don’t need to be online for a full search.

– how will PRP do on a backup test spanning multiple tapes ?

Doug? BRU will properly span tapes for a single backup or archival operation into the 16 Exabyte range.

– how will BRU do on the backup tests which I’ve started today ?

Hopefully I’ve answered that all above.

– I still have to look into PRP to see if there’s a way to compare a live file to a backup reference file in order to check if the live file has been corrupted for example

Because BRU’s proprietary format applies the 32bit CRC to the file data as it;s being read from the filesystem in 2K chunks, you will always be able to verify the tape’s contents WRT the file as it was read from the filesystem. If you wish, BRU will also allow you to performa a file by file comparison of the files in the archive against the original files on your system.

– apart from the power being cut during a backup, what are the other risks of using the PRP implementation of ltfs ?”

I don’t know that this is LTFS endemic or something the PRP has worked out, but you read the note about bad tapes when filling LTO-5 tapes in an LTO-6 drive (LINK), right? This is being fixed in the drive firmware for HP (done), Quantum (??), and Tandberg/Overland (??). I don’t know about IBM’s status. BRU does not suffer from this issue on any vendor’s tape drive.

What happens when you move to the next version of OS X or Windows and the FuSE layer hasn’t caught up? Since LTFS’s connection with your desktop and those apps that don’t natively write to tape is completely dependent on FuSE, and FuSE is a separate OSS project with no direct responsibility to the LTFS teams … I’m not trying to simply scare you here, we saw this with both Windows 7 SP 2, and OS X Mavericks. The FuSE layer was late arriving and if you made the OS switch early, you had no LTFS access.

Tim
—
Tim Jones
CTO – TOLIS Group, Inc.
https://www.productionbackup.com
BRU … because it’s the RESTORE that matters!

[Raul Zeummes]

Hi Tim,

Thanks for taking the time to address my points in detail.
My first BRU backup test ran overnight with no errors.

My main question at this point is as follows :

Do you recommend using BRU during a shoot as part of the offload workflow ? If so, how ?

I’m looking for a simple, GUI, time-effective way to copy footage off the cards onto a hard drive and a tape, and have both copies verified, before we format the card and give it back to the camera operator for re-use.

Thanks

[Tim Jones]

As always – my pleasure.

Here’s what a couple of road warriors are doing in your situation:

Mount the camera cards on the Mac (no need to copy them first)
Back the card contents up to LTO tape with BRU PE
Restore the backed up data from the LTO tape to the local disk using BRU PE’s “Restore to Alternate Location” option
Lather, rinse, repeat…

Because BRU is checksumming the data from the cards onto the LTO tape, you are sure that you have a correct original copy. Because BRU then also verifies the checksums on the restore pass, you are sure that you have a correct copy written to the local disk.

After a few “reassurance runs”, most users simply returned the cards to use as soon as the BRU backup to tape is completed since they know that the copy is 100%.

HTH,

Tim
—
Tim Jones
CTO – TOLIS Group, Inc.
https://www.productionbackup.com
BRU … because it’s the RESTORE that matters!

[Tim Jones]

I did have some additional feedback about the points that you posted about disadvantages of using BRU:

[Raul Zeummes] “I’d be keeping my files in a proprietary format. There’s only this one company that’s responsible for it, and who knows where they’ll be in 30 years.

Two things are in place to sort this as I’ve described in this blog post: Dealing with BRU’s Proprietary Nature

You can’t backup to a tape and a disk in one operation. (which you can in PRP) If they did add this functionality I would assume the files would be in bru format on the disk, which seems undesirable.

We did this because we don’t see a win in duplicating the functionality that your operating system already provides. You want to simply copy files from your media to your disk? Drag and Drop. Works in Finder, Explorer, and the various media managers on Linux.

As mentioned in my answer to your scenario earlier, if you’re more interested tin validating the copies, then BRU PE provides a proven mechanism for insuring that both the copy of the files written to your tape and the copy then written to your disk. In each operation, the files are checksummed via an inline process based on a 2K level of granularity. This is more consuming that a simple copy, but it does provide inline checksumming that requires no additional intervention on the users’ part.

On the other hand, manually copying is only as good as the underlying copy tool and I’d rather trust the OS’ tool than a 3rd party tool other wise.

The interface is much less straightforward, even confusing at times. Why does it say LTO5 everywhere when I’m using LTO6 ?

I’m curious about this statement. The only way that this would be happening is if you were running the 2.3 version of BRU PE that was released before LTO-6 was available. In the 3.x builds, we refer to the drive by it’s reported type regardless of whether it’s LTO-1, 2, 3, 4, 5 or 6.

Many comments on the web about BRU being frustrating to use.

My suspicion is that if you read those comments to the end of the threads, you’ll see that in almost each instance the issue was a problem with the drive or a misunderstanding of how tape access works.

I would be curious about your feelings on these notes after you’ve had a chance to read our “what if…” stance and you’ve had a chance to work with the latest version of the app.

Tim
—
Tim Jones
CTO – TOLIS Group, Inc.
https://www.productionbackup.com
BRU … because it’s the RESTORE that matters!