‘Offline’ tape backup, or ‘image’ and backup

[Jerzy Zbyslaw]

Its INCREDIBLY EASY if you use the ZFS file system that comes with either Solaris, one of the BSD’s like https://www.freebsd.org/ which are the rock solid ZFS systems, you can also install ZFS On Linux (ZOL) https://zfsonlinux.org/ which should be OK but it’s not regarded as production stable as the first two options.

The reason I say this is that you can create a ZFS file system on either entire disks, or slices (Solaris way of doing partitions) or just plain ORDINARY FILES, see here for an explanation using Solaris https://www.thegeekdiary.com/zfs-tutorials-creating-zfs-pools-and-file-systems/

You then mount the file and access it as you would do a regular disk, so in your case where you said you had a 5TB disk you could for example create say a 2.45 TB file and mount it, you then copy data to it until full and then you unmount it, you then copy the 2.45 TB file to a 2.50 TB LTO6 tape in your tape drive using regular unix TAR, DD, or CPIO commands and then post the LTO6 tape in the mail (it doesn’t only have to be an LTO tape as it could just as easily be an RDX cartridge)

The recipient at the other end just has to reverse the process by loading the tape into their drive and copying the monolithic file to their hard drive or raid array and simply mount it (obviously using ZFS as well, and preferably using the same version of ZFS) and then they can easily access the data.

Because you are creating ZFS using files and not disks or partitions and therefore you don’t have to worry about passing through disk accesses through to the VM itself (e.g. fiddling with VM and BIOS settings for LSI SAS cards and SAS drives) then the easiest way to do this using your existing computer would be to get a free hypervisor like https://www.virtualbox.org/ which runs on “Windows, Linux, Macintosh, and Solaris hosts” or you could use Hyper-V with Windows and I think the Mac has https://www.parallels.com/ available and simply install one of those three ZFS capable systems although to use Solaris professionally it would have to be licensed whereas the BSD’s and ZOL would be entirely free, you don’t have to have a 2.5TB VM as you can just set aside say 50GB for the VM’s operating system and just give that VM network access to let it create a 2.45TB file somewhere else on the computer, network or NAS, once your running the VM and in there you mount the file, then copy 2.45TB’s worth of data from somewhere else to that filesystem (the 2.45 TB file), unmount the file and shutdown and exit the VM.

So to answer your question “Is this feasible on any software?” yes it can and it can even be totally free but the only downside is that suddenly you have to learn one of either unix, Solaris, linux, or FreeBSD but this would be a very good investment for yourself now and in the future especially as data storage sizes are set to explode with 4K and 8K coming in, another benefit is that with ZFS each block in the filesystem if fully checksummed so if it gets sent to the destination and has any errors you’ll know about it if you do a Zpool Scrub command on the entire filesystem, more introductory information about ZFS is here https://pdfs.semanticscholar.org/presentation/bba6/a91ab9573e4284d041b0ae1c41f737d8cb96.pdf and it is far better than Windows NTFS, Linux’s Ext4 and the Mac’s APFS.

Lastly, a second answer to your question is if you don’t want to use ZFS you could just as easily create a 2.45 TB VM file using whatever hypervisor you want containing say an OS of your choosing and as an example together with say a free version of Blender and the Blender files you are working on and then you shut the VM down and then send that actual VM file (together with any relevant config files) and just send all that on the LTO tape, the recipient just has to import the VM into their hypervisor and fire it up. e.g. this article here https://www.techrepublic.com/article/how-to-move-virtualbox-vms-from-one-drive-to-another/ and although this is just as do-able you miss out on the integrity features that ZFS provides and so isn’t my preferred recommendation but this would suffice for say an interim period until you get up to speed with ZFS.

I think the ZFS file and VM image is the only real neat and tidy way to virtualize file systems as these can be any size whereas actual disk images are usually for set sizes (unless you just create a primary partition that doesn’t fill the entire disk) and also contain pesky information like MBR and GPT partition table information that you also have to deal with when transferring which makes actual disk images a bit messier to deal with.

Cheers

[Tim Jones]

Hi Neil,

The key to such a mechanism is in the format that the archive container is written. As long as the container format for the two types of containers is the same, it is quite “doable”. You just need a tool that can tape the file and write it as raw data to the tape so that the format and any metadata in the container is retained.

BRU Server already does this (D2D and D2D2T) and you can do it through a script with BRU PE, disk archiving and the tapewrite command line tool that is included. Drop our support team a note for more details.

Tim
—
Tim Jones
CTO – TOLIS Group, Inc.
https://www.tolisgroup.com
BRU … because it’s the RESTORE that matters!

[Neil Sadwelkar]

Thanks Tim.
I was wondering if I make a disk backup using Bru-PE, can I take the resulting file (its named .bru?) and then write that to tape somehow?

———————————–
Neil Sadwelkar
neilsadwelkar.blogspot.com
twitter: fcpguru
FCP Editor, Edit systems consultant
Mumbai India

[Tim Jones]

Hi Neil,

Yes – the BRU container format is the same on disk and tape, so you would use the “tapewrite” command that is part of BRU PE.

There are a lot of more advanced things that you can do, but the simplest mechanism is:
cat /PathTo/archive.bru | tapewrite -b 128k -f ntape0
If you shift to BRU Server, it’s an automatic process called “UpStage”. When you set the BRU Server job for Disk 2 Disk, you can then follow up at a later time with an UpStage that automatically transfers the disk file to tape with the options of deleting the original disk archive or retaining it for local operations while sending the resulting tape(s) offsite.

As always, contact the support team if you have any specific workflow questions.

Tim
—
Tim Jones
CTO – TOLIS Group, Inc.
https://www.tolisgroup.com
BRU … because it’s the RESTORE that matters!

[Jon Allen]

Hi Tim

When you say contact support…I tried to get them to explain the upstaging maybe 3 or 4 times and never got a satisfactory answer. Due to the delay in BRU PE for linux I have had to use Server. The support team is very slow to get back to me, and often when they do, they don’t read the full ticket and ask for information that has already been provided.

I’m happy to pay $279 USD/annual for support, if I get support. It takes alot of man power to make that happen and so maybe it is being pragmatic to consider putting together a knowledge base? There must be some sort of internal wiki that the tech support use to save time, why not make a version for the customers to help themselves? Charge $150/year for “basic” support with the current 48 hour ticket response time, plus a wiki, and then the $279 for actual one to one support with same day response.

Your thoughts?