[Neil Sadwelkar]

Have mailed you my address. You can send it to me.

Unless its a lot of sequences and it takes me all day, I won’t charge you. But if you insist on paying, then I’ll let you know an amount multiples of which will get a certain number of medical workers in India with PPE suits. Fair?

Neil

[Neil Sadwelkar]

I’ve used Yoyotta to backup to a Tandberg half height SAS LTO drive. Done nearly a Petabyte so far. Done backups from 4-bay and 8-bay TB3 RAIDs, 2.5″ SSDs, NVMe SSDs, 2xSSD RAIDs and fo course just drives. So, devices with read speeds from 150 MB/s to 1,500 MB/s.

Backup speed doesn’t exceed 300 MB/s. Its faster with a small number of large files (Sony Venice) compared to a large number of small files (Arriraw).

I usually do Quick verify not full verify. If verify is a must, I call back the tapes a few months later when my machines are free, and do a verify run.

[Neil Sadwelkar]

As Bob said, quit waiting and move on.

Those who have a huge collection of Bru written LTO tapes, and working system with a Bru install, don’t worry, Bru itself will continue to work till the machines work. So your data is safe.

If your tapes are old, LTO4 or LTO5 then in any case you need to transition them to LTO7 or LTO8 because a working LTO drive that can read LTO4 or LTO5 tapes is not going to be around much longer.

12 LTO4 tapes can be restored to a 12TB hard drive and then to 1 LTO8 tape. You can then reuse the 12TB drive, or buy as many as you need. So if you have a 100 Bru written LTO4 tapes, you can fit them on just 8 LTO8 tapes, and have 8 hard drive backups for instant access too.

5 years from now, you can take those 8 12TB drives, and copy them to 2 50TB drives which will be available by then. And so on.

Similarly, 7 LTO5 tapes can fit on one 12TB hard drive and then be copied to one LTO8 tape.

To write LTO8 LTFS tapes, you can use a variety of LTFS software. A system that can run the last version of Bru and the latest version of any LTFS software is easily available now. But may not be in the future. As newer systems move to Big Sur, and Apple Silicon, a Bru compatible system would be hard to come by in about 3 years from now.

At least in this transition, you have a software (Bru PE) that’s free for restore, and still available. Think of those ho have hundreds of tapes written in Windows NTBackup or Retrospect, or some other proprietary LTO software which ran on a version of Windows that’s no longer available.

I’m in India and due to the Covid situation, I know of many LTO systems and people sitting idle. If anyone needs a large number of Bru tapes to be restored and transitioned to LTFS, you can contact me at neilsadwelkar at me dot com.

Even if you’re not confident of sending out the tapes, and if you don’t have the time to do this, as long as you have a working Bru system, and can arrange for the tapes to be changed, I could have someone do the transition remotely from here.

[Neil Sadwelkar]

You could get a JBOD, connect it to a Mac mini, or an older unused Mac or PC (depending on your drive’s format), connect that to your office network and access drives from home. After use, you can manually unmount the drives. In some JBOD enclosures drives spin down when unused.

I have a 4-bay Orico enclosure which spins down drives when I unmount the drives from my Mac. To access them again, I mount them again with disk utility.

Not sure if this improves the life of drives compared to leaving them spinning. But since your archive data is already on drives, migrating that to LTOs will have a substantial cost depending on how many drives you have. This could be a cost effective method.

For larger JBODs take a look at Datoptic. They make 16 bay JBODs. With Thunderbolt too.

[Neil Sadwelkar]

If you use Resolve to make proxies, and use the same proxies inside Resolve, you’ve found your off-line NLE. I’ve edited with DNxHD36 or ProResLT (converted from Red Raw by Resolve), inside Resolve and its been smooth. I haven’t observed this render-after-trim behaviour.

Are you perhaps working with camera raw files and Render cache in Resolve? That would explain the need to render after trim.

I second the suggestion to use XMLs instead of EDLs. They carry more information than EDLs. FCP X XMLs work very reliably in Resolve.

[Neil Sadwelkar]

That’s a great bit of advice Paul. And more convenient than LTOs.

What I use to track my ‘archive’ drives is Diskcatalogmaker. It saves’catalogs’ of your drives down to the last file, and you can ‘browse’ the drive without it being connected.

For an entire feature film original camera files, we often have them backed up to, say, 8 drives of 12 TB. I ‘scan’ all 8 into one Diskcatalogmaker catalog, so I can search for a particular file across 8 drives and 96 TB without connecting a single drive.

Diskcatalogmaker also exports all or selected files/folders as csv so you can make a Excel database of file names, sizes and paths and use an AppleScript or Excel formulae/macros to parse EDLs/XMLs and create a ‘copy script’ to extract selected files.

Neil

[Neil Sadwelkar]

Thanks Bob.

Santanu, I’m in India too. And I too had to move my setup home in March as well. And am still operating from home. From your profile and web site it looks like you’re in Mumbai (Thane). I’m in Mumbai too (Andheri W), so we have similar climates. I’ve had similar challenges in temp and dust.

About your server, you haven’t mentioned what kind of a server and how you connect to it. Or how many users currently connect to your present server.

For 40TB of data, I don’t think Backblaze is a viable option. or practically any other cloud service. In fact, for these very use cases, in the last NAB I went to (2019, which was the last NAB anyone went to) I had meetings with Backblaze as well as Wasabi to figure out if we could work out a cloud backup structure based in India for Indian users. With the backup ‘seeded’ with shuttle drives, and then only updated online. That didn’t work out as neither was interested.

Anyway, for 40TB of data which is shared between more than 3 users, who have to access the same media at the same time, a NAS is needed. If they only need to be able to occasionally share files, and never work on the same files, then its more cost effective to provide each user with sufficient local storage, and have them all connected over a fast network (not WiFi).

Another interesting new cloud shared storage service is Postlab. For under Rs 10,000 per month, 3 users can share proxies up to 3 TB, and keep their FCP X libraries in sync over the cloud. Basically edit from anywhere with up to 28-50 hours of proxy footage in the cloud.

That being said, the case for LTO drives is that they extend the capacity of your local storage – direct storage or NAS. Projects that have been delivered and are not likely to return immediately, but might one day, can be backed up to LTO tapes and erased off your primary storage. To be restored when the project returns for versions/changes.

Cost-wise, LTO tape costs Re. 1 per GB now. Of course, you need the drive and software which will run about Rs 3.5 lakhs. Assuming a life of 5 years for the LTO drive (with moderate use of about 20-25 tapes per annum) and assuming you ‘sweep out’ data at the rate of about 10 TB per month, your effective tape ownership/backup cost goes to Rs 1.50 per GB. With heavy use, it will be about Rs 2.5 per GB. This is way cheaper than any kind of drive storage. Which is about Rs 2 per GB for the cheapest consumer drives up to about Rs 7-10 per GB for good RAID drives or NAS.

There are also LTO service houses who do LTO Backups only. That might be more cost effective if your backup requirements are under 5 TB/month.

So, drives+LTO effectively is sort of like unlimited storage.

Hope this helps. Contact me offline and we can talk.

My email is my name surname (one word all small) at me dot com

Neil

[Neil Sadwelkar]

Hari,

Unless its a typo, I noticed that you’ve written you are using an Atto R680 card and not an Atto H680. The R680 is a RAID card as far as I remember, and (hope Martin doesn’t mind) I’m reproducing this from Yoyotta’s support page…

“ATTO ThunderLink SH1068, SH2068 TB-SAS, ThunderStream SC4808 adaptors, R680 or R644 PCIe RAID SAS cards do not support TLR which is a required protocol. So they are not compatible with LTFS running on any operating system.”

So, maybe that’s the problem in your installation. The SAS card you’re using R680 doesn’t support TLR, hence LTFS.

Have you tried, or can you try with this drive and connecting via a H680 or H1280, and see if that makes a difference.

Neil Sadwelkar

[Neil Sadwelkar]

Hari,

Unless its a typo, I noticed that you’ve written you are using an Atto R680 card and not an Atto H680. The R680 is a RAID card as far as I remember, and (hope Martin doesn’t mind) I’m reproducing this from Yoyotta’s support page…

“ATTO ThunderLink SH1068, SH2068 TB-SAS, ThunderStream SC4808 adaptors, R680 or R644 PCIe RAID SAS cards do not support TLR which is a required protocol. So they are not compatible with LTFS running on any operating system.”

So, maybe that’s the problem in your installation. The SAS card you’re using R680 doesn’t support TLR, hence LTFS.

Have you tried, or can you try with this drive and connecting via a H680 or H1280, and see if that makes a difference.

[Neil Sadwelkar]

I don’t know if myLTO is the same as PreRoll Post, also from Imagine. I had a license of that, and had the same issue. It was dog slow backing up large file sequences like what one has from an Arri Alexa camera, or DNG from a drone. It used to be something to do with their database if I remember right.

Bru-PE, Yoyotta, or Canister do a better job. I’ve used all three for file sequences. I’ve also used them with a QNAP connected over Thunderbolt3 or over 10GigE and they do not have this issue with folders with a large number of files.

You can get a trial license of either, and try for yourself, with the exact same media.