[Neil Sadwelkar]

The performance of external USB drives on M1/M2 Macs has been posted about before.

There is a reduction in R/W speeds for USB drives on M1/M2 Macs compared to Intel Macs. It’s to do with USB 3.1 Gen 2 compatibility in the M1/M2 Macs. This issue was more pronounced in older M1 based MacBooks but has improved somewhat with newer M2 Max and M2 Ultra Macs. But still not as fast as the same drive on an Intel Mac.

On solution that some propose is to use a Thunderbolt hub between the Mac and the USB drives. There are some YouTube videos demonstrating this. But the improvement is marginal to substantial depending on the drive and cable. Its too unpredictable.

I now use only Thunderbolt external drives with my M1/M2 MacBook Pros and Mac Studios. They cost a bit more but they work well.

Neil

[Neil Sadwelkar]

Are you using DaVinci Resolve free or DaVinci Resolve Studio.

Resolve Free cannot import certain types of MP4 files. Specifically 10-bit MP4 files from a Sony A7s3.

These 10-bit MP4 files import in Resolve Studio.

Neil

[Neil Sadwelkar]

Davinci Resolve Free has some limitations about the kind of clips it can import. For example, some 10-bit file types, MP4 or MXF do not open in Resolve Free. You can import them but they show the ‘Media Offline’ screen when played.

In case you’re using Resolve Studio for the export and the Resolve Free for the import, then this what you describe, is likely. If you’re using Resolve Studio for both, then this is very unusual.

[Neil Sadwelkar]

Mark,

Like I wrote, if the media files drive and the clone of that drive are both attached to your editing system, then Chronosync can keep them in sync. And make an archive of what’s different so in case you accidentally delete a file on the media drive, Chronosync will also delete it from the cone, but keep a backup in an archive folder.

Carbon Copy Cloner can also do exactly that. As will SuperDuper

If the media drive and its clone are not attached to the same computer, but are on the same premises but on a network, then too Chronosync or Carbon Cloner will do the job. They will mount it as required.

If the media drive and its clone are in different premises and connected via Internet, then the free SyncThing, or Resilio sync can clone them over the Internet. Of course, copying TBs of data over the Internet would take time.

[Neil Sadwelkar]

You’ve not mentioned what editing system you use (Pc or Mac, Avid or Prem Pro or some other) and whether you have projects and media on the same drive, or projects on internal drive and media on external drive.

Anyway, for an Avid MC system (would apply to Prem Pro too), system I have Carbon Copy Cloner clone my internal drive to an external drive once a day. Back when external drives could be used to boot a Mac from (pre M1/M2 Mac), an external SSD clone of the internal drive would enable one to boot from one external drive and have all of one’s work intact. If once a day was insufficient, then that external SSD could be left connected and cloning could be set to happen once every 2-3 hours.

As for the external drive with the media files, you can keep a spare drive connected, and have Chronosync sync the two, daily or once every few hours. So that as you add media to the external drive, the Chronosync clone would automatically have that media updated with folder structure intact.

If your external media drive failed, you could rename the clone drive with the same name as the original failed drive, and once you open your project all media will automatically relink because the system would ‘see’ that clone as the original.

For offsite backup of your project files, you could use Google Drive, or Dropbox to keep a continuous cloud backup.

If you’re not comfortable saving these on the cloud, then you can keep a clone drive at another location (office/home) and have a utility like Resilio Sync or SyncThing, to sync a folder/s on your work system in sync with a folder/s on the remote backup.

[Neil Sadwelkar]

Rodion,

As Bob described, it’s possible to use U.3 SSDs for editing, in a RAID form, inside a NAS.

For your use case, using one 15TB $2000 U.3 SSD may be overkill.

A U.2 or U.3 SSD will deliver that speed only if the interface supports it. Like if it’s installed on a PCIe card, or directly in a U.2/U.3 compatible slot internal to your PC.

If you’re on a Mac, you’ll need to be using a Thunderbolt to U.2/U.3 adapter of some kind (OWC makes enclosures which are U.2 SSD compatible). Used in this manner, a U.2/U.3 SSD will work at about 2,500-3,000 MB/sec which is fast enough for 8k video file processing. But then these speeds are also achievable with M.2 NVMe SSDs.

You could also take a look at Iodyne which makes SSD based storage which can be connected to multiple Macs simultaneously.

Also, bear in mind that U.2 and U.3 in some situations/enclosures/adapters are not interchangeable.

[Neil Sadwelkar]

Jon,

All the LTO backup I do is only on Mac systems. For the past few years, it been an Intel i7 Mac Mini.

I use Yoyotta mostly, as it offers built-in cataloguing and project management. I also have Canister which I use for quick backups. And restore from tapes that were not made by me.

I have a different older MacBook Pro with macOS Sierra still on it, and a working Bru install. That I use when I need to retrieve an old Bru tape. Many of my clients have had their backups done by me which are on Bru, so I need to keep this system operational just for Bru.

Neil

[Neil Sadwelkar]

I have both of them. I use Diskcatalogmaker for its ability to make self-contained catalogs, to have multiple drives in one catalog, to copy drive catalogs from one catalog to another, and the ability to export a catalog as a .csv so I can do stuff with that data in Numbers/Excel.

I use NeoFinder to be able to import old Bru catalogs and create a browsable, searchable catalog of Bru tapes.

For the past couple of years, for some projects, I’ve also maintained a database of assets by importing raw camera files into a Resolve project. Compared to DCM or NF, Resolve stores more detailed metadata like timecode, resolution, codec camera details. And a Resolve media pool can be exported as .csv and imported into Numbers/Excel for further analysis.

Neil

[Neil Sadwelkar]

I wasn’t aware of RapidCopy. I use Hedge for all my copying tasks. Costs more than RapidCopy but lets you make two or more copies at the same speed as a single copy.

I like the idea of backup to spinning disks. I use enterprise drives in USB 3 docks, so the drives become almost like ‘cassettes’. For spinning drives Mac HFS extended (GUID) is a preferred format. Spinning disks are more easy to access at a later date, than LTO tapes.

For some tasks I have also used LTO plus drive and stored the contents of the LTO and the drive in a DiskCatalogMaker catalog. I prefer making plain catalogs (.dcmf) rather than the default catalog with thumbnails (.dcmd) because dcmf is a single file while dcmd is a package which can get corrupt across file systems.

One issue you could come across as you add backups is, while DiskCatalogMaker lets you ‘browse’ backups without mounting the drive, you don’t get any idea of what the asset looks like. I’m now considering creating 1/100th size H.264 or H.265 proxies of everything I backup so that this can be used to do quick ‘browse and select’ shortlisting without touching the backup.

Resolve lets you create proxies very fast while preserving the original folder structure. So, you could have 20 TB of video files in a complex folders structure, compressed to 200 GB with the exact same folder structure saved away in a local drive. If you have 50 backup drives of 20TB each, (1 Petabyte total) then proxies will take up just 10 TB.

Neil

[Neil Sadwelkar]

Your assets are on LTO-5 tapes and LaCie rugged drives. The LaCie drives will run about 80 MB/sec and the LTO tapes about 100-120 MB/sec for retrieval.

Today, bare SATA drives of 20 TB each can run at two times this speed even over USB3.0. Bare drives run about $ 16 per TB. Many drives in a RAID enclosure, run about $ 25 – $ 40 per TB. This is your cost per TB for 5 years of storage.

Your 200 LTO-5 tapes will become 16 LTO-9 Tapes if you migrate then now. So, restoring the Retrospect LTO-5 tapes to hard drives, and then writing them to LTFS LTO-9 tapes will extend their life.

After 5 years, this cost of hard drives will halve, and most likely, take up half the physical space as well. So, your 300 TB of data, which takes up 15 drives of 20 TB now, will fit in 6 drives of 50 TB each of which will most likely cost exactly what you paid for the 20 TB drives now. If you paid $ 5,000 for 15 drives of 20 TB today, 6 drives of 50 TB will cost about $ 2000, 5 years from now. And they will hold the same amount of data. (I’m basing this on the fact that in 2017-18, 8 TB drives cost nearly what 20 TB drives cost today)

As for your clients, your best bet is to catalog their data using something like DiskCatalogMaker and send them a mail, showing their data as a pdf (made from DiskCatalogMaker), and offering them a certain amount per TB for 5 years. I suspect this will be less than what they would have paid for a cloud service. You could also let them pay annually.

Even if you bill your client $ 10 per TB per year, some clients with about 10 TB, may even be willing to pay for 5 years up front. And even if you manage to convince 100 TB worth of clients, you’re looking at recovering part or all of your costs for this storage venture.

There is, of course, the cost of the labour of doing this retrieval and migration.