Neil Sadwelkar
4
Apple Final Cut Pro X Coins
2
Avid Coins
66
COW Coins
10
DaVinci Coins
61
Storage & Archiving Coins
Forum Replies Created
-
Your assets are on LTO-5 tapes and LaCie rugged drives. The LaCie drives will run about 80 MB/sec and the LTO tapes about 100-120 MB/sec for retrieval.
Today, bare SATA drives of 20 TB each can run at two times this speed even over USB3.0. Bare drives run about $ 16 per TB. Many drives in a RAID enclosure, run about $ 25 – $ 40 per TB. This is your cost per TB for 5 years of storage.
Your 200 LTO-5 tapes will become 16 LTO-9 Tapes if you migrate then now. So, restoring the Retrospect LTO-5 tapes to hard drives, and then writing them to LTFS LTO-9 tapes will extend their life.
After 5 years, this cost of hard drives will halve, and most likely, take up half the physical space as well. So, your 300 TB of data, which takes up 15 drives of 20 TB now, will fit in 6 drives of 50 TB each of which will most likely cost exactly what you paid for the 20 TB drives now. If you paid $ 5,000 for 15 drives of 20 TB today, 6 drives of 50 TB will cost about $ 2000, 5 years from now. And they will hold the same amount of data. (I’m basing this on the fact that in 2017-18, 8 TB drives cost nearly what 20 TB drives cost today)
As for your clients, your best bet is to catalog their data using something like DiskCatalogMaker and send them a mail, showing their data as a pdf (made from DiskCatalogMaker), and offering them a certain amount per TB for 5 years. I suspect this will be less than what they would have paid for a cloud service. You could also let them pay annually.
Even if you bill your client $ 10 per TB per year, some clients with about 10 TB, may even be willing to pay for 5 years up front. And even if you manage to convince 100 TB worth of clients, you’re looking at recovering part or all of your costs for this storage venture.
There is, of course, the cost of the labour of doing this retrieval and migration.
-
I re-read this whole thread from 2018. So much changed. Bru went away, but got revived as Argest. Even if one had Bru licences, one needed a MacOS 10.14 or earlier system to run it. Argest import of Bru catalogs off tape is a bit iffy, so, large quantities of Bru archives are best restored using an older MacOS 10.14 system.
On-prem is being looked at once again, because those that adopted cloud storage, realised that over a long period, the monthly ‘holding’ charges quickly add up. Then there are ‘egress’ fees to get back your own data.
LTO backup’s one big issue is availability of tape drives, and the software they were written in, after extended periods. Like, those who archived to LTO-1 to LTO-3 in the early 2000s, may or may not find the software and the system, to restore those tapes. Like NTBackup on Windows was quite popular (in my region) and I know people with stacks of those tapes. They have no easy way to read those tapes now, as the current Windows doesn’t support the software. Same with many other pre-LTFS softwares.
‘LTO migration’, and in general ‘data migration’ is a fact of life, and no on-prem storage is ‘forever’. I tell clients that any on-prem or off-site archive storage is good for about 5 years. After that it needs to be migrated to something else.
The next 5 years will see (hopefully not) a few cloud storage players go down, and with that, a scramble to restore the TBs of data that’s on their servers.
Neil
-
This is something that many media houses that have a large quantity of video and audio assets, will have.
I’m intrigued by your multi-tiered approach. The assets some here some there, some in multiple places, is typical, and it will all get sorted over time.
But one question I have, since you’re making projections for the future. How are you going to afford to pay for the ‘holding costs’ that most cloud services charge per month which over many years and many TBs quickly run into six figures (in $$). Plus with some providers, there are ‘egress fees’, so you pay to get your own data back.
The question actually is, are the assets worth these costs? Do they return these investments?
Maybe they are.
About projecting future data requirements, the ‘formula’ you have, seems fine. Although data estimation is tricky. One cannot accurately predict how much (more or less) data you’ll create, going forward. And, with new formats coming out all the time, one cannot predict how small (or large) future assets will be. Back in the day, archives were 10-bit dpx. then came ProRes and files shrunk, but more files were created. Now we have J2k, JPEG-XS, HEVC, so you may have to factor that in.
The other thing some people have begun to consider is ‘data audit’, ‘data pruning’, ‘data mining’. To make the data stored and archived, richer and more usable. But that’s a whole new story.
Neil
-
Trevor,
It’s likely that your previous LTO 5 model came with some software that wrote LTFS tapes using a free software that used to ship with HP and Tandberg drives. Does that same software not work with this new LTO-7 drive? I Googled it and found HP has the HPE StoreOpen software still available for download. That may work with your drive.
Two issues I found with LTO tapes written through free drag-and-drop software is that there seems to be no verification of what’s written. And, LTO tapes written like with these softwares, take a very long time to mount and extract data from. As a test you could try your old LTO-5 tapes written with your older software, and check that retrieval is easy and speedy.
Re: Yoyotta vs Canister, yes there is a price difference and Canister will probably cost more since it’s annual, but even Yoyotta could charge for future upgrades.
Whichever you decide, its money well spent as easy cataloging and retrieval of tapes is as important as backing up. Whichever you choose, I can also suggest a really good cataloguing software Diskcatalogmaker which lets you create browsable, searchable, offline catalogs of LTO tapes.
Neil
-
“Check that you have selected “tag frames with clip name” and “export frames with tag name” in the export settings. You can find these options in the “Output name” tab of the export settings.”
Where are these settings exactly?
-
What’s the make and model of your LTO drive? Some LTO drives come with all the required software.
But even if you don’t have it, downloading a 10 day demo of Yoyotta or Canister will help install all the necessary drivers. Canister automatically downloads all the necessary LTFS, ICS frameworks, and MacFuse drivers which will make your LTO drive recognised by the Mac and then you can decide on which one to buy.
Get Yoyotta if you need very deep organising of your LTO assets into clients, projects, and any other criteria. Like if you’re writing 50-100 LTOs a year.
Get Canister if you occasionally write LTOs for backups (about 10-30 a year), and don’t have more than few projects.
Both have their strengths and weaknesses. Both have excellent online support.
Canister costs less and is very easy to use. Yoyotta costs more, but has a large amount of controls.
I use both, for different reasons and use cases.
-
For a 4-part Netflix show that was being finished during the lockdown, I did the finishing for that on DaVinci Resolve. And I did most of it remotely, going into the facility about once a week to attach drives, check the aircon and other such.
Finishing consisted of doing motion graphics, supers, applying grades, making edit changes, subtitles, adding mixed audio and such tasks. The show is currently running on Netflix.
You can DM me if such a remote arrangement works for you. I use Jump Desktop and it works almost exactly like being there. All communications are through Zoom/Teams/Facetime.
-
To be able to use a networked drive as a media drive for editing/transcode/consolidate you need two things
Avid Media Composer Ultimate
Avid Nexis/Unity/ISIS or Nexis compatible shared storage
Alternatives to Avid Nexis are many. Simplest one is Mimiq which is $199 per license. With that installed, any shared storage can be used like a local media drive over SMB. Multiple Avids need Mimiq on each, and they can all can mount the same shared drive and use it for project and bin sharing.
Even if you have only one Avid connected to shared storage, you’ll still need Mimiq to make Avid treat it like a shared drive.
-
Thunderbolt 3 (40Gbps) and USB 3 (5 or 10 Gbps) have different top speeds, but the 2×3.5″ SATA hard drives that you use inside this LaCie dock, as RAID0 would give you a top speed of 500 MB/sec, or 4 Gbps. Either port can manage this speed.
But both types of ports are provided because, as Glenn correctly pointed out, some systems may not have a Thunderbolt 3 port, or have one free, so the USB port can be used.
The other reason is, this is a dock. It also has CFast and CFExpress card readers built in. These can read at 500 to 800 MB/sec. And there’s a hub USB 3.0 port where you could have another drive or an SSD connected. And there’s a display port where you could have a monitor hooked up.
Assuming you’re using one or both these card readers and the drives, and another downstream drive/SSD, then the combined bandwidth would exceed USB, so then a Thunderbolt 3 port would be faster.
-
That’s such a common mistake that on some LaCie Units, I have taped off the USB port so that someone doesn’t plug a Thunderbolt cable in it. LaCie hasn’t make it any easier by spacing the ports equally and having a small (non 40-plus eyes friendly) symbol below the port so its hidden once one connects a cable.
For this particular (2-bay RAID0) drive, its speed should be nearly the same over Thunderbolt 3 vs USB 3.1. Provided, one uses a Thunderbolt 3 cable with a Thunderbolt 3 port, and a USB-C cable with a USB-C port. The Mac’s Thunderbolt 3 ports double up as USB-C ports, but work as USB, and have optimal USB 3.1 speeds only with USB-C cables. Preferably the supplied USB cables.