Creative Communities of the World Forums

The peer to peer support community for media production professionals.

Activity Forums Storage & Archiving I scratch your back, you scratch mine

  • Neil Sadwelkar

    May 2, 2024 at 10:32 am

    Santanu,

    Ever growing archives of media, is an issue facing any content creator. 70TB is what a feature or series generates in about 40 days of shoot, so it’s not a huge amount in today’s standards.

    The most common method of archive is backup to LTO. I do a lot of this. Like over a petabyte in a year. Every year since the past 7+ years. Like almost any ‘on-prem’ archive, this one is not a ‘forever archive’. LTO tapes change generations every couple of years, and while backward compatibility is assured within about 1-2 generations, after about 10 years, one needs to ‘migrate’ entire LTO tape archives to the current generation. For example, I’m migrating large collections of LTO-5 and 6 tapes created in the 2012-15 period, to LTO-8 and LTO-9 tapes now. 10 years from now, they’ll need to migrate again.

    If archives are stored on hard drives, those too have a life, and even hard drive based archives need to be migrated to the current largest capacity drives. I’ve migrated many 2-4TB drives from 2009-2012 to 8-12TB drives around 2017-2020. Some of the original 2-4TB drives still work, some don’t. But the dupes made in 2017-2020 are still readable, but not for long. So, I’ll need to migrate them once again, next year.

    A convenient aspect of migrating is that hard drive prices halve every 2-4 years, and capacity also doubles in about that time. So also, LTO generations. Each LTR generation stores 1.5x-2x of the previous generation.

    Cloud archive has a monthly cost which varies depending on which platform one chooses. People who archived to the cloud in the 2018 or before, have now paid for about 5+ years. And they continue to pay huge sums of money to keep this data in the cloud. The amount of data they have in the cloud is so huge now that taking it back is impractical, due to the time, and egress costs. So, most likely, some will keep their local copy and delete the cloud backup eventually.

    I get many queries from people paying hundreds of thousands of $ per annum just to keep their data in the cloud. They ask, can we not own a setup and keep this ourselves, maybe in another premises in some other part of the city or some other city. This is the new ‘private cloud’, which I’m devising for some of my clients. With the advantage that if any data was needed, all one has to do is connect a drive and copy it. No egress time, no egress charges.

    Neil

  • Mark Grance

    May 8, 2024 at 3:00 pm

    I would comment that you started talking about your “content” and then you referred to it is “data”. There is a big difference.

    Does any of your content involve rights managed material: music, talent, imagery (both for hire and stock)?

    If so, do you have the rights to give that content to another company for their use?

    (Make no mistake they are using that content.)

    Are you able to prove you have the rights for each file in that 70TB?

We use anonymous cookies to give you the best experience we can.
Our Privacy policy | GDPR Policy