LTO Archive vs Near line storage

[Bob Zelin]

an update on what topic. You are not clear.

The wonderful companies that make LTO software today are Imagine Products, YoYotta, and Hedge.
Thunderbolt LTO drives can be purchased from Magstor, and MLogic.

Today, LTO 8 is the “latest thing” – but LTO 8 cannot read an LTO 5 or LTO 6 tape. It will only read LTO 7 and LTO 8.

As for near line storage, QNAP and Synology are the most popular products.

If you have further questions – please ask SPECIFIC questions, and I will answer them.

Bob Zelin

Bob Zelin
Rescue 1, Inc.
bobzelin@icloud.com

[Eric Strand]

Thanks for your reply Bob; I meant on your exact initial post:

“I cannot help to acknowledge that as the price of inexpensive NAS products from Synology, QNAP, and Netgear
continue to drop in price, the reality that NEAR LINE STORAGE (secondary shared storage with cheaper products) is becoming a viable alternative to LTO tape backup.”

What is your and other execs thoughts two years later/the current state of near line as archive?

@ericstrand11

[Bob Zelin]

near line is great.
systems from QNAP and Synology are dirt cheap, and drives are super reliable today. When you create a RAID 6 configuration, this allows for 2 drives to fail. So your backup “nearline” solution is RAID 6, and your master system is RAID 6, so you can have FOUR drives fail, and you will still have your data intact. Of course, you should change your defective drive the second that it happens.

So will a RAID array like near line storage last for 30 years ? Of course not. But neither will LTO. I don’t care if it’s “theoretical” for 30 years. No one will write drivers for products like the Cache-A for example. So what do you do with all of your LTO 4 and LTO 5 (and now LTO 6) tapes. You create the LTO 6 archive. You keep it in a storage vault.
Your place burns down, and today you can only get LTO 8 drives, which will not read an LTO 4, or LTO 5 or LTO 6.
So someone will come back and say “well, that is ridiculous, you can still find LTO 5 and LTO 6 drives out there” –
well what about 10 years from now – or even 5 years from now. Can you find an LTO 4 except on eBay ?

Everything becomes obsolete. Nothing is going to last for 30 years. The appeal of cloud site storage is that “someone else does it for you”. So all the big cloud companies are continuously migrating their technology to new technology, as it approaches obsolescence.

This is why near line storage is appealing. You keep your near line system with it’s 5 year warrantee drives for 5 – 6 years, and in 5 – 6 years, you buy a NEW SYSTEM (this time with 20 TB drives – because 14 TB drives are out right now for $499 a piece, and in 5 years, I assure you 20 TB drives will be available) – and you migrate your data to the new larger Near line system.

If you say – “well, I don’t want to keep buying new stuff every 5 – 6 years – I want it to last for 30 years” – this is my advice to you. Open up a sandwich shop, near video production facilities. Because that meat slicer and refrigerator may in fact last for 30 years. But NOTHING that is “technology” is going to keep going for 30 years, or 20 years, and if by some miracle it lasts for 10 years today – well, it’s a Miracle.

Bob Zelin

Bob Zelin
Rescue 1, Inc.
bobzelin@icloud.com

[Eric Strand]

Thanks Bob, along the lines of what I thought you may say. Small production facilities, as always, are continually fighting the budget vs new technology battle.

Also interesting to note that although 20TB drives will be available in the future, the file sizes that cameras generate continues to grow. Although one could look at new Blackmagic Raw with it’s lower data rates and say that camera and codec manufacturers will also keep innovating and giving us better quality for similar or smaller file sizes.

Thanks again for your insights.

@ericstrand11

[Jerzy Zbyslaw]

[Eric Strand] “”I cannot help to acknowledge that as the price of inexpensive NAS products from Synology, QNAP, and Netgear continue to drop in price, the reality that NEAR LINE STORAGE (secondary shared storage with cheaper products) is becoming a viable alternative to LTO tape backup.””

Yes, for the amount of data they store they are perfectly viable, the answer may swing back towards LTO drives, auto loaders and tape libraries if suddenly you need to shoot in 8K and find you may need five to ten times as much storage as before. Realistically though, people will most likely have a mix of both in having a NAS for stuff they are working on and tape for stuff that’s current but inactive at the moment.

[Bob Zelin] “So what do you do with all of your LTO 4 and LTO 5 (and now LTO 6) tapes. You create the LTO 6 archive. You keep it in a storage vault. Your place burns down, and today you can only get LTO 8 drives, which will not read an LTO 4, or LTO 5 or LTO 6. So someone will come back and say “well, that is ridiculous, you can still find LTO 5 and LTO 6 drives out there” – well what about 10 years from now – or even 5 years from now. Can you find an LTO 4 except on eBay ?”

Dell still sell two different models of brand new LTO4 drives so that covers you down to LTO2 tapes https://pilot.search.dell.com/lto%20drive , all higher versions (LTO5+) are also available everywhere else so I don’t see this issue as being a problem at the moment although it may change in the future as you suggest, but I’m guessing that people will still be building old versions of LTO drives well into the future because if they are still going to be priced at four grand apiece the reply to any query is probably going to be “how many of them do you want to buy?”, I don’t know who would still be sitting on LTO1 tapes today as they are only 100GB so I presume a used LTO2 or LTO3 off Ebay would suffice to get the data off those onto something a bit more modern, LTO2 was introduced in 2003 so people have had plenty of time to migrate LTO1 data which was introduced in 2000.

[Bob Zelin] “This is why near line storage is appealing. You keep your near line system with it’s 5 year warrantee drives for 5 – 6 years, and in 5 – 6 years, you buy a NEW SYSTEM (this time with 20 TB drives – because 14 TB drives are out right now for $499 a piece, and in 5 years, I assure you 20 TB drives will be available) – and you migrate your data to the new larger Near line system.”

In theory yes, but the cost of the hard drives on a $ per TB basis since they consolidated to only two main manufacturers is not declining that rapidly, if say a 6TB drive cost $300 and they introduced 8TB and 10TB hard drives they would come in at $400 and $500 respectively, gone are the days when people went crazy buying 1 TB drives and I more wisely would buy two 750GB ones for the exact same price. Hard drive prices have to decline by 50% every 2.0-2.5 years to keep pace with LTO storage density roughly doubling each new generation for pretty much the same price. You can have an 8 bay NAS with 14TB drives or even 20TB drives you mentioned but then if say you need to back up this data a second NAS is going to be expensive whereas I’ve seen a bare LTO8 drive for sale at just under USD $3K and LTO7 type M8 tapes are just under AUD $100 (USD $72.50) and they store 9 TB uncompressed. And then as I mentioned previously if storage requirements blow out for 4K or 8K then this question becomes even more acute if you read the reduser forums at https://www.reduser.net/forum/forum.php on this topic or even on the Reddit Datahoarder forums https://www.reddit.com/r/DataHoarder/ where LTO technology is starting to be discussed more and more where people have a need to backup their stuff in the 50TB – 100TB+ range.

[Bob Zelin]

Jerezy writes –
In theory yes, but the cost of the hard drives on a $ per TB basis since they consolidated to only two main manufacturers is not declining that rapidly, if say a 6TB drive cost $300 and they introduced 8TB and 10TB hard drives they would come in at $400 and $500 respectively, gone are the days when people went crazy buying 1 TB drives and I more wisely would buy two 750GB ones for the exact same price. Hard drive prices have to decline by 50% every 2.0-2.5 years to keep pace with LTO storage density roughly doubling each new generation for pretty much the same price

REPLY – well, in November 2018, a Seagate Ironwolf 7200 RRPM 6TB drive costs $184. And to keep going with your model, a 10 TB costs $299, not $500. For $500 you get the new 14 TB drives, and those prices of course will come down within 6 months.

Nearline also offers the ability of INSTANT ACCESS. No one has to restore a nearline archive. And if you are not using the LTFS tape format, good luck trying to restore any LTO tape (like LTO1, LTO 2, LTO 3) that has been done in a proprietary format, whose manufacturer (like Cache-A) has gone out of business.

One of the downfalls of LTO, which has been greatly simplified by wonderful companies like Hedge, YoYotta and Imagine Products, is that the RESTORE process needs to be easy. The minute that any technology product becomes someone’s career (oh – we are experts in LTO archive) – then this technology is useless. In 2018, people want stuff to be easy, and not require an IT professional to assist them with it, or have an IT staff person just to restore an archive.

The observance of Zip drives AIT tape, DLT tape, DAT tape, and early LTO makes all of this very nerve racking.
And the recent release of LTO 8 tape does not make people feel more secure – only because it’s not very backward compatible. Will LTO 10 read an LTO 8 tape when it comes out.

But do drives fail – OF COURSE THEY DO. And I see on these very forums, some poor slob with his old Rorke Data array, or Apple Xserve RAID array, or Sonnet Port Multiplier array, or Medea RAID 3 array, saying “I can’t read my data anymore”
So drives are no miracle. But they certainly last 5 – 6 years, and if you simply are tired of making continuing investments into our business, then you should NOT be in this business anymore.

I look at guys that have spend a fortune in cameras, only to have new wonderful cameras from companies like Blackmagic come out, that run circles around their old expensive “old gear”. I often bring up the $80,000 Media Composers, and $100,000 AVID Symphonies. You know what you can do with these ? Put them in the trash bins.
And this applies to the original $400,000 Davinci Resolve systems, Ampex ADO’s, Chryon’s, GVG Kaleidoscopes, Quantel everything, and countless other deadly expensive products that have basically put countless people out of business because of their massive investments, and not getting a good ROI on them.

A current LTO 8 system from MLogic is about $6400 and that includes Hedge LTFS software. I love this product, and I love Hedge software. Will this give the purchaser of this product life beyond 2023 ? And the REAL question is –
will Thunderbolt 3 even exist in 2023 ? (the MLogic LTO 8 drive is Thunderbolt 3 interface only). For clarity – 2023 is 5 years from now – about the life of a nearline NAS.

Bob Zelin

Bob Zelin
Rescue 1, Inc.
bobzelin@icloud.com

[Eric Strand]

I love Hedge, but am slightly wary of their Canister LTFS product simply because they are a smaller company. Even though LTFS is open source, where will technology and companies be in 5 and then 10 years? What happens if a newer technology comes out meant to replace LTFS?

@ericstrand11

[Bob Zelin]

Eric writes –
I love Hedge, but am slightly wary of their Canister LTFS product simply because they are a smaller company. Even though LTFS is open source, where will technology and companies be in 5 and then 10 years? What happens if a newer technology comes out meant to replace LTFS?

reply –
really? You do know that YoYotta is a tiny company. So exactly who do you want to rely on?
Lets pick Tolis Group (who does not support the LTFS format) –
look at this recent post on Creative Cow –
https://forums.creativecow.net/readpost/330/2897

what do you think about that ?

How long was Cache-A in business ? ProMax bought them. Does ProMax support Cache-A products now ?

OH – I know – you can get Archiware P5
https://p5.archiware.com/products/p5-backup

it’s about $5000 for the software – and now you buy the LTO –

how about this –
https://www.gblabs.com/products/EasyLTO/

It’s about $10,000

how about this –
https://storagedna.com
I understand that their SUPPORT CONTRACT is about $4000 per year (perhaps I am wrong about this) –

so you tell me, Eric – what is the right decision ?

Bob Zelin

Bob Zelin
Rescue 1, Inc.
bobzelin@icloud.com

[Eric Strand]

I understand your point and fully agree with you; Cache-A is a great example. Additionally I learned about YoYotta from your articles.

There is no right answer. I thought your original post from 2016 was interesting and decided to start a conversation given the downward trends in pricing. We live in a time of amazing, affordable technology, as your articles post NAB often highlight.

@ericstrand11

[Tim Gerhard]

I’m the LTO guy so I’ll stick to the LTO talking point, but I don’t think it should be disk storage vs LTO, you should have both.

LTFS is just the way to go now when using tape. The old tarball way of making 1x file that is proprietary to a single version of software is a thing of the past like Casch-A.

I see Hedge has the canister app now, which I intend on running through the ringer when I find some time. Imagine Products and YoYotta have been in the LTFS game much longer than Hedge and have much more experience. When pondering over the single deck LTFS type support, these are all small companies, but you need to take their location / time zone into account. Imagine Products are in Indiana USA, YoYotta is in the UK, and Hedge is in Amsterdam or something? (Their website has no contact us or about us section, NOR support page).

StorageDNA is probably the largest company for LTFS single deck, with their myltoDNA product, which is built off of Imagine Products myLTO, but it has the “HyperTape” version. It isn’t $4000 for the support, that is for the library version I would guess.

I wouldn’t worry too much about Thunderbolt going out of style, and you can always just remove the drive from the enclosure later as they’re all SAS drives.

I sell Imagine Products and YoYotta with our MagStor line of LTO Thunderbolt & SAS solutions, and both have proven to provide top notch support.

I can’t speak for Hedge or Tolis group as I have no experience, but it seems like most of this forum is complaining about Tolis group most of the time, which could just be unwarranted because their ticket wasn’t responded to immediately.

Tim Gerhard
MagStor Inc.
614-505-6333
tgerhard@magstor.com