-
newbie to LTO6/need help choosing a drive (for slower <160MB writing)
Hello
I need help understanding the world of LTO Ultrium tape backup. I’ve read things which are online but I cannot find answers to any of my questions because I don’t know where such conversations normally turn up. I will have a series of questions I may have to post separate threads on though the most important one always comes down to “which drive” and the most critical feature needed is the ability to write to the tape MORE SLOWLY than the full rate without what apparently is called “shoeshining” (simply because I am told this is “Bad”) because I do not expect to have hard drives fast enough to keep up, at least not across the entire drive surface and am not looking to use RAID0/5 either. I expect to have a NAS of multiple 3tb drives and want to write directly from each drive for convenience.
I am not concerned about writing rate much at all, if it takes 8 hours or 24 hours, I do not have a volume of backup so critical that this matters. At the same time the difference between writing at say 55MB/sec and 65MB/sec is not much, as long as I don’t have to maintain 160MB/sec like the standard seems to max out at maintaining the absolute minimum rate is not the only question as long as it keeps up with the slowest area of unfragmented 5400rpm 3TB hard drives.
I’m told “shoeshining” is when the tape goes so slow that it has to back up and it wears out the tape and drive faster. My understanding is that simply writing slower, as long as it is not shoeshining the tape, should not damage/increase wear or anything else. So using a 3TB drive which starts writing at maybe 140MB/sec and slows to 85MB/sec at the other side shouldn’t be a problem if the tape drive can step down it’s speed (and maybe some step or vary speed better than others as buffers fill and empty – also those aren’t verified numbers of my drives just a random example not unlike what i’ve seen on other drives)
I would like to use LTFS to write files directly for ease of recovery but am not sure how groups of smaller files are handled/normally in Windows for instance, groups of small files will write and copy much slower. I would like to know if it’s possible to make a “tape image” to write to tape in LTFS format (meaning the ability to directly read files from the tape, not locked up in a TARball, where I am concerned that if I lose a bit somewhere in a 2.5gigabyte tarball I can’t access any other file, but my assumption is that a lost bit on a file would lose just that file, like it is on a CDrom) similar to the way we used to have to write ISO images of maybe tens of thousands of small picture or text files to our hard drives, and then write the ISO to disc which was a monolithic file and would read faster.
I would strongly prefer to use SAS because I expect to have a controller card for accessing hard drives anyway, over any other alternative. (like fibrechannel or even USB3)
I am pretty price sensitive due to a student’s budget – I see LTO6 drives down around $1400 (though i’m curious whether they are likely to drop with LTO8 being out) so if someone suggests a model that is $1900 instead I would want to know what critical things it does so much better to justify the cost.
I don’t have a preference between full height or half height drives other than hearing half height ones ‘may not write at fastest speed’ which probably doesn’t matter too much to me though it would be nice for the future when faster hard drives are used. Given a choice an external LTO drive is better than an internal but not required – provided it has a serviceable PSU because I expect to be using this for probably 10-12 years.
Can any experts here help me out? 🙂
-
4 Replies
-
LTO drives include speed matching, which adjusts quite well to slower speed sources.
I think for LTO-6 the range of speeds is between 55MB/s and 160MB/s. So as long as your source drives can provide data in that range then the tape speed will adjust so that rewinding is minimised.
LTO-6 drives haven’t dropped that much in price, probably because there is a lot of LTO-6 media so people are hanging onto their drives.
So if buying a new drive I would get LTO-7 for future proofing. If looking second hand then LTO-6 or even LTO-5, as the great thing about LTO is that older media will be available for a long time.There is no need to get a full height drive, it’s only LTO-8 where the full height drive is faster and then not by that much. (360MB/s vs 300MB/s)
If you use LTFS as the format on tape then each file can be read separately. The speed does drop when copying small files, but image sequences and media files all copy at full speed. It’s not worth zipping or taring prior to writing.
Your tape can also be read by anyone else using LTFS on Windows, Mac or Linux which is handy for file interchange.
Best,
Martin Greenwood
CTO
YoYotta.com -
So in theory EVERY LTO6 drive should write fine as slowly as 55MB/second? Are any better than others? Is there anything specification or featurewise to suggest buying one drive over another then or not really/I can just buy on price?
I’m pretty sure I want LTO6 – by the time LTO7 media is cheap enough to matter i’m willing to upgrade to an LTO8 drive (and use M8 format) which can’t read my older tapes anyways so i’d still need the older drive. I see no point in paying more for an LTO7 drive now just to use LTO6 media – I see no point in using LTO7 media without an LTO8 drive to make better use of it. And economically the higher cost of the LTO8 drive doesn’t pay for itself over the LTO6 in my expected time window of usage and minimum vs potential writing needs.
Also if i’m going to newer drives i’m going to need higher performing writing systems and such – i’ve run a spreadsheet of how many hundreds of TB I realistically expect to have to use or need to scale up to over a period of time and nothing newer ever makes sense economically when i’d have to upgrade HD’s and change intended workflows even of migrating data around to faster drives. I just want to write straight to tape from each drive in an array of 3tb internals which I already have (along with overbought spares) which on the slow side max around 80MB/sec I think.
I’m sure the writing of small files to the drive will keep up with the cache, i’m just concerned about the reading side and how many seconds deep the cache is if I hit a slow point of too many files. I don’t plan to be storing lots of small files, it’s more like say a large video file then hundreds of small res snapshots at different timestamps in a preview directory, then another large video file. I’d still like to know if there’s any way to make a “tape image” of contiguous files that can read at full speed without thrashing around the FAT and whatever usual overhead is done… that’s less expensive than setting up an SSD or something.
-
LTO-7 SAS drives are not that much more expensive and I would guess that within 6 months LTO-7 media is the same price per TB as LTO-6, it’s getting closer all the time. Meanwhile you can use the LTO-7 drive to write LTO-6 media if required. Fitting 5.7TB on an LTO-7 tape makes sense and if you move away from single drives then you can take advantage of the faster write speed.
However the minimum speed matching for LTO-7 is ~ 100 MB/s so writing directly from single hard drives would be slow as the tape stops and starts. This is also the case for LTO-6, if you have lots of small files the transfer rate will drop. But you don’t want to worry too much about this, just leave it archiving.
Martin
CTO
YoYotta.com
-
Minimum LTO speeds are specified in this PDF here https://tapepower.fujifilmrmd.com/Shared/PDF/knowledgebase/LTO6-TECHFLASH-2015.pdf
Probably the easiest way to achieve maximum throughput is to add another hard drive slightly larger in size than the largest tape cartridge you are going to use and simply format this drive before each use and then copy the data to it from your main drive (as long as you don’t write to it in multi-threaded mode which for example the Windows Robocopy command defaults too). The logic behind this is that data will be laid down sequentially on an empty drive (because you previously formatted it) and hence when you read off this drive it will be read back in a sequential manner. On a typical 3TB drive the data starts off at around 200 MB’s on the outer tracks and drops down to around 100 MB’s on the inner tracks. e.g. have a look at the HD Tune 5.0 benchmark here https://www.technologyx.com/pc-hardware/toshiba-dt01aca300-3tb-sata-iii-hdd-review-premium-hdd-storage-for-less-than-0-03gb/2/
If by some chance you would want to maintain the full speed of 160 MB’s you would need to either get something like a 6TB (8TB?/10TB?) hard drive and format into two 3TB partitions and just use the first (outer) partition where 160 MB’s sequential speed is possible https://www.myce.com/review/toshiba-x300-6tb-high-performance-hdd-review-81607/benchmark-results-3/ and this is called “short-stroking” the drive, another advantage not important here is that on a short-stroked drive the heads don’t have far to travel but that is more important for random access and not relevant here for the mainly sequential access usage as you will use. The only other way to have high disk read speeds is to use Raid-0 https://thepcenthusiast.com/wd-red-pro-6tb-nas-drive-review/6/ so you would probably have to purchase two 2TB drives to create a 4TB Raid-0 array which is easily do-able on a SAS controller.
Reply to this Discussion! Login or Sign Up