[Shawn Convey]

Ill check out the Thunderbay 4 — re: SAS ports I have the mTape thunderbolt unit so no SAS adapter unfortunately…. thanks for the ideas!

[Shawn Convey]

Thanks for the tips!… I should have mentioned that I was looking for something on the more portable side as I am on location all the time and will occasionally need to fly with both my LTO and my drive solution… I was (am) looking at the OWC GEMENI which I found several hours after writing this post.. the speed isn’t crazy but the +/- 400 MB/sec reads seem like they should be ‘enough’ for LTO writing… It still isn’t as small as I had hoped but…. It would appear there is a “middle-speed” void in the market when it comes to backup solutions… either they are big (sometimes huge) with slow speeds or they are super tiny with super quick speeds but nothing in between… I was sorta shocked to learn that a SSD isn’t much if any cheaper than an m.2 yet, and that there are not many options for a portable SSD RAID for 2-4 drives (there are a few but not many)… not that that matters I suppose as if you are going to spend that amount of money on SSD’s you may as well get m.2’s now and there are just as many RAID solutions for those chips as there are for the SSD’s… Odd.

[Shawn Convey]

@neilsadwelkar

Thanks again for your additional thoughts ( I think I posted my response to your first post at the same time your last post came through 🙂 )

All of this info is very helpful but I do have a few follow up questions:

1. Cataloging software — I checked out the 2 you mentioned and I didn’t see any specific LTFS / LTO support. Would I simply drag the mounted LTFS volume on my HD to either of these software choices for them to make the catalog or can I import a manifest or is there a better way to connect through Canister?

2. Remote Ops — that is very interesting proposition and not one I considered as possible… I would like to figure out a good workflow for my needs moving forward as once I am done archiving the past 10 years (which as a one man band isn’t nearly as bad as even a small production house) I will simply be archiving shoot data as it comes so it should all be very manageable moving forward… that is once I work out the workflow and the kinks. But with all that said knowing that your remote services even exist gives me some more piece of mind so thank you for mentioning that!

[Shawn Convey]

LTO write speed matches the read speed of the hard drive you’re reading from. I haven’t come across a lower limit, and I often have to backup to LTO-8, from 2.5″ portable drives that read at 60 MB/sec when full. I’ve even seen speeds as low as 40 MB/sec.

Writing to LTO takes longer with file sequences like .dpx, .ari, .arx, .tga etc. meaning 100 GB consisting of 14,000 files, will take longer than 100 GB consisting of 30 files. Longer means about 1.5-2 times as long sometimes. Nowhere near 5 hrs for 9 GB. So that’s definitely an anomaly.

Thanks for the insight @Neil!

– I don’t have an intel mac at the moment unfortunately.
– I will have to double check how all of my source drives are formatted but normally it is HSF+, I did have some ExFat / NTFS drives for shuttling between PC and Mac but I don’t believe I tried to use any of those as source drives in this scenario.

But I don’t think the problem is with Canister, in your setup. It either to do with SAS and LTO drivers for your M1 host computer, and/or errors on the drive with data. Have you tried doing a verify and repair on your source drives?

Right now the team at Canister is communicating with IBM trying to figure it out as well..
In the end it looks like I have a complete LTO 8 tape which which had about 8-9 sessions written to it using different source drives, and connection configurations write and verify. With the exception of that ULTRA-SLOW 9.5GB back up session everything else seemed to go smoothly with varying write / verify speeds depending on drive speed and connectivity. I did notice a much slower transfer when I had the mTape TB3 plugged into my OWC m.2 raid unit which was then plugged into the MBP, which makes sense I suppose, but even that session wasn’t as slow as the 9.5GB write.

[Shawn Convey]

Therefore, during the course of backup/archive of your folder, you’ll find the data-rate to tape will slow while reading many small files, and speed up when reading large files. There’s not a lot that you can do to overcome this, just the way of things.

OK that is great to know… I still think something was acting funny as 5.5hrs for 9.5GB (not TB) of data transfer / verification seems extremely excessive. But I have the developer looking into this as well…

Can you point me in a direction to find practical info such as this (small files of same data total will take longer to write / verify, how much space should be left on a LTO tape as buffer, Issues to watch out for, LTO workflows etc….) online? I am happy to do my own research on all of this and don’t mean to single out a forum or person on a forum individually for all these questions but I have been coming up short finding a location (or locations) that have non IT, end-user practical info on the basic in’s and outs of LTO archiving…

Also as you work for a software developer does Archiware have any plans on releasing a lower cost single user version of their software? The investment in a a LTO system (hardware and software) is very large for a single user / small shop and while I understand that hardware is regulated or controlled by IBM and they set the price I probably still would have justified the cost long ago IF the software wasn’t similarly expensive… It seems a shame that there are only one or two software options for the “little guy/gal” and the film / video field is becoming far more populated with single owner shops who have huge amounts of data that needs archiving… Anyway, I guess I am saying there seems there may be a growing demographic that could be served by more lightweight / low-cost options for software. 🙂

[Shawn Convey]

So your LTO 8 drive will be spinning at its ‘top speed’ when date is arriving at 360Mb/sec, but as the incoming data-rate slows, the drive will be able to slow down to match. There is however, a lower lower limit, at which point the tape speed can no longer match the data speed. At this point, the drive will stop the tape, fill up its internal buffer with data, and then run the tape again when there’s enough data to write some blocks to the tape.

That is great to know David and much appreciated information which makes sense now that you spell it out for me 🙂 — You were also correct in the fact that mTape LTO 8 does have a max at 300Mb/sec and as such I suppose that the 250Mb/sec my Oyen Digital Novus 12TB External USB-C (3.1,Gen2) 7200RPM Hard Drive has wouldn’t slow down the process all that much.

NEW QUESTION
Bad Session Write — I have made several successful large transfers to my LTO tape however the folder that is currently being backed up is acting problematic (VERY SLOW transfer and verify speeds taking 100x longer than it should with the drive spinning up and down often) and I suspect that there is a good chance that this write session will not verify successfully in the end AS SUCH — What happens to the remainder of the data that is on the tape? Do I need to reformat the tape and start from scratch OR can I somehow delete that last block of data and continue?

[Shawn Convey]

I actually do have more questions… I emailed you a bit ago… if you have the time to respond that would be great! thx!

[Shawn Convey]

Hey there Michael,
Thanks for your reply but I am not sure if we are on the same page exactly…
I have all the proper audio tracks assigned L , R, C, LS, Rs, LFE (then the stereo mixed pair L,R)
when I play it in Resolve and I look at the audio meter everything is tracking as one would expect.

The issue is certainly with the way I have either the BUS set up OR (and more likely this) the output assignment in the delivery section.

I am not using another software to author the DCP as in Resolve 15 it isn’t necessary as it makes and outputs a validated package for you.

I am currently testing using my ProRes (8ch) file and authoring in DCP-o-Matic and that is working however DOMs image isn’t nearly as nice as Resolves… so my next test is taking Resolves video file and marrying it with DOM’s audio file.

[Shawn Convey]

bump

[Shawn Convey]

Fantastic… thanks so much I am going to check this out this weekend…
Much appreciated!