-
LTO – faster to archive audio than video files
Hi,
We regularly write video, photo, and audio data to LTO tape. We recently noticed that when writing audio files, the tape write operation goes much faster. I am anxious to know why this is.
Note: We always write only one type of file to a particular tape (e.g. one LTO tape will contain only video files)
We’re using Tandberg LTO-4 SAS Half height standalone drives, and for a typical tape of 650GB photo or video it takes about 2hrs40min to write a tape (which works out to be around 70 mbps), but when we write audio files, the speed reaches around 110 mbps and for 650GB the job finishes in under 2 hrs.
We use BRU as our backup software – which we have been very happy with for the past 5 years. The only thing I can think of is that compression is being applied to the files, and the audio files (wav + mp3) are able to be compressed much more, and therefore it is actually writing less data.
However, we have not specified any compression options in the BRU config (since most of our data is video) we will not benefit from compression.
I’ve gone through our logs over the past 5 years (700+ tapes) – and checked the write and verify timings… and every time it as a tape containing audio files (15 tapes) I’m seeing the same thing.
If anyone can explain why this is happening, I would really appreciate knowing. As the drives have an advertised speed of 75mbps… if I’m sometimes getting 110 mbps then it feels like something odd is going on.
Kind Regards
Swami
-
2 Replies
-
Bru, seems to write some error correction and redundancy information along with every file. So, when one is backing up a large number of small files, Bru is slower compared to a small number of large files.
The extreme comparison is when we backup 2 TB of Arriraw (which is file sequences, 7 MB per file) compared to 2 TB of Red raw.———————————–
Neil Sadwelkar
neilsadwelkar.blogspot.com
twitter: fcpguru
FCP Editor, Edit systems consultant
Mumbai India -
Hi Swami,
Remember that the compression that is discussed in CODEC land is very, very different from the compression used by backup tools. When you modify an audio stem from 96Khz, 24bit BWAV to something like a 44.1K, 16bit WAV file, you’re actually losing data. When your backup solution compresses a file, you will get bak 100% of what you put in.
In the case of most audio and video CODEC formats at the edit level, the lossy compression has already been applied before you back it up or archive it. This means that the backup tool (BRU, ZIP, etc.) will not gain any further compression since the DATA is already compressed to the nominal level. However, in the case of photo data – especially RAW or TIFF formatted data – there IS room for data compression and you will see both higher performance and higher capacity for a given tape. In fact, we just witnessed a large collection (100’s of TB) of NASA black and white TIFF image data compress at over 8.5:1.
The good news is that the adaptive compression algorithms used in tape drives (all, not just LTO) is smart enough to recognize non-compressible data and just pass it through to the tape as it is received.
75MB/sec is the average for LTO-4 and non-compressible data, so your 110MB/sec is indicative of a good level of compression on your image data.
Tim
—
Tim Jones
CTO – TOLIS Group, Inc.
https://www.tolisgroup.com
BRU … because it’s the RESTORE that matters!
Reply to this Discussion! Login or Sign Up