-
Ideas for Archiving 5,000 Minutes of ProRes Video
I’m currently collecting thousands of minutes per week of video encoded with ProRes SQ (equates roughly to 1GB/Min). I would like to keep all of this video in ProRes for archiving but we’re talking 1 Terabyte per 1,000 minutes (or 16.5 hrs) of recording. I am racking my brain on some solid solutions and I thought you might have some ideas.
Some of this footage will be used again but some will just sit on the shelf until one day a client requests the video from x-meeting when y-presenter had a guest appearance 3 years ago now we want to see this again…(you know the story).
I doubt any of this footage will ever need color work or compositing in the future so I’m not too concerned with color compression. I am concerned with how much space I can save with out loosing any visual quality in my video.
I’m considering using H.264 my default archival solution. I can encode 1080 HD Video limited to 8,000 Kbps and get no loss in visual quality (files usually come out to 8.2 to 8.5 Mbps) from the original footage. Now I’m closer to some storage I can handle – 1 TB = 260 hrs. or 15,600 Minutes!
I have some questions that I’ve not been able to answer; here it goes:
1. Should I consider another I-Frame only format format for archiving or will I,P,B Frame format work for my usage? Any reasons NOT to use h.264 for archiving?
2. I’m using Compressor and can’t find handles to encode my H.264 with a 4:2:2 color. In fact, I can’t find any detailed documentation on how quicktime encoding works with H.264. I can’t tell if my file is encoding with 4:2:2 or 4:2:0. I know that Episode gives you the handles to make this happen but I don’t have Episode yet. Any thoughts (besides “get episode”)?
3. I currently capture all of my footage with an AJA IOHD at 1080i/59.94 except it doesn’t really capture at that frame rate. Nope, it captures at 29.97 which increases motion blur. I’m thinking about switching my entire workflow to 720p because the AJA will capture 720p @ the proper 59.94 fps. Although this provides me with the same file size as 1080i, it saves me a deinterlacing step in my workflow. Since most of my material is destined for a progressive display this makes sense to me. Any suggestions?
4. I’m trying to steer clear of hard drives sitting for extended periods so I’m just burning files to 25GB Blu-Ray discs for storage. Do you think Blu-Ray readers will be around for the next 10 years? (I think so, DVD data drives have been around quite a while.)
5. Am I way off the rocker here? Is there anything else that you would consider working with HD material that provides a direct-to-disk workflow?
My recordings are working well and the clients love them but I’m running out of storage space fast. I currently have over 10 Terabytes on my desk and I’m still fighting for storage.
Thanks for playing along.
Brian
—
If video was easy, just anybody could do it! -
3 Replies
-
[Brian Alexander] “1. Should I consider another I-Frame only format format for archiving or will I,P,B Frame format work for my usage? Any reasons NOT to use h.264 for archiving?”
Quicktime H.264 has a subtle gamma shift bug which you may not like.
[Brian Alexander] “2. I’m using Compressor and can’t find handles to encode my H.264 with a 4:2:2 color. In fact, I can’t find any detailed documentation on how quicktime encoding works with H.264. I can’t tell if my file is encoding with 4:2:2 or 4:2:0. I know that Episode gives you the handles to make this happen but I don’t have Episode yet. Any thoughts (besides “get episode”)? “
compressor (or rather Quicktime) does not support the H.264 Levels and Profiles required for 4:2:2 support
[Brian Alexander] “Since most of my material is destined for a progressive display this makes sense to me. Any suggestions? “
I’d capture at 720p
[Brian Alexander] “4. I’m trying to steer clear of hard drives sitting for extended periods so I’m just burning files to 25GB Blu-Ray discs for storage. Do you think Blu-Ray readers will be around for the next 10 years? (I think so, DVD data drives have been around quite a while.) “
Most certainly.
[Brian Alexander] “5. Am I way off the rocker here? Is there anything else that you would consider working with HD material that provides a direct-to-disk workflow?”
Probably 😉 – Have you considered Data tape?
__________________________________________________________________
Two years from now, spam will be solved. – Bill Gates, World Economic Forum 2004 -
Hi Daniel. Thanks for taking the time to read this.
[Daniel Low] “Quicktime H.264 has a subtle gamma shift bug which you may not like”
I know and have lived the gamma issue. My current workflow is completely unaffected by this bug. My IOHD to ProRes to QuickTime H.264 gamma stays the same. Files I get from clients are susceptible to this gamma shift and unfortunately I never know when or where it will strike. When it does strike my fix is to add a Gamma of 1.2 in Compressor and that leaves the video unchanged – no gamma bump. I’m thinking this happens when the video shifts from an RGB to YUV or possibly (not as likely) when we go from a 10bit to 8bit.
[Daniel Low] “compressor (or rather Quicktime) does not support the H.264 Levels and Profiles required for 4:2:2 support
“I’ve notice that leaving the settings to “Automatic” and “Best Quality” that the bit rate usually ends up around 16 Mbps, double my standard target of 8 Mbps. I figured this might double because we’re going from a 4:2:0 to a 4:2:2 but I’ve found no documentation that supports this; only wishful thinking.
[Daniel Low] “Have you considered Data tape? “
Nope, I do not know much about tapes. Are they expensive? What does a Data Tape Deck cost? What format do I lay to Data tape or is it just Data in general? Would I still want to compress my video to save on tape costs? I’m set up for an all digital and optical workflow so I haven’t considered much else since I have so much material coming in.
I appreciate it the feedback.
Thanks again. -
[Brian Alexander] “I’ve notice that leaving the settings to “Automatic” and “Best Quality” that the bit rate usually ends up around 16 Mbps, double my standard target of 8 Mbps. I figured this might double because we’re going from a 4:2:0 to a 4:2:2 but I’ve found no documentation that supports this; only wishful thinking. “
https://en.wikipedia.org/wiki/H.264#Software_encoder_feature_comparison
[Brian Alexander] “Nope, I do not know much about tapes. Are they expensive? What does a Data Tape Deck cost? What format do I lay to Data tape or is it just Data in general? Would I still want to compress my video to save on tape costs? I’m set up for an all digital and optical workflow so I haven’t considered much else since I have so much material coming in. “
There are a number of different tape formats to consider and yes the initial outlay can seem expensive but tape nearly always works out the cheapest per bit in the long run but there are many other factors to consider i.e. speed, accessibility and so on
The main ones to look at are AIT, DAT and LTO. If you are happy with Blu-Ray then it’s likely that this will be the best on-going option for you as media is easy to come by will just get cheaper and you have already made the investment in hardware.
__________________________________________________________________
Two years from now, spam will be solved. – Bill Gates, World Economic Forum 2004
Reply to this Discussion! Login or Sign Up