4k edit: storage solutions

[Tero Ahlfors]

[Gabriel Sanchez] “a file for theatres”

https://en.wikipedia.org/wiki/Digital_Cinema_Package

[Ht Davis]

They’re talking to you in total storage for raw footage, not in total storage for working footage. Big difference. Working footage should be internal. RAW should not if it is too big to store internal. You will need some SAS.

You will need ram. You will need proxies, you will need to edit in your output resolution, but interpret your proxies to that for your previews. Use a server farm to build your 240tb for footage and have them catalogue it to that location (this is a logistics bomb).
You will need a fiber based connection of 10-20gb/s. You will also need to be able to run the video out to proxies.

How to:
1. set servers up with AE and AME, set up watch folders\scripts for auto copying and proxy creation
2. Set up your internal system with the max it will hold with 1-2tb drives in a RAID formation and use this for cacheing
3. Add a SAS with an exterior shell casing that can handle up to 30th RAID for holding working files
4. Make a disk image big enough to hold the proxies, project and previews, store it on the External SAS and begin building the folder structure on that. Every night, back up the changes to servers by imaging, and every two weeks have changes burn to discs (look into acronis software solutions). This way, you can move the external SAS and the disk image anywhere, and your entire project is contained. You only need to relink the original media and render effects when you cross system platforms.

When you finish with the video itself, you’ll need to move that entire project to the servers to output your final work. You’ll probably need more like 300-400tb. However, you’ll be able to work on your main editing system while the servers are exporting the final file, so you’ll still be able to work. You may want to look into compressor and an apple server setup… …this would give you the ability to not only share storage but processing cores for compression of the final output. That final output will be placed on a disc or a hard drive for playing in theaters, and it will be compressed formatting. You get better quality and compression in Compressor.

[Gabriel Sanchez]

When they told me i´ll need 120 TB for storage they meant this will be in the case they shot with Odyssey attached to the FS700, which produces CinemaDNG files in Raw mode. The cheapest solution is to record in 4k ProRes HQ 4:2:2 through the Odyssey, which will produce about 5 times or more smaller files, but then we won´t have a Digital Negative.
So finally we´ll be shooting with the sony AXS-R5 instead, producing Sony Raw mxf files, which are 2,5 times smaller than the Odyssey´s Cinema DNGs, and very much easier to handle in PPro.
This means that the storage needed will be about 48-50 TB, not 120.

I´ve talked to a people expert in storage solutions, and they suggested a 64Tb NAS for de Digital Negative backup, that can be moved to another location with all the original files plus the exported DPX sequence or Cineform file of the final edit for another people to make the DCP. We think about a NAS because it can be connected to any other machine with no specific card installed.
By the way, thanks Walter for making me notice what i haven´t (forget to think about the backup, whitch obviously doubles the storage needs!)

For the work, i´ve been told (as David said) to use one Direct Attached Storage cabin of 16x4Tb Raid5, through an Areca Fiber Card (PCI Express), with all the original footage, and other DAS same storage capacity (or maybe half the storage needed for original footage) through another Areca Card for the proyect assets, animations, exports, previews, etc.. Will this be enough?

Also, i´ll have an internal 6x3TB Raid0 for media cache, i have no more free slots for making a raid with more disks (other slots in my mobo are taken by a bunch of SSD drives).

Sony Raw MXFs play smooth in PPro, although i haven´t made any upgrades to my workstation yet, so i think i wont make proxies and going to work directly with the raw files. Do you think i could be wrong? I´ll work with Speedgrade, Red Giant Mojo, and eventually using Twitch and some Optical Flares.
Anyway i can always convert all the raw files in Cineform files or DNxHR HQ to make the edit if i encounter any problem working with the original Raw files.

I´m planning to put another graphics card for the 4k monitoring (Dell P2715Q UHD external monitor through a GTX980) and use my older GTX780Ti for MPE in GPU mode and for program interface in an extended desktop of two 1080p monitors. As far as i know PPro CC2014 won´t use both for MPE playback, but he´ll use them both for encoding/exporting, which would be of help.

Also i plan to upgrade my RAM from 32GB to 64GB.

What do you think? Will it be enough?
Thank you all very much for your inputs.

[Ht Davis]

Heh… …Must be nice… …Yeah, that will do it. If you have 2xgfx and CUDA capable windows system, you should fly right through. Just remember to get 2gb gfx cards or higher for 4k throughput and make sure their data path is wide enough for the playback of the video. Alternatively, render out a preview first thing. That will give you iframe playback speed in premiere but let you edit away at similar to native quality. Ram to 64 is probably a bit much, but you might be glad you have it when multitasking your edits.