-
FCPX library size
Hi All,
I am about to start work on a feature doc and much of it will be shot in 4k. With that in mind I am looking for advice on max library file sizes for FCPX.
I have spent the last 18 months using nothing but FCPX for broadcast docs and short corporate projects and have moved from the spitting at my screen phase to loving FCPX. But my library files can be huge. A recent 8 minute corporate project weighed in at 900gb with all the media transcoded ingested into the project and most o the rushes for this (bar some drone footage) was HD.
I have tried working with files sitting on my main ruses server, but FCPX does not seem to play too nicely with this set up, so I am wondering just how much disc space to dedicate to this new project. Or perhaps I abandon FCPX for Premier or Davinci…?
Thanks for any thoughts & suggestions.
Andy
Andy Johnstone
Wild Dog Limited
film & multimedia production
http://www.wilddogworld.com -
9 Replies
-
“I have tried working with files sitting on my main ruses server, but FCPX does not seem to play too nicely with this set up”
What exactly was the problem you experienced?
As you know, you have two options with Final Cut: to add media to your library or to leave files in place. The “leave files in place” option is the only option in DaVinci Resolve; I’m not sure about Premiere. On one Resolve project I’m working on currently, I have about 3 terabytes of footage but my Resolve database is only 15 megabytes in size; my project backup files are only 1.2 megabytes in size. I keep all my footage on a Thunderbolt-connected RAID. Of course, if I disconnect the RAID, all my files go offline.
When I use Final Cut, that’s how I work too: I choose the “leave files in place” option and Final Cut doesn’t copy any of my original media into the library. I would only copy files to my library if I needed to share the library with someone else, or if I needed to work on the library on another computer offsite.
-
If you have background rendering on, FCPX’s cache files can get massive if you don’t remember to delete unused render files from time to time.
—-
Jeff Kirkland | Video Producer & Cinematographer
Hobart, Tasmania | Twitter: @jeffkirkland -
[Andrew Johnstone] “I have tried working with files sitting on my main ruses server, but FCPX does not seem to play too nicely with this set up, “
More details please.
What kind of server? How is it connected to your computer? Where are your libraries stored? Your media? Your cache?
Please give us a snapshot of how everything is connected and how you organize the libraries and media.
-
Thanks for all the comments so far. Deleting old cache files is a good idea and something I should do more often.
I have a thunderbolt 2 Raid on my system where I keep all my rushes. The for each project (or ‘library’ in FCPX-ese) I import that material and also create proxies. The project libraries live on external SSD cards and I have one for each project. I also create a back up copy to another disk, as well as exporting the XML files of the final cut to store on my cloud server.
Andy
Andy Johnstone
Wild Dog Limited
film & multimedia production
http://www.wilddogworld.com -
How are the “external SSD cards” connected?
Where is the cache?
Where is your imported media?
-
[Andrew Johnstone] “I am about to start work on a feature doc and much of it will be shot in 4k. With that in mind I am looking for advice on max library file sizes for FCPX.”
I have edited a large documentary on FCPX with 8,500 4k H264 clips in a single library. This included 220 camera hours, about 130 multi-camera interviews, occupying 20 terabytes. Most of the media was in just three events. Machine was a 2017 iMac 27. The media was on two redundant 32TB OWC Thunderbolt RAID arrays. Actual library file size was small, about 120MB since all media, proxies and cache were external to the library.
In general the performance was OK but I was using proxies. FCPX proxy management can be difficult since they are tied to the original disk volume name where they were created, also proxies cannot be relinked. However they work OK.
An alternative approach is relink to the external proxies as original media, edit, then relink back to full-res media for final conform. There are several relink constraints and this should be tested on the specific codecs and file types before committing significant work.
If using lots of effects an iMac Pro is better because it is quieter under sustained high CPU/GPU load. A more recent doc I worked on used an approx. 2 hr timeline and had many hundreds of effects, including about 50 clips with Neat Video. It took several hours to render on my 10-core Vega 64 iMac Pro. Media in that case was about 14-16 TB, spread across a 4-drive SSD array and several smaller SSD drives. Media, proxies and cache were all external to the library, using “leave files in place” import and other media placed in designated locations using the library inspector>Storage Locations>Modify Settings.
-
My normal work flow is that:
External SSDs connected by Thunderbolt 3. Cache is in the Library file on the external SSD, imported media is in the Library file too.
I have previously cut using ‘Leave files in place’ (as I would on Premier) but this creates issues when transferring projects (libraries) between my system and the online system 9for grade & dub) and I have found it is much simpler to keep everything bundled into one FCPX library file.
I would anticipate getting a much larger external T3 RAID with a dedicated project back up drive as well, but just how big is the Q…and does the sze of the FCPX library file affect performance after a while. I assume not if FCPX has everything converted to its favourite file format.
Andy Johnstone
Wild Dog Limited
film & multimedia production
http://www.wilddogworld.com -
Thanks that is all very useful info. Good to hear that FCPX does manage very large project (library files) – all be it on a much speedier/hefitier machine…
Andy Johnstone
Wild Dog Limited
film & multimedia production
http://www.wilddogworld.com -
[Andrew Johnstone] “Good to hear that FCPX does manage very large project (library files) – all be it on a much speedier/hefitier machine…”
I edited the original 20TB library using a top-spec 2017 iMac 27, however an iMac Pro is really better for tasks of that size.
Basic guidelines:
– If 4k H264, it will generally require transcoding to proxy for optimal editing performance.
– If 4k ProRes it can probably be edited without proxies, if the I/O is fast enough and if on a 2017 or later iMac 27.
– Use media imported with “leave files in place”. If the camera media is tree-oriented and doesn’t allow this type of import, consider re-wrapping (not transcoding) with EditReady before import: https://www.divergentmedia.com/editready
– Do not ever import bare .mts files from an AVCHD package using “leave files in place”. This can create a large I/O burden due to how FCPX handles that internally.
– Use FCPX library inspector Storage Locations>Modify Settings to place cache and storage of generated media external to the library itself. This includes proxies and optimized media. Doing so keeps the library small and facilitates easy file-level backup by duplicating the library with Finder.
– Use only HFS+ or APFS external drives.
– Do not copy clips or projects between libraries unless they are inside a “transfer event” created for that purpose. Otherwise spurious duplicates can occur.
– Efficient post production starts in production. Make sure all cameras and recorders have the correct time of day, and re-verify this each production day. If shooting multi-cam try to use a slate. If doing stop/start multicam, consider using timecode devices like Tentacle: https://tentaclesync.com
– Similarly, try to ensure all cameras are using the same frame rate, and this matches the expected frame rate for editing and final distribution. Even with modern optical flow rate conforming, mixing non-multiple frame rates like 23.98 and 29.97 is not perfect. In a perfect world all cameras on a given production would shoot even multiple frame rates such as 24/48/72 or 29.97/59.94/89.91. Admittedly this isn’t always possible.
– Ideally before importing any media, make sure all camera filenames are *globally* unique across the entire production. Not doing this can cause spurious duplicates if XML is loaded, also having a unique filename greatly streamlines post production turnover. My doc team simply appends a 5-digit unique serial number to each offloaded file using the tool “A Better Finder Rename”: https://www.publicspace.net/ABetterFinderRename/index.html
However Finder itself has batch rename ability as shown in this brief tutorial on MacMost Now: Finder itself has batch rename ability. This short tutorial explains how to use it: https://macmost.com/step-by-step-using-the-mac-batch-rename-tool.html
– Do as much work as possible in the Event Browser before adding material to a timeline. Fully leverage FCPX database features; try to avoid thinking in terms of bins. This video “One Smart Collection To Rule Them All” shows one possible approach:
https://youtu.be/sjuCfJFhdo0Some contents or functionalities here are not available due to your cookie preferences!This happens because the functionality/content marked as “Google Youtube” uses cookies that you choosed to keep disabled. In order to view this content or use this functionality, please enable cookies: click here to open your cookie preferences.
Reply to this Discussion! Login or Sign Up