[Joe Marler]

[David Lawrence] ” he discusses his logging method and why tags and metadata do not work for him”

He said he doesn’t like metadata tagging because other people might not tag the right shots. That means he can’t use CatDV or any other MAM, just markers that he himself applies. That might work on 6 Below which was mainly a one-camera shoot with an approx 66:1 shooting ratio. I’m not sure how scalable that is to the 200:1 ratio used on some films or the 400:1 ratio used in reality TV.

But he actually IS using metadata, as shown at 15:00-16:00 into the video. Using a dry-erase marker board and 3×5 cards, he essentially constructed a paper version of FCPX’s Event Browser, with tags hand-written on each card. Instead of rejecting the clip via software, he wrote a red X on the white board.

OTOH if a film of this complexity can be done without software-assisted tags and metadata, maybe that means it’s less important than often depicted. He obviously got the job done, and effectively.

However this was a scripted narrative. The script forms a pre-defined organizational backbone and he used timeline markers to quickly find scenes and takes. For observational documentary there is no tightly defined script, the shooting ratio is higher and the storyline is often discovered during the editorial process. In that case I’m not sure his method would work as well.

[Joe Marler]

[Oliver Peters] “….if you have a lot of clips, it will take a long time to load…”

I’m working on a library of 1.4 terabytes comprising 1,600 4k H264 clips, 58 hours of material. On my 2015 iMac 27 it loads within about five seconds. The media is on an 8TB Thunderbolt RAID-0 array.

Previously I had 6,000 clips comprising about 2.4 terabytes in a library on a conventional spinning 16TB RAID-5 array, and that loaded within about 10 seconds. Most of the clips were in a single event.

I had waveforms turned off in the Event Browser, but didn’t do anything special beyond that.

There can be performance problems (some severe) if you import any AVCHD content “in place”. Also, the Event Browser in filmstrip mode can take a while to generate thumbnails when scrolling through a huge library. Unlike Lightroom you can’t tell it to pre-generate thumbnails — they are done dynamically as you scroll down/up, then cached. However none of these affect “loading time”. I’ve never seen FCPX take very long just loading a library, event or project.

[Joe Marler]

[Dan Bowman] “I’m trying to import Sony FS5 AVCHD footage into FCPX 10.2.3 without embedding the clips…But FCPX won’t let me leave files in place…having to copy clips before importing to FCPX is a deal breaker for me. Has anyone found a solution?”

If the AVCHD content is in the original tree format (despite being copied to hard disk before import), FCPX will not import that “in place”. You could copy the video files outside the tree and import in place from there — but this should NEVER be done with FCPX, for two reasons:

(1) Risk of missing metadata causing problems with clip spanning, or other unknown issues. However new cameras using SDXC cards automatically use the exFAT file system which has no practical file size limit, so clip spanning might be less a problem than previously. Unfortunately you never know — the camera might split the file even though the exFAT file system does not mandate that.

(2) Possible I/O performance problems from “in place” AVCHD files. This causes excessive small random I/Os when scrolling through the Event Browser in filmstrip mode. Even a small % of “in place” AVCHD content mixed in a large library of non-AVCHD content may cause this. I don’t know if this happens with all AVCHD content or just from certain cameras. I haven’t tested FS5 content but I’ve seen it from other cameras. It is potentially quite serious because there’s no obvious error — everything just seems slow. Isolating, removing, then properly re-importing just those AVCHD “in place” clips can be tricky — especially if they are already in projects.

By far the easiest, fastest solution for AVCHD is simply re-wrapping with EditReady before import. It is lightning fast, preserves all metadata, allows in-place import, and the import itself is much faster: https://www.divergentmedia.com/editready

[Joe Marler]

[Jerry Jones] “…I finally made the switch to FCPX a month or so ago – and am loving it. But I still have a few questions. For example, how to best organize my folders where the data is stored….Do I need to combine these two folders?…I do not want to move things around at the file level (as pictured above) and then find out I have screwed myself. Is it safer to do this organizing within FCPX somehow?… how can I better understand how to use “Events?”.”

It is difficult to answer these in a forum post. You are thinking about media management in physical, hierarchical, folder-oriented terms. However FCPX primarily uses a database metaphor to organize content, and does this based on keywords and ratings you provide. One reason is to free you from dealing with files and folders at the filesystem level. When you query a relational database, you don’t know or care what file that was stored in — it’s simply in the database. It’s generally best to have all your media for a given “project” in a single folder tree, not scattered across multiple locations or disks. But how much organization is done within that media folder is up to you. Normally you don’t need to move media very often at the file level. Re physical management you can move content at the Finder level and re-link it or move it within FCPX. The methods and issues are too extensive to cover here. Ripple has an entire class on this.

To give an extreme example, you could import all your content to FCPX using “leave files in place” and without generating proxy or optimized media, and without doing any evaluation, folder organization or culling of material beforehand. You could import it all to a single event. Then once imported you could go through the content using the ultra-fast skimmer, marking range-based favorites, rejects, and keywords. Then you could begin querying this carefully curated, compact material to start assembling your timeline. This is just one method. The exact keyword/rating scheme is up to you. There is no single right way. FCPX gives you the tools to construct your own scheme, without mandating a specific system.

Obviously your media files and folders exist out on the disk. You may have previously had the practice of organizing content in physical terms — folders, sub-folders, etc. Transitioning editors often attempt to bring this philosophy into FCPX, but that’s not how it’s designed to work. Unfortunately the existing tutorial content (while high quality) generally doesn’t give much background and orientation about the philosophical and conceptual differences between FCPX and track-oriented editors. However they do explain how to use the product, so I recommend them, esp. Ripple Training.

FCPX has some “bridge”-type features to span from its database-centric philosophy to the outside file system. E.g, it can optionally auto-keyword media based on folder names, or keyword them based on Finder tags. FCPX has folders but this is solely to group keyword collections, not to mimic some on-disk organizational structure.

FCPX provides Events (which are sort of like folders) and these can be used various ways — but they are not mandatory. You could put all content in a single event and merely use the database methods to organize and find material. But Events provide an additional organizational tool.

The old way of organizing content was spending lots of time culling material, naming folders and copying files. This was partially because editing software had such poor organizational tools — it was mostly for editing not organizing. The lack of organizational tools and lack of ability to rapidly skim content then forced editors to use multiple timelines as staging areas. This in turn bred a timeline-oriented copy/paste approach, however that was a workaround for a product deficiency. It also encouraged only importing a minimum set of content, since it was so hard to manage inside the editor. This led to a preliminary phase of content organization using relatively primitive and slow methods. E.g, no video player is fast as skimmer, organization can only be clip-based, and a clip (unless duplicated) can only be in one folder.

FCPX is designed for (but doesn’t mandate) a different workflow, which is import lots of content without doing so much file/folder organization outside, then do most of your evaluation, rating and keywording inside, then work from the Event Browser to the timeline, not from one timeline to another. This takes some getting used to. Copying stuff around multiple timelines is a spatial and visual method, whereas querying a database is a more cerebral method. However, range-based keywording allows multiple, overlapping criteria. E.g, one long clip could have multiple marked ranges (like sub clips), each range having multiple overlapping tags: favorites, day, night, interior, exterior, interview, child, man, woman, location. All those can be easily queried. Here is one example:

One Smart Collection To Rule Them All: https://www.youtube.com/watch?v=sjuCfJFhdo0

Some contents or functionalities here are not available due to your cookie preferences!

This happens because the functionality/content marked as “Google Youtube” uses cookies that you choosed to keep disabled. In order to view this content or use this functionality, please enable cookies: click here to open your cookie preferences.

[Joe Marler]

Further study shows most (but not all) of the problems I see are related to using external proxies and two-editor workflow with different drive volume names. In that case the original media relinks but the proxies do not. If the volume name is the same, relink is not required upon receiving the updated library, and both original media and proxies work (usually).

If volume name is different you’re supposed to relink but this doesn’t seem to work for external proxies. That might be understandable since those aliases contain the original volume name and now the volume name is different. However even after a successful relink of the original media, then using the previous-stated procedure of manually recreating the proxy aliases within the library (which at this point uses the current volume name) no longer works.

There is an approved procedure for using external proxies but it involves consolidation which is extremely time-consuming if the proxies already exist within the library. I haven’t tested that for the case involving a volume name change and media relink to see of the external proxies created by that procedure keep working.

I have also seen some weird cases in two-editor workflow with external proxies where changing the destination volume name to the source volume name, then starting FCPX, then changing the volume name back will sometimes make the external proxies work. There is something odd about this area.

I also see cases where a few proxies just go off line by themselves after loading an updated library file, even when consistently using the same volume name.

[Joe Marler]

[Michael Angelo] “If there was a way to relink proxies I think it would solve both our problems”

After a fashion, you can move and relink proxies using this procedure. However I have tried it and it doesn’t work reliably in my case. Note: to create alias links in Finder via drag/drop, you must first select the files, do the drag, keep the mouse button down, then hold down OPT+CMD to invoke alias creation. This is not a documented or approved FCPX procedure from Apple’s standpoint, but a lot of people use it.

https://www.fcp.co/final-cut-pro/tutorials/1828-cheating-final-cut-pro-x-proxies-to-store-where-you-want-by-using-aliases

[Joe Marler]

[Michael Angelo] “We are starting to see many clips suddenly showing up as “missing proxies” when all we have done is quit and relaunch the app. …Looking in the package contents the FCPX generated proxies are still there but the application no longer sees them. The behavior was intermittent but now those proxies aren’t coming back at all….Has anyone seen this kind of behavior?…it’s an edit that we share between multiple systems and we are seeing similar behavior on both systems. They both have their own proxy media in self contained proxy only libraries that point to external drives with the optimized/original media….We’ve…trashed the preferences file, still the same behavior….”

I am seeing some of these on 10.3.2 when using external proxies and a two-location workflow where the library file is emailed back and forth, and the media requires relinking due to change in disk volume name. The media itself re-links OK but sometimes the proxies show up missing. Other times a few proxies will go missing in FCPX for no obvious reason — the files are still there and the aliases in the library still point to them. This may or may not be related to what you’re seeing. I am trying to investigate.

[Joe Marler]

[Ronny Courtens] ” I don’t see the MacPro as something this vast majority uses or even needs. Not even those of us working in television or film production….a friend who works at a large tv production company…decided to buy 42 maxed out iMacs instead because the price difference did not justify the performance gains. And these guys need performant hardware because they produce long-form television programs that often involve heavy 4K multicam projects that they want to edit natively….”

Is their “native” 4k content H264 or ProRes? In general a top-spec iMac 27 does well on ProRes and lots of content producers use ProRes acquisition. It also does well on 4k H264 with proxy, but you must generate the proxies. We became accustomed to fast camera-native editing on H264 1080p, but that doesn’t work so well at 4k — even using FCPX on the fastest possible iMac. It is borderline usable for limited single camera editing, and it’s much faster than Premiere on the same hardware, but it’s not remotely fast enough for multicam without proxy. That would be expecting too much.

From one standpoint this is sort of a niche within a niche — 4k H264 acquisition, native editing, no proxy. However 4k is everywhere now, regardless of final distribution resolution. H264 acquisition is very common and nobody who’s gotten used to native editing wants to go back to transcoding. For this scenario either a well-equipped Mac Pro or an improved iMac 27 would help a lot. Fortunately it appears Apple is working on both of these, based on the recent announcements.

[Joe Marler]

[David Hidalgo] “I´m rendering at the moment a 20 min video in 1080p from Canon 550D original files. It has a lot of color grading and noise reduction. But its taking already 10 hours ????
All in all, it doesn´t seem that the problem is in the CPU usage. “

The CPU is likely low because you are rendering a lot of computationally-intensive effects which are running on the GPU. Whether an effect is CPU-bound or GPU-bound is highly variable. Some like Neat Video noise reduction are configurable to use either CPU, GPU or both. In general video noise reduction is one of the most time-consuming, computationally-intensive effects — no matter what it runs on.

There is a saying, “a chain is only as strong as the weakest link”. For this specific workload, you are likely already bottlenecked on the GPU, so even having an infinitely fast SSD hard drive would probably not speed things up much.

You can confirm this by watching disk I/O on Activity Monitor during this task. The things to watch are read/sec, writes/sec, data read/sec, data written/sec, and also divide data read/sec by reads/sec and divide data written/sec by writes out/sec. The latter two gives the average number of bytes per I/O. However it is likely that all of those will be quite low for this workload. They are not low because your disk is slow but because FCPX is not issuing a high rate of I/Os. It is not doing that because on average it is waiting on the GPU. Even changing to a Thunderbolt SSD RAID-0 array would likely not help much because you’re already bottlenecked further up the chain.

The best steps are probably evaluate what effects are causing this by doing a timed export on a small clip with one effect at a time added. E.g, if a 1 min. clip with no effects takes 30 sec to export to H264, then adding effect “A” increases this to 40 sec, then adding effect “B” increases this to 5 min, it is mostly caused by effect “B”.

If it is video noise reduction causing this, then who makes the effect? Is there some way to configure it for better performance (on Neat Video there sometimes is). Also ask whether you need video noise reduction on all 20 min of a clip (or whatever ranges you are using it on). Some effects are simply very expensive from a CPU/GPU standpoint and are best used sparingly.

[Joe Marler]

[David Hidalgo] “I want…some more performance when reendering video….MacbookPro Retina…2,3 GHz Intel Core i7, 16 GB 1600 MHz DDR3…GT 750M…CANON 550D, 5D (in raw) and sometimes from Canon C300, Black Magic and similar….It takes a long time when rendering files with lots of effects….Should I make a RAID0 or buy a single external hdd…”

By “rendering” most people mean exporting. However the timeline must also be rendered either beforehand or as part of the export. Rendering the timeline can be CPU or GPU bound, whereas exporting (esp. if H264 or similar) is usually CPU bound. Neither are often I/O bound, so normally adding more I/O bandwidth won’t help.

You can inspect your system and verify this by looking at Activity Monitor or using iStat Menus. If the CPU is already mostly high during the render/export, adding more I/O bandwidth normally won’t help. This is because if it was already I/O bound, then CPU could not be high — it would already be waiting on I/O.

You can also check Activity Monitor for how many bytes per sec, I/O per sec and bytes per I/O. This will help determine if you are waiting on I/O during render/export. Only if you are already bottlenecked on that would increasing I/O bandwidth make the process speed up.

You might get some improvement by exporting to single-pass H264 if that meets your needs, or maybe by enabling background rendering so the timeline will already be rendered at export time.