[Joe Marler]

[Brett Sherman] “Any 150 mpbs 4K has to be inter-frame. I’m sure it has a slightly different H.264 implementation than XAVC 4K. But not that would explain the massive performance difference between the two formats.”

If you could point me to about 5 sec of that material I’d be happy to test it on FCPX and Premiere on both iMac Pro and 2017 iMac.

[Joe Marler]

Both FCPX and Premiere CC can be sluggish on many different 4k H264 codecs, even on fairly high-end hardware.

If you are on a 2013 Mac Pro which does not have Quick Sync or any other encode/decode acceleration, in general H264 editing or transcoding will be quite sluggish. It might even be slower than a 2015 MacBook Pro.

I haven’t tested C200 material, but all 8-bit XAVC 4k from the A6300, A6500, A7R2 and A7R3 is similar. Likewise 8-bit 4k H264 material from our Panasonic DVX-200 and GH5 is also similar, also from our DJI Inspire 2 drone. So it’s not unique to Sony or the XAVC-S variant of H264.

The fastest machine I’ve tested on 4k H264 editing response in FCPX is the 2017 i7 iMac. It is about 2x faster than the equivalent 2015 model, about 2x faster than a 12-core D700 Mac Pro. Those are rough approximations based on decoding and creating optimized media or encoding from ProRes to 4k H264 output.

Harder to measure is editing smoothness in the timeline. But the 2017 iMac is much smoother on 4k H264 than the 2015 or earlier iMac.

The 10-core Vega 64 iMac Pro is not dramatically faster on 4k H264 than the 2017 iMac in most cases and even slower in a few scenarios, although faster than the 2013 Mac Pro. This is apparently because Xeon doesn’t have Quick Sync and Apple is using AMD’s UVD/VCE hardware instead. It’s faster than software decoding but (at least in FCPX 10.4) it’s not as fast as Quick Sync. Hopefully this can be optimized in future versions: https://en.wikipedia.org/wiki/Unified_Video_Decoder

There are many different encoding parameters for H264: bit rate, frame rate, bit depth, chroma sampling, GOP size, GOP pattern, I or P frame interval, etc. This article shows how many variations there are: https://www.tiliam.com/Blog/2015/07/06/effective-use-long-gop-video-codecs

Some of these can be examined with tools like MediaInfo but it doesn’t reveal this for all variants. The XAVC-S material is all *inter*-frame; I don’t know about the C200’s 150 mbps internal 4k codec.

You can also have *intra* frame H264 encoding which is typically much quicker to decode. E.g, the 4k intra-frame 300 mbps material from a Canon XC15 is very smooth to edit. If the C200 codec is intra-frame that could explain why it’s so much smoother, but at 150 mbps I suspect it’s inter-frame just like Sony’s XAVC-S.

On my 2017 top-spec iMac, a single stream of XAVC-S 4k h264 is fairly quick in FCPX 10.3.4. By “quick” I mean 1x playback never drops frames, JKL lag is fairly low, fast forward and reverse playback is usable though not perfectly smooth. The 10-core Vega 64 iMac Pro running 10.4 is roughly the same, maybe slightly more laggy. On my 2015 top-spec iMac it’s much more laggy.

Multicam requires proxies or optimized media, even on a 2017 iMac or iMac Pro. This is for XAVC-S or most other 4k H264 interframe material I’ve tested. This isn’t unique to FCPX — Premiere CC is worse (on Macs) as it apparently doesn’t use Quick Sync acceleration. Supposedly starting in 2016 Premiere (on Windows) started using Quick Sync, five years after it was released with Sandy Bridge CPUs.

[Joe Marler]

[Patrick Jones] “iMac 27″ Late 2015 64gb RAM, 4 GHz Intel Core i7, AMD Radeon R9 M395X 4 GB – Internal Storage is the 3TB Fusion….I’m thinking about upgrading the Fusion to SSD and maybe changing out the Blade SSD to a 1TB. Not sure. Do you think this would speed things up to keep up with simple 100Mbps 4K, under 10min projects?”

A “simple” 100 mbps 4k project is H264, which is difficult for most hardware or software to handle with total smoothness. However it should not crash, hang or “quit”. The solution is not modifying your Fusion Drive, and I’m not sure that’s possible anyway. Our lead editor is using a 2013 i7 iMac 27 with a 3TB Fusion Drive and the FCPX library is 20 terabytes on a Thunderbolt array. It uses proxies but runs mostly OK on FCPX 10.3.4.

What version of FCPX and macOS are you running, is the media all on the internal drive, and how much free space exists on the internal drive? Are you editing the camera native files, or did you transcode to proxy or optimized media? If general you don’t need optimized media but proxies can help with performance and editing smoothness.

If you have background rendering enabled in FCPX preferences, that can sometimes cause problems so try turning that off, then delete all render files in case any are corrupt or damaged.

By “quitting and not rendering” do you mean FCPX crashes or hangs, or CTRL+R won’t render the timeline or it won’t export?

You can also try resetting preferences, as described in this article: https://support.apple.com/en-us/HT203477

[Joe Marler]

[Casey Longden] “I had in mind that transcoded footage gave you increased speed when using effects, retimes etc. and that you had more ‘room’ in the footage for grading although I can’t remember my original source for that info nor can I find any additional evidence (which is why I asked the question, I guess).”

ProRes (or optimized media) can give improved speed when using certain effects. However you may not need that. Your C100 Mk II footage is “only” 1080p H264, which is quick to edit on almost anything.

Re transcoding to ProRes giving more latitude in post, FCPX is always editing ProRes anyway, regardless of the source codec. When you create a project and pick “Use Custom Settings” or modify an existing project, you are given the option of selecting what *type* of ProRes, plus the option of Uncompressed 10-bit 4:2:2. But it’s always internally rendering to one of those, whether the source material is H264 or whatever. It is not using an H264 buffer or intermediate scratch file — if it were, the more edit passes you made the worse the image would become.

Thus I don’t see the argument that transcoding all source material to ProRes gives more color grading latitude. Maybe the people who say that are mixing up ProRes *acquisition* with transcoding in the NLE. On some cameras that can acquire in ProRes (whether internal or external) it may record greater bit depth, higher bit rate, or better chroma sampling. In those cases it could give better latitude in editing but that’s not because it was transcoded from H264 within the NLE.

[Joe Marler]

[Steve Guiles] ” I tried the deinterlace option, but it doesn’t do much. It changes slightly.
It blurs it just a little, but it still doesn’t look good. “

I think the XA10 if set to 60i produces interlaced output, and in “PF30” mode produces PsF (Progressive Segmented Frame):https://en.wikipedia.org/wiki/Progressive_segmented_frame

By default FCPX 10.3.4 will create an interlaced project if the first clip added is interlaced. I don’t remember how it handles PsF, since I haven’t shot that for five years. But in general for an interlaced project the rendered, encoded output will also be interlaced, and the assumption is (like NTSC TV or DVDs) that the playback device will deinterlace.

If you apply deinterlacing within FCPX to an interlaced project it does nothing because by definition it must be interlaced — at least that was the interpretation on 10.3.4. I think they changed the behavior on 10.4 so the deinterlace filter works on an interlaced project (at least for MXF) but I haven’t tested this extensively.

If you want it “hard deinterlaced”, you put that in a 1080p project, then apply deinterlacing in the FCPX inspector. Then when you render & export, the output will be deinterlaced.

If hard deinterlacing is improperly done, it’s then “baked in”. With some codecs if you put interlaced content in an FCPX progressive project, then *don’t* deinterlace, the encoded output may have every other line discarded (with some codecs). Normally Youtube will deinterlace upon upload, but you often see interlaced Youtube content which was apparently processed improperly before upload, which locked in interlacing artifacts.

When dealing with interlaced content it’s often difficult to trust what you are seeing. Different playback methods handle it various ways. Quicktime Player 10 automatically deinterlaces under some conditions based on the video header. VLC by default does not deinterlace but has various optional deinterlace modes. QT7 only deinterlaces if enabled in Window>Movie Properties>Video Track>Visual Settings.

I think the Viewer playback logic in FCPX is similar to QT Player 10, so you can’t always trust what you’re seeing. Sometimes it appears to visually deinterlace for viewing (regardless of the project) and other times it doesn’t.

For this reason it’s often good to verify the media file characteristics before import and after export using a tool like MediaInfo or Invisor. Invisor can display multiple files in a spreadsheet-like grid, side-by-side.

Since VLC allows explicit control over deinterlacing, that’s also an option but you are making a visual judgment not examining the video file header.

MediaInfo for Mac: https://mediaarea.net/en/MediaInfo/Download/Mac_OS
Invisor: https://itunes.apple.com/us/app/invisor-media-file-inspector/id442947586?mt=12

[Joe Marler]

Re Philip Bloom’s statement about smug FCPX and Resolve users who “never have issues”, I personally have never used any NLE which never crashed. I’ve had FCPX crash a lot, although it seems to crash *less* than Premiere.

NLEs have a difficult challenge. The combination of myriad code paths and codecs, stringent real-time latency windows, when combined with complex workloads form a murky multi-dimensional space where software load paths intersect with features, hardware and user actions.

Lurking in there are hidden “islands of instability”. A certain combination of user actions, hardware, codec and system state can cause repeated crashes. Another user doing almost the same action on almost the same hardware may never experience that.

From the standpoint of software development and testing, it is a grinding, tedious job to ferret out all those problem areas. Maybe Apple does a better job, or maybe they have an advantage because of fewer total test permutations via less hardware variation. OTOH it seems Avid (which also runs on Mac and Windows) has a better reputation for reliability.

It’s very difficult to objectively measure software reliability across products. However since Mac users can opt to send application crash data to Apple, this would include both Premiere and FCPX crashes. Adobe being only an app vendor wouldn’t have that data. I wonder if the Apple system people forwarded this data to the Pro Apps development team.

That said, in the book “Behind the Seen: How Walter Murch Edited Cold Mountain Using Apple’s Final Cut Pro”, by Charles Koppelman, Murch was asked was he scared of FCP crashing or losing data. He response was Avid crashed and corrupted data so much (at least back then) that FCP could not possibly be any worse, and he figured they could afford more workstations so at least a few would be up and running.

[Joe Marler]

[Richard Herd] ” it ain’t processing stuff to conform to what “optimized for apple hardware” means. Even though my computer is lame, surely, activity monitor should be redlined, like a tachometer on a street bike.”

“Optimized for Apple hardware” means it’s using the hardware efficiently, not squandering it. You can’t merely go by CPU utilization.

For example I just tested a 60 sec 4k H264 test clip in both FCPX 10.3.4 and Premiere CC 2017.1.2, both on the same 2017 iMac running macOS 10.12.6.

In Premiere, just scrubbing forward/backward on the timeline pegged all CPU cores, even though player resolution was only at 1/4. The viewer update rate was about once per second and playhead responsiveness to JKL commands was very sluggish.

On FCPX the viewer update rate when skimming the same timeline was about 10x faster, yet the CPU cores were quite low. Playhead responsiveness to JKL commands was much quicker. This is on the exact same hardware and operating system.

Exporting that clip to 4k H264 in Premiere CC took 2 min 9 sec, and the CPU cores were very high. Exporting the same clip to the same parameters in FCPX took 38 seconds (3.4x faster), yet the CPU cores were only about 50%.

Your screen caps show background rendering. In most cases FCPX does not need this and you can disable it in preferences. It is generally fast enough on most machines to edit most 1080p codecs natively. If you are editing 4k H264, almost no machine is fast enough to edit that natively with perfect smoothness, even on FCPX.

If you need smoother, faster editing performance on H264 or similar “difficult” codecs, it’s best to create proxies. Both FCPX and Premiere can do this. You can also transcode to ProRes optimized media, however this takes about 6x the space vs the camera-native H264.

[Joe Marler]

[Gabriel Spaulding] “Until recently I also kept all of my media and FCP X Libraries on external 7200rpm RAID drives. With my iMac Pro I keep media on external drives but move the Library (cache files inside) to the internal SSD and now audio waveforms draw significantly faster. Certainly the CPU and GPU play a role in drawing thumbnails and waveforms, but it seems to me that the storage itself makes the most noticeable impact. FCP X generates thumbnails and waveforms for different cameras at different speeds. “

FCPX I/O can be characterized as two very different profiles. One is typical large sequential I/Os for media. This is what benchmarks like Black Magic measure, and spinning RAID arrays are good at delivering that.

The other I/O profile is small random I/Os used for metadata management, in particular thumbnail generation, waveforms, info.plist files, etc. I assume the SQLite calls FCPX makes for database management also generate lots of small random I/Os. Unfortunately RAID arrays are not good at that. SSD drives are better, but even they have limits on random I/O per second (vs MB/sec) rates.

Here is an I/O histogram I generated using the terminal dtrace utility bitesize.d when FCPX was scrolling through a library with AVCHD .MTS files. There are lots of small I/Os: https://joema.smugmug.com/Computers/FCPX-Event-Browser-Perf-Data/n-M7bG7L/i-bCH2XFX/A

Which files it does I/O to can be inspected with the dtrace command iosnoop.

Regardless of I/O size or rate, a key item is whether the I/O system is overloaded. This can be examined with the dtrace command iopending which produces a histogram of how many async I/Os are backed up waiting to be serviced.

Using these commands is more difficult than older versions of macOS because of security restrictions. Starting with El Capitan you have to first disable System Integrity Protection. I wouldn’t recommend anybody use these unless they are quite familiar with terminal: https://apple.stackexchange.com/questions/208762/now-that-el-capitan-is-rootless-is-there-any-way-to-get-dtrace-working

How much of what I/O type varies based on what FCPX is doing, and also the codec. Scrolling through a large library with the Event Browser in filmstrip view incurs a lot more small I/Os than in list view. These thumbnails are then cached and subsequent scrolling is a lot faster. However if you resize the thumbnails there’s a threshold whereby they must be regenerated. Unfortunately there’s no manual control over this, such as in Lightroom where you can say “generate previews” and they are persistent.

As you said there are cases where putting the library and cache files on an SSD (even the system SSD) can help performance. However it can be difficult to determine whether this helps. Just because your spinning RAID array is chugging loudly doesn’t mean it’s overloaded. Lots of people speculatively put items on SSD, yet it may not help performance if the workflow isn’t I/O-limited. If you measure the before/after timing of a specific workload and it’s faster with library on an SSD, then that’s good evidence but methodically doing such things is time consuming.

I usually use a 4-drive spinning RAID-0 array and often put both library and media there. Sometimes if I have space and it’s a “lean” library I’ll put it on the system SSD or another external Thunderbolt SSD. If you use RAID-5 there’s a write penalty so it might be more important to put scratch/library files elsewhere in that case. However SoftRAID is very good at optimizing writes on RAID-5, so if you use that the penalty is often less than you expect.

[Joe Marler]

[Chad Greene] “I have three editors cutting different sections of a documentary and need to hand projects and compound clips back and forth. We have been using temporary Libraries to “transfer” these to one another, but are wondering about XMLs. “

You can do that. However if the media filename paths differ, relink may be required. If you are using proxies, relink can be problematic, especially if the disk volume name differs between systems.

I have seen cases on large libraries where loading a project XML may create spurious duplicate clips. This is on 10.3.4, I haven’t tested it on 10.4 yet. It’s not consistently reproducible but I’ve seen it a lot.

There is another issue if your original filenames are not globally unique, e.g, two files with the same name in different folders. In that case loading an event XML will also create spurious duplicate clips. This one is 100% consistent even on small libraries, at least on 10.3.4. The workaround is rename all files before import so they are globally unique across the entire documentary.

In general I suggest using a lean library and duplicating it in Finder as a backup before loading any XML, then examine the library for obvious problems before committing major work. Otherwise you can get stuck in a “data fork” situation where you’ve committed lots of work and only later find the XML did something undesirable to the event. Of course if using proxies they must be outside the library, else it’s too large to easily duplicate. Using external proxies entails another set of complications.

The MergeX utility allows combining event XMLs while giving priority to various metadata from one side or the other. However it does not work for projects, and to merge an event you must have all your projects removed to another event:

https://www.merge.software/

[Joe Marler]

[Sean Hynes] “…refurbished 2011 21.5 inch iMac 2.8 GHz Intel Core i7…Final Cut Pro X…10.4…external 1tb usb drive connected to the back of the mac…store the files for the video library itself…gets imported into the library stored on this 1tb usb drive. I have events with a few hours worth of footage in each (no more than 8 hours in either case), and the drive is only about half full….on various occasions e.g when simply selecting raw clips in the events page, or pressing the space bar to play etc, the program often comes to a halt with the spinning beach ball of death. Sometimes for a few seconds, sometimes indefinitely, causing a “not responding” status to appear on the force quit applications window….”

The 2011 iMac 21.5″ had only USB 2.0 ports. Even if the portable drive is USB 3, you are using it at USB 2.0 rates which is very slow. If your media is 4k H264 this entails a major CPU burden, but a 2011 “Sandy Bridge” CPU should be able to handle that if using proxies. If it is 1080p H264 it should handle it no problem. If the media is optimized that greatly increases the I/O burden and with USB 2.0 you don’t have much I/O bandwidth.

If eight hours of video roughly fills half a 1TB drive, this could imply a bit rate of about 100 megabit/sec which is generally 4K H264. Let us know what camera and codec, including resolution, frame rate and bit rate, and whether you are using original media, proxies or optimized media.

That said, it should not lock up in a perpetual beach ball or become non responsive no matter how slow the machine is — it should just be slow.

Is there any possibility the external drive is not HFS+ but is formatted ExFAT or FAT-32? Any active FCPX media should only be on an HFS+ volume. I don’t remember the specific problems if you put a library on a non-HFS volume, but that should not be done.

I’d suggest you run Apple Hardware Test as a 1st step, also run Disk Utility First Aid on all drives. Even if the hardware test passes this is not a guaranteed clean bill of health, but it’s an easy to run test: https://support.apple.com/en-us/HT201257

It would also be useful if you ran Black Magic disk test on all volumes, especially the external USB portable drive, and make sure it is formatted HFS+: https://itunes.apple.com/us/app/blackmagic-disk-speed-test/id425264550?mt=12

Even if your media drive is only half full, if the system drive is nearly full you could be running out of paging file space. That will often cause unpredictable behavior, so make sure you have at least 20% free space on the boot drive.