[Joe Marler]

[chris walker] “….the original footage is 4k but the timeline is 1080p. I’m exporting to prores 1080p…so when I reduce it down to sd in a separate step to make a dvd, the downscaling looks better. Ive used this workflow a lot with 1080p original footage, and if I export to a 100gb prores file it takes up to several hours. I figured that with 4k footage reduced down on a 108p timeline, it might take several hours more because the 4k has to be transcoded first, but instead its taking much longer than that. Now Im 60% done after 18 hours…If it is simply that with my relatively old computer and small amount of memory it is typical for it to take this long, I feel like Ive been mislead…I’m wondering what workarounds there are….I really dont want to get a new computer if i dont have to..”

You’re saying it takes much longer to export 1080p from a 4k file than from a 1080p file. That is correct — just because the timeline is 1080p doesn’t alter this. The timeline is only for intermediate renders. When you need to generate the final output it must go back and process the original 4k file. If it did not do that, you could not zoom in on content using the original 4k resolution.

On my 2015 iMac 27 it takes roughly 2.4x longer to export 1080p ProRes from a 4k H264 source than a 1080p source. This should not be surprising — it is 4x the data. If it were 4k output it would be about 4x longer. Exporting to 1080p saves some time, but taking 2.4x longer is expected. This is not unique to FCPX. Premiere Pro CC 2017 running on the same hardware takes 2.3x longer to export to 1080p H264 from a 4k H264 file than from a 1080p file, plus the overall time is 3.8x slower than FCPX.

On my iMac it was a bit faster to export from an H264 source to H264 output than ProRes. You could test this on a small file and see if it applies to yours. Are you certain that a 1080p ProRes file will look better than H264 if downscaled to DVD? Did you actually test this and inspect it, or are you just assuming it might be better? If you test it and can’t tell any difference, you could save some time that way.

Handling large amounts of 4k H264 is one of the most difficult things a computer can do — regardless of what editing software. It can be challenging on a top-spec 2015 iMac 27 or even a Mac Pro. Proxy enables the edit phase to work faster, but when you do the final export, that original 4k content must be processed — if you want the benefit of 4k. If you don’t want that benefit you could shoot in 1080p or externally transcode the 4k to 1080p before importing it.

[Joe Marler]

[chris walker] “fcpx 10.2.1, 8 gigs of ram, 2011 imac, osx 10.10.5…80 minute 1080p timeline using 4k footage from panasonic g85 and gx85. exporting to prores rather than h264…i have done this many times before, using hd footage rather than 4k, and the speed was always reasonable. but this time its extremely slow; its at 2% after 20 minutes…should it be this slow, just because its 4k? “

FCPX won’t export 4k from a 1080p timeline, so I don’t understand that part. However your performance seems very slow. OTOH 4k is 4x the data of 1080p, so if by “always reasonable” before, you mean it was 4x faster when exporting to 1080p, that would make sense. I did several timed export tests of 4k Panasonic H264 content using a top-spec 2015 iMac 27 and FCPX 10.3.2. I then compared this to export time of 1080p H264 material from a 5D Mark III and lastly the export time of H264 4k from Premiere Pro CC 2017.0.2.

The only Panasonic camera I have lots of footage from is the DVX200. I did the following export tests using 30 minutes of 100 mbps UHD 4k H264 material, so you’ll need to multiply by 2.66x to equal your 80 min timeline. Note there are three possible resolutions in play (1) resolution of original material (2) resolution of timeline (3) resolution of export.

Export 30 min. 4k H264 from 4k timeline to 4k H264: 21:00
Export 30 min. 4k H264 from 4k timeline to 4k ProRes 422: 21:00
Export 30 min. 4k H264 from 1080p timeline to 1080p H264: 9:18
Export 30 min. 4k H264 from 1080p timeline to 1080p ProRes 422: 12:14 (repeated twice to check)

1080p Export Tests from Canon 5D Mark III using 30 mbps IPB H264 codec:

Export 30 min. 1080p H264 from 1080p timeline to 1080p H264: 5:39
Export 30 min. 1080p H264 from 1080p timeline to 10800 ProRes 422: 5:12

Export test from Premiere Pro CC 2017:

Export 10 min. 4k H264 from 4k timeline to 4k H264 at same output bitrate as FCPX: 26:47 (Premiere is 3.8x slower than FCPX on this test)

[Joe Marler]

[John Rofrano] “…most memory cards use the FAT32 filesystem which is limited to 4GB as the largest file size. Depending on the format that you are shooting, this could be as little as 5 or 10 minutes of video…”

BTW, all SDXC cards 64GB and larger are automatically formatted exFAT by the camera itself — it’s part of the SDXC spec. With these there is no file size limit. This is increasingly the standard, especially as 4k becomes dominant. Almost all the cards in my doc team are 64GB or larger, whether in a camera, drone, GoPro, etc.

[John Rofrano] “..Consumers are more likely to be recording a school play or other event where they are recording for a long time. When these broken up files are just dropped onto a timeline without any stitching, there is often video frames or audio missing. The metadata on the cards give instruction on where one file leaves off and the other begins allowing for a seamless concatenation of the two.”

For this very reason, the way FCPX handles “leave files in place” often creates a problem. It appears programmatically possible to distinguish between media on an SD-type card vs that same folder tree on a hard drive. You can even do it from terminal. Apple could do this within FCPX and only disallow in-place imports from actual SD-type cards, not from hard drive copies of the card. They could also provide an override or warning dialog which allows the user to make the decision. Or they could do both. They do neither, which encourages users to copy bare video files outside the folder tree and do in-place import from there. This in turn creates the possibility of missing metadata — which is missing because users were forced to copy the video files out of the tree to get in-place import to work. Basically, FCPX behavior is encouraging users to do the wrong thing.

It is especially bad for AVCHD files which seems to cause I/O performance problems if imported in place from outside the original folder tree. It appears FCPX may be dynamically re-wrapping each AVCHD file upon each reference — it performs a huge number of small I/Os when using the Event Browser in filmstrip mode. It even happens if only a tiny % of the media in the library is AVCHD imported in place. This behavior doesn’t happen for XAVC-S or other files I’ve tested. However as currently designed FCPX doesn’t handle AVCHD content properly for the in-place import case, which can “poison” the performance of a large library. Until this I/O situation is fixed, in-place import of AVCHD material probably shouldn’t be allowed — whether inside or outside the original tree structure. It works fine on Premiere.

Copying media files outside the folder tree is not generally a good practice, even when it seems to *currently* cause no problems. You never know when missing metadata will come back to bite you, but FCPX as currently designed actually encourages this poor practice. These behaviors are not documented by Apple, and I haven’t seem them described in any FCPX tutorial or book, and I have most of them. It’s no wonder users get confused.

[Joe Marler]

[Andrew Kimery] “For a lot (I dare say most) of the people that need to edit on a somewhat regular basis a ‘normal’ desktop or laptop is adequate.”

This was true before widespread use of 4k cameras. The nMP was released in 2013, and probably some design work done in 2012. In 2012 few people envisioned that by 2017, 4k acquisition would be this widespread. Back then people thought the limited 4k distribution and playback infrastructure would diminish the need for 4k acquisition and editing, leading to slow adoption.

What happened is 4k has become the “new color”. Content producers with no immediate plans for 4k distribution still shoot 4k to improve shelf life, plus editors like the compositional flexibility. Today, inexpensive drones, GoPros, even cell phones are shooting H264 4k. This happened much faster than most people anticipated.

This has greatly impacted editing. Adobe’s Mercury Playback Engine that was like quicksilver on 1080p is sluggish on 4k. Even FCPX on the highest-end iMac can struggle. Compute-intensive effects on 4k material are maddeningly slow, and they can be laggy even on a nMP.

The near term answer is use proxy. That’s fine but it knocks us back to the pre-Mercury era when everything must be transcoded before editing. Editing camera-native content with no transcoding was pretty nice. As shooting ratios skyrocket, we need camera native editing more than ever. Yet we are increasingly knocked back to transcoding due to 4k performance issues.

I’ve seen countless cases where recreational editors are mystified at why their computer became so slow at video editing — “it’s only a GoPro/iPhone/DJI clip”, they say.

If we want similar editing performance (without transcoding) on H264 4k as we had on 1080p, this takes a lot more hardware muscle on both CPU and GPU sides.

If Apple knew this was going to happen back in 2012, and if they weren’t going to release an updated nMP until 2017, they might have designed the nMP differently with more upgrade options. If people can’t get the performance they need on a Mac platform, they’ll just use Windows. Whenever the updated nMP and iMac are released, a lot of people will buy them because this time the video editing workload as truly changed and the additional performance is needed. This assumes they haven’t already given up and moved to Windows.

[Joe Marler]

[Jordan Sarkisian] “…exporting a few different projects and different ways from FCPX to test my new laptop (maxed out MBPTB except SSD size) and the exports are giving me very different results. Some projects will use all the CPU cores….and other times when exporting it’ll use barely any CPU at all…For $2700 I feel like I should be using at least all 8 cores even if it’s low on each right? “

Exporting consists of two tasks (1) The timeline must be fully rendered, and (2) The rendered timeline must be encoded to the designated export codec.

If you are encoding to H264 this cannot be accelerated by traditional GPU methods since the core algorithm is inherently sequential. Each frame in a GOP (Group Of Pictures) must be encoded before the next. Each GOP can be processed in parallel but there aren’t enough GOPs in a typical file to effectively harness the thousands of lightweight threads a GPU offers. Thus a multi-core CPU is typically used for this, one core per GOP, then walking down the file.

The only exception to this is speeding up the sequential encoding thread by using dedicated hardware. That is what Quick Sync does — it’s essentially dedicated hardware logic which processes the innermost encoding algorithm faster. Although logically separate from the GPU, Quick Sync requires resources from the on-chip GPU so it cannot exist independently as currently designed. If Quick Sync is used during export, the CPU core utilization may be lower since much of the work is being done by Quick Sync. That is not bad — it’s good. You can typically export 4x or 5x faster to H264 using Quick Sync, yet the CPU will be lower. Your goal is do the job faster not have the highest CPU numbers.

Why your different results? If the timeline contains unrendered effects, these must be rendered before being encoded. Depending on the effect that could require CPU, GPU or both. It will be highly variable depending on your timeline.

Another variation is whether you have background rendering enabled. If enabled (which is the default), FCPX will have done varying amounts of background rendering. If the timeline has been fully rendered and you export to H264 using Quick Sync, you might see lower CPU utilization. If the timeline has not been rendered and requires CPU to achieve this, you might see higher CPU levels during export.

Your CPU doesn’t have 8 physical cores. It is hyperthreaded and has 4 physical cores and 8 logical cores. While this can help many CPU-intensive workloads, it’s also possible to encounter “cache thrashing” in the CPU instruction/data cache. It appears the macOS thread scheduler tries to avoid this and sometimes will only schedule 4 cores not 8, or some combination in between. I don’t know what criteria it uses.

If you want to see high CPU numbers, you can export from Premiere — your CPUs will be maxed out and it will export at 1/4 the speed.

[Joe Marler]

[Lisa Defelice] “It is actually a Grass Valley AVI . The client sent me a ” HQX White Paper” “

I see another user reporting that as of FCPX 10.3.x, it seems HQX doesn’t work properly: https://forums.creativecow.net/thread/344/46890

If you import into FCPX then export as ProRes 422, then re-import that, will it play in Quicktime player OK? If so try to import it in that format.

If nobody responds with specific experience on tested conversion paths from HQX to FCPX 10.3.x, this will be a trial-and-error situation. The question is what transcode utility and path will convert the HQX .AVI file to the highest quality format FCPX will accept and not yield any degradation or behavioral quirks.

You are correct EditReady will not work in that case. Supposedly ffmpeg will convert HQX to other formats but I’ve never tried this: https://ffmpeg.org/

I see mixed answers as whether Handbrake will handle the HQX codec. If it will that is a free solution: https://handbrake.fr/

If this is a one-time thing and you still have access to Premiere 5.5, another option would be importing it then exporting it to a format FCPX will accept. I think CS 5.5 can export in Quicktime, but I don’t know what the constraints are on bit rate and resolution. If nothing else H264 at the highest possible bit rate would work but you normally don’t want to use that as an intermediate codec.

[Joe Marler]

[Lisa Defelice] “I am new to the Cow and FCPX. I did my first importing of file and transcoding to Pro Res. But it failed and gave me a ProRes green playback screen with sound? What did I do wrong? “

In general you don’t need to transcode to ProRes on import. Just like Premiere, FCPX can edit camera native files with good performance. If the files are already copied to hard disk, you can import with the “leave files in place” option and do not select either proxy or optimized media transcoding. The import will be very fast.

Exceptions to this are cameras which have a hierarchical folder structure, and you copy the entire camera structure to hard disk. In that case FCPX will not allow import with “leave files in place” but will rewrap the files and copy them to the library. In most cases you can avoid this by copying the video files outside the folder structure. However in the case of AVCHD you should not do this because it may create performance problems after import. For AVCHD, IMO it’s better to (a) not use it in the first place or (b) externally rewrap before import using the 3rd party tool EditReady: https://www.divergentmedia.com/editready

If you are editing multicam, 4k H264, or multicam 4k it may be necessary to transcode to proxy for better editing performance. This can be done during import or afterwards. After transcoding the viewer must be set to proxy. Before the final export, set the viewer back to optimized/original, else the export will be in proxy resolution.

Separately, it’s a good idea to install Apple Pro Video Formats 2.0.5 which supports some newer professional codecs: https://support.apple.com/kb/DL1898?locale=en_US

[Joe Marler]

[Myreille Abaya] “Playback is good for the most part, when I playback a drone footage in 4K, which plays back choppy. I have a 2011 Macbook Pro.”

4k H264 content is difficult to edit using almost any software on most computers, and especially on a 2011 Macbook Pro. It is 4x the data of 1080p, so in theory you’d need a computer 4x as fast to provide the same performance.

If you are using FCPX, there’s a built-in facility to transcode that to proxy media — either during import or afterwards. That will provide much better editing performance. You must remember to set the viewer to proxy, then before your final export set it back to optimized/original or it will export in proxy resolution.

Premiere Pro now has proxy capability, but I assume you are using FCPX since this is a FCPX forum.

[Joe Marler]

The 1TB HGST Touro S (about $79) is a 7200 rpm USB 3.0 bus-powered drive. It is one of the faster drives in this range: https://a.co/id7p0Wm

Roughly double that speed is the 4TB Seagate Backup Plus Fast (about $199). It is internally RAID-0 so is faster but this theoretically impacts reliability: https://a.co/e1g0aea

By far the fastest would be an SSD portable drive. The Samsung T3 is available at several price points and capacities: https://a.co/8uXsh92

[Joe Marler]

[Noah Kadner] “Tell them to forget everything they know.”

There is truth to this. While the higher artistic layers of editing don’t change, how to optimally implement this using FCPX is very different from a track-based editor. I hate to be negative but IMO there is a significant deficiency in the available tutorial content. Some of it is high quality and slickly produced but it focuses on the lower-level aspects — what buttons to push, not the new paradigm itself. There’s lots of info about the trees but not much about the forest.

It would be nice if there was a dedicated tutorial which focused on conceptual-level paradigm issues and the workflow implications uniquely raised by FCPX. It could include the common pitfalls for an editor transitioning from a track-based product and explain how to do each one the “FCPX way”. It should also include the underlying philosophy and explanation for current FCPX behavior. IOW rather than say “do it THIS way not THAT way”, explain the design rationale and why FCPX behaves as it does in each specific case.

I don’t know why Apple has not even provided a white paper on this. When they popularized the GUI they were not shy about formally articulating the philosophical underpinnings of this and why it was advantageous. There is nothing like that for FCPX. There is almost a subtle mentality that if you’re one of the “cool kids” you don’t need this explained — that people will quietly discover it by themselves. Yet the lack of formally stating these guiding principles may explain why six years after the release people are trying to patch tracks and wonder why they can’t have several timelines open to organize their content. There is no focused comprehensive treatment of the conceptual-layer issues that transitioning editors commonly struggle with.