What is making my project file so large?

[Bill Davis]

Mark,

Do I understand you’re applying effects to the RAW full length clips rather than to just the RANGES you want to use out of them?

This seems like a possible point of inefficiency. For example, If I’d shot 30 minutes of footage, but only was going to use 5 minutes of it, it would seem much more efficient to apply the computation heavy effects only to the ranges of the clips that I was intending to use.

I’m not sure this is your problem – but I’m suspicious of applying calculation intensive effects to large swathes of stuff that I’ll never actually want to use. It seems like a lot of unnecessary calculation and “under the hood” references to a whole bunch of frames for no purpose.

YMMV, just thinking out loud here.

“Before speaking out ask yourself whether your words are true, whether they are respectful and whether they are needed in our civil discussions.”-Justice O’Connor

[Mark Morache]

Bill… I’m only affecting the audio of the master clips. I adjusted the level of two of the audio channels, and I applied the “loudness” enhancement to the other two, basically adding an audio compressor.

FCPX boasts about its ability to analyze and fix things. I can’t imagine it would be a big problem to apply these to the clip in the event browser.

I wonder if my problem came because I opened the clips in timeline view and applied the effect to the clip “underneath the hood” so to speak.

I wonder how FCX handles this.

Follow this and see if you think this might be what happens: When I import my clip, it treats the media just like any other clip that references the media on your hard drive. When I right-click the clip in the event browser, open it up in the timeline view, and modify the audio inside, then perhaps it starts considering the clip a compound clip instead of the original clip.

In FCX, we can open up a clip in timeline view, and actually make edits inside the clip. Now it’s no longer a clip, but it acts like a compound clip.

This is different from applying a level adjustment or an audio enhancement or even a color correction to the “outside” of the clip in the event browser.

I’ll do an experiment sometime, and try this. Meanwhile I need to get my editing done, so I painstakingly made notes for my three dozen edits, and imported my two clips fresh, and recreated my timeline.

My project file has gone from 90mb, down to 5. This should edit much more quickly!

I think perhaps the warning might be that we shouldn’t take a clip in the event browser, open it in timeline view and modify it inside. If anyone else can verify this, I’d be interested in hearing from you.

———
FCX. She tempts me, abuses me, beats me up, makes me feel worthless, then in the end she comes around, helps me get my work done, gives me hope and I can’t stop thinking about her.

Mark Morache
Avid/Xpri/FCP7/FCX
Evening Magazine,Seattle, WA
https://fcpx.wordpress.com

[Rob Mackintosh]

My guess is it’s the markers. Each instance of your two clips in the timeline will contain a reference to the original markers. This seems to bloat the FCPX project file and slow things down. See my experience here: https://forums.creativecow.net/readpost/344/5732

If you still have the original project file export XML, open in a text editor and see how many marker references there are.

[Mark Morache]

Rob…

Good detective work.

I only added five markers to each of my two clips, so even if each clip in my timeline had references to those 5 markers. A quick look at the xml reveals about 268 marker references in my timeline. Every time I cut a clip into two clips, it would have added 5 more markers. However, splitting a clip into two made the beachball come out for 10-15 seconds.

It’s interesting that my 98mb project created a 106kb xml file, and my 5mb project created a 45kb xml file. Hardly proportional.

I had also applied five different keywords to five non-overlapping sections of each clip. These are in the xml several times as well.

I didn’t read the entire thread you linked to, but you had placed a clip inside a compound clip and edited with that… I still wonder if that’s what’s making the project get so large. I suspect that by modifying my clip in timeline mode, FCX starts treating it like a compound clip, turning every clip in my timeline into a nested sequence, and making the project size abnormally huge.

I’d sure love to hear from Apple.

———
FCX. She tempts me, abuses me, beats me up, makes me feel worthless, then in the end she comes around, helps me get my work done, gives me hope and I can’t stop thinking about her.

Mark Morache
Avid/Xpri/FCP7/FCX
Evening Magazine,Seattle, WA
https://fcpx.wordpress.com

[Rob Mackintosh]

Mark, I struggled to read through my original post, perhaps a little too much detail!

In my experience compound clips within projects are problematic, whether created in the timeline or within an event. I agree, by opening and editing your event clips in their own timeline FCPX is treating them like compound clips.

T.Payton has this clip on You Tube illustrating how compound clips grow project size and bog FCPX down:

https://www.youtube.com/watch?v=bRIeTZ-RNzc

[Mark Morache]

Thanks for the link.

I guess it makes some sense, that every iteration of a clip or compound clip is a separate entity, that can be modified separately.

This is perhaps a big difference between compound clips and nested clips from FCP7. In legacy Final Cut Pro, when you nested a sequence and used it more than once, you could modify the original sequence and the changes you make ripple down to all the clips already in the timeline.

That’s not the case here. Every time you cut a compound clip in to, it basically gives you a brand new version, and if you change one, the other does NOT follow.

I suppose there is a usefulness for something like that, but it seems to me that every time you cut a compound clip, both halves should refer to the same original clip, not a separate clone of the clip.

Again, in FCP7 if I want to duplicated a nested clip and make changes to the duplicate, I can do that. I don’t seem to have that sort of control in FCX.

Apple may need to rethink the compound clip paradigm.

For me, I want to see the compound clip show up in the event broswer, and make it so there is only one iteration of the compound clip that is referred to in the timeline, rather than cloning the entire thing each time you use it.

Right now, I mainly use compound clips either to keep my timeline clean by collapsing a large stack of clips, or I use it to combine a stacked effect if I want to apply a transition or clip effect on the entire bunch.

I really like X, but it has a long way to go.

———
FCX. She tempts me, abuses me, beats me up, makes me feel worthless, then in the end she comes around, helps me get my work done, gives me hope and I can’t stop thinking about her.

Mark Morache
Avid/Xpri/FCP7/FCX
Evening Magazine,Seattle, WA
https://fcpx.wordpress.com

[T. Payton]

The way the actual project files are stored in FCP X is indeed a big problem. I’ve done many more tests and looked at the SQLLite files (that is what FCP X project and events are stored in) and frankly it is very inefficient and redundant. It almost looks like there is debugging code in there.

The Project files store much more information than the FCPXML files, hence the larger sizes, but it’s not THAT much more information that — there is some extra bloat in there too.

The underlying technology of CoreData is a good one, and built upon some really cool things like unlimited undo and constantly saving, but it is just not optimized in FCP X at the moment.

I believe Apple is going to get this fixed–well they have to–it is completely foundational to everything in FCP X. It effects timeline edits, compound clips, keyword ranges and markers — which is what FCP X is.

My current advice is to collapse large compound clips (by using break apart), and not storing complex sequences in events as compound clips.

——
T. Payton
OneCreative, Albuquerque

[Rob Mackintosh]

Thanks for doing more tests T.

Whatever the technical implementation, which at the moment is poor, conceptually FCPX works differently to what you would expect:

– Every project points to the source media, not just “clips” in an Event. Perhaps this is necessary to communicate with other software. It suggests it would be trivial to implement a reconnect media function.

– All the metadata applied to a clip within an Event appears to be duplicated in the Project.

– You can reconstruct an Event, keywords and all, with just the project file (limited to the media contained within the project).

Why not leave the metadata (and source file info) in the Event and only hook into it when exporting a project or XML?

[T. Payton]

Rob – great points.

I’m purely speculating, but from the looks of it even a very complex edit with many nesting compound clip with lots of markers the project file doesn’t need to be that large.

I have a feature length project, about 2500 edits, and while the project file is 96MB, the FCPXML is just 2 MB. What in the world is in the other 94MB in the project file?

I think it is interesting now that having used FCP X for a while, I appreciate the incredible simplicity that they have introduced in the App. I was thinking about locking tracks and track targets the other day and the hoops I had to jump through to get a few simple edits done. It is funny now, that I realized many of the objections I had initially with FCP X are really areas where I wanted to be able to jump through the hoops that I had to jump through in FCP 7.

The entire main storyline always in ripple mode is brilliant! Trackless editing is brilliant. The fact that at a glance we can see the audio level of any clip is amazing. The “throw any format on the timeline and it will work” is incredible. The Proxy workflow is amazing. I could go on.

It’s very much akin to when Apple recreated the smart phone and removing all the buttons in the iPhone except one — they simplified things to the point of brilliance. I tried to help someone with their Nokia phone the other day, and was amazed at how difficult it was to use with all those buttons.

However…. the problem is not with the radicalness of FCP X concept, but the execution. I know this isn’t the case but I picture in my mind the FCP X team made up of just a handful of people that are completely overwhelmed and unable to get anything but the most basic fixes done. The bug list has to be huge! I have been keeping a record of the bugs, and I have sent over 50 to Apple. 50 bugs that just one user has seen on a shipping product! That is crazy! In all my years beta testing I have never been able to report 50 bugs on a beta. This is just not a thoroughly executed program on the programming level.

Hopefully Apple will be creating a professional beta program so that actual users will be giving feedback to the product. If Apple would just devote the resources… and they have the resources… then FCP X would be executed wonderfully just as it was designed to be.

——
T. Payton
OneCreative, Albuquerque