Large projects

[Jeremy Garchow]

Also, I noticed you’re working with a 22k mp3.

Does the type of media matter?

[Andrew Richards]

Reading the experiences here and via your link my hunch is the weakness is in the storage hosting the project and event databases. IOPS are critical to snappy database activity, and I suspect most editors are storing projects like media, on storage that is very big and can stream big bitrate data. That same storage, likely HDDs (even if it is RAID) will not be particularly good at serving database queries against large databases.

If anyone has an SSD in their rig, testing a large project or event on the SSD vs an HDD might yield a considerable performance gap in favor of the SSD.

Best,
Andy

[T. Payton]

Well I didn’t mean to say too many negative things about FCP X. 😉 I really do like the program, in fact I think it is brilliant concept, but a poor execution at the moment.

So pardon me for throwing Apple under the bus, but in my view Apple has had a bad track record of creating apps from scratch on the Mac. The original iPhoto, Garageband, Pages and Numbers were painfully slow. iMovie (the new one that is like FCP X) was and is very slow. I encouraged some friends to get an iMac (an i5) last spring to do some student video work (actually very complex) and they came back and said “Why is it that every time we do something in iMovie it has that spinning ball?”.

Final Cut Pro was not like that, and it wasn’t begun at Apple. It was speedy from day 1. But the home grown apps like Live Type, Motion, and even the beloved SoundTrack Pro were slow and awkward — especially if you compared them to other apps like After Effects and Pro Tools on my same hardware.

Now the good news is that many of these Apps Apple has fixed. Pages for example. At my advertising shop my fellow designers LOVE Pages. It can do about 90% of what we used to do in QuarkXpress back in the 90’s. SoundTrack Pro got much better and Motion has matured too.

I know Apple can make FCP X work if they want to. I know this because I use GarageBand on my iPhone. yeah my iPhone. It is amazing! I’m recording song ideas in multitrack on my iPhone that I did 8 years ago in ProTools. The program is very responsive, and it just works. Every little detail has been well thought out, and when I use it on my first gen iPad it is even better.

So what is up with Apple? This is a company who are able to make Garageband work on a tiny device with a tiny amount of ram. If they put their mind to it they can do almost anything. Now I don’t know what happened with FCP X development, but something went horribly wrong. As we all know the launch was very un-Apple like, and a branding disaster; Where were the endorsements from people like AJA, Blackmagic, Broadcast stations, famous film editors? So the features changed, but as those of us who have stuck with the App for a while know, the lack of features are not really a problem. In fact I continue to be amazed at the brilliance and simplicity that went into the design of FCP X.

The design on paper is brilliant, but the execution fell short. In my book the concept gets and A, execution and attention to detail gets a D. (No keyboard commands in the precision editor? Can play back nearly anything in real time at whatever resolution, but too many edits on the timeline and it stalls for several seconds when trimming a clip?)

It’s kinda funny because in my business, advertising, we see the opposite true most of the time. Horrible concepts but unbelievable slick executions. It’s lipstick on a pig. But in the case of FCP X, its ragged clothes on a sawn.

So I have great respect for Apple, and the programmers. But they have shown that they are human, like the rest of us, and have blown it. I’m willing to give them the benefit of the doubt, as I have been given the benefit of the doubt many a time. Plus FCP 7 still works well if I need it.

My specs:
MacPro, 2006, 2.66 GHZ (4 core)
14GB RAM
Radeon 5770, 1GB
eSata RAID (130MB/sec)
10.6.8

P.S. While my machine is no spring chicken, the problems I am giving as examples effect any machine.

——
T. Payton
OneCreative, Albuquerque

[T. Payton]

I apologize for my previous post was so long–the effects of posting tool late at night.

Just did a quick test that showcases my concern about the project file inefficiency:

Took my 1 hour 15 minute timeline, about 2000 edits.
FCP X Project File: 126mb
Exported FCPXML. 1.2 MB

Imported 1.2 MB FCPXML on my MacBook Pro:
Took about 5 minutes
New Project File: 43MB

Therefore FCP X Created 41.5 MB of data when importing the FCPXML. There was no additional “real” project information because no media was available, as it was on a completely different machine. i.e. it couldn’t create new render files, waveform cache or thumbnails. This is needless bloat. BTW. Maneuvering in the imported timeline with no media to access is about the same speed it is on my MacPro that has all the media available – very slow.

For comparison, via Automatic Duck I moved the timeline to FCP 7.

FCPX Project File: 43MB
AAF Export from Automatic Duck: 13 MB (link to existing media)
FCP 7 project file: 5.3 MB.

Concerning the Database Records, although I’m sure speed would help, I would conclude that it isn’t SSD’s or a Thunderbolt RAID that is needed, but instead a serious look at the database structure. Concerning the database, I have several large mySQL databases that run web sites on my web server. The hardware is 2.1 GHZ, Dual Core, 4 MB of Ram, and a regular old 200 GB sata internal drive. Typically the database files are small, about 2mb. But I have a very large site with a 67MB database, 168,683 records. Does the size of the database effect the speed? Not that I can tell at all. Typically my queries to return a hundred or so records are less than .5 seconds. Adding a record to the database just took 0.2085 seconds. Just as they are on a site with a small 2MB database.

Now I don’t know alot about SQLLite (which as far as I can tell is the structure of the FCP X project and event files) but I did a quick Google and found this: (it may or may not be relevant) Basically it says “As you can see, most operations are slower on SQLite3 and “write” operations (create/update/delete) are really bad.”

https://zenadmin.org/en/blog/post720.html

So there are two issues here. 1) project databases has unbelievable bloat. 40 units of bloat for every 1 unit of real data. 2) database access is incredibly slow, which seems to get worse with size.

I would conclude from this that the database format that we currently see in FCP X is horribly flawed. If I were a project manager, I would get as many programmers as possible and set this as the top priority. Either scrap it completely or do a rewrite. Although a complete rewrite sounds like a big deal, and I can’t dream of working on a large project like FCP X, I have done smaller rewrites of database structures and it is often needed during the coarse of a project. It is going to be what determines if FCP X flies or not. No amount of hardware you throw at it can help.

I would encourage all of us to send Final Cut feedback to Apple regarding this issue. They have to know it is a problem, but perhaps our feedback will let them know how problematic it is.

So I have not lost heart with FCP X, not at all! Apple can get this fixed, and I’m confident they will.

Happy Thanksgiving!

——
T. Payton
OneCreative, Albuquerque

[Simon Ubsdell]

Thanks Timothy for this in-depth and impeccably argued overview of what is clearly one of the biggest hurdles that FCPX has to overcome – and quickly!

Have a great holiday!

Simon Ubsdell
Director/Editor/Writer
http://www.tokyo-uk.com

[Aindreas Gallagher]

good christ.

also its strange to see calm collected investigation as opposed to the native froth loopiness of crazed natives like, well, me.

if…

if apple have in fact walked FCP this far down architectural FUBAR – if it is at subsidence level – is there a basis for thinking they have the wherewithal to walk it back out?

how many software engineers are left working on this thing anyway?

ah god.

annnyway besides – and indeed – happy thanksgiving peeps one and all.

(last ever time for this)

ancient quote from the assembled mass of humanity on this day of thanksgiving, from their gods, their elders and their forebearers:

“Randy Ubillos, you scion of editing intelligentsia, you visionary of time – we thank you Randy for this steaming, brown, curled, strangely smelling, but surely wonderful, wonderful editing bread. thanks for what you took the time to do once you got bored by the strictures of editing as we understood it.
And so it is that we truly thank you Randy, for what you did to Final Cut Pro.”

http://www.ogallchoir.net
promo producer/editor.grading/motion graphics

[T. Payton]

[Aindreas Gallagher] “good christ. “

Actually yes He is very good. 😉

I do think that Apple is able to fix this. (hence my example Garage Band for iOS. It is incredibly well executed.) Although the database problem is pervasive and seems to effect everything in FCP X, it is still the execution that is the problem and not the concept. While I don’t want to make this sound trivial, from my perspective it is fixable.

Kinda like getting into editing and realize that everything that was shot that day won’t work and you have to shoot again. An annoyance? yes. Will it cost you money? yes. Fixable? Absolutely.

The most difficult call for Apple is simply devoting the resources, and making the call to go over how “CurrentVersion” is stored and accessed with a fine tooth comb. Even if they do a complete rewrite on that one “module” or Framework, it is actually one of many elements that make up FCP X. They will make a translator in a future version of FCP X, and no-one will notice. Every indication I have heard Apple say publicly about Final Cut, they want to fix FCP X.

——
T. Payton
OneCreative, Albuquerque

[Aindreas Gallagher]

sorry sir, I do a tad tend to use the lord’s name in an Irish fashion…

True indeed with garageband, it is mindboggling on the 1st gen ipad. I just wonder a little if the banal intricacies of a good editing system, fit to perform in a niche market, is realistically within the scope of their efforts.

as you say, we’ll know in fairly short order. I’m minded by looking at their native efforts with STP and motion though. they were both, ultimately, kludgy software with almost no market traction outside of the wake of FCP. I don’t actually think either of them are objectively particularly good software in hindsight? neither did they have a real life outside of the halo of FCP.

all the serious stuff was brought in – FCP itself, shake, colour, logic – (which I don’t know)
I think… it is an unspoken truth that apple’s own efforts show an attempt at pro software from a company not outfitted to produce pro software?
Motion was heavy and unwieldy (although haven’t used the newest) – and it was buggy. I had to personally try to architect it to be database driven to provide automated multiple GMT lower third endboards for a station launch and I found it genuinely horrible in action. just heavy and chrome filled. make comparisons at will…

So was STP. And STP I knew pretty well.

Apple, I find, make a form of software that looks sort of right when you launch it, sounds interesting, and appears to have interesting workflows, but it ultimately tends to be a slightly gloopy curio. It is as if the software engineers never knew professional hunger – there is a lack of operating empathy in how they go about things – they are just wandering around making idealised scenarios that please in the boardroom demo? that there is, and has never really been any ear at all to the ground?

that aloofness represents brilliance in the consumer market and allows them to leapfrog entire markets intuitively, but as an impulse, it is utter poison to the provision of workable, scalable professional software. I don’t think apple are any longer in a mind position to produce workable professional software. They are completely deaf to any critical signals – and, if larry jordan is to be believed, they wilfully ignore deafening signals coming in on private channels during their rather paper thin betas.

Ultimately – I think that Apple are institutionally (and in a way, for the best of reasons more broadly,) incapable of fostering any of the co-operative customer base relationships critical to the production of this type of software.
You cannot simply toss the puck in the air and skate to where you think it lands in five years: because you are not talking about the whims of consumer habits, you are talking about entire invested industries and countless livelihoods.
That’s not really a puck throwing scenario.

I’d argue that any professional software produced by Apple corporation is currently, (and will for the foreseeable future be), somewhat beautifully made but largely unusable white elephants on sale.

(long reply!)

http://www.ogallchoir.net
promo producer/editor.grading/motion graphics

[Oliver Peters]

The surprising aspect to this is that Apple isn’t totally clueless about databases. After all, once indexing is done, Spotlight returns search results at blinding speeds. Filemaker Pro is highly regraded and that company is a subsidiary of Apple. Although Bento isn’t particularly fast.

As a point of comparison, take a look at Vegas versus FCP X or Numbers versus Excel. In the latter example, Numbers is friendly and attractive, but completely chokes on 3,000-entry spreadsheets that don’t even give Excel a pause. All on the same machine. I can run Vegas under Parallels and Windows on my Mac Pro and have a very satisfying experience.

It’s quite possible that too much of the database handling came over from FC Server. It’s not a particularly good asset management tool and when it works well, it’s running under Mac OS X Server usually on an Xserve. If that’s the case, then there are some built-in design problems.

Remember, that Ubilios gave us FCP 1 as well as X, so a better product was a possible result. I believe Aperture came out of the same team and the market goes back-and-forth between it and Lightroom as to the preferred photo app. I suspect that if ProApps were spun off as a subsidiary forced to justify its existence based on products, you might actually have better results.

– Oliver

Oliver Peters Post Production Services, LLC
Orlando, FL
http://www.oliverpeters.com

[Frank Gothmann]

Actually, your mentioning of Soundtrack Pro here is scary and brings back very unpleasant memories of working with it.
SP shares some code borrowed from Logic but it has been written by a completely different team at Apple. Logic still has veteran guys from the original Emagic on board.
The differences in performance and stability between the two apps is like night and day. So much so that it is hard to believe they were sold by the same company.