-
FCPX developers, please optimize disk access
-
Lasislav,
Perhaps this will help you look at things in a different light…
One of the absolutely best things about X is that it’s taking constant “snapshots” of all the decisions you make – every edit, every color correction, every sound change – it tracks the state of your project.
All FCP-X does is track text changes. That’s the metadata that everyone is always talking about.
So you can think of it as whenever you point to ANYTHING in the X interface, the program must refer back to the original text that describes the state of the thing you’ve pointed to.
So access to “historical” data is critical – since every new change has to be linked to that. Always.
That’s the problem it sounds like you’re facing. Your system is extremely slow to access the historical editing data X needs to attach your new instructions. So it’s driving you nuts.
You might try using the MOVE command to re-locate the PROJECT and EVENT files you’re working with to the same drive as your footage. It’s possible that if the machine can locate everything it needs in the same place, it might fractionally speed things up since that drive is where the controllers will look for everything and therefore it will never go to “sleep” and slow things down more.
But at the heart of things, you’re simply working with a system that’s doesn’t shuffle enough data around fast enough to keep up with the editing your doing.
As others have noted, I think that USB-2 just isn’t fast enough to work with anything larger than Proxy files.
Good luck.
“Before speaking out ask yourself whether your words are true, whether they are respectful and whether they are needed in our civil discussions.”-Justice O’Connor
-
Bill and all guys here,
I appreciate your answers. I work with high speed RAIDS. This USB2 experience was extreme a occasioanal case.
As old SQL database developer I’m aware of not optimal programming. For example not optimal SELECTs work fine with small databases on speed hardware, but with thousands of gigabytes of data such SELECTs are weak. And if thousands of users run such SELECTs at same time, servers are in trouble then.
My USB2 experience tells me that something can be made better.
Do you remember problem with doubling project file size when compound clip was made? Apple fixed it.
So, I believe that FCPX developers can avoid some unnecessary disk activity and make editing on slow drives and machines faster. -
[Ladislav Zamba] “As old SQL database developer I’m aware of not optimal programming. For example not optimal SELECTs work fine with small databases on speed hardware, but with thousands of gigabytes of data such SELECTs are weak. And if thousands of users run such SELECTs at same time, servers are in trouble then.”
As a SQL developer you must also know that the most effective way to accelerate a database is to give it lower latency storage to operate on. Depending on the size of the database, that is often a matter of loading the entire thing into RAM. Moore’s Law and Brute force have been the saving grace of accelerating large databases for a long time, and the same is true for a 64-bit, hardware-optimized NLE.
[Ladislav Zamba] “My USB2 experience tells me that something can be made better.”
My USB2 experience tells me that it is some of the slowest, highest-latency storage you can get short of trying to edit over 100BASE-T.
[Ladislav Zamba] “Do you remember problem with doubling project file size when compound clip was made? Apple fixed it. So, I believe that FCPX developers can avoid some unnecessary disk activity and make editing on slow drives and machines faster.”
Maybe they could if you had a huge glut of RAM they could load more of the utilized source media into, since they have to store the timeline media somewhere if it isn’t going to be read from disk when called upon. However, for a large project, this too is untenable. Think about how much RAM you have, and then how much media you are using in your large project’s timeline. I can’t imagine what they could do to achieve what you are describing. Sometimes you just need better hardware.
That said, I do agree they need to optimize how FCPX assets are stored- specifically, allowing for separation of Render media from the Project database storage. The Events and Projects databases will be much happier on SSD, while the media can tolerate higher density, lower cost HDD arrays that are built for streaming media and not so good with the IOPS.
Best,
Andy -
Andy – I have Project and Event folder on internal drive. Only original media files was on USB2 disk.
And with SQL optimisation I was pointing to sequential vs. index searching in tables. Even if you have modern servers with lot of RAM, it’s not always possible to put 500GB table into RAM 🙂
So, modern HW is good to have, but programmers must optimize their code always.
What if Apple is planing to implement multiuser access to Events. Imagine 100 editors accessing same media at same time.I still hope that there is no reason for FCPX to access all original media used on timeline.
-
[Ladislav Zamba] “Andy – I have Project and Event folder on internal drive. Only original media files was on USB2 disk.”
I understand, but I maintain that a single USB2 HDD is usually inadequate media storage for Pro Res. What brand/model USB2 drive are you using? They vary a lot.
[Ladislav Zamba] “And with SQL optimisation I was pointing to sequential vs. index searching in tables. Even if you have modern servers with lot of RAM, it’s not always possible to put 500GB table into RAM :-)”
Yes, developers should always optimize their code to be efficient, I don’t dispute that. All I’m arguing is that hardware is often a much more significant variable when it comes to NLE responsiveness. Software tuning will only get you so far.
500 GB tables? https://www1.hp.com/products/quickspecs/14212_div/14212_div.HTML“>No problem. Now if you’re getting into the 10s of TBs, then you’ll really have to spend some money!
[Ladislav Zamba] “What if Apple is planing to implement multiuser access to Events. Imagine 100 editors accessing same media at same time.”
Oh, that’s easy. Xsan (StorNext) can scale any which way you need. It ain’t cheap, but if you really have 100 editors, it can be built to handle it.
I stand by my original point- hardware performance is a lot more of a factor than software optimization when it comes to NLEs. Software optimization is very important, and should certainly be stressed, but at some point you bump up against physics and you just need a bigger boat.
Best,
Andy -
It was desktop 3.5” 7200 rpm drive with 32MB cache in USB2/eSata enclosure
BTW playback and skimming was fine and acceptable.
-
USB 2 is not even up to the task of capturing DV.
The minimum storage requirement for FC historically has been FW400.
USB2 offers a similar transfer rate but with many peaks.
rafael -
I have the program on internal drive, project and render files on on a second drive and then all media on a larger third drive. Regardless, I think you’re fighting the obvious. USB for video isn’t fast enough, never has been. Hence the importance of firewire. I doubt that Apple will take the time to optimize FCPX for USB2 when its looking to thunderbolt for the future.
-
I only wanted to say that working with large timeline on very slow media disk can reveal some bottlenecks in SW too.
-
Larry Jordan has new article
And there is workaround:
“However, for larger projects, compound clips that aren’t nested inside each other can actually make the system run better. If you are loading lots of clips into the Timeline, Final Cut needs to track each of these clips individually. Instead, if there is a section of the Timeline that you are done editing, select all the clips in that section and convert it to a compound clip using File > New Compound Clip.
This tells Final Cut to treat those clips as a group. This improves memory management and overall performance, especially as projects and clip counts get larger.”
Reply to this Discussion! Login or Sign Up