Joe Marler
22
Apple Final Cut Pro X Coins
39
COW Coins
Forum Replies Created
-
Normally FCP will import AVCHD files by copying and re-wrapping them inside the library. Some people copy the .mts files outside the library bundle with Finder, then import them with “leave files in place”. That should never be done due to I/O performance issues.
In either case, simply deleting the references to those files within FCP will not cause red clips. If clips are red that implies he manually deleted the on-disk media files themselves with Finder.
FCP will never delete external media files and will only delete internal media files if no references exist, IOW the files are not used in any project (hence no red clips since they aren’t being used anywhere). Even in this case it warns you with the pop-up “Media Moving to Trash…One or more media files will be moved to the trash since the last reference to them is being deleted. External file will remain where they are”.
Due to inode resolution, FCP will even find external media files if all symlinks to those are deleted from within the library, if every media file is renamed and each renamed file is moved to another location on that volume. This assumes either a Mac OS Extended or APFS volume.
If the AVCHD file was internal and was deleted via Finder from within the library, and if you have the original non-rewrapped .mts file from within the AVCHD bundle, that can be relinked. In FCP find the red clip, select it, then pick File>Relink Files>Original Media and point to that file.
If the AVCHD file was extracted from the bundle with Finder and the bare .mts files imported with “leave files in place”, that file can also be relinked.
You can verify where FCP is looking for the media file by picking the red clip, do File>Relink Files>Original Media, then do not relink but look at the bottom of the dialog and in grey letters it will show the pathname where it expects the file.
If the machine has a Time Machine or any other backup type, as a contingency it might be good to preserve that before it is overwritten. The missing media files or earlier versions of the library before the problem might be preserved there.
-
Reformatting the external drives as HFS+ or APFS (if SSD) is a good idea, regardless of whether if fixes the current problem. ExFAT is not a transactional file system so in some shutdown situations the file system metadata may not be properly maintained.
FCP is technically supported on ExFAT but it doesn’t support hard links which FCP uses frequently. Only HFS+ or APFS volumes support “inode lookup” for media files (see below details). Hard links are esp. used when moving media between libraries on the same HFS+ or APFS drive. There is a fallback code path for non-Mac volumes but I don’t remember the exact behavior.
Not recently but I have infrequently seen the problem of sporadic de-linked files following large cross-database operations and/or if renaming disk volumes. This was possibly related to some of the below-listed issues. That was on HFS+ so the file system itself does not fix all problems but it’s a good idea.
You had a bunch of smaller libraries and were copying clips and (apparently) projects between libraries. Unfortunately there are some poorly-documented issues with how FCP handles this.
Whether external drives, internal drives or NAS, when copying clips and projects between libraries, they should be inside a “transfer event”. Sam Mestman discusses this from 06:30 to 11:00 in the below video. He demonstrates this on a Lumaforge NAS but it’s not unique to a NAS. “Final Cut Pro X Virtual User Group #7”:
https://youtu.be/NAv89cGexIM?t=386
Another complication can arise if all media filenames are not globally unique. FCP will attempt to handle this by adding a “uniquifier” suffix of (fcp1), (fcp2), etc, but this doesn’t always work perfectly. The best course is avoid the problem by renaming all media files before ingest by adding a unique incrementing serial number. See Macmost #1651, “Batch Renaming Files in Mac Finder”: https://youtu.be/fDcvqyHduHc
Yet another complication is FCP has two independent lookup methods for each media file. It normally tries to resolve the file location via a pathname lookup based on data in SQL tables in the library. If that fails it will do an “inode lookup”. That is why for “in place” media on a Mac-formatted volume, you can rename every file, delete every symlink in the library, then relaunch FCP and it will still connect to the media. However — inodes are only maintained within a volume. If media is moved to a different volume it can only do a path lookup. If the drive volume is renamed it may not find the media, but in theory you can relink.
If all media files are globally unique, if all drives are HFS+ or APFS and if you move clips/projects between libraries inside a transfer event, this situation normally doesn’t happen. Unfortunately these procedures are rarely mentioned in any tutorial.
The above may partially explain how the problem arose, but how to fix it?
You should be able to use Finder and copy (not move) media out of the library to an external drive, then relink to that location. On the current version 10.5.4, relink to those will replace the internal media files with symlinks pointing to the new location. I don’t remember if it always did that.
As a wild guess, before doing any of the above steps you might run Disk Utility First Aid on all volumes (including system) and rebuild all Spotlight indexes on all volumes. I don’t know if FCP uses Spotlight lookups for relink but it wouldn’t hurt to run these. To properly run First Aid on the system volume you must be in Recovery Mode (booth with CMD+R), else it cannot lock the system volume for changes.
How to rebuild Spotlight indexes: https://support.apple.com/en-us/HT201716
Some contents or functionalities here are not available due to your cookie preferences!This happens because the functionality/content marked as “Google Youtube” uses cookies that you choosed to keep disabled. In order to view this content or use this functionality, please enable cookies: click here to open your cookie preferences.
-
Are all drives used for FCP libraries and media — inc’l the new SanDisk 1TB SSD — formatted as MacOS Extended Journaled (aka HFS+), APFS, or something else like ExFAT? In general all drives used by FCP should be formatted HFS+ or APFS.
After you re-launch FCP and see the new errors, what if you pick one such file and begin the relink dialog. Before you actually relink it, note at the bottom of the relink dialog and it will state in grey letters the pathname where it expected the file. Does that pathname give any clues about why it can’t find the file? E.g, is it looking for a different drive name, or is it looking inside the previous small library which may no longer exist?
-
Exporting a timeline of a clip selection (or consolidate with trim) with extra user-specified handles is a common need, especially when working remotely. I really wish FCP had that — provided it worked comprehensively and reliably.
In FCP you can export all clips used in a timeline with the event browser “used media” filter, but (a) not with additional user-specified handles, and (b) multicam clips cannot be exported.
The 3rd-party tool Worx4 X can do export with user-specified handles, albeit with several limitations. It will not trim MXF, compound, multicam or sync clips: https://worx4.com
Resolve has this feature (called media management) but in my testing it has been unreliable. A few months ago I tested numerous codecs on Resolve Studio 17.1.1, and it caused frequent data corruption of some Long GOP destination clips if using the “copy” (not transcode) option. Most All Intra clips worked OK, but it failed on MXF-wrapped All Intra from a Sony FX6. Resolve formerly had both “move” and “copy” options for this; the “move” option has been removed, maybe because of reliability problems.
I just re-tested it on 17.2.2 and it seems to work better now. However because of all the previous problems I’d have to test it rigorously before I trusted it on an actual project.
There may be a technical challenge in doing this safely. In Resolve’s implementation, you can select “copy” or “transcode”. The copy case is a bulk “no encode” trim of many (potentially hundreds or even thousands) of clips, possibly comprised of many different complex Long GOP codecs. When you do a no-encode trim of a single clip using Quicktime Player, you know immediately if it worked or not. However a bulk operation could jeopardize data integrity of some clips. Even if it worked OK 98% of time, in a big operation numerous destination clips could be damaged. Most people aren’t going to check every frame of every clip.
Other implementation options might include doing only re-encode trims, which would likely be safer but would be much slower and involve generational loss, esp. for Long GOP codecs.
For the transcode case, it can’t be that hard because FCP essentially does that now, just doesn’t allow user-specified handles and won’t do multicam.
-
The built-in FCP color match tool is fairly simple and is not designed for complex cases. It does not know there’s a color card in the frame. This often gives the impression it’s not doing much.
Without using a 3rd party tool, you can use the procedure in this video to match shots from different cameras based on a ColorChecker chart. However IMO he should be adding correction effects before the Log-to-Rec709 conversion LUT, not after. The LUT has no idea the color balance is off and will blindly translate those colors, which might make it harder to correct if “grading under the LUT”.
There are ways to automate this. The Color Finale plugin can recognize a standardized color card (at least the XRite ColorChecker Video; don’t know about others). You draw a little box around the card on both source and destination clips and it visually identifies each tile on the card and matches based on that.
Resolve Studio has a similar feature built in.
If both cameras were in a log profile, that’s better than a “baked in” color look, but each camera has its own unique log implementation (often more than one). E.g, Canon CLog2, CLog3, Sony SLog2, SLog3, Blackmagic FilmLog, etc. In theory the camera-specific log conversion LUT should help but the color science is nonetheless different between cameras.
There’s a plugin for Premiere and Resolve (and planned for FCP) called CineMatch which tries to match different profiles and sensor characteristics. It doesn’t replace the above matching methods but is designed to better match cameras which are properly white balanced and exposed: https://www.cinematch.com
Some contents or functionalities here are not available due to your cookie preferences!This happens because the functionality/content marked as “Google Youtube” uses cookies that you choosed to keep disabled. In order to view this content or use this functionality, please enable cookies: click here to open your cookie preferences.
-
Try to export a full library XML, then create a new library and load that XML.
-
Matt, you already did several good troubleshooting steps. It crashed in the method FFEffectStack due to an unrecognized selector. I think it was technically an Objective-C language exception. The question is what effect was involved. I can’t tell from the crash log. Prior to that it was in the FCP framework “flexo”, which implements several key parts of the app.
Frame 12 and 13 mention the method FFAnchoredSequence, which might be a secondary storyline.
Toward the end of the crash log is the section called binary images. That lists all code loaded in the process at the time of crash. There was code from several plugins loaded into FCP address space. Many FCP plugins prior to Big Sur on Apple Silicon run within the address space of the FCP process. That means any bug in any thread of any plugin can crash or destabilize FCP.
That should be improved as developers re-write their plugins using the FxPlug 4 framework, which is mandatory for Apple Silicon. But until then any mysterious crashes where 3rd-party plugins are used require scrutiny of all plugins. Your plugins within FCP’s address space include:
CoreMelt
Color Finale
Pixel Film Studios (several different plugins)
Digital Anarchy Flicker Free
FxFactory
I suggest you update all those effects or remove them if needed as part of debugging. CoreMelt plugins can be updated using their installers: https://coremelt.com/pages/downloads
There have been many problems traced to Pixel Film Studio plugins. If possible try removing those or make a test library and copy a project over there, then remove those from that test timeline. Here is how to remove those plugins: https://store.pixelfilmstudios.com/how-to-uninstall-a-plugin/
Digital Anarchy is a good company but they have an updated version of Flicker Free. Make sure you are running the latest version: https://digitalanarchy.com/Flicker/main.html
FxFactory is like an apps store for plugins. The host FxFactory app is frequently updated. Make sure you’re running the latest version.
Even though you didn’t report any MacOS kernel panics, you might want to review what 3rd-party kernel extensions are in use. Those can destabilize or crash MacOS or applications. To inspect this, type this command in terminal:
kextstat | grep -v com.apple
-
Sorry to hear this. Normally FCP data integrity is excellent, esp. on directly-attached hard drives which are formatted APFS or Mac OS Extended Journaled (HFS+).
First make sure you have at least 20% free space on all drives. Then run Disk Utility First Aid on all drives to make sure there’s no filesystem problem.
The fact that weeks-old library backups also crash may imply a system-level problem, or plugin issue, not corruption in your current library.
A common cause of FCP crashes (but not necessarily data corruption) is 3rd-party plugins. Can you examine any 3rd-party plugins and remove or update them? Plugins often have an external deinstallation procedure or the plugin developer publishes methods to manually deinstall them.
Don’t mess around inside your library unless you have a complete backup of it.
One relatively safe thing is delete all render files. You can (carefully) do that in Finder by deleting the library folders named Render Files. However there is a 3rd-party utility which safely does that called Final Cut Library Manager: https://www.arcticwhiteness.com/finalcutlibrarymanager/
After doing all of the above, if it still crashes, if you post the crash log I will examine it. Please post it as an attached text file, not an in-line post.
Application crash logs can be found by holding the OPT key in Finder and selecting menu Go>Library>Logs>Diagnostic Reports.
Please also state the year and model of your Mac and what version of MacOS and FCP you are running, also the make and model of the external drive and while filesystem it uses (from Finder Get Info).
-
The term for the envelope that contains clips in a secondary storyline is the “shelf”.
Individual clips above the primary storyline are simply connected clips, or anchored clips. When one or more of those connected clips are placed in a secondary storyline, a visual shelf is created to contain them. Way back in 2012, Ripple Training discusses this in MacBreak Studio #183:
Some contents or functionalities here are not available due to your cookie preferences!This happens because the functionality/content marked as “Google Youtube” uses cookies that you choosed to keep disabled. In order to view this content or use this functionality, please enable cookies: click here to open your cookie preferences.
-
How do those media files look at each stage of the offload, conversion and import sequence?
If you play the original camera media file in Quicktime Player or VLC, what does it look like?
What about those same files played in Quicktime or VLC after conversion by EditReady but before import to FCP?