-
I’m having a nightmare with repeatedly missing files.
Using FCPX 10.4.10 on Mojave here, and normally running sweetly with no problems but I’ve started working on a bigger project and having major headaches.
Over the past year I’ve filmed around 30 short pieces, each with their own libraries, with all files consolidated within those libraries.
Now I need to edit a larger project taking clips and stills from all those libraries, so I’ve been creating a new library containing all the smaller library projects, and setting up events which correspond to those projects.
It’s been a slow and laborious process, and I’ve kept having odd glitches where I’ve dragged files from the smaller libraries into the events I’ve created in the master library, and then discovered they didn’t copy after all.
Still, I’ve preservered, and this afternoon after the last batch of copying, then making sure any missing files got relinked, and finally consolidating the whole master library and seeing the magical “no files to consolidate” notification, I thought it was done. No warning triangles on any projects or events, nothing to consolidate, all projects appearing to open up as they should.
EXCEPT…after closing the library, when I reopened it, there were several warning triangles, and dozens of files missing again. How can this be? I’m saving everything on a fairly new SanDisk 1Tb SSD and nothing seems to be corrupting, just certain files missing again.
JUST BEFORE POSTING THIS, I closed down FCPX and then reopened again, and this time there are fewer files missing, but some of them are different! Shutting down and opening again, and once more a few random files missing.
This is so random, deeply frustrating, and very time-consuming – clearly there’s no point relinking and consolidating because it appears each time I open the library, different files are supposedly missing.
Any ideas what I can try, to troubleshoot and solve this. FCPX seems to be working fine on all other smaller libraries, so I’m stumped.
-
10 Replies
-
A bit more info:
If I try relinking one of the missing files, it tells me it’s expecting it to be in an event I created called “Projects” which is where I’ve put all the original projects from the smaller libraries.
So, I locate the large library in the ‘relink’ dialogue, it appears to find and verify the file, so I press relink, and it still appears missing 🙁
-
This is just a shot in the dark, but I wonder if there’s a conflict by using “Projects” in the actual event name. What happens if you rename your “Projects” event to something else, like “Original Edits?”
-
Trid that on the off chance, but no cure I’m afraid. I was sceptical because I’ve always created an event called PROJECTS in all my smaller libraries and never had these problems before.
The weird thing is that when a file is reported as missing, mostly if I open up pacakge contents and look in the original media of that event, it IS actually there. So I drag a copy out, relink, reconsolidate, and it’s sorted (until the next time).
My PROJECTS/ORIGINAL EDITS folder only appears to have projects in it in the browser, but if I look in the Original Media folder of that ‘event’ I can see that dozens of media files are sitting there too, doubling up om where they’re supposed to be. I don’t know if this is connected to my issues, or why they are there.
-
Are all drives used for FCP libraries and media — inc’l the new SanDisk 1TB SSD — formatted as MacOS Extended Journaled (aka HFS+), APFS, or something else like ExFAT? In general all drives used by FCP should be formatted HFS+ or APFS.
After you re-launch FCP and see the new errors, what if you pick one such file and begin the relink dialog. Before you actually relink it, note at the bottom of the relink dialog and it will state in grey letters the pathname where it expected the file. Does that pathname give any clues about why it can’t find the file? E.g, is it looking for a different drive name, or is it looking inside the previous small library which may no longer exist?
-
Hi Joe
I wasn’t aware of this requirement re drive format, and yes the SanDisc is exFAT.
I’m not confident this is the issue as I’ve realised one of my main 4Tb archive discs is exFAT with many dozens of libraries on it all running fine.
However, I’m copying on to a back up disc, will reformat the SSD as MacOS Extended Journaled and let you know if that helps once I’ve copied back.
re Relinking – it’s always showing it’s looking for the file within the new large library, not other discs or libraries. As mentioned above, on inspecting package contents I’ve seen a lot files are sitting in the original media folder of my Projects folder, but they don’t show up in the browser.
AS A DIFFERENT APPROACH so I can move on, does anyone have suggestions of how they would go about combining a load of smaller libraries into one larger one – maybe there was something about my methodology that confuses FCPX, so I’d be very interested how others would approach this in a painless and orderly way.
-
Opening the library on the new MacOS Extended Journaled disc, there were, once again a few files missing from one of the events. When I try to relink, the pathname points to the ‘Original Edits’ folder in the library on the new disc. As mentioned above, inspecting package content reveals there’s a whole load of media sitting in that folder that doesn’t show up on the browser. The browser is only showing the projects, which is what I WANT that event to hold. (Before you ask ‘All Clips’ IS selected in browser). The missing media is in an event where I want all the footage for that particular project, and inspecting the package contents of that folder, I can see that the supposedly missing clip actually IS there.
As an experiment, in the relink dialogue, I selected the library (since I know the media is in there), and after a little search, it comes up OK as found and ready to link, but when I finish linking, it still shows as a missing file.
The only way I’ve found to fix it, is to drag a copy of the missing file out of the package contents folder onto my desktop, relink it there, consolidate the library, and all seems fine.
The maddening problem is that next time I open the project, a few other files will have gone AWOL in the same way, seemingly randomly. They can all be fixed the same way, but another relaunch and a new set of problems.
AAARGH!
Unless someone’s got an idea what’s going on here, I think I need to start from scratch rebuilding a new library from all the older smaller ones, but I’d love pointers about the smoothest way to do that as I don’t need any more glitches or to spend to much time doing so.
-
Reformatting the external drives as HFS+ or APFS (if SSD) is a good idea, regardless of whether if fixes the current problem. ExFAT is not a transactional file system so in some shutdown situations the file system metadata may not be properly maintained.
FCP is technically supported on ExFAT but it doesn’t support hard links which FCP uses frequently. Only HFS+ or APFS volumes support “inode lookup” for media files (see below details). Hard links are esp. used when moving media between libraries on the same HFS+ or APFS drive. There is a fallback code path for non-Mac volumes but I don’t remember the exact behavior.
Not recently but I have infrequently seen the problem of sporadic de-linked files following large cross-database operations and/or if renaming disk volumes. This was possibly related to some of the below-listed issues. That was on HFS+ so the file system itself does not fix all problems but it’s a good idea.
You had a bunch of smaller libraries and were copying clips and (apparently) projects between libraries. Unfortunately there are some poorly-documented issues with how FCP handles this.
Whether external drives, internal drives or NAS, when copying clips and projects between libraries, they should be inside a “transfer event”. Sam Mestman discusses this from 06:30 to 11:00 in the below video. He demonstrates this on a Lumaforge NAS but it’s not unique to a NAS. “Final Cut Pro X Virtual User Group #7”:
https://youtu.be/NAv89cGexIM?t=386
Another complication can arise if all media filenames are not globally unique. FCP will attempt to handle this by adding a “uniquifier” suffix of (fcp1), (fcp2), etc, but this doesn’t always work perfectly. The best course is avoid the problem by renaming all media files before ingest by adding a unique incrementing serial number. See Macmost #1651, “Batch Renaming Files in Mac Finder”: https://youtu.be/fDcvqyHduHc
Yet another complication is FCP has two independent lookup methods for each media file. It normally tries to resolve the file location via a pathname lookup based on data in SQL tables in the library. If that fails it will do an “inode lookup”. That is why for “in place” media on a Mac-formatted volume, you can rename every file, delete every symlink in the library, then relaunch FCP and it will still connect to the media. However — inodes are only maintained within a volume. If media is moved to a different volume it can only do a path lookup. If the drive volume is renamed it may not find the media, but in theory you can relink.
If all media files are globally unique, if all drives are HFS+ or APFS and if you move clips/projects between libraries inside a transfer event, this situation normally doesn’t happen. Unfortunately these procedures are rarely mentioned in any tutorial.
The above may partially explain how the problem arose, but how to fix it?
You should be able to use Finder and copy (not move) media out of the library to an external drive, then relink to that location. On the current version 10.5.4, relink to those will replace the internal media files with symlinks pointing to the new location. I don’t remember if it always did that.
As a wild guess, before doing any of the above steps you might run Disk Utility First Aid on all volumes (including system) and rebuild all Spotlight indexes on all volumes. I don’t know if FCP uses Spotlight lookups for relink but it wouldn’t hurt to run these. To properly run First Aid on the system volume you must be in Recovery Mode (booth with CMD+R), else it cannot lock the system volume for changes.
How to rebuild Spotlight indexes: https://support.apple.com/en-us/HT201716
Some contents or functionalities here are not available due to your cookie preferences!This happens because the functionality/content marked as “Google Youtube” uses cookies that you choosed to keep disabled. In order to view this content or use this functionality, please enable cookies: click here to open your cookie preferences.
-
Wow, thanks for all the analysis, links and suggestions. Much appreciated.
I think I was so used to what you describe as ‘inode’ lookup working flawlessly, that I assumed I could just drag stuff from one library to another and it would sort itself out, and clearly there was something about the way I was proceeding that caused the problem.
I think I’ve worked out the practical cause – I’d drag a project over, ticking the ‘without media’ option, but as well as the project it would bring unlinked named files with it. My assumption, based on previous flawless experience, was that when I drag across the relative events (WITH original media), everything would link up OK, so then I deleted the apparently unlinked versions and assumed the project would find the right files in the right folders. But for whastever reason (maybe due to disc format or something else), the linking went awry, and I seem to have ended up with some original media in the project folder, while FCPX is convinced it should be in the event folder.
I did try pulling the files out of the package contents project folder to desktop and then relinking and consolidating – one of the fixes you suggest – but this is where it got weird, because although I kept fixing stuff like this, each app restart would throw up random unlinked ones again.
After a talk through with a nice chap from Apple, we decided to rebuild the master library from scratch with a workaround method, while he consults engineers. He agreed I wasn’t doing anything particularly weird and FCPX wasn’t playing fair with what I was trying to do, and he’s going to get back to me next week, but here’s how it’s worked for now.
For each small library, I first select events I want to bring into the new master library, and I copy them WITH original media. One complication is that I often use a starter template library which has a few compound clips for logos etc, and I don’t want to have multiple folders in the new library with these, so I’ll leave them behind.
Next, I select the projects I want, and use the File menu comman – copy to library selecting ‘without media’. This will bring over the projects, but because the events folders are already in place, it’s not dragging a whole load of files with it (which was the source of my problem before). If it has brought anything over that I’d missed (and those pesky repeated compound clips), I throw them into the events as required (including a generic ‘Compounds’ event for the logo repeats).
Doing it in this order seems to have kept everything neat and without problems. I’ve ended up with 30 or so projects from their original libraries, which I’ve numbered, and then I’ve matched the numbering and have all their original event folders still in place, ie 001 footage, 001 stills, 001 compounds, 001 music, 001 graphics, 002 footage and so on.
This is all I ever wanted and it seemed crazy I couldn’t just drag the projects over without media, drag all the events folders over, and for FCPX not to be able to work it out for itself, but doing it in the specific order one by one seems to have resolved it.
If I get any more info from Apple I’ll post it here in case anyone else comes across similar issues.
Thanks again Joe for your care, time and input.
-
Ever considering upgrading?
I’ve had issues similar but never bothered to fix them.
But after upgrading it seemed to have woken up the links.
-
Not keen on Big Sur as I have a lot of plug-ins, quite a few of which might not yet be ready, and versions of FCPX above my current one don’t offer me much other than Big Sur compatibility. Apart from this one issue, now resolved, I have a very smoothly running system I’m happy not to mess with.
Reply to this Discussion! Login or Sign Up