CatDV 9.0.1 crashing using Import Directory

[Craig Seeman]

Temp Licensed Pro, switched Java SE 6 32 bit above 64 bit.
Tested each media drive. Each one crashed CatDV at what seems to be around the same point.
Crash logs seem to be the same as the ones I sent. Same for the CatDV Error logs. Either that means the same file is causing the crash or there’s some kind of cache or temp file filling up maybe?

[Bryson Jones]

I’m betting you’re running out of RAM. I’m not sure anything is “broken”.

A lot of folks dispute this, but I never advise that folks use CatDV to “scan” a drive.

You’d never drop 70,000 assets into FCP at once, for instance.

I advise that my clients import them in by folder, organizing them into catalogs as they go… managing assets, as the term says.

The hardest part of any asset management project is when we realize how much work there is to do before the software kicks in.

I’ll bet if you do it a folder at a time, following the best practices for catalog size laid out on this forum, you’ll get better results.

https://forums.creativecow.net/thread/278/1073#1073

bryson

bryson “at” hidefcowboy.com

hidefcowboy.com

[Rolf Howarth]

Hi Craig,
Did you get the email I sent you a few days ago after looking at your log file? The two crash logs you sent seemed to indicate either a corrupt MPEG file (in which case it would probably crash QuickTime consistently each time you process the file) or a problem extracting large thumbnails from the file system (which you can turn off via a Preferences option).
-Rolf

[Jorgen Gustafson]

Why does a program crash cause of a corrupt file? Is it possible to make the program just ignore a corrupt file? Maybe even make a note about it and it’s location? So after import you just get list of files that CatDV refused to import cause of corruption or whatever? Isn’t that the normal routin for other programs importing files?

[Rolf Howarth]

Currently analaysis of files during import is done within the CatDV process. If a particular QuickTime codec crashes when processing a file then CatDV will crash (though it does autosave the catalog at periodic intervals, and the log file will show which file it was working on). We do have using a separate process to handle the import down as a possible enhancement but as this is quite a rare occurrence in normal operation it’s not a top priority.

If you do experience problems on import some things to try include:

– turn off the Details panel during import
– switch to a Concise view during import
– turn off “Generate full size thumbnails from file system” prefs option (Mac OS X only)
– import files in smaller batches and try to determine if a particular file is failing repeatably

[Craig Seeman]

I’ve been horribly busy (of course that’s not horrible).

The file in question on one disk is an MPEG-2 Program Stream 4:2:2 Main Profile. This is not something that Quicktime would be able to play even with Apple’s MPEG-2 plugin. Quicktime can only play 4:2:0 MPEG-2 Program Streams. I’ve used this format with some VNRs and it’s also what I use for delivering 30 second spots for broadcast. I always check these files in VLC and this plays just fine.

I tested by importing the folder containing this file rather than the entire hard drive. Sure enough it crashed CatDV.

I tested by pointing to other folders that have MPEG-2 PS 4:2:2 Broadcast masters. They didn’t crash CatDV but they did throw warnings. I also note the progress bar slowed when it hit these files (as it did with the previous file in question)

The Import Notes show:
File only partly scanned (1/5.5), statistics are approximate
and there’s no poster frame generated.

THe only difference between those that worked and the one that crashed CatDV was the duration. The one that crashed CatDV is about 5 minutes long whereas the ones that worked about 50 seconds or so (short bars and slate).

So this would seem to indicate that I can’t catalog any of my MPEG-2 PS 4:2:2 files that are even a few minutes long.

In Preferences I turned of “Generate full size thumbnails from file system” as a next test and pointed it the offending folder and . . . again it crashed.

Only shutting off Thumbnail generation altogether allowed me to catalog the file. Even the smallest size caused a crash with this file.

This is very awkward for me because I use such files for broadcast delivery. If only there were a way to turn off thumbnails for specific codecs over a certain duration. For me I’d imagine a series of attempted imports and crashes and then tracking down the offending files in each folder.

The only workaround would be to start pulling these files out (across all my drives) and quarantine them and turn off thumbnails for just these files. Otherwise it’s not using thumbnails at all and I really don’t want to disable that feature.

BTW attempting to play the offending clip even when imported without thumbnail, caused CatDV to crash. The other MPEG-2 PS files just didn’t show video (to be expected using Quicktime), no crash.

I still haven’t investigated why the other hard drive caused a crash but I’ll investigate thumbnails as a cause.

[Rolf Howarth]

CatDV normally uses QuickTime to extract a thumbnail image from these files. Even though QT doesn’t support 4:2:2 files it obviously shouldn’t crash when you try to open one. If you have an old version of the Apple MPEG2 component it’s worth downloading the latest one because I know there were some crashing problems with it in the past. Also, if you can forward us a copy of the crash log in Home > Library > Logs > Diagnostic Reports that would enable us to confirm whether it really is the QuickTime codec crashing when opening these files.

As for how to handle these files, you could possibly import all the MPEG files first with thumbnails turned off (by using Spotlight to find all the .mpg files in a folder), then turn thumbnails on and reimport the folder but add “mpg” to the list of file extensions to exclude in CatDV’s Import preferences.