-
Time Lapse Problem “too many files” for OS X
Trying to put together a time lapse project that would include over 600,000 files. OS X doesn’t seem to be able to deal with that many files! Has anyone ever encountered this problem? I can’t believe that this is a new problem but I can’t find any info on it.
Have 6 cameras, each with folders/sub folders. Trying to get each cameras images into one folder leaving me with 6 folders with about 100,000 jpg’s. Then I would hopefully “open image sequence” in quicktime 7..
Tried searching for all .jpg’s within my root folder, then copy and paste into a new folder. Cam back with error “more than 10,000 files”
tried importing into fcp then media manage into one big folder but fcp freezes on import.
Any help would be much appreciated.
-
2 Replies
-
Hi Michael,
FC is a Video editing tool. A bit of compassion.
100.000 stills at 1 frame, makes a 55 minutes sequence (30fps).
Thinking that the only no-insane task you can do in FC with 100.000 stills is to export as QT Movie, you shouldn’t have any problem in splitting the job in few sequences, projects or whatever.
Graphics applications (AE, SHAKE,..) are more fitted to manage huge stills sequences.
rafael -
Long Answer:
The OS has limits on the maximum number of file handles that can be on on a system wide, per user and per process basis. I’m guessing you’re hitting the max files per process limit (kern.maxfilesperproc) which defaults to 10240. You can see the limit by using “sysctl kern.maxfilesperproc” at a shell prompt. “sysctl -a” will show you all sorts of limits and settings of the system. Some/most of these you can change with sysctl -w parameter=value. Some must be set at boot time, so you put the entry in /etc/sysctl.conf (this will make a change permanent, too) and reboot. Some limits are hard coded into the kernel. You will also need to look at kern.maxfiles. I’m not sure if these values are 32bit or 64bit integers. Assuming 32bit, the biggest I’d expect this to be is a touch over 65,000. I’m also pretty sure you’ll chew up more kernel memory in doing this — especially if you can get to your 600,000 target. Remember that when you start messing with these settings, you’re playing with a loaded gun. Adjusting kernel settings is something you want an experienced Sys Admin to do based on well thought out understanding of the impacts of the change. Don’t complain if you shoot yourself with this loaded gun.
Shorter Answer:
Don’t try to open 600,000 files on your system. That’s an insanely huge number of open files and sure to cause you some other kind of heart ache. Since this is an FCP forum, I assume you have Final Cut. Use QT to create multiple time lapses with more reasonable number of files per video and then pull those videos into Final Cut and combine together.
Also be careful with the number of files you put into a single file system or directory. All filesystems hit a point where their performance starts to go down the tubes with huge numbers of files in a single directory, some with total number of files in the file system, regardless of directory location. If really want to store 600k+ files like this, you really need to read up on the internals of HFS+ (the Mac’s file system) and understand how best to accomplish this. My guess is that you’ll want multiple smaller file systems with few files in them. You may be able to also do this with a number of disk images — essentially nesting file systems on top of each other, but performance probably won’t be so great.
Good luck.
Reply to this Discussion! Login or Sign Up