-
Multiprocessing CS3 – AE Render Engine – AE with Apple QMASTER
Hello again guys!!
I’m working on an 12min AE project with chroma-key, and it takes 54h (aprox.) to render it out with the Animation codec, resulting in a 20GB file. I was using a 2.8Ghz 8 core MacPro with 6GB 800Mhz of RAM but recently i’ve upgraded the system, and now the mac is with 22GB RAM, ATi 4870 and an external OWC Mercury Rack Pro with 4TB running on RAID 5, but i can only render the same project with no errors following this settings:
Max Memory Usage 120%, Max RAM cache Size 35%, Enable Disk cache, Max Disk Cache Size 99000 MBThe Multiprocessing Enable and Output options are ON and if I try to increase the RAM Cache the render will fail (I also changed the After Effects 8.0 Prefs.txt to enable 7 cores).
After this upgrade I tried to render the project again, and guess what… it took the same time to do it, no improves were noticed, except the fact that during the render process, i checked the activity monitor and only 2 aeselflink were activated.
Recently I noticed that, before opening a project I should open AE first, and check on activity monitor until it loads the aeselflink’s 1 by 1, and after that, I open my project. When I try to render using that process, the 7 aeselflink’s remain active but only 2 or 3 work over 230% of cpu usage, while others stuck on 2% and start to shut, so after 5min only 3 aeselflink’s are active but none of them using more than 10% of CPU, at the same time that the AE 8.0 app is consuming around 290% of the CPU, with no render time improvement (still varying from 54h to 220h).
In short words: the multiprocessing is freaking me out! The weirdest thing is that when i create a new and simple project with other medias, all 7 aeselflink’s are used during the render process.
So i ask you guys if you could help me in this multiprocessing issue and i’d like to know if is possible to use the AE Render Engine in the cores, simulating every core as a machine in the network, without duplicating a 60GB project.
Also, is it possible to use the Apple Qmaster to create a kind of core cluster, because now i can do it using Compressor. Has anyone already used AE with Qmaster?Any info about multiprocessing, render engine and qmaster is welcome.
Thank you all in advance for reading this!
Marco.
-
3 Replies
-
hi marco,
a few things to make sure of first. make sure you have the latest update to cs3, it should be 8.0.2. also, if you are using keylight (or any third-party keyer or other effects) make sure you have the latest updates for cs3. for keylight you can download the latest update here (you’ll want version 1.2v8 for mac):
https://www.thefoundry.co.uk/pkg_downloads.aspx?ui=A5FB8E97-D752-4E01-AC7D-17B39B7E1192
as an fyi, keylight is multithreaded and will take advantage of multiple cpus without mp being enabled in ae. so it is possible that the render may not be much faster, if at all, if keylight is the only effect that you are using…
now, just to be sure, you are not running the powerpc version of cs3…? to check, select the after effects application in the finder (applications>adobe after effects) and choose file>get info. make sure that the little check box for ‘run under rosetta’ is not checked. you definitely don’t want to run the powerpc version of cs3 on an intel mac, unless you absolutely have to.
also, to avoid issues with your current preference settings, go into users/username/library/preferences/adobe/after effects and move the folder named ‘8.0’ onto your desktop. then run after effects, goto preferences>multiprocessing and enable mp (let’s not change anything else yet).
open your project, wait for all the processes to start up and try a render. use the activity monitor to check and see how things are going for a while.
if things are going well (no errors or crashes), then you might try enabling the disk cache, but make sure you set the cache to use the system (boot) drive. you’d like to keep the cache on a separate bus from your media array to try to avoid getting a data bottleneck on a single bus.
another thing that can effect performance and stability in ae, is the codec of the footage that you are working with in your comps. if you are using hdv, mpeg-2, mpeg-4 or h.264 then you should convert that footage in another app (an nle, perhaps) to a codec that uses only intraframe compression (sometimes called i-frame compression). apple’s lossless animation codec is a good one to use, but if you don’t want those large files eating up drive space, quicktime photo-jpeg at high quality (90-95) is very good, pro res 422 or avid’s dnxhd (free from avid) are good and dvcprohd will work too… of course image sequences (tif, tga, png, psd, jpeg) work very well to, if you want to go that route.
i don’t know of any way to get qmaster to work with ae, but i’m definitely no expert there…
Kevin Camp
Senior Designer
KCPQ, KMYQ & KRCW -
Hi Kevin!
Thank you for replying this thread.
Here are my software specs:
CS3: 8.0.2.27 Intel based, not Rosetta enabled;
Keylight was 15v5, now 15v8;I copied the folder 8.0 to my desktop and then all AE settings were reset, so I tried to render it again; but it failed. I had to change the settings for the following: 35% Max RAM cache size, disk cache disable and multiprocessing enabled, and set 6 active cores in the prefs.txt (eventhough I have 22GB of RAM, which is enough to run 8 cores, i would be happy with those 6 hehehe).
I always used the Animation codec, there is a plenty of space in the hard drives so I opt to have uncompressed files when possible (my psycho always tells me to do this way hehe).
Here are the facts: last night I ran a render test with the settings above without the Keylight v8 (just updated from v5 to v8, so i haven’t tried it yet but I don’t think this could be the problem, because in this 50h render, believe it or not, only the virtual set was on the queue, chromas were not being rendered out because their medias were already rendered in a Animation mov, in a kind of proxy process).
This test had ran for 11h with no errors with the disk cache disabled until I interrupted it this morning. 1300 frames had been rendered at that time in a total of aproximately 19000 frames.
In summary, disabling disk cache, 22GB of RAM, new VGA (ATi 4870) and 4TB Raid didn’t improved the render process and now it seems to be worse than before the upgrade! I don’t know what is going wrong and what to do to fix it.
I have chosen another disk to the outputs (overflow volumes) instead of the RAID (without it the render fails too), so it’s not being used for now, the disk cache even being disabled is set to the mac HD, and the final rendered file is being saved on another internal hard drive.
Checking the activity monitor, I noticed that only 1 aeselflink keeps opened after a while and this one doesn’t stress so much the processor and memory, while the AE app leeches aproximately 300% of processor. The strangest thing is that the 8-6 cores can be used in simple test project; would it be a plugin issue?!
Same plugins used in this project:
Magic Bullet Looks
Trapcode
Proanimator
Primatte Keyer Pro
Datamator
Red Giant Plugins…I just instaled CS4, so I’m going to make some tests in it and I hope it goes better than CS3, but before migrating to CS4 need to check all plugins so this could take a while.
This thread even targets for Qmaster in AE and Render Engine in the same machine suggestions. Any help is welcome!
Thanks in advance.
Marco
BRAZIL -
i think we are using similar machines. mine is a 2×2.8ghz (8-core) macpro, 12gb of ram and very rarely have any issues with cs3. i can say that the macpros are very picky about the configuration… one of our engineers here had installed my ram incorrectly and it gave me a lot of issues for the first few days until i reconfigured the ram.
since your problems began after your upgrade, how is your ram installation configured? you can see the configuration if you choose apple>about this mac, click more info… then select memory under hardware on the left side of the window.
i’m going to guess that it came originally with 2x1gb sticks. those would have originally been configured like this:
riser a / slot 1 – 1gb
riser a / slot 2 – empty
riser b / slot 1 – 1gb
riser b / slot 2 – empty
all other slots empty.if you had added 2x2gb sticks to that to get 6gb total, i think your config should have looked like this:
riser a / slot 1 – 2gb
riser a / slot 2 – 2gb
riser b / slot 1 – 1gb
riser b / slot 2 – 1gb
all other slots emptyto get to 22gb i’ll assume that you add 4x4gb sticks. the configuration for that should be like this:
riser a / slot 1 – 4gb
riser a / slot 2 – 4gb
riser b / slot 1 – 4gb
riser b / slot 2 – 4gb
riser a / slot 3 – 2gb
riser a / slot 4 – 2gb
riser b / slot 3 – 1gb
riser b / slot 4 – 1gbthe early 2008 macpros (and i believe the ones before that) have a relationship not just between slots 1 & 2 on the same riser, but also between riser a/ slots 1&2 and riser b/ slots 1&2. the same for the those risers for slots 3&4. so they will benefit from ram of the same size being installed in those interleaved slots. when that relationship gets broken, it can cause stability issues. i’m wondering if that may be why you need to restrict your ram to only 35% to get stability…
another thing you might want to do is run the apple hardware test on the original osx install disc instruction to run it are printed on the disc itself, run the extended test to fully test your ram — note it may take 30 min to 1 hour to run this test with 22gb of ram, but it will tell you if you have any bad ram.
Kevin Camp
Senior Designer
KCPQ, KMYQ & KRCW
Reply to this Discussion! Login or Sign Up