-
TARGA in AE equals excessive render time
Let me state first that I am in this business for quite some time. I don’t want to make it sound like this is my first rodeo.. 🙂
A familiar topic but I was hoping to get some help / advise on how to handle this.
Basically I have a big local client that is part of a big global company. Normally I provide the needed videos for this, but something global sends something – for branding purposes – that need to be adapted locally.
About 2 weeks ago I received a 200GB After Effects project. Now big projects don’t scare me but as soon as I opened it and my iMac (i7 3.4GHz 16GB) started chocking on the project I got a heart attack. I don’t even know where to start explaining….
Basically there are a couple of comps that need to be rendered and in these comps there are some pre-comps. In the precomps there are about 15 layers of TARGA sequences. I can’t RAM preview it.
When I started digging I found a couple of comps that where using proxy files. Now those files wheren’t delivered in the project (First question: If you collect footage, does it include the proxies and pre renders?)
Since I couldn’t find the pre-rendered files I decided to render them myself. -> I got another heart attack. Currently I am pre-rendering a 750frame comp and after 5 hours i managed to render 180 frames. After Effects is struggling to respond (but its processing so that makes sense). In activity monitor I see that my CPU usage is 120-220% (AE), yet only one core is being fully used.
Question 2: Is there a way to optimise the performance?And lastly. Obviously I contacted the client, telling them there are items missing in the project (Pre-renders) and that I want to get them from the creator. They contacted the agency and said its not available and if they have to re-render that will ask a lot of money
How would any one handle this?
I really appreciate your inputs.
Regards,
Paul -
5 Replies
-
First, a general point: Targa files can be pretty big, especially if RLE compression was turned off. Without compression A 32 bpp (that is, 8bpc RGBA) TGA for a 1920×1080 frame is almost 8 MB. If you have to read 15 of those, that’s 120 MB per frame. If you have, say, 4 precomps and each precomp has 15 layers, that’s 480 MB per frame — or 14 GB/s.
That means that your storage can very quickly become the bottleneck — the computer simply can’t read from the disk fast enough. What disk are you storing these TGAs on, and how fast is it?
If your storage is a bottleneck, you will likely find that you can improve performance by turning multiprocessing off. It’s a little counter-intuitive, but allowing multiprocessing means multiplying the amount of storage I/O, and once you saturate your storage, multiprocessing can’t help anyway.
[Paul Emous] “Basically there are a couple of comps that need to be rendered and in these comps there are some pre-comps. In the precomps there are about 15 layers of TARGA sequences. I can’t RAM preview it.”
Do you get an error message when you try?
[Paul Emous] “When I started digging I found a couple of comps that where using proxy files. Now those files wheren’t delivered in the project (First question: If you collect footage, does it include the proxies and pre renders?)”
Pre-renders are always included with collect files. Pre-renders are regular footage items; they are not special. When you pre-render something, the rendered file is imported back into the project, and all usages of the pre-rendered comp are replaced with the file.
Proxies may be optionally included or excluded from collect files via the “Obey proxy settings” checkbox.
[Paul Emous] “Question 2: Is there a way to optimise the performance?”
There are numerous suggestions on the Improve performance [link] page, but I think the first thing we should address in troubleshooting this is your storage configuration.
Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog – What I’m thinking when my workstation’s thinking
Creative Cow Forum Host: Live & Stage Events -
Hi Walter,
Thank you for your reply. Very much appreciated!
I very much agree with you that storage looks like the potential problem here. I was running this from a FW800 hard disk and then store the output on a different location. Unfortunately due to other projects I had to shift this nightmare over too a dedicated Mac Mini i5 with FW800 hard disks.
If I look at the hard disk I/O its not under very high load. Also it currently uses (only) 70~80% RAM
13h25m on the clock and I rendered 193 frame (out of 750)…. What a nightmare.
-
[Paul Emous] “If I look at the hard disk I/O its not under very high load.”
How are you measuring load? It’s not enough to consider bandwidth — the disk queue length (number of unfulfilled requests to the disk) is important. I am not in front of a Mac now, but I think running “su iopending” in the Terminal will tell you the length of the disk queue.
Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog – What I’m thinking when my workstation’s thinking
Creative Cow Forum Host: Live & Stage Events -
I have no idea how to read this…
2013 Nov 27 02:52:14, load: 1.95, disk_r: 680 KB, disk_w: 387048 KB
value ————- Distribution ————- count
< 0 | 0
0 |@@@@@@@ 3648
1 |@ 590
2 | 236
3 | 118
4 | 6
5 | 6
6 | 10
7 | 6
8 | 14
9 | 14
10 | 18
11 | 79
12 | 71
13 | 60
14 | 125
15 |@@ 756
16 |@@@@@@@@@@@@@@ 6987
17 |@@@@@@@@@@ 4952
18 |@@@@ 1860
19 | 142
20 | 26
21 | 19
22 | 61
23 | 20
24 | 84
25 | 12
26 | 10
27 | 12
28 | 18
29 | 12
30 | 26
31 | 0 -
iopending shows a histogram of the number of pending disk transactions over the period of time it samples. A pending disk transaction is any read/write request that has been made but not yet fulfilled by the disk.
Persistently high queue lengths over 1 indicate disk saturation. Your queue length is very often between 15-18.
Your render cannot go any faster because the system can’t read frames from the disk any faster than it is. This can be improved by placing the footage on a faster disk — internal SSD, miniSAS RAID, Thunderbolt SSD/RAID, etc.
Again, if multiprocessing is on, you should try turning it off as that will lower the number of simultaneous disk requests.
Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog – What I’m thinking when my workstation’s thinking
Creative Cow Forum Host: Live & Stage Events
Reply to this Discussion! Login or Sign Up