TARGA in AE equals excessive render time

[Walter Soyka]

First, a general point: Targa files can be pretty big, especially if RLE compression was turned off. Without compression A 32 bpp (that is, 8bpc RGBA) TGA for a 1920×1080 frame is almost 8 MB. If you have to read 15 of those, that’s 120 MB per frame. If you have, say, 4 precomps and each precomp has 15 layers, that’s 480 MB per frame — or 14 GB/s.

That means that your storage can very quickly become the bottleneck — the computer simply can’t read from the disk fast enough. What disk are you storing these TGAs on, and how fast is it?

If your storage is a bottleneck, you will likely find that you can improve performance by turning multiprocessing off. It’s a little counter-intuitive, but allowing multiprocessing means multiplying the amount of storage I/O, and once you saturate your storage, multiprocessing can’t help anyway.

[Paul Emous] “Basically there are a couple of comps that need to be rendered and in these comps there are some pre-comps. In the precomps there are about 15 layers of TARGA sequences. I can’t RAM preview it.”

Do you get an error message when you try?

[Paul Emous] “When I started digging I found a couple of comps that where using proxy files. Now those files wheren’t delivered in the project (First question: If you collect footage, does it include the proxies and pre renders?)”

Pre-renders are always included with collect files. Pre-renders are regular footage items; they are not special. When you pre-render something, the rendered file is imported back into the project, and all usages of the pre-rendered comp are replaced with the file.

Proxies may be optionally included or excluded from collect files via the “Obey proxy settings” checkbox.

[Paul Emous] “Question 2: Is there a way to optimise the performance?”

There are numerous suggestions on the Improve performance [link] page, but I think the first thing we should address in troubleshooting this is your storage configuration.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog – What I’m thinking when my workstation’s thinking
Creative Cow Forum Host: Live & Stage Events

[Paul Emous]

Hi Walter,

Thank you for your reply. Very much appreciated!

I very much agree with you that storage looks like the potential problem here. I was running this from a FW800 hard disk and then store the output on a different location. Unfortunately due to other projects I had to shift this nightmare over too a dedicated Mac Mini i5 with FW800 hard disks.

If I look at the hard disk I/O its not under very high load. Also it currently uses (only) 70~80% RAM

13h25m on the clock and I rendered 193 frame (out of 750)…. What a nightmare.

[Walter Soyka]

[Paul Emous] “If I look at the hard disk I/O its not under very high load.”

How are you measuring load? It’s not enough to consider bandwidth — the disk queue length (number of unfulfilled requests to the disk) is important. I am not in front of a Mac now, but I think running “su iopending” in the Terminal will tell you the length of the disk queue.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog – What I’m thinking when my workstation’s thinking
Creative Cow Forum Host: Live & Stage Events

[Paul Emous]

I have no idea how to read this…

2013 Nov 27 02:52:14, load: 1.95, disk_r: 680 KB, disk_w: 387048 KB

value ————- Distribution ————- count
< 0 | 0
0 |@@@@@@@ 3648
1 |@ 590
2 | 236
3 | 118
4 | 6
5 | 6
6 | 10
7 | 6
8 | 14
9 | 14
10 | 18
11 | 79
12 | 71
13 | 60
14 | 125
15 |@@ 756
16 |@@@@@@@@@@@@@@ 6987
17 |@@@@@@@@@@ 4952
18 |@@@@ 1860
19 | 142
20 | 26
21 | 19
22 | 61
23 | 20
24 | 84
25 | 12
26 | 10
27 | 12
28 | 18
29 | 12
30 | 26
31 | 0

[Walter Soyka]

iopending shows a histogram of the number of pending disk transactions over the period of time it samples. A pending disk transaction is any read/write request that has been made but not yet fulfilled by the disk.

Persistently high queue lengths over 1 indicate disk saturation. Your queue length is very often between 15-18.

Your render cannot go any faster because the system can’t read frames from the disk any faster than it is. This can be improved by placing the footage on a faster disk — internal SSD, miniSAS RAID, Thunderbolt SSD/RAID, etc.

Again, if multiprocessing is on, you should try turning it off as that will lower the number of simultaneous disk requests.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog – What I’m thinking when my workstation’s thinking
Creative Cow Forum Host: Live & Stage Events