New After Effects PC

[Ian Mapleson]

Mark writes:
> Thanks for all of the deep insights Ian!

Most welcome!

> This looks like quite the machine. …

Short of going expensive multi-socket, it’s about as good as it gets
apart from the GPUs used.

Note that I’ve just secured two GTX 580 cards, so I’ll be able to
test a wide range of configs soon.

> … My thoughts right now are that at
> current I’m going to stick with just one GFX card …

If so, then get a GTX 580 3GB with the highest core clock you can afford.

> … since I’ve been reading
> that After Effects only pulls from one GFX card for rendering. Can you
> confirm or deny this?

No, that’s not true at all. Ray Trace 3D mode uses multiple GPUs, as can
RAM Preview in many cases.

However, depending on the effects used, _sometimes_ the performance
scaling from multiple GPUs is not as one might expect. It depends on the
nature of the scene. I have one particular test scene (fire effects, 3D
letters, motion blur, lens flare, lots of particles) which imply that
perhaps the use of particles can affect how well AE exploits multiple
GPUs, but I’ve yet to confirm this – I need to set it up again and do
proper tests.

Certainly though, in numerous cases, multiple GPUs do scale the
performance nicely, which (if you have the option with respect to mbd
slots/spacing) means more flexible choices with what you can buy in terms
of how to spend your budget. For example, two cheaper cards might cost
less but outperform a single better card, though cheaper cards do tend to
have less RAM. This is why I initially fitted the system I built with two
800MHz 1GB GTX 460s (they only cost 87 UKP each). Combined with a Quadro
4K as the primary display card, they’re faster than a GTX 580. However,
as mentioned above, if one is doing something that can’t exploit multiple
GPUs quite so well, then performance may be limited to the power of a
single card, but this isn’t usually the case.

Ideally, AE itself would be able to monitor what’s happening with multiple
GPUs and adjust how it uses them accordingly, but I don’t think that
happens atm.

For example, see:

https://forums.creativecow.net/thread/2/1019120

I call that test, “CS6-11.1”, in my notes.

The original poster states they get 5 mins and 42 secs with their 3930K +
GTX 580 3GB (an expensive GPU).

At the time, my test systems were (both of these are 4-core with 8 threads):

ASUS Maximus IV Extreme
5GHz i7 2700K
16GB DDR3/2133
Quadro 4000 2GB
2x 800MHz GTX 460 1GB

Asrock X58 Extreme6
3.5GHz XEON X5570 (similar to an i7 950)
24GB DDR3/1600
3x 850MHz GTX 460 1GB

Render results were as follows:

2700K system:

1 Card: 17m 53s (rendering via the Quadro 4000 only)
2 Cards: 8m 35s (Quadro 4000 + 800MHz GTX 460)
3 Cards: 5m 59s (Quadro 4000 + 2x 800MHz GTX 460)

X5570 system:

1 Card: 11m 58s
2 Cards: 7m 15s
3 Cards: 5m 17s

So the first system comes close to a GTX 580, while the 2nd system beats
a GTX 580 (not surprising really).

In the case of the 3930K system I built, the two 800MHz 460s only cost 87
each – a lot less than a 3GB 580. Also, some models of GTX 460 can oc to
crazy levels (I have one that happily zips along at 1025MHz core), and
indeed the two 2GB cards I’m sending to the end user will be forced to
run at 800MHz instead of their default 700 in order to match the
performance of the old 1GB cards (I’ve modified them with extra heatsinks
on both sides of the PCB, etc. to cope with the extra heat under load).
Anyway, that’s a side issue, I just wanted there to be no performance
drop with the replacement cards. With the two 800MHz 1GB cards returned,
I’ll then have 4x 800MHz 460s for testing with the replica 3930K system
I’ve built, but I’ll run the same tests on a very different board which
can also use four GPUs (ASUS P7P55 WS Supercomputer with an i7 870, a P55
board with NF200 to support x8/x8/x8/x8).

In all cases, I’ve maxed out the RAM on the boards to allow for the best
possible render tests that involve a lot of RAM, to show what’s possible
with a more limited budget (P55, X58, P67, Z68, etc.)

> How much does this rig rely on overclocking …

If the task is CPU-bound, then obviously it relies a lot on the oc’d CPU,
eg. rendering via Classic 3D mode, but this is the big win with respect
to price/performance, because a 6-core SB-E at this speed easily
outperforms a much more expensive 8-core XEON. It certainly leaves my
dual-X5570 Dell T7500 completely in the dust.

> … – my goal would be to keep
> everything in as stock a configuration as possible …

IMO that’s just wasting free performance. At the very least one can
exploit what oc is possible without raising the core voltage.

That’s not to say some effort isn’t required to sort out an oc’d
configuration, but the effort is well worthwhile. I built the AE setup
in two days.

Having said that, I wouldn’t bother oc’ing a GPU like the GTX 580. It
already generates quite a bit of heat and uses a fair amount of power,
so the gains via oc’ing are less useful considering the impact on the
rest of the system. It would be more sensible to just add a 2nd GPU.

Likewise, I don’t oc the RAM. Instead, I just bought a kit rated high
enough (DDR3/2400) so that I could run the RAM at the desired speed (2133)
without any issues.

Btw, the end user of the AE system I built provided me with his own test
scene consisting of a number of moving ray-traced 3D words with multiple
reflections, etc., a less complex scene than the CS6-11.1 test above. I
can’t provide a screenshot yet, so I hope that description will suffice.
Anyway, the render scaling results were as follows:

2700K system:

1 Card: 4m 8s (rendering via the Quadro 4000 only)
2 Cards: 2m 2s (Quadro 4000 + 800MHz GTX 460)
3 Cards: 1m 28s (Quadro 4000 + 2x 800MHz GTX 460)

X5570 system:

1 Card: 2m 44s (1x 850MHz GTX 460)
2 Cards: 1m 36s (2x 850MHz GTX 460)
3 Cards: 1m 17s (3x 850MHz GTX 460)

Note that I obtained the same X5570/3-card result when testing with 3 x
800MHz GTX 460s on a simple P55 board with a 4.7GHz dual-core i3 550 (the
CS6-11.1 result was 6m 35s).

I actually signed up for an account on this site because I was planning
on writing a lengthy summary post for the above mentioned benchmark
thread, but your query prompted me to post early.

Hope this helps! 🙂

Ian.

SGI Guru

[Mark Laslo]

Thanks again Ian!

All of the above makes sense to me, just working out some last minute logistics issues if you don’t mind.

The issue I’m now seeing, as I’m sure you’re aware, is it is quite difficult to find new GTX 580’s except for the hydro 4 GB models which is not something I wish to tackle with this build. My company is not too keen on buying used cards so at the moment unless I can persuade them I’m looking at getting one of the GTX Titans since in the benchmark test that was linked earlier in the thread it only slows down the 11.1 test by 30-40 seconds. Any reason I should avoid this besides the fact it is more expensive than I could probably secure two GTX 580’s?

Also if I do secure two GTX 580’s do they need to run in SLI mode, or is it enough to just have them in the box?

My planned build now matches pretty much the specs you sent over

Case: Cooler Master HaF 932
Board: ASUS P9X79 WS LGA 2011
GFX: ASUS GTXTITAN-6GD5 GeForce GTX TITAN
PSU: CORSAIR Professional AX1200 Modular
Processor: Intel Core i7-3930K Six Core 3.2 Ghz
Cooler: Phanteks PH-TC14PE – hoping that will fit in the case, looks like it should
Internal HDDs: Samsung 840 Pro – 128 GB for Cache & 256 GB for Applications
Memory: 2x GSKill Trident X series 32 GB DDR3 2400
Monitor: Dell UltraSharp U2713HM

Let me know your thoughts!
Mark

[Ian Mapleson]

Mark writes:
> All of the above makes sense to me, just working out some last minute
> logistics issues if you don’t mind.

No prob! If you’d like more detailed info, feel free to email me (send to
mapesdhs@yahoo.com), I can explain a few other issues specific to a build
using that case, data that doesn’t belong here, eg. small fans for
cooling the chipset, where/how to put them, pictures, etc.

> The issue I’m now seeing, as I’m sure you’re aware, is it is quite
> difficult to find new GTX 580’s except for the hydro 4 GB models which is

I was really only referring to buying used. 😀

I’ve won two on eBay this past week. The first should arrive tomorrow,
the 2nd either tomorrow or Monday. See:

https://cgi.ebay.co.uk/ws/eBayISAPI.dll?ViewItem&item=171044128930
https://cgi.ebay.co.uk/ws/eBayISAPI.dll?ViewItem&item=200925198261

However, they are just the 1.5GB models. 3GB models are harder to find &
obviously usually go for higher amounts.

Are you in the UK? Or in the US/Canada? Or elsewhere?

The 580 is still available new in the UK, and there are some good deals
available from reputable sellers, eg. right now Scan has the EVGA 3GB 580
for 223 UKP as a refurb:

https://www.scan.co.uk/products/3gb-evga-geforce-gtx-580-classified-refurb

Hmm, might call the guy who bought the system I built, let him know…

> looking at getting one of the GTX Titans since in the benchmark test that

I suppose it does have the large RAM advantage.

> … Any reason I should avoid this besides the fact it is
> more expensive than I could probably secure two GTX 580’s?

Well, let’s see… (checking!) I’ll use UK prices here, but I expect the
same ratios would likely apply elsewhere.

A basic Titan is 800 UKP (837MHz), good ones about 888 UKP (876MHz). If
one is going to spend that much, may aswell get a good version, so let’s
say 900 UKP typical.

With some hunting around, you could maybe get 2 or 3 new GTX 580s for
that much, which would hammer a single Titan. For example, here:

https://cgi.ebay.co.uk/ws/eBayISAPI.dll?ViewItem&item=221230167382

However, something one must be careful of with these cards is how many
real slots they use up. Although the backplate of most 580s is 2-slot
(ASUS is an exception, one of their 580s does have a 3-slot plate), in
reality many of the coolers are so big that overall the card uses 3
slots. The above 3GB Gainward Phantom is typical, it uses 3 slots because
of the cooler. Thus, you’d only be able to fit two on the board. Of
course if one had the budget then one could use watercooling, but that’s
a whole other scenario. Anyway, at least for my research, I’ve been
deliberately choosing 1.5GB cards which definitely only use 2 slots.

Or instead of a Titan, you could get two GTX 780s, which would be about
a 3rd more than the cost of a single Titan, but obviously more than a
3rd better CUDA performance. OTOH the 780s only have half the RAM.

Having said that, if you did get a Titan, it would at least give you the
maximum upgrade potential for the future (4x Titan, blimey! Need a better
PSU though), and the luxury of large RAM. It’s just that in terms of
upfront initial performance, without a doubt 2 or 3 580s would be faster
and cost about the same or less.

I guess it’s a tossup between having better performance right now but
less RAM with multiple 580s, or a better upgrade path in the future & the
luxury of large RAM now.

Note I was going to mention the latest GTX 770, but I see those only have
2GB RAM. Nah, too much of a sacrifice really, though they are cheaper,
about 330 UKP each here, so for not really much more than the cost of a
Titan you could get three 770s, but the RAM would only be 2GB; hmm, I’d
rather expend the effort to find 3GB 580s.

> Also if I do secure two GTX 580’s do they need to run in SLI mode, or is
> it enough to just have them in the box?

No, do not use the SLI connector. Not needed for CUDA tasks.

> Case: Cooler Master HaF 932
> Board: ASUS P9X79 WS LGA 2011

Definitely email me about this, I can pass on some build tips. For
example, I wasn’t impressed with the stock rear 140mm fan in the 932, so
I replaced it with a much better Noctua NF-A15.

> PSU: CORSAIR Professional AX1200 Modular

Yup, that should do.

> Processor: Intel Core i7-3930K Six Core 3.2 Ghz

Remember to get the C2 stepping. Always ask the seller to confirm (ie. if
their specs are not absolutely clear). Ask for the SPEC code from the CPU
box, which you can check via ark.intel.com. The code should be ‘SR0KY’.

> Cooler: Phanteks PH-TC14PE – hoping that will fit in the case, looks like
> it should

Yes, it does; it’s what I have, though note the side panel 240mm fan will
have to be replaced with 2x 120mm instead on the lower half of the panel.
Naturally, I used AS5 for the HS attachment instead of the paste that came
with the fan. I use Articlean 1&2 for CPU/HS surface preparation, and
lint-free cloths for cleaning/wiping/etc.

And remember to get a 3rd fan for the Phanteks. The Black edition is
widely available now and it’s often the cheapest aswell (biggest demand).
Here’s a pic of my own board with the HS/fans in place before I put it
into the 932 case (bottom of the picture):

https://www.sgidepot.co.uk/misc/pcbenchmarking2.jpg

Email me, I can send you some build pics.

Likewise, for my own build, I put the PSU at the top, so the upper 240mm
fan had to be removed; I’ve installed an extra 120mm towards the front of
the case to act as an extra intake, giving fresh air for the CPU cooler.

Doing it this way means one can fit four 2-slot GPUs on the board. To
compensate for the missing top fan, I fitted two 120mm exhaust fans at
the bottom of the case (the side panel fans are intake). I use only Gelid
Wing Blue UV 12 fans for 120mm jobs, PWM editions where that’s sensible.
They’re normally available in a 4-pack for a good price (I invested,
bought 8 packs last autumn). If you live in a hot climate, faster fans
are available, but they’ll be louder.

> Internal HDDs: Samsung 840 Pro – 128 GB for Cache & 256 GB for Applications

Excellent!!

For general data storage, I recommend Enterprise SATA if you can afford
them (I struck lucky, bagged several lightly used drives locally for a
silly price). In the build I did for the AE guy, I fitted 2x 2TB
Enterprise SATA in RAID1 for his long term archive, and a 3rd just on its
own as the main work area disk (RAID speed just isn’t needed when working
with AE, ie. the key processing bottlenecks are CPU, GPU and cache I/O).

If you can’t obtain or afford Enterprise SATA, then stick with the upper
end of normal SATA such as WD Black (don’t bother with low-end SATA). For
Seagate, go for the models which have NS at the end, they’re more reliable
than the generic consumer AS models.

Oh, if you want a really nice & tidy way to hold the SSDs, I definitely
recommend the Startech 4-bay SAS/SATA enclosure. Holds 4x 2.5″ devices,
uses one 5.25″ bay. Looks like this:

https://cgi.ebay.co.uk/ws/eBayISAPI.dll?ViewItem&item=370668792942

Includes LED power/activity indicators on the front for each sled. I gave
one to my brother for xmas, here’s a pic of it installed in his Antec 300:

https://www.sgidepot.co.uk/misc/keithpc2.jpg

If you’re in the USA, Thermaltake makes a case that holds 6x 2.5″ devices
in a single 5.25″ bay (not available in the UK); I bought two, asked a
friend to post them over.

> Memory: 2x GSKill Trident X series 32 GB DDR3 2400

Note you’ll have to remove the top fins from each DIMM as they won’t fit
under the HS. Again, I have pics.

I’ve just fitted one of these kits to my P67/2700K setup; somewhat to my
surprise I didn’t have to change the BIOS settings at all, just up &
running straight away, 2133 as it was before with 4 x 4GB.

> Monitor: Dell UltraSharp U2713HM

Now that’s funny; after a morning of 2560×1440 monitor hunting, trying
to find something that supports sync-on-green so it’ll work with my
SGI stuff aswell, I bought that very model of Dell about an hour ago. 😀

Cheers! 🙂

Ian.

SGI Guru

[Ian Mapleson]

Hey Mark, how did you get on in the end? Did it all go ok?
I hope so!

Ian.

SGI Guru