[Ian Mapleson]

I’m intrigued by your mbd/CPU setup; what model XEONs do you
have on the board? How did you find the experience of oc’ing
them compared to normal boards? Any issues specific to working
with two XEONs? Also, is the RAM ECC, and at what speed is it
running? Nothing to do with CUDA of course, but your system is
similar to something I’ve been discussing with a friend.

Ian.

SGI Guru

[Ian Mapleson]

Marco writes:
> for my use and ‘better off buying a picture k600 or card type nvida
> geforce gtx?

I suppose it depends on what you can afford. Personally I would recommend
a platform upgrade since at this level of performance you’re going to be
hitting bottlenecks with RAM capacity/speed, and the CPU.

I’d recommend a Quadro card for the primary display (more reliable,
better viewport performance, precision, etc.) and a separate card (or
cards) for CUDA. My 3930K setup has a Quadro 4000 and three GTX 580s.
If you can only use one card though, then a GTX 580 is the best option.
I make no assumptions as to space issues in your system though.

> GTX (xxx) Ti with Cuda vs. HD 6850 or similar with OpenCL ??

Best card for CUDA is the GTX 580. Get a 3GB model if you can, but even a
1.5GB model is a far better option for CUDA than any of the 600 series.
Indeed, the 580 can beat the 700s most of the time for CUDA. They’re
quite cheap too now, especially 2nd-hand. I’ve bought ten of them in
recent weeks, four of them being 3GB versions.

> release that you can insert without major physical obstacles, and
> you can feed with my power supply atx 600 watt

Ah, again we have more possible bottlenecks. A 580 is a large card, and
personally I’d want a better PSU than a 600W no matter what newer card
one used. My own rule for powerful cards is: 750W+ for one card, 1kW+ for
2 cards (though for lesser cards these are fine for running 2 or 3
respectively, and I’ve run three 580s with a 1kW PSU before, on a 5GHz
2700K system).

> unfortunately I can not spend as much, and I ask you what ‘s the best
> video card that I can take as value for’ price?

Definitely a GTX 580. Get a used one. Your PSU bothers me though. Q9550
isn’t exactly light on the power usage.

> already ‘that there are, I ask you the model and brand of some laptops

I’d never use a laptop for this kind of work. Waste of money IMO.

Overall though, I’m not sure upgrading a platform that old is so wise.
Your CPU, etc. won’t really be able to get the most out of a good card
like a GTX 580, especially if it’s limited by having so little RAM.
Lots of RAM for AE is essential.

Honestly? Save up, replace the system. Get a used Z68 board, max the
RAM to 32GB DDR3/1600 or 1866, move over the SSD, better PSU, GTX 580
for gfx, used 2500K is good value. Much better idea IMO than trying to
squeeze more out of your old platform.

Feel free to email/PM me, I can give you specific examples of the items
I’ve obtained, etc.

Hope this helps!

Ian.

PS. By ‘best’ card for CUDA I did of course mean from your point of view
with respect to value, etc. In absolute terms the best is a Tesla, or
a Titan with a display output, but that’s a whole different league of
pricing pain. :}

PPS. Sorry I couldn’t reply earlier! I’ve been on holiday for the past
2 weeks. 😀

[Ian Mapleson]

It’s all to do with aggregate bandwidth per CUDA core and
other differences. See:

https://www.tomshardware.com/reviews/geforce-gtx-760-review-gk104,3542-19.html

A 770 or 780 has lots of cores, but they don’t have
correspondingly higher bandwidth to feed them all.

Ian.

PS. Edit: I’ve been told by C. Angelini @ toms that the
shaders in the 580 run at twice the clock rate of the
later shaders used in 600/700 series cards, so that’s
another major factor for the strength of the 580. He
said NVIDIA switched to having a larger number of lower
clocked shaders because that made thermal issues easier
to deal with, but of course it means a newer card must
have a lot more shaders to beat a 580, which some do,
but then they don’t have the mem bw per core to match.
Must be a combination of these factors I suppose.

SGI Guru

[Ian Mapleson]

Also disable drive indexing. Full guide here:

https://www.tomshardware.com/reviews/ssd-performance-tweak,2911.html

Ian.

SGI Guru

[Ian Mapleson]

Jean Piche writes:
> Dual Xeon 8-core (E5- 2687W) @ 3.1Ghz, liquid cooled

Sweet setup! 8)

> 256 SSD boot/app drive

I hope you have some kind of backup strategy for this. At
the very least, backup to a mechanical drive somewhere
using Macrium Reflect or something.

As for the SSD itself, I recommend the Samsung 840 Pro,
OCZ Vector or OCZ Vertex4. If pushed for one, I’d say
go for the Samsung, though I’d be happy with any of them.

> 64Gb flash cache

(this is for the AE cache, yes?)

Only 64GB? Get at least 128GB, there’s a definite
performance advantage, and longer term write indurance.
Personally, I’d just get another 256GB.

> 14Tb external array

Is this a SAS setup? If not, try and get Enterprise SATA
if possible, or at the very least the more reliable versions
of normal SATA (Seagate models with NS, or others marked as
24/7 editions; definitely not any eco model). Recommend you
use RAID10 (not RAID5!!). Don’t use RAID0, too risky.

> Nvidia GTX780, 3GB graphics

Given the cost of this, I think it’s a better idea to have
two or three used GTX 580 3GB cards instead, which would be
cheaper (for 2) or about the same (for 3) and much faster.
Unless of course you want to buy all-new parts, but I’ve
always been one for sensible cost savings where possible.

With additional initial performance, perhaps this would
enable you to scale up to multiple Titans/Teslas later.

> Is this overkill for CS6 (+ Touch Designer) under Windows7?

Nope, one can never have enough compute power for AE. 😀

One question: what mbd will you be using? How many PCIe
slots does it have? Would be good if you could use a board
that allowed you to expand up to 4+ GPUs in the future.

Ian.

SGI Guru

[Ian Mapleson]

That correlates nicely with my own testing; a Quadro 4000 on
a P67 board with 5GHz 2700K gave 17 mins 53 secs.

Ian.

SGI Guru

[Ian Mapleson]

Thanks for the extra info Teddy! Yup, the H100 (or newer H100i)
is good, assuming one has an appropriate case. Not sure about
the H80, that might be tad too far down the scale.

Várbjørnin, if you do use a water cooling kit such as an H100,
remember to ensure that extra dedicated cooling is somehow
directed at the mbd chipset areas, since normally an air
cooler naturally tends to blow some air over these parts; see
my earlier posts for examples of how this can be done with an
air cooler like the large Phanteks PH-TC14PE.

Teddy’s right about oc’ing btw, it really is just free
performance waiting to be exploited. At the very least, you
could work out a suitable overclock which does not require any
extra core voltage (and thus no change to power/cost ratios).

Ian.

SGI Guru

[Ian Mapleson]

Most welcome! Do hunt around for other opinions though since
I don’t know how much of what I said would apply to Premiere
and Resolve.

Ian.

SGI Guru

[Ian Mapleson]

Várbjørnin writes:
> Hey guys. Me and my partner are planning to invest in a new computer
> for video editing, mostly used for AE, Premiere and also Resolve. We

What case are you going to use for this build? I recommend something
sizeable with plenty of potential airflow. I used the Coolermaster HAF
932, with the PSU positioned at the top of the case to permit 4 GPUs to
be installed.

> RAM – Kingston 64GB DDR3 1600MHz (8×8)HyperX

Hmm, not what I would choose. GSkill would be better IMO. I’ve never had
good experiences with Kingston RAM (all too often I found they just would
not run at their rated speed, whereas GSkill and Mushkin had no problems).
Also, given memory bandwidth is useful for AE, two kits capable of 1866
or 2133 would be preferable. I decided to ensure extra stability by getting
kits rated at 2400 so that I could set them at 2133 without worry, though
ironically at the time the 2400 kits were cheaper than 2133 kits.

> CPU – 1x Intel Core i7-3930K Sandy Bridge-E

Make sure you get the C2 stepping. I think the relevant SPEC code is SR0KY.

> CPU Cooler – Phanteks PH-TC14CS CPU Cooler – Blue

Are you going to oc the CPU? If so, then get the Phanteks PH-TC14PE instead.

> STORAGE – Western Digital WD Green – 3TB 2x

Eek! No! Such disks are not designed for professional use, and buying
disks aimed at power saving fanatics is a bit pointless when you’re
building a 6-core system with 64GB RAM and a Titan. 😀 You’re just
asking for a disk failure with disks like the Green. At the very least,
get WD Black instead, or the Seagate models that end with NS (more
reliable than consumer AS). I went a step further and managed to obtain
some 2TB Enterprise SATA drives.

> MOBO – ASUS Rampage IV Extreme

That’s really a board designed for gamers; its features reflect this.
For long term investment in an AE setup, I would have thought the newer
version of the board I’m using is better, namely the ASUS P9X79 WS-E
(mine is the older ASUS P9X79 WS). See:

https://www.tomshardware.co.uk/ASUS-LGA2011-X79-P9X79-E-WS,news-44472.html

Compare the WS-E to the Rampage:

https://www.asus.com/Motherboards/P9X79E_WS/#specifications
https://www.asus.com/Motherboards/RAMPAGE_IV_EXTREME/#specifications

The WS-E has a better and more flexible PCIe configuration, and it has
more 6Gbit SATA ports. It also supports XEONs, though that would only be
relevant if Intel bothers to release an 8-core IvyBridge-E at a decent
clock speed later this year (unknown atm); point being, Rampage boards
don’t support XEONs.

Both boards have the same overclocking features. The WS-E costs more,
though even the older WS would be more appropriate than a Rampage IMO.
I spent many days examining a wide range of boards for use with AE,
including the Asrock X79 Extreme11 (because it has onboard SAS), but
in the end I felt the WS was the best choice; today it’d be the WS-E.

> PSU – Corsair AX1200i – 1200W PSU

That should be ok to begin with. Do you think you’ll ever expand the
system to have four gfx cards for max CUDA performance? 1200W is probably
ok for 2 or 3, but I’d want something a bit better to handle 4, especially
if you plan to oc the CPU.

Looking around, I see that the PSU I bought recently is pretty much the
same price as that Corsair, so why not get a 1475W Thermaltake Toughpower
XT Gold instead? Then you’d have plenty of headroom for the future. See:

https://uk.thermaltake.com/products-model.aspx?id=C_00001812

Certainly though, to begin with, any PSU at this level will be just fine
for a 3930K and just one GPU. Best to plan ahead though.

> What we are most unsure of, is the gfx choice. Is our Titan choice the
> best card for pure performance for AE, premiere and resolve?

(note that I can’t comment on Premiere and Resolve, I’ll leave that to
others, but with respect to AE…)

That depends on your performance priorities. For a single card, it’ll
certainly give the best performance for any task that only uses the
primary GPU (perhaps others can comment on how this relates to using
Premiere/Resolve because that’s beyond my knowledge, I’ve only been
researching AE), and it means the maximum unused slots for future
expansion. On the other hand, two older GTX 580s would cost less but be
faster for any task that can use multiple GPUs, such as CUDA rendering (a
single Titan would use less power, but take longer to complete a task).

One thing though, Titan does have the option of operating in 64bit mode
for which 1/3rd of the cores are available. By contrast, all the other
cards (including the 780) only provide 1/24th of the cores for 64bit.
I’m not sure yet how this difference might translate into better workflow
or rendering with AE, but certainly for the benchmark discussed on this
page it doesn’t seem to help, assuming of course that those with Titans
who have posted here have checked to see if there’s a difference between
standard and 64bit mode for running the test. Alas I don’t have a Titan
yet for my testing, so I can’t say.

Mind you, how realistic is the test used here? It’s really the same
general image rendered multiple times. I’m working with someone on a more
complex & varied test, based on real commercial work. The goal is to have
a test suite that consists of a single-frame render (typical for when one
is creating a large billboard poster advert, for example) and then an
animation render test, which tomshardware will use in their CUDA review
tests. Someone for whom I built an AE system is designed the tests, might
be ready this week or next perhaps.

> How does the 690GTX fare against the TITAN? …

Similar issues when comparing against two 580s or a 590, though note the
580 is _faster_ than the 680 for CUDA (the 600 series suffers from
inferior memory bandwidth issues). Indeed, the 580 beats many of the 700
series cards in a lot of CUDA tests.

In the past, AE had difficulties exploiting both GPUs on a dual-GPU
card. Not sure if this is still the case. Have a look at the results
here, see if those posting 590/690 results show any relevant speedup
compared to a single 580/680.

> … And finally how do the
> Quadros 4000-6000 compare to the two mentioned geforces?

Quadro cards offer better display port performance, but it seems likely
this is less relevant for AE given the way many functions are GPU
accelerated (don’t know about Premiere/Resolve). However, Quadro cards
have far lower CUDA performance as they have fewer cores, so a Quadro
would not be as good as a Titan for GPU accelerated rendering. Other
differences include driver optimisations, reliability, warranty level,
etc., plus & minus things on both sides – see Teddy’s recent post for a
summary, and my followups.

Absolutely without question though, if you only want to get one card to
begin with, then don’t get a Quadro as it’ll be nothing like as good as a
Titan, 2×580, etc. for GPU-accelerated functions.

Overall, if you do want to go for max CUDA performance while minimising
initial cost (at the expense of power consumption), then multiple GTX
580s are definitely best (find 3GB editions if you can). 1.5GB cards are
going pretty cheap on eBay – for less than the cost of a Titan, you could
get four 1.5GB cards, but this means initially more limited VRAM (mind
you, the faster performance means you could get tasks done quicker,
perhaps help to bring in money faster, upgrade to multiple Titans sooner;
just a thought). Alternatively, and even better, get a Titan as the
primary card (buy it new), and add two 2nd-hand 3GB GTX 580s for extra
CUDA oomph, when you can find them. You can always replace the 580s later
with more Titans.

Ian.

SGI Guru

[Ian Mapleson]

Teddy Gage writes:
> AE being, of course, another exception.

I assume you mean from a performance perspective, yes; presumably such users would
still benefit from the other differences you mentioned.

> … Although it really depends on a) what your definition of “faster” is …

The results are pretty clear for the Viewperf tests.

However, Viewperf can indeed be misleading and should be used with care. For
example, those tests do not take into account situations where some kind of
host processing is required which is unrelated to the 3D rendering pipeline
in any direct sense. I recall an oil rig database which, for every frame in a real
time vis sim, the database info had to be converted into Performer for rendering.
This required a lot of preprocessing (8 CPUs working together). Likewise, as I’m
sure you know, AE can gobble huge amounts of RAM, whereas Viewperf 11.0 only needs
8GB to run ok.

But as a basic 3D comparison in those apps where RAM/CPU is less critical, the
differences can be enormous. The ProE, TCVis and SNX results are particularly stark.

> … I wasn’t aware of the massive difference in viewport performance for maya etc. …

Maya seems to be an inbetween case. I’ve not checked for a while, but I bet the
situation I found many years ago is still the case, ie. if one examines the individual
test results that make up the SPEC Maya suite, gamer cards will be good at some things,
but bad at others, the combination of which evens out the final average. Alas it’s still
not enough for a gamer card to outpace a Quadro, just look at how a GTX 580 compares
against a Quadro 4000.

What I was told long ago is that the Quadro drivers provide strength for functions such
as antialiased lines (not needed in games), while gamer drivers are strong for other
things such as 2-sided textures (generally not used in pro apps). Lots of new features
since then, but I expect similar things still apply.

> Now whether that built into the drivers or is a factor of unlocked hardware is
> hard to tell. I would favor an answer in software. …

Probably both.

> But I would disagree about the Titan being overpriced. It is really an amazing

It’s fast, but it’s not sufficiently faster than a 780 to warrant the price
difference IMO, and the 64bit fp lockout on the 780 is deliberate (I view this
as the 780 not having something it should have, rather than Titan having something
it normally wouldn’t). Likewise, the use of a narrow bus in all these recent cards
means the cores cannot be fed fast enough to fully exploit them, hence the bizarre
differences in CUDA performance vs. a 580 which has massively fewer cores:

https://www.tomshardware.com/reviews/geforce-gtx-780-performance-review,3516-26.html

Or to put it another way, with respect to AE, how is it possible for two 580s to
be so much quicker for your test than a single Titan? It’s all about the mem bw.
NVIDIA could have made Titan a massively quicker card by using a wider bus, but
as I say that would harm Tesla/Quadro sales. My own results are quickly leading me to
the conclusion that overall accelerated GPU performance in AE is all about balanced
aggregate bandwidth per CUDA core, though it’s also scene-dependent.

> … Especially once you’ve seen what it can do on multi-monitor 2k openGL performance, …

On that front you’re absolutely right. 😀

In the same way though, why is there no 6GB 780? As a gamer card it’s often little
slower than a Titan, eg.:

https://www.tomshardware.com/reviews/geforce-gtx-780-performance-review,3516-9.html

Likewise, why does the 770 only have 2GB RAM? (though there’s a Gigabyte with 4GB)

NVIDIA is being very careful how it positions these products, so it won’t harm its
pro card sales too much. It’s akin to Intel making the 2500K far too good for oc’ing
(took me less than 3 minutes to get mine to 4.8GHz stable); those with that chip see
no reason to upgrade, so Intel used a garbage TIM with IB so it would run a lot hotter
(replace the TIM and temps can drop by up to 30C). If NVIDIA released a card like the
780 with a 512bit bus (etc.), it would be a price/performance CUDA killer, in which
case bye bye lots of Tesla/Quadro sales. The only things that hold this back in other
areas are ECC RAM and issues such as the restricted GPU to host return path in gamer
cards for CUDA.

I know someone who does a lot of CUDA programming for financial transaction processing;
he’s explained many of these issues to me in recent months.

> … Add to this the ability to enable double precision floating point calculation. …

Indeed, but I guess my viewpoint tends to approach this from the other direction; most
will regard it as ‘good’ that the Titan has this feature, and I agree, but my immediate
reaction is it’s bad that other cards don’t have it when clearly they could have,
ie. the lockout is deliberate. The potential is there, but not permitted.

> I honestly believe GPU rendering is the future, and this card can compete and outperform
> a $3000 “pro” equivalent on that front, so I think it is a great value. …

Definitely agree there, when it comes to GPU acceleration these cards are great, though
it’s a pity NVIDIA ignores other aspects of the design which mean their cards continue to
badly trail AMD in some areas, and again show the strength of the older 580:

https://www.tomshardware.com/reviews/geforce-gtx-780-performance-review,3516-27.html
https://www.tomshardware.com/reviews/geforce-gtx-780-performance-review,3516-28.html

Imagine if NVIDIA simply activated the 64bit 1/3rd option on the 770/780. But they won’t.

> Just maybe not to your average gamer who just wants to run the latest call of duty.

As it happens though you’re actually right again there, because the Titan’s big
RAM means in many cases it’s a better option for multi-screen gaming at high
resolutions, ie. the scaling with Titan SLI works far better than two 690s or
multiple 7970s, etc. If one could get a 780 with 6GB RAM though, that would be
more sensible than Titan given its enormously lower price.

What AMD doesn’t have is a product akin to Tesla. I guess as long as that’s the case,
NVIDIA has no need to produce a version of GK110 that really is as good as it can be,
ie. consumer-type RAM setup with restricted return path, but full 64bit activation
and a wide memory bus to fully feed all its cores. Such a card would blow everything
else away for CUDA (it would be at least 2X faster for AE than the current Titan),
but such a card would also be too good – NVIDIA just doesn’t need to produce
something like this yet, it’d eat too much into Tesla/Quadro sales.

Personally I think it’s sad that SGI’s old ethos of gfx product design is gone. Think
back to the days of MaxIMPACT; it was seriously expensive, but when it launched it was
10X faster than anything else available, so it sold like hotcakes because the price
was well worthwhile. Ditto for IR. Their philosophy in those days was, what can we make
with a budget of $10K? Or $50K? Imagine that now: what could NVIDIA build if other
factors (such as harming other areas of their own product range) didn’t matter? How much
would you be willing to pay for such a thing? Imagine Titan with a 1024bit bus, even
more RAM, full 64bit activation and no restricted return path – it’d cost more, but not
that much more, though they could price it way higher no problem.

Alas it won’t happen. Not unless AMD does something unexpected in the pro space which
suddenly pushes available OpenCL performance massively ahead.

Ian.

SGI Guru