Eric Fiegehen
Forum Replies Created
-
Alex – Cubix has not yet tested this GTX card with any of the Xpander models. I don’t anticipate any issues with using it, but keep this info in mind as you purchase products for use with Xpander. You might contact Mic Grover in Cubix Support (micg@cubix.com, ph 775-888-1000, ext 220) before making a decision one way or the other.
Eric Fiegehen
Director, Visualization & GPU Compute Solutions
Cubix Corporation
ericc@cubix.com
https://www.cubix.com -
Hi Chris,
Would you mind if either Mic Grover or I contact you about this configuration? Might also be able to solve the noise question (given it’s a Cubix fan and not the NVIDIA fans).
Eric Fiegehen
Cubix -
Hi Steff,
Keep in mind that PCIe works differently than other I/O transport technologies. Although all 4 Xpander slots share the same host PC / Mac Pro 16-channel slot, each of the Desktop 4 slots are receiving the full 80Gbps, 16-channel bandwidth to the host system’s bus. This is due to the way Cubix programs the timing sequences of the PCIe switch sets we use in our products.
Another way to put this is that when you perform benchmarks, each Xpander slot appears in the management utility as 16-channel PCIe receiving full bandwidth from the host system. If the theoretical 80Gbps maximum bandwidth were being used continually (without the application letting up on CUDA cores used and bandwidth maximized), then you would eventually see bus saturation occur over a period of time. However, Cubix has never experienced this condition running Resolve on OSX or Windows (I don’t believe Linux configurations have done this either)when connected at PCIe Gen2 16-channel speeds.
Please feel free to contact me or Cubix Technical Support for additional information on this or other related subjects.
Eric Fiegehen
Director, Visualization & GPU Compute Solutions
Cubix Corporation
ericc@cubix.com
https://www.cubix.com -
Keep in mind that, at least for some people,it might make sense to utilize both types of NVIDIA cards if you’re using the workstation to support multiple applications.
The same situation also applies to studios where multiple users with different applications would share 1 or more Xpander Desktop units. Each workstation would have an Xpander HIC installed in an open 16-channel slot, and users would connect and disconnect the box to multiple hosts as needed.
An Xpander Rackmount Series II or Xpander Elite, mounted in a mobile 19″ cabinet or cart, would also serve the same purpose.
Eric Fiegehen
Cubix -
Hi NE,
From Cubix’s perspective, yes the Xpander Desktop 4 part number you referenced will support the 2x GTX 580 cards you listed, plus two single-slot cards drawing less power than the 2x GTX 580 cards.
If you intend to go with additional GTX cards in the same Xpander, I recommend purchasing the standard, hi-flow exhaust fan from Cubix, a $25-$30 item.
As to your first question, I’ll check with my Engineering Department for input, but DaVinci and Avid would be able to answer this question more definitively.
Eric Fiegehen
Director, Visualization & GPU Compute Solutions
Cubix Corporation
ericc@cubix.com
https://www.cubix.com -
Todd – the following info comes from Cubix Technical Support in reference to your question:
“Cubix has not tested NVIDIA GeForce GTX 690 running in Xpander and connected to Mac Pro yet; however, this should work. Install the HIC in MacPro_Slot2. Install only two GTX 690s in Xpander Desktop 4, for example. Connect two 8-pin auxiliary power connectors to each 690. Connect only one display to only one 690. Boot the Mac Pro and install the Mac OS X driver version 304.00.00f20 dated 20120910 located here:
https://www.nvidia.com/object/macosx-304.00.00f20-driver.html.”Eric Fiegehen
Cubix Corporation
ericc@cubix.com
https://www.cubix.com -
Hi Todd – I’ll ask Mic Grover, Cubix Technical Support Manager to comment, but we recommend installing the GUI card in Slot 1 of the Mac Pro. Doing so provides maximum bandwidth for both hardware acceleration processes plus other types of I/O traffic such as reads and writes from 8-channel SAN or RAID cards which you might want to run from inside the Xpander.
Eric
-
Todd – keep in mind that OSX has a 4 GPU limitation. Think GUI + 3x CUDA accelerators. Other than this limitation, yes the Xpander would be able to host at least 2x GTX 690 cards.
Eric Fiegehen
Cubix -
Hi Peter,
I’ll probably need to clarify or amend this info later today or tomorrow after review by my SE or an Engineer, but there’s two parts to the answer.
Robert H basically has it correct in one of his earlier comments. Right now, the PCIe switch vendors are only shipping up to 48-lane Gen 3 chipsets in volume. This is why Cubix has not yet released a Gen3 expansion chassis – why bother when you can only get 1 additional Gen3 16-channel card attached (use one in the host, get 2 in return)?
The other part of the answer lies with the cards installed in those Gen3 slots. A Gen2 I/O controller will not operate at any faster rate than industry defined PCIe Gen2 speeds, even if a 16-channel Gen2 card is installed into a Gen3 16-channel slot. Also, you cannot expect a Gen2 16-channel I/O card to communicate at 80Gbps in a Gen3 8-channel slot, even though a Gen3 8-channel (electrical) slot has the capacity to communicate at 80Gbps, the equivalent of Gen2 16-channel performance.
That’s why if you plug a 16-channel Gen2 GUI or GPU card into an 8-channel Gen3 slot, even though that Gen3 slot has the ability to communicate at 80Gbps, maximum throughput will be 40Gbps since the I/O controller only has access to 8-lanes physical and electrical. Same condition comes up when you run out of Gen3 PCIe lanes – system architecture will limit you to a set number of lanes as specified by ASUS or whoever is manufacturing the motherboard.
My recommendation – again, without benefit of Cubix Engineering’s technical input as of writing this message – is to opt for a motherboard capable of supporting 4x (electrical) 16-channel Gen2 graphics cards, or 2x 16-channel Gen2 physical + Xpander. A great feature of PCIe is that Gen2 technology has progressed to the point that a single Gen2 slot can share its 80Gbps bandwidth with 3 more 16-channel slots, plus a couple of additional 8-channel slots. This is what we’re doing with the new Xpander Elite system shipping this week. 96 PCIe lanes let’s us daisy-chain 6 PCIe slots (4x dual-width 16-channel + 2x 8-channel), using up only 1 slot in the host computer. And yes, we designed it for running 4x GTX 580/680 cards plus a couple of RED Rockets, among other things.
Eric
-
“None of that Winblows hassle, great people doing modified cards for it and if you really need power, you get a proper Cubix 8X and put 3 GTX 580s in.”
Hi Sascha,
Glad to hear things are running well. Keep in mind that DaVinci recommends using the 16-channel versions of GPU-Xpander for Resolve 8+. Also, feel free to contact Mic Grover in the Cubix Support Department, micg@cubix.com, with any additional technical questions. Some of the PCIe lane comments I’m reading in this thread are a bit off the mark or incomplete in their analysis, and Cubix Support / Engineering can answer any PCIe technical question you might have.Eric Fiegehen
Cubix