• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

CUDA and render machine with four graphics cards

K

Kepakiano

n00b
Joined
Feb 11, 2014
Messages
5
Hi there,

I'm building a system that should be able to render to twelve monitors with a resolution of 1400x1050 pixels each and serve as a CUDA machine. The system will run on linux.
My idea was to put four graphics cards into it so that each card provides for three displays.

I thought of the following components:
Motherboard: ASUS P9X79-E WS or ASRock X79 Extreme11, socket 2011
CPU: Intel Core i7-4930K (6x 3.0GHz)
GPUs: 4x GTX 780. Either Inno3D's Accelero Hybrid with water cooling or a different GTX 780 model with a radial fan
RAM: 32GB oder 64GB
Netzteil: LEPA G 1600W
Gehäuse: Antec P280
HDD: Don't know, two 2TB drives for a software raid, I guess.

This raises some questions:

1. There are two PLX PEX 8747 to turn the 40 PCIe Lanes provided by the CPU into the 68 Lanes I need to have x16 PCIe 3.0 for every GPU. Does this affect CUDA unified memory in any way? Will it be possible to address the complete combined RAM of CPU and GPUs?

2. I wrote an email to Inno3D, but, sadly, they did not reply. But maybe someone here can help me: The specs of the Accelero Hybrid state that it's a triple slot card (http://www.inno3d.com/product/ichillgtx780lcs.html). But I cannot see why. In the images they provide it seems to be a dual-slot card. Does someone know for sure? I would really like to put four of these into the system, because water cooling would reduce the heat problem significantly!

3. What are your generals thought on the system? Will it work the way I plan?

Thank you for your time and maybe answers.
 
1) Probably not but I'm not an expert on CUDA
2) Part of the heatsink (well mainly the tubes) actually sticks out past the second slot. Hence why it would be considered a "triple slot" card.
3) I don't think it's going to work the way you plan. Both motherboards aren't that good in terms of quality or stability. The only solid X79 motherboard that I can find with the correct placement of four PCI-E3.0 x16 slots would be the Asus P9X79 WS:
http://www.newegg.com/Product/Product.aspx?Item=N82E16813131798
 
Maybe you could just buy an HP z820? It can only do SLI with Quadro cards, unfortunately.
 
@Dangman:
1) Okay, thanks anyway
2) Hm... so I would have to buy PCIe risers to be able to use these cards. If I won water cooled cards with this approach, it would probably be worth the trouble.
3) The P9X79 WS only features PCIe 3.0 with x8/x8/x8/x8 lanes if four cards are connected :/
Are the two boards I found really that instable? Some people on newegg complain about bad customer service, but Asus for example answers all those people in the thread. I can't imagine that they release a flawed product with such a high price... (naive?^^)...

@Snoflo: HP's configuration tool lets me add only three graphics cards, which is not enough, unfortunately :/
 
I highly doubt you'll notice much, if any kind of difference between a 780 on a 2.0 PCI-e 16x slot and a 3.0 PCI-e 8x slot.

PCI-e 2.0 16x slot = ~7.8GT/s
PCI-e 3.0 8x slot = ~7.6GT/s

What I am trying to say here is an 8x PCI-e 3.0 slot should handle the 780s bandwidth just fine. Considering a PCI-e 3.0 slot isn't required to get the full potential of a 780 anyway.
 
Kepakiano said:
2) Hm... so I would have to buy PCIe risers to be able to use these cards. If I won water cooled cards with this approach, it would probably be worth the trouble.
Click to expand...
Not really no. Too much of a hassle and too much awkward spacing. In addition, it would probably require a new video card altogether.

If you really want water cooling, you're better just doing an actual water cooling setup rather a kludged together solution with liquid AIO.
Kepakiano said:
3) The P9X79 WS only features PCIe 3.0 with x8/x8/x8/x8 lanes if four cards are connected :/
Are the two boards I found really that instable? Some people on newegg complain about bad customer service, but Asus for example answers all those people in the thread. I can't imagine that they release a flawed product with such a high price... (naive?^^)...
Click to expand...
Yes Asus is more than capable of releasing flawed products for such a high price. Happened last time with some of their really high-end LGA 1366 motherboards. Thus I would not go with either motherboards, especially considering the above average complaints about Asus' customer support in general.
 
@Dangman:
Okay, I reread the complaints about the boards carefully and have decided that I won't buy one of these. Thanks for the warning!
And as Skillz mentions.... x16 is not even needed! Wow, I didn't know the memory bandwidth of the GTX 780 was limited to 6Gbps... (http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780/specifications). So, thanks for the info.

So, a board with 4x x8 will suffice, nice.
I'm thinking of one of these:
http://www.newegg.com/Product/Product.aspx?Item=N82E16813131800R
http://www.newegg.com/Product/Product.aspx?Item=N82E16813128562
http://www.newegg.com/Product/Product.aspx?Item=N82E16813157285
They all seem the be decent boards, I'd probably take the Gigabyte board, because it's the cheapest one here in Germany.
What do you think?
 
+1 for Gigabyte just because IMHO they're a better company than ASUS and I'm still not sold on AsRock's products.
 
*facepalm* I just checked: The boards I posted only support three-way SLI, hence the board suggested by Dangman (Asus P9X79 WS) is probably the way to go. Too bad^^
 
Ah, you're right, of course. Hm, okay, so the decision is between the Asus and the Gigabyte board...
 
I'm not 100% sold on the Asrock as well. Between the Asus and Gigabyte, that's going to be a toss-up:
- Asus makes some really solid and well-thought out motherboards in general. However, their customer support is apparently very very horrible. In other words, there's a good chance that your Asus mobo won't be defective and will probably last for years but god help you if it is defective.

- Gigabyte generally makes decent motherboards that aren't quite as solid or well thought out. I still see the same stupid double boot issue with Gigabyte mobos at work. However, there haven't been any widespread reports of Gigabyte's support being horrible.
 
You must log in or register to reply here.
Back
Top