Dual GPU Scaling

[natermeister]

I currently have one X1900XTX Folding and another one will be on the way shortly. I'm wondering how well production scales running two clients. I remember hearing that it didn't scale perfectly, usually you only got about 80-90% of the usual single card performance on the second card. Not to mention the small production hit you'd take when you shut off the standard client on one of the cores.

Anyone out there have any info for me?

Thanks,

-Nater

 

[Tigerbiten]

The production hit is something like that.

I've been reading up on it as I have/will run a Dual GPU rig.
DX9 which is needed to poll the clients only runs single theaded hence the slight hit in points going from a single GPU to dual GPU's.

If both cards are the same then your ok,
If there is a big difference in the speed between the cards then the faster card takes an even bigger hit as it waits for the slower card.

Going to more than dual cards means the slow down is even worse.

My cards where dual X1950xtx's running at 3d speeds.
I went from ~300 PpD dual CPU to ~900 PpD (750 GPU + ~150 CPU) to 1350 PpD with dual GPU's.
Frame times went from ~6:20 to ~7:10.
Power draw went from 140 watts for twin CPU's clients -> 250 watts with mixed clients -> 360 watts with twin GPU clients.

Luck ...........

 

[natermeister]

Alright, that sounds good. Both of my cards are X1900XTXs and I had no intention of running more than two, I didn't even know you could do that.

 

[JBavousett]

So what's the word on dual GPU and SMP?

 

[MixManSC]

So what's the word on dual GPU and SMP?

I'd be interested in that as well. I don't really need high end video in my new dual xeon workstation but I am considering adding dual 1950's to the machine just to fold on, and then eventually upgrade to dual quad core xeons. 2 smp clients and 2 gpu clients on one box should net me 5000+ ppd on a single box.

 

[natermeister]

I think you could probably run two GPU clients and a single SMP client if you had four cores. If you had eight, you could add another SMP console.

 

[marty9876]

I dunno....

1x SMP client will use 4x cores by itself at 100% across all cores. The points increase on going form 2x cores to 4x cores while not perfect is pretty dang linear. Dang near 2x the points going from 2x cores to 4x cores on a SMP instance.

2x GPU's don't use 2x cores at 100%, I forget the exact numbers reported but I think it was 100% on the first core and 60% on the second. The GPU client just does the stupid polling stuff or what not and needs 100% of a core for 1x instance. Take away the CPU ticks from the CPU client and you start taking hits on the GPU output big time.

It's all trial and error. I tried SMP on a P4D and SMP/GPU on the same box (P4D/x1950 Pro) and at the end of the day the total PPD was the same. C2D rigs scale better from what I read (I thought that would scale worse). More science was being done I think, more juice being drawn from the wall for the same PPD too.

No offical word on my 2x posts on what is the preferred config for dual core boxes w/ GPU's.

On a dual core box w/ dual GPU's it's interesting... 1st off on straight installs of all being equal priority I wonder if the SMP client would ever finish by the deadlines to begin with.

On a quad core box w/ dual GPU's things would be far better off for deadlines at least. The thing is the PPD by the GPU's simply might not equal the PPD lost by the heavily bonus SMP clients, currently.

Disclaimer: All numbers under the condition of watching HD porn 24/7.

 

[Tigerbiten]

I got around a 300 PpD increase running a SMP-GPU combo on my C2D.
I set the affinity for the GPU client to core 0 and the SMP client to core 1.
The GPU client stayed on core 0 as each new protien started but the SMP client needed to be reset for each new protein.
The SMP client drop from ~1450 PpD running on two core to 950 PpD running on only one core.
But I added 750 PpD from the GPU which more than up for that drop.

Never did try running two SMP client on my C2D.
I think it would depend on how overloaded the memory bus got as to how well it would cope.

In theory I would loose around 1,000 PpD going 4 core -> 2 core on my Opterons but gain 1,350 running dual GPU's.

Luck ..........

 

[Xilikon]

Disclaimer: All numbers under the condition of watching HD porn 24/7.

I watched HD pr0n 24/7 prior to draw my own conclusion :

I believe it's more efficient to sell the two GPU, get money to build a extra C2D box and you will get higher PPD. The only time GPU folding is interesting is if it's used in a box with a single-core or a old generation P4D, or even a low-end X2.

 

[marty9876]

@ Tiger I'm still surprised the SMP client ran so well on 1x C2D core. Needless to say, I'm guessing Stanford would not like the idea of jacking the SMP client into a glorified uni client...

Also I think your setting affinity manually which is not so much fun in my book. Not a unattended solution.

 

[JBavousett]

My SMP runs ~12 minutes per frame on my e6700 @3.5Ghz. I'm running an 8800GTS...but if it gave me a good PPD boost I'd consider tossing an x1950xtx in. At 20 hours per WU, I'm getting a bit over 1800PPD

As for twin SMP clients on a C2D...I tried it. The two clients run, but the second client you start only get's ~40% of the CPU's time. The first client went up to 22 minutes per frame (~1100ppd) while the second core ran 32 minutes per frame (~680ppd). So it would be a slight PPD hit. Perhaps it would run better if you set the CPU affinity to lock each client to a single core...but that kills the point of SMP. Maybe locking a client's to a pair of cores on a C2Q...Hmmm, time to by a quad I think.