it might be time to consider SB-E

jebo_4jc

[H]ard|DCer of the Month - April 2011
Joined
Apr 8, 2005
Messages
14,566
I noticed today that the Xeon E5 2660 is selling for $350-$400 apiece if you buy a spicy variant or scoop up a good deal on an ebay auction. This cpu is an 8 core, 16 thread, 2.2ghz nominal, 3ghz max turbo, 20M cache, 95W CPU. I also noticed that Supermicro dual 2011 boards can be found as low as $350. So, for a little over $1000, you could have the base of a nice folding system.

The question in my mind is, at what clock speed will these chips fold? I'm sure the 3ghz advertised max turbo is for single threaded performance, but I wonder if the 30x multi could be forced 100% of the time, or at least a 29x multi. At 2.9 to 3.0ghz, 16 cores/32 threads would be pretty competitive.

Could somebody educate me on what I could expect in terms of a 24/7 100% load clock speed with these chips? Has anybody tried throttlestop or a similar method to force a higher multiplier?
 
I have had several and I currently run an unnamed chip that is advertised at 1.6ghz with 2.1ghz max turbo speed. It seemed to have settled at 2.0ghz when folding on all 8 cores even with the short/long duration power limit turned up. I experienced the same with a pair of 2650's, they seem to fold 1 bin below max turbo.

Go 2P if you do it, as a single chip any but the ones that will run at 2.9ghz+ 24/7 (2680/90, 2687W) will be beaten out by a 2600k/3770k in SMP folding.

You can see my machines here, the E5 at 2.0 falls squarely between my 3570k at 4.4ghz and the 3770k at 4.4 as well, the results were the same on both 6.34 under Ubuntu 10.10 and now using the V7 beta client under the same OS. The plus side is it uses much less power, about 50w less than the 3570k machine.
 
Great info. Thanks.

So the 2660 would probably fold at 2.9ghz 24/7
 
I believe the spicy 8-core 2.2ghz chips run at 2.5ghz all-core turbo.
 
I believe the spicy 8-core 2.2ghz chips run at 2.5ghz all-core turbo.

That sounds about right.

Also you have to be sure you can get a board that supports ES chips in 2P configurations.

Oh... and for $350 you get a basic 2P motherboard. For ~$550 you can get an onboard LSI SAS2308 plus either 4x GbE (not including the IPMI NIC) or dual 10GbE. If you ever wanted slightly slower folding performance trading off for a storage server for example.
 
Also a note: almost none of the supermicro boards will work with spicy chips in a 2P configuration, your best bet is an Asus board unless you get OEM/retail chips.
 
The supermicro x9dai works fine with every spicy chip I've ever tried with it.
 
The supermicro x9dai works fine with every spicy chip I've ever tried with it.

I think thats the one most of the auctions list for compatibility, since there are a ton of SM boards thats why I said almost none, I couldn't remember the one that did off the top of my head.
 
Any idea about the 2ghz E2650?

I ran a 2P setup with those and they typically ran at 2.4ghz turbo after I increased the short/long power limits, the chips run really cool at a max load temp of around 50*C so there is plenty of thermal headroom. I was running bigadv and only getting 8101's, I'll see if I can find my TPF data for those.
 
I ran a 2P setup with those and they typically ran at 2.4ghz turbo after I increased the short/long power limits, the chips run really cool at a max load temp of around 50*C so there is plenty of thermal headroom. I was running bigadv and only getting 8101's, I'll see if I can find my TPF data for those.

That would be appreciated
 
That would be appreciated

Running Ubuntu 10.10 and the 6.34 client (kraken wrapper) with an 8101 WU it was getting an average TPF of 23:02 with 280w draw at the wall, very little deviation in the frame time. Minimum was 22:59 and maximum was 23:07. I wish I had a chart for you, but I wasn't using HFM at the time so it wasn't logged anywhere besides where I wrote it down.
 
How does this compare with the 4P AMD stuff $ to $ ?

Seems like, if I were to guess, the 4P AMD still has this beat? For example, $1,100 spent on 4P would net you more PPD (and more PPD/watt) then with Intel?
 
How does this compare with the 4P AMD stuff $ to $ ?

Seems like, if I were to guess, the 4P AMD still has this beat? For example, $1,100 spent on 4P would net you more PPD (and more PPD/watt) then with Intel?

For $1,100, you could build a 4P Opteron 6128 system, which would net around 170K-230K PPD with 8101 WU, depending on overclocking.
 
For $1,100, you could build a 4P Opteron 6128 system, which would net around 170K-230K PPD with 8101 WU, depending on overclocking.

So it seems to me that given the current information we have that the SB-E dual setup is now better price/point, but only slightly. Given the running temps, I wonder how well these (temp wise) would run in a 2U case. I'm intrigued to know this as 280w for 250K PPD seems rather impressive. Is it also possible to get a quad board and only run 2 like with the optys? Seems if you could get a quad you would have a pretty awesome setup running at ~500W. Just curiosity here speaking hypothetically since I need to buy an engagement ring soon and all PC spending for me has to be halted, but maybe someone else could use the info.
 
Just in the process of closing out a 2660 deal myself, the 2680's give much better PPD for not much more power but the setup costs are higher, going for an asus mobo for now as i plan on upgrading again later. My daily driver and slower rig are both being retired from folding, depending on power draw my slower dedicated rig will either be switched back on early next year, rent a boxen'd if there is any interest or sold.

260w from the wall will give 600PPD/w on a 8101 - not bad and much better than the 300PPD/w i currently get
 
Which Asus mobo Nathan? Will it support espaniol chips?
 
Z9PE-D8 WS works well with ES chips.

I would like to try and get a Z9PA-D8 board to play with (YAY ATX FORM FACTOR! YAY!), but that is down the line for the moment.
 
4p G34 es is the winner of PPD per dollar, but 2p/4p E5 might be better at PPD per watt.

How does this compare with the 4P AMD stuff $ to $ ?

Seems like, if I were to guess, the 4P AMD still has this beat? For example, $1,100 spent on 4P would net you more PPD (and more PPD/watt) then with Intel?
 
The compatibility issue of E5 ES chips is a bit complicated, it's highly dependent on the CPU stepping and even the QDF code. In general, C0/C1 stepping is perfect and free of compatibility issues (no matter 2p or 4p), while B1 stepping is bogus.
One thing should be noted is that Z9PE-D8 WS only works well with E5 26xx ES chips, it doesn't support any E5 46xx ES chips.

Z9PE-D8 WS works well with ES chips.

I would like to try and get a Z9PA-D8 board to play with (YAY ATX FORM FACTOR! YAY!), but that is down the line for the moment.
 
Based on the Linuxforge Bonus Point Calculator estimated total points would be 251345. With an estimated completion time of 1.6 days (average tpf of 23:02) to complete the WU that would net you an estimated157136 PPD.
So about 157K PPD
For $1,100, you could build a 4P Opteron 6128 system, which would net around 170K-230K PPD with 8101 WU, depending on overclocking.
Looks like 4P 6128 wins on the PPD/$ still, but it will pull more power
So it seems to me that given the current information we have that the SB-E dual setup is now better price/point, but only slightly. Given the running temps, I wonder how well these (temp wise) would run in a 2U case. I'm intrigued to know this as 280w for 250K PPD seems rather impressive. Is it also possible to get a quad board and only run 2 like with the optys? Seems if you could get a quad you would have a pretty awesome setup running at ~500W. Just curiosity here speaking hypothetically since I need to buy an engagement ring soon and all PC spending for me has to be halted, but maybe someone else could use the info.
It is 157K PPD for 280W, which is still pretty good, 560PPD/W. Thermodynamics is thermodynamics, I have a 2P 1366 rig that pulls 315W. With good coolers and adequate air flow you should be fine. It does not matter what the core temps are as far as heat production goes. 280W at 70*C is the same amount of total heat as 280W at 50*C, the delta T will just be higher on the 70*C one. As long as you are within equipment specs you are fine, but lower is better and a better cooler and better air flow will drop the temps.

Only the 4xxx series can run in 4P mode, 2xxx are 2P. I do not know if the 2xxx series will boot in a 4P 2011 board. Last I remember, 4650 ES were in the $700+ per CPU range if you got lucky and are at $1500 per CPU buy it now at this time on eBay. The 4P 2011 boards are around $1000.

A high overclock on a 4P G34 with 6180SE / ES (around 3Ghz) will net you 477K PPD on 8101, and a 4P 4650 (2.7Ghz) machine will get you around 631K PPD on 8101.

The 4P 4650 machine is no joke with more PPD and less watts than a 4P AMD, but it is 2-3x the cost of a 4P AMD machine, and the CPU's are also scarce.
 
I think 157K PPD is not a good score for 2P E5 rigs, 2P Xeon E5 3.0G (QA92) with slightly overclocking could get 280K PPD on 8101, and QA92 is also cheap.
And as far as I see, 46xx chips could work on many 2p boards without difficulties, but no 26xx chips could work on 4p boards at the moment.

So about 157K PPD
Looks like 4P 6128 wins on the PPD/$ still, but it will pull more power

Only the 4xxx series can run in 4P mode, 2xxx are 2P. I do not know if the 2xxx series will boot in a 4P 2011 board. Last I remember, 4650 ES were in the $700+ per CPU range if you got lucky and are at $1500 per CPU buy it now at this time on eBay. The 4P 2011 boards are around $1000.
 
I think 157K PPD is not a good score for 2P E5 rigs, 2P Xeon E5 3.0G (QA92) with slightly overclocking could get 280K PPD on 8101, and QA92 is also cheap.
And as far as I see, 46xx chips could work on many 2p boards without difficulties, but no 26xx chips could work on 4p boards at the moment.

It looks as if there is a steep price jump from the 2650/2660 (around $350-$400 per CPU) to the 2670/2680/2687W range ($650-$720 per CPU).

I did not see any QA92's, but the QA91's are going for $725-$785

2($350) + $350 for motherboard would get you 157K PPD ($1050)
An SR2 with dual hexen a health OC would be in this range of PPD and price but more power.

2($650) + $350 for motherboard would get you 280K PPD ($1650)
Around the price of a 4P 6172 AMD machine which could get anywhere from 300K to 350K PPD on 8101 depending on overclock.

If power is a concern, then the 2P 2011 should do fairly well. Looks like 2P 2011 is pretty competitive now. We will have to wait and see if 4P Intel prices drop any for more of those to get adopted. We will also have to see how Pile Driver shakes things up on the G34 socket coming soon.
 
For $1,100, you could build a 4P Opteron 6128 system, which would net around 170K-230K PPD with 8101 WU, depending on overclocking.

What motherboard would you be using on that 6128 system? The only board I see on Newegg runs close to $800
 
Supermicro H8QGL (preferred) or H8QGi. Check wiredzone.
 
Z9PE-D8 WS works well with ES chips.

I would like to try and get a Z9PA-D8 board to play with (YAY ATX FORM FACTOR! YAY!), but that is down the line for the moment.

Eyeing up both boards, one now and one early next year, for all flavours of chips
 
I am the very lucky owner of a 4P 4650 machine, and it fold the 8101 with a TPF of 8:52 that give a PPD of 658K at the most. The average is 655K PPD for the 8101, but the 6901 is much better with average of 690K. My 4650 run at 3.1GHz in turbo mode on all cores. The power draw is 725W from the wall, meaning a PPD/Watt at 907.5 for the 8101. This machine cost me around $8000 ++ (if my memory is not damage from Alzheimer in this hour), that is very expensive because I have the barebone box from SM, and I live in Norway. At the moment all my 5 rigs is working with 6901, that is very unusual this days with only 8101 around.

My mother died of Alzheimer the 21 of January this year, as 2 of her siblings has done the last 5 year. As you can understand, I am a little worried for that I can have inherited this horrible disease my self, but I do not think really so! Anyway, this is the reason why I do all this folding for the research in regi of Stanford. Have a nice day & tonight all of you folding people.

So about 157K PPD
A high overclock on a 4P G34 with 6180SE / ES (around 3Ghz) will net you 477K PPD on 8101, and a 4P 4650 (2.7Ghz) machine will get you around 631K PPD on 8101.

The 4P 4650 machine is no joke with more PPD and less watts than a 4P AMD, but it is 2-3x the cost of a 4P AMD machine, and the CPU's are also scarce.
 
Alias, its great to hear your folding story.

What does "regi" mean? It doesnt translate :)

That's an amazing rig you have. Enjoy that monster, and I hope we can find the cure for you and your family!
 
Regi in my language, means: "on behalf of"...Sorry about my bad english writing!
 
I am the very lucky owner of a 4P 4650 machine, and it fold the 8101 with a TPF of 8:52 that give a PPD of 658K at the most. The average is 655K PPD for the 8101, but the 6901 is much better with average of 690K. My 4650 run at 3.1GHz in turbo mode on all cores. The power draw is 725W from the wall, meaning a PPD/Watt at 907.5 for the 8101. This machine cost me around $8000 ++ (if my memory is not damage from Alzheimer in this hour), that is very expensive because I have the barebone box from SM, and I live in Norway. At the moment all my 5 rigs is working with 6901, that is very unusual this days with only 8101 around.

My mother died of Alzheimer the 21 of January this year, as 2 of her siblings has done the last 5 year. As you can understand, I am a little worried for that I can have inherited this horrible disease my self, but I do not think really so! Anyway, this is the reason why I do all this folding for the research in regi of Stanford. Have a nice day & tonight all of you folding people.


Wow, dude that is way impressive. I don't kno what else to say but that is some serious dedication, and that 4P box is NUTZ fast! Good work sir!
 
I am the very lucky owner of a 4P 4650 machine, and it fold the 8101 with a TPF of 8:52 that give a PPD of 658K at the most. The average is 655K PPD for the 8101, but the 6901 is much better with average of 690K. My 4650 run at 3.1GHz in turbo mode on all cores. The power draw is 725W from the wall, meaning a PPD/Watt at 907.5 for the 8101. This machine cost me around $8000 ++ (if my memory is not damage from Alzheimer in this hour), that is very expensive because I have the barebone box from SM, and I live in Norway. At the moment all my 5 rigs is working with 6901, that is very unusual this days with only 8101 around.

My mother died of Alzheimer the 21 of January this year, as 2 of her siblings has done the last 5 year. As you can understand, I am a little worried for that I can have inherited this horrible disease my self, but I do not think really so! Anyway, this is the reason why I do all this folding for the research in regi of Stanford. Have a nice day & tonight all of you folding people.

No worries sir! I am just trying to get the most output from the lest money possible. The more output we can get online the more science that can get done! The 4P 2011 is prohibitively expensive for most folders so it does not get recommended. Since for $8000 you could build 3-4 AMD 4P's for that price which would put out anywhere from 1 million to 2 million PPD depending on work units folded, and overclock obtained.

I have great sympathy for you. We are all here to fight these diseases the best we can. We try to help everyone get the most out of their systems, and budgets so that we can bring the most production against these diseases.

Don't take my responses as anything negative against you. Your information is valuable to us as well!
 
Back
Top