Magny Cours "Black Edition"

10e

2[H]4U
Joined
Jul 20, 2006
Messages
3,383
So I recently picked up a couple of Opteron 6168 (Magny-Cours) "extra spicy" editions from another folder on my other team (37412) in hopes of being able to do a little comparo between overclocking Xeons and overclocking Opterons.


While I realize it's nigh impossible to get these Opterons in the wild, I still wanted to report on my findings of what I think are a pair of pretty damn good CPUs. Just because this is the [H]ardforum right? ;)

I plugged 'em in on the weekend and let 'em run some F@H BigAdv units and at stock speeds they are, well, exactly the same as regular 6168s.

But, these chips are special because they come with an unlocked multiplier, thus the "black edition" label on this thread.

The 6168's stock speed is rather low at 1.9Ghz, but with 12 real cores and quad channel RAM access, there is a lot of performance even there.

The lowest BigAdv (2684) results are 40+K PPD for a pair of 6168s, and in general, they are competitive at stock with a pair of Xeon x5650s. In fact, for once I'm actually looking forward to getting a 2684 to see how it chews through one of these.

So anyway, up 500mhz from stock speed @ 2.4Ghz with a slight voltage bump from 1.075 to 1.10v on VCore, they are folding rather nicely. Stock on a 2685 the 6168 pair gets somewhere around 53K PPD with a 23 minute TPF. Right now at 2.4Ghz they are getting a TPF of 19:40 and 70K PPD.

All this with the chips being only 49 - 50 celsius, so I'm thinking there is a fair bit o' head room here. My aim is to see if I can hit the magical 3.0Ghz mark, but 2.7 - 2.8 should also be pretty cool.

So far they've been rock-solid, and strangely enough, having run LinX on 'em for a couple of hours last night I find it strange that the GFlops reported are over 145 Gflops. Not that it's a benchmark, but rather strange that this is nearly double what my two overclocked Xeon x5650s get.

Alternatively this unit's PPD on my SR-2 config is around 135K PPD, but I'm sure that when I settle on a final OC on these chips, they will likely be able to pump out nearly 100K PPD on the fastest BigAdv work units. And I highly doubt the power usage will be anywhere close to the 500+ watts the SR-2 configuration is using. Kill-a-Watt to the rescue.

The process for overclocking these chips is far different from the Xeons on SR-2. Since the bus speed cannot be moved up past 200mhz, and the multiplier cannot be changed via AMD OC tools, a combination of tools is needed.

Using a program called RW-Everything and some WRMSR calls (Write MSR or "memory specific registers" the multiplier, VID and NB voltage can be changed per chip, and the actual multiplier can be changed per core.

I was able to find out the MSR bits that needed to be changed from the fine gentleman who sold them to me, and I also used K10Stat's Debug mode to find out how these bits relate to multi, VID and NB voltage. Taking this output and copying them to a read-write script/text file for RW-Everything was easy. I just had to create a text file with a .rw extension to get that done, and it uses three entries per CPU core. The first entry puts the chip in P-State 1, the second modifies P-State 0 (full performance) VID and multi, and the third returns the core to P-state 0, which activates the changes, as they need to be refreshed.

I just thought I'd post this to help everyone remember that the hardware part is half the fun of folding. I'll update my findings as I update my overclocks :)
 
This is awesome to hear! I was wondering how these chips perform; If I build myself a folding machine anytime soon, it'll be with these!

So far they've been rock-solid, and strangely enough, having run LinX on 'em for a couple of hours last night I find it strange that the GFlops reported are over 145 Gflops. Not that it's a benchmark, but rather strange that this is nearly double what my two overclocked Xeon x5650s get.

It's because there's twice the physical cores. HT doesn't increase Gflops performance that much.
 
Amazing, unfortunately, JF-AMD confirmed that there will be no unlocked Bulldozer server chips, the G34 socket 16 core version.
 
should have no problems hitting 3ghz. but the heat is exponential on these things since its dual 6 core Istanbul's under each heatspreader. there was one guy that hit 3ghz on the 1.7ghz ES chips @ 1.27v i believe but ran into heat issues since there were no aftermarket G34 heatsinks out at that time. so because they are Istanbul chips they can handle pretty high voltages plus have a 62C temp limit(though ive noticed AMD has changed a lot of the athlon II/phenom II processors max temps to 72C). so theres a shit ton of headroom there. i wouldnt be all that surprised if you could hit the 3.5 maybe 3.8ghz mark if ya had water cooling available.

thread with the overclock info:
http://www.xtremesystems.org/forums/showthread.php?t=233565

nightbird AMD has never officially released unlocked opteron chips. but ES chips are usually unlocked for testing purposes. so its quite possible it will be the same with the bulldozers but we can only wait and see what happens.
 
Last edited:
Amazing, unfortunately, JF-AMD confirmed that there will be no unlocked Bulldozer server chips, the G34 socket 16 core version.

That's true, but hopefully there will be desktop parts with 8 cores that will. Hopefully BullDozer can bring a bit of parity to the AMD/Intel performance race.

dam nice.....always wondered what the ES chips could do :D

Yeah, the whole reason I posted this was, while knowing that it's not a common configuration, I wanted to show that these chips have such great potential.

Tonight I'm going to try 2.6 - 2.7 Ghz and see how she goes.
 
Really, really interesting stuff. I bet since those chips are rare and really attractive to enthusiasts they demand a pretty penny, eh?

Keep those babies cool and let's see what they can really do.

Memo to AMD: we want unlockable server chips and overclockable MP boards!
 
Memo to AMD: we want unlockable server chips and overclockable MP boards!

Yeah! :D

P.S. I remember JF-AMD being interested in Magny Cours overclocking. He posted in one thread where the person only could OC one core, said he(JF) wanted info he could send to engineers.
 
Wow, that is sweet! You have my dream folding rig, and it got even better.

That seems like an aweful lot of hassle that you had to go through to overclock these though. I just hope that we will see a SR2 like motherboard when the Bulldozers come out. We know that eVGA's mobo team went to Sapphire, so I wonder if they are working on something special.
 
Using a program called RW-Everything and some WRMSR calls (Write MSR or "memory specific registers" the multiplier, VID and NB voltage can be changed per chip, and the actual multiplier can be changed per core.

I was able to find out the MSR bits that needed to be changed from the fine gentleman who sold them to me, and I also used K10Stat's Debug mode to find out how these bits relate to multi, VID and NB voltage. Taking this output and copying them to a read-write script/text file for RW-Everything was easy. I just had to create a text file with a .rw extension to get that done, and it uses three entries per CPU core. The first entry puts the chip in P-State 1, the second modifies P-State 0 (full performance) VID and multi, and the third returns the core to P-state 0, which activates the changes, as they need to be refreshed.

Would this work on an Intel server board? I would need Vtt, Vcore, and bclk, but I would think that they could all be changed with MSR sets. I just never figured out how to find the ones I needed.
 
Wow, that is sweet! You have my dream folding rig, and it got even better.

That seems like an aweful lot of hassle that you had to go through to overclock these though. I just hope that we will see a SR2 like motherboard when the Bulldozers come out. We know that eVGA's mobo team went to Sapphire, so I wonder if they are working on something special.

Half the reason I posted was for you my friend!

Up to 2.8Ghz now with temps still in the mid 50s. Just running LinX and getting 167 Gflops :eek:

If I can get 2.9Ghz, or a 1 Ghz OC with temps under 60 I think I will be ecstatic with that. TJMax on these chips is listed as 70 in CoreTemp so this gives me 10+ celsius "cushion" as it will be hotter in the summer months.

Would this work on an Intel server board? I would need Vtt, Vcore, and bclk, but I would think that they could all be changed with MSR sets. I just never figured out how to find the ones I needed.

I'm 100% sure it would be possible with this utility, but finding the MSRs would be the difficult part. Obviously you're talking about regular server boards I gather?

I know that there is "SetFSB" which manipulates the clockgen chip to help pump up bus speeds. That might be something worth looking into, as Intel Xeons aren't bus locked like the Opterons are. Alternatively the voltage part would need a different utility in regards to VCore, VTT or PLL. Of course there are a number of Xeons out there that can probably go up in BCLK and get a bit of an OC going without those values getting upped.

I believe that the AMD findings were in the actual AMD reference documents, so I'm sure poring through the Intel versions shouldn't be impossible in terms of VID and VTT. But these are "legacy" settings that have existed since the Athlon FX days.

I'll take a look once I've completed my Opteron OC "project", ie. like tonight.
 
I've had my eye on these but the lack of OC has kept me from buying. Thanks for the info.
 
Nehalem based Xeons can clock real high without volt increases. Just getting the bus speed up would unlock huge potential depending on the limitations of the mobo.
 
I'm 100% sure it would be possible with this utility, but finding the MSRs would be the difficult part. Obviously you're talking about regular server boards I gather?

I know that there is "SetFSB" which manipulates the clockgen chip to help pump up bus speeds. That might be something worth looking into, as Intel Xeons aren't bus locked like the Opterons are. Alternatively the voltage part would need a different utility in regards to VCore, VTT or PLL. Of course there are a number of Xeons out there that can probably go up in BCLK and get a bit of an OC going without those values getting upped.

I believe that the AMD findings were in the actual AMD reference documents, so I'm sure poring through the Intel versions shouldn't be impossible in terms of VID and VTT. But these are "legacy" settings that have existed since the Athlon FX days.

I'll take a look once I've completed my Opteron OC "project", ie. like tonight.

Yeah, this is a regular server board (the Asus in my sig.) SetFSB works, but it will only go to 135-136. I think that is telling me that it needs more Vtt to go higher. Vcore is probably fine for a reasonable jump, although adjusting that would be nice, too.
 
but can it run Crysis?

:D

Damn right it can! All cores no threads. The integrated server video board sucks bowls though.

Yeah, this is a regular server board (the Asus in my sig.) SetFSB works, but it will only go to 135-136. I think that is telling me that it needs more Vtt to go higher. Vcore is probably fine for a reasonable jump, although adjusting that would be nice, too.

I took a look at Intel's reference dox last night but didn't find anything. I'll probably have a gander at Xtremesystems to see if any one else has cracked this.

Update: I had to dial it down to 2.5Ghz... It's really strange. It passed over 10 hours of LinX and 3 hours of Intel Burn Test last night/this morning, BUT as soon as I started the SMP folding client it died within 5 minutes and I just stopped it before it sent the error back to Stanford.

Now at 2.5Ghz it's been folding for 1.5 hours without issue. TPFs on the 6900 unit are at a respectable 17:55 which is about 5.5 - 6 minutes lower than stock, for 80K PPD.

I'm thinking once it finishes this unit early tomorrow I will take down the client and try Prime95 to see if it gives me a better indication of stability. I know the previous owner of these "extra special" chips had them at 2.7Ghz, but was getting EUEs on 670x units.

Weird stuff.
 
Throw enough cores at it and you could probably get by with a software renderer...

4 sockets, 48 cores? ;)
 
Throw enough cores at it and you could probably get by with a software renderer...

4 sockets, 48 cores? ;)

You know what? That's a great idea! I have 4 6168s now. Let's see if I can beat may I be worthy's monster SR-2 setup with my SR-2 and quad Opteron monster death machine!

Or just play Crysis. :D
 
.....you have 4 now with unlocked multis !?!?!?!?!?!?......and it may nto beat him but will be way more bad ass :D
 
.....you have 4 now with unlocked multis !?!?!?!?!?!?......and it may nto beat him but will be way more bad ass :D

No, just two with unlocked multis, I wish I had 4. Good lord that would be insane. 2 are standard 6168s with locked multi.

But still, on the foldingforum someone had 4 x 8 (32) cores at 2.0Ghz and they were rocking SMP units at nearly my SR-2's levels.

Pretty scary. My SR-2 gets A3 SMP units out at about 1:01 - 1:02 minutes per frame, and this 32 core box does them in 1:07. Imagine 16 more cores.

Here's the URL http://foldingforum.org/viewtopic.php?f=55&t=15899&p=157514#p157514
 
Last edited:
There are people that fold with quad socket magny cours, I did a search on them a while ago. These are people that plop down 8k$ for server systems though, with ppd in the millions per day. (Was like 7 or 8 mil, something crazy)
 
No, just two with unlocked multis, I wish I had 4. Good lord that would be insane. 2 are standard 6168s with locked multi.

But still, on the foldingforum someone had 4 x 8 (32) cores at 2.0Ghz and they were rocking SMP units at nearly my SR-2's levels.

Pretty scary. My SR-2 gets A3 SMP units out at about 1:01 - 1:02 minutes per frame, and this 32 core box does them in 1:07. Imagine 16 more cores.

Here's the URL http://foldingforum.org/viewtopic.php?f=55&t=15899&p=157514#p157514

Considering the sheer number of cores, I'd have expected 32 AMD cores to put up a bigger fight than just barely nipping at your heels. Them 12 cores on the SR-2 board are some mean number crunchers. Are there any boards out there that will do 4x6 1366? A 24 core Intel box would be madness...
 
no, one of the disadvantages of the Xeon architecture is you would have to upgrade to a whole new platform to get 4P Xeons.

You'll need 7000 series xeons which run on LGA1567
 
Considering the sheer number of cores, I'd have expected 32 AMD cores to put up a bigger fight than just barely nipping at your heels. Them 12 cores on the SR-2 board are some mean number crunchers. Are there any boards out there that will do 4x6 1366? A 24 core Intel box would be madness...

Dell will happily sell you an R910 with four 8-core Xeons. With X7560s (2.26GHz) prices start at $29K. :)

To put that in perspective, the software licensing costs probably exceed the hardware cost for businesses that would buy such a server.
 
Considering the sheer number of cores, I'd have expected 32 AMD cores to put up a bigger fight than just barely nipping at your heels. Them 12 cores on the SR-2 board are some mean number crunchers. Are there any boards out there that will do 4x6 1366? A 24 core Intel box would be madness...

Yes, but my SR-2 is running smoothly at 3.9Ghz, which is close to twice as fast per core than a 24 core or this 32 core Magny Cours box that PennyPincherP has, so obviously the extra 8 cores at a slightly higher clock speed are making up some of the difference between the 32 core 2.0Ghz 6128s vs my 24 core 1.9Ghz 6168s.

For example, my time-per-frame on BigAdv 2692 units on the SR-2 is around 11:52 - 12:02 on average and the 1.9Ghz Opteron 6168s with 24 total cores do this in about 21:50 - 22:00 minutes.

Of course, there is probably not a fully linear extrapolation on cores vs. units, so the SMP A3 units may scale better in terms of more cores than BigAdv.

Regardless I took the plunge and ordered a Supermicro board to house the 4 CPUs, and all in all, with RAM costs (extra 8 1GB DIMMs) and board cost, it's not a big difference after I sell my Asus KGPE-D16 board.
 
Dell will happily sell you an R910 with four 8-core Xeons. With X7560s (2.26GHz) prices start at $29K. :)

To put that in perspective, the software licensing costs probably exceed the hardware cost for businesses that would buy such a server.

I am fortunate to have a link at MS where I can get Server 2008 r2 for a very, very, very low $ amount.
 
Yes, but my SR-2 is running smoothly at 3.9Ghz, which is close to twice as fast per core than a 24 core or this 32 core Magny Cours box that PennyPincherP has, so obviously the extra 8 cores at a slightly higher clock speed are making up some of the difference between the 32 core 2.0Ghz 6128s vs my 24 core 1.9Ghz 6168s.

For example, my time-per-frame on BigAdv 2692 units on the SR-2 is around 11:52 - 12:02 on average and the 1.9Ghz Opteron 6168s with 24 total cores do this in about 21:50 - 22:00 minutes.

Of course, there is probably not a fully linear extrapolation on cores vs. units, so the SMP A3 units may scale better in terms of more cores than BigAdv.

Regardless I took the plunge and ordered a Supermicro board to house the 4 CPUs, and all in all, with RAM costs (extra 8 1GB DIMMs) and board cost, it's not a big difference after I sell my Asus KGPE-D16 board.

Ahh, yeah...forgot about the clock speeds.
 
I am fortunate to have a link at MS where I can get Server 2008 r2 for a very, very, very low $ amount.

I'm not talking about the OS, I'm talking about VMware, SAP, etc. Just to install SQL Server 2008 R2 Datacenter would be $220K.
 
I'm not talking about the OS, I'm talking about VMware, SAP, etc. Just to install SQL Server 2008 R2 Datacenter would be $220K.

Oracle is another good example of that. More power equals big bucks. All I need is the O/S. Folding client is FREE :)
 
All this right here is why I would probably never go above a 2 processor system if I found the funds to build such a system. We know that Windows 7 Ultimate is compatible with up to 2 processors, and those Windows server OS's appear to get crazy expensive after that. I don't have connections, so I would probably have to pay those full retail prices :(
 
Windows 7 Pro also supports 2 procs, so you don't have to pay for Ultimate. I have Pro on one SR-2. The other has Server 2008 R2, through "connections" (legitimate).
 
Windows 7 Pro also supports 2 procs, so you don't have to pay for Ultimate. I have Pro on one SR-2. The other has Server 2008 R2, through "connections" (legitimate).

I didn't know that, as I emailed a Microsoft Windows 7 rep a while back who told me that only Ultimate supports 2 processors.

Folding1 is running on W7 home premium, but folding2 is running on Windows 7 Pro although it is an OEM version I bought from Micro Center. Gaming rig is running Ultimate.
 
I didn't know that, as I emailed a Microsoft Windows 7 rep a while back who told me that only Ultimate supports 2 processors.
I can confirm Pro works fine with two sockets as well.
 
I can confirm Pro works fine with two sockets as well.

Out of curiosity, do you guys know how OEM versions of Windows 7 work when you get ready to do a complete system overhaul? I'm now curious whether I would even be able to transfer the OS if I were to go to a dual processor system at some point far in the future.

Some of you guys might remember how folding2 had a bad dimm slot which shorted out and took all 3 ram modules with it. After Micro Center replaced the motherboard, I had one hell of a time trying to reactive Windows 7 Pro. I ended up having to call them up and getting an activation key.
 
Out of curiosity, do you guys know how OEM versions of Windows 7 work when you get ready to do a complete system overhaul? I'm now curious whether I would even be able to transfer the OS if I were to go to a dual processor system at some point far in the future.

Some of you guys might remember how folding2 had a bad dimm slot which shorted out and took all 3 ram modules with it. After Micro Center replaced the motherboard, I had one hell of a time trying to reactive Windows 7 Pro. I ended up having to call them up and getting an activation key.


theres ways around that.. but thats all ill say. i too have the activation problem with my windows 7 ultimate. got fed up with calling/emailing them so used my work around. but it also means i cant install MSE :(
 
Yup Windows 7 64bit Pro OEM on all my SR2s.

(I do like answering questions that other people have already answered, and better) ;)
 
theres ways around that.. but thats all ill say. i too have the activation problem with my windows 7 ultimate. got fed up with calling/emailing them so used my work around. but it also means i cant install MSE :(

One subscription to TechNet pretty much covers you for unlimited installs of all MS products, although it's more useful if you have multiple systems. I think I have 10 keys I can use as many times as I want, and the keys stay valid after your subscription expires. It's nice not having to jump through hoops to get your installs to work.
 
One subscription to TechNet pretty much covers you for unlimited installs of all MS products, although it's more useful if you have multiple systems. I think I have 10 keys I can use as many times as I want, and the keys stay valid after your subscription expires. It's nice not having to jump through hoops to get your installs to work.

Yes, my msdn sub has proven invaluable. Although it's provided by my company, I would renew out of my own pocket if needed.
 
Technet is a cheaper alternative to msdn that is better suited to people like us. I've paid for two Technet subs in the past and will do it again if/when needed.
Posted via [H] Mobile Device
 
Back
Top