SM H8DGU-F 2P @ 125K PPD Improvements?

[bowlinra]

I've been running this 2P box for about 3 weeks solely on P8101s. Am I get everything I can out of this rig? Any suggestions?

Performance:

8101 - TPF 26:36 - 26:50 ~125K PPD w/ 233K WUs drops

Hardware:

2x AMD Opteron 6176SE 2.3Ghz CPUs   
Supermicro H8DGU-F AMD Dual G34 Socket Motherboard    
4x Crucial Ballistix 4GB (2 x 2GB) 240-Pin DDR3 SDRAM DDR3 1600 (Cas 8 8-8-8-24) Desktop Memory Model BLT2KIT2G3D1608DT1TX0

Software:

Standard [H]ard Ubuntu 10.10 install with TheKraken and Langouste installed

BIOS r2.0b settings to check 
- PowerNow set to Disabled 
- NUMA/SRAT set to Enabled 
- Node Interleaving set to Disabled 
- Bank/channel interleaving set to Auto or Enabled

MemTest86+ Results: I still don't understand why the memory isn't auto discovered at what it's rated?


Uploaded with ImageShack.us

 

[musky]

Your memory is running at the standard JEDEC spec for 1333 because server boards have no way to load an XMP spec, which is what the advertise 8-8-8-24 is. The only way to get the XMP spec is to flash the memory. This is from a PM I sent a while ago - different memory specs, but still applicable here:

Each memory module can have up to 3 different sets of timing programmed into its SPD (internal memory.) I'll touch on these separately.

Every module has a "base" value, usually referred to as the "SPD timings" or the "JEDEC timings". These are intended to be fail-safe timings. Most memory modules I have seen have the JEDEC DDR3-1333H standard specifications programmed into their SPD, which is in part the 9-9-9-24 timings and 666/1333 (DDR) speed you see by default. JEDEC is just an organization that issues standards for various thing, kind of like ANSI. If you are interested, google them. For the purpose of this conversation, all you need to know is that every memory module has some set of specs programmed into it, and usually the specs are a lower JEDEC standard.

Each memory module can also have up to 2 XMP profiles programmed into them. These are normally the more aggressive speed and timings that you see advertised when buying memory - in your case, the 7-7-7-21 is part of an XMP profile for those modules.

Memory with an XMP profile assumes you are going to use the memory in a motherboard that can use the XMP values. Most desktop MBs do give you an option to load an XMP profile for your memory. If you install such memory and just boot up the system without going into the bios to load the XMP values, your memory will run at the lower JEDEC specs. Once you do load the XMP values in the bios, you will see the XMP speeds and timings for your memory.

That brings us to our current situation - we are running memory intended for performance desktop motherboards in server motherboards. Server board assume you are not going to "tweak" anything in the system (or at least very little), sacrificing performance for stability. Because of this, we have no way to load an XMP profile in the bios like you traditionally would in a desktop system. When you boot up a server like this, the memory will use its default JEDEC specs, which is exactly what you are seeing and what I would expect.

So, given all of this, how do we run memory at XMP timings n a server board? There are basically two options - flash the SPDs on your memory with the XMP values, or use tear's O/C bios for SuperMicro 4p motherboards. Since you are talking about a board that is not a SM 4p, you are left with only one option - flashing the SPD on your memory.

Flashing the SPDs on memory modules is not that difficult to do. There are commercially available tools to do it in Windows (Thiaphoon Burner is one of them), or there is the open source eeprog available in Linux. Our friend tear has also written a utility to make the process much easier in Linux using eeprog. There is no documentation on this tool yet, as there hasn't been that much demand for it. In a nutshell, what you do is overwrite the "base values" (normally the JEDEC values described above) on your memory module SPDs with a more aggressive set of specifications. What we normally do is use the specs from the memory module's XMP profile - this is actually what tear's utility does. The thought is that if the memory module is "rated" for the XMP specs, then flashing these specs to be the default should be safe.

If/when you are ready to do this, please log into our IRC channel - search the forum for the server/channel. It will be MUCH easier to walk you through it real-time than for me to try to write instructions for you.

 

[bowlinra]

Thanks Musky! That's the piece I was missing.

Is there an easy way to determine if the memory support XMP timings? One would think this would clearly be on NewEgg's specs http://www.newegg.com/Product/Product.aspx?Item=N82E16820148488 so, either it's not or XMP is common place at this point?

You said the info was from an older PM, do we still not have a guide for memory flashing? Is "eeprog" aka Tear's Memory Flashing Utility, should have call it TMFU

I'm sure I've seen the 1-2-3 of the IRC channel, but can't seem to find it. Could someone shot me a link to the details?

 

[AXm77]

eeprog

d3sak

Sorry, fixed first link.

 

[musky]

eeprog

d3sak

These are as close as you are going to get now to instructions (although I think they both link to the same place.)

 

[bowlinra]

I saw something from another thread, can 6100s support memory speed 1600? I had plan to flash the memory to the actually memory rating but wanted to check first.

No special requirements. 6200 chips can additionally handle DDR3-1600.

 

[tear]

DDR3-1600 sticks can be pushed above 1333 (up to 1600, depending on refclock) with
OCNG; this is not officially supported neither by CPUs nor by OCNG but it mostly just
works.

With regular ROMs you're stuck with 1333 though.

 

[bowlinra]

With regular ROMs you're stuck with 1333 though.

With my stock BIOS (unless you would like to port you OCNG to the 2P board ) Is it worth flashing the 1600 memory to go to (Cas 8 8-8-8-24 be default) over the auto detected 9-9-9-9 24 from the memtest picture?

Are we talking a minute or a few seconds off the TPF?

EDIT Add the -Dram

bowlinra@amd2P:/fah$ sudo tpc -dram
[sudo] password for bowlinra:
Turion Power States Optimization and Control - by blackshard - v0.41

DRAM Configuration Status

Node 0 ---
DCT0: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=3 Trfc2=0 Trfc3=0 MaxRdLatency=52
DCT1: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=3 Trfc2=0 Trfc3=0 MaxRdLatency=52

Node 1 ---
DCT0: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=3 Trfc2=0 Trfc3=0 MaxRdLatency=50
interface disabled on node 1 DCT 1
DCT1: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=6 TrwtTO=5 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=3 Trfc2=0 Trfc3=0 MaxRdLatency=57

Node 2 ---
DCT0: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=3 Trfc2=0 Trfc3=0 MaxRdLatency=51
DCT1: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=3 Trfc2=0 Trfc3=0 MaxRdLatency=51

Node 3 ---
DCT0: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=3 Trfc2=0 Trfc3=0 MaxRdLatency=53
DCT1: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=3 Trfc2=0 Trfc3=0 MaxRdLatency=51


Done.

 

[musky]

It would be more like 7-7-7-20 timing @ 1333, and yes, it would be worth flashing to XMP specs. The "how" part is a bit tricky with that board - the automated dump and flash tools only work with 4p G34 boards. It is definitely possible, though.