Improve Opteron 6200's performance (and PPD) by using TurionPowerControl

Discussion in 'Distributed Computing' started by quickz, Aug 2, 2012.

  1. quickz

    quickz Limp Gawd

    Messages:
    256
    Joined:
    Jul 30, 2012
    Hi all, I don't know whether it was posted here before, recently I find that TurionPowerControl can improve Opteron 6200's performance significantly. All we need to do is running the cmd:

    TurionPowerControl -psmax 1

    I have made a test on a 4p Opteron 6272 box, and detailed results are shown as follows:
    (DLB is enabled in all cases)

    Before running TurionPowerControl:
    p6901: TPF=06m44s, PPD=338k
    p6903: TPF=14m56s, PPD=459k
    p8101: TPF=14m12s, PPD=325k
    p8102: TPF=10m55s, PPD=482k

    After running TurionPowerControl:
    p6901: TPF=06m18s, PPD=374k
    p6903: TPF=13m56s, PPD=510k
    p8101: TPF=12m48s, PPD=379k
    p8102: TPF=09m58s, PPD=552k
     
  2. tear

    tear [H]ard|DCer of the Year 2011

    Messages:
    1,567
    Joined:
    Jul 25, 2011
    Looks like pretty good find :) Congratulations!

    By limiting P-state to 1 we're basically containing CPU to "all-core turbo" bin.
     
  3. Kendrak

    Kendrak [H]ard|DCer of the Year 2009

    Messages:
    22,871
    Joined:
    Aug 29, 2001
    Soooooo..... 62xx chips good now? Or just on par now.
     
  4. DooKey

    DooKey [H]ard DCOTM x4

    Messages:
    5,726
    Joined:
    Apr 25, 2001
    Nice find! Might make it worth grabbing 62xx chips now. Interesting.
     
  5. tear

    tear [H]ard|DCer of the Year 2011

    Messages:
    1,567
    Joined:
    Jul 25, 2011
    I'd actually be interested in logging into 6200 system (don't have one) and checking few things out before the command is run...

    Anyone? :)
     
  6. tkam

    tkam [H]ard|DCer of the Month - Dec. 2012

    Messages:
    436
    Joined:
    Dec 18, 2007
    Does seem to be helping I'm seeing about a 2 min decrease in TPF on an 8101 on my 2 x 6272 box.
     
  7. 402blownstroker

    402blownstroker [H]ard|DCer of the Month - Nov. 2012

    Messages:
    3,156
    Joined:
    Jan 5, 2006
    People...... make this happen.
     
  8. Linden

    Linden [H]ard|Gawd

    Messages:
    1,140
    Joined:
    Sep 8, 2005
  9. tkam

    tkam [H]ard|DCer of the Month - Dec. 2012

    Messages:
    436
    Joined:
    Dec 18, 2007
    Hmm so much for the lower TPF - this change seems to have created some instability as that box is now crashing. Should that change carry over through a reboot? What -psmax do I need to set to get it back to "default" settings.
     
  10. tear

    tear [H]ard|DCer of the Year 2011

    Messages:
    1,567
    Joined:
    Jul 25, 2011
    It shouldn't cause instability per se...

    I would expect -psmax 0 would revert it to normal
    and no, it does not persist across reboots.

    How is the box crashing? What are your temps? I'm assuming that Turbo is enabled in the BIOS, correct?
     
  11. tkam

    tkam [H]ard|DCer of the Month - Dec. 2012

    Messages:
    436
    Joined:
    Dec 18, 2007
    Temps seem ok but it's crashing almost immediately now after folding starts so it's hard to get any read on what they are like under the load. This box was working 100% completely fine till I did the -psmax change. Maybe just a coincidence?.
     
  12. tear

    tear [H]ard|DCer of the Year 2011

    Messages:
    1,567
    Joined:
    Jul 25, 2011
    If you power-cycled the box and symptoms persist I'd call it a coincidence...

    What are the symptoms of a crash? Freeze? FahCore crash? Something else?
     
  13. tkam

    tkam [H]ard|DCer of the Month - Dec. 2012

    Messages:
    436
    Joined:
    Dec 18, 2007
    The box is resetting on it's own. Except this last time - can't connect via ssh and no ipmi on this box. Will have to wait till I'm back in the office on Monday to check on it and figure out what's wrong with it.
     
  14. ChelseaOilman

    ChelseaOilman [H]ard|Gawd

    Messages:
    1,231
    Joined:
    Mar 27, 2001
    I would be happy to let you experiment on my machine but I'm 2,000 miles away from it right now. :(
     
  15. quickz

    quickz Limp Gawd

    Messages:
    256
    Joined:
    Jul 30, 2012
    Acording to TurionPowerControl, the Opteron 6272 CPU has 7 pstates:
    core 0 pstate 0 (pb0) - En:1 VID:19 FID:14 DID:0.00 Freq:3000 VCore:1.3125
    core 0 pstate 1 (pb1) - En:1 VID:35 FID:8 DID:0.00 Freq:2400 VCore:1.1125
    core 0 pstate 2 (p0) - En:1 VID:41 FID:5 DID:0.00 Freq:2100 VCore:1.0375
    core 0 pstate 3 (p1) - En:0 VID:41 FID:5 DID:0.00 Freq:2100 VCore:1.0375
    core 0 pstate 4 (p2) - En:0 VID:41 FID:5 DID:0.00 Freq:2100 VCore:1.0375
    core 0 pstate 5 (p3) - En:0 VID:41 FID:5 DID:0.00 Freq:2100 VCore:1.0375
    core 0 pstate 6 (p4) - En:1 VID:49 FID:12 DID:1.00 Freq:1400 VCore:0.9375

    But it seems that '-psmax 0' is invalid, after such a setting I find following infomation in the result of 'TurionPowerControl -l':

    Processor Maximum PState: unsupported feature
    Processor Startup PState: 6
    Processor Maximum Operating Frequency: 3000 MHz

    Since the Processor Startup PState is shown as 6 (for my 6272), I think the default setting might be 'TurionPowerControl -psmax 6'.

     
    Last edited: Aug 2, 2012
  16. tear

    tear [H]ard|DCer of the Year 2011

    Messages:
    1,567
    Joined:
    Jul 25, 2011
    Ahh, gotcha (opened docs just now). I made psmax=max performance connection (wrong).

    You're right, -psmax 6 is the way to go.

    So, the manipulation in question moves the lower (performance) limit up. Interesting.
    Almost smells like a flaw in the CPU -- I wouldn't expect this to be settable beyond
    2 (numbooststates). Great find indeed!

    I created small script that measures frequency (incl. turbo) -- http://darkswarm.org/freqcheck.sh
    I'm very curious to see what it reports (w/FAH load) fresh out of power-cycle
    (psmax 6) and with psmax 1 (set via TPC).
     
  17. quickz

    quickz Limp Gawd

    Messages:
    256
    Joined:
    Jul 30, 2012
    I'm sorry to hear that. But on my 4p 6272 box I haven't met such a problem, it has been working fine for several days. It should be noted that I have measured the power consumption before and after '-psmax 1' setting and find it has been increased by ~100Watts. What is your PSU? Is it sufficient for such a change?

    Default setting:
    p6901: 660Watts
    p6903: 670Watts
    p8101: 670Watts
    p8102: 670Watts

    After '-psmax 1':
    p6901: 740Watts
    p6903: 750Watts
    p8101: 780Watts
    p8102: 770Watts

     
  18. quickz

    quickz Limp Gawd

    Messages:
    256
    Joined:
    Jul 30, 2012
    In my test, I have also made a frequency check by use of turbostat. Seems it works, but in some conditions it will report incorrect results. I will try your frequency-check script later, thank you!

    The stock/all-core/one-core frequencies for Opteron 6272 are 2.1/2.4/3.0GHz, and the effective frequencies reported by turbostat during my test are:

    Default setting:
    p6901: Freq=2.26GHz
    p6903: Freq=2.24GHz
    p8101: Freq=2.18GHz
    p8102: Freq=2.20GHz

    After '-psmax 1':
    p6901: Freq=2.40GHz
    p6903: Freq=2.40GHz
    p8101: Freq=2.40GHz
    p8102: Freq=2.40GHz

     
  19. jebo_4jc

    jebo_4jc [H]ard|DCer of the Month - April 2011

    Messages:
    14,628
    Joined:
    Apr 8, 2005
    Wow. Nice ppd there. This is interesting indeed.
     
  20. tear

    tear [H]ard|DCer of the Year 2011

    Messages:
    1,567
    Joined:
    Jul 25, 2011
    NICE. Seems like you've already done all the homework.

    It looks like testing w/my script may be redundant at this point -- I'll leave it for you to decide.

    Cool. We can only hope that AMD carries this... er... feature... into future CPUs :eek:
     
  21. Posidon42

    Posidon42 Limp Gawd

    Messages:
    146
    Joined:
    Dec 20, 2005
    Is this limited to the 62xx series or would the 61xx series benefit at all?
     
  22. tear

    tear [H]ard|DCer of the Year 2011

    Messages:
    1,567
    Joined:
    Jul 25, 2011
    6200s only.

    If your PowerNow is already disabled (w/6100s) then you're already using optimal configuration.
     
  23. Posidon42

    Posidon42 Limp Gawd

    Messages:
    146
    Joined:
    Dec 20, 2005
    cool. not using anything yet as I still don't have a motherboard :( Have to get my car up and running first, but I have a ton of parts lying around just waiting for that to happen.
     
  24. -alias-

    -alias- Limp Gawd

    Messages:
    374
    Joined:
    Jul 15, 2012
    I tried this on my IL 6272 4P rig, but got some error messages! The mobo is a Tyan 8812.
    What have I done wrong?

    The error messages says this:

    vidar@G34Rigg1:~/fah$ TurionPowerControl -psmax 5
    Turion Power States Optimization and Control - by blackshard - v0.43

    ReadPciConfigDwordEx: pread: Success
    Interlagos::getNumBoostStates unable to read boost control register
    ReadPciConfigDwordEx: pread: Success
    ReadPciConfigDwordEx: pread: Success
    Warning: unable to detect multiprocessor machine
    ReadPciConfigDwordEx: pread: Success
    Interlagos.cpp::getMaximumPState - unable to read PCI register
    ReadPciConfigDwordEx: pread: Success
    Interlagos.cpp::setMaximumPState - unable to read PCI register

    Done.
    vidar@G34Rigg1:~/fah$
     
    Last edited: Aug 5, 2012
  25. tear

    tear [H]ard|DCer of the Year 2011

    Messages:
    1,567
    Joined:
    Jul 25, 2011
    sudo modprobe msr
    sudo modprobe cpuid
    sudo TurionPowerControl -psmax 1

    should get you going...
     
  26. Jeanjean

    Jeanjean [H]Lite

    Messages:
    99
    Joined:
    Nov 22, 2011
  27. -alias-

    -alias- Limp Gawd

    Messages:
    374
    Joined:
    Jul 15, 2012
    Thanks tear!

    I did the:

    sudo modprobe msr
    sudo modprobe cpuid
    sudo TurionPowerControl -psmax 1

    And it replayed with: Done

    So far it seems correct, but the machine change in to be very slow, and when it did not progress with another % after 25 min. I stopped folding, and tried restarted the machine. I did not shut down all the way, so I have to push the button. It then came on again at TPF is back to normale.

    It seems that there is some thing wrong in my way of doing this. Should the TurionPowerControl be started before I start fah6 or could it be done while the machine is folding?
     
  28. tear

    tear [H]ard|DCer of the Year 2011

    Messages:
    1,567
    Joined:
    Jul 25, 2011
    It shouldn't matter, just make sure that PowerNow along with Turbo Core / CPB are enabled in the BIOS.

    If it still isn't working for you, you could try and set "performance" scaling governor first thing after boot (before running TPC):
    Code:
    sudo -s
    for i in /sys/devices/system/cpu/cpu[0-9]* ; do echo performance > $i/cpufreq/scaling_governor ; done
    exit
    These are production CPUs we're talking about, right?
     
  29. -alias-

    -alias- Limp Gawd

    Messages:
    374
    Joined:
    Jul 15, 2012
    Yes it is production CPUs! I belive I have the PowerNow disablet, and will look in to it, and belive that is the problem why it did not respond.

    Update:
    Works great, PPD up from 305K to 366K for Project: 8101 (Run 23, Clone 1, Gen 21).:)
    But it produces some more heat and are using 100W+ from the wall, and total consume is now 680W for this rig.

    Thanks tear!
     
    Last edited: Aug 6, 2012
  30. thinklet

    thinklet Limp Gawd

    Messages:
    142
    Joined:
    Jul 4, 2012
    Hi, What is the TPF on the 8101? Thanks, Best regards, Charlie
     
  31. -alias-

    -alias- Limp Gawd

    Messages:
    374
    Joined:
    Jul 15, 2012
    At the moment TPF is 13:05
     
  32. sirmonkey1985

    sirmonkey1985 [H]ard|DCer of the Month - July 2010

    Messages:
    18,611
    Joined:
    Sep 13, 2008
    61k ppd for 100 more watts, worth it.

    just out of curiosity what are the 6100 series chips doing 8101's at?
     
  33. -alias-

    -alias- Limp Gawd

    Messages:
    374
    Joined:
    Jul 15, 2012
  34. quickz

    quickz Limp Gawd

    Messages:
    256
    Joined:
    Jul 30, 2012
    I'm glad to see 4P 6272 now can produce more PPD than 4P 6176se when folding on P8101. What is the power consumption number of these two 4P 6100 rigs when folding?

     
  35. -alias-

    -alias- Limp Gawd

    Messages:
    374
    Joined:
    Jul 15, 2012
    The 6176se rig uses 665W, and 6172 595W from the wall. On the 6172 I did change the PSU a while ago from a 80+ to a 80+ Silver, and the consumption drop from 635 to 595W for that reason. All Power consumption measured with P8101 in folding. Before the update with TurionPowerControl the 6272 rig use 575 - 580W from the wall. It may vary a little from WU to WU. Where I live we have 230V system
     
    Last edited: Aug 6, 2012
  36. sirmonkey1985

    sirmonkey1985 [H]ard|DCer of the Month - July 2010

    Messages:
    18,611
    Joined:
    Sep 13, 2008
    thats not bad at all.. looks like the 6200 series finally became useful again. hope this works on the piledriver chips as well, those numbers should be pretty damn good with the improvements.
     
  37. Jeanjean

    Jeanjean [H]Lite

    Messages:
    99
    Joined:
    Nov 22, 2011
  38. sirmonkey1985

    sirmonkey1985 [H]ard|DCer of the Month - July 2010

    Messages:
    18,611
    Joined:
    Sep 13, 2008
    lets just say F@H is really really slow to changes. so even if AVX was added don't expect to see it in F@H for a few years.
     
  39. quickz

    quickz Limp Gawd

    Messages:
    256
    Joined:
    Jul 30, 2012
    As pointed by -alias-, looks like that Opteron 6100 would lose its performance when folding on P8101. I'm curious to see what made it happen.
    By checking Opteron 6272's working frequencies for different BA WUs (listed int #18 of this thread), we can see that in fact Opteron 6200 (without the "-psmax 1" option) is not good at P8101 either (working at 2.26GHz on P6901 while only 2.18GHz on P8101), but obviously Opteron 6100 plays worse.
    So, just a guess, would it be possible that Opteron 6100 also decrease its working frequency without being noticed when folding on P8101?
     
  40. Nathan_P

    Nathan_P [H]ard DCOTM x2

    Messages:
    3,076
    Joined:
    Mar 2, 2010
    There has been speculation that 8101 favours the faster memory thats available with the 62xx chips but it was only talked about for a couple of days and i haven't seen any more on it for a while now.