[H]ard|Forum  

Go Back   [H]ard|Forum > Bits & Bytes > Distributed Computing > DC Guides
Register [H]ard|OCP Community Today's Posts Search

Reply
 
Thread Tools Search this Thread
  #1  
Old 01-29-2012, 10:13 PM
musky [H]ard|DCer of the Year 2012, 4.9 Years
 
Status: musky is offline  
Multiple Processor G34 Checklist

Thanks to tear and the rest of the Area 51 crew for this list.
8/28/2012 - Updated to TPC version 0.43
2/19/2013 - Updated to TPC version 0.44-rc2 (previously)
10/9/2013 - remove i2c-tools/i2c-dev references

Here are some basic things to have installed and to check for with a 2p/4p G34 system. Please continue to join the irc channel for more specific help. You need these basics in place first, though.

Packages

If you plan to use TurionPowerControl prior to a reboot, the following modules need to be activated via modprobe:
Code:
sudo modprobe cpuid
sudo modprobe msr
Adding the modules to /etc/modules will activate them when your system loads the OS, so you will not need to modprobe them after each reboot.

Code:
sudo nano /etc/modules
You should see something like this:
Code:
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.

lp
rtc
Add msr and cpuid to this list:

Code:
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.

lp
rtc
msr
cpuid
Exit and save. The next time you reboot, these modules will load with the OS.

The build-essential package will give you the compilers you will need. You may have downloaded part of this package (gcc) when you installed thekraken. Go ahead and get the rest of it.
Code:
sudo apt-get install build-essential
TurionPowerControl is a very useful utility for system information on any AMD-based system. Version 0.41 was the most recent as of this post. To install this, we are going to:
1. Download the tarball
2. Un-tar the tarball
3. move to the src directory
4. compile it (make and sudo make install)
5. Make a hard link from TurionPowerControl to tpc

First, make sure you have the ncurses-dev package installed. With Ubuntu, use apt-get install:
Code:
sudo apt-get install ncurses-dev
Then go through the install steps:
Code:
cd ~
wget http://turionpowercontrol.googlecode.com/files/tpc-0.44-rc2.tar.gz
tar xvf tpc-0.44-rc2.tar.gz
cd tpc-0.44-rc2/src
make
sudo make install
One common use case - show your current CPU temperatures:
Code:
sudo tpc -temp
Just type 'tpc' to see a list of all of the options.


Configuration

Temps - need to be under 70C, else you will run into thermal throttling
Code:
sudo tpc -temp
Memory
- populated evenly, 4 sticks/CPU minimum
- if your board has 8 slots/CPU, the slot closest to each CPU should not be populated
- run
Code:
grep MemTotal /sys/devices/system/node/node[0-9]*/*
Make sure that:
1. There are 8 nodes total
2. Each node has the correct amount of memory (they should normally be the same amount unless you have an odd memory configuration)
Example output - 16 x 1Gb memory modules:
Code:
sys/devices/system/node/node0/meminfo:Node 0 MemTotal:        2096696 kB
sys/devices/system/node/node1/meminfo:Node 1 MemTotal:        2095744 kB
sys/devices/system/node/node2/meminfo:Node 2 MemTotal:        2097152 kB
sys/devices/system/node/node3/meminfo:Node 3 MemTotal:        2097152 kB
sys/devices/system/node/node4/meminfo:Node 4 MemTotal:        2097152 kB
sys/devices/system/node/node5/meminfo:Node 5 MemTotal:        2097152 kB
sys/devices/system/node/node6/meminfo:Node 6 MemTotal:        2097152 kB
sys/devices/system/node/node7/meminfo:Node 7 MemTotal:        2097152 kB
- Check memory timings:
Code:
sudo tpc -dram
All timings should be identical across all nodes and DCTs (MaxRdLatency values may vary a little, which is fine)

Sample ouput - basic 1333 C9 memory (most memory should default to something close to this by default)

Code:
Turion Power States Optimization and Control - by blackshard - v0.41

DRAM Configuration Status

Node 0 ---
DCT0: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=2 Trfc2=0 Trfc3=0 MaxRdLatency=52
DCT1: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=2 Trfc2=0 Trfc3=0 MaxRdLatency=52

Node 1 ---
DCT0: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=2 Trfc2=0 Trfc3=0 MaxRdLatency=50
DCT1: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=2 Trfc2=0 Trfc3=0 MaxRdLatency=51

Node 2 ---
DCT0: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=2 Trfc2=0 Trfc3=0 MaxRdLatency=52
DCT1: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=2 Trfc2=0 Trfc3=0 MaxRdLatency=51

Node 3 ---
DCT0: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=2 Trfc2=0 Trfc3=0 MaxRdLatency=49
DCT1: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=2 Trfc2=0 Trfc3=0 MaxRdLatency=50

Node 4 ---
DCT0: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=2 Trfc2=0 Trfc3=0 MaxRdLatency=51
DCT1: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=2 Trfc2=0 Trfc3=0 MaxRdLatency=51

Node 5 ---
DCT0: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=2 Trfc2=0 Trfc3=0 MaxRdLatency=50
DCT1: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=2 Trfc2=0 Trfc3=0 MaxRdLatency=49

Node 6 ---
DCT0: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=2 Trfc2=0 Trfc3=0 MaxRdLatency=52
DCT1: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=2 Trfc2=0 Trfc3=0 MaxRdLatency=52

Node 7 ---
DCT0: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=2 Trfc2=0 Trfc3=0 MaxRdLatency=51
DCT1: memory type: DDR3 frequency: 1332 MHz
Tcl=9 Trcd=9 Trp=9 Tras=24 Access Mode:1T Trtp=5 Trc=33 Twr=9 Trrd=4 Tcwl=7 Tfaw=20
TrwtWB=8 TrwtTO=7 Twtr=5 Twrrd=2 Twrwr=4 Trdrd=3 Tref=2 Trfc0=0 Trfc1=2 Trfc2=0 Trfc3=0 MaxRdLatency=49


Done.
- PowerNow - this should be disabled for all ES and MC systems. It should be enabled for all non-ES LI and AD systems so you can get the turbo multipliers. Run the following:
Code:
ls -l /sys/devices/system/cpu/cpu[0-9]*/cpufreq
This should return 'ls: cannot access /sys/devices/system/cpu/cpu[0-9]*/cpufreq: No such file or directory' if you have PowerNow diabled. If this returns something else, you have PowerNow enabled.

BIOS settings to check
- PowerNow set to Disabled (see above for the check)
- NUMA/SRAT set to Enabled
- Node Interleaving set to Disabled
- Bank/channel interleaving set to Auto or Enabled
- Tyan boards - Ungang DCTs set to Always

If you have any problems or questions, please join our irc channel for assistance. Several people that frequent the channel are familiar with all of this, and it will be easier to help you there.
__________________
The [H] 4p Mafia!
Proud member of the [H]orde - [H]ardOCP Folding@Home team #33
"If it ain't broke, you're not trying" - Red Green
"Need a lift? Then shove a jack up your ass." - Jennifer Spencer (Sudden Impact)
"Never let the truth stand in the way of a good story" - Mark Twain

Last edited by musky; 10-09-2013 at 08:53 AM.
Reply With Quote
  #2  
Old 01-29-2012, 10:35 PM
chaandan Limp Gawd, 3.4 Years
 
Status: chaandan is offline  
One more awesome guide...keep it up Musky!!
Reply With Quote
  #3  
Old 01-29-2012, 10:36 PM
Grandpa_01 [H]ard|DCer of the Year 2013, 3.4 Years
 
Status: Grandpa_01 is offline  
I just learned more about Linux commands by reading this than I have learned in a year of running it. Thanks musky
__________________
Folding rigs
1- 6274 - 4P @ 3Ghz
3- 6348 - 4P @ 3.8Ghz
5 - 4650 - 4P @ depends on there mood
2 - 980X @ 4.3Ghz
1 - 2700K 4.6Ghz, 3-GTX 680
1 - ASUS G73JH Laptop 740QM @ 2Ghz, ATI 5870M
Currentley folding / crunching 472 cores 3 - GTX 680's and 1 - GTX 770
Reply With Quote
  #4  
Old 01-29-2012, 10:54 PM
firedfly [H]ard|DCer of the Month - February 2012, 3.5 Years
 
Status: firedfly is offline  
What you all don't realize is that he makes these guides so he knows what to do the next time he sets up a system!

Very useful stuff. Thanks for compiling it all together musky.
Reply With Quote
  #5  
Old 01-29-2012, 11:55 PM
Untitledone [H]ard|DCer of the Month - April 2012, 3.7 Years
 
Status: Untitledone is offline  
Better than his pile of .txt on his desktop
__________________
Gaming rig: E5620 4 cores 8 Threads 3.61Ghz, ASUS P6TSE, 24GB DDR3 1866, GTX 470
Folding1: Dual x5570 8 cores 16 threads 3.2Ghz, ASUS Z8NA-D6C 6GB DDR3 1600
Folding2: Octo 8431 48 cores 48 threads 2.4Ghz, TYAN S4985+M4985, 22GB DDR2 ECC
Reply With Quote
  #6  
Old 01-30-2012, 12:01 AM
WOLF_TEAM_LEADER [H]Lite, 2.8 Years
 
Status: WOLF_TEAM_LEADER is offline  
Thanks musky.
Should help when I am setting up the new 4P system within a couple of weeks
__________________
P67: i7 2600k @ 5.0GHz Water Cooled + P8P67 WS Revolution + 2x GTX780 ACX SC + 16GB 2133MHz + MaxRevo 1500W + 3x Dell U2713HM (7680x1440) + HAF X
G34: 4x 6174 w/ CPU-370 + H8QGi-F + 32GB 1333MHz + 9x120mm External WCing
Plus assorted folding/mining systems.
Reply With Quote
  #7  
Old 01-30-2012, 12:30 AM
Linden Gawd, 9.1 Years
 
Status: Linden is offline  
Thank you very much, Musky.
__________________
1. Supermicro H8QGi+-F, 4 X AMD Opteron 6172, Seasonic M12D 850, 32GB Crucial Ballilstix 1600, 4 X Cooler Master Hyper 212+, Cooler Master HAF X modified, Linux; 2. Supermicro H8QGi+-F, 4 X AMD Opteron 6174, Corsair AX850, 32GB G.Skill Ripjaws DDR3 1333, 4 X Cooler Master Hyper 212+, Cooler Master HAF 932 modified, Linux; 3. Supermicro H8QGL-iF, 2 X Opteron 6174, 2 X Opteron 6176, Enermax Revolution 85+ 950, 64GB G.Skill Ripjaws DDR3 1600, 4 X Cooler Master Hyper 212+, Rocketfish/Lian Li modified, Linux; 4. Supermicro H8QGi+F, 4 X AMD Opteron 6174, Corsair AX750, 32GB G.Skill Ripjaws DDR3 1333, 4 X Cooler Master Hyper 212+, custom wood/metal platform, Linux
Reply With Quote
  #8  
Old 01-30-2012, 09:09 AM
rascal65 Limp Gawd, 2.8 Years
 
Status: rascal65 is offline  
Thanks Musky
Great guide, very useful.
__________________
#1 4P SM, 6166he, 32gb, 60gb ssd, 1250w, HAF case
#2 SM 4P 6180, 32gb, 250 gb, 1250w,........................thermal meltdown attempting resurrection
Reply With Quote
  #9  
Old 01-30-2012, 10:33 AM
pjkenned [H]ard|Gawd, 4.8 Years
 
Status: pjkenned is offline  
+1 for musky on this.
__________________
http://www.ServeTheHome.com
Reply With Quote
  #10  
Old 01-30-2012, 10:48 AM
tjmagneto [H]ard|DCer of the Month - Aug. 2011/Aug. 2014, 6.2 Years
 
Status: tjmagneto is online now  
Quote:
Originally Posted by Untitledone View Post
Better than his pile of .txt on his desktop
Hey now I resemble that remark.
__________________
The [H] 4p Mafia!
Voltron: Opteron 6176 x 4 @ 2.65 Ghz
Habu 2.0: i7-4790K Cañón del Diablo, EVGA 780ti SC ACX
A garage full of spare parts

"Just a heads up: We're gonna have a superconductor turned up full blast and pointed at you for the duration of this next test. Honestly, we're throwing science at the wall here" -Cave Johnson

“He’s so dumb he thinks a Mexican border pays rent” -Foghorn Leghorn
Reply With Quote
  #11  
Old 01-30-2012, 02:25 PM
Untitledone [H]ard|DCer of the Month - April 2012, 3.7 Years
 
Status: Untitledone is offline  
Quote:
Originally Posted by tjmagneto View Post
Hey now I resemble that remark.
How big is your pile?
__________________
Gaming rig: E5620 4 cores 8 Threads 3.61Ghz, ASUS P6TSE, 24GB DDR3 1866, GTX 470
Folding1: Dual x5570 8 cores 16 threads 3.2Ghz, ASUS Z8NA-D6C 6GB DDR3 1600
Folding2: Octo 8431 48 cores 48 threads 2.4Ghz, TYAN S4985+M4985, 22GB DDR2 ECC
Reply With Quote
  #12  
Old 01-30-2012, 03:00 PM
W.Feather [H]Moderator, 5.0 Years
 
Status: W.Feather is offline  
however big it is i bet mine is bigger ...
__________________
Moderator of: For Sale/For Trade, Overclocking & Cooling, and Distributed Computing

Desktop: 3770K | Maximus Formula V | 32GB G.Skill Sniper 1600 | 3x Corsair Force GT3 120gb |Asus GTX680 | Seasonic x1250

Folding/Simulations: 2x E5-2670 | Gigabyte GA-7PESH1 | 64GB ECC Ram | Nvidia GT640 | Nvidia K10 | HP P420 | Some Drives | Crappy Laptop Drive|Antec TP-750 New
Reply With Quote
  #13  
Old 02-01-2012, 03:08 PM
kevinksu [H]Lite, 5.1 Years
 
Status: kevinksu is offline  
Quote:
Originally Posted by Grandpa_01 View Post
I just learned more about Linux commands by reading this than I have learned in a year of running it. Thanks musky
that is what you get when learning with passion
Reply With Quote
  #14  
Old 02-01-2012, 09:45 PM
jebo_4jc [H]ard|DCer of the Month - April 2011, 9.6 Years
 
Status: jebo_4jc is offline  
Great work everyone
Reply With Quote
  #15  
Old 03-03-2012, 10:52 AM
DSee 2[H]4U, 7.2 Years
 
Status: DSee is offline  
Great great job.. Not just for your contributions, but also for sharing the knowledge.
__________________
TBD
Reply With Quote
  #16  
Old 03-04-2012, 06:55 PM
Axdrenalin [H]ard|DCer of the Month - Nov. 2009, 10.8 Years
 
Status: Axdrenalin is online now  
Musky, Thanks for sharing this guide with us. I just printed it out for my own use, and I'm in the process of putting together a 2P unit right now.

Robert
__________________
Main Boxen - i7-970 / ASUS Sabretooth X58 / 12GB DDR3 / Win 8 Pro / 2x EVGA GTX 760 2GB SLI

HEAT = Axdrenalin ~ EBAY = Axdrenalin ~ Email = robATdatawerksDOTnet (You know the routine) apt-get install common-sense

"Never Forget, even for an instant, that the one and only reason anybody has for taking your gun away is to make you weaker than he is, so he can do something to you that you wouldn't allow him to do if you were equipped to prevent it. This goes for burglars, muggers, and rapists, and even more so for policemen, bureaucrats, and politicians."
Reply With Quote
  #17  
Old 02-23-2013, 03:54 PM
bowlinra Limp Gawd, 2.5 Years
 
Status: bowlinra is offline  
I'm trying to update my tpc 0.41 to the 0.44-rc2

I'm get the follow error on the make statement:
Code:
bowlinra@amd4p:~/tpc-0.44-rc2/src$ make

/bin/sh: 1: svn: not found
/bin/sh: 1: svnversion: not found
g++ -O2   -MMD -MF obj/x86_64/.sysdep-linux.d -MT obj/x86_64/sysdep-linux.o -c -o obj/x86_64/sysdep-linux.o sysdep-linux.cpp
sysdep-linux.cpp:2:18: fatal error: term.h: No such file or directory
compilation terminated.
make: *** [obj/x86_64/sysdep-linux.o] Error 1
bowlinra@amd4p:~/tpc-0.44-rc2/src$
Any Ideas?
Reply With Quote
  #18  
Old 02-23-2013, 11:33 PM
tear [H]ard|DCer of the Year 2011, 3.3 Years
 
Status: tear is offline  
From OP:
Quote:
First, make sure you have the ncurses-dev package installed. With Ubuntu, use apt-get install:
Code:
sudo apt-get install ncurses-dev
Reply With Quote
  #19  
Old 02-24-2013, 05:04 PM
bowlinra Limp Gawd, 2.5 Years
 
Status: bowlinra is offline  
That was it.. Apparently, I looked passed it several times..

Thanks!
Reply With Quote
  #20  
Old 03-05-2013, 02:57 PM
knopflerbruce Limp Gawd, 7.0 Years
 
Status: knopflerbruce is offline  
How do you run tpc at startup, with predefined OC settings? I know I saw a script once, but couldn't find it now.
Reply With Quote
Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 03:13 PM.


Powered by Vbulletin - Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Copyright 2000 - 2014 KB Networks, Inc.