BOINC client optimized for multisocket machines [Linux]

tear

[H]ard|DCer of the Year 2011
Joined
Jul 25, 2011
Messages
1,568
Here's NUMA-aware, affinity-setting BOINC client for Ubuntu 12.04 LTS (Precise Pangolin)
[basically a matured version of http://hardforum.com/showpost.php?p=1040429103&postcount=9].

It's meant for dedicated, multisocket boxes and its objective is to improve performance.
Some old performance data (4P Socket F box) are available there: http://boinc.berkeley.edu/trac/ticket/853.
Testing w/more current setup is pending.

It currently needs to be installed manually but soon enough we'll have a PPA running which
will simplify the processes of installation and upgrades.

Modified version is based on Ubuntu's stock BOINC -- 7.0.65.
If you're currently running version newer than 7.0.65 (from third party PPA or otherwise),
note that downgrade to the modified version has not been exercised -- YMMV.

To install:
  1. Open fresh terminal
  2. Determine BOINC version and make sure it's not newer than 7.0.65 (or not installed)
    Code:
    dpkg-query -W boinc-client
  3. Satisfy BOINC dependencies:
    Code:
    sudo apt-get -y install libxss1 libwxbase2.8-0 libwxgtk2.8-0
  4. Download BOINC packages:
    Code:
    wget -O boinc-client_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb http://darkswarm.org/ubuntu-boinc/boinc-client_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb
    wget -O boinc-manager_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb http://darkswarm.org/ubuntu-boinc/boinc-manager_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb
    wget -O boinc_7.0.65+dfsg-3~ubuntu12.04.1~a3_all.deb http://darkswarm.org/ubuntu-boinc/boinc_7.0.65+dfsg-3~ubuntu12.04.1~a3_all.deb
    wget -O libboinc-app7_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb http://darkswarm.org/ubuntu-boinc/libboinc-app7_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb
    wget -O libboinc7_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb http://darkswarm.org/ubuntu-boinc/libboinc7_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb
  5. Install BOINC packages
    Code:
    sudo dpkg -i boinc_7.0.65+dfsg-3~ubuntu12.04.1~a3_all.deb boinc-client_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb boinc-manager_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb libboinc7_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb libboinc-app7_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb
  6. Start boinc manager: click the dash icon and type "boinc" to find it; once found, press Enter to launch
  7. Proceed normally

To check if it's working correctly, examine /var/lib/boinc-client/stdoutae.txt, it should now print extra [numa] lines, e.g.
Code:
30-Nov-2013 18:16:56 [---] Starting BOINC client version 7.0.65 for x86_64-pc-linux-gnu
30-Nov-2013 18:16:56 [---] log flags: file_xfer, sched_ops, task
30-Nov-2013 18:16:56 [---] Libraries: libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
30-Nov-2013 18:16:56 [---] Data directory: /var/lib/boinc-client
30-Nov-2013 18:16:56 [---] Processor: 48 AuthenticAMD AMD Engineering Sample [Family 16 Model 9 Stepping 1]
30-Nov-2013 18:16:56 [---] Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr npt lbrv svm_lock nrip_save pausefilter
30-Nov-2013 18:16:56 [---] OS: Linux: 3.2.0-45-generic
30-Nov-2013 18:16:56 [---] Memory: 31.42 GB physical, 32.00 GB virtual
30-Nov-2013 18:16:56 [---] Disk: 426.95 GB total, 398.63 GB free
30-Nov-2013 18:16:56 [---] Local time is UTC -8 hours
30-Nov-2013 18:16:56 [---] No usable GPUs found
30-Nov-2013 18:16:56 [---] Config: GUI RPCs allowed from:
30-Nov-2013 18:16:56 [World Community Grid] URL http://www.worldcommunitygrid.org/; Computer ID 2575649; resource share 100
30-Nov-2013 18:16:56 [World Community Grid] General prefs: from World Community Grid (last modified 22-Nov-2013 20:43:13)
30-Nov-2013 18:16:56 [World Community Grid] Computer location: home
30-Nov-2013 18:16:56 [---] General prefs: using separate prefs for home
30-Nov-2013 18:16:56 [---] Reading preferences override file
30-Nov-2013 18:16:56 [---] Preferences:
30-Nov-2013 18:16:56 [---]    max memory usage when active: 24128.98MB
30-Nov-2013 18:16:56 [---]    max memory usage when idle: 28954.78MB
30-Nov-2013 18:16:56 [---]    max disk usage: 100.00GB
30-Nov-2013 18:16:56 [---]    (to change preferences, visit a project web site or select Preferences in the Manager)
30-Nov-2013 18:16:56 [---] gui_rpc_auth.cfg is empty - no GUI RPC password protection
30-Nov-2013 18:16:56 [---] Not using a proxy
Initialization completed
30-Nov-2013 18:16:56 [---] [numa] /proc/cpuinfo: 48 CPUs
30-Nov-2013 18:16:56 [---] [numa] examining node 0
30-Nov-2013 18:16:56 [---] [numa] node 0: 6 CPUs (map: 0000003f)
30-Nov-2013 18:16:56 [---] [numa] examining node 1
30-Nov-2013 18:16:56 [---] [numa] node 1: 6 CPUs (map: 00000fc0)
30-Nov-2013 18:16:56 [---] [numa] examining node 2
30-Nov-2013 18:16:56 [---] [numa] node 2: 6 CPUs (map: 0003f000)
30-Nov-2013 18:16:56 [---] [numa] examining node 3
30-Nov-2013 18:16:56 [---] [numa] node 3: 6 CPUs (map: 00fc0000)
30-Nov-2013 18:16:56 [---] [numa] examining node 4
30-Nov-2013 18:16:56 [---] [numa] node 4: 6 CPUs (map: 3f000000)
30-Nov-2013 18:16:56 [---] [numa] examining node 5
30-Nov-2013 18:16:56 [---] [numa] node 5: 6 CPUs (map: fc0000000)
30-Nov-2013 18:16:56 [---] [numa] examining node 6
30-Nov-2013 18:16:56 [---] [numa] node 6: 6 CPUs (map: 3f000000000)
30-Nov-2013 18:16:56 [---] [numa] examining node 7
30-Nov-2013 18:16:56 [---] [numa] node 7: 6 CPUs (map: fc0000000000)
30-Nov-2013 18:16:56 [---] [numa] found 48 CPUs
30-Nov-2013 18:16:56 [World Community Grid] [numa] Found CPU 0 (at node 0)
30-Nov-2013 18:16:56 [World Community Grid] Restarting task FAHV_x3AVMbINfbA_0336649_0426_1 using fahv version 706 in slot 20
30-Nov-2013 18:16:56 [World Community Grid] [numa] Found CPU 1 (at node 0)
30-Nov-2013 18:16:56 [World Community Grid] Restarting task faah377324_ZINC00194283_x4GVMaINleA_00_0 using faah version 715 in slot 21
30-Nov-2013 18:16:56 [World Community Grid] [numa] Found CPU 2 (at node 0)
30-Nov-2013 18:16:56 [World Community Grid] Restarting task faah377324_ZINC00227031_x4GVMaINleA_00_0 using faah version 715 in slot 3
30-Nov-2013 18:16:56 [World Community Grid] [numa] Found CPU 3 (at node 0)
30-Nov-2013 18:16:56 [World Community Grid] Restarting task faah377324_ZINC23478308_x4GVMaINleA_00_0 using faah version 715 in slot 9
Once it starts churning out units, you can also run:
Code:
ps -eopid=,pcpu=,psr=,comm= | awk '{ if ($2 > 10) print }'  | sort -n -k3,3

You should see a list of workers sorted by CPU number, e.g.:
Code:
33292 99.4   0 wcgrid_faah_7.1
34038 98.8   1 wcgrid_fahv_vin
34061 98.5   2 wcgrid_fahv_vin
32450 99.9   3 wcgrid_faah_7.1
33931 99.8   4 wcgrid_mcm1_7.2
33188 99.9   5 wcgrid_faah_7.1
32443 99.9   6 wcgrid_faah_7.1
34055 98.4   7 wcgrid_fahv_vin
33848 99.9   8 wcgrid_mcm1_7.2
32436 99.9   9 wcgrid_faah_7.1
25708 98.7  10 wcgrid_fahv_vin
32365 99.9  11 wcgrid_faah_7.1
33634 99.9  12 wcgrid_mcm1_7.2
33492 99.9  13 wcgrid_faah_7.1
33787 99.9  14 wcgrid_mcm1_7.2
34372 85.8  15 wcgrid_fahv_vin
32347 99.9  16 wcgrid_faah_7.1
19265 98.4  17 wcgrid_fahv_vin
31950 99.9  18 wcgrid_faah_7.1
31433 99.9  19 wcgrid_faah_7.1
31648 99.9  20 wcgrid_faah_7.1
33919 99.9  21 wcgrid_mcm1_7.2
34067 98.5  22 wcgrid_fahv_vin
34367 93.7  23 wcgrid_fahv_vin
34043 98.6  24 wcgrid_fahv_vin
34071 98.8  25 wcgrid_fahv_vin
33941 99.9  26 wcgrid_mcm1_7.2
33494 99.9  27 wcgrid_mcm1_7.2
31219 99.9  28 wcgrid_faah_7.1
31058 99.9  29 wcgrid_faah_7.1
34041 99.6  30 wcgrid_faah_7.1
33991 99.9  31 wcgrid_mcm1_7.2
29909 99.8  32 wcgrid_cep2_qch
33903 99.9  33 wcgrid_faah_7.1
34046 98.6  34 wcgrid_fahv_vin
19797 98.8  35 wcgrid_fahv_vin
33927 99.9  36 wcgrid_mcm1_7.2
34069 98.1  37 wcgrid_fahv_vin
32682 99.9  38 wcgrid_faah_7.1
30129 99.9  39 wcgrid_faah_7.1
34063 97.9  40 wcgrid_fahv_vin
33899 99.9  41 wcgrid_mcm1_7.2
34077 91.8  42 wcgrid_fahv_vin
33682 99.9  43 wcgrid_mcm1_7.2
33759 99.7  44 wcgrid_mcm1_7.2
32668 99.8  45 wcgrid_faah_7.1
31905 99.9  46 wcgrid_faah_7.1
33797 99.9  47 wcgrid_mcm1_7.2

Special thanks go to Grandpa_01 for letting me test the code on his machine :)
 
Last edited:
Couple questions for you tear. When I switched to BOINC I installed your Ubuntu Folding Appliance on a spare HD and installed it in my 2P socket F machine. Thank you and the Area 51 crew! That was the easiest Ubuntu install I ever did.

My 1st question is, is this new code something I should run or is it already incorporated in the Folding Appliance?

2nd question is, I notice at the BOINC site there seems to be a new official version of the BOINC client, version 7.2.42. Is there any need to update to this newer version? Will you incorporate the new version into the Ubuntu Folding Appliance at some point?
 
2nd question is, I notice at the BOINC site there seems to be a new official version of the BOINC client, version 7.2.42. Is there any need to update to this newer version? Will you incorporate the new version into the Ubuntu Folding Appliance at some point?

The answer to whether you should update really depends on your setup. Here is the revision history which is updated with each version.

http://boinc.berkeley.edu/trac/wiki/VersionHistory
 
Couple questions for you tear. When I switched to BOINC I installed your Ubuntu Folding Appliance on a spare HD and installed it in my 2P socket F machine. Thank you and the Area 51 crew! That was the easiest Ubuntu install I ever did.

My 1st question is, is this new code something I should run or is it already incorporated in the Folding Appliance?

2nd question is, I notice at the BOINC site there seems to be a new official version of the BOINC client, version 7.2.42. Is there any need to update to this newer version? Will you incorporate the new version into the Ubuntu Folding Appliance at some point?

I think tear is traveling at the moment, the boinc included in the appliance is the optimized version.

The last I heard he was not going to update the version until he saw a reason to do so. ;)
 
The answer to whether you should update really depends on your setup. Here is the revision history which is updated with each version.

http://boinc.berkeley.edu/trac/wiki/VersionHistory

I'd be very interested in a newer version being developed, specifically the following updates interest me:

Version 7.2.38
Fix: Various CPU and GPU scheduling issues.

Version 7.2.34

released: 3 December 2013
New: Sub second CPU throttling. (All Platforms)
Fix: Fix several issues with CPU Throttling and GPU applications. (All platforms)

Version 7.2.28

released: 8 November 2013

New: OpenCL CPU support for Intel and AMD CPUs.
New: Add new AMD GPU descriptions.
New: Add new Windows version descriptions. (Windows Only)
Fix: Various changes to CPU and GPU scheduling and work-fetch.

I'm running 6*7950's alongside 4*6176's.
I'd be willing to part with 0.25btc for this update for your troubles :)

Regards,

Customminer.
 
grr trying to install this on Ubuncraptu ( Ubuntu) and am getting errors
:(
r select Preferences in the Manager)
dir_open: Could not open directory 'slots'.
01-Dec-2014 15:29:54 [---] GUI RPC bind to port 31416 failed: 98
gstate.init() failed
Error Code: -180
linux@linux-H8SGL:~$
 
Shadowmeph, you need to provide more context. Paste as much terminal output
as you can. Specifically, all commands (and their output) leading to the error.

Customminer, somehow I missed your post. Do you still need this?
 
In case it matters, I think they eventually pulled the sub second throttling back out because the wrapper apps didn't work right with it. So, you may want to double check the client version notes if that was the selling point for updating. I honestly don't remember off the top of my head.

http://boinc.berkeley.edu/dev/forum_thread.php?id=8378&postid=52198#52198

Preliminary Change Log 7.2.38 --> 7.2.39:


client: shouldn't show suspend/resume msgs for CPU throttling.
client: fix bugs in accounting elapsed time.
client: don't use sub-second CPU throttling.

The developer forgot that the wrapper has a 1-second poll for suspend and resume, so sub-second throttling won't work properly for wrapper apps. Revert to a variant of the old scheme, in which the min of the suspended and resumed periods is 1 sec. Also, fix task start/suspend/resume log messages.
That sounds like they pulled it, but I'm not sure. It could just be when you use vbox wrapper apps.
Reading through client changes, there were quite a few adjustments in regards to sub second throttling and GPU apps as well. A newer client does support more hardware such as MIPS and ASICs. So, it would probably be best to get the latest and greatest.
 
Last edited:
Shadowmeph, you need to provide more context. Paste as much terminal output
as you can. Specifically, all commands (and their output) leading to the error.

Customminer, somehow I missed your post. Do you still need this?
Yeah, if you're able to update it to the latest version of boinc & it's still compatible with boinctasks then totalls. 0.25btc still sound good? :)
 
Bump.
How you getting on with the client?
What is it you need to do in order to get the client updated?

Would it be possible to get the latest version - Version 7.4.36
 
Last edited:
Sorry about the delay. I'm testing 7.4.37 w/NUMA patch at the moment. ETA is ca. 4 hours.
 
Is anyone here using Tear's optimized client and if so, what are you seeing in the way if improved results? I have a 4P with 6180 SEs that is a dedicated Crunching machine and is currently running BOINC ver 7.2.39. under Ubuntu 12.04 LTS.

I may have missed it somewhere, but could not find any information about the results from this NUMA aware BOINC client. I'm not married to a particular version of BOINC unless it provides some performance reason to change it. So, before I take off and make any changes, I'd like to hear from those that have used this modified client for multi-socket rigs.
 
There isn't typically performance differences between the versions of BOINC but rather bug fixes, hardware support, etc.. in the newer versions. The improvements is the NUMA feature. However, I don't have any numbers. tear is probably the man to talk to about that.
 
Gil, yes I was thinking of the NUMA-aware BOINC client when asking about the "performance" that people are seeing. Tear's work on Folding performance was widely known and put into use by a great many Folders across lots of teams.

Before I go through the work to implement this NUMA-aware BOINC client, I wanted to see if I could get a feel for what to expect from it. When I used to Fold, I was using Tear's creations to push the best performance out of Bigadvs on this 4P. It has been dedicated to Crunching for over a year now, so I'm interested in what people are experiencing with this custom client for Crunching. :D
 
Hi!

Tear, did you publish the modified code? This topic seems to be dead, so someone else could step in and make it work again.

I have to say I could try to upstream a NUMA change, if there's Bitcoin involved :) Customminer, are you still interested?
 
Back
Top