BOINC client optimized for multisocket machines [Linux]

Discussion in 'DC Guides' started by tear, Feb 8, 2014.

  1. tear

    tear [H]ard|DCer of the Year 2011

    Messages:
    1,568
    Joined:
    Jul 25, 2011
    Here's NUMA-aware, affinity-setting BOINC client for Ubuntu 12.04 LTS (Precise Pangolin)
    [basically a matured version of http://hardforum.com/showpost.php?p=1040429103&postcount=9].

    It's meant for dedicated, multisocket boxes and its objective is to improve performance.
    Some old performance data (4P Socket F box) are available there: http://boinc.berkeley.edu/trac/ticket/853.
    Testing w/more current setup is pending.

    It currently needs to be installed manually but soon enough we'll have a PPA running which
    will simplify the processes of installation and upgrades.

    Modified version is based on Ubuntu's stock BOINC -- 7.0.65.
    If you're currently running version newer than 7.0.65 (from third party PPA or otherwise),
    note that downgrade to the modified version has not been exercised -- YMMV.

    To install:
    1. Open fresh terminal
    2. Determine BOINC version and make sure it's not newer than 7.0.65 (or not installed)
      Code:
      dpkg-query -W boinc-client
    3. Satisfy BOINC dependencies:
      Code:
      sudo apt-get -y install libxss1 libwxbase2.8-0 libwxgtk2.8-0
    4. Download BOINC packages:
      Code:
      wget -O boinc-client_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb http://darkswarm.org/ubuntu-boinc/boinc-client_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb
      wget -O boinc-manager_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb http://darkswarm.org/ubuntu-boinc/boinc-manager_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb
      wget -O boinc_7.0.65+dfsg-3~ubuntu12.04.1~a3_all.deb http://darkswarm.org/ubuntu-boinc/boinc_7.0.65+dfsg-3~ubuntu12.04.1~a3_all.deb
      wget -O libboinc-app7_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb http://darkswarm.org/ubuntu-boinc/libboinc-app7_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb
      wget -O libboinc7_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb http://darkswarm.org/ubuntu-boinc/libboinc7_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb
    5. Install BOINC packages
      Code:
      sudo dpkg -i boinc_7.0.65+dfsg-3~ubuntu12.04.1~a3_all.deb boinc-client_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb boinc-manager_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb libboinc7_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb libboinc-app7_7.0.65+dfsg-3~ubuntu12.04.1~a3_amd64.deb
    6. Start boinc manager: click the dash icon and type "boinc" to find it; once found, press Enter to launch
    7. Proceed normally

    To check if it's working correctly, examine /var/lib/boinc-client/stdoutae.txt, it should now print extra [numa] lines, e.g.
    Code:
    30-Nov-2013 18:16:56 [---] Starting BOINC client version 7.0.65 for x86_64-pc-linux-gnu
    30-Nov-2013 18:16:56 [---] log flags: file_xfer, sched_ops, task
    30-Nov-2013 18:16:56 [---] Libraries: libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
    30-Nov-2013 18:16:56 [---] Data directory: /var/lib/boinc-client
    30-Nov-2013 18:16:56 [---] Processor: 48 AuthenticAMD AMD Engineering Sample [Family 16 Model 9 Stepping 1]
    30-Nov-2013 18:16:56 [---] Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr npt lbrv svm_lock nrip_save pausefilter
    30-Nov-2013 18:16:56 [---] OS: Linux: 3.2.0-45-generic
    30-Nov-2013 18:16:56 [---] Memory: 31.42 GB physical, 32.00 GB virtual
    30-Nov-2013 18:16:56 [---] Disk: 426.95 GB total, 398.63 GB free
    30-Nov-2013 18:16:56 [---] Local time is UTC -8 hours
    30-Nov-2013 18:16:56 [---] No usable GPUs found
    30-Nov-2013 18:16:56 [---] Config: GUI RPCs allowed from:
    30-Nov-2013 18:16:56 [World Community Grid] URL http://www.worldcommunitygrid.org/; Computer ID 2575649; resource share 100
    30-Nov-2013 18:16:56 [World Community Grid] General prefs: from World Community Grid (last modified 22-Nov-2013 20:43:13)
    30-Nov-2013 18:16:56 [World Community Grid] Computer location: home
    30-Nov-2013 18:16:56 [---] General prefs: using separate prefs for home
    30-Nov-2013 18:16:56 [---] Reading preferences override file
    30-Nov-2013 18:16:56 [---] Preferences:
    30-Nov-2013 18:16:56 [---]    max memory usage when active: 24128.98MB
    30-Nov-2013 18:16:56 [---]    max memory usage when idle: 28954.78MB
    30-Nov-2013 18:16:56 [---]    max disk usage: 100.00GB
    30-Nov-2013 18:16:56 [---]    (to change preferences, visit a project web site or select Preferences in the Manager)
    30-Nov-2013 18:16:56 [---] gui_rpc_auth.cfg is empty - no GUI RPC password protection
    30-Nov-2013 18:16:56 [---] Not using a proxy
    Initialization completed
    30-Nov-2013 18:16:56 [---] [numa] /proc/cpuinfo: 48 CPUs
    30-Nov-2013 18:16:56 [---] [numa] examining node 0
    30-Nov-2013 18:16:56 [---] [numa] node 0: 6 CPUs (map: 0000003f)
    30-Nov-2013 18:16:56 [---] [numa] examining node 1
    30-Nov-2013 18:16:56 [---] [numa] node 1: 6 CPUs (map: 00000fc0)
    30-Nov-2013 18:16:56 [---] [numa] examining node 2
    30-Nov-2013 18:16:56 [---] [numa] node 2: 6 CPUs (map: 0003f000)
    30-Nov-2013 18:16:56 [---] [numa] examining node 3
    30-Nov-2013 18:16:56 [---] [numa] node 3: 6 CPUs (map: 00fc0000)
    30-Nov-2013 18:16:56 [---] [numa] examining node 4
    30-Nov-2013 18:16:56 [---] [numa] node 4: 6 CPUs (map: 3f000000)
    30-Nov-2013 18:16:56 [---] [numa] examining node 5
    30-Nov-2013 18:16:56 [---] [numa] node 5: 6 CPUs (map: fc0000000)
    30-Nov-2013 18:16:56 [---] [numa] examining node 6
    30-Nov-2013 18:16:56 [---] [numa] node 6: 6 CPUs (map: 3f000000000)
    30-Nov-2013 18:16:56 [---] [numa] examining node 7
    30-Nov-2013 18:16:56 [---] [numa] node 7: 6 CPUs (map: fc0000000000)
    30-Nov-2013 18:16:56 [---] [numa] found 48 CPUs
    30-Nov-2013 18:16:56 [World Community Grid] [numa] Found CPU 0 (at node 0)
    30-Nov-2013 18:16:56 [World Community Grid] Restarting task FAHV_x3AVMbINfbA_0336649_0426_1 using fahv version 706 in slot 20
    30-Nov-2013 18:16:56 [World Community Grid] [numa] Found CPU 1 (at node 0)
    30-Nov-2013 18:16:56 [World Community Grid] Restarting task faah377324_ZINC00194283_x4GVMaINleA_00_0 using faah version 715 in slot 21
    30-Nov-2013 18:16:56 [World Community Grid] [numa] Found CPU 2 (at node 0)
    30-Nov-2013 18:16:56 [World Community Grid] Restarting task faah377324_ZINC00227031_x4GVMaINleA_00_0 using faah version 715 in slot 3
    30-Nov-2013 18:16:56 [World Community Grid] [numa] Found CPU 3 (at node 0)
    30-Nov-2013 18:16:56 [World Community Grid] Restarting task faah377324_ZINC23478308_x4GVMaINleA_00_0 using faah version 715 in slot 9
    Once it starts churning out units, you can also run:
    Code:
    ps -eopid=,pcpu=,psr=,comm= | awk '{ if ($2 > 10) print }'  | sort -n -k3,3
    You should see a list of workers sorted by CPU number, e.g.:
    Code:
    33292 99.4   0 wcgrid_faah_7.1
    34038 98.8   1 wcgrid_fahv_vin
    34061 98.5   2 wcgrid_fahv_vin
    32450 99.9   3 wcgrid_faah_7.1
    33931 99.8   4 wcgrid_mcm1_7.2
    33188 99.9   5 wcgrid_faah_7.1
    32443 99.9   6 wcgrid_faah_7.1
    34055 98.4   7 wcgrid_fahv_vin
    33848 99.9   8 wcgrid_mcm1_7.2
    32436 99.9   9 wcgrid_faah_7.1
    25708 98.7  10 wcgrid_fahv_vin
    32365 99.9  11 wcgrid_faah_7.1
    33634 99.9  12 wcgrid_mcm1_7.2
    33492 99.9  13 wcgrid_faah_7.1
    33787 99.9  14 wcgrid_mcm1_7.2
    34372 85.8  15 wcgrid_fahv_vin
    32347 99.9  16 wcgrid_faah_7.1
    19265 98.4  17 wcgrid_fahv_vin
    31950 99.9  18 wcgrid_faah_7.1
    31433 99.9  19 wcgrid_faah_7.1
    31648 99.9  20 wcgrid_faah_7.1
    33919 99.9  21 wcgrid_mcm1_7.2
    34067 98.5  22 wcgrid_fahv_vin
    34367 93.7  23 wcgrid_fahv_vin
    34043 98.6  24 wcgrid_fahv_vin
    34071 98.8  25 wcgrid_fahv_vin
    33941 99.9  26 wcgrid_mcm1_7.2
    33494 99.9  27 wcgrid_mcm1_7.2
    31219 99.9  28 wcgrid_faah_7.1
    31058 99.9  29 wcgrid_faah_7.1
    34041 99.6  30 wcgrid_faah_7.1
    33991 99.9  31 wcgrid_mcm1_7.2
    29909 99.8  32 wcgrid_cep2_qch
    33903 99.9  33 wcgrid_faah_7.1
    34046 98.6  34 wcgrid_fahv_vin
    19797 98.8  35 wcgrid_fahv_vin
    33927 99.9  36 wcgrid_mcm1_7.2
    34069 98.1  37 wcgrid_fahv_vin
    32682 99.9  38 wcgrid_faah_7.1
    30129 99.9  39 wcgrid_faah_7.1
    34063 97.9  40 wcgrid_fahv_vin
    33899 99.9  41 wcgrid_mcm1_7.2
    34077 91.8  42 wcgrid_fahv_vin
    33682 99.9  43 wcgrid_mcm1_7.2
    33759 99.7  44 wcgrid_mcm1_7.2
    32668 99.8  45 wcgrid_faah_7.1
    31905 99.9  46 wcgrid_faah_7.1
    33797 99.9  47 wcgrid_mcm1_7.2
    Special thanks go to Grandpa_01 for letting me test the code on his machine :)
     
    Last edited: Sep 29, 2014
  2. tear

    tear [H]ard|DCer of the Year 2011

    Messages:
    1,568
    Joined:
    Jul 25, 2011
    Placeholder for future content.
     
  3. ChelseaOilman

    ChelseaOilman [H]ard|Gawd

    Messages:
    1,270
    Joined:
    Mar 27, 2001
    Couple questions for you tear. When I switched to BOINC I installed your Ubuntu Folding Appliance on a spare HD and installed it in my 2P socket F machine. Thank you and the Area 51 crew! That was the easiest Ubuntu install I ever did.

    My 1st question is, is this new code something I should run or is it already incorporated in the Folding Appliance?

    2nd question is, I notice at the BOINC site there seems to be a new official version of the BOINC client, version 7.2.42. Is there any need to update to this newer version? Will you incorporate the new version into the Ubuntu Folding Appliance at some point?
     
  4. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    5,946
    Joined:
    Jan 29, 2006
    The answer to whether you should update really depends on your setup. Here is the revision history which is updated with each version.

    http://boinc.berkeley.edu/trac/wiki/VersionHistory
     
  5. Grandpa_01

    Grandpa_01 [H]ard|DCer of the Year 2013

    Messages:
    1,176
    Joined:
    Jun 4, 2011
    I think tear is traveling at the moment, the boinc included in the appliance is the optimized version.

    The last I heard he was not going to update the version until he saw a reason to do so. ;)
     
  6. ChelseaOilman

    ChelseaOilman [H]ard|Gawd

    Messages:
    1,270
    Joined:
    Mar 27, 2001
    Sounds good to me. If it isn't broke, don't fix it. :)
     
  7. customminer

    customminer n00bie

    Messages:
    45
    Joined:
    Mar 17, 2014
    I'd be very interested in a newer version being developed, specifically the following updates interest me:

    Version 7.2.38
    Fix: Various CPU and GPU scheduling issues.

    Version 7.2.34

    released: 3 December 2013
    New: Sub second CPU throttling. (All Platforms)
    Fix: Fix several issues with CPU Throttling and GPU applications. (All platforms)

    Version 7.2.28

    released: 8 November 2013

    New: OpenCL CPU support for Intel and AMD CPUs.
    New: Add new AMD GPU descriptions.
    New: Add new Windows version descriptions. (Windows Only)
    Fix: Various changes to CPU and GPU scheduling and work-fetch.

    I'm running 6*7950's alongside 4*6176's.
    I'd be willing to part with 0.25btc for this update for your troubles :)

    Regards,

    Customminer.
     
  8. Shadowmeph

    Shadowmeph Limp Gawd

    Messages:
    154
    Joined:
    Dec 3, 2011
    grr trying to install this on Ubuncraptu ( Ubuntu) and am getting errors
    :(
    r select Preferences in the Manager)
    dir_open: Could not open directory 'slots'.
    01-Dec-2014 15:29:54 [---] GUI RPC bind to port 31416 failed: 98
    gstate.init() failed
    Error Code: -180
    linux@linux-H8SGL:~$
     
  9. tear

    tear [H]ard|DCer of the Year 2011

    Messages:
    1,568
    Joined:
    Jul 25, 2011
    Shadowmeph, you need to provide more context. Paste as much terminal output
    as you can. Specifically, all commands (and their output) leading to the error.

    Customminer, somehow I missed your post. Do you still need this?
     
  10. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    5,946
    Joined:
    Jan 29, 2006
    In case it matters, I think they eventually pulled the sub second throttling back out because the wrapper apps didn't work right with it. So, you may want to double check the client version notes if that was the selling point for updating. I honestly don't remember off the top of my head.

    http://boinc.berkeley.edu/dev/forum_thread.php?id=8378&postid=52198#52198

    That sounds like they pulled it, but I'm not sure. It could just be when you use vbox wrapper apps.
    Reading through client changes, there were quite a few adjustments in regards to sub second throttling and GPU apps as well. A newer client does support more hardware such as MIPS and ASICs. So, it would probably be best to get the latest and greatest.
     
    Last edited: Dec 2, 2014
  11. customminer

    customminer n00bie

    Messages:
    45
    Joined:
    Mar 17, 2014
    Yeah, if you're able to update it to the latest version of boinc & it's still compatible with boinctasks then totalls. 0.25btc still sound good? :)
     
  12. customminer

    customminer n00bie

    Messages:
    45
    Joined:
    Mar 17, 2014
    Bump.
    How you getting on with the client?
    What is it you need to do in order to get the client updated?

    Would it be possible to get the latest version - Version 7.4.36
     
    Last edited: Jan 6, 2015
  13. tear

    tear [H]ard|DCer of the Year 2011

    Messages:
    1,568
    Joined:
    Jul 25, 2011
    Sorry about the delay. I'm testing 7.4.37 w/NUMA patch at the moment. ETA is ca. 4 hours.
     
  14. texinga

    texinga Limp Gawd

    Messages:
    182
    Joined:
    Dec 5, 2011
    Is anyone here using Tear's optimized client and if so, what are you seeing in the way if improved results? I have a 4P with 6180 SEs that is a dedicated Crunching machine and is currently running BOINC ver 7.2.39. under Ubuntu 12.04 LTS.

    I may have missed it somewhere, but could not find any information about the results from this NUMA aware BOINC client. I'm not married to a particular version of BOINC unless it provides some performance reason to change it. So, before I take off and make any changes, I'd like to hear from those that have used this modified client for multi-socket rigs.
     
  15. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    5,946
    Joined:
    Jan 29, 2006
    There isn't typically performance differences between the versions of BOINC but rather bug fixes, hardware support, etc.. in the newer versions. The improvements is the NUMA feature. However, I don't have any numbers. tear is probably the man to talk to about that.
     
  16. texinga

    texinga Limp Gawd

    Messages:
    182
    Joined:
    Dec 5, 2011
    Gil, yes I was thinking of the NUMA-aware BOINC client when asking about the "performance" that people are seeing. Tear's work on Folding performance was widely known and put into use by a great many Folders across lots of teams.

    Before I go through the work to implement this NUMA-aware BOINC client, I wanted to see if I could get a feel for what to expect from it. When I used to Fold, I was using Tear's creations to push the best performance out of Bigadvs on this 4P. It has been dedicated to Crunching for over a year now, so I'm interested in what people are experiencing with this custom client for Crunching. :D