PSA - DIMES is something everyone can run, no impact on other DC'ing

Discussion in 'Distributed Computing' started by CaptainUnlikely, Sep 30, 2013.

  1. CaptainUnlikely

    CaptainUnlikely [H]ard|DCer of the Month - May 2014

    Messages:
    311
    Joined:
    Mar 20, 2013
    Just wanted to make everyone aware that DIMES is an NCI project, meaning non CPU intensive. This means it can be run alongside other projects, whether that's F@H, one of the many projects using BOINC, or others, without affecting production.

    The aim of DIMES is to "study the structure and topology of the Internet" via pings and traceroutes. This means it requires a constant net connection to do any work, but it should barely use any bandwidth. It also requires the corresponding version of Java to be installed (32 bit for the 32 bit client, 64 bit for the 64 bit client).

    Gilthanis wrote an excellent guide on getting DIMES set up, which can be found here - http://hardforum.com/showthread.php?t=1768712

    I would urge anybody not currently running DIMES to get it set up, as it's quick and easy, won't affect your other production, and it helps our team's ranking!
     
  2. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,902
    Joined:
    Jan 29, 2006
    I would also encourage those not familiar with the client to stick with version .5.5.3 instead of version 1. Version 1 is currently only running as a service and makes it difficult to change settings and monitor. You can find the client here: http://www.netdimes.org/new/

    This project as well as FAH also helps us out at the DC-Vault . http://www.dc-vault.com/index.php


    [​IMG]
     
    Last edited: May 21, 2015
  3. CaptainUnlikely

    CaptainUnlikely [H]ard|DCer of the Month - May 2014

    Messages:
    311
    Joined:
    Mar 20, 2013
  4. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,902
    Joined:
    Jan 29, 2006
  5. musky

    musky [H]ard|DCer of the Year 2012

    Messages:
    3,154
    Joined:
    Dec 14, 2009
    Is there any way to confirm that the client is doing what it should? I installed this on one machine, and I see stuff like this in the "Messages" tab:

    Code:
    9/30/13 3:21 PM dimes.measurements.Measurements execute
    INFO: Executing default op.execute
    9/30/13 3:21 PM dimes.measurements.Measurements execute
    INFO: PING 198.94.49.1 ICMP
    9/30/13 3:21 PM dimes.measurements.Measurements execute
    INFO: Executing default op.execute
    9/30/13 3:21 PM dimes.measurements.Measurements execute
    INFO: TRACEROUTE 198.94.49.1 ICMP
    9/30/13 3:21 PM dimes.measurements.Measurements execute
    INFO: Executing default op.execute
    9/30/13 3:22 PM dimes.measurements.Measurements execute
    INFO: PING 192.118.76.98 ICMP
    9/30/13 3:22 PM dimes.measurements.Measurements execute
    INFO: Executing default op.execute
    9/30/13 3:22 PM dimes.measurements.Measurements execute
    INFO: TRACEROUTE 192.118.76.98 ICMP
    9/30/13 3:22 PM dimes.measurements.Measurements execute
    INFO: Executing default op.execute
    9/30/13 3:22 PM dimes.measurements.Measurements execute
    INFO: PING 80.160.54.1 ICMP
    9/30/13 3:22 PM dimes.measurements.Measurements execute
    INFO: Executing default op.execute
    9/30/13 3:22 PM dimes.measurements.Measurements execute
    INFO: TRACEROUTE 80.160.54.1 ICMP
    9/30/13 3:22 PM dimes.measurements.Measurements execute
    INFO: Executing default op.execute
    9/30/13 3:23 PM dimes.measurements.Measurements execute
    INFO: PING 168.234.153.1 ICMP
    9/30/13 3:23 PM dimes.measurements.Measurements execute
    INFO: Executing default op.execute
    9/30/13 3:23 PM dimes.measurements.Measurements execute
    INFO: TRACEROUTE 168.234.153.1 ICMP
    9/30/13 3:23 PM dimes.measurements.Measurements execute
    INFO: Executing default op.execute
    I'm showing nothing for User Statistics, which is probably correct since I haven't been running very long. Is there anything else I can look at to confirm that the client is doing what it is supposed to?

    Also, does anyone have this running on a CLI Linux install? I'd like to verify your statement about running alongside F@H, but my folding machines are running CLI Linux and the install instructions for Linux are lacking.

    Thanks
     
  6. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,902
    Joined:
    Jan 29, 2006
    musky, your messages look like what all of mine look like. I run on Windows, so I can't help you on the Linux setup. I haven't actually tried installing it on Linux yet. As far as the points, check in routinely through the day. They tend to update multiple times. Glad to see more taking interest.
     
  7. CaptainUnlikely

    CaptainUnlikely [H]ard|DCer of the Month - May 2014

    Messages:
    311
    Joined:
    Mar 20, 2013
    Indeed, looks the same as mine. I think the points tend to update every hour or so, but it took me a while before they started to show up - after that, they seem to be updating more regularly.
    I don't have a Linux box to set up DIMES on yet, but I hope to do so when I have time. Unless someone else does it first, I'll do a guide for it when I do set it up, however I'm very much a Linux novice so bear with me :)
    Thanks for joining up musky.
     
  8. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,902
    Joined:
    Jan 29, 2006
    I only did Linux VM's when I was running BOINC projects that didn't support Windows. Yes there are some projects that refuse to support Windows. :mad: I would extend the offer to try it out on Linux, but my personal time is filled to the max with the baby right now. I barely get to check my stats or troubleshoot the random hardware glitch at home before bed.

    I also see musky has points posting now. http://stats.free-dc.org/stats.php?page=user&proj=dimes&name=musky
     
  9. musky

    musky [H]ard|DCer of the Year 2012

    Messages:
    3,154
    Joined:
    Dec 14, 2009
    Yep, I am posting points... :).

    I have a Linux VM I am playing with now as well. I think it will be pretty easy with a GUI. A CLI install may require something different.
     
  10. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,902
    Joined:
    Jan 29, 2006
    I did scroll through their *ahem* "forums" to see if there was any useful info on Linux. Didn't find a lot and most of it was from like 2009.
     
  11. musky

    musky [H]ard|DCer of the Year 2012

    Messages:
    3,154
    Joined:
    Dec 14, 2009
    With a huge assist from tear, I got to the same point described in this thread. Given that it was started over a year ago and has still not bee addressed, I'm going to have to say the DIMES isn't going to work with Linux.
     
  12. CaptainUnlikely

    CaptainUnlikely [H]ard|DCer of the Month - May 2014

    Messages:
    311
    Joined:
    Mar 20, 2013
    I'm currently working my way through trying to get DIMES installed and working on Linux myself...complete novice here as I've never used Linux before so I'm making very slow progress so far.
    I don't even think I'm as far as you are, musky, but if I get to that point I will post back to say if I have the same issue or not.
     
  13. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,902
    Joined:
    Jan 29, 2006
    Yeah...this is certainly something we need to get ironed out if we expect our FAH brethren to run it alongside the FAH client. I will try and find some time to play with Linux again. (I too am not a Linux adept)

    I'm not sure if this helps, but here is an older version listing with some instructions from this website: http://www.cs.huji.ac.il/labs/danss/p2p/dimes-linux/download2.html

    And GoogleFu tends to bring up a lot of old posts in various forums from like 2005. http://www.xtremepccentral.com/forums/showthread.php?p=163417#post163417
     
    Last edited: Oct 1, 2013
  14. CaptainUnlikely

    CaptainUnlikely [H]ard|DCer of the Month - May 2014

    Messages:
    311
    Joined:
    Mar 20, 2013
    I got it installed thanks to some help from Linux-savvy friends, but ran into the same issue as you, musky, with one difference - I didn't get the last line saying "Checking Java version......1". No option to enter my username or team or anything, so it doesn't seem to be working. I ran out of time last night but will try again tonight.
     
  15. musky

    musky [H]ard|DCer of the Year 2012

    Messages:
    3,154
    Joined:
    Dec 14, 2009
    There is a configuration file - /usr/local/DIMES/Classes/Base/conf/properties.xml. You will find mention of editing it directly with no luck.

    I am running something now with the Sun java binary:
    Code:
    dave@ubuntu:~$ ps aux | grep java
    root      2508  0.2  3.0 1020328 31452 pts/1   Sl   07:26   0:01 /usr/lib/jvm/jr
    e1.7.0_40//bin/java -Djava.util.logging.config.file=/usr/local/DIMES/scripts_lin
    ux/../Classes/Base/\conf\logging.properties -classpath /usr/local/DIMES/scripts_
    linux/../Classes/ AgentLauncher /usr/local/DIMES/scripts_linux/../Classes//JARs 
    /usr/local/DIMES/scripts_linux/../Classes/Base//conf/properties.xml
    dave      3737  0.0  0.0  13584   916 pts/2    S+   07:35   0:00 grep --color=au
    to java
    A lot of these paths look incorrect - extra back slashes and forward slashes - maybe some bad scripts in the tarball?
     
  16. CaptainUnlikely

    CaptainUnlikely [H]ard|DCer of the Month - May 2014

    Messages:
    311
    Joined:
    Mar 20, 2013
    I've made no further progress with this yet, going to try installing it afresh when I have a few hours to set aside to grapple with it. I'll perhaps try an older client and see if it helps, thanks for the links.
     
  17. bigted

    bigted [H]ard|DCer of the Month - Oct. 2013

    Messages:
    3,334
    Joined:
    Jan 20, 2006
    i have put my work laptop to work on DIMES :)
     
  18. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,902
    Joined:
    Jan 29, 2006
    Awesome, glad to have more on board. :) And just in time. Within the next 3 days we should overtake Team Ninja.
     
  19. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,902
    Joined:
    Jan 29, 2006
    I see musky is ramping up some good numbers. :)
     
  20. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,902
    Joined:
    Jan 29, 2006
    We should overtake Team Ninja later today and then Computing Family in about a week. I'm still waiting on my motherboard, but when it arrives it will probably be mostly DIMES, BOINC GPU, and then mainly used as a file server in the home. I will have to see how many VM's it will run under that load since it will only have a dual core.
     
  21. CaptainUnlikely

    CaptainUnlikely [H]ard|DCer of the Month - May 2014

    Messages:
    311
    Joined:
    Mar 20, 2013
    Excellent, nice work everyone!
     
  22. musky

    musky [H]ard|DCer of the Year 2012

    Messages:
    3,154
    Joined:
    Dec 14, 2009
    Welcome, Grandpa - I knew you could do it!

    One interesting fact - Windows 2000 works just fine with the older 5.5.1 client and an older version of java and a bit of patience. I can throw together a quick VM guide if anyone is interested, or make my VM disk file available for copy.
     
  23. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,902
    Joined:
    Jan 29, 2006
    Yeah..I have a P3 Laptop running Win2k pretty well on DIMES. I believe it is running .5.5.3 like the rest of my systems. Both P3's are in my top 3 DIMES producers: http://stats.free-dc.org/stats.php?page=user&proj=dimes&name=Coleslaw One of them has XP the other is 2k.

    The reason I usually stick to XP though is because I tend to put BOINC even in VM's for certain projects and not all of the BOINC projects play well with Win2K. Just something for others to keep in mind if they plan on the VM route.
     
    Last edited: Oct 7, 2013
  24. bigted

    bigted [H]ard|DCer of the Month - Oct. 2013

    Messages:
    3,334
    Joined:
    Jan 20, 2006
    Does running multiple vms on the same machine improve performance?
     
  25. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,902
    Joined:
    Jan 29, 2006
    On DIMES, yes because you can run one client per VM. So, the more VM's you run the better you will produce at DIMES. However, VM's may interfere with performance on other clients like FAH and BOINC. So, manage accordingly.
     
  26. musky

    musky [H]ard|DCer of the Year 2012

    Messages:
    3,154
    Joined:
    Dec 14, 2009
    If by "improve performance" you mean more points per day, then yes..
     
  27. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,902
    Joined:
    Jan 29, 2006
    Some others may be able to chime in on which OS's work best when running multiple VM's. I believe I was recommend to use Win2k8 R2 due to its better memory management. However, I don't have a copy to play with to see if there really is a difference on non-server grade equipment. That is all I have for now. :(
     
  28. musky

    musky [H]ard|DCer of the Year 2012

    Messages:
    3,154
    Joined:
    Dec 14, 2009
    I am seeing what I think is a performance hit with Win2K compared to at least Win7. I'll keep playing - I think I do have a Win2K8 R2 iso somewhere as well
     
  29. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,902
    Joined:
    Jan 29, 2006
    I would also give it time. I have seen some machines that for whatever reason just outperformed others. I can't explain why my P3 laptops that are also running BOINC at full load are outperforming in DIMES compared to my i7 that actually has ran longer with less issues. I don't know the ryme or reason nor the logic behind it. However, some just do better. You will also find that points can fluctuate quite a bit from day to day too.
     
  30. Grandpa_01

    Grandpa_01 [H]ard|DCer of the Year 2013

    Messages:
    1,175
    Joined:
    Jun 4, 2011
    And it only took a couple of tries. The first try I did my usual and only read the instructions I wanted to :p the second I did as I usually do after a mess-up on the 1st try. I read all the instructions and it worked, hmmph imagine that :D
     
  31. EXT64

    EXT64 DCOTM x3

    Messages:
    494
    Joined:
    Mar 27, 2013
    Wow, have you all seen some of the places the DIMES client claims to have pinged? I think I even saw Iran pop up once - hopefully the NSA won't come after me :D
     
  32. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,902
    Joined:
    Jan 29, 2006
    The NSA probably already has you as part of their BOTNET. :eek:
     
  33. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,902
    Joined:
    Jan 29, 2006
    We should be moving up another position today overtaking Crunching Family. Keep up the good work guys.
     
  34. CaptainUnlikely

    CaptainUnlikely [H]ard|DCer of the Month - May 2014

    Messages:
    311
    Joined:
    Mar 20, 2013
    Looks like we're outproducing everyone by a significant margin, actually, so there should be more overtakes on the cards soon :)
    OcUK have recently ramped up their production though, although they're a long way below us right now they may be worth keeping an eye on.
     
  35. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,902
    Joined:
    Jan 29, 2006
    I keep squeezing what I can out. The only thing I could foresee on my side would be if I shut down a few of the VM's I have. Otherwise, I doubt I remove the clients. We should keep a pretty consistent if not growing production. :) I agree that we need to watch other teams though. You never know when a large farm will decide to setup a 100 VM's and just make a massive push. On the bright side, I now have a copy of Server 2008 R2 to play with. It is now on my long list of things to do/test.
     
  36. CaptainUnlikely

    CaptainUnlikely [H]ard|DCer of the Month - May 2014

    Messages:
    311
    Joined:
    Mar 20, 2013
    Exactly, they seem to be waking up to farming VMs, and you never know when one or more of their members might make a big push on it. Like you say though, DIMES is something with no impact on other projects, so most of our clients should in theory never be shut down, bar machines getting formatted or replaced if they don't re-install the client, so I agree our production should really stay stable or increase over time. We do look to have a decent gap between us and any potential threats below, and we're gaining quickly on those above us.
     
  37. musky

    musky [H]ard|DCer of the Year 2012

    Messages:
    3,154
    Joined:
    Dec 14, 2009
    AH HA!
     
    Last edited: Apr 9, 2014
  38. CaptainUnlikely

    CaptainUnlikely [H]ard|DCer of the Month - May 2014

    Messages:
    311
    Joined:
    Mar 20, 2013
  39. Brak710

    Brak710 [H]ard|Gawd

    Messages:
    1,424
    Joined:
    Oct 27, 2008
    I have like 48GB RAM and 2 quad core xeons free on my ESX farm at home... What would be the best method for me to run a bunch of these? I have two fiber lines at my house, so I can split between the two.

    Should I just run a really small XP install with it, or would linux be better? I'd like to be able to make one and clone it 40 times if possible.
     
  40. bigted

    bigted [H]ard|DCer of the Month - Oct. 2013

    Messages:
    3,334
    Joined:
    Jan 20, 2006
    well linux has the benefit of not having to buy/worry about licensing XP.

    musky seems to be using centos, i'm sure there will be instructions to follow :)