setting up phi hardware

Discussion in 'Distributed Computing' started by cdabc123, Feb 28, 2018.

  1. cdabc123

    cdabc123 [H]ard|Gawd

    Messages:
    1,888
    Joined:
    Jun 21, 2016
    I have been trying to setup one of my rigs on boinc to see how it would do and have been having plenty of issues.

    the rigs are x200 intel phi based (64 cores 256 threads) of course windows doesnt like them much. on mining they will do around double what a dual 2699v4 setup does so i was hoping to get close to the same gap on this (150k a day ish) i havent got anywhere near that yet. I was thinking of letting someone remote into one of the boxes and give it a shot with whatever magical touch you people have. if they produce good results i wouldnt mind throwing some comoputing power @ any events (~3,000 threads worth)
     
  2. Skillz

    Skillz [H]ard DCOTY 2017

    Messages:
    20,560
    Joined:
    Aug 14, 2004
    I could be wrong, but I am pretty sure its not possible. The projects would have to support them directly like a GPU.
     
  3. cdabc123

    cdabc123 [H]ard|Gawd

    Messages:
    1,888
    Joined:
    Jun 21, 2016
    thease are the bootable cpus. although finicky they act as a normal chip

    [​IMG]
     
  4. Skillz

    Skillz [H]ard DCOTY 2017

    Messages:
    20,560
    Joined:
    Aug 14, 2004
    Do a clean install of the BOINC client and post the log when you first load it up. See if the BOINC client sees the processors.
     
  5. cdabc123

    cdabc123 [H]ard|Gawd

    Messages:
    1,888
    Joined:
    Jun 21, 2016
    still reinstalling linux on the box for now. any recomendation for distro?
     
  6. Skillz

    Skillz [H]ard DCOTY 2017

    Messages:
    20,560
    Joined:
    Aug 14, 2004
    I use CentOS on my Linux boxen.
     
    McCHillin likes this.
  7. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,317
    Joined:
    Jan 29, 2006
    I've never played with one to know their inner workings. I know there was a lot of interest when they were introduced but no projects were willing to help much. I remember GPUGrid (not sure if it was via PM at the moment) said they tested it once and it did poorly for them. However, I also know that they aren't the best when it comes to development. We had approached Collatz about doing it but the admin Slicker said he was already supporting way too much hardware and he didn't have one to test with. We offered to provide one. He said only if it was on loan as he prided himself with not accepting any form of donations on his project. He never followed up after that with further interest. To my knowledge, none of the DC projects have really given a serious attempt to embrace them. Mostly because there are so few of them out there in the donors hands. Currently the most sold and used computer device is the cell phone which could bring us processing power much greater than the PC's out there. But very few people are willing to use them for DC. And for good reason. So, since users aren't bringing them to DC, many projects won't waste their time developing apps for them. It is a catch 22. Nobody can bring these devices to the table if they aren't supported much. I would love for someone to get a Xeon Phi working at a project to see what they are really capable and not capable of.

    I highly encourage providing screenshots of your successes and failures as I'm sure there are plenty out there even outside our team that may be interested in your efforts.

    We also have a CentOS install guide here if you run into troubles: https://hardforum.com/threads/centos-7-boinc-install-guides.1854569/
     
    McCHillin likes this.
  8. cdabc123

    cdabc123 [H]ard|Gawd

    Messages:
    1,888
    Joined:
    Jun 21, 2016
    just to clear up some confution the intel phis went through a MAJOR reconstruction alittle bit ago. all the x100 phis and before were the ones you needed to write spacific code to run on the pcie cards. many of the x200 phis (the ones i have) are a full blown 64 core cpu (with 16gb super fast ram on board) that can boot any os and run pretty much any program a normal cpu can. no special software is needed. and i did run boinc on the chips but can a bunch of issues with it not using more then a few tiles on the cpu (cores are aranged by tiles and are presented to the os as 5 sockets, boinc doesnt seam to like that) unforcuantly im a pretty busy person so i was looking to see if anyone wanted to remote into the box that knows more about linux then i do to give it a shot.
     
  9. Skillz

    Skillz [H]ard DCOTY 2017

    Messages:
    20,560
    Joined:
    Aug 14, 2004
    I'll give it a shot.
     
  10. cdabc123

    cdabc123 [H]ard|Gawd

    Messages:
    1,888
    Joined:
    Jun 21, 2016
    Sweet I'll pm the info when I get home
     
  11. Skillz

    Skillz [H]ard DCOTY 2017

    Messages:
    20,560
    Joined:
    Aug 14, 2004
    Is it a fresh install? I would rather start from scratch if its not.
     
  12. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,317
    Joined:
    Jan 29, 2006
    I'm wanting to say that BOINC had some kind of a limit on total number of CPU's it could use. Let me know what you find out. Plus if you run into an issue like that and the cc_config flag for setting the number of CPU's doesn't work, try running multiple clients. I'm not sure how to run multiple clients in Linux though.

    I know that BOINC.exe has a limit of ~1,000 work units in progress as we found out testing with multiple clients for GoofyxGrid.
     
  13. cdabc123

    cdabc123 [H]ard|Gawd

    Messages:
    1,888
    Joined:
    Jun 21, 2016
    yea it's a fresh centos install. And yea it had a lot of issues waiting for memory under windows (it has 24gb) another thing is these chips don't support vd-t preventing me from firing abunch of vms and doing it that way
     
  14. ChristianVirtual

    ChristianVirtual [H]ard DCOTM Mar 2016,Aug 2017

    Messages:
    2,266
    Joined:
    Feb 23, 2013
    CentOS is a good choice; Multiple clients are easy to be installed via the source code of boinc and should be able to run in parallel.

    If I’m not wrong we have one black night (or whatever name it had) with F@H and regular core CPUs; but as I recall the performance was not as hoped/hyped.
    But still really worth trying with boinc ; if still needed I’m also willing to help trying

    ;-)
     
  15. cdabc123

    cdabc123 [H]ard|Gawd

    Messages:
    1,888
    Joined:
    Jun 21, 2016
    interestingly enough it still appears intel is moving forward on the bootable cpus (knights mill was released q4 17) and for some programs there is a MASSIVE performance gain from these chips. for example on mining it just about doubles every other chip as a result of the mcdram.
     
  16. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,317
    Joined:
    Jan 29, 2006
    I am still just interested in how well it works. Just the idea of someone seeing a host with so many "cpu's" and it not be a cc_config manipulation would be cool.
     
    ChristianVirtual likes this.
  17. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,317
    Joined:
    Jan 29, 2006
  18. Skillz

    Skillz [H]ard DCOTY 2017

    Messages:
    20,560
    Joined:
    Aug 14, 2004
    I never got the login.
     
  19. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,317
    Joined:
    Jan 29, 2006
    EXT64, did you ever get your older model to work with anything DC?
     
  20. cdabc123

    cdabc123 [H]ard|Gawd

    Messages:
    1,888
    Joined:
    Jun 21, 2016
    In the process of moving. All rigs are unplugged right now. Will update when I get back on it
     
  21. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,317
    Joined:
    Jan 29, 2006
    Any secret places to get those reasonably cheap? lol I see on eBay they are attempting to sell for $400+ still
     
  22. cdabc123

    cdabc123 [H]ard|Gawd

    Messages:
    1,888
    Joined:
    Jun 21, 2016
    CPUs are a decent deal for under 450 each. I can sell motherboards but even with a discount you would be looking at 600 per. (They don't work on normal 3647 boards) every little bit you will see abunch of chips pop up on eBay for 250 each but good luck buying those as every time they pop up I normally buy all of them :p
     
  23. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,317
    Joined:
    Jan 29, 2006
    What boards are these limited too? A specific generation or anything that feeds a typical Xeon or is it pretty much just specific 3647 boards?
     
  24. cdabc123

    cdabc123 [H]ard|Gawd

    Messages:
    1,888
    Joined:
    Jun 21, 2016
    s7200ap (I have abunch of these)
    k1spe

    there are afew others but those are the easiest to get.
     
    Last edited: Mar 14, 2018
  25. Skillz

    Skillz [H]ard DCOTY 2017

    Messages:
    20,560
    Joined:
    Aug 14, 2004
    Well as soon as you get me access ill spend some time playing with it.
     
    cdabc123 likes this.
  26. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,317
    Joined:
    Jan 29, 2006
    Skillz, please screen shot so we have future reference too. It isn't like these come along every day. :D
     
    Skillz likes this.
  27. ChristianVirtual

    ChristianVirtual [H]ard DCOTM Mar 2016,Aug 2017

    Messages:
    2,266
    Joined:
    Feb 23, 2013
    And run a few FAH on it ; we had one other donor with a Phi but I never got a log file for my database ... would be nice to see some a7 core running on it.
     
  28. EXT64

    EXT64 [H]ard DCOTM x2

    Messages:
    363
    Joined:
    Mar 27, 2013
    No Gil, I never got anything meaningful running well on the old Phis. Though I think (and hope) these will be a different story.
     
  29. Skillz

    Skillz [H]ard DCOTY 2017

    Messages:
    20,560
    Joined:
    Aug 14, 2004
    If I can get it operating on all the cores I'll get a tutorial written up. Gotta get it working first.
     
  30. EXT64

    EXT64 [H]ard DCOTM x2

    Messages:
    363
    Joined:
    Mar 27, 2013
    Yeah, and if it does work, drive over to cdabc123 's place for me and grab me a couple. Thanks!
     
    Skillz, ChristianVirtual and cdabc123 like this.
  31. Skillz

    Skillz [H]ard DCOTY 2017

    Messages:
    20,560
    Joined:
    Aug 14, 2004
    Ill grab them all!
     
    Pocatello and cdabc123 like this.
  32. Skillz

    Skillz [H]ard DCOTY 2017

    Messages:
    20,560
    Joined:
    Aug 14, 2004
    Any word on this? We could use them right now.
     
  33. cdabc123

    cdabc123 [H]ard|Gawd

    Messages:
    1,888
    Joined:
    Jun 21, 2016
    still in the prosses of moving. sadly it will probrably be a month or so
     
  34. Skillz

    Skillz [H]ard DCOTY 2017

    Messages:
    20,560
    Joined:
    Aug 14, 2004
    Please tell me you're a week early with moving!
     
  35. cdabc123

    cdabc123 [H]ard|Gawd

    Messages:
    1,888
    Joined:
    Jun 21, 2016
    It was actually pushed a week later. :( Let me see if I can find some place I'm currently staying that can run one without me going insain.
     
    Red Falcon and Skillz like this.
  36. Skillz

    Skillz [H]ard DCOTY 2017

    Messages:
    20,560
    Joined:
    Aug 14, 2004
    That would be awesome!
     
    Red Falcon likes this.
  37. Gilthanis

    Gilthanis [H]ard|DCer of the Year - 2014

    Messages:
    7,317
    Joined:
    Jan 29, 2006
  38. cdabc123

    cdabc123 [H]ard|Gawd

    Messages:
    1,888
    Joined:
    Jun 21, 2016

    Unfortunately not and it may be Abit before anything happens. I currently have all the servers in storage. Im also trying to get my hands on a 7235 or simular so I could easily run multiple instances