The [H]arbringiner is alive

Discussion in 'Distributed Computing' started by Patriot, Jun 29, 2011.

  1. Patriot

    Patriot [H]ard|DCer of the Month - March 2011/June 2013/De

    Messages:
    2,496
    Joined:
    Dec 15, 2010
    Meet [H]arbringer

    [​IMG]

    Specs

    48cores @ 3.48ghz
    refclock 240, nb 2160

    SM GL w/ OCNG4 (thank you everyone who had a hand in this...)
    16x 2gb G.skill cas 6 ddr3 1600 @ 1600
    Antec HCP1200w ( a real trooper... looking to get it some help soon... 1432w AC est 1267DC)
    2x RX360 rads
    1x RS240 rad
    8x GT-15s
    Laing-d5 swiftech rebranded
    4x DT G34 snipers (skripka told me to ask DT... and thus the G34 sniper was born)

    32gb samsung ssd
    Ubuntu 10.10 (destined to change)





    Performance


    Fah performance: [Classified: Need to know] :)

    Cinebench 11.5 45.3
    I haven't bothered updating hwbot to the new WR yet.
    Partly because I already own it.


    Sponsors: (Also known as the best folding team ever)

    For:
    TPC ,OCNG4 , The Kraken, Langouste, d3sak, and all those random scripts...
    Tear, musky, dsee firedfly and the rest of the faithful testers of tears handiwork...
    Thankyou!



    Revision History
    • 0.8= 2.6 ghz serial wc 1333 cas 9 ecc ram 59c load @ 2.6
    • 0.9 = 2.8 cores 50-53c
    • 1.0 = final overclock target 3.1ghz (3ghz achieved)
    • 1.1 = ripjawsX cas 6 ram
    • 1.2 = spd hack to cas 5
    • 1.3 = fan upgrade GT-15s
    • 1.4 = replace xspc pumps with Laing D5 and RS240 with RX240 or larger radiator vertically in front of case Combined with 1.9
    • 1.5 = 1500w or greater platinum psu Canceled
    • 1.9 =Swap to SM GL for OCNG and 3x 12v aux plugs for cpu voltage stability

    Future planned revisions

    • 2.0 = IL ES chips aiming for 3.5ghz-4ghz clock

    Current parts list:
    • 4x AMD Magny Cours ES chips 2ghz suggested clock.
    • Supermicro H8QGL-iF+
    • 16 x G.skill ripjawsX ddr3 1600 cas 6
    • hitachi 500gb sata drive
    • 1x laing D5
    • 2x XSPC RX360
    • 1x XSPC RS240
    • 4x DT G34 Snipers
    • < 7' primochill tubing with anti kink coil
    • 8x GT-15s
    • Enzo copper ram sinks (not going to count)
    • Antec HCP 1200w
    • Xigmatek Elysium (now heavily modified)


    Revision Walk-through, Original build log coming soon....



    Rev 0.8

    Limited to 2.6ghz because of the serial wc loop could not keep temperatures comfortable for the last cpu in the line.

    [​IMG]




    Rev 0.9

    wc loop redone to have 2 cpus per radiator, 2.8ghz temps 50-53c
    enzo copper ramsinks added to vrms (donated by Rtype) ​

    [​IMG]


    Rev 1.0

    Final overclock 3ghz nb 2200 (has been pushed beyond but not stable for folding past 3ghz)


    Rev 1.1

    RipjawsX ddr3 1600 cas 6 ram (generously donated by Dsee)

    [​IMG]



    Rev 1.2

    spd hack to cas 5 1333 (board recognizes as cas6, early Guinea pig of tears d3sak with him guiding me though it)



    Rev 1.3

    Fan upgrade GT-15s, I had originally planned on doing this in the summer but 1 of the fans on the rs240 started to fail so I went ahead and swapped all the rad fans out. I may or may not order another set in the summer and do push pull. Temps mid 50s for 3ghz load.

    [​IMG]


    Rev 1.5

    Postponed/scrapped.


    Rev 1.9

    Swapped to a SM GL board, gained 3rd 12v Aux plug.
    Added an RX360 radiator and 3 more GT-15s.
    Swapped to a Laing D5 pump.
    Lots of little case mods required for the switches and yes its upside down ;) ...
    Swapped ES chips with R-type so that I have 4x 2ghz chips now.

    New overclock tbd ... more cooling, more power, better chips. Time to see what she has got.

    [​IMG]
     
    Last edited: Jan 25, 2013
  2. FrEaKy

    FrEaKy [H] Movie and TV Show Review Guy

    Messages:
    13,377
    Joined:
    Jan 31, 2003
    SPEED HACK!

    Bake it!
     
  3. csmith811

    csmith811 Limp Gawd

    Messages:
    274
    Joined:
    Nov 13, 2007
    Maybe "Doc Ock" would be a good name, too. :)
     
  4. cbarbour71

    cbarbour71 [H]Lite

    Messages:
    88
    Joined:
    May 26, 2011
    Fap, fap, fap, fap....(gotta love folding pron)
     
  5. sbinh

    sbinh Gawd

    Messages:
    969
    Joined:
    Jul 12, 2008
    What if you run tube from 2nd cpu to dual rad then go back to 3rd cpu ? would that help last cpu cooler ?
     
  6. Kendrak

    Kendrak [H]ard|DCer of the Year 2009

    Messages:
    21,142
    Joined:
    Aug 29, 2001
    And this is the answer to the question: how bad do you want the #1 spot back?
     
  7. W.Feather

    W.Feather Hard DC'er for Aug 2012/ Feb 2019

    Messages:
    11,139
    Joined:
    Nov 10, 2009
    alot it appears.....


    this is better than i imagined
     
  8. DSee

    DSee 2[H]4U

    Messages:
    2,134
    Joined:
    Aug 31, 2007
    Congrats on the system!
     
    Last edited: Apr 5, 2012
  9. Activate: AMD

    Activate: AMD [H]ard|Gawd

    Messages:
    1,985
    Joined:
    Nov 6, 2004
  10. rmdashrrootsplat

    rmdashrrootsplat Gawd

    Messages:
    921
    Joined:
    Apr 2, 2008
    D is not a big truck... it's a series of tubes.
     
  11. Patriot

    Patriot [H]ard|DCer of the Month - March 2011/June 2013/De

    Messages:
    2,496
    Joined:
    Dec 15, 2010
    as noted... the picture does not include the retubing... I was too tired at 1:45 to take pictures... the picture is also dated the 19th...

    in quadrant style (pump > 360 rad to Q1 cpu > Q3 cpu >pump 240 rad > Q4 cpu >Q2 cpu >)

    Its quite a crisscross but cools much much better...

    I will of course take a picture and show you 0.9
     
    Last edited: Jun 29, 2011
  12. jebo_4jc

    jebo_4jc [H]ard|DCer of the Month - April 2011

    Messages:
    14,521
    Joined:
    Apr 8, 2005
    make with the pics yo

    congrats on making it to 2.8. looking forward to more results
     
  13. pjkenned

    pjkenned [H]ard|Gawd

    Messages:
    1,971
    Joined:
    Jan 8, 2010
  14. rmdashrrootsplat

    rmdashrrootsplat Gawd

    Messages:
    921
    Joined:
    Apr 2, 2008
    Works for me.
     
  15. Nathan_P

    Nathan_P [H]ard DCOTM x2

    Messages:
    3,356
    Joined:
    Mar 2, 2010
    Dead for me too
     
  16. jebo_4jc

    jebo_4jc [H]ard|DCer of the Month - April 2011

    Messages:
    14,521
    Joined:
    Apr 8, 2005
    lol works for me. a pic of tons of RAM
     
  17. Patriot

    Patriot [H]ard|DCer of the Month - March 2011/June 2013/De

    Messages:
    2,496
    Joined:
    Dec 15, 2010
    aaah the ripjaws... tis what dsee donates to this monstrosity ....
     
  18. sbinh

    sbinh Gawd

    Messages:
    969
    Joined:
    Jul 12, 2008
    are those RAM coming my way ? lolz ... Darn .. you guys are folding [H]ard .. :D
     
  19. DSee

    DSee 2[H]4U

    Messages:
    2,134
    Joined:
    Aug 31, 2007
    Nothing compared to what you've donated to the [H]orde so far.
    Keep us posted!!! We want numbers :)
     
  20. rmdashrrootsplat

    rmdashrrootsplat Gawd

    Messages:
    921
    Joined:
    Apr 2, 2008
    And more HW pr0n.
     
  21. R-Type

    R-Type [H]ard|DCer of the Month - October 2011

    Messages:
    2,542
    Joined:
    Mar 6, 2006
    Glad to hear the revised tube routing is working better for you.

    3.1ghz with spd hack is probably going to set the TPF records that will hold until they make the client scale well over 48 cores.
     
  22. Roman79

    Roman79 [H]ard|Gawd

    Messages:
    1,393
    Joined:
    Oct 2, 2000
    Mother of God....

    What is that? Could I get some hardware details please? :D
     
  23. W.Feather

    W.Feather Hard DC'er for Aug 2012/ Feb 2019

    Messages:
    11,139
    Joined:
    Nov 10, 2009
    its 4 Dodeca AMD Magny Corus, all ES chips, under water
     
  24. Activate: AMD

    Activate: AMD [H]ard|Gawd

    Messages:
    1,985
    Joined:
    Nov 6, 2004
    don't these chips need ecc ram?
     
  25. musky

    musky [H]ard|DCer of the Year 2012

    Messages:
    3,154
    Joined:
    Dec 14, 2009
    Nope, or at least mine don't. I assume those don't either.
     
  26. Shlomo

    Shlomo 2[H]4U

    Messages:
    2,969
    Joined:
    Oct 8, 2006
    So what kind of PPD will this rig produce. Will you run 2 instances of bigadv or one 48 core mega-bigadv-monster?
     
  27. DSee

    DSee 2[H]4U

    Messages:
    2,134
    Joined:
    Aug 31, 2007
    My guess:

    850K ppd @ 6904 CAS5.
    Maybe we should make a poll
     
  28. Untitledone

    Untitledone [H]ard|DCer of the Month - April 2012

    Messages:
    1,486
    Joined:
    Feb 28, 2011
    My guess in at around 817K PPD on 6903 last time I played around with the numbers.
     
  29. Patriot

    Patriot [H]ard|DCer of the Month - March 2011/June 2013/De

    Messages:
    2,496
    Joined:
    Dec 15, 2010
    playing around with the numbers 960k @ 3.1 6903 I am hoping cas 6 will boost it over 1m
     
  30. gwarren007

    gwarren007 [H]ardForum Junkie

    Messages:
    12,763
    Joined:
    Apr 15, 2006
  31. Untitledone

    Untitledone [H]ard|DCer of the Month - April 2012

    Messages:
    1,486
    Joined:
    Feb 28, 2011
    Well I hope you are right! That should boost you up pretty good!
     
  32. tjmagneto

    tjmagneto [H]ard DCOTM x2

    Messages:
    3,011
    Joined:
    Aug 6, 2008
    What? Nobody asked how it plays Crysis?

    (my apologies for the cliche) :D
     
  33. pjkenned

    pjkenned [H]ard|Gawd

    Messages:
    1,971
    Joined:
    Jan 8, 2010
    It creates Crysis sequels.
     
  34. nitrobass24

    nitrobass24 [H]ard|DCer of the Month - December 2009

    Messages:
    10,460
    Joined:
    Apr 7, 2006
    Holy crap!

    That just made my day :D
     
  35. jebo_4jc

    jebo_4jc [H]ard|DCer of the Month - April 2011

    Messages:
    14,521
    Joined:
    Apr 8, 2005
    No, Crysis plays it.
     
  36. Kendrak

    Kendrak [H]ard|DCer of the Year 2009

    Messages:
    21,142
    Joined:
    Aug 29, 2001
    When did we move to Soviet Russia?
     
  37. Patriot

    Patriot [H]ard|DCer of the Month - March 2011/June 2013/De

    Messages:
    2,496
    Joined:
    Dec 15, 2010
    something about free cooling.....was a big draw...
     
  38. Kendrak

    Kendrak [H]ard|DCer of the Year 2009

    Messages:
    21,142
    Joined:
    Aug 29, 2001
    Well yes it is free, however when you have to do your own maintenance work.... not sure it is worth it.

    [​IMG]
     
  39. jebo_4jc

    jebo_4jc [H]ard|DCer of the Month - April 2011

    Messages:
    14,521
    Joined:
    Apr 8, 2005
    Soviet Russia moved to you

    (Too easy :p)
     
  40. sfield

    sfield [H]Lite

    Messages:
    74
    Joined:
    Nov 2, 2010
    For the EK AMD block, I used a metal file to file down 1-2 mm of the acetal around the left and right motherboard mounting screws, which allows the block to be properly aligned with the chip. Probably doesn't make a big difference in block performance -- mostly asethetics and stability in your case -- but something to consider. In my situation, this was required given the design of the custom mounting brackets -- there's basically no play possible with the blocks once the brackets cover the blocks.

    Is cas5 working for you with that MB? This nut hasn't yet been cracked with the supermicro quad -- appears to be a bios limitation. I'm still running at 6-7-5-20-1T @ ddr3-1440, even though the modules have tested fine at 5-7-5-20-1T at equivalent frequency on an SR-2 system.

    Somebody mentioned FAH scaling limits beyond 48 cores -- scaling beyond this already works fine and is basically linear.

    You should have the fastest single system currently -- though a couple of others have the same chips -- unless somebody sneaks out a "-smp 160" 8x E7-8870 (...or even bigger system...).

    best AMD TPF for me is 6903: 11:45, 6904: 16:00. These are retail chips.


     
    Last edited: Jun 30, 2011