G34 4P Supermicro Pics

rvborgh

Weaksauce
Joined
Jul 1, 2014
Messages
88
i currently run the Noctua NH-9UDO A3 at full speed...

i upgraded one of the fans on the Noctuas to the newer higher speed A9 PWM fans that run at 2000 rpm... for a tad better cooling as well.

these: http://noctua.at/en/products/fan/nf-a9-pwm.html

the 3 fans behind the drive cage and the 3 fans at the rear are run at low speeds with the fan controller. Some run faster than others (i found that running some faster actually hurt cooling instead of helping). There is an additional SuperMicro GPU style fan at the rear that i added as well.

Replacing the passive coolers with the Noctuas will really help reduce the noise... as you can run the internal and external fans slower. The Noctuas are very quiet... however they are not capable of keeping this rig under 60C when i overclock past 3 Ghz (these days i run 16 of the cores at 3.4GHz and the rest at 1.7 Ghz, and set thread affinity to the faster cores for my commonly run apps).

Of course as with all things in life YMMV.

hope this helps.


PS: if you want to know how it sounds like watch this video:

https://www.youtube.com/watch?v=ppvOUAVrHfQ

At the beginning where i am talking... that is what it normally sounds like. The sensivity on the camera mic starts to adjust as i stop talking and the benchmark commences... and it makes the fans seem loud... but they sound like what they sound like at the beginning of the video.

I have scored (at 800$, more of a steal, i think) myself one of those as well:

- Supermicro H8QG6-F
- 4x 6274 with passive coolers
- Supermicro SC748 chassis w/2 redudant 1400W powersupplies
- 16 dimms x 8 Gb each

and I must say: the loudness overrides the nice points. Boinc is chugging along nicely, running 16 MCM1 tasks @ about 5 hours 30 minutes each. That is the same time as a dual X5650 Xeon system. It won't download more tasks at the moment, both CEP2 and UGM1 complain about 'transient HTTP errors'.

Do I read correctly that better CPU coolers (Noctua NH-9UD0) and a fan-controller on the existing fans are enough to bring the noiselevel down? Did you do anything with the fans behind the drivecage inside? This system will move to my office, so it needn't be absolutlely silent, but right now even the bios option Balanced gives a lot of noise.
 
Last edited:

Jurriaan

n00b
Joined
Feb 14, 2016
Messages
7
So if you kept the existing case fans, did you (or anybody else) have any success in slowing them down from the BIOS or with ipmitool? Will the ocng bios perhaps allow the fan-speed more fine grained?

Upgrading the fans on the NH9 is one way, of course. The 120 mm Noctua NH-U12DO probably won't fit, right? Would that be a problem with the height or also the width (particularly for the highest socket) ?
 

rvborgh

Weaksauce
Joined
Jul 1, 2014
Messages
88
i don't run ocng... i run ES procs, and run a Turion Power Control script to set all the cores appropriately... so i can't really comment on the OCNG.

The Noctuas are so quiet though that you can run them at full blast (the highest performance mode in the BIOS whatever that is) and you can barely hear them. That is what i do.

The U12DO as far as i know is too tall (barely) for this case, but if you want i'll do some exact measuring tonight. The top socket doesn't have the clearance for them though (like you mentioned)... bottom socket would block a PCI-e slot (this might not matter for you however).
 

Jurriaan

n00b
Joined
Feb 14, 2016
Messages
7
I would love exact measurements. I read that both noctua coolers can be rotated 90 degrees - is there a reason you mounted them vertically? I would expect that with the airflow running from front to back the other way around would be better?

TurionPowerControl has boosted my 6274's to 2.5 GHz (according to the ocng clockspeed utility) - highest cpu temperature is 42 degr C (but the computer is in a very cold room - I'd guess 5 degr C).
 
Last edited:

orion

Weaksauce
Joined
Nov 19, 2011
Messages
113
i don't run ocng... i run ES procs, and run a Turion Power Control script to set all the cores appropriately... so i can't really comment on the OCNG.

The Noctuas are so quiet though that you can run them at full blast (the highest performance mode in the BIOS whatever that is) and you can barely hear them. That is what i do.

The U12DO as far as i know is too tall (barely) for this case, but if you want i'll do some exact measuring tonight. The top socket doesn't have the clearance for them though (like you mentioned)... bottom socket would block a PCI-e slot (this might not matter for you however).

Are those the MC's you got from me?
 

rvborgh

Weaksauce
Joined
Jul 1, 2014
Messages
88
i'll do some measurement tonight...

when i only had two processors installed i mounted the coolers east west...

with 4 processors...i had to mount #4 north south with one fan missing... so i remounted all the rest to match.

i didn't notice any temperature difference really. i got a small gain by taping the gaps on the top of the Noctua coolers, and mounting them such that they blow down and up for #1 and #3, and #2 and #4 respectively... i bought an additional SuperMicro GPU cooler when i was looking at the thermal images and noticed that because of blowing heat down from #1 it was collecting down in the PCIe area and needed to be extracted:




 

rvborgh

Weaksauce
Joined
Jul 1, 2014
Messages
88
yes... those are the ones :) working well! i've been experimenting with running a bunch of them at lower speeds while running others at higher speeds (a type of home rolled turbocore i suppose). Currently with the Noctua's 3.4 GHz on 2 cores of each die (total of 16 cores out of the 48), and the rest at 1.7 Ghz works great. i can run all 48 cores at 3.0 Ghz just fine, but they get too hot for the Noctuas at anything over that.

Are those the MC's you got from me?
 

orion

Weaksauce
Joined
Nov 19, 2011
Messages
113
yes... those are the ones :) working well! i've been experimenting with running a bunch of them at lower speeds while running others at higher speeds (a type of home rolled turbocore i suppose). Currently with the Noctua's 3.4 GHz on 2 cores of each die (total of 16 cores out of the 48), and the rest at 1.7 Ghz works great. i can run all 48 cores at 3.0 Ghz just fine, but they get too hot for the Noctuas at anything over that.
Great! Glad that they're still running strong for you
 

Jurriaan

n00b
Joined
Feb 14, 2016
Messages
7
Well, my system crashed - numerous NMI lockup messages - perhaps using TurionPowerControl to overlock 6274's from 2.2 to 2.5 GHz wasn't such a good idea.
I'm not going to invest anything in new coolers before Boinc can run for 48hrs without problems. If that works, I understand 4 noctua NH9's and a fan controller
are on the list. Perhaps 2 outtake fans at the top of the case, right over the cpus?

Anyway, with the cpu's at stock speed, and fans on 'balanced', I still get

Code:
[  117.823513] perf interrupt took too long (3655 > 2500), lowering kernel.perf_event_max_sample_rate to 50000
[  515.252108] perf interrupt took too long (6275 > 5000), lowering kernel.perf_event_max_sample_rate to 25000
[  837.969610] perf interrupt took too long (10063 > 10000), lowering kernel.perf_event_max_sample_rate to 12500
[ 3710.024383] INFO: rcu_sched detected stalls on CPUs/tasks: {} (detected by 60, t=5252 jiffies, g=35007, c=35006, q=20348)
[ 3710.024434] All QSes seen, last rcu_sched kthread activity 5251 (4295820540-4295815289), jiffies_till_next_fqs=1, root ->qsmask 0x0
[ 3710.024439] poempp_2.21_x86 R  running task  0  2332  1521 0x00000000
[ 3710.024447]  ffff882026d16580 ffffffff81a54f40 ffffffff810c9816 0000000000000000
[ 3710.024453]  0000000000000096 ffffffff00001780 ffff88141370a5c0 0000000000000000
[ 3710.024458]  0000000000004f7c ffffffff81a54f40 ffff882026d03e50 ffff882026d03e50
[ 3710.024462] Call Trace:
[ 3710.024466]  <IRQ>  [<ffffffff810c9816>] ? rcu_check_callbacks+0x6d6/0x6e0
[ 3710.024487]  [<ffffffff810dd360>] ? tick_sched_do_timer+0x40/0x40
[ 3710.024493]  [<ffffffff810ced06>] ? update_process_times+0x36/0x60
[ 3710.024496]  [<ffffffff810dd360>] ? tick_sched_do_timer+0x40/0x40
[ 3710.024500]  [<ffffffff810dcda4>] ? tick_sched_handle.isra.15+0x24/0x60
[ 3710.024503]  [<ffffffff810dd39b>] ? tick_sched_timer+0x3b/0x70
[ 3710.024506]  [<ffffffff810cf81c>] ? __hrtimer_run_queues+0xdc/0x210
[ 3710.024513]  [<ffffffff8101c325>] ? read_tsc+0x5/0x10
[ 3710.024516]  [<ffffffff810cfc2a>] ? hrtimer_interrupt+0x9a/0x190
[ 3710.024524]  [<ffffffff81557a79>] ? smp_apic_timer_interrupt+0x39/0x50
[ 3710.024528]  [<ffffffff81555c5b>] ? apic_timer_interrupt+0x6b/0x70
[ 3710.024529]  <EOI>
[ 3710.024533] rcu_sched kthread starved for 5251 jiffies! g35007 c35006 f0x2

I wonder if 100% cpu usage of all cpu's is too much?
 
Last edited:

Jurriaan

n00b
Joined
Feb 14, 2016
Messages
7
Well, it crashed after 26 hrs of folding with default settings as well.
CPU temperatures according to tpc stayed under 35, no ht retries, fans running at 4500/6500 rpm, so I'm a bit stumped. If it's not temperatures, and everything is stock, what is happening?
Do these motherboards/cpu's 'age'? I'm not about to invest $$ in silent cooling if it keeps crashing, of course - the resale value of G34 cooling seems somewhat low :)

Any hints on what to check next?
 

Core32

[H]ard|Gawd
Joined
Mar 3, 2012
Messages
1,065
Well, it crashed after 26 hrs of folding with default settings as well.
CPU temperatures according to tpc stayed under 35, no ht retries, fans running at 4500/6500 rpm, so I'm a bit stumped. If it's not temperatures, and everything is stock, what is happening?
Do these motherboards/cpu's 'age'? I'm not about to invest $$ in silent cooling if it keeps crashing, of course - the resale value of G34 cooling seems somewhat low :)

Any hints on what to check next?

Run comprehensive memory tests.
What ever device you are booting/running from, swap to something else (USB stick, HDD, SSD, etc.)
Pull each CPU and check for contamination on each pad.
Run FAH one CPU installed at a time with minimal memory. Repeat for each CPU individually using the same socket and memory
Do this ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ with the same CPU for each socket.
If no failures then cycle through all the memory sticks.
If you have redundant power supplies run with just one, then swap to the other.
 

ChelseaOilman

[H]ard|Gawd
Joined
Mar 27, 2001
Messages
1,555
@Jurriaan, are you using tears overclocking bios? If not, that's the way to go with stock non ES cpus.

OCNG5: OC firmware for Supermicro AMD G34 platforms

"OCNG5 works by altering motherboard's reference clock early at boot and tuning
processor components for performance and stability."

You really should start your own thread. All this stuff is way off topic for this thread. Maybe a mod could edit out these posts into a new thread for you.
 
Last edited:

TheWolf

n00b
Joined
Jan 29, 2016
Messages
9
I use to have access to the above link, but had a linux crash and now I'm no longer able to get those those threads about OCNG5: OC firmware for Supermicro AMD G34 platforms stuff?
Can anyone help me with this? Trying to get my H8QG6 back working and overclocked. Bios is still intact I just need some of the commands to put everything back in order.
 
Last edited:

auntjemima

Supreme [H]ardness
Joined
Mar 1, 2014
Messages
6,515
I use to have access to the above link, but had a linux crash and now I'm no longer able to get those those threads about OCNG5: OC firmware for Supermicro AMD G34 platforms stuff?
Can anyone help me with this? Trying to get my H8GL6 back working and overclocked. Bios is still intact I just need some of the commands to put everything back in order.

Hey! With the new forum software the admins are still working on getting subforums with passwords working. Pretty sure they might have been in the strategy area, but I could be mistaken. Sorry!
 

Gilthanis

[H]ard|DCer of the Year - 2014
Joined
Jan 29, 2006
Messages
8,200
Naw..those were in the open section. However, we are working on getting access to the Guides section again. Hoping that happens soon. Pocatello is working on making that happen.
 

TheWolf

n00b
Joined
Jan 29, 2016
Messages
9
Thanks guys for the help, sorry for bringing up a old thread, but was about the only thing a search could find about the subject. Hope to see it working soon, good luck.
Using a Win server 20012 OS ATM, but only seeing half of the CPU's 4x 6276.
 
Last edited:

auntjemima

Supreme [H]ardness
Joined
Mar 1, 2014
Messages
6,515
Thanks guys for the help, sorry for bringing up a old thread, but was about the only thing a search could find about the subject. Hope to see it working soon, good luck.
Using a Win server 20012 OS ATM, but only seeing half of the CPU's 4x 6276.

Was that a common problem? I am running some 6278's in Linux on that same BIOS and I never had to run any commands to enable some of the cores. There's no HT, so windows should be seeing them all, no?
 

TheWolf

n00b
Joined
Jan 29, 2016
Messages
9
Well I'm no longer sure, I'm only running this OS until I'm able to get Linux set back up. It was a quick fix to get folding again.
We moved 3 years ago and all my hardware has been setting in storage, just starting to bring some of my old rigs out to play with. lol
My mind is no longer sharp as it use to be I've forgotten alot and need those forums to refresh my memory.
I didn't think the OS had a socket license limit I used it in the past and if I recall right it seen them all, there are like 4 options to pick from while doing the install I may have picked the wrong option.
32 cores folding is okay for now, plus one GPU 660Ti
 
Last edited:

EXT64

DCOTM x4
Joined
Mar 27, 2013
Messages
598
Yep, for that version Standard is limited to 2 sockets, however Datacenter should let you use all 4.
 

auntjemima

Supreme [H]ardness
Joined
Mar 1, 2014
Messages
6,515
Win server 2012 R2 standard Eva with GUI x64
Not sure if I can upgrade over what is already installed, but will give it a try. Should I use one of the datacenter versions to see all four sockets?
Yep, for that version Standard is limited to 2 sockets, however Datacenter should let you use all 4.

Sounds like you guys are on to something! Awesome.
 

TheWolf

n00b
Joined
Jan 29, 2016
Messages
9
Yep, for that version Standard is limited to 2 sockets, however Datacenter should let you use all 4.
Is DataCenter with GUI also limited to only two as well? I installed it and its the same as the other only seeing 2 sockets.
I could use a list of the basic commands to run in thermal such as Mphrobe, tmax, OCNG5 & temp? I remember clockspeed. lol but not everything that goes in front or behind the others.
 
Last edited:

EXT64

DCOTM x4
Joined
Mar 27, 2013
Messages
598
Datacenter shouldn't be limiting sockets, though who knows with MS and their licensing games. Its been so long since I've tried to use Windows on a server unfortunately. Linux brings its own problems, but at least license shenanigans isn't one of them.
 

TheWolf

n00b
Joined
Jan 29, 2016
Messages
9
Datacenter shouldn't be limiting sockets, though who knows with MS and their licensing games. Its been so long since I've tried to use Windows on a server unfortunately. Linux brings its own problems, but at least license shenanigans isn't one of them.
Turns out it wasn't the OS, but the FaH software only setting up 31 cores for CPU folding -1 for GPU.
I setup a 2nd CPU folding with -1 and it only set 16 cores folding with the default -1 setting.
Adjusted it to 30 and seeing 95% load on CPU's. I'm still not happy with the OS having to babysit the Lan to get units to return
so ASAP as I can I'll get some flavor of Linux installed. Suggestions?
 
Last edited:

auntjemima

Supreme [H]ardness
Joined
Mar 1, 2014
Messages
6,515
Turns out it wasn't the OS, but the FaH software only setting up 31 cores for CPU folding -1 for GPU.
I setup a 2nd CPU folding with -1 and it only set 16 cores folding with the default -1 setting.
Adjusted it to 30 and seeing 85% load on CPU's. I'm still not happy with the OS so ASAP as I can I'll get some flavor of Linux installed. Suggestions?

Most guys around here are running nah flavour of u Ubuntu. It's the simplest option for what we are doing. There are many flavours, but I think Mint and Pop! OS are both popular.
 

TheWolf

n00b
Joined
Jan 29, 2016
Messages
9
Most guys around here are running nah flavour of u Ubuntu. It's the simplest option for what we are doing. There are many flavours, but I think Mint and Pop! OS are both popular.
Ubuntu 15.10 was what I was running before the crash. After I started having problems after updating the OS.
I read there was some problems with newer versions no longer supporting stuff needed to run the Fahcontrol.
Trying the workaround "right from the FaH forum" is when I got the major crash.
 

auntjemima

Supreme [H]ardness
Joined
Mar 1, 2014
Messages
6,515
Ubuntu 15.10 was what I was running before the crash. After I started having problems after updating the OS.
I read there was some problems with newer versions no longer supporting stuff needed to run the Fahcontrol.
Trying the workaround "right from the FaH forum" is when I got the major crash.

Honestly not sure. I haven't really run F@H at all and story on the street is CPU's are basically useless at this stage .. apparently some sort of BigAdv being gone or something. I don't know, before my time.

Guys here are running BOINC as well and I haven't had any issues with the 18.04 LTS or 19.xx versions.
 

Nathan_P

[H]ard DCOTM x2
Joined
Mar 2, 2010
Messages
3,431
Mint 18 here, with no kernel updates. thats for both CPU and GPU slots for F@H. Not sure on current PPD for those 62xx chips, may be worth running a couple of WU and reporting back what you are seeing PPD wise
 

auntjemima

Supreme [H]ardness
Joined
Mar 1, 2014
Messages
6,515
Mint 18 here, with no kernel updates. thats for both CPU and GPU slots for F@H. Not sure on current PPD for those 62xx chips, may be worth running a couple of WU and reporting back what you are seeing PPD wise

I am running a 64c 6378 and I have only been able to get a single task for it so far, but it slammed through it
 

Nathan_P

[H]ard DCOTM x2
Joined
Mar 2, 2010
Messages
3,431
someone just posted on the internal slack channel that their 3900x can get around 300k PPD on the current cpu projects. My how times have changed
 

TheWolf

n00b
Joined
Jan 29, 2016
Messages
9
Think I found the problem my 1200W power supply went up in smoke. Hope it didn't take anything out with it. Funds are low so I'll have to put a update PS on hold.
 

auntjemima

Supreme [H]ardness
Joined
Mar 1, 2014
Messages
6,515
Think I found the problem my 1200W power supply went up in smoke. Hope it didn't take anything out with it. Funds are low so I'll have to put a update PS on hold.

Boooo! Sorry to hear that. Scout eBay when you get some funds. Bought all my big PSUs there.
 

7TomCatT7

n00b
Joined
Sep 15, 2019
Messages
2
I still haven't fully played with Linux on my rig. WIndows side I like Enterprise with Workstations as a runner-up. Server 2019 and lower are okay, they "feel stiff until you find the scripts to run against the REG files.

Hope you got your PSU replaced.
 

rvborgh

Weaksauce
Joined
Jul 1, 2014
Messages
88
Just thought i'd let you know that they are still running strong and just fine 4 years later :)

currently running OCNG, which let me run DDR3-1600 ram. Unfortunately DDR3-1866 did not run at that speed (ran fine at DDR3-1600 at slightly lower latency).

Are those the MC's you got from me?
 
Top