• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

402blownstroker's 4P Build

4

402blownstroker

[H]ard|DCer of the Month - Nov. 2012
Joined
Jan 5, 2006
Messages
3,257
Here is the worklog for my new 4P build.

First and for most I have to give a big thanks to musky and tear for helping out on a problem. When I was doing a test run when MB was assembled I was getting a lot of HT Link SYNC Errors and NB Watchdog Timeout Errors. These would happen when I would put a light load on the system and it would make booting take several trys. Tear and musky where able to diagnose the issue on the IRC channel within minutes. Turned out that CPU1 and CPU2 had dirty pads on the CPUs or pins in the socket. After cleaning both pads and pins for CPU1 and CPU2, things started working great.

And now onto the build.....

Components of the build
  • Supermico H8QGi-F Quad G34 Motherboard
  • (4) AMD Opteron 6166HE 12-core CPU
  • (8) 4GB DDR3-1333 ECC/Register DIMM
  • (4) Dynatron A6 Heatsinks
  • Antec 1200w Power Supply
  • Mountain Mods UFO Case
  • (7) 120mm Blue LED Fans
  • (4) 80mm Blue LED Fans
  • Supermicro 5-in-3 SAS/SATA-2 Drive Cage
  • Sony Blu-Ray Burner
  • Sunbeam Fan Controller
  • XFX GTX 275 896MB Video Card
  • Adaptec 5805 RAID Controller with 256MB Cache and BBU

I ended picking a Mountain Mods UFO case off of crapBay for pretty cheap. It is an older version and has the Gun Metal wrinkle finish. It has a nice 9 HD holder in the rear and a nice 5 1/4" 5-bay up front. It has a crap pile of fans:
  • (7) 120mm fans
  • (4) 80mm fans
My big complaint about the case is that the motherboard tray would flop around like a fish out of water. I fixed that by making some stabilizers from some old PCI slot covers using a dremel. With the stabilizers, I would say about 90% of the play has been eliminated.

UFO Bare Case:


Here are some pics of the stabilizers:


Now I am sure the people paying attention are wondering how in the world is that huge H8QGI-F board going to fit onto the ATX motherboard try of the UFO? First answer, not a chance in hell :D Mounting pattern of the H8QGi-F only maps up to 4 of the mounts on the tray. Final answer..... adapter plate. I had a lid from an old Rackable server laying around. After careful measurements and analysis, I determined it was good enough to use. I traced the H8QGi-F size onto it along with mounting holes. After a short bit of work with the dremel I now had an adapter plate. I did a quick test fit and it worked great. I was a little concerned I would have to shorten the standoffs, but all the expansion cards fit right in perfectly. After a little clean up and several coats of glass black enamel paint, the plate was done.

Pics of the plate and it installed:


With the adapter plate installed, it is time to put in the power supply.


With the power supply in, now comes the task of installing all the fans and grills. Kind of a PITA installing everything, but looks good at the end.


After a beer and a pat on the back for getting in the fans, now it it time to setup the motherboard. The motherboard was bought new from wiredonline, the CPUs and heatsinks come from musky, and the memory was leftover/upgraded from my SR-2. In general everything went together nice. The CPUs and heatsinks where pretty clean. Some other CPUs I have bought have been cover in thermal paste. After appling ArticClean thermal remover and purifier, time to start assembling. I also used Artic Ceramique for thermal paste as it is good stuff.

Pics the motherboard assembly.


Now with the motherboard assembled, it is time for fire it up and test things. For testing, I added several extra standoffs and put the motherboard on top the H8QG-i-F box. Initially everything fired right up and I even ran memtest for a while.



Next I installed the OS, Fedora 16 x86_64. That went fairly smoothly. Next I installed the FAH client to see what 48 cores of goodness would do..... and that is where the problems started. I started to see a huge amount of instability. The system would reboot itself at various periods in time. Each time booting back up, it would error out in the BIOS with HT Link SYNC Errors and NB Watchdog Timeout Errors. It would require a cold boot to clear them. At first I thought it might be a driver and/or 3.2 kernel issue with the board. I installed Fedora 14 x85_64 which is based on a 2.6 kernel. Damn it, same problems. After playing around and good amount of swearing and cussing, I found that if I run the FAH client with 16 cores, things where pretty stable. If I ran more than 16 core, issue would start up. After a day of fighting things and good amount of wrong information from the internet, I gave up and asked for some help on the IRC channel. I really felt like a dumbass because of the problem was diagnosed and fixed with an hour. If I would have gone there in the first place, I would not have wasted a whole day. At least with being a dumbass, you can learn from your mistakes :D

Next it is time to populate the front drive bay and install it.


Now it is time to install the motherboard and the add on cards.


Here is a shot of the back with all the wiring taken care of. Most of my past builds, the wiring has been a complete mess like a plate of spaghetti dumped into the case. This build I took a bit of time to do it right.


With everything installed, it is time to close things up and call it done.


All in all, I am pretty happy with the build. The other things I would like to do to it are:
  • Quad Channel Memory: Either 64GB( 16 x 4GB ) or 32GB ( 16 x 2GB )
  • Watercool it!!!! - These Dynatron A6 are killing me :D
  • Install the OC BIOS and get these 6166HE up to 2.0Ghz
 
Damn, that is a nice system!

How many 2P/4P builds have been made in the past month? Did everyone get their tax returns early or something?
 
I've got that same case (in mirror black) that I used to run my SR-2 in...that's a nice build you've got there 402.
thumbsup.gif
 
Very nicely done! Its good to see some other cases among this nudist colony of a team.

I just picked up an Extended Ascension with the Pedestal 24 and have been mulliing over my configuration options for getting a couple 4P's in there. Given MM's custom panel options, I think we could get them to design us a midplate that will bolt in stock for these and allow double sided mounting. What you have done looks very clean though!
 
You'll eventually learn to drown out the dynatrons lol

Very slick build, love it lol
 
R-Type said:
I just picked up an Extended Ascension with the Pedestal 24 and have been mulliing over my configuration options for getting a couple 4P's in there.
Click to expand...

Did you pick up that grey one in the general F/S section? I almost pulled the trigger on that one a couple of times.

Thanks for all the compliments guys :) It has definitely been the most build I have done so far and I have lot count how many I have done.

So far I am very pleased with it. Though it has the same Ghz sum as the SR-2 system of mine( 24 x 3.6 and 28 x 1.8 ), the 4P does 23m23s frames on the 6903 WU while the Sr-2 does 29m45s. I definitely need to flash on the new OC BIOS on this thing :D
 
Great looking rig man! :)

Btw, 23:23 for the 4P must be the TPF on 6904, not 6903. IDK about the SR-2 TPF though...
 
theGryphon said:
Btw, 23:23 for the 4P must be the TPF on 6904, not 6903. IDK about the SR-2 TPF though...
Click to expand...

Nope that is for 6903 frames. It has zero tweaks right now.... until tonight :D It should have 'the kraken' and the OC BIOS running. That should get the TPF down in the 18 to 19 minute range.
 
That's interesting. I have the same CPUs, not overclocked as well. The only difference it seems the Kraken, which I have, and the memory (mine is at 1333 CL7). My 6903 TPF is around 16:55, and 6904 TPF is around 23:20.
I'm amazed by the difference made by only these two factors. :eek:
 
theGryphon said:
The only difference it seems the Kraken, which I have, and the memory (mine is at 1333 CL7). My 6903 TPF is around 16:55, and 6904 TPF is around 23:20.
Click to expand...

Hmmmm.... I believe mine is CL9. Do you have dual or quad rank memory? I am thinking maybe I should switch out memory. I know that non-ecc unregistered DIMMs have faster CL speeds, but is there a real disadvantage in running non-ecc memory other than the obvious error correction?
 
For comparison, My OC 4P 6166HE gives me a TPF of 15:28 for 6903s, and 21:27 for 6904s
 
Good God stop please, depression is setting in. What tweaks are you running other than what the guide mentions?
 
Oh boy, I just noticed you're running dual channel. Now that would make a huge difference compared to quad channel. I strongly suggest getting more memory to run quad channel.
I have non-ecc unreg memory and I don't think there is any disadvantage for folding. Only for mission-critical tasks ECC may have a real advantage. For folding, as long as the memory is running as it should (at spec'ed timings and voltage), it should be fine AFAIK.
 
Yes, your problem is definitely the dual channel memory. That is only a single channel for each quad core die in the chips.
 
OK, good info there on the memory. Anyone have any suggestions for 2GB DIMMs?
 
How much difference would CL9-24 be to CL7-21? For the price of 4 of the 4x2GB kits, I can pick up another 32GB of 4GB ECC/Reg DIMMs and reach quad channel at 64GB.
 
16x2GB exceeds current folding requirements thrice.
Faster 2-gig modules are a better choice.

Plus, if you don't go above 16 DIMMs you can mix
them any way you want.
 
I'm running these Crucials: http://www.newegg.com/Product/Product.aspx?Item=N82E16820148488
They're lower profile than GSkill's and that's what I needed for my Noctua 92mm's (perfect clearance).
I wish I got these though, they're a bit cheaper: http://www.newegg.com/Product/Product.aspx?Item=N82E16820148487

Both the GSkill linked above and the Crucials above will default to 1333 CL9. For 1333 CL7, you will have to flash the spd using tear's method. I don't think it's posted anywhere publicly, so you may wanna go to the irc channel :) It was stupidly easy for me, and it saved at least 15 seconds from 6903 and 6904 TPF's.
 
theGryphon said:
I'm running these Crucials: http://www.newegg.com/Product/Product.aspx?Item=N82E16820148488
They're lower profile than GSkill's and that's what I needed for my Noctua 92mm's (perfect clearance).
I wish I got these though, they're a bit cheaper: http://www.newegg.com/Product/Product.aspx?Item=N82E16820148487

Both the GSkill linked above and the Crucials above will default to 1333 CL9. For 1333 CL7, you will have to flash the spd using tear's method. I don't think it's posted anywhere publicly, so you may wanna go to the irc channel :) It was stupidly easy for me, and it saved at least 15 seconds from 6903 and 6904 TPF's.
Click to expand...

With the new overclocking method you apparently don't flash the ram, the modded bios is smart enough to grab the faster timings.
 
OK, I think I am sold on the G.SKILL Ripjaws X Series. I will be ordering (4) of the 8GB (4 x 2GB) kits tonight or tomorrow in the morning.
 
R-Type said:
With the new overclocking method you apparently don't flash the ram, the modded bios is smart enough to grab the faster timings.
Click to expand...

Oops, yeah, with the new OC, things may have changed. I didn't know it grabbed the faster timings. Cool! :cool:
 
theGryphon said:
I'm running these Crucials: http://www.newegg.com/Product/Product.aspx?Item=N82E16820148488
They're lower profile than GSkill's and that's what I needed for my Noctua 92mm's (perfect clearance).
I wish I got these though, they're a bit cheaper: http://www.newegg.com/Product/Product.aspx?Item=N82E16820148487

Both the GSkill linked above and the Crucials above will default to 1333 CL9. For 1333 CL7, you will have to flash the spd using tear's method. I don't think it's posted anywhere publicly, so you may wanna go to the irc channel :) It was stupidly easy for me, and it saved at least 15 seconds from 6903 and 6904 TPF's.
Click to expand...

For those who are not aware, tear's method is in the middle of the page http://www.amdzone.com/phpbb3/viewt...sid=ad5b5e1bdb59299c13d2d37436b6248a&start=50

Also, this memory (for those who have IL) http://www.newegg.com/Product/Product.aspx?Item=N82E16820231444&Tpk=12800CL6 has no problem with clearance with Noctua 120mm. My 4P temp is 35-39C (ambient 30C) on full load.
 
I'm using 8x F3-12800CL7D-4GBXM with tear's classic memory OC, reflashed to CL6 in quad-channel. Post-flash got 9% speed up.
 
musky said:
The Kingstons are 1.65V. Your board can only push 1.5V to the memory. Those would be a bad choice.
Click to expand...

You are right. I did not notice that at first.

I just placed an order for (4) of the G Skill 8GB kits. Hopefully it can be here in 2 to 3 days as newegg claims. Anyone looking for 32GB of DDR3-1333 ECC/Reg memory for cheap :D

With the new OC BIOS, does the memory skill need to be reflashed to get the CL6 as rhavern is getting? 9% speed up is a very good improvement.
 
402blownstroker said:
You are right. I did not notice that at first.

I just placed an order for (4) of the G Skill 8GB kits. Hopefully it can be here in 2 to 3 days as newegg claims. Anyone looking for 32GB of DDR3-1333 ECC/Reg memory for cheap :D

With the new OC BIOS, does the memory skill need to be reflashed to get the CL6 as rhavern is getting? 9% speed up is a very good improvement.
Click to expand...

No, it does not. The bios takes care of setting memory timings appropriately. No flashing is required.
 
Well I got the 16 2GB G.Skill DIMMs in and it has made an improvement. With the ECC/Reg dual channel setup up, I was getting aout 23m23 TPF on the 6903 WU. With quad channel, a 6903 WU is doing 19m45s TPF. Nice improve of 3m38s or 18%. Next install 'the kraken' and see what that does. The kraken will most likely be this evening. Not enough free time here :(
 
Oh yes, I'd recommend adding Kraken even before doing any OC work as Kraken is virtually free
performance.
 
tear said:
Oh yes, I'd recommend adding Kraken even before doing any OC work as Kraken is virtually free
performance.
Click to expand...

I think this guy knows about Kraken. It is free performance, and it also can sometimes engage DLB which is another performance boost :)
 
gero1369 said:
Pick up this stuff:
Samsung Electronics Extreme Low Voltage 30nm UDIMM 4 Dual Channel Kit DDR3 1600 (PC3 12800) 240-Pin DDR3 SDRAM MV-3V2G3D/US

Great write-ups on it (mostly on Intel systems, but should be great on AMD too):
http://www.techpowerup.com/reviews/Samsung/MV-3V4G3/6.html

http://www.techpowerup.com/forums/showthread.php?t=159320
Click to expand...

The problem is going to be that you aren't going to get much out of it with current functionality. Server boards and G34 chips are going to have these modules running 1333 C9 and no easy way to do any other speed.
 
As an Amazon Associate, HardForum may earn from qualifying purchases.
402, how many boxes do you have folding? I see you are on my threat list.
 
You must log in or register to reply here.
Back
Top