Ivy-E folding on the Z9PA

Nathan_P

[H]ard DCOTM x3
Joined
Mar 2, 2010
Messages
3,490
As those of you who join IRC know I recently managed to obtain a pair of Ivy-E 12core cpu's for the bargain price of $1400, I say bargain as the auction had them listed as 8 core parts but the actual picture was of a HCC chip (the 12 cores look different). CPU-Z identifyies the cpu as a 2640 8 core, HWinfo 64 lists it properly

Purchased these, and an ASUS Z9PA intending to use existing parts where possible, after all v2 is supposed to be a drop in replacement for v1 cpu's with a BIOS flash

Well, not quite.

1. These CPU's are in short supply, unlike v1
2. They must have ECC ram for 2p mode, normal desktop stuff doesn't work even if your mobo manual says it will.
3. Like v1 there have been reports, of CPU's not behaving properly etc, well mine do work but they are B2 stepping, not sure about the earlier ones.
4. They have a high voltage requirement on start up, some psu's may not have the juice to power everything up.

That's the caveats out the way, so here's my tale.

Got everything in the mail, opened everything up and hit the first snag. On the Z9PA the cpu sockets are right close together, you will need to look very closely at heatsink sizes, and it will probably be the same for other ATX boards as most won't fit and if you turn them sideways you may have an issue clearing your ram

Anyway for testing to make sure everything worked I did use my oversized intel HSF and it wouldn't boot in 2p mode. 1p worked fine.

Sourced some cheap HSF that just fit - photo's will come later, and started proper testing.

1p would boot fine on either cpu with stock desktop ram but still no 2p boot, tried 3 different sets of normal ram and no joy. Boot kept failing on Q code b2, Memory init.

Ordered 2 cheap ECC modules to see if it would boot. still nothing so I called on Tear's expertise.

We started disabling things in the BIOS, ISOC, VT, Option Roms, slowing QPI links etc. and still nothing. We pondered and the general consenus was it was still a RAM issue so I went to try and source some RAM from the very short approved list from asus.

That arrived a couple of days ago and a quick test still wasn't having any joy so today I set aside as long as it would take.

Installed 2nd cpu, hsf and started from there, at this time we had it stuck on q code b2 on the boot sequence.

Installed a cheap 5450 gfx card, disabled on board graphics and still b2
disabled ISOC - this has been stated on several websites - b2
disabled Lan option roms - A2, this is a result as A2 is near the end of post
disabled SATA raid option rom - A2
At this point Wyluliraven asked about the PSU and suggested a different one as v2 needs a lot of current on start up
Ripped apart my x5670 rig and put in the 650AX - still A2
disabled Vt and Vt-d - A2
disabled the USB 3 controller and got AE, more progess as AE means its looking for a boot device, in my haste I had forgotten to reconnect power to the HDD
Reconnected that and voila - we now have a working 48 thread machine:)

So here are some numbers.

1p folding 12c/24t at 2.1ghz, 8gb Gskill 1600 CAS9 ram, 400w seasonic fanless PSU.
Power 132w from the wall folding.

8104 tpf 16:33 for 170k PPD
8105 tpf 20:56 for 176k PPD.

2p folding 24c/48t at 2.1ghz, 2gb Hynix HMT125U7, 650w AX PSU.
Power 234w from the wall folding

8552 tpf 3:36 for 84k PPD - this is single channel mode, I will post updates as I get more ram. When I say single channel, its 1 stick between both CPU's so lots of shunting around of data.
 
Congrats. Looks like a BIOS bug of Z9PA? I think supermicro boards would do better with IVB ES chips.
What's the default clock & QPF code of the chip? Can CPU-z recognize all 12 cores now?
 
CPU-Z still doesn't recognise the 12 cores, but since I now havce it running Ubuntu i'm not playing with it any more, its working and just needs more ram.

Default clock is 2.0 with all core turbo up to 2.1 and the QPF code is QDUD. I've never had a problem with asus boards before using ES chips but then I've not been as close to the front of the pack in using new cpu's.

Probably is a BIOS bug and at some point I will log it with asus, but the current job is RAM and a case for my x5670 rig
 
So here are some numbers.

1p folding 12c/24t at 2.1ghz, 8gb Gskill 1600 CAS9 ram, 400w seasonic fanless PSU.
Power 132w from the wall folding.

8104 tpf 16:33 for 170k PPD
8105 tpf 20:56 for 176k PPD.

2p folding 24c/48t at 2.1ghz, 2gb Hynix HMT125U7, 650w AX PSU.
Power 234w from the wall folding

8552 tpf 3:36 for 84k PPD - this is single channel mode, I will post updates as I get more ram. When I say single channel, its 1 stick between both CPU's so lots of shunting around of data.

Nathan_P,

The last WU in 2P is interesting. Reference AndyE's work also: http://hardforum.com/showthread.php?t=1782201
I did a double-take on the WU 8552 and see that it is a GRO-A3 core versus the A5 core and ends up having such a low ppd/W. I've not recorded any GRO-A3 core WU's on the 4P I am running. Is it still possible to get an A3 core if the smp settings are correct?

Thank you for all the information.

Sincerely,
 
Yeah, A3 is still being used, just remove the -bigadv option from your config. My personal observation is that v7 clients get the A4 smp work and v6 clients get the A3 smp work.

Hoping to have nough ram in the next week to get BA units going again
 
Nathan_P,

I only meant that if the -bigadv/cpu/smp settings are correct, is it still possible to see A3/A4 WU on a bigadv capable system? Since I am so new, I had not read if this was a possibility.

I do have a part time Win7 1P/12c running v7 and get GRO_A4 (62 WU's), GRO-A4 (25 WU's), and GRO_A3 (30 WU's) with the last of each on 11/11, 11/13 and 11/16 respectively.

Sincerely,
 
Mike_Shaffer, it is possible to get A3/A4 WUs even with bigadv flag. bigadv flag is telling PG that you are capable of big, does not guarantee that is all you will get.
 
Okay. Thank you. Apologies for being so dense. Nathan_P, good luck finishing your build and let us know how the new ram works out.
 
Glad you got them running! I wonder if they're very early steppings (or similar) and that amplified some of the issues?
 
Not sure, there are B0 and B1 chips out there, I suspect its more a crap early bios version for ivy-e chips.

on a side note running at full speed may take a while, finding ram is not easy
 
It's strange that the all-core clock is only 2.1GHz. One of my friends also has a pair of QDUDs, but the all-core clock is 2.4GHz and CPU-z can see all 12 cores. He is using a supermicro X9DAi board (with R3.0 BIOS).

CPU-Z still doesn't recognise the 12 cores, but since I now havce it running Ubuntu i'm not playing with it any more, its working and just needs more ram.

Default clock is 2.0 with all core turbo up to 2.1 and the QPF code is QDUD. I've never had a problem with asus boards before using ES chips but then I've not been as close to the front of the pack in using new cpu's.

Probably is a BIOS bug and at some point I will log it with asus, but the current job is RAM and a case for my x5670 rig
 
Last edited:
Hmm, I'll look into that turbo thing again, but probably another bios bug
 
quick update:-

Managed to get a 2nd stick of RAM installed so each cpu is now running in single channel mode.

8552 tpf went down to 2:46, PPD now 126k
 
Dual channel mode took it up to 160k PPD on 8574

Quad channel mode on same WU went upto 176k.

Power consumption at 271w from the wall.

BA numbers to follow after this WU.
 
Power draw went up over night to 279w, managed to get the on board gpu working again.

Power now 268w and PPD 494k on an 8103. That's 1,843 PPW :)

Edit:- forgot to add the total rig cost was £1,500. CPU's, Mobo, Case, HSF, Ram and a USB stick. The only thing resued from an older build was the PSU
 
Last edited:
Back
Top