Dual socket build, which Xeons to look for?

X

x-cimo

Limp Gawd
Joined
Mar 26, 2012
Messages
209
I'm looking to replace my 3930K 2011 Server (it will become my desktop) with a Dual Socket Xeon setup.
That all in one server runs ESXi with a dozen VM, ZFS with 16 Drives, run Emby/Plex transcoding.

I think for my kind of load I should look for frequency optimized Xeon ( dual socket should provide the extra cores ).

Would the following xeons make sense (I'm looking for used v3 or v4 on ebay but they are rare!):
E5-2637
E5-2643
E5-2667

I'm looking at filling as many PCIe as possible currently have:
2x PCIe 16x Supermicro JBOD controller
1x PCIe 8x Intel 10G X520-DA2

Looking to add(future)
1x PCIe 16x GPU (Steam in-home streaming)
1x PCIe 16x NVMe adapter for 4 NVMe M2 drive.

Thanks.
 
Your workloads are all well-threaded, so I'd probably go for the E5-2680. Max turbo is 3.6GHz so you should have ample gaming performance; I don't think even a GTX 1080 is bottlenecked by 3.6GHz Sandy Bridge cores.
 
e5-2670 might be a little cheaper and more cost effective but still fast enough. should check out the natex mb cpu mem combo.
 
I was confused about the [email protected] until I realized it's a V2. Which in the end might be a good thing, there are more V2 on ebay, and 128GB of ECC DDR3 looks much cheaper than ECC DDR4. So V2 might be the way to go. CPU are still super expensive however.

Wildpig, which Version of the 2670?, Will look at natex, thanks

EDIT: 2670 are V1, natex deals are pretty sweet, but MB don't support PCIe x16, and that wouldn't be much of an upgrade over my 3930K. But thanks :)
 
Last edited:
Bill1024 said:
Asrock makes a nice dual lga 2011 with a PCIE 16 I do believe. Around 300$ for just the board though.
I have one of these with two e5-2670, it runs real good.

ASRock EP2C602 SSI EEB Server Motherboard Dual LGA 2011 Intel C602 Supports DDR3 1866 / 1600 / 1333 / 1066 R/LR ECC and UDIMM-Newegg.com
Click to expand...

but the asrock do have trouble with sleep in win10 though and many people also experience random crashes in win 10 with it. .. if you are sticking with lga2011 v1/v2, should also consider oem WS like Lenovo d30 dell t7600 hp z820. the Lenovo only have 4 pcie so might not fit your build but hp and dell has 6 pcie slots. they also have built in usb3 and sound onboard and good sleep implementation and will play nice with win10 all the way. v1/v2 Server boards lack built in sound and usb3 and they don't play completely nice with win10 sleep a lot of time.
 
Last edited:
Been running win10 pro for a while on my Asrock server and never had a problem.
Never use sleep mode so I can not say to that. System shuts down and starts up fast enough with a SSD I would not use sleep anyway.
Besides it is crunching data 24/7/365
 
x-cimo said:
I was confused about the [email protected] until I realized it's a V2. Which in the end might be a good thing, there are more V2 on ebay, and 128GB of ECC DDR3 looks much cheaper than ECC DDR4. So V2 might be the way to go. CPU are still super expensive however.

Wildpig, which Version of the 2670?, Will look at natex, thanks

EDIT: 2670 are V1, natex deals are pretty sweet, but MB don't support PCIe x16, and that wouldn't be much of an upgrade over my 3930K. But thanks :)
Click to expand...

yeah all the v2 cpu are still very expensive relative to v1 and that won't change til a massive server recycle hit like with v1. some says might never happen b/c a lot of companies didn't upgrade from v1 to v2. v3 is still very current so price still very high, even ES cpu are still high and I don't recommend it unless you don't care about how your data is treated.


Bill1024 said:
Been running win10 pro for a while on my Asrock server and never had a problem.
Never use sleep mode so I can not say to that. System shuts down and starts up fast enough with a SSD I would not use sleep anyway.
Besides it is crunching data 24/7/365
Click to expand...

https://forums.servethehome.com/index.php?search/4986111/&q=asrock+ep2c602&o=relevance

check out a few of those threads. there are several people random crash using it with win10. bill, maybe you can help them out?

a few people able to maybe get it to work stable w win10? but I haven't hear too many people say working with win10 and sleep. so just keep that in mind if win10 and/or sleep is something you want.
 
I'm still looking, so far mostly at v3 xeon. Why?, because of motherboard ( 5 pci-e ).

I've found good deals on "refurbished" xeons. Any ideas what that would mean? I can't be bent pins... and I really don't see anyone being able to fix broken cpus.

I'm currently looking at:
E5-2640v3: 8 core 2.6/3.4ghz 90W
and
E5-2663v3: 10 core 2.8/3.5ghz (NO Hyper-threading).. 120W...

I wonder which is better. the 2663 is slightly cheaper.
 
Last edited:
You
x-cimo said:
I'm still looking, so far mostly at v3 xeon. Why?, because of motherboard ( 5 pci-e ).

I've found good deals on "refurbished" xeons. Any ideas what that would mean? I can't be bent pins... and I really don't see anyone being able to fix broken cpus.

I'm currently looking at:
E5-2640v3: 8 core 2.6/3.4ghz 90W
and
E5-2663v3: 10 core 2.8/3.5ghz (NO Hyper-threading).. 120W...

I wonder which is better. the 2663 is slightly cheaper.
Click to expand...
I would recommend getting the 2683v3 or 2686v3 if your jobs benefit from more cores. If i recall correctly the 2683v3 offers 16 cores and the 2686v3 offers 18 cores. You could also get the QS versions. Check eBay.
 
seems like the e5-2686 v3 has become rare now. I can't find much of any oem or qs at all. e5-2683 v3 seems a lot more common. e5-2683 v3 is 14 cores
 
Last edited:
Here is how ebay prices looks so far: (exclude ES, and QS, only OEM/Retail)

E5-2640v3 2.6/3.4 8/16c 600
E5-2663v3 2.8/3.5 10/10c 579
E5-2660v3 2.6/3.3 10/20c none
E5-2666v3 2.9/3.3 10/20c 900
E5-2673v3 2.4/3.2 12/24c 399 <-- Feels like good balance of core, clock and price
E5-2680v3 2.5/3.3 12/24c 1200+
E5-2683v3 2.0/3.0 14/28c 349
E5-2683v4 2.1/3.0 16/32c 799
E5-2686v3 2.0/3.5 18/36c none
E5-2686v4 2.3/3.0 18/36c 1100
 
Last edited:
Last edited:
As an eBay Associate, HardForum may earn from qualifying purchases.
The 2686v3 comes into stock from time to time on ebay, either as OEM or QS (pretty much as good as OEM imo). Just keep checking every few days. Its another one of those custom chips made for a particular client. You might also consider Taobao (a chinese auction site) if your chinese is good. The chips are cheaper and the availability is much better since you're getting them from the source.

Yeah I'd recommend the 2683v3 if you're going to be running transcoding/VMs. Those tasks can really benefit from more cores. You might also consider getting the Z10PE-D16 WS version, which has more workstation friendly features if you plan on working on the machine as opposed to just sticking it in a closet somewhere as a server.
 
I think the e5-2686 v3 while more expensive at $500 is the best deal compared to 2673 or 2683. you get better clockspeed and a lot more cores for not that much more.
 
@wildpig Did you see any 2686 v3 anywhere?

Thanks StrategyFreak, I did intend to say the WS. I could have done a mistake ordeing.

Is the RAM link I posted alright?
 
don't see really much 2686. like strat said, it's a custom chip so not as much avail. I am actually surprised that it's doesn't cost higher than 500 b/c the only other 18 core is the 2699 which is like $1k.
 
Just wanted to say thanks for the help! I would have bought a higher clocked, fewer core otherwise.

I believe I found the best compromise between clock and speed at a decent price. The E5-2678v3. It's a 2.5Ghz/3.3Ghz Turbo 12C/24T beast.

There is actually very little information about it; it's a OEM version of the E5-2680v3 (which sells for 1500$ US on ebay), however the E5-2678v3 can be found for 600 US. I made an offer for two at 500 and got them :)

Here are my tips which might help people looking a building a Xeon monster.

1) Get a current Generation(v4) or previous generation(v3): Because older motherboard will be hard to find and will lack current features (e.g NVMe M.2, PCI-E 3.0, or many PCI-e slots).
2) Buy a brand new motherboard. Motherboard will die a lot faster than a CPU. The ASUS ASUS Z10PE-D16 WS was flawless to get up and running.
3) Look for good Freq / # core ratio. Going too low in freq won't be good for single (or few threads) tasks.
4) Look at turbo boost potential. Some Xeon have pretty poor Turbo Boost, also turbo boost apply even if all core are 100% ---> This matters!!

I found this page very handy: List of Intel Xeon microprocessors - Wikipedia

E5-2678v3: 2.5 GHz 4/4/4/4/4/4/4/4/5/6/8/8

This mean that this CPU will do: 2.5 + 4/4/4/4/4/4/4/4/5/6/8/8 * BCLK (100Mhz) which translate to: 2.9/2.9/2.9/2.9/3.0/3.1/3.3/3.3 Ghz (2.9Ghz if all core are 100%, or 3.3Ghz! if only 2 core are 100%).

So having this +4 boost frequency even if all core are busy is a big deal which you don't see if you just look at the base frequency.

In comparaison, the 2640v3 I was originally looking at has a base clock higher 2.6Ghz, but a boost of only 2/2/3/4/5/6/8/8, and only has 8 cores.



For anyone ending up here to get more info on this CPU here is a CPU-Z for it, (I could only find ES CPU-z of this CPU, here is an Retail one):

E5_2678v3.png





IMG_2572.jpg

IMG_2578.jpg
 
Last edited:
For my duallie, I stayed with the Romley-EP platform. No FIVR, and WAY better clock speeds than Grantley-EP. I'm in the process of swapping in a pair of E5-2673 V2s into it. Basically, the they are the same as the E5-2687W V2 but much lower TDP. It shares the 2687W V2s all core turbo of 3.6GHz and single core turbo of a full 4.0GHz. It'll be a very good server and backup machine, and has a miserly TDP of only 110W to boot (vs the 2687W's whopping 150W for the same clock speeds).
 
lutjens said:
For my duallie, I stayed with the Romley-EP platform. No FIVR, and WAY better clock speeds than Grantley-EP. I'm in the process of swapping in a pair of E5-2673 V2s into it. Basically, the they are the same as the E5-2687W V2 but much lower TDP. It shares the 2687W V2s all core turbo of 3.6GHz and single core turbo of a full 4.0GHz. It'll be a very good server and backup machine, and has a miserly TDP of only 110W to boot (vs the 2687W's whopping 150W for the same clock speeds).
Click to expand...
Actually i am fairly positive that lower TDP means in this case that it will boost less on heavy workloads [read: AVX].
 
lolfail9001 said:
Actually i am fairly positive that lower TDP means in this case that it will boost less on heavy workloads [read: AVX].
Click to expand...

TDP is the power dissipated at the specified clock speed. A chip with a lower TDP rating and the same clock speed will usually be a higher binned chip that's capable of running at the specified clock speeds (including those clocks specified as boost speeds) while dissipating less power, which is usually accomplished through reduced voltage. If one tried to overclock an unlocked low TDP chip without increasing the TDP limit to compensate for the increased clock speed, indeed, reduction of clock speed will occur to ensure the TDP isn't exceeded. But at default clock speeds (which all 2P Xeons run at) the E5-2673 V2 is specified to be able to run at those clock speeds which remaining at or below the quoted TDP (and usually run at considerably lower than the specified TDP).

The TDP limits specified in the chip are simply there ensure that those limits aren't exceeded, a situation that definitely should not occur when running at default clock speeds.
 
lutjens said:
TDP is the power dissipated at the specified clock speed. A chip with a lower TDP rating and the same clock speed will usually be a higher binned chip that's capable of running at the specified clock speeds (including those clocks specified as boost speeds) while dissipating less power, which is usually accomplished through reduced voltage. If one tried to overclock an unlocked low TDP chip without increasing the TDP limit to compensate for the increased clock speed, indeed, reduction of clock speed will occur to ensure the TDP isn't exceeded. But at default clock speeds (which all 2P Xeons run at) the E5-2673 V2 is specified to be able to run at those clock speeds which remaining at or below the quoted TDP (and usually run at considerably lower than the specified TDP).

The TDP limits specified in the chip are simply there ensure that those limits aren't exceeded, a situation that definitely should not occur when running at default clock speeds.
Click to expand...

Sorry, but you are wrong, TDP throttling happens on default clocks and in case of OC it is usually possible to overwrite any automatic clock scaling. It is especially obvious on laptops, but on server chips it was reported as well, heck, it was reported on 2679v4 of all chips!
And yes, they usually run way below quoted TDP but a quick look at powertop reveals why.
 
lolfail9001 said:
Sorry, but you are wrong, TDP throttling happens on default clocks and in case of OC it is usually possible to overwrite any automatic clock scaling. It is especially obvious on laptops, but on server chips it was reported as well, heck, it was reported on 2679v4 of all chips!
And yes, they usually run way below quoted TDP but a quick look at powertop reveals why.
Click to expand...

Since you know everything I'm probably wasting my time, but I'll make an effort to educate you nonetheless. The E5-2679 V4 is not a mainstream chip. It's a custom chip for Dell. The E5-2698A V3 is another example...a custom chip designed for a particular client, in this case Lenovo. Both were intended to be water-cooled. These chips have very different temperature throttling thresholds than the run-of-the-mill chips. The E5-2698A V3, for example, throttles hard at a mere 65 deg C. This strategy ensures that the chips are actually water-cooled as intended, which is actually an interesting approach by Intel. But again, since you know everything, you were probably already aware of this...;)
 
lutjens said:
Since you know everything I'm probably wasting my time, but I'll make an effort to educate you nonetheless. The E5-2679 V4 is not a mainstream chip. It's a custom chip for Dell. The E5-2698A V3 is another example...a custom chip designed for a particular client, in this case Lenovo. Both were intended to be water-cooled. These chips have very different temperature throttling thresholds than the run-of-the-mill chips. The E5-2698A V3, for example, throttles hard at a mere 65 deg C. This strategy ensures that the chips are actually water-cooled as intended, which is actually an interesting approach by Intel. But again, since you know everything, you were probably already aware of this...;)
Click to expand...
Well, 2679v4 from what i recall is ebay-only [as in made for Ebay] chip, but otherwise, that's a curious piece of info. Though that does not do anything about my TDP point since i am positive even 2699v4 TDP throttles.
 
The point I was trying to make is that there are many reasons why a chip would throttle, not just TDP. The actual TDP limit for Xeons at their default clock speed (which are rated VERY conservatively) is rarely reached, except when overclocking.
 
lutjens said:
The point I was trying to make is that there are many reasons why a chip would throttle, not just TDP. The actual TDP limit for Xeons at their default clock speed (which are rated VERY conservatively) is rarely reached, except when overclocking.
Click to expand...

E5-2679V4 on R5E

I doubt thermal throttling is present here. Though it may have to do with wacky BIOS, indeed.

Anyways, if you will have time, you can probably verify it yourself with your chips by leaving it to run some stress utility for few hours and see the actual clock. Gl.
 
One would have to take a look at the actual values being used by the system for TDP via a readout through HWInfo or AIDA. Sounds like the BIOS is throttling very prematurely (it's probably little confused with a factory processor that has a 200W TDP rating). Given that most of the time you can usually adjust the TDP values in the BIOS, the user in that thread could try manually specifying 200W in the BIOS instead of leaving the system to its own discretion.;)
 
You must log in or register to reply here.
Back
Top