Dual CPU vs single CPU

[Kelter]

I guess this question kind of half fits in this forum. Anyways, a little background of what I am looking at. Currently I am up in the air between a dual quad core xeon solution vs a single 8-core solution.

My gut tells me that a dual CPU solution would be faster all things equal. Unfortunately, there is no way to even compare since none of the newer processors are quad core anymore. On the quad core side, I would be looking at a Xeon 5600 series processor vs an 8-core e5-2600 series processor.

Now the newer e5s are probably faster per core, but does that offset the benefit of dedicated memory per processor and additional data paths?

Anyways, was hoping some folks might have experience with a similar situation.

 

[bwcall]

Crucial data missing: what the heck are you using it for?

 

[Kelter]

Crucial data missing: what the heck are you using it for?

I wasn't too sure if what I was planning to use it for was crucial information or not, I simply wanted to see what type of performance differences I might expect to see. ie. What type of activities would be impacted being that my system memory bandwidth effectively gets halved?

But, if it really helps, the primary goal would be to run Oracle RAC, which is the main reason for capping CPU counts. 8-cores of licensing is ~$240k per node. So the cost of hardware is a fraction of the total cost.

The underlying hardware will be all carrier grade maintaining a 5 9's SLA. I currently have other similar deployments using dual CPUs (some with dual quad cores, and others with dual 8-cores but I am using hardware partitioning and essentially only using 4 cores from each CPU). I want to get away from the hardware partitioning as it just adds another layer of complexity to the system which is why the question of whether dual quads is the way to go or single 8-core.

 

[mikeblas]

Hunh? What is it that halves your memory bandwidth? How did you measure that?

 

[ToddW2]

I've seen 2 x 5xxx Series XEONs compared to a SINGLE E3-1230 v2 and the newer CPU wins, so the E5-2600 would FOR SURE be better.

 

[Kelter]

Thanks Todd. Ya, I'm pretty sure pure CPU power, a single e5-2690 would be faster than the 2 quad core xeon 6xxx CPUs. I guess my main concern would be memory. Most of these multiprocessor systems have separate memory channels per CPU. With a low amount of memory, I imagine it's not an issue, but at what point does it become a bottleneck? I am planning on 128GB of memory total. With a single 8-core, it would be 128GB for the 1 CPU. With the dual quad core, it would be 64GB per CPU. I just don't know how much of a difference this makes.. if any.

Which leads into Mike's question. Given that each CPU has access to it's own memory, I would think that memory throughput would be higher in a multiCPU solution, but I don't know for sure or even how much.

 

[mikeblas]

As far as I know, all modern multi-socket PC systems have CPU-local memory. This architecture is called NUMA -- non-uniform memory architecture. In a NUMA motherboard, each socket has a channel to its processor-local memory. It also has a channel to the other processor. The arrangement is the opposite of symmetric multi processing, where each processor has equal access to equal resources.

Software that's written to be NUMA aware will allocate memory local to the processor that will access the memory. If CPU 1 needs some memory, it will have the OS guarantee that the memory is satisfied in the bank local to CPU 1. Then, when CPU 1 runs code touching that memory, it does so in isolation from the other processors.

This actually ends up being a savings, since all processors have the opportunity to execute without causing contention over the same memory -- or the acess path to that memory. It also saves processors from assuring that they have a consistent view of every other processor's memory in their cache.

It can -- and must, in fact -- happen that one processor accesses memory local to another processor. When that happens, the processors use a communications channel that links them to eachother at the same speed as their memory bus. The processor that owns the memory receives the request and handles the memory

The potential for non-local memory access is substantial; the processor making the request waits longer for the remote access than it would for hte local access. The servicing processor has to service the request and interrupt its own memory access. It also must assure cache coherency between the two processors (and any other processor in the system, too). The remote access isn't slow, but it has much more latency and residual cost than a local access.

I still don't feel confident that I understand your concern, but I hope that helps you understand what's going on.

 

[Kelter]

Thanks Mike,

That is actually very informative. So if I understand correctly, between each CPU and it's respective memory is a channel. And between each CPU and the other CPU's memory is another channel that can handle traffic at the memory bus speed.

In a single CPU, the CPU would just have it's single channel access.

So, it really comes down to the application and where it puts data in the memory respective to which CPU needs to access it. IF it were to be perfect and the memory needed was always access by the correct CPU, I would imagine your performance would increase, but realistically, it would use the memory bus speed channel between CPU/memory pairs, which I would imagine is slower?

I guess I probably wouldn't notice memory performance between the two setups, but the difference between the faster processor vs the older ones would be apparent.

Or I could have just interpreted it all completely wrong. lol.

 

[mikeblas]

So, it really comes down to the application and where it puts data in the memory respective to which CPU needs to access it. IF it were to be perfect and the memory needed was always access by the correct CPU, I would imagine your performance would increase, but realistically, it would use the memory bus speed channel between CPU/memory pairs, which I would imagine is slower?

Ideally, each CPUs is executing code and reading and writing its own local memory. They don't need to talk to each other. Carefully coded applications (like SQL Server or Microsoft Exchange Server) allocate memory locally very carefully, then schedule work on threads locked to the processor where memory is allocated. This is usually the fastest way to get work done, but there are some exceptions.

If one CPU needs data from memory local to another processor, the access to the memory has the same bandwidth but a lot more latency than getting the memory locally. Latency hurts -- it's far worse than a cache stall -- but it also causes some disadvantage for the processor that owns the memory which is being accessed.

 

[AndyE]

For products with high per core licensing cost, Intel introduced the high-frequency / low core count versions.
Take a look at E5-2643 and E5-2637 CPUs.

These CPUs have high per core performance, a comparatively low core count and good memory bandwidth.

Further reading (its on SQL Server, but applies likewise to Oracle)
http://www.sqlskills.com/blogs/glen...rocessors-for-sql-server-2012-oltp-workloads/
http://www.sqlskills.com/blogs/glenn/deciding-what-processor-to-choose-for-sql-server-2012/

hth,
Andy

 

[Kelter]

Thanks for the help everyone.

Andy, I wish I had looked at those CPUs earlier to see if the hardware vendor had those as an option. I did ask for a dual quad solution or a single 8 core solution, and it seemed the only item available was the single 8-core solution. We are using Hitachi Blade Symphony 2000s and there really isn't much in the way of seeing what options are available. Their website is much better now than it used to be... previously when we purchased these, any customization, the VARs had to go back to Hitachi SEs, who inturn had to go back to their japan counterparts to see if it would be available in the US market.