ARM server status update / reality check

P

pxc

Extremely [H]
Joined
Oct 22, 2000
Messages
33,063
We're just what, 12 months from ARM microservers taking over 20% of the server market? Yeah...

Worldwide Server Market Revenues Increase 5.2% in the Fourth Quarter as Demand in China Once Again Drives the Market Forward, According to IDC

press release said:
  • Demand for x86 servers improved in 4Q15 with revenues increasing 8.0% year over year in the quarter to $12.5 billion worldwide as unit shipments increased 4.0% to 2.6 million servers. HPE led the market with 29.0% revenue share based on flat revenue growth over 4Q13. Dell retained second place, securing a 20.5% revenue share following 5.3% year-over-year revenue growth.
  • Non-x86 servers experienced a revenue decline of -5.4% year over year to $2.9 billion, representing 18.6% of quarterly server revenue. IBM leads the segment with 75.8% revenue share on the back of a year-over-year revenue increase of 8.9% when compared with the fourth quarter of 2014. ARM server sales fell in 4Q15 compared to the same time in 2014, with HPE Moonshot system deployments representing the largest single component.
Click to expand...

tl;dr
x86 server sales up YoY by 8% and 4% by revenue and units
non-x86 server revenue down, despite IBM's growth from POWER (suggesting Oracle and every other non-x86 server tanked hard)
ARM server sales fell year over year, from so small it's hard to measure to somehow even less, and the biggest ARM server player is the practically dead HP Project Moonshot.

This is not a trajectory for ARM's server success.
 
they are expensive no? new and expensive = death...

meanwhile for me Rpi's are taking over a lot of what I used to do with x86 boxes... they are great for asterisk and other simple small business duties that can be achieved with linux... thing is a lot (and i mean a LOT) of stuff is either still married to M$ software or just not worth the migration to ARM for most people... maybe the facebook and amazons of the world can afford the investment to lower power consumption but most companies don't really concern themselves with what their server infrastructure is using as far as power

more money to be saved in low power desktops... but again, Microsoft reigns supreme... so getting low power x86 stuff I think is going to kill any need for ARM in that department
 
FLECOM said:
they are expensive no? new and expensive = death...

meanwhile for me Rpi's are taking over a lot of what I used to do with x86 boxes... they are great for asterisk and other simple small business duties that can be achieved with linux... thing is a lot (and i mean a LOT) of stuff is either still married to M$ software or just not worth the migration to ARM for most people... maybe the facebook and amazons of the world can afford the investment to lower power consumption but most companies don't really concern themselves with what their server infrastructure is using as far as power

more money to be saved in low power desktops... but again, Microsoft reigns supreme... so getting low power x86 stuff I think is going to kill any need for ARM in that department
Click to expand...

Yeah, so far there's no quantifiable power savings once you make the hardware grow-up. Who knew that gobs of cache clocked at 3+ GHz uses the same amount of power on any architecture?

Pi is only an industry because they hit the right price/performance/power point first. The popularity does not signify there's a growing need for ARM in the server room, but simply recognizes the fact that tweakers like a challenge and x86 is too easy/expensive/high power.

Any alternative CPU architecture could have done that :D

You'll find that your average company tech department has exactly the opposite values concerning what powers their server rooms.
 
defaultluser said:
Yeah, so far there's no quantifiable power savings once you make the hardware grow-up. Who knew that gobs of cache clocked at 3+ GHz uses the same amount of power on any architecture?
Click to expand...

Not only that, but Intel's Bay Trail stuff is pretty competitive with ARM in terms of power consumption. The reason we haven't seen a huge performance bump since Sandy Bridge is because Intel's main focus is now on competing with ARM on power consumption and feature integration. They're approaching being on par with them even in smartphones, only issue is that companies already invested in ARM designs and don't want to port everything back over to x86. They also like all the customization they get to do with ARM, whereas Intel likes to design once and ship several copies of the same thing.
Pi is only an industry because they hit the right price/performance/power point first. The popularity does not signify there's a growing need for ARM in the server room, but simply recognizes the fact that tweakers like a challenge and x86 is too easy/expensive/high power.
Click to expand...
I'm sort of a tweaker, but I ended up getting a x86-based SoC for a low-power application after debating with myself about getting a Hummingboard or a Raspberry Pi. It was pre-assembled and half the price of the Hummingboard, with guaranteed compatibility and a better fanless cooling design. It was so much better than any ARM system due to backwards compatibility and graphics acceleration alone. Even Intel's worst IGP offerings are better than the VideoCore IV nowadays. Besides, when I really want a challenge, it's more fun to build your own Z80 motherboard from scratch and implement VGA graphics and SD cards by hand. Best thing about a Z80 is it's backwards compatible with the 8080, which means you can play around with legacy software without even doing emulation.

In any case, I really think the argument for ARM servers is weak... I mean, if your server is low-power, Intel has that covered too. No need to port stuff over to ARM. Everyone that did this, either did it back when Intel didn't offer what they needed in terms of power consumption, or they needed a level of hardware customization for specific workloads they couldn't get on x86.

I think ARM's niche is in embedded. An embedded server is still just another embedded application that can get away with lower power. Server market is tied up by Intel and IBM... if people wanted a non-Intel architecture, they'd probably go with IBM's POWER servers.
 
Last edited:
Yeah, they still sell Z80 dev boards. I used a Rabbit in a class about ten years back :S

Mind you, there are signs of SOME demand for Arm in servers, but nobody has announced a design win yet. We'll have to see how much Qualcomm aimed the design of Kryo at servers, and if they can actually beat Intel.

Google Looking To Tap Qualcomm To Replace Intel As Chip Supplier For Its Data Centers: Report

It would be an impressive win if they get Google interested. But even if they do, Google is a bit of an island - nearly everything they use on a daily basis is both FAIRLY NEW and custom code. This makes porting between architectures easier, because you still have access to the brain trust that created the code.
 
Last edited:
ChristianVirtual said:
Exclusive: First Cavium ThunderX Dual 48 Core (96 core) Benchmark
Click to expand...

That's the best showing so-far of the Xeon competitors, if your load is massively-threaded and integer-heavy.

For float-heavy rendering the performance is the same as an Intel 6c/12t Haswell at 2.4 GHz. Not exactly one of the most expensive Xeons, so this thing will have a tough time gaining ground. You can also cut costs and go with the 4c/8t parts for 95w and get the same performance.

Power consumption estimates are similar as well: around around 95w estimated for the 48-core ThunderX full-tilt.

ARM Servers: Cavium Is A Contender With ThunderX

Their extravagant claims are so-far busted:

when you build systems out of Xeon and ThunderX chips, for a given unit of performance, the ThunderX platform would have around a 50 percent thermal advantage.
Click to expand...

Unless you're running compression all day, this doesn't hold-up. And if you are, then it's only 20% better (slower than Xeon 8 core at 120w).

So this core had better be cheaper to make, because it's not better at anything. Even the Atom Avoton would match it easy, if Intel could be bothered to make a 32/64-core version :D

Of course it would be nice to have a full review instead of this "sneak peek."
 
Last edited:
The real key is that x86 has years and years and years of compiler optimizations piled up.

It'll take awhile to catch up just in that area alone.
 
Google talking about "porting" software to POWER arch:

“For good software developers enabling Power is just modifying a flag in a config file and off they go,” said Maire Mahony, an engineering manager in Google’s platform group that is responsible for its data center hardware and a director of the Open Power Foundation.


I've been saying this for awhile.

When you target a non-retarded OS it's quite easy to move between architectures.
 
...If you build custom software, and If what you do doesn't require custom hardware to get it's job done at a reasonable pace.

Otherwise, you're tied to where the drivers roam. Luckily Nvidia supports Power for HPC :D
 
jwcalla said:
Google talking about "porting" software to POWER arch:

“For good software developers enabling Power is just modifying a flag in a config file and off they go,” said Maire Mahony, an engineering manager in Google’s platform group that is responsible for its data center hardware and a director of the Open Power Foundation.


I've been saying this for awhile.

When you target a non-retarded OS it's quite easy to move between architectures.
Click to expand...
That version of the quote completely missed the context of what work had to be done to enable that, and what it actually means:

in context said:
“We have ported our infrastructure onto the Power architecture. And most importantly, what that means is that our toolchain supports Power and so for our Google developers, enabling Power for their software applications is simply a matter of modifying a config file and off they go.”

This is not the first time that Google has ported its software stack to alternative platforms, of course, and McKean was very clear with us last year that Google is always testing out its software on alternative architectures to prevent “bit rot” and keep its options open. But what seems very clear now is that for certain workloads at least, Google is now in position to actually start rolling out its applications on Power. The fun bit about this is that we, as Google users, will never know if it does.
Click to expand...

Inside The Future Google Rackspace Power9 System

What Google is talking about is a platform based on Power9 (not out yet and not due until 2017... and btw POWER8 was very late and underwhelming, certainly not a model of power efficiency either, to put expectation in perspective) and a new efficient power delivery high density servers (also not out yet and still in the planning stage). This is what ARM and other alternative MPU hype does to people. It makes the imaginary "real". :p
 
I recognize Google's example was POWER and not ARM but I needed some thread to put my snark in and this one was best suited. :)
 
But you know better than that. Aside from Netware boxes, Power VASTLY predates x86 servers/compute. So it's no surprise it's on the short list of desirable architectures, and has full support from Nvidia.

By comparison, Arm on server/compute is like a third world country, raw and lacking basic infrastructure. Going to take huge mountains of money to fix that problem.

I mean, Nvidia opened CUDA on arm 64 two years ago, but that still requires the effort to integrate it into each platform's development environment. That's why Cavium is so far the only one to announce support.
 
I'm not really sure that Google will be having much of anything to do with NVIDIA or HPC for their POWER-based servers.

But my point there is that if your software wasn't written in a closed ecosystem, it's trivial to port to another ISA, whether that is ARM or POWER.
 
Related:
IDC: Intel server CPU unit share in 2015 was 99.2%. That's not 99.2% of x86/x86-64 server CPUs, but 99.2% of ALL server CPUs (~22.1 million units). That's up from 93.0% Intel had in 2010. Ouch.

Gradual Change in Server Microprocessor Market; IDC Expects Competition and Evolving Workloads to Change Supply Ecosystem in 2017 | Business Wire

Worldwide x86 and ARM Server-Class Microprocessor Forecast, 2016–2020 (summary only unless you pay)

This is what the ARM server revolution looks like. lol
 
This brings us more clarity on that ThunderX Serve The Home crapfest of a "preview."

Intel Lines Up ThunderX ARM Against Xeons

Since Cavium won't publish benchmarks, Intel has happily helped them out. Not sure how trustworthy they are, but no worse than any other marketing literature.

Biggest insights:

Those ThunderX chips burn some serious power, at 130w @ 2 GHz, and 160w @ 2.5! This is sourced from David Kanter, and then backed-up by Intel's single-socket 180w whole system estimate. That's WAY ABOVE the 90-100w I thought these beauties were burning at 2.0!

And the power management is just as broken as is is on XGene 2. Haven't these people ever heard of reducing vampire load? Servers aren't heavily loaded most of the time, but they need to be ON ALL OF THE TIME. And it's not Gigabyte's fault, as Intel sourced a Xeon server from them too (just to make sure).

Power consumption will drop to more competitive levels with 16/14nm, but since this is a big chip, they will have all the same fun everyone else already had getting big chips ramped-up on this problematic node.
 
Last edited:
From that article:

Networking chip maker Cavium is one of the ARM server chip upstarts that is taking on Intel’s hegemony in the datacenter, and is probably getting the most traction among its ARM peers in the past year with its ThunderX multicore processors.
Click to expand...

HP's Moonshot is still the top ARM server chip, according to all the prior IDC releases that have mentioned ARM vendors, including the most recent one from around a week ago. Worldwide Server Market Revenues Decline by 3.6% in the First Quarter, Curbed by Hyperscale Slowdown, According to IDC

IDC said:
  • Non-x86 servers experienced a revenue decline of 28.7% year over year to $1.8 billion, representing 14.7% of quarterly server revenue. IBM leads the segment with 62.7% revenue share despite a 32.9% revenue decline when compared with the first quarter of 2015. IDC also continued to track falling revenue from ARM server sales in 1Q16, with the HP Moonshot system deployments representing the largest single component [by far, representing about 90% of the ARM server market IIRC].
Click to expand...

If Thunder-X has gained any traction, it's unmeasurable. The truth is that no ARM vendor has gained traction. ARM servers peaked a few quarters ago to barely measurable and have fallen for 3 straight quarters since. x86 servers had normal rates of growth in the same time period.

Thunder-X didn't do too well in AT's testing: ARM Challenging Intel in the Server Market: An Overview The one potential benefit that Johan suggested for these type of ARM server chips has long been plugged with 20-45W Xeons released since.

I don't have anything against the author of that article, but those types of articles are pieced together from press releases and press packages. Great for generating content for his site, but is not objective reporting.
 
Last edited:
Yeah, it's the same old story every time. Sparc couldn't stop x86 plus Linux. Even though it was "open," nobody except Sun and Fujitsu put any serious effort into core designs.

And PowerPC looked like a serious effort to retake the Workstation and small server market...except that Motorola didn't really design it's chips for that market, and IBM was too busy putting PowerPC into everything BESIDES workstations. The 970 and G3 were lost opportunities.

It's so easy to get distracted, because competing with Intel head-on WITHOUT x86 is near impossible. And it's still hard when you HAVE an x86 license.

Both Motorola and IBM had to coordinate and put everything they has into PowerPC to stop Intel's monster Pentium Pro in the 1990s, but none of them wanted to. It's not going to magically change 20 years later.
 
Last edited:
defaultluser said:
Sparc couldn't stop x86 plus Linux.
Click to expand...

Well if I recall SPARC (or other RISC alternatives) was the market leader for awhile but once x86 + Linux came on the server market it undercut the prices of the competition tremendously. And then eventually Intel's single-threaded performance was too good to turn down.

It's hard to compete with Intel because (w/ the consumer market) they have such great economies of scale, allowing them to sell server chips so cheaply.

x86 + Linux was definitely "good enough" but a crapton cheaper, so eventually it became impossible to argue against adopting it.

Meanwhile, is it just me or is ARM (the company) completely uninterested in anything other than small mobile devices? I mean they don't even want to have anything to do with opportunities like Chromebooks, etc.
 
Nothing is stopping ARM from being used in Chromebooks, it's just a lack of performance competitive with Bay Trail once you can feed tons of power to the chip. Might as well get the real deal :D

ARM designs their chips for high efficiency, which means the target design frequency of the chip is intentionally lower than they could make it. It means they miss out on design wins in larger devices like Chromebooks, but I don't think it worries them.

They know that Chromebooks are a waste of time and effort: a land of low-margin Linux boxes fishing for people tired of Windows, and some sales in schools.

And yeah, I was referring to Sparc in the mid 90s, after they won the internet server market by default. SGI MIPS imploding, HP PA RISC disappearing in preparation for Itanic, DEC Alpha late to the party, then imploding, and PowerPC having to build Workstation/Server credibility from the ground-up. That left only one high-end chip marker able to make moves.

Then when Intel surprised everyone with the Pentium Pro right about the same time Linux became solid, the internet forgot who Sun was overnight. They're lucky they made Java a standard before their hardware cratered, or else they wouldn't have survived beyond 2001.
 
Last edited:
defaultluser said:
They know that Chromebooks are a waste of time and effort: a land of low-margin Linux boxes fishing for people tired of Windows, and some sales in schools.
Click to expand...

Everything ARM is low-margin. And it's not like they sell the chips, just license the designs. So it would be money in the bank for them.
 
Well, Cavium is not as bad as Intel's marketing would have you think. Big surprise there :D

Investigating Cavium's ThunderX: The First ARM Server SoC With Ambition

Looks like in the hands of someone who knows what they're doing, ThunderX is not so horrible at integer-heavy workloads, except in performance/watt. That could be forgivable at the price they charge, if they can just fix the horrible power management.

I'm now a little more interested in how well they fix things in ThunderX 2. Too bad we'll have to wait two years for benchmarks.
 
^lol It's a hobbled together pierce of crap software-wise, power hungry as hell at just 2/3 its target frequency and isn't even competitive with current Xeons. It's half as efficient per watt as the Xeon-D. Nothing in that review suggests it's going to turn around ARM's infinitesimal and falling market share in servers. Especially this:
Gigabyte could not tell us how much the R120-T30 would cost, only that the pricing would be "competitive with similar midrange Xeon E5 servers".
Click to expand...

Same price as E5, but slower and more power hungry. That should work out great. :p

BTW, Johan also wrote the last article which tested Thunder-X that I linked above. Same turd sammich, just on different bread.
 
Yeah, they need to fix the damn efficiency and ludicrously high idle power.

I give them credit for being the first to come anywhere close to real Xeon performance/dollar with a mostly viable RHEL platform, even if they completely forgot about performance/watt in the process. That's why my interest has elevated from "couldn't be bothered" to "slightly interested."
 
6 months later (Q4'15 sales in original post, Q2'16 in latest IDC and Gartner analyses), ARM in servers is still going nowhere.

Worldwide Server Market Revenue Declines 0.4% in the Second Quarter, Awaiting Imminent Hyperscale Buildouts, According to IDC

IDC also continued to track minimal revenue from ARM-based server sales in 2Q16; ARM sales have yet to make an impact on the server market.
Click to expand...

Gartner didn't even bother to mention tiny ARM server revenues: Gartner Says Worldwide Server Revenue Declined 0.8 Percent in the Second Quarter of 2016, While Shipments Increased 2 Percent
 
juanrga said:
Apples and oranges...
Click to expand...

Not necessarily. If you look at the underlying reason of why those other architectures failed under Windows (software availability), the problem remains when Windows Server ARM comes out. You'll have a hardware platform that runs Windows with only a subset of all the Windows software available. For some, that might be enough. For others, they simply won't see the point in running Windows on multiple hardware platforms when they can simply standardize on an Intel/AMD platform and run with it.
 
niomosy said:
Not necessarily. If you look at the underlying reason of why those other architectures failed under Windows (software availability), the problem remains when Windows Server ARM comes out. You'll have a hardware platform that runs Windows with only a subset of all the Windows software available. For some, that might be enough. For others, they simply won't see the point in running Windows on multiple hardware platforms when they can simply standardize on an Intel/AMD platform and run with it.
Click to expand...

Alpha, PowerPC, Itanium, and MIPS have failed or disapeared, independently of the software used: Microsoft, IBM, linux, or whatever. The problem with Windows RT wasn't on the ARM side and it seems seems is not repeating same mistakes with the new Windows 10 for PC.

Microsoft already announced more than half of its datacenter will switch to ARM. And another two big datacenter announced using ARM as well. There is no software problem for those are making the announcements neither for those already using ARM (Paypal has been using ARM servers for a while). Legacy guys can continue relying on Xeon and Opteron.
 
Reminds me of IBM trying to push Linux on POWER. Sure, they're running it and can probably get customers to run on their hardware but who else is going to jump on. Azure can help but if Microsoft is the only one really doing Windows Server on ARM, eventually the economics work out better to dwindle back to x86 again. The problem is that every time another CPU architecture comes along, Microsoft tries it out. And, no MIPS was still going pretty strong back with SGI, Tandem, and Cray all running MIPS processes when Microsoft bowed out. Same story for DEC with the Alphas handling Linux, VMS, and Digital UNIX. PowerPC is still around in the POWER processors. Microsoft could easily be running on that architecture if they chose to. They tend to bow out of alternative CPUs relatively easily.

So good luck to Microsoft and trying out yet another CPU architecture. I'll sit back and see how this one goes as it just strikes me as another, "no, this time will be different!" Maybe it will. Maybe they've compiled enough lessons learned from past attempts. Only time will tell.
 
This should be good!

Now it just has to compete with AMD's EPYC, which pulls the same dandy trick of NUMA-on-the-cheap, but with x86-compatability. See here, it's under half the cost of the Xeon 8176, and has lower power consumption under full-load. And beats the Xeon in everything EXCEPT databases.

https://www.anandtech.com/show/11544/intel-skylake-ep-vs-amd-epyc-7000-cpu-battle-of-the-decade/22

That's where the real battle will unfold. Will Qualcomm have enough of an advantage to win mainstream ports of server software?

Like I said at the top of the page, getting wins in MS and Google is easy, as they do nothing but custom code and cloud. But for mainstream acceptance, you need to convince server application writers to port. I'd guesstimate that that's around 80-90% of the server market.

Even if MS ports their ARM x86 emulator to handle AMD64, the perf/watt advantage disappears (or might go backward) with emulated code.


EDIT: this is the Cloudflare Blog Post that everyone's just skimming-over:

https://blog.cloudflare.com/arm-takes-wing/
 
Last edited:
defaultluser said:
This should be good!

Now it just has to compete with AMD's EPYC, which pulls the same dandy trick of NUMA-on-the-cheap, but with x86-compatability. See here, it's under half the cost of the Xeon 8176, and has lower power consumption under full-load. And beats the Xeon in everything EXCEPT databases.

https://www.anandtech.com/show/11544/intel-skylake-ep-vs-amd-epyc-7000-cpu-battle-of-the-decade/22

That's where the real battle will unfold. Will Qualcomm have enough of an advantage to win mainstream ports of server software?

Like I said at the top of the page, getting wins in MS and Google is easy, as they do nothing but custom code and cloud. But for mainstream acceptance, you need to convince server application writers to port. I'd guesstimate that that's around 80-90% of the server market.

Even if MS ports their ARM x86 emulator to handle AMD64, the perf/watt advantage disappears (or might go backward) with emulated code.


EDIT: this is the Cloudflare Blog Post that everyone's just skimming-over:

https://blog.cloudflare.com/arm-takes-wing/
Click to expand...

Ananadtech review crippled the performance of Xeons by 40--60%
 
Oh, well then I guess I'll just wait for someone who knows what they're doing to compare them.

But it sounds like EPYC is a lot tougher sell
 
An image is more than 1000 words


Same performance, about half power consumption and cheaper acquisition costs.
 
Very interesting reading. Just porting SIMD code optimized for x86 SSE, the ARM server was noticeably slower than Skylake Xeon. After a bit of optimization for ARM NEON ISA, the ARM server beats the Xeon

With the new implementation Centriq outperforms the Xeon at batch reduction for every number of workers. We usually run Polish with four workers, for which Centriq is now 1.3 times faster while also 6.5 times more power efficient.
Click to expand...
 
Last edited:
You must log in or register to reply here.
Back
Top