Specialized Processors are Seemingly Overtaking General Purpose Chips

AlphaAtlas

[H]ard|Gawd
Staff member
Joined
Mar 3, 2018
Messages
1,713
The "CPU" has been the heart of the modern computer for decades. As chip design costs. and speeds, have increased, the tech industry has typically defaulted to dumping more resources into flexible, integrated general purpose processors rather than spending millions on specialized chips for specific tasks. But, citing a paper from Thompson and Spanuth, The Next Platform believes that the era of general purpose computing may be coming to an end.

The meteoric rise of graphics processors is perhaps one of the earliest and most visible examples on the trend. At first, these "semi-specialized" processors only took over very specific graphics workloads from the CPU. Eventually, specialized logic was added to handle video decoding and encoding, general purpose compute, and as as of recent times, machine learning workloads. But processors specifically tailored for machine learning are already starting to overtake GPUs, while demands from the IoT market are also making custom tailored, efficient processor designs more economically viable. Thanks to cageymaru for the tip.

Thompson and Spanuth offer a mathematical model for determining the cost/benefit of specialization, taking into account the fixed cost of developing custom chips, the chip volume, the speedup delivered by the custom implementation, and the rate of processor improvement. Since the latter is tied to Moore's Law, its slowing pace means that it's getting easier to rationalize specialized chips, even if the expected speedups are relatively modest. "Thus, for many (but not all) applications it will now be economically viable to get specialized processors - at least in terms of hardware," claim the authors. "Another way of seeing this is to consider that during the 2000-2004 period, an application with a market size of ~83,000 processors would have required that specialization provide a 100x speed-up to be worthwhile. In 2008-2013 such a processor would only need a 2x speedup."
 
That will be the only way we end-round the death of Moore's Law. I don't believe there is any magic new physics coming along anytime soon to make the lone CPU's leaps-and-bounds advancement comparable to what it was in the past.
 
"Coming to an end" is silly.

As long as there are needs for general purpose computers there will be needs for general purpose CPU's.

That said, a general purpose design will always come with some overhead, and as we get closer and closer to the limits of silicon it is not surprising that more and more specialized tasks may benefit from specialized silicon.


That will be the only way we end-round the death of Moore's Law. I don't believe there is any magic new physics coming along anytime soon to make the lone CPU's leaps-and-bounds advancement comparable to what it was in the past.

I can sort of see this. I think maybe we will have machines centered around a general purpose CPU, but many more co-processors for specialized tasks, just like how we have dedicated GPU's today.

I just don't believe the general purpose CPU is going anywhere, just that more and more of its responsibilities may be offloaded to specialized silicon dedicated to the task at hand.
 
I think with Moore's Law and our possibility of shrinking dies in a cost effective manner is going to lead to general computing cores losing their effectiveness and eventually end up leading to hybridization of specialized core structures on dies, AMD on GPUs make have started the shift as early as using the async compute added component with NVidia eventually adding compute RT and Tensor cores into the mix, we may be approaching that brick wall where we simply can't increase performance generally but instead need optimizations of more complex software and specialized hardware to unburden current generalized hardware structures to free up additional performance.
 
So, how many specialized processors does it take to make one general purpose processor?
Means and ends and all that jazz...
 
Another grand prediction to add to the dustbin of history!
I think the article was referring specifically to large scale enterprise stuff like Amazon's cloud servers. Instead of a universal Xeon or Epyc cpu, a specialized CPU will be used no different than how consoles used Jaguar cpu's instead of throwing a bulldozer or Intel I3 cpu. It's just more cost effective that way.

If it comes to pass I think it's fantastic. Should lead to a lot more variety in choosing a CPU, and it will force CPU makers to think outside the box a lot more and gain a lot more experience and know how on creating completely different architectures more regularly which should lead to more innovation. Effectively polishing off what AMD has already started to dismantle, the Intel 5% IPC bump rehash model.

It also opens the door for ARM CPU's to make a push to the forefront and provide incentive to develop new instruction sets.

That's probably just me being overly optimistic though...
 
I don't doubt that there will be some of this going on, but I've been down this road and one of the big problems is getting the data to and from the specialized processor. The CPU has to gather up the work to be done and push it over to the function processor, get the results back and put it where it belongs. The bookkeeping can very easily swamp out the gains. I ran into this sort of thing some years ago in connection with a DBMS system and a bespoke encryption chip; it could encrypt way faster than the CPU of the day, but we couldn't feed it fast enough to be worth the cost.

This isn't to pooh-pooh the whole idea, because when it works it's great stuff. But to extrapolate and say that the "era of GP computing is coming to an end" is clickbait nonsense.
 
I think the article was referring specifically to large scale enterprise stuff like Amazon's cloud servers. Instead of a universal Xeon or Epyc cpu, a specialized CPU will be used no different than how consoles used Jaguar cpu's instead of throwing a bulldozer or Intel I3 cpu. It's just more cost effective that way.

If it comes to pass I think it's fantastic. Should lead to a lot more variety in choosing a CPU, and it will force CPU makers to think outside the box a lot more and gain a lot more experience and know how on creating completely different architectures more regularly which should lead to more innovation. Effectively polishing off what AMD has already started to dismantle, the Intel 5% IPC bump rehash model.

It also opens the door for ARM CPU's to make a push to the forefront and provide incentive to develop new instruction sets.

That's probably just me being overly optimistic though...

Ah ok that alters things a touch. Even so we've gone from 2 types of chips doing 99.99% of everything everywhere - cpus and gpus - to 3 types - cpus, gpus, inference engines. All 3 will happily exist beside each other.

You're right though things have been majorly shaken up. Who would've thought a few years ago amazon and Google would be making their own processors? And apple made an arm cpu that was knocking on the door of desktop chips? The future is very non-xeony looking.
 
Meh, the cycle will eventually come full circle once more and general computing devices will become king again.

Basically what is happening is akin to animal evolution in a stagnate ecosystem... they start to specialize and become hyper adapted to their environments in order to compete with other species. Its all great and dandy, until the environment changes and the specialized species go extinct because they can't adapt and compete in a changing environment. Once the animals that can rapidly adapt fully populate, the ecosystem becomes stagnate once more and things start specializing again.

Given how quickly tech has been developing, its safe to say that making specialized processors is surefire way to go bankrupt in a few years. If a specialized process is needed by enough people, a way to miniaturize it is usually found then it gets incorporated all into the same chip, or it gets emulated after the generalized computing device exceeds the computing power of the specialized chip in its given task.

Im just waiting for the eventual rise of the adaptive computing device, a combination of a general computing device that uses machine learning or even AI to increase efficiency with specialized tasks.
 
"Coming to an end" is silly.

As long as there are needs for general purpose computers there will be needs for general purpose CPU's.

That said, a general purpose design will always come with some overhead, and as we get closer and closer to the limits of silicon it is not surprising that more and more specialized tasks may benefit from specialized silicon.

I can sort of see this. I think maybe we will have machines centered around a general purpose CPU, but many more co-processors for specialized tasks, just like how we have dedicated GPU's today.

I just don't believe the general purpose CPU is going anywhere, just that more and more of its responsibilities may be offloaded to specialized silicon dedicated to the task at hand.

Yeah, I think we're a ways off from general purpose CPUs becoming obsolete any time soon. Clearly corporations are creating their task specific processors, however, the general industry as a whole would have to completely adopt a new standard for this to happen. I don't see Microsoft disappearing from Client / Server setups any time soon.

It's kind of funny, because all of this has technically been done in one capacity or another before. If you look at the old Amiga architecture the system had dedicated processors all over their board that accelerated tons of specific functions in the computer. Processor for Audio, general ops, disk management, etc. It was a specialized "multi" processor setup. It's been a while since I've really even thought of Amiga... I was always amazed at how much better their games were than "more powerful PCs" were. You could go a long way with just a couple megahertz when the system had all its specialized components working in harmony. Sorry about the tangent...

What I think we will see more of in the near future is General Purpose CPUs that have specific purpose silicon embedded on the same die or on a chiplet or stacked with it. AMD is already an example of this... They have an ARM core on the Ryzen that handles security after all.

"Zen added support for AMD's Secure Memory Encryption (SME) and AMD's Secure Encrypted Virtualization (SEV). Secure Memory Encryption is real-time memory encryption done per page table entry. Encryption occurs on a hardware AES engine and keys are managed by the onboard "Security" Processor (ARM Cortex-A5) at boot time to encrypt each page, allowing any DDR4 memory (including non-volatile varieties) to be encrypted. AMD SME also makes the contents of the memory more resistant to memory snooping and cold boot attacks"
 
I don't doubt that there will be some of this going on, but I've been down this road and one of the big problems is getting the data to and from the specialized processor. The CPU has to gather up the work to be done and push it over to the function processor, get the results back and put it where it belongs. The bookkeeping can very easily swamp out the gains. I ran into this sort of thing some years ago in connection with a DBMS system and a bespoke encryption chip; it could encrypt way faster than the CPU of the day, but we couldn't feed it fast enough to be worth the cost.

This isn't to pooh-pooh the whole idea, because when it works it's great stuff. But to extrapolate and say that the "era of GP computing is coming to an end" is clickbait nonsense.
In the end the CPU itself would have to become a specialized processor, designed specifically to optimize movement of data to and from memory and it's coprocessors.
 
As long as we don't get TOO specialized.....


1405475112.jpg
 
It's like we can't have toned down titles anymore.
 
Yeah, I think we're a ways off from general purpose CPUs becoming obsolete any time soon. Clearly corporations are creating their task specific processors, however, the general industry as a whole would have to completely adopt a new standard for this to happen. I don't see Microsoft disappearing from Client / Server setups any time soon.

It's kind of funny, because all of this has technically been done in one capacity or another before. If you look at the old Amiga architecture the system had dedicated processors all over their board that accelerated tons of specific functions in the computer. Processor for Audio, general ops, disk management, etc. It was a specialized "multi" processor setup. It's been a while since I've really even thought of Amiga... I was always amazed at how much better their games were than "more powerful PCs" were. You could go a long way with just a couple megahertz when the system had all its specialized components working in harmony. Sorry about the tangent...

What I think we will see more of in the near future is General Purpose CPUs that have specific purpose silicon embedded on the same die or on a chiplet or stacked with it. AMD is already an example of this... They have an ARM core on the Ryzen that handles security after all.

"Zen added support for AMD's Secure Memory Encryption (SME) and AMD's Secure Encrypted Virtualization (SEV). Secure Memory Encryption is real-time memory encryption done per page table entry. Encryption occurs on a hardware AES engine and keys are managed by the onboard "Security" Processor (ARM Cortex-A5) at boot time to encrypt each page, allowing any DDR4 memory (including non-volatile varieties) to be encrypted. AMD SME also makes the contents of the memory more resistant to memory snooping and cold boot attacks"

Yep, the good old Amiga 500 was the first to come to mind when I was thinking about this thread as well. I didn't think mentioning it made for a very good example though, as few people on here have any familiarity with it.
 
When transistors are cheap and operating them is expensive (power/heat), then it inherently makes a *ton* of sense to make a lot of ASICs. You put more pressure on the general purpose processors to be glue/pipe tasks between specialized hardware subsets.

I don't see what is surprising about this at all.
 
I used to have 2-3 amigas and am still trying to get a decently priced complete setup for the 500. You do see them on EBay but the full set-ups can be absurdly priced and getting each bit one by one is also expensive. (Or can be)
 
Last edited:
Yeah, I think we're a ways off from general purpose CPUs becoming obsolete any time soon. Clearly corporations are creating their task specific processors, however, the general industry as a whole would have to completely adopt a new standard for this to happen. I don't see Microsoft disappearing from Client / Server setups any time soon.

It's kind of funny, because all of this has technically been done in one capacity or another before. If you look at the old Amiga architecture the system had dedicated processors all over their board that accelerated tons of specific functions in the computer. Processor for Audio, general ops, disk management, etc. It was a specialized "multi" processor setup. It's been a while since I've really even thought of Amiga... I was always amazed at how much better their games were than "more powerful PCs" were. You could go a long way with just a couple megahertz when the system had all its specialized components working in harmony. Sorry about the tangent...

What I think we will see more of in the near future is General Purpose CPUs that have specific purpose silicon embedded on the same die or on a chiplet or stacked with it. AMD is already an example of this... They have an ARM core on the Ryzen that handles security after all.

"Zen added support for AMD's Secure Memory Encryption (SME) and AMD's Secure Encrypted Virtualization (SEV). Secure Memory Encryption is real-time memory encryption done per page table entry. Encryption occurs on a hardware AES engine and keys are managed by the onboard "Security" Processor (ARM Cortex-A5) at boot time to encrypt each page, allowing any DDR4 memory (including non-volatile varieties) to be encrypted. AMD SME also makes the contents of the memory more resistant to memory snooping and cold boot attacks"
I agree, its just a new time period in the routine evolution.
Its entirely likely all these new tasks will be absorbed by " general purpose " cpus at some point in the not so far future, just like you mentioned, embedded circuits, even extra but different cpus inside... Nothing surprising really, things might look right now like they exclude each other, but odds are it won't remain that way forever... Probably not even for long.
 
I used to have 2-3 amigas and am still trying to get a decently priced complete setup for the 500. You do see them on EBay but the full set-ups can be absurdly priced and getting each bit one by one is also expensice. (Or can be)
There's an Amiga movement out there that is recreating the platform from scratch, I forget the group that's doing it but it's pretty damn interesting.

There is an official page somewhere but this incarnation allows you to drop in discrete accelerators to run the various Amiga platforms. Pretty cool actually .

https://www.google.com/url?sa=i&sou...aw1z3g4YhEabO63xtNNXwLjk&ust=1549664651539377
 
Last edited:
I don't think so.

There will be CPUs, GPUs, and industrial-scale GPU-like processors such as Google's TPUs.

The problem is that there are really only two ways to make processing fast. You can get high performance by either sacrificing throughput to minimize latency like a CPU does, or sacrificing latency to maximize throughput like a GPU does.

There's some architectural wiggle room where you can drop certain features, run at half precision, etc. to save on costs and heat - but unless you are Google, you won't be able to afford the same manufacturing processes Nvidia, Intel, and AMD do, and larger chips generate more heat. So you'll just buy a general-purpose CPU or GPU instead.
 
This is great- companies will now have a bunch of new things to sell us for our PCs...
 
I imagine if this trend continues we might see Intel integrate a specialized video processor on-die in their desktop chips. Predict many gamers won't be happy about paying full price for half full silicon tho.
 
We already have APU's - maybe AMD should start making threadripper APU's with RX580 caliber GPU's on them. If we had DDR5 that might make even more sense.
 
At what point will code and programming be instruction set agnostic?

All the asic designs and specialized silicone needs to be told what to do, no?

I could see some sort of ai/ml process optimizing c code (as an example) for specific CPUs one day. Lots of games (and important RL stuff) would in fact benefit from heavy optimization.
 
At what point will code and programming be instruction set agnostic?

All the asic designs and specialized silicone needs to be told what to do, no?

I could see some sort of ai/ml process optimizing c code (as an example) for specific CPUs one day. Lots of games (and important RL stuff) would in fact benefit from heavy optimization.
I'm pretty sure there are already programs that do this, though maybe not very well.
 
The Commodore Amiga started this, back in the day. It's nothing new for anyone who owned one of those great machines.
 
I use a general purpose CPU to run 18 x 400MHz DSP Chips From Analog Devices.
Only in the last 10 years have I started using the CPU for similar audio chores.
I need 50% of 4 Cores @ 4GHz to do what 15% of one ADSP-21369 Chip does.

Nice having both to share workloads.
If I didn’t have to use Micro$oft, and had a custom operating system, probably wouldn’t need Intel or AMD.

I’ve seen another DSP/CPU combo from Merging Technologies that steals a single core from an i7 by somehow bypassing Windows.
It’s incredible audio processing @ 192k used for 17.1 Surround or IMAX.
Maybe a reduced instruction set or refine the permission process of an operating system might be worthier than these 32core behemoths.
 
The Commodore Amiga started this, back in the day. It's nothing new for anyone who owned one of those great machines.

Amiga was cool and made great use of offloading, but it hardly started it. Heck, the Atari 800 (many of the same engineers, I believe) had the CTIA, POKEY, ANTIC. I made games for it back in the day - amazing what you could do with a handful of instructions fired off to those little buddies.
 
The Next Platform believes that the era of general purpose computing may be coming to an end.

The meteoric rise of graphics processors is perhaps one of the earliest and most visible examples on the trend.

Graphics processors are just the opposite example, which have slowly evolved from being only-graphics accelerators GPUs to being general purpose accelerators (or GPGPUs) by adding more and more non-fixed hardware to the pipeline.
 
It's like x87/FPU all over again and I would bet we'll see the same end result again.
 
As stated, if you're building a computer for an unknown purpose, it needs to be able to do any task imaginable. You'll always need something that is fully programmable to accomplish that. Even CPUs right now dedicate a ton of their silicon for specific instructions, so although it's general purpose, it has a lot of specialized bits tacked on where it makes sense. ASICs are definitely the temporarily solution to the chicken / egg problem, where you don't want to add support to a CPU for a task that has no code that can benefit from it, and no code was written because there is no hardware support for it. Once it becomes popular support for it will just be rolled into a CPU, since there will be a bit enough market to make it worthwhile. Case in point is a lot of the AI engines that are on dedicated silicon. If those become widely popular to the point where half the market products contain one, it makes sense to just integrate it into the CPU.

The flip side however, is if there is a market for it, a specialized product will exist. When you start talking about super computers, the CPU will likely play less and less of a role in that space. That's already happened with GPUs, where most of the TFLOPs of a super computer comes from a GPU. If you're building a computer that needs to last for 5 - 6 years, and it's only ever going to be used for 1 exact purpose, then using a general purpose processor doesn't make sense. In that space I think you'll continue to see the decline in importance of a CPU, because it will be cost effective to build specialized hardware for those cases.
 
The flip side however, is if there is a market for it, a specialized product will exist. When you start talking about super computers, the CPU will likely play less and less of a role in that space. That's already happened with GPUs, where most of the TFLOPs of a super computer comes from a GPU. If you're building a computer that needs to last for 5 - 6 years, and it's only ever going to be used for 1 exact purpose, then using a general purpose processor doesn't make sense. In that space I think you'll continue to see the decline in importance of a CPU, because it will be cost effective to build specialized hardware for those cases.

And the post-K supercomputer, scheduled to begin operation in 2021, just kill that idea. The post-K is CPU-based. GPUs aren't needed for anything.

Also, special purpose hardware is subject to physical and other limits. People start to talk about an accelerator wall, a new concept to be added to the classic frequency, power, and ILP walls.
 
Back
Top