NVIDIA’s Vera Team Says Chiplet Designs Sacrificed Single-Threaded Performance For Core Counts, And Agentic AI Is Now Forcing A Reversal

“NVIDIA Vera's monolithic die​

Question: Why did NVIDIA choose a monolithic die for Vera instead of the chiplet architecture used by competitors?

Answer: Unlike legacy x86 manufacturers who must design complex, flexible architectures to serve both everyday consumers (client PCs) and high-end data centers, NVIDIA entered the standalone CPU space completely unencumbered by historical segments. This unique positioning gave them the luxury of designing Vera exclusively for a single target: the ultra-high-performance AI data center.

"...truly Vera is the first one that we're positioning after, is that we are unencumbered by segments past... AMD and Intel, they have to serve not only data centre, but they also have to serve client. They have different segments that have to serve with a single architecture... We really had the luxury of designing for the data centre market."

Click to expand...

To satisfy the intense memory-fetching loops of AI agents and data analytics, NVIDIA deliberately rejected the multi-chiplet trend of its competitors in favor of a single monolithic die. Basically, by keeping all cores on one continuous piece of silicon, NVIDIA was able to completely eliminate the internal performance "tax" that occurs when data is forced to cross-communicate boundaries between separate chiplets.

"...having a monolithic die, where the core in the top left corner of the chip can talk to the core at the bottom right corner of the chip without crossing I/O boundaries and incurring attacks, that was a valuable design point for us... it was important to our design point for like maximising data movements for not only agentic AI, but also just straight up data processing and data analytics."

Click to expand...

This monolithic engineering translates into a massive performance advantage for collaborative computing. The Vera CPU achieves an unprecedented 3.4 terabytes per second of bisection bandwidth, allowing individual processor cores to collaborate and share contextual information 3x faster than the leading x86 chips.

"...we can deliver 3.4 terabytes per second of bisection bandwidth, which is 3x higher than the leading x86 chip. So that means like, you know, sometimes cores have to collaborate on shared context. And so they don't, you know, it's three times faster for them to collaborate on shared context than it is for an x86 chip."

Click to expand...

While the monolithic approach was mathematically required to achieve Vera's specific performance targets, NVIDIA maintains a highly pragmatic, agnostic stance on chiplet technology for the future. The company will pivot to chiplet designs if a future market opportunity dictates it, choosing engineering utility over rigid design philosophies.

"...I think we're just going to be very balanced about like, what are the design points we want to meet for what market opportunity?... But I would say it's sort of like, we're not, I guess I wouldn't say we don't have a religion around it. It's more like what is right for what we're trying to achieve."

Click to expand...

TL; DR: Because NVIDIA is unencumbered by legacy segment demands [like balancing data center and client needs], it has the luxury of designing its chips strictly for data center and agentic AI. A monolithic die maximizes data movement by allowing cores across the chip to communicate without crossing I/O boundaries, yielding 3.4 terabytes per second of bisection bandwidth - 3x higher than leading x86 chips. Even so, NVIDIA remains open to chiplet architectures in the future if market demands necessitate a different architecture.“
Source: https://wccftech.com/exclusive-nvid...nts-and-agentic-ai-is-now-forcing-a-reversal/

Pricing power via vertical integration of the CPU is also great for this, Apple is able to use expensive monolithic die over a large range of product has well, they are better in everyway if they fit the reticle and can pay to make them.

Not sure Nvidia could go around making 850mm giant monolithic CPU on frontier node that made commercial sense if they were not to be placed in extremelly expensive racks. They would have to take the intel-amd chiplet road and its compromise to make it work.