Next-Gen HBM Targets Embedded GPU Cores

[erek]

"Kim Joung-ho, a professor in the School of Electrical Engineering at KAIST, said, "The speed of technological transition where the boundary between memory and system semiconductors collapses for AI advancement will accelerate," and added, "Domestic companies must expand their ecosystem beyond memory into the logic sector to preempt the next-generation HBM market." AMD's recently unveiled Instinct MI430X accelerator built on the next-generation AMD CDNA architecture supports 432 GB of HBM4 memory and 19.6 TB/s of memory bandwidth. NVIDIA's "Vera Rubin" Superchip takes a different approach. Each Rubin GPU integrates two reticle-sized compute chiplets paired with eight HBM4 stacks, delivering around 288 GB of HBM4 per GPU and roughly 576 GB of HBM4 across the full Superchip. From those potential changes companies that have strong packaging and logic capabilities will benefit, while pure memory vendors may need to expand into system-level semiconductor technologies to stay competitive."

Source: https://www.techpowerup.com/343313/next-gen-hbm-targets-embedded-gpu-cores

 

[LukeTbk]

that could release pressure on the regular dram production, has HBM bottleneck will be more at the complex packaging needed at TSMC/intel/samsung, not how much ram hynix/micron/samsung can make.

 

[Baenwort]

Not much help as HBM production has been entirely bought out for the last 3 years for AI as the extra bandwidth is important and they'd rather use HBM for training.

DDR is getting hit now as that is used more for the actual running of the AI as you can scale the memory even higher quantities and cheaper. More of the AI companies as trying to switch from a primary training focus to rolling out the scale to serve posts of clients.

However, training isn't stopping so I don't think HBM supplies will get freer.

 

[LukeTbk]

Not much help as HBM production has been entirely bought out for the last 3 years for AI as the extra bandwidth is important and they'd rather use HBM for training.

that exactly why changing the bottleneck to cowos complex packaging capacity instead of how much dram production can you shift into HBM can help.

DDR is getting hit now as that is used more for the actual running of the AI as you can scale the memory even higher quantities and cheaper.

And because HBM use the same production capacity a lot (they share the dies/wafer part), so they are shifting dram production into HBM one more and more lines. The reason they still "wasting" some wafer of dram is the complex packaging/3d stacking/ bottleneck of HBM (they cannot cut more dram to make more HBM, wafer is not the constraint) and the more complicated and slow it become, the more there is free wafer for dram.