HBM Issues In AI Systems

[erek]

You'd think there'd only be great things with HBM performance going forward! You'd be wrong?

"How does this create a new problem? The static power draw for the DRAM core will remain essentially the same and will increase with capacity. HBM is more power-efficient in terms of bits per watt, and enables higher transfer rates. Total power and energy may increase significantly if the memory is being used at full capacity.

Then we look at the impact on compute. “The major challenge with all most designs is that they can put in lots of compute, and they can add more compute, more parallelism, but it is actually a compute and memory problem,” says Ron Lowman, strategic marketing manager at Synopsys. “Systems have been constrained by bandwidth and fighting that bottleneck to memory.”

So what happens when that limitation goes away? “HBM provides an unprecedented amount of bandwidth between the CPU and memory,” says Cadence’s Greenberg. “HBM2E provides 2.4Tbit/s of bandwidth, with further specification enhancements on the horizon. By using an interposer-based technology, the energy-per-bit is kept low, but power — the product of energy-per-bit and number of bits transferred per second — may be relatively high at terabits-per-second transmission rates.”

This starts to cause some new problems. “One of these includes accounting for the power noise impact from HBM I/Os,” says Calvin Chow, senior area technical manager at ANSYS. “Even though the power per pin is lower, there are many more I/Os firing in parallel, resulting in significant increase in current consumption. Though the signal traces are shorter, there is still a noise concern due to the simultaneous switching of a large number of I/Os.”"

https://semiengineering.com/hbm-issues-in-ai-systems/

 

[Armenius]

Was trying to think of something witty to say, but this is all I got:

 

[clockdogg]

Was trying to think of something witty to say, but this is all I got:

Me too. What about: "The more (current) you spend, the more you save!"

 

[defaultluser]

I could have told yoiu that. You're exchanging one thing (high-seeed mostly serial communications) for low speeed mostly parallel. You''re going to gt different kinds of interferece, but both are just as tricky.

 

[erek]

I could have told yoiu that. You're exchanging one thing (high-seeed mostly serial communications) for low speeed mostly parallel. You''re going to gt different kinds of interferece, but both are just as tricky.

Thanks bud, what about a hybrid, half serial, half parallel communications mesh or ring bus

 

[defaultluser]

I mean, we invented rs232 so that we didn't have to think too hard about interference or max range. But that cost you tons of power, and kHz max speed

W e like our busss to be LVDS and FAST, so of course interference becomes important.

 

[erek]

I mean, we invented rs232 so that we didn't have to think too hard about interference or max range. But that cost you tons of power, and kHz max speed

W e like our busss to be LVDS and FAST, so of course interference becomes important.

https://ieeexplore.ieee.org/abstract/document/7295984

 

[AceGoober]

Of course there is going to be higher power draw and a greater level of noise. Basic electronics teaches that in its simplest form.