Nvidia Unveils Custom High-Bandwidth Memory Solution for AI Accelerators
Nvidia has announced a new custom high-bandwidth memory solution called NVHBM, designed for its partners in the NVLink Fusion program. This custom implementation promises up to 30% higher bandwidth and 15% lower power consumption compared to traditional HBM4e.
NVHBM moves the memory controller into the base die of the HBM stack, freeing up precious package real estate that can be used for additional compute die area. According to Nvidia, this approach allows for up to 30% more compute on the primary silicon die and simplifies interposer routing.
Nvidia's NVHBM also provides power savings versus off-the-shelf HBM4e stacks. The company claims that NVHBM uses 15% less power than commodity HBM4e, which can be banked for performance-per-watt improvements or reallocated to support larger numbers of accelerators within the same fixed power envelope.
Nvidia has announced its first partner on NVHBM as Amazon's Annapurna Labs, which will use this technology in its next-generation Trainium 4 AI chips. The company is expected to follow up with more details on how NVHBM will be integrated into future products.