Towering Memory: HBM and Verticality

The AI arms race is heating up, with AMD and NVIDIA releasing cutting-edge products and hyper-scalers like Google and Meta developing custom accelerators. This expansion strains the supply chain, driving investments in advanced packaging. A critical focus is on high bandwidth memory (HBM), essential for maximizing data center performance.

JEDEC's recent update increased the allowable package height for HBM, enabling stack heights up to 16 dies. Contemporary HBM packages use a 3D structure with multiple DRAM dies stacked on a controller die, connected through silicon vias (TSVs) and thermal compression micro-bumps (µ-bumps). The top-most DRAM die is thicker to account for height variations and to provide a planar surface for further packaging.

To achieve higher stacks, industry discussions are shifting towards hybrid bonding, which offers increased interconnect density and reduced bond height. TSMC's system-on-integrated chip (SoIC) technology, with significantly reduced bond heights, could be key. Thinning DRAM dies could allow for a 16-die stack within the new height limit.

This evolution in HBM technology is crucial for advancing AI and data processing, enabling more efficient stacking of DRAM dies to meet future data center demands.

For more insights and exclusive content, including detailed infographics on HBM Package Structure, visit the TechInsights platform.