The constraint on AI growth isn't raw computing power — it's memory. This episode breaks down the "memory wall," why generating AI output is fundamentally a memory-bandwidth problem, and what a global shortage of a specialized chip component means for the pace of AI deployment.

AI-generated (NotebookLM) audio overview. Source: HexLocal in-house research — Research - Memory Chip Bottleneck (HBM) - 2026-06-21 (Dr. Priya Nair). Primary external sources include Goldman Sachs and Counterpoint Research analyst forecasts, and an arXiv paper on AI and the memory wall.

- The "memory wall" explained: generating each token of AI output requires streaming the full model weights out of memory — no amount of extra compute can fix that

- HBM (high-bandwidth memory) stacks DRAM dies vertically to create the wide data highway inference actually needs, making it far harder to manufacture than standard memory

- CoWoS advanced packaging — integrating GPU and HBM onto a single substrate — is the deeper, less-discussed constraint in the supply chain

- HBM is sold out across SK Hynix, Micron, and Samsung through 2026; the acute shortage is forecast to run through 2027–2028, with some estimates extending into the early 2030s

- NVIDIA's Vera Rubin platform is the named hardware response to the memory constraint, with vendor-announced specs of 22 TB/s bandwidth and 10x projected throughput gains

- The practical implication: throwing more GPUs at inference doesn't solve the latency problem — and may make it worse