AI Is Insatiable
While browsing our website a few weeks ago, I stumbled upon “ How and When the Memory Chip Shortage Will End ” by Senior Editor Samuel K. Moore. His analysis focuses on the current DRAM shortage caused by AI hyperscalers’ ravenous appetite for memory, a major constraint on the speed at which large language models run. Moore provides a clear explanation of the shortage, particularly for high bandwidth memory (HBM). As we and the rest of the tech media have documented, AI is a resource hog. AI electricity consumption could account for up to 12 percent of all U.S. power by 2028. Generative AI queries consumed 15 terawatt-hours in 2025 and are projected to consume 347 TWh by 2030. Water consumption for cooling AI data centers is predicted to double or even quadruple by 2028 compared to 2023. But Moore’s reporting shines a light on an obscure corner of the AI boom. HBM is a particular type of memory product tailor-made to serve AI processors. Makers of those processors, notably Nvidia and AMD, are demanding more and more memory for each of their chips, driven by the needs and wants of firms like Google, Microsoft, OpenAI, and Anthropic, which are underwriting an unprecedented buildout of data centers. And some of these facilities are colossal: You can read about the engineering challenges of building Meta’s mind-boggling 5-gigawatt Hyperion site in Louisiana, in “ What Will It Take to Build the World’s Largest Data Center? ” We realized that Moore’s HBM story
As the world continues to grapple with the rapid advancements in artificial intelligence, one of the most pressing issues emerging is the insatiable appetite of AI for memory. Senior Editor Samuel K. Moore's analysis, "How and When the Memory Chip Shortage Will End," sheds light on this critical constraint, particularly focusing on the current shortage of DRAM, a type of memory crucial for the speed of large language models.
Moore's report highlights the role of high bandwidth memory (HBM), a specialized memory product designed to serve AI processors. Companies like Nvidia and AMD, which manufacture these processors, are increasingly demanding more memory for each chip. This surge in demand is driven by the needs of tech giants such as Google, Microsoft, OpenAI, and Anthropic, which are spearheading an unprecedented expansion of data centers worldwide.
The scale of these data centers is staggering. For instance, Meta's Hyperion site in Louisiana, a 5-gigawatt facility, presents significant engineering challenges. The article "What Will It Take to Build the World’s Largest Data Center?" delves into these complexities, underscoring the monumental scale of the infrastructure required to support AI's growing needs.
The AI boom is not only consuming vast amounts of electricity and water but also exerting immense pressure on the global supply of memory chips. Projections indicate that AI electricity consumption could reach up to 12 percent of all U.S. power by 2028. Generative AI queries alone consumed 15 terawatt-hours in 2025, with estimates predicting a surge to 347 TWh by 2030. Similarly, water consumption for cooling AI data centers is projected to double or even quadruple by 2028 compared to 2023 levels.
This insatiable demand for memory is not limited to high-end AI applications. The memory-chip shortage is also driving up the price of low-cost computers, such as the Raspberry Pi, as highlighted in a recent article by Contributing Editor Matthew S. Smith. The resulting price pressure on consumer electronics is a hidden consequence of AI's rapid expansion.
The critical question now is: When will this memory shortage end? While the tech industry continues to innovate and invest in new solutions, the demand for memory from AI hyperscalers remains relentless. As the world becomes increasingly reliant on AI for various applications, the need for sustainable and efficient memory solutions becomes paramount. The challenge lies in balancing the exponential growth of AI with the constraints of current technology and resources, ensuring that the future of AI development is both viable and sustainable.
