Breaking Through The Generative AI Memory Wall

Forbes November 21, 2024
Steven Woo

By Dr. Steven Woo, fellow and distinguished inventor at Rambus.

The term “memory wall” was first coined in the 1990s to describe memory bandwidth bottlenecks that were holding back CPU performance. The semiconductor industry helped address this memory wall through DRAM architecture innovations, rapidly increasing memory bus widths and data rates, and better process technologies that made faster memory and interfaces manufacturable. The torrid pace of innovation in AI processing has given rise to a similar bottleneck that has created an “AI memory wall.”

The challenge for modern systems is that dramatically wider memory buses, much faster data rates and better process technologies—mainstays of the industry that helped break down the previous memory wall—are no longer as easily achieved. The...

Today's Sponsors

Today's Sponsor