Forbes January 19, 2025
Seven years and seven months ago, Google changed the world with the Transformer architecture, which lies at the heart of generative AI applications like OpenAI’s ChatGPT.
Now Google has unveiled a new architecture called Titans, a direct evolution of the Transformer that takes us a step closer to AI that can think like humans.
The Transformer architecture has no long-term memory, limiting its ability to retain and use information over extended periods – an essential part of human thought. Titans introduces a neural long-term memory, along with short-term memory and a surprise-based learning system—tools our own minds use to remember unexpected or pivotal events.
In simple terms, Transformers have a sort of “spotlight” (called the attention mechanism) that looks at...
Today's Sponsors
Today's Sponsor