Table of Contents
The Google research that created the Transformer architecture is responsible for what we know as AI today. In Google's Titans paper they describe a new, improved architecture that adds long term memory in a really cool way. The Titans architecture outperforms the Transformer architecture and I expect it will improve Google's capabilities, whether it is for LLM based chat, AI agents, or Search.
Here is the paper, published December 31, 2024: Titans: Learning to Memorize at Test Time.
The paper talks about “attention.” Attention is what made it possible via the Transformer architecture to attend to large amounts of information by understanding what is important. The Titans paper is a new neural long term memory module that learns to memorize historical context. It's not just memorizing everything verbatim. Rather, it learns which parts are important to place in its memory, both short term and long. It can then attend to the current conversation while utilizing historical information.
Titans are a new family of architectures. They offer three variants for incorporating memory into the architecture. Here is the abstract of this paper:
A brief summary of what's important to know about this paper
- Transformer attention architectures learn to store key-value associations and retrieve them by computing pairwise similarity between queries (i.e. search signals) and keys (i.e. contexts).
- The traditional transformer architecture is limited to the information in the context window.
- The paper describes a neural memory that learns how to memorize and store data into its parameters. It is inspired by the human long-term memory system.
- An event that violates the expectations (being surprising) is more memorable.
- There is also a forgetting mechanism to allow the memory to forget information not needed anymore as memory is limited.
- Titans have three components:
- Core - short term memory and responsible for the main flow of processing data
- Long term memory
- Persistent Memory
- Titans outperform Transformers and can scale to larger than 2 million context window size.
Is Gemini using the Titans architecture?
I had a good conversation with Gemini 2.0 Flash Experimental to see if my summary of this paper was correct. Then I asked whether it was using the Titans architecture. It said it was not as of yet.
Now, we don't know for certain whether this is true, as it is possible that Gemini is using the Titans architecture and is simply not aware of it yet.
This was the question I really wanted to know though...
How does the Titans architecture change how Search works?
We know that Search uses a complex mixture of traditional algorithms and machine learning systems. In the Pandu Nayak testimony from the DOJ vs Google trial we learned that deep learning systems like RankBrain, DeepRank and RankEmbed BERT are vitally important to Google's systems which are continually learning how to present searchers with results they are likely to find helpful.
I asked Gemini how the Titans architecture might change Google's Search systems:
I really like this answer. Especially, “Think about follow-up questions users might have and address those proactively.” This is in line with my theory I discuss in my course on optimizing for both user intent and related “micro-intents.”
Essentially our goal is to create content that aligns with what the searcher is trying to find.
Really, if you're focusing on your users and creating content that they are likely to find helpful, then you're already ahead of the game.
There are several points in Google's documentation on creating helpful content that we can consider here. I expect that advances in AI such as the Titans architecture will improve Google's capabilities of determining which pages excel on these points:
- Does the content provide original information, reporting, research, or analysis?
- Does the content provide a substantial, complete, or comprehensive description of the topic?
- Does the content provide insightful analysis or interesting information that is beyond the obvious?
- If the content draws on other sources, does it avoid simply copying or rewriting those sources, and instead provide substantial additional value and originality?
- Does the main heading or page title provide a descriptive, helpful summary of the content?
- Does the content provide substantial value when compared to other pages in search results?
Titans are a big step in improving AI's capabilities and performance
The Titans architecture represents a significant leap forward in AI's ability to understand and process information in a more human-like way. It's not just memorizing everything, but instead, choosing what details are important to memorize and forgetting those that are likely no longer needed, acting more and more like a human brain!
This article started off as an entry in my paid newsletter, Marie's Notes in which I write in great detail about new AI news and anything I can find that can give us an advantage in Search.