Google Research Titans: Learning to Memorize at Test Time

​Our guest Ali Behrouz from Google Research will introduce to the BuzzRobot community Titans, a new neural architecture that combines recurrent models with attention mechanisms.

​Neural memory modules complement attention by storing long-term context while attention serves as short-term memory with precise dependency modeling.

Titans architectures integrate these approaches, allowing for parallelizable training and context windows exceeding 2M tokens.

Experiments across language modeling, reasoning, and time series tasks demonstrate superior performance compared to baseline models.

Read the paper
Join BuzzRobot Slack to connect with the community