Google DeepMind’s Dream-RSI helps AI agents improve by replaying past searches
Researchers from Google and Google DeepMind have built a method called Dream-RSI that helps AI agents get better at hard search tasks by replaying past attempts instead of running every new idea from scratch. The approach reuses data from a completed search to test alternative strategies, then applies the best one to the next live run.
The team tested Dream-RSI on three problem areas: competitive programming, math optimization, and GPU kernel writing. Results showed faster solutions and fewer attempts compared with a fixed-strategy baseline.
What problem Dream-RSI solves
Self-improving AI agents work by proposing a solution, checking the result, learning from it, and trying again. Over many attempts, they gradually close in on a good answer. For complex tasks, the space of possible approaches grows enormous. The agent has to decide which paths to pursue, which to try in parallel, and which to abandon. That exploration step often decides whether the search pays off or burns compute on dead ends.
Existing approaches handle exploration in two ways. A fixed strategy can’t learn from experience, so the agent can hit the same dead ends over and over. Adapting the strategy mid-search avoids that rigidity, but testing a new strategy usually means rerunning the whole expensive process to see if it helps. Dream-RSI attacks that cost.
How Dream-RSI reuses past search data
The method records every attempt and its result as the agent searches, building a stored search tree. New strategies can then be run against that stored data instead of against live calls to the model or evaluator. The researchers compare it to exploring an unfamiliar area: on the first visit you hit dead ends, but once you have a mental map you can plan a better route without walking every path again.
Because the results already exist in the search tree, thousands of alternative strategies can be tested without generating or evaluating fresh solutions. The team calls this replay step “dreaming.” The agent plays through thousands of variations, picks the best, and applies it to the next live search. Only the search strategy changes. The underlying model stays the same.
The system alternates between live searches and replays of stored histories. After each live run, the agent uses the recorded results to test improved strategies, then carries the winner into the next round.
Results across coding, math, and GPU tasks
The researchers tested Dream-RSI with Gemini 3.1 Pro and Gemini 3.7 Flash on eight tasks across the three areas. Each comparison used a baseline with the same starting conditions but a fixed search strategy.
One task asked the system to write the fastest possible program for a statistical calculation commonly used in genomics and finance. Dream-RSI’s program ran faster than the established libraries sklearn and glmnet on all six test datasets. With Gemini 3.1 Pro, average runtime fell from 3,587 to 2,931 milliseconds, while the number of attempts dropped from 550 to 317. Against a competing system called SimpleTES, which needed 51,200 runs, Dream-RSI needed only 317 attempts.
The same pattern held for math optimization and GPU kernel writing. On two GPU tasks, Dream-RSI matched performance while cutting the number of runs by a factor of up to 2.43. On two others, it delivered up to 2.09 times the performance within the same budget.
Why explicit instructions can hurt
In a follow-up analysis, the team tried a different way to use search histories: condensing them into written instructions telling the agent where to search. On one GPU task, the instructed version performed worse than the version without instructions. The researchers suggest that overly specific directions can narrow the search space too much, keeping the agent from exploring a broader range of approaches.
The same analysis showed how the learned strategy adjusted its effort over time. As performance improved, it initially reduced the number of attempts. When progress stalled, it increased search effort again, and that increase coincided with further gains.
Where Dream-RSI fits in the wider research
Recursive self-improvement has drawn growing attention. Google DeepMind introduced AlphaEvolve in 2025, where Gemini Flash generates code proposals, Gemini Pro analyzes them, and an evolutionary algorithm selects the best versions. Dream-RSI works one level above that process by optimizing the search strategy itself.
AutoTTS takes a related approach, using a coding agent to search for algorithms in a simulated environment that decide when a language model should start, expand, or abandon reasoning paths. Google Research presented WikiSkill, which records failures and successes in a wiki and turns them into reusable instructions for the agent. Dream-RSI’s follow-up analysis suggests that explicit instructions like these can restrict exploration on open-ended search tasks. Meta goes further with Hyperagents, which lets agents rewrite the mechanism controlling how they improve.
The Dream-RSI team has shared code and more details on GitHub.
FAQ
What is Dream-RSI?
Dream-RSI is a method from Google and Google DeepMind that lets AI agents test new search strategies against stored results from past searches, then apply the best strategy to the next live run.
How does Dream-RSI cut compute costs?
The system records every attempt and result as the agent searches. New strategies are tested against that stored search tree instead of running the model or evaluator again, so thousands of alternatives can be checked without repeating expensive computations.
What did Dream-RSI achieve in tests?
On a competitive programming task, Dream-RSI with Gemini 3.1 Pro cut average runtime from 3,587 to 2,931 milliseconds while dropping attempts from 550 to 317. On GPU tasks, it matched performance with up to 2.43x fewer runs or achieved up to 2.09x higher performance within the same budget.
Related coverage
- Weekly Search News Recap: AI Local Results Slop, Google AI Home Page Buttons, and Meta Building a Search Engine – LocalSEOBot
- Google Search Console Links Report Not Updated In A Month – LocalSEOBot
- Inside Google Maps: 72 Ranking Signals and the Architecture Behind Local Search – LocalSEOBot
SEOScanPro
SEOScanPro has the Search Console analytics view, runs a full technical audit of a site and shows the measured result behind every check. Open the Search Console analytics view.