Google AIs Learn from Past Searches to Improve Efficiency

Google researchers have developed a system called Dream-RSI that uses an AI agent's search history to improve future exploration and reduce the cost of automated discovery.

The framework works by recording each search as a tree, which contains the alternatives explored, the order in which they were explored, the computational cost, and the results. The system then replays this recorded history to test alternative search strategies without running the original coding agent and evaluator again.

Dream-RSI was tested on eight algorithm, mathematics, and GPU programming tasks, where it matched or improved on competing approaches while using fewer AI-agent calls in several tests. In one task, Dream-RSI produced a solver with an average runtime of 2,931 milliseconds, compared to 3,587 milliseconds for a fixed exploration policy.

The researchers found that Dream-RSI can improve the performance of an AI system without retraining its underlying model by changing how it allocates time and computing resources during a search. This approach has some limitations, including the inability to determine what would have happened along branches that were never explored.