Google's WikiSkill Framework Boosts AI Performance with Persistent Knowledge

Researchers at Google have developed a framework called WikiSkill that enables AI agents to learn from their past mistakes and improve their performance over time. Unlike traditional learning methods, WikiSkill doesn't rely on continuous learning but instead uses a persistent knowledge base that grows with each iteration.

The framework organizes an AI agent's workspace into three layers: the Raw Layer, which stores execution traces; the Wiki Layer, where raw data is distilled into structured insights; and the Skill Layer, which holds active procedural instructions. A 'Wiki Maintainer' analyzes execution traces to identify failure patterns and successful strategies, while a 'Skill Proposer' uses this information to suggest targeted skill changes.

WikiSkill was tested on five benchmarks covering various tasks such as math reasoning, web search, spreadsheet manipulation, and document question-answering. The results showed that WikiSkill consistently outperformed other skill evolution methods and the no-skill baseline, with larger models benefiting more from evolved skills. However, smaller models using WikiSkill were able to match the performance of larger models without the framework.

One notable finding was that skills developed by one model often transferred to another, sometimes even working better than skills built by the receiving model itself. The researchers recommend checking transferability on a case-by-case basis.