Google's Autofinetune Automates Large Language Model Fine-Tuning

The Google team has developed an AI agent that can automate the process of fine-tuning large language models (LLMs) using their full AI stack. This autonomous research loop, called autofinetune, uses Google's Tunix, Gemma, and Cloud TPUs to optimize LLMs for specific tasks. The agent is able to modify hyperparameters, run experiments, and commit improvements to a Git repository, all without human intervention.

The team demonstrated the effectiveness of autofinetune on two case studies: Supervised Fine-Tuning (SFT) on FunctionGemma and Reinforcement Learning (GRPO) on Gemma for Math Reasoning. In both cases, the agent was able to improve model performance by adjusting hyperparameters such as LoRA rank/alpha, optimizer, learning rate, and rollout temperature.

The developers hope that autofinetune will inspire others to automate their LLM fine-tuning workflows using Tunix on TPUs. They have made the code and sample runs available in the autofinetune GitHub repository for anyone to explore and build upon.