Google's DeepMind Converts Existing Model into Text Diffusion Breakthrough

Google's DeepMind has made significant advancements in text diffusion models by converting an existing model, Gemma-4-26B-A4B, into a diffusion model. This approach proves that it's not necessary to train from scratch to build a text diffusion model.

The team used less than ten percent of the original training token budget to convert the Gemma 4 model into DiffusionGemma. The results are impressive: DiffusionGemma delivers several times the output speed of the Gemma 4 models and previous diffusion models while maintaining comparable accuracy.

DiffusionGemma uses a combined approach of reinforcement learning and sampler distillation, which Google calls SD·RL. This process boosts answer quality and lets the model get by with fewer compute steps. The model can also correct itself during later denoising steps, unlike standard language models that have to commit to the first digit of an answer before they've worked through the reasoning.

DiffusionGemma was fine-tuned for tasks such as Sudoku solving and structured outputs like JSON or code repairs. It solves close to 85 percent of puzzles correctly after minimal fine-tuning, while the base model fails at the task entirely.