Gemma 3 12B Outperforms Larger Model in High-Concurrency Generation Tasks

Google Cloud recently published benchmark results comparing Gemma 3 12B and Gemma 3 27B on TPU v6e infrastructure. The tests found that the smaller 12B model continued to scale more strongly at higher concurrency levels in generation tasks, while the larger 27B model hit a throughput ceiling.

For generation workloads, the two models tracked each other closely before separating sharply under heavier load. At 128 users, the 12B model climbed to 8.19x while the 27B model rose only slightly to 4.12x, indicating a plateau. The larger model appeared to hit memory or compute limits earlier under those conditions.

In classification tasks, both models showed broadly similar scaling at the highest concurrency tested. However, in prefill-heavy workloads, where the model processes large prompts and produces short answers, the difference in parameter count does not translate into a major throughput penalty at scale.