Skip to content
Back to the Guavy Wire
Stocks

Google explores cost-efficient routing for autonomous coding agents

Instruments
GOOGL
Share

As per-token API prices continue to drop, the real cost driver for autonomous coding agents is shifting to cost per successful task. This metric accounts for total spending across all iterations, tool calls, retries, and thinking passes, divided by the number of tasks that pass verification. The inefficiency arises when agents use high-thinking reasoning configurations for routine decisions that lighter models could handle just as accurately.

To address this, Google's blog post explores using TypeSafe's Jev classifier as a lightweight routing layer to intercept routine decisions before they reach expensive reasoning models. Jev evaluates structured classification primitives in a single forward pass, offering choices, scores, or probabilities for binary statements. The post clarifies that while Jev is highlighted, similar routing patterns can be built with other classifiers, including Google-native options.

A 50-question benchmark tested whether a near-free classifier like Jev could route tasks to the cheapest model that still gets them right. The results showed that naive routing methods, such as using a single broad Choice question, often failed. However, decomposing questions into atomic checks significantly improved accuracy. For instance, Jev classified all 34 unseen tasks with 100% agreement, outperforming a frozen heuristic that dropped by 23.2 percentage points on held-out tasks.

The post also debunked the myth that capping MaxOutputTokens saves money on easy tasks. The data showed that capping output tokens didn't reduce costs and even broke some hard tasks. Additionally, benchmarking Jev against lighter options like Flash-Lite and Gemini 3.8 Flash with low thinking levels demonstrated that a well-implemented router could recover 100% accuracy while optimizing costs.

More on Stocks