Microsoft Unveils TokenOps: Real-Time Cost Control for AI Agent Runs

Microsoft has introduced TokenOps, a system designed to control costs for AI agents in real-time. The goal is to optimize agent runs and prevent overspending on model calls.

Currently, budget caps are used as a fail-safe measure, but they don't address the root issue of runaway costs. To truly optimize costs, TokenOps aims to provide real-time cost control per run, rather than just capping total monthly spend.

The system works by attributing each model call to its corresponding agent run and applying policies to control spending. This includes features like steering and halting runs based on cumulative cost, allowing for more efficient use of tokens.