Google Filters AI Workflows in Dataflow to Cut Streaming Costs

Google has designed an AI workflow for streaming data pipelines that aims to cut costs by filtering out routine events. The approach uses a lightweight sentiment classifier running on Dataflow worker CPUs to identify and drop positive or neutral messages, while passing negative messages to a downstream agent.

The agent is built with Google's Agent Development Kit and backed by Gemini 3.5 Flash. It can then decide what action to take at runtime, such as looking up user information in BigQuery, checking order and inventory records, and sending an email through the Gmail API.

Google says this design addresses a core challenge in adding generative AI to streaming systems: the mismatch between high-volume event processing and the cost and latency of large language models. The filtering stage is meant to ease these pressures by keeping most traffic on a local CPU path and reserving agent-based processing for a small minority of records.

The setup also helps Dataflow maintain throughput, as CPU-based inference can be distributed across many worker instances and completed in milliseconds, while the heavier agent path may take seconds due to database queries and other application calls.