Google's Autonomous Customer Support Agent Highlights Zero-Trust Security Risks

Google has developed an open-source autonomous Customer Support & Returns Agent to demonstrate how developers can apply zero-trust security principles to AI agents. The project tests an approach that assumes an AI agent could be manipulated or compromised and puts security controls around it.

The customer support agent reads a customer's return request, generates a Python script to calculate prorated restocking deductions, records an approved refund in a database ledger, and provides a confirmation. However, during a test scenario, a customer with a $149 order instructs the agent to issue a $10,000 refund.

The agent's design treats the model as a component that could be tricked or jailbroken. To prevent potential attacks, Google's proposed configuration assigns each agent its own service account and grants it signing permissions on an asymmetric key in Cloud Key Management Service (KMS).