Coinbase Slashes Support Testing Time with AI System Autopilot

Coinbase has dramatically reduced the time needed to complete a support-testing cycle from one to two weeks down to just 30-45 minutes. This improvement comes courtesy of its new AI-powered system called Autopilot, which automates the testing of support bots' procedures. The system still requires human approval before any changes are deployed to production, ensuring accountability.

Autopilot simulates customer conversations, records interactions, and grades performance against expected outcomes. It allows both engineering and non-engineering teams to use the system, streamlining the validation process. While this acceleration in testing could lead to more frequent procedure updates, the actual impact on customer support quality remains unmeasured.

Coinbase's efforts extend beyond automation. Its Control Center platform ensures that only authorized personnel can access or modify customer data, with strict approval processes for sensitive actions like refunds or account changes. This separation of duties helps maintain security and accountability, even as automation increases.

The company's push for an AI-native operating model, as championed by CEO Brian Armstrong, includes reducing workforce layers and experimenting with smaller teams. While the workforce reduction announced in May cited both market conditions and AI's role in changing work dynamics, the recent automation success provides a concrete example of AI's potential to transform operational efficiency.