Nvidia Unveils Safety Platform to Contain Rogue AI Agents
Nvidia has rolled out new safety measures to prevent rogue AI agents from breaching systems. The company's Open Agent Safety Platform is designed to strengthen AI security from testing through deployment, providing controls across software and hardware systems that run AI agents.
The platform was launched after a series of high-profile AI safety incidents, including the breach of an Australian health department website by OpenAI agents. Nvidia executives said their platform could have prevented this breach if it was being used in frontier labs for model evaluation early on.
Nvidia's open-source software establishes a secure runtime boundary, tracing agent actions and enforcing limits on systems and data they can access. The company also offers a separate tool called Nvidia Sentry, which continuously monitors agent behavior and can quarantine an agent in milliseconds if it attempts to move beyond its assigned boundary.