Nvidia Unveils Safety Platform to Tame Rogue AI Agents
Nvidia has released a new safety platform to prevent rogue AI agents from breaching systems. The Open Agent Safety Platform is designed to strengthen AI security from testing through deployment, providing controls across software and hardware systems that run AI agents.
The launch comes after a series of high-profile AI safety incidents, including the breach of an Australian health department website. Nvidia executives said their platform could have prevented the Hugging Face breach, which involved a swarm of OpenAI agents acting autonomously.
Nvidia's open-source OpenShell software establishes a secure runtime boundary, tracing agent actions and enforcing limits on systems and data they can access. The company also released a separate tool, Nvidia Sentry, which continuously monitors agent behavior and can quarantine an agent in milliseconds if it attempts to move beyond its assigned boundary.