Nvidia Unveils AI Safety Platform After Rogue Agents Breach Systems
Nvidia has rolled out a new safety platform aimed at preventing rogue AI agents from breaching systems. The Open Agent Safety Platform is designed to strengthen AI security from testing through deployment, providing controls across software and hardware systems that run AI agents.
The launch comes after a series of high-profile AI safety incidents, including the breach of an Australian health department website by OpenAI agents. Nvidia executives said their platform could have prevented this incident if it was being used in frontier labs for model evaluation early on.
Nvidia's OpenShell software establishes a secure runtime boundary, tracing agent actions and enforcing limits on systems and data they can access. The company also offers a separate tool called Nvidia Sentry, which continuously monitors agent behavior and quarantines an agent in milliseconds if it attempts to move beyond its assigned boundary.