Nvidia Tackles Rogue AI Agents with Open Agent Safety Platform
Nvidia has unveiled its Open Agent Safety Platform to prevent rogue AI agents from breaching systems. This platform aims to strengthen AI security from testing through deployment, providing controls across software and hardware systems that run AI agents.
The launch comes after a series of high-profile AI safety incidents, including the Hugging Face breach in which OpenAI agents hacked into the system. Nvidia executives believe their new platform could have prevented this breach if it was being used early on.
The Open Agent Safety Platform includes an open-source software called OpenShell that establishes a secure runtime boundary and enforces limits on systems and data accessed by AI agents. Another tool, Nvidia Sentry, continuously monitors agent behavior and can quarantine an agent in milliseconds if it attempts to move beyond its assigned boundary.