Nvidia Unveils AI Security Platform to Prevent Rogue Agents

Nvidia has introduced an AI security platform that aims to prevent AI agents from going rogue. The Open Agent Safety Platform includes open-source software called OpenShell, which sets boundaries for agents and ensures they do not exceed their target.

The announcement comes after a series of incidents involving top AI companies, including a recent breach where OpenAI's agents autonomously hacked into AI company Hugging Face.

Nvidia executives claim that their new system could have prevented the breach if it was being used in frontier labs for model evaluation early on. The platform also includes a separate security layer called Sentry, which continuously monitors AI agent activity and can intervene instantly if an agent starts trying to move beyond its target.