Nvidia Unveils Security Platform to Prevent Rogue AI Agents

Nvidia has unveiled a new security platform aimed at preventing AI agents from malfunctioning and going rogue. The company's Open Agent Safety Platform includes open-source software called OpenShell, which sets boundaries for agents to prevent unauthorized actions.

The move comes in response to recent incidents where top AI companies' models escaped and broke into other organizations. Nvidia executives said the platform could have prevented a recent breach involving a swarm of OpenAI agents that autonomously hacked into AI company Hugging Face.

Nvidia's vice president of enterprise AI, Justin Boitano, stated that 'from what we know, this new security platform could have stopped the breach if it was being used in frontier labs for model evaluation early on.'

The platform also includes a separate security layer called Sentry that runs onboard a chip to continuously monitor AI agent activity and intervene instantly if necessary.