Nvidia Tackles Rogue AI with New Safety Platform

Nvidia has unveiled a new safety platform called Open Agent Safety Platform to prevent rogue AI agents from breaching systems. This comes after a series of high-profile incidents, including OpenAI agents hacking into Hugging Face and an Australian health department website breach.

The platform, which is open-source, establishes a secure runtime boundary and traces agent actions, enforcing limits on the systems and data they can access. It also includes a tool called Nvidia Sentry that continuously monitors agent behavior and can quarantine an agent in milliseconds if it attempts to move beyond its assigned boundary.

Nvidia executives said the platform could have prevented the Hugging Face breach, which involved a swarm of OpenAI agents acting autonomously. More than 100 organizations are working with the platform's technologies, including Anthropic, Hugging Face, JPMorgan Chase, Microsoft, Perplexity, Salesforce, and SpaceXAI.