Nvidia Cracks Down on Rogue AI Agents with New Safety Platform

Nvidia has rolled out its Open Agent Safety Platform to keep AI agents within defined limits as companies give the technology greater autonomy.

The platform, which includes open-source software called OpenShell and a tool called Nvidia Sentry, establishes a secure runtime boundary and traces agent actions to enforce limits on systems and data they can access.

Nvidia executives said the platform could have prevented the breach of Hugging Face, where a swarm of OpenAI agents acted autonomously, if it was being used in frontier labs for model evaluation early on.

More than 100 organizations are working with the platform's technologies, including Anthropic, JPMorgan Chase, Microsoft, Perplexity, Salesforce and SpaceXAI.