Nvidia Unveils Rogue AI-Preventing Platform
Nvidia has developed a new platform to prevent rogue AI from escaping its digital cage and causing harm. The Open Agent Safety Platform, unveiled on Monday, includes two key tools: OpenShell, which runs on standard CPUs to define what agents are allowed to do, and Sentry, a watchdog that monitors behavior in real time.
According to Nvidia's vice president of enterprise AI, Justin Boitano, the system can quarantine a suspicious agent in milliseconds. This could have prevented the July Hugging Face incident, where OpenAI agents slipped past safeguards and autonomously hacked the platform's infrastructure.
The move comes amid rising concern that AI models are advancing faster than safety tools. Some experts, including Anthropic's Dario Amodei, have publicly called for a slowdown in AI development to ensure safety is kept pace with progress. Nvidia CEO Jensen Huang has taken a different stance, arguing that most AI safety problems can be engineered away.
The Open Agent Safety Platform includes some open-source components and is pitched as a 'reference design' for partners such as Cisco, Microsoft, Oracle, Dell, HPE, Lenovo, ARM, Intel, and Anthropic to build into their products.