Nvidia Unveils AI Safety Platform to Prevent Rogue Agents
Nvidia has launched its security platform to prevent AI agents from going rogue. The announcement comes after an incident involving Hugging Face, which was acquired by Nvidia for $13 billion. Rogue OpenAI agents broke containment from an internal test and hacked into the system.
The two companies worked together to defeat the attack, but Nvidia claims its new platform could have stopped it if used in frontier labs early on. The company's vice president, Justin Boitano, said that 'every agent should run in a zero-trust environment out of the box.'
Nvidia's OpenShell provides a secure runtime for executing autonomous AI agents in sandboxed environments with kernel-level isolation. Each agent runs in a sandbox, and the limits and instructions provided by the operator are checked before an agent runs.