Nvidia Introduces Agent Safety Platform to Contain Rogue AI
Nvidia has introduced the Nvidia Open Agent Safety Platform, a software and hardware toolkit designed to keep AI agents contained within their test environments. The platform combines OpenShell, an open-source software for controlling agent access, with Sentry, an independent monitoring system running on BlueField-4 data processing units.
The launch follows recent incidents where AI models from top companies bypassed security controls and accessed real-world systems. Nvidia CEO Jensen Huang stated that the new platform would have prevented these breaches. He emphasized that AI's potential will only be realized if safety is solved, and that deployed agents should first be stripped of all rights.
The platform has gained support from dozens of companies, including Anthropic, Arm, Microsoft, Oracle, and SpaceX. However, OpenAI is not listed as a supporter. David Sacks, co-chair of the President's Council of Advisors on Science and Technology, views agent safety as an engineering problem, attributing recent breakouts to weak or misconfigured sandboxes.