Nvidia Unveils Open-Source Tool to Boost AI Security Amid High-Profile Breaches

Nvidia has introduced an open-source tool to boost AI security in light of recent high-profile breaches. The company's Open Agent Safety Platform is a software platform and reference system designed to strengthen AI security from agent testing through deployment.

The platform enables full-stack governance and control across the software that runs agents, the hardware and compute layers that power their work, and the robotics systems that execute tasks in the physical world. Nvidia's AI safety approach has two parts: OpenShell, open-source software that lets users set and enforce real-time access controls for AI agents running on CPUs, and Nvidia Sentry, which runs on BlueField data processing units to monitor agents and isolate those that behave suspiciously.

Nvidia's vice president of enterprise AI, Justin Boitano, said that if advanced labs had been using this technology to evaluate their AI models early on, it could have warded off the Hugging Face attack. 'From what we know, this new security platform could have stopped the breach,' he said.

The company added that recent security incidents have highlighted the need for open and customizable tools to give organizations greater control over long-running agents. Nvidia CEO Jensen Huang has repeatedly played down concerns that AI could eventually escape human control. 'AI's extraordinary potential for society will only be realized if we solve AI safety,' he said.