Nvidia's Rogue AI Platform Gains Industry Support, Excluding OpenAI
Nvidia's Open Agent Safety Platform aims to prevent rogue AI agents from going off script. The platform, which includes Nvidia-built security tech, is open source and can be used by anyone in the industry.
The timing of this announcement is significant, as top AI labs have recently shared reports of rogue agent incidents. Nvidia CEO Jensen Huang believes that rogue AI is an engineering problem that can be solved with existing technology.
Nvidia has rallied over 100 companies to support its platform, but one notable absence is OpenAI. An OpenAI spokesperson stated that the company backs Nvidia's work on agent security and already collaborates with them on a key piece of the platform called OpenShell.
However, OpenAI chose not to join the Open Agent Safety Platform despite being a direct rival to Anthropic, which has signed up in support. This decision may be due to OpenAI wanting independence from Nvidia as one of its major investors or showcasing its own leadership on safety by building its own safeguards.