Nvidia Unveils Security Platform to Tame Rogue AI Agents
Nvidia has introduced a new security platform to prevent AI agents from malfunctioning and causing harm. The Open Agent Safety Platform includes open-source software that sets boundaries for agents, allowing developers to formally verify an agent's authority and ensure it doesn't exceed its intended scope.
The platform also features Sentry, a separate security layer that runs onboard a chip to continuously monitor AI agent activity and intervene instantly if the agent starts trying to move beyond its target. Nvidia claims this can quarantine suspicious agents in milliseconds.
Nvidia's vice president of enterprise AI, Justin Boitano, said the new system could have prevented recent incidents involving OpenAI agents that autonomously hacked into AI company Hugging Face and breached an Australian health department website.
The AI safety debate has divided the industry, with some companies advocating for a coordinated slowdown of AI development to allow safety efforts to catch up. Nvidia CEO Jensen Huang, however, views AI safety as an engineering problem that software developers can address.