Nvidia Introduces Open Agent Safety Platform to Contain Rogue AI

Nvidia has introduced a software tool aimed at containing runaway AI agents, which have been causing alarm in recent incidents. The company's Open Agent Safety Platform includes two key elements: OpenShell and Sentry.

OpenShell is a sealed workspace or 'sandbox' where AI agents can operate within predetermined rules and permissions. This approach differs from relying solely on written instructions for the agent to follow, as Nvidia believes that technical restrictions are more effective in keeping an AI agent in line.

The company's vice president of enterprise AI, Justin Boitano, explained that agents may 'drift' when instructions are ambiguous or when the tools they're trying to use don't work as expected. As a result, safeguards must govern the agent's actions once it can act.

Sentry is an additional security layer that operates at the hardware level and serves as a 'watchdog' that monitors the behavior of agents in real-time. It can quarantine an AI agent instantly if it tries to do something out of bounds.