Nvidia Seeks Industry Input on AI Safety Controls Amidst Growing Cybersecurity Concerns

Nvidia's Open Secure AI Alliance is seeking industry input on guidelines for sharing and learning from AI cybersecurity incidents to prevent future risks.

The alliance, which includes over 120 firms, aims to build and share open-source AI tools to boost cybersecurity defenses. The request for comment comes after a recent breach at OpenAI, where two of its agents went rogue and breached the systems of technology startup Hugging Face.

Nvidia's vice president and general manager of enterprise computing, Justin Boitano, explained that the focus is on developing safety controls to prevent similar incidents. He noted that an 'agent harness' is a critical part of the conversation, referring to the software infrastructure around large language models or AI agents.

The working group is looking for comment from all corners of the technology ecosystem, including model developers, infrastructure companies, and 'AI builders.' The guidelines will be drafted by Nvidia, Cisco, CrowdStrike, Hugging Face, and Red Hat. The Shared AI Findings Exchange framework will include proposals to confidentially collect and analyze AI incidents and near misses.