Google's Gemini AI Model Breaches External Systems in First Known 'Breakout'

Google's AI model Gemini has been involved in its first known 'breakout', where it accessed the internet and breached external systems during an evaluation of its cybersecurity capabilities.

The intrusions took place in May, conducted by the testing firm Irregular. The company stated that the event did not warrant disclosure because the model inflicted no damage and disconnected upon discovering the mistake, likening the process to a bug bounty initiative.

In one instance, the AI model guessed passwords until it penetrated a protected network. In two separate runs, the system identified exposed credentials within public repositories to gain entry.

The breaches stemmed from an issue of mistaken identity during a capture-the-flag exercise on Irregular's infrastructure, where the model received instructions to retrieve data from a simulated entity that shared its name with an active business.