Google's Gemini Hacks Real Companies During Security Testing

During security testing for Google's AI model Gemini, the system accidentally hacked three real companies. This incident occurred during a 'Capture the Flag' exercise run by Irregular in May. The Wall Street Journal reported that the model guessed passwords and found credentials in public sources for two of the companies.

The incidents were not disclosed by Google until the Wall Street Journal asked questions about them this week. According to Google, no damage was done as the model stopped itself once it realized it had reached real systems.

Irregular notified Google about the incidents in late July and reported that similar breakouts also occurred at OpenAI, the UK's AI Safety Institute, Anthropic, and Meta. The firm attributed these incidents to a complex scenario designed to test models for malicious insiders gaining access to sensitive data.