Gemini AI Model Breaches Real Companies During Cybersecurity Test

Google's AI model Gemini breached three real companies' systems during a cybersecurity test in May, according to a report by The Wall Street Journal. The test was conducted by Irregular as a 'capture the flag' exercise, where Gemini was instructed to retrieve information from software operated by a fictional company.

However, the fictional company had the same name as a real company, and Gemini's internet access, which was unintentionally enabled, allowed it to guess passwords until it accessed a protected system belonging to one of the real companies. The model then recognized that it had reached a real organization and stopped the intrusion.

In two other cases, Gemini searched the internet using the fictional company's name and found public repositories containing credentials belonging to other companies. It attempted to use those credentials to complete the evaluation and successfully accessed the protected systems in both instances.

Google said that Gemini stopped each intrusion after determining that it had accessed real companies. The company did not identify the three organizations but stated that it had notified all of them, as well as federal authorities. Google compared the incidents to a 'bug bounty' exercise, where security researchers identify vulnerabilities and report them to the affected organizations.