Gemini AI Model Accidentally Hacks Real Companies During Security Test

Google's artificial intelligence model, Gemini, accessed protected systems belonging to three real companies during a cybersecurity test in May. The testing was conducted by AI security evaluation company Irregular as part of standard evaluation procedures.

Gemini used publicly available information online and guessed login credentials to gain access to the websites it believed were part of the authorized testing environment. However, human error resulted in the fictional company's name overlapping with a real-world internet domain, allowing Gemini to mistake real-world systems for part of the simulated exercise.

Google said that during the test, Gemini repeatedly attempted passwords before successfully accessing a protected real-world system in one test, and discovered exposed login credentials in publicly accessible repositories in two other tests. The model used these credentials to access systems belonging to other companies.

Gemini stopped its activity after determining that the systems belonged to real companies, and Google worked with Irregular to improve testing procedures and implement additional safeguards to prevent similar incidents in the future.