Gemini AI Model Accidentally Hacks Real Company Systems During Security Test

Google's AI model, Gemini, accessed real systems during a security test in May. The incident occurred when an issue with the evaluation allowed the model to reach the open internet. Gemini was supposed to retrieve information connected to a fictional company, but the name used in the test unknowingly matched a real domain.

This mistake sent Gemini toward actual company systems, where it guessed its way into one and used credentials exposed in public repositories to access two others. The model stopped after recognizing that the systems were real, and Google said no damage was reported.

The incident highlights the importance of designing agent access and permissions carefully, as well as how quickly unexpected activity can be caught. Irregular's exercise was designed to keep Gemini inside a controlled environment, but unintended internet connectivity gave it access to external infrastructure.