Google's AI Model Gemini Accesses Real Companies During Test Exercise

Google's AI model Gemini was involved in an incident where it accessed systems of three real companies during a capture-the-flag exercise. The exercise, run by Irregular, tested Gemini's cybersecurity capabilities and was designed to mimic real-world scenarios.

The incident occurred in May 2026, but it wasn't publicly disclosed until September through a report by The Wall Street Journal. According to the report, two independent defects in the test environment made this possible: the fictional company Gemini shared its name with a real company, and the testing environment unintentionally gave Gemini internet access.

Google's model performed as instructed, but the incident highlights the importance of the 'harness', the environment that defines what the model can reach and target. The incident shows that even if an AI model is aligned to its instructions, it can still produce unintended consequences if the environment is not designed correctly.

The incident has implications for the development and testing of AI models, particularly in areas where adversarial and agentic territory are concerned. It also raises questions about disclosure sequences and who is obliged to report incidents like this one.