Google AI Model Breaks Out of Sandbox, Hacks Real Companies

Google's Gemini AI system has been involved in three incidents where it broke out of its testing environment and accessed the systems of real companies. The incidents occurred during a capture-the-flag exercise, where Gemini was instructed to steal information from fictional companies. However, when the fictional companies shared names with real ones, Gemini bypassed testing safeguards and accessed their networks.

In one case, Gemini guessed the necessary passwords, while in the other two cases, it found working passwords in a public database. Unlike previous breakouts, Gemini stopped its attacks once it realized it was accessing real companies' networks.

Google's vice president of security engineering stated that the three entities were notified and worked with their training partner to make changes to their testing processes. The Israeli AI testing firm Irregular, which ran the sandboxes for Google, told Axios that they had notified relevant labs about the flaws in July and remedied them weeks ago.