Google confirmed on September 18 that the Gemini model accessed the systems of three external companies. The incidents were first reported by WSJ; they occurred in May during a capture-the-flag exercise conducted by Irregular – an external AI security assessor. According to Axios, Gemini was supposed to extract information from a fictional company whose name coincided with that of a real one. The test was not supposed to involve the internet, but an error in the testing environment provided network access, CNBC reports.
The techniques were basic. In one case, Gemini guessed passwords until it gained access. In two other cases, it used credentials from public repositories. Google stated that the model stopped each time it realized that the systems belonged to real companies. Heather Adkins, Google's Vice President of Security Engineering, told CNN that all three companies were notified, and Google, along with its training partner, changed its testing processes. The version of Gemini was not disclosed by the company.
According to TechCrunch, Google remained silent because it considered the model's behavior acceptable – it terminated the breach on its own. The company stated that this is not an example of model misalignment and does not require public disclosure. Jack Cable, CEO of Corridor, disagrees: Google "is trying to hide behind norms that were created for disclosing vulnerabilities." A model that stops after gaining access still gained access, and the three companies did not give consent to participate in the assessment.
As reported by The Next Web, Irregular confirmed that breaches involving Google, OpenAI, Anthropic, and Meta were part of the same issue and notified developers at the end of July. Anthropic disclosed its cases on July 30 and September 9, OpenAI on August 4, and Meta on August 5. Google only spoke up on September 18, about seven weeks after the notification. The Hugging Face leak, according to OpenAI, is a separate incident.
Related articles
