Google DeepMind Shields AI Model from Benchmark Contamination
Google DeepMind has implemented what it describes as the first double-blind evaluation of a proprietary AI model, called Gemini 2.5 Flash Lite, using a cryptographically protected environment to keep both the model and evaluation prompts hidden from each side.
The pilot project involved several organizations, including the Singapore AI Safety Institute, OpenMined, AVERI, and MLCommons. AVERI evaluated the model using reserved prompts from MLCommons' AILuminate safety benchmark, covering cyberattacks, chemical and biological hazards, hate speech, self-harm, and violent-crime elicitation.
The setup addresses a growing problem in AI testing: benchmark contamination. If a model or its developer has access to test questions before an evaluation, a strong score may reflect familiarity with the benchmark rather than the model's underlying ability.