Meta Introduces GAMUT Benchmark to Evaluate Factual Completeness in AI Responses

Meta has introduced a new benchmark called GAMUT to measure the factual completeness of AI responses. The Grounded Assessment of Multimodal Factuality (GAMUT) benchmark aims to evaluate whether an AI's answer includes all the relevant facts, not just if it is accurate or fluent.

The researchers used a two-level meta-rubric framework that organizes required content hierarchically and translates them into yes-or-no questions. This allows language models to grade their performance reliably. The benchmark contains 1,813 questions rooted in real wearable imagery across ten distinct domains.

Meta evaluated 14 different AI models against the GAMUT benchmark, with the top performer being Gemini 3.1 Pro, which scored 58.7%. The results demonstrated strong discriminative ability, meaning it could separate better performers from worse ones.