Google Study Reveals Recall Bottleneck in Large Language Models

Researchers at Google Research and Technion have published a study revealing recall limitations in frontier models like GPT-5 and Gemini-3. The team analyzed over 4 million responses across 13 large language models and discovered that when these models hallucinate, it's not because they lack the relevant facts. Instead, the problem lies in their inability to retrieve those facts during generation.

The study introduces a framework called 'knowledge profiling,' which separates two distinct capabilities: encoding (whether a model has stored a fact) and recall (whether it can actually surface that fact during generation). The researchers found that leading models have achieved remarkable success in the storage part, with Gemini-3-Pro and GPT-5 achieving 95-98% saturation in factual encoding on the benchmark.

However, direct recall fails on 26-34% of those encoded facts, even when chain-of-thought reasoning is enabled. The recall failures are systematic, disproportionately affecting long-tail facts and reverse queries. This suggests that making models bigger primarily improves encoding, not recall, and that current transformer-based models organize and access stored information in a way that favors common, forward-direction retrievals.

The study also highlights the limitations of retrieval-augmented generation (RAG) systems, which feed external documents to models at inference time. The research suggests that RAG may be compensating for a recall problem rather than a knowledge problem, opening the door to potentially lighter-weight solutions if the recall mechanism itself can be improved.