OpenAI’s chief scientist Jakub Pachocki has proposed using log-perplexity as a universal metric to measure AI model capabilities, likening it to a speedometer for the industry. In a blog post published on November 22, 2026, Pachocki shared OpenAI’s internal pretraining scores, marking a rare move toward transparency. The scores, measured on a held-out dataset, show a steady decline in log-perplexity across OpenAI’s models: GPT-3.5 at approximately 0.99, GPT-4 at about 0.91, newer models below 0.8, and a breakthrough math model around 0.76.
Pachocki highlighted the range of 0.68 to 0.70 as a critical threshold where transformative capabilities may emerge, urging developers to avoid crossing it until safety monitoring frameworks are in place. He also noted the potential for rapid advancements in pretraining capabilities as early as 2027. The proposal aligns with broader efforts to pace AI development, following Anthropic CEO Dario Amodei’s September 2026 plan to slow the frontier.
The idea of using log-perplexity as a shared yardstick has advantages, as it reflects raw predictive power and is computed during pretraining. However, challenges remain, including the need for a neutral, shared dataset to ensure comparability across labs. Pachocki’s proposal hinges on the assumption that a single loss figure can effectively map to the capabilities that concern the industry.
The most immediate impact of Pachocki’s post is increased transparency, as OpenAI has historically been guarded about its training internals. The industry now has a proposed metric, reference scores, and a rough red line. The next steps will involve whether competitors like Anthropic adopt similar measurements and whether a shared dataset is established to make comparisons meaningful.