DeepMind Researcher Quits Over 'Terrifying Chance' of Major Harm from Unchecked AI

Josh Engels, a researcher from Google DeepMind's AGI safety team, has left his position to join independent AI evaluation group METR. In a post on X, he expressed concerns about the rising stakes in efforts by leading AI companies to build systems more capable than humans across a broad range of tasks.

Engels warned that if safety work fails to keep pace with the development of increasingly capable artificial intelligence systems, it could cause immense harm. He cited recursive self-improvement as a particular concern, where AI systems contribute to developing increasingly capable successor systems without sufficient alignment with human intentions.

METR evaluates frontier AI systems for autonomous capabilities and potential catastrophic risks. Engels plans to work on investigations into alignment incidents and study the circumstances in which AI systems act contrary to human intentions at METR.