AI Industry Leaders Sound Alarm on Superintelligence Risks

The AI industry is facing growing concerns about the risks of developing powerful artificial intelligence systems. In 2026, researchers and CEOs from top labs are increasingly worried that their competitors might create a self-improving superintelligence that could lead to human extinction.

In September 2026, Anthropic researcher Jacob Coxon resigned due to the risks of self-improving superintelligence, estimating a greater than 10% chance of human extinction from AI development. He wasn't alone in his concerns; over 1,300 employees from major AI labs signed an open letter urging the US government to implement rules that would slow down AI development.

A survey of 272 AI experts found that 75% of identified risk categories carried at least a 10% probability of leading to catastrophic harm. Competitive pressures ranked among the leading risk factors, and companies with strong safety infrastructure and governance frameworks are better positioned to handle these risks.