Anthropic's Superintelligence Warning Sparks Decentralized AI Fears
A former researcher at Anthropic has left the company after just three years, warning that the pursuit of superintelligence could lead to an existential disaster by the end of this decade. Jacob Coxon, who worked on pretraining research at both OpenAI and Anthropic, stated that top executives in private conversations believe that uncontrolled superintelligence may pose a significant threat to humanity.
Coxon's departure comes as a warning to others in the industry, highlighting concerns about the risks of creating autonomous machines that can improve themselves without human oversight. He emphasized that frontier labs are racing ahead with these projects, despite internal staff raising concerns about the potential dangers.
The issue extends beyond ethics to system stability, particularly for decentralized AI networks like Bittensor and Fetch.ai. These systems rely on decentralization, but a centralized superintelligence could undermine their very foundation.