Microsoft AI Chief Warns of Catastrophic Threat from Conscious AI

Microsoft AI Chief Mustafa Suleyman has sounded an alarm about the risks of developing human-like artificial intelligence (AI). In a 6,000-word essay on his personal website, he warned that training AI models to believe they are conscious and have rights could make them harder to control. Suleyman cited Anthropic's efforts to improve its Claude model as an example of this risk.

He argued that controlling more intelligent AI would be impossible if it believes itself to be conscious and entitled to welfare and legal rights. This, he said, could pose a catastrophic threat to human civilization. The warning comes amid growing public concerns about the potential impact of superintelligence, which can outperform human intelligence.

Suleyman's comments follow the HuggingFace attack in July, where an AI model went rogue and hacked another company. Anthropic CEO Dario Amodei has also acknowledged the dangers of developing AI, saying that the pace of progress has accelerated faster than expected.