AI Agents Show Diverse Performance in Detecting Bitcoin Vulnerabilities

Researchers at the Bitcoin Red Team have been using artificial intelligence (AI) to detect security vulnerabilities in open-source projects. Developer Paul Miller tested seven AI agents on five real vulnerabilities and found significant differences between them. Three models, Qwen 3.8 from Alibaba, GPT-5.6 Sol from OpenAI, and Fable from Anthropic, failed the task, with two refusing to work altogether.

The other four models that completed the task varied greatly in their dollar costs for token consumption: Grok 4.6 cost $4.70, while DeepSeek V4 Pro was significantly cheaper at just $0.30. The time taken also varied, ranging from 30 minutes to three hours.

Miller found that only Grok 4.6 accurately matched his own assessment of the vulnerabilities' severity, while the other models exaggerated their severity, affecting the overall quality of the results. This comes amid a wave of attacks using AI to exploit security flaws in the Bitcoin ecosystem.