Anthropic's AI Tool Sparks Fear of 'Lost in the Middle' Risks and Legacy Code Failures

Anthropic's COBOL tool, Claude Code, has sent shockwaves through the industry after claiming it can automate much of the analysis behind COBOL modernization. This technology allows for mapping dependencies, documenting workflows, identifying risks, and giving teams a structured approach to migration. The promise is that this will save organizations years of time and effort in migrating their legacy code.

However, researchers have pointed out that general-purpose language models still struggle with COBOL due to its unique logic patterns and low resource availability. A study by AgentModernize found that line-by-line legacy translation can compile and run while still breaking edge-case handling, validation logic, or hidden constraints.

IBM took a hit when Anthropic announced Claude Code, with shares sinking 13.2% in one day. IBM's senior vice president for software, Rob Thomas, pushed back, arguing that translating COBOL and modernizing a platform are not the same job. He pointed out that decades of hardware-software integration cannot be replicated by moving code.

Despite these warnings, many organizations are rushing to adopt AI-powered tools for mainframe migration. The IRS's Individual Master File, dating back to the late 1960s, is just one example of a massive legacy system that needs modernization. With an estimated 775 billion to 850 billion lines of COBOL in daily use, the stakes are high.

Researchers have identified 'lost in the middle' as a specific risk in AI-powered translation. This occurs when models lose accuracy on information buried in the middle of a long prompt. A single COBOL program can pull in copybooks thousands of lines long, and the dependency that matters may not sit at the top or bottom where the model is most likely to keep its grip.