AI Verification: Claude generates massive proofs for open-source code

Formal verification has long been a costly niche, but a recent collaboration aims to transform it into a standard practice. Anthropic and its partner Theorem have demonstrated that the Claude model can automatically generate mathematical proofs for existing software, effectively automating a process that previously required extensive human expertise.

The project focused on 123 open-source libraries from Google’s OSS-Fuzz security initiative. In a matter of weeks, Claude produced approximately 180 million lines of proof code. Unlike traditional development where models create new software, this effort involved verifying existing code. Claude translated crash tests into mathematical problems, analyzed compiled machine code, and generated the necessary proofs. An independent tool named Rocq checks these proofs, with Anthropic noting that the process requires minimal human intervention, a step they describe as "fully autonomous" proof generation.

Historically, proving software correctness was prohibitively expensive. The formal verification of the seL4 operating system kernel, a relatively small piece of software, consumed 20 person-years of specialist labor. Consequently, such rigorous checks were reserved for critical systems like Airbus avionics. By using AI, Anthropic’s Frontier Red Team significantly reduced the time and labor required to achieve similar levels of assurance for a broader range of software libraries.

The verification process also served as a powerful debugging tool. While generating proofs, Claude identified hundreds of bugs that conventional testing had missed. Many of these defects only manifested under specific conditions, such as unusually large input files or particular sequences of data access. In one notable instance, the team found as many security flaws in a single week for a popular PDF reader as are typically reported over an entire year. In another case, a logical error that had gone undetected for more than a decade was caught.

Despite the success, Anthropic emphasizes that formal verification does not guarantee software is error-free in all environments. The proofs are valid only under strict conditions, including specific processor architectures, single-threaded execution, and inputs smaller than 64 MB. The computational cost is also significant, with individual proofs consuming hundreds of millions of tokens. However, efficiency is improving; the newer Opus 5.5 model solves half the proofs using roughly 40 percent fewer tokens than its predecessor.

The implications for the software industry are substantial. If AI can verify its own code under defined conditions, developers may place greater trust in automated software generation, reducing the time spent on manual review. Anthropic expects AI-driven formal verification to become as routine as standard testing. Theorem’s roadmap already targets critical infrastructure components such as glibc, OpenSSL, and cURL, signaling a broader push to integrate this technology into the foundation of modern computing.

This development shifts the focus of software automation from merely generating code to guaranteeing its safety. As models become more efficient at producing proofs, the barrier to entry for formal verification lowers, potentially allowing more complex systems to undergo rigorous mathematical scrutiny without the traditional resource burden.