The International Congress of Mathematicians met in Philadelphia last month, its first U.S. meeting since 1986. Minutes after accepting the Fields Medal, mathematics’s highest honor, Jacob Tsimerman told reporters he was leaving academia to work on AI safety. Two months earlier, he had been one of nine mathematicians asked to review whether an AI model had truly refuted an 80-year-old central pillar of modern geometry, known as the Erdős unit distance conjecture.
Tsimerman and his peers determined that it had, and their commentary alongside the 18-page proof helped translate the machine's argument into mathematics a human could understand. The breakthrough, and subsequent report, reflected how the bottleneck that holds us back from forging new discoveries in mathematics, and, increasingly, in every other field, is changing. For most of history, the scarce resource was discovery.
With AI, discovery is nonstop, and human confirmation is now what is scarce. To be clear, we are AI optimists. We believe AI tools will complement human ingenuity and expand what we can know and build, but our infrastructure for vetting and certifying discoveries was built for human throughput, and that is now the binding constraint.
That is what holds innovation back. The answer to this problem is formalization: translating AI’s outputs into precise forms whose correctness can be checked automatically. Building the infrastructure to make verification routine is now a national-scale engineering problem.
Machines are outrunning us in more than math. The same is happening to the code that runs hospitals, banks, and power grids. In April, for example, Anthropic disclosed that its Mythos model could find unknown vulnerabilities in major operating systems and browsers, and restricted access to fifty organizations racing to patch them.
Soon after, Microsoft engineers found 90 critical flaws in a widely used product; and in June, Sen. Mark Warner told a Senate hearing, citing the NSA director, that the tool "broke into almost all of our classified systems, not in weeks but in hours." All of this arrives in a world that already runs on code we assume is correct. In July 2024, a single faulty software update, not a cybersecurity attack, grounded flights and disrupted hospitals worldwide.
Extract — continue reading at the source.
