For three years, Jacob Coxon helped train increasingly powerful AI systems at OpenAI and Anthropic. On Sept. 8, he walked away. They are racing straight to self-improving superintelligence and gambling with our lives.” His dire warning amassed more than 90 million views in less than 24 hours.
What did Coxon see that made him walk away? No single breakthrough, he says. His resignation was driven instead by two conclusions: “One, it’s obvious that things are speeding up, and two, they’re not under control,” the 27-year-old Brit tells TIME.
Coxon’s departure is unusual. Many of the most prominent researchers to leave frontier AI companies with public warnings worked on safety. But Coxon helped build the capabilities he now fears, spending roughly three years conducting pretraining research at OpenAI and Anthropic.
His resignation offers a glimpse of how concern about the pace of development has spread beyond the teams specifically charged with making advanced AI safer. AI’s rapid improvement has been particularly evident in mathematics, says Coxon, who studied the subject at Cambridge before moving into AI. In recent months, AI labs have announced solutions to several longstanding mathematical problems.
Most notably, OpenAI claims to have resolved the Navier–Stokes existence and smoothness problem—one of mathematics’ seven Millennium Prize Problems—using roughly 10,000 concurrent agents over 88 hours. If that trend continues, and extends to AI research itself, Coxon fears it will kick off a feedback loop of further acceleration. Read more: What happens when AI builds itself?
Meanwhile, in the recent Hugging Face incident, OpenAI’s models broke out of the infrastructure meant to contain them and hacked another AI company to cheat on a cybersecurity benchmark. The episode drove home the fact that the problem of controlling AI systems remains unsolved. To Coxon, it made the sci-fi scenario of AI models escaping human control seem plausible, and perhaps urgent.
Extract — continue reading at the source.
