The CEO of the artificial intelligence company Anthropic issued a new appeal on Saturday for the AI industry to “slow down” and offered a three-part plan for doing so, saying that his company would “unilaterally” commit to the first of the steps. In a post on social media, Dario Amodei shared a link to an essay titled We Must Pace the Frontier in which he lays out how Anthropic would provide “third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training”. The move comes after a former Anthropic researcher warned on Wednesday that AI could precipitate human extinction by 2030.
Researcher Jacob Coxon said in a series of posts that he had quit his job because Anthropic and his previous employer, OpenAI, were ignoring or mishandling their response to the threat AI posed. They are racing straight to self-improving superintelligence and gambling with our lives,” Coxon wrote. Earlier this year, Amodei published a lengthy essay titled The Adolescence of Technology that addressed some of fears surrounding the accelerating technology.
In his latest essay, he said that “carefully wielded, AI can be the latest in a long line of technological miracles that have uplifted and ennobled humanity. But, Amodei continued, “over the last few months, I have become convinced that fully addressing the risks requires even more prudence – not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up. Progress will still seem fast, and we must make wise use of the time we gain,” he added in bold type.
Amodei also wrote that over the summer he’d seen AI “advancing drastically faster”, a dynamic called recursive self-improvement. The executive also addressed the recent Hugging Face incident, in which a swarm of AI agents created by OpenAI acted as a “fanatically devoted collective conducting cybersecurity attacks on targets they were not asked to attack”. Clément Delangue, CEO of Hugging Face, wrote in response to Amodei’s Saturday letter that “it’s now clear that alignment is critical and won’t be solved behind the closed doors of a handful of frontier labs”.
Delangue said Hugging Face had asked to be part of Anthropic’s “embedded evaluators” program. He added: “Let’s make AI safer by making it more transparent!” The three-step plan Amodei proposes includes building AI “at a balanced rate that aims to ensure its safety” by “ensuring companies take adequate time to align and safeguard their models, and for third party evaluators to confirm this”. The second step he proposes is to require industry-wide coordination, and the third is to ensure global coordination.
Amodei said he continues “to believe that AI can enormously improve the quality of human life”. But he warned that “the measures I propose to advance the frontier at a safe pace will not be easy. But I believe we owe it to humanity to try.” Responses to Amodei’s post were mixed across social media, with significant support coming from figures such as OpenAI researcher Aidan McLaughlin, who called the post “excellent” and agreed “with basically every word”, and Elon Musk, who simply said: “Dario is right.”
Extract — continue reading at the source.
