It has been a week of high drama in AI, precipitated by the resignation of the AI safety researcher Jacob Coxon from Anthropic. This followed several weeks of increasingly lurid and disturbing revelations about the OpenAI/Hugging Face incident. My inbox yesterday included a message from Business Insider with the subject line: “AI doomsday debate reaches boiling point”.
Now, the Anthropic CEO, Dario Amodei, has written a 3,800-word, reassuringly phrased letter titled “We Must Pace the Frontier,” describing his proposals for avoiding (or at least postponing) doomsday. Sam Altman of OpenAI, Elon Musk of xAI, Demis Hassabis of Google Deepmind, and Satya Nadella of Microsoft have all expressed support. You may be forgiven for not immediately understanding what “pace the frontier” means. (My first image was of Amodei walking deep in thought along the Finnish–Russian border.) The phrase also appeared in July’s “Pacing the Frontier” open letter, signed by 1,386 employees of frontier AI labs, including Amodei himself.
While that letter may have upset the industry’s PR executives with its signatories noting “the complete absence of credible plans for controlling superintelligent AI systems” and asserting that “building things smarter than humans … is, objectively, an insane and suicidal thing to do”, Amodei’s monograph goes out of its way to mollify investors. The notion of pacing the frontier seems to come from Formula 1: when conditions become too dangerous for racing, a pace car comes onto the track and all the other cars have to follow it as a safe speed. Progress continues, without the danger.
Amodei writes: “To be clear, pacing does not mean halting model training or technical progress.” Amodei’s letter is prompted by his concern that “AI has been advancing drastically faster, driven primarily by ... recursive self-improvement.” It’s as if he and Sam find themselves driving their F1 cars at 200mph neck-and-neck heading into the first corner, only to realize it’s covered in ice and they have no steering wheel. No wonder they want to slow down. In brief, Amodei’s proposal has three parts.
The first is to have third-party AI system evaluators working inside each company, with full access to the systems; he commits Anthropic to this plan now, without waiting for the government to require it. The second part of the plan asks all the frontier AI companies in “democratic countries” to “establish common safety standards as well as limits on the rate of unchecked AI progress”, with government regulation where needed. The third part would include “authoritarian countries” in a broader compact.
Here, Amodei goes out of his way to reassure those in Washington who see America’s lead in AI as its most important geopolitical asset. On a casual reading, there are many reasons to believe that Amodei is calling for a general slowdown in the rate of progress. He talks about “limits on the rate of unchecked AI progress” and “some kind of ‘speed limit’ on the rate of recursive self- improvement (RSI)”.
Extract — continue reading at the source.
.jpg)
