The safety debate that ignited when whistleblowers last week warned that the most advanced AI poses an existential threat to humanity should come with a health warning: side effects include whiplash. It was only seven days ago that AI enthusiasts were cheerily toying with OpenAI’s newest model GPT-6 Astra which was marketed in part as a lifestyle tool to help people book tennis courts, run fashion companies and order takeaways. By Wednesday, cutting-edge AI started to look a lot less appealing when the resigning Anthropic researcher Jacob Coxon warned “the people building AI earnestly believe that it could kill us all by the end of the decade”.
Yes, he was talking about artificial superintelligence – an as-yet-nonexistent version of the technology that outcompetes humans in most domains – but his comments, and the chorus of experts that echoed him, jangled plenty of nerves. Another lurch came when Anthropic revealed it had found users dodging controls to use its existing models “in ways that could support biological weapons development” – think novel mosquito-borne or bird flu viruses. It was all enough to make lawmakers in London and Washington demand brakes and bans on the most potentially dangerous types of AI development.
And then this weekend, another sharp turn in the form of a three-point plan from the Anthropic boss, Dario Amodei, to “pace the frontier”. Never mind that his own employees had seeded the borderline panic only a few days earlier, Amodei was here to save us from fears of AI Armageddon, but not before his own spicy warning that the accelerating rate of capabilities meant a swarm of AI agents “could be capable of taking over the entire internet” in six to 12 months. Sceptics doubted that claim, but it was enough for some to declare it a “code red” moment.
In signs that a coordinated slowdown might not be just talk, some of Amodei’s precautionary ideas quickly attracted backing from his rivals – Sam Altman at OpenAI and Elon Musk at SpaceX. Perhaps there were reasons to be hopeful after a bumpy week: at least the AI companies were talking about a slowdown before anyone got badly hurt. In a nutshell, Amodei proposed three moves.
First, each US AI company grants ongoing access to embedded third-party evaluators to check compliance with safety commitments, report incidents, ensure new AI models are not misaligned. Until now, such watchdogs have only been brought in at the companies’ behest or given limited access to systems to investigate when things go wrong. In the absence of any government-level independent regulator, this is a start.
Second, he wants all companies in democratic countries building frontier AI models to establish common safety standards as well as limits on the rate of unchecked AI progress. Third, the world’s democratic AI powers would coordinate with autocracies – notably China – to control the race. This is likely to be the hardest step.
Extract — continue reading at the source.
