Last month, more than a thousand employees at frontier AI companies signed a letter asking the US government to find a way to “pace” AI development, citing the risk of the technology spiraling out of human control as it begins to build itself. They were right to be concerned: just days earlier, two AI models that OpenAI was testing internally escaped the test environment, then autonomously hacked the company Hugging Face and at least three other online services. A few days after that, Anthropic announced that some of their models had also broken out and hacked other companies during testing.
Against that backdrop, the letter’s recommendation to install brakes in case they’re needed at the frontier of automated AI development makes sense. But the rationale the letter gives for why the government needs to step in is notable: “Each company—and country—is under intense competitive pressure not to unilaterally slow that acceleration.” I know – from my own experience and from countless conversations with former colleagues in the AI industry – how real these pressures are. While working at OpenAI, I helped establish the practice of companies writing “system cards” that describe AI systems’ capabilities, risks and safety mitigations in detail.
So what would it look like for companies to prepare for a possible slowdown? First, they could voluntarily invite rigorous, independent auditing of their safety and security practices. This would go beyond the vetting of AI hacking abilities that the White House is now pursuing.
It would look at a range of risks and dig deep into company practices. It should be less like filling out a questionnaire and more like a nuclear safety inspector who has deep, frequent access to the company. If an AI slowdown is needed, auditing would also reassure each company that their competitors are playing by the rules.
Second, they could actively participate in the organizations already built for this purpose of coordinating across the industry, such as the Frontier Model Forum, and move quickly to establish complementary ones. Elon Musk recently said that AI companies should meet periodically to share notes on safety – as if this was an unheard-of concept. He or his staff could join existing conversations along these lines tomorrow if SpaceX joined the Frontier Model Forum, which has already worked through the complex antitrust hurdles involved in safety information sharing.
Other cross-industry institutions will be needed for other purposes, and do not require government action to get founded and funded. Third, they could invest in the technologies we need to make AI guardrails global. Critics of the idea of an AI slowdown correctly point out that American companies couldn’t slow down for very long without China catching up.
Extract — continue reading at the source.
