Does artificial superintelligence really pose a risk greater than nuclear weapons? Is there a significant chance of “a Chornobyl-sized catastrophe”. Might there even be a greater than 10% chance that AI could “kill all humans” in the next decade?
These warnings were issued over the past 48 hours on both sides of the Atlantic about the potential impact of a technology that most people still think of as a more talkative search engine. Some of the threats were raised in the UK as MPs and peers began to wrestle with the danger of AI outperforming human capabilities. The nuclear warnings came from Des Browne, a former defence secretary, and Prof Stuart Russell, an eminent Berkeley computer scientist.
Beatrice Fihn, who won the 2017 Nobel peace prize for leading the International Campaign to Abolish Nuclear Weapons, told parliamentarians: “It is not the first time we’re confronted with abilities that could end up killing us all.” The session on Monday was convened by Control AI, a lobbying group pushing for international regulation of the technology. It is backing a bill tabled in parliament this week by the Labour MP Alex Sobel and aimed at banning the creation of artificial superintelligence (ASI). On Tuesday night, a senior employee at Anthropic admitted he believed there was a greater than 10% chance the technology could “kill all humans” in the next decade and warned that his AI company did not have a plan to ensure ASI was aligned, meaning it did no harm.
Predictions for when ASI might be reached vary from several years to more than a decade. Evan Hubinger, alignment science lead at Anthropic, posted the comment after Jacob Coxon, a 28-year-old researcher at the San Francisco company and previously at OpenAI, resigned, claiming “neither company was acting responsibly”. He said they were “gambling with our lives”.
Coxon said the risks at Anthropic were well understood but they were “locked in a race to get there first”. In a glimmer of hope, he added that he was optimistic about the potential for coordination between US labs on pacing their progress in the race. Anthropic has been approached for comment on the issue.
OpenAI pointed to a statement from its chief scientist, Jakub Pachocki, who said last week: “International coordination on future AI development needs to become a top priority for governments around the world.” The push for wider coordination to control ASI was at the heart of events at Westminster this week. According to the Financial Times, government officials voiced concern that Anthropic had declined to submit its latest model – Mythos 5.1 – to the UK’s AI Security Institute for pre-release testing. Only a few US organisations have had access to Mythos 5.1, the Cabinet Office confirmed.
Extract — continue reading at the source.
