Here in Silicon Valley, the experts think that within the next couple of years we’ll see an extraordinary takeoff for artificial intelligence. More prosaically, the imminent breakthrough is described as “recursive self-improvement” – the point at which AI itself trains each successive model of AI, resulting in an exponential development to something which, in many significant respects, is more intelligent than us humans. As Robert Wright puts it in his book The God Test: “Never before has the near-term future … held such a wide array of not-implausible paths for humankind that would be so wildly transformative.” But will this be heaven or hell?
Heaven, says Elon Musk, who predicts “an age of amazing abundance” – although he also sees a 10-20% chance of killer robots murdering us all. Hell is more likely according to Geoffrey Hinton, one of the intellectual founding fathers of AI. And talking to Sebastian Mallaby, author of The Infinity Machine, a book about the quest for superintelligence, Hinton estimates p(doom) – the probability of human extinction – at 50%, “because I haven’t got a clue how to estimate the real number”.
But whether it’s God or Godzilla, artificial superintelligence is just around the corner. Now, maybe they’re all wrong – in which case, prepare for a crash in US financial markets, which are massively invested in a few big-spending tech companies gambling on a great leap forward to artificial general intelligence, a theoretical state of AI where systems gain human levels of intelligence. But elementary prudence demands that we should think fast and hard about the Pandora’s box of futures that eminent scientists and cutting-edge technologists assure us will soon be thrown open.
As a matter of fact, everyone from the Chinese president, Xi Jinping, to Pope Leo XIV has been weighing in on the opportunities and dangers of AI, and how to regulate it. Xi insists AI should be “always under human control” (and preferably that of the Chinese Communist party). Appealing explicitly to the “global south”, Beijing has inaugurated a World Artificial Intelligence Cooperation Organisation.
Meanwhile, the head of the world’s oldest and largest international NGO – the Roman Catholic church – discusses the AI challenge in his encyclical Magnifica Humanitas (Magnificent Humanity). Either we will build a new tower of Babel, the pope argues, or we will follow the example of the Jewish leader Nehemiah, who got the walls of Jerusalem rebuilt by involving the whole community in the project. But looking around the world today, I see more Netanyahus than Nehemiahs.
With salutary realism, a paper from the British thinktank Chatham House argues that it will take a major crisis to catalyse global coordination of AI governance. Unfortunately, I think even that sober prognosis is too optimistic. I’ve no idea what the p(doom) is, but I’m sure the p(somedisaster) is more than 90%.
Extract — continue reading at the source.
