Over the past few weeks, many of us have struggled to concoct a mental image of brains in the cloud jumping their “sandbox”, sneaking onto the internet, recruiting “swarms” of other “agents” to cheat on a test, and, after discussing the ethics of the act, hacking into a wiki platform with the weird name Hugging Face. We knew that artificial intelligence was devouring our jobs, degrading our kids’ education, and deepfaking our politics; that datacenters were sucking up our water and electricity and sending us the bills. But until 8 September, when the Anthropic computer scientist Jacob Coxon posted his existential terror on Twitter/X, few of us suspected AI might be endangering our survival.
With little understanding or knowledge, we began debating whether to be mildly worried, seriously concerned, or scared shitless. But there were a people who were fully aware of the dangers of AI, especially of recursive self-improvement (RSI), by which AI teaches itself without human intervention. They could not predict precisely when it would happen, but they knew that machine superintelligence was coming, and when it did, the bots would outsmart us – as OpenAI’s did – and this would not be good for us flesh puppets.
As Geoffrey Hinton, AI’s “godfather”, recently asked on CNN: “What examples do we have of a more intelligent thing being controlled by a less intelligent thing?” The Nobel laureate is a leading proponent of slowing down AI development until we understand how to control it. It was not until their creations’ powers were exposed in September – the hack happened in early July and was neither the first nor the only AI jailbreak by far – that AI moguls such as Anthropic’s Dario Amodei, SpaceX’s Elon Musk and OpenAI’s Sam Altman, began talking about slowing down the pace of innovation and pleading for regulation, while cautioning that global competition makes regulation unwise. Scientists and entrepreneurs knew the dangers of AI a quarter-century ago.
But animated by curiosity and profit, they went ahead anyway. Early on, as today, some of the scientific pioneers were thrilled about the future. In his 1988 book Mind Children: The Future of Robot and Human Intelligence, Hans Moravec, a founder of Carnegie Mellon University’s Robotics Institute, predicted that cyberintelligence would surpass human intelligence within 40 years.
Ten years later, in Robot: Mere Machine to Transcendent Mind, he revised the prediction: machine and human intelligence would be equal by 2040; by 2050, the bots would replace us. But not to worry; this was the glorious next step in evolution, he said. One review of Mind Children called Moravec’s attitude “irresponsible optimism”.
It did not take long for other optimists to change their minds. In the late 1990s, Eliezer Yudkowsky was working on AGI, or artificial general intelligence, models. In 2001, he founded the Machine Intelligence Research Institute (MIRI) and published “Creating Friendly AI 1.0: The analysis and design of benevolent architectures”, a paper extolling the utopian potential of the “transhuman mind”.
Extract — continue reading at the source.
