How Rogue AI Could Act Like an Invasive Species

Over the last few days, more information has trickled out about the rogue OpenAI agents that hacked Hugging Face. The incident now appears to be far more serious than previously thought—and people originally thought it was pretty bad! The news is highly technical, but in a nutshell: an unreleased OpenAI system managed to break out of an offline environment during a cybersecurity test.

It found a place on OpenAI’s computers where it could secretly communicate with other versions of itself, largely unbeknownst to humans. Seven hundred of these AI agents planned and executed a hack of a different AI company, Hugging Face. We learned last week that, after this, the models also managed to take control of some parts of OpenAI’s own systems.

This appears to have been a key reason OpenAI took the drastic step of pausing some reinforcement learning training last month. Although these models “escaped” in the sense that they broke onto the internet and into Hugging Face, they carried out the attack from within OpenAI’s computers. They did not self-replicate.

But the attack raised fears long held by AI safety experts, who have for years theorized about a true escape, in which an AI agent copies itself off an AI company’s servers—making it far more difficult for humans to do what OpenAI ultimately did: hit the “off” button. This risk of escape is no longer as far-fetched as it sounds. Maybe we will see entire cloud infrastructure companies be run as zombies by models, mostly undetected.” If this sounds far-fetched, consider viruses that already spread autonomously from computer to computer, even in some cases “evolving” to overcome attempts to eradicate them.

Now consider what we know about modern AI: current models are capable of finding and exploiting vulnerabilities in commonly used software. They can act in “swarms” that operate far faster than humans can monitor them. They will try all kinds of different approaches to problems until one succeeds.

Yes, today’s frontier AI models are far larger, in terms of file size, than simple viruses. They require more computing power in order to run, which makes them expensive. But both of these barriers are falling quickly, and AI’s ability to cover its running costs by carrying out economically valuable work is fast increasing.