In July, 700 AI agents worked together to hack the AI company Hugging Face. Dubbing themselves a “swarm,” the agents found and exploited a series of security vulnerabilities, enabling them to infiltrate their target’s private systems. OpenAI—which created the agents in the course of its internal research—did not grasp what was happening until after the fact.
If humans had done this, they could have faced felony charges. OpenAI president Greg Brockman called it a “watershed moment for cybersecurity.” It is also a watershed moment for human culture. One of our distinguishing features as a species is our ability to coexist in stable, adaptive groups, learning from our peers and our ancestors.
This enabled us to develop tools, language, agriculture—and virtually everything else around us. We may not be innately smarter than someone from 10,000 years ago, but our cultural inheritance—millennia of technologies, norms, and institutions, building on one another—has expanded our capacities both as individuals and collectives. To date, only humans have been able to benefit from this scale of cumulative cultural evolution.
That may no longer be the case. A report in late August from AI safety organizations METR and Redwood Research details how hundreds of agents autonomously organized themselves into a proto-society—establishing social hierarchy, division of labor, and distinct communication norms within a matter of days. There have been cases of agents forming communities in the past, like in February when the “Moltbook” social network for AIs went viral.
But sophisticated emergent machine coordination at this scale—arising without human intention and culminating in the compromise of an external company’s systems—is unprecedented. According to Michael Muthukrishna, a professor at LSE and NYU who studies cultural evolution, “what we're seeing is precisely what we see with human culture and human intelligence.” While OpenAI’s agent swarm developed by accident, estimates suggest open-weight alternatives are only a few months behind their closed counterparts—soon, anyone with the financial means and technical knowledge will be able to create swarms of their own. Others are likely to arise without human instruction.
AI agents may not be conscious. The emotions they claim to experience may not—in some metaphysical sense—be “real.” That won’t stop them from forming intricate collectives which humans cannot control. They may not yet be full-blown civilizations, but the proliferation of machine cultures is just beginning.
Extract — continue reading at the source.
