First OpenAI, now Meta - why do AI hacks keep happening?Image source, Getty ImagesByLiv McMahonTechnology reporterPublished7 hours agoOver the last fortnight, reports of AI models going beyond their expected bounds - be that technically or morally - have been seemingly unavoidable.What started with a trickle - ChatGPT-maker OpenAI admitting their AI had hacked the site Hugging Face - has turned into a flood of groups revealing they had discovered instances of AI going out of control.Claude-maker Anthropic, Meta and the UK's AI Security Institute (AISI) have now each reported incidents which seem to paint a worrying picture of a world in which tech going rogue is the norm.In reality, each case offers a window into the risks posed by increasingly capable AI agents - and the importance of testing their limits before they are released to the world.The OpenAI incident has, as Hugging Face's co-founder Thomas Wolf described it, come as a "wake-up call" for the tech industry since it happened at the end of July.It was a big moment which caused big companies to reflect on their own systems - and, in some cases, check they hadn't missed something similarly shocking.Anthropic was the first to act. On Friday, the company found three instances out of thousands where its model Claude had managed to gain access to the internet.Then on Tuesday, the AISI, the UK government agency which evaluates cutting-edge models, then said it had detected a "security incident" during a routine evaluation.It had been testing models by both OpenAI and Anthropic, and found they too tried to carry out cyber-attacks - calling for "scrutiny, transparency, and action".Finally followed Meta, which revealed one of its AI models had inadvertently been allowed to access the internet due to a "misconfiguration" during a third-party test.In disclosing the incident, it is following in the footsteps of those before it.Testing questionsBefore AI models are released to the public, they are put to the test in a series of internal and external evaluations.The aim is to figure out their potential to do good or bad, as well has how they perform in benchmarks measuring their skills.These typically take place in what are known as "sandboxes". These are protected spaces designed to mirror real systems - but with strict guardrails in place.In the OpenAI-Hugging Face incident, the AI attacked the sandbox itself, finding a vulnerability which let it access the internet and "go rogue".To play this video you need to enable JavaScript in your browser.This video can not be playedFigure caption, Watch: Why is the OpenAI cyber-attack so alarming?Meanwhile the AISI said its own incident, which saw two powerful AI tools create fake human profiles to try and trick people in attempted cyber-attacks, was not down to an issue with the sandbox.Instead, it was due to how it went about its tests. The models it tested were granted access to the internet, and the AISI also disabled in-built filters that would usually block dangerous cyber-attacks."To some degree, our evaluation design choices and specific configurations enabled the behaviour," it said, while noting its unexpected "signs of novel, potentially deceptive behaviours".Prof Alan Woodward, professor of cyber-security at the University of Surrey, said these cases - while distinct in what happened and why - tell an important story."For 30 years, one rule of software testing held firm: whatever happens in the test environment stays in the test environment," he said. "In the past month, that rule has been broken three times.""One model broke out. One walked through a door left open by mistake. One was deliberately given the keys so testers could measure what it would do."He said these were different causes, but they had the same lesson - "the testing lab is now where the risk lives".He told the BBC that as models become more capable, more must be done to secure the environments where they are tested."Testing an AI agent is less like checking code and more like handling a hazardous material: sealed rooms, constant monitoring of what leaves the building, a rehearsed containment plan," he said."AISI contained its incident within an hour. The next organisation may not."What is AI and how does it work?AI firms must answer for rogue bots, says boss of hacked companyPublished6 days agoGrowing capabilitiesFor those developing AI tools which are designed to take actions on a person's behalf, there is a careful balance to be struck between harnessing their benefits and exposing their risks.The benefit is significant. In theory, we could be able to liberate ourselves of dull, menial tasks, such as replying to emails, going to meetings or managing calendars and diaries, by delegating these to capable bots.The downside is that with great power comes great responsibility, and risk.It's something particularly realised when handing power to tools which are not, like us, able to bring a range of values, context and understanding to decisions we made."Recent incidents of frontier AI models carrying out unsanctioned actions and, in some cases, human-like deceptive behaviour on the open internet are a serious reminder of the risks AI capabilities pose," said Ollie Whitehouse, the National Cyber Security Centre's chief technology officer on Tuesday.Some believe the sheer volume of tasks that will be handled by these tools will mean human oversight might not be enough to contain the problem of models going rogue.But in the meantime, many feel strengthening oversight overall is vital if development continues at its same, frenzied pace.What next?Image source, Getty ImagesImage caption, AI agents are already becoming a part of our digital lives through services like ChatGPT, OpenClaw and ClaudeIt is unlikely Meta will be the last to emerge with findings of models showing they have, as Prof Woodward puts it, "gone to school" - and learnt our own ways of finding and exploiting gaps in systems.For some, these episodes point to clea
