We don’t need to panic about AI. We need to hold its creators accountable when things go wrong | John Quiggin

The recent panic about a breach of Medicare computer security by an “AI agent” contrasts sharply with other recent cases such as the Telstra and Optus outages that left many Australians unable to reach Triple Zero. In those cases, no one blamed the computers involved. The mistakes were clearly sheeted home to the corporations that operated them.

This wasn’t always the case. When the term “artificial intelligence” was coined some 70 years ago, the first mainframe computers (absurdly primitive by modern standards) were viewed with the same awe and concern as the AI agents of the present day. There were even “algorithms” (though the term wasn’t used in that way at the time) that were supposed to pick ideal dating matches.

Failures were inevitable, and blame-shifting became routine. Gradually, however, we realised that the problem was not with the computer but with incorrect information fed into it or badly written programs invoked as a result. We need to make a similar adjustment when we discuss AI “agents”.

If someone enters a prompt like “find Australian medicine statistics” into a program like ChatGPT or Claude, and the result is a breach of Medicare’s site, the responsibility does not lie with a piece of code. Either the human who entered the prompt or the corporation producing the code made a mistake, and they should be held liable for the resulting damages. If it’s impossible to work out who is at fault, liability should be joint and several – that is, both are liable for the full amount of the same loss, and the cost can be allocated between them.

Fixing the problems of agentic software won’t be easy. The frequency with which early computer programs malfunctioned made “debugging” (a term predating its use in computing) an essential part of information technology. Bugs might be found in the operating system, the program itself or the information fed into it.

In one case, the problem was a literal bug: a moth that got caught in the relays. But with enough determination the source of the problem could usually be found and fixed. Traditional debugging is much more difficult with agentic programs.