Earlier in August, cryptography professor Matthew Green wrote a controversial thread on X and a longer blog post that went viral within the cybersecurity community. Green, who has long been a close observer of the debate around the use of hacking tools by governments to fight crime and the need for strong encryption to protect the privacy of innocent people, posited a provocative thought: What if AI makes bugs so scarce that law enforcement and intelligence agencies are unable to lawfully hack criminals anymore? Law enforcement have long claimed that encryption made it difficult to catch criminals and terrorists.
The concept of “going dark” was popularized in 2014 at a time when then-FBI director James Comey warned that encryption could hamper authorities from being able to listen in on conversations or access data on devices. Around this time, apps like Signal, WhatsApp, and Apple’s iMessage rolled out end-to-end encryption to the masses, making traditional real-time wiretapping of calls and text messages almost impossible. Tech giants like Apple also began making data on their devices encrypted by default, making it harder to break into iPhones protected by a strong PIN code or passphrase.
Since then, authorities have still been able to catch criminals — including by hacking into their devices — and innocent people have been able to enjoy a good level of privacy thanks to encryption. In part, as Green explains, that is because of “an uneasy kind of truce.” That is, instead of incorporating backdoors into devices to help authorities get the data, governments have instead invested money into buying hacking tools and spyware that can subvert the security of devices and their owners. For Green, that truce is about to be disrupted by AI, because, as proponents promise and some early data suggests, LLMs are becoming better and faster at finding security vulnerabilities at scale.
That, in theory, suggests we will get to a point where companies can make their software and systems significantly less bug-ridden — and prone to attacks. The end result, per Green, is that governments could ask for backdoors again, making everyone’s devices less secure by design. We asked several people to chime in on Green’s argument, from privacy and cybersecurity experts to hackers who have experience developing hacking tools for governments.
Some agree with Green, some disagree, and some see it both ways. Luna Tong, a researcher who has previously worked at two prominent companies that search for bugs and develop exploits to help governments break into systems, agreed with Green, saying that there is a “gold rush of bugs right now but it’s a temporary phenomenon and bugs will get scarce again soon.” Another researcher, who has more than a decade of experience working at offensive security firms, said that he is worried AI could make human security researchers obsolete because it will be much harder to find bugs, and that defenders will eventually have the edge over offensive researchers. The person asked not to be named so that they could speak more freely.
Stagno explained that the current process of requiring governments to exploit security flaws to break into devices is the “most democratic system we have,” but that the status quo may not last if bugs become too hard to find. Three other people who currently work in the offensive cybersecurity industry, and one who used to, disagreed. Their arguments boil down to: Easy bugs will be easier to find; more complex bugs that are generally more valuable and useful for governments will not go away; and, AI can actively assist the researchers who sell bugs to government authorities.
Extract — continue reading at the source.
