In recent weeks, leaders of some of the biggest AI companies have warned that the very tech they are developing is dangerous. Last weekend, Anthropic CEO Dario Amodei argued that AI carries serious risk and that progress should be slowed. OpenAI CEO Sam Altman responded on X: “I agree with Dario that we need to pace the frontier.” Those posts came a few days after the AI researcher Jacob Coxon announced that he was leaving a role at Anthropic, charging that neither it nor OpenAI (where he had also worked) was acting responsibly.
Another Anthropic employee, Evan Hubinger, publicly agreed with him. Let’s take a closer look at why. A bioweapon might be a highly lethal virus that targets people according to their genes.
It could be a fungus that wipes out a crop and causes food insecurity. Perhaps it would be a tasteless, odorless toxin that could be slipped into a region’s water supply, undetected. The concern is that AI tools can be used to help generate agents like these.
In 2022, researchers at Collaborations Pharmaceuticals found that it was remarkably easy to do so using an AI “molecule generator” they’d developed to find potential drugs for human disease. In less than six hours, the model generated 40,000 molecules with the potential to serve as chemical warfare agents. Some of them were designed to be even more toxic than known nerve agents.
It was a wake-up call for David Magnus, a professor of medicine and biomedical ethics at Stanford University, even though he had been assessing the risks associated with the misuse of medical science and biotechnology since the late 1990s. Anyone can use large language models trained on the knowledge and experience of “almost every scientist who ever lived on this planet,” says Dunja Sabra, a biosecurity researcher at the University of Hamburg in Germany. Those models can provide instructions and video training on how to conduct experiments.
Combine that with advances in biotech that have made gene editing and synthetic biology tools much more accessible (the “DIY biology” movement has already enabled many people to set up labs at home), and you’ve got a potentially very dangerous situation. There are safeguards in place. People who want to build new genomes must typically order the pieces of DNA from companies that screen for suspicious requests.
Extract — continue reading at the source.
