Concerns surrounding the potential for an AI-driven apocalypse are intensifying from the tech hubs of Silicon Valley to the corridors of Washington, DC.
One of the prevalent theories about how a rogue AI could lead to humanity's demise involves the misuse of biological weapons.
This past summer, leading figures in the AI industry advocated for stricter regulations on synthetic DNA production. In a recent study by researchers at Stanford University and the Arc Institute, findings revealed that AI has the capability to create new viral genomes. The topic gained further traction last week following a report from Anthropic, which noted attempts to utilize their AI, Claude, in ways that could facilitate biological weapons development. The report authors labeled biological misuse as “one of the most serious risks of frontier AI models.” Subsequently, CEO Dario Amodei called on the government to assist AI labs in managing their advancements responsibly.
Despite the growing concerns regarding an AI-generated pandemic, many experts consider the risk to be relatively minor in the grand scheme of global issues.
David Bellamy, a research scientist at the Institute of Foundation Models in Sunnyvale, California, argues that AI does not represent a fundamentally new threat in terms of bioweapons creation. The scientific community has long navigated the dual-use dilemma, which contemplates whether to publish research that could be misappropriated for harmful purposes.
Before the rise of large language models, the internet and open-access scientific journals, along with translation services like Google Translate, had already facilitated widespread access to biological information, including laboratory techniques, according to Bellamy.
“AI serves as a tool that enables both beneficial users, like researchers, and malevolent actors to access information more efficiently and pinpoint relevant protocols quickly,” he explains. “However, these abilities do not constitute the main obstacle in bioweapon production.”
Bellamy points out that the true challenge lies in constructing a virus or biological agent, which necessitates specialized materials, equipment, and expertise. For a malicious actor to create a harmful agent, they would need to procure the appropriate gene fragments, assemble a complete genome, and ensure that the resulting virus could infect humans, cause illness, and spread from person to person. While robotic lab assistants can streamline processes, a human expert is still essential for conducting experiments and gathering the necessary resources.
Jason Kelly, CEO of the biotech startup Ginkgo Bioworks, believes it is improbable that an artificial general intelligence (AGI) could take control and distribute existing pathogens, let alone engineer and deploy new ones. Ginkgo Bioworks is known for its autonomous laboratories and recently collaborated with OpenAI, utilizing its GPT-5 model to operate a lab.
“The AI couldn’t dominate the lab operations,” he points out, noting that human staff could simply refuse to provide the requested materials needed to develop a bioweapon. The presence of humans as critical checks means that “you would require a significantly greater number of robots to achieve the level of control needed for AGI to conduct the intricate virology tasks involved.”
Immunologist Derya Unutmaz echoes this sentiment, asserting that humans would be able to thwart attempts by AGI to create a lethal pathogen. He argues that even if a highly intelligent AI were to somehow release a supervirus, scientists would likely leverage other AI tools to rapidly create a vaccine.
However, even those who downplay the immediate threat of AGI autonomously launching a bioweapon acknowledge concerns about how nefarious actors might exploit AI technology, as well as the uncertainties that lie ahead.
