Artificial intelligence (AI) has been used to search for breakthrough drugs to help improve human life, but a new study has found a huge potential for creating deadly biochemical weapons instead.
Researchers at Collaborations Pharmaceuticals used a conference at the Swiss Spiez Laboratory to examine how technologies for drug discovery could be misused.
And in just six hours of operating their model produced more than 40,000 potential deadly molecules, including some closely related to nerve agent VX.
The researchers, in a report entitled 'Dual use of artificial-intelligence-powered drug discovery' for the journal Nature Machine Intelligence, said they had never previously considered the use of AI for nefarious purposes.
"We were vaguely aware of security concerns around work with pathogens or toxic chemicals, but that did not relate to us," they wrote.
"Our work is rooted in building machine learning models for therapeutic and toxic targets to better assist in the design of new molecules for drug discovery.
"We have spent decades using computers and AI to improve human health - not to degrade it. We were naive in thinking about the potential misuse of our trade."
Normally the company's molecule search system, 'MegaSyn', uses machine learning to predict the bioactivity of the new drugs it finds.
Their usual model penalises predicted toxicity and rewards molecules that fit the bioactivity they are looking for. This time they inverted that logic and rewarded toxicity instead.
They trained the AI using molecules that are both synthesizable and likely to be absorbed with data from a public database, specifying a lethal dose level to narrow the field slightly.
To focus it further, they chose to drive the model towards compounds such as VX, one of the most toxic chemical agents there is. Just a few salt-sized grains can kill a person, the researchers said.
The 40,000 they found in just a few hours was something of a wake-up call to the researchers.
"In the process, the AI designed not only VX, but also many other known chemical warfare agents," the designers said.
"Many new molecules were also designed that looked equally plausible. These new molecules were predicted to be more toxic than publicly known chemical warfare agents. This was unexpected because the datasets we used for training the AI did not include these nerve agents.
"Our proof of concept thus highlights how a nonhuman autonomous creator of a deadly chemical weapon is entirely feasible. The results should serve as a wake-up call to colleagues," the researchers said.
"The reality is that this is not science fiction. We are but one very small company in a universe of many hundreds of companies using AI software for drug discovery," they wrote.
"By going as close as we dared, we have still crossed a grey moral boundary, demonstrating that it is possible to design virtual potential toxic molecules without much in the way of effort, time or computational resources.
"We can easily erase the thousands of molecules we created, but we cannot delete the knowledge of how to recreate them."
