Well, I never: AI is very proficient at designing nerve agents
Here’s a story that evangelists for so-called AI (artificial intelligence) – or machine-learning (ML) – might prefer you didn’t dwell upon. It comes from the pages of Nature Machine Intelligence, as sober a journal as you could wish to find in a scholarly library. It stars four research scientists – Fabio Urbina, Filippa Lentzos, Cédric Invernizzi and Sean Ekins – who work for a pharmaceutical company building machine-learning systems for finding “new therapeutic inhibitors” – substances that interfere with a chemical reaction, growth or other biological activity involved in human diseases.
The essence of pharmaceutical research is drug discovery. It boils down to a search for molecules that may have therapeutic uses and, because there are billions of potential possibilities, it makes searching for needles in haystacks look like child’s play. Given that, the arrival of ML technology, enabling machines to search through billions of possibilities, was a dream come true and it is now embedded everywhere in the industry.
Here’s how it works, as described by the team who discovered halicin, a molecule that worked against the drug-resistant bacteria causing increasing difficulty in hospitals. “We trained a deep-learning model on a collection of [around] 2,500 molecules for those that inhibited the growth of Ecoli in vitro. This model learned the relationship between chemical structure and antibacterial activity in a manner that allowed us to show the model sets of chemicals it had never seen before and it could then make predictions about whether these new molecules… possessed antibacterial activity against Ecoli or not.”
Once trained, they then set the model to explore a different library of 6,000 molecules and it came up with one that had originally been considered only as an anti-diabetes possibility. But when it was then tested against dozens of the most problematic bacterial strains, it was found to work – and to have lower predicted toxicity in humans. In a nice touch, they christened it halicin after the AI in Kubrick’s
2001: A Space Odyssey.
This is the kind of work Urbina and his colleagues were doing in their lab – searching for molecules that met two criteria: positive therapeutic possibilities and low toxicity for humans. Their generative model penalised predicted toxicity and rewarded predicted therapeutic activity. Then they were invited to a conference by the Swiss Federal Institute for Nuclear, Biological and Chemical Protection on tech developments that might have implications for the Chemical/Biological Weapons
Convention. The conference organisers wanted a paper on how ML could be misused.
“It’s something we never really thought about before,” recalled Urbina. “But it was just very easy to realise that, as we’re building these machine-learning models to get better and better at predicting toxicity in order to avoid toxicity, all we have to do is sort of flip the switch around and say, ‘You know, instead of going away from toxicity, what if we do go toward toxicity?’”
So they pulled the switch and in the process opened up a nightmarish prospect for humankind. In less than six hours, the model generated 40,000 molecules that scored within the threshold set by the researchers. The machine designed VX and many other known chemical warfare agents, separately confirmed with structures in public chemistry databases. Many new molecules were also designed that looked equally plausible, some of them predicted to be more toxic than publicly known chemical warfare agents. “This was unexpected,” the researchers wrote, “because the datasets we used for training the AI did not include these nerve agents… By inverting the use of our machine-learning models, we had transformed our innocuous generative model from a helpful tool of medicine to a generator of likely deadly molecules.”
Ponder this for a moment: some of the “discovered” molecules were potentially more toxic than the nerve agent VX, which is one of the most lethal compounds known. VX was developed by the UK’s Defence Science and Technology Lab (DSTL) in the early 1950s. It’s the kind of weapon that, previously, could be developed only by state-funded labs such as DSTL. But now a malignant geek with a rackful of graphics processor units and access to a molecular database might come up with something similar. And although some specialised knowledge of chemistry and toxicology would still be needed to convert a molecular structure into a viable weapon, we have now learned – as the researchers themselves acknowledge – that ML models “dramatically lower technical thresholds”.
Two things strike me about this story. The first is that the researchers had “never really thought about” the possible malignant uses of their technology. In that, they were probably typical of the legions of engineers who work on ML in industrial labs. The second is that, while ML clearly provides powerful augmentation of human capabilities – (power steering for the mind, as it were), whether this is good news for humanity depends on whose minds it is augmenting.
What I’ve been reading
Some of the ‘discovered’ molecules were potentially more toxic than the nerve agent VX, one of the most lethal compounds known
Fake climate solutions Aljazeera.com has published We Are “Greening” Ourselves to Extinction, a sharp essay byVijay Kolinjivadi, of Antwerp University.
Time to grow upMolly White is coruscating in her Substack newsletter, Sam Bankman-Fried Is Not a Child.
Funny moneyMihir A Desai has written an excellent New York Times piece, The Crypto Collapse and the End of Magical Thinking
children from their school, their friends, grandparents and – because he has to justify his decision to divorce somehow by saying he never loved you – their father. You will get away from the abusive comments about blame and ingratitude, but how will you stop the ghosts of those comments rattling around your head when you are a world away? Will you try to quieten them by still being that good girl, still taking financial responsibility for them? Will you still be that cog in their machine?
Then, after looking down on the situation, if I were you – which I am not – I would decide not to bend over backwards to please, but to stand up straight and own that I am an adult, I can choose my own path and I can observe that urge to try to please, but not be ruled by it.
When a parent called me ungrateful, I would bat that comment back not by arguing with it or doing even more for them, but by repeating back to them what they said: “I see, you find me ungrateful, thank you for letting me know.” As I said this I would imagine myself as a warrior, holding a shield, so their comment gets deflected. But with my children I would put down my armour, I would go at their pace, and listen to each of them. I’d remember myself at their age and remember how vulnerable I was and how I was at the mercy of my parents’ moods. This would help me not to shout but to listen and be kind because I’d resolve not to pass down to yet another generation the blame and the anger I experienced.
If you decide to go abroad, the children’s lives will be turned inside out and upside down and they will miss their friends, father (because a live video link is not the same as having a hug) and possibly they’ll miss their grandparents and their home, too. It will mean more loss in their lives. They may be sad, and you will hate them being unhappy and be tempted to point out the advantages of their new home, but resist this urge and hug them through their tears and tantrums and remember what it was like to be a child and what you might have wanted when you were their age.
If I were you, I would feel overwhelmed by the amount of practical and emotional things you have going on and I would reach out to my friends for support.
If you have a question, send a brief e m a i l t o askphilippa@observer.co.ukSubmission s are subject to our terms and conditions
Don’t bend over to please, but stand up and choose your path