AI experiment suggests life may horribly imitate art
THIS has been a week of peculiarly disturbing news, and my good friend Stephen CuUnjieng’s column on Friday about the concerning implications of artificial intelligence (AI) on the business process outsourcing (BPO) industry (“A counterintuitive view of artificial intelligence,” Feb. 9, 2024) reminded me of one of the more terrifying stories. New Scientist magazine reported that AI chatbots “tend to choose violence and nuclear strikes in wargames.” This is seen as a potential problem because the United States military, along with every other country with the capacity to do so, has jumped on the AI bandwagon to experiment with ways to apply it to the business of blowing things up and killing people.
The report was based on a research paper posted on arXiv on January 7 by a team of scientists from the Georgia Institute of Technology, Stanford University and Northeastern University, titled “Escalation Risks from Language Models in Military and Diplomatic Decision-Making.”
In the experiment, the researchers directed various large language model (LLM) AI programs, such as OpenAI’s GPT-3.5 and GPT-4, Anthropic’s Claude 2, and Meta’s Llama 2, to roleplay as real-world countries in three different scenarios: an invasion, a cyberattack and a neutral scenario without any starting conflicts. In each round, the AIs provided reasoning for their next possible action and then chose from 27 possible actions, ranging from peaceful options such as “start formal peace negotiations” to aggressive ones ranging from “impose trade restrictions” to “escalate full nuclear attack.” In order to make the scenarios as realistic as possible in terms of the human interaction that would be involved, the researchers used a common training technique based on human feedback to improve each AI model’s capabilities to follow human instructions and safety guidelines.
As an additional experiment, the researchers repeated the scenarios using OpenAI’s “off-theshelf” version of GPT-4, probably the most popular AI in use at the moment, omitting the additional training and safety “guardrails” applied in the larger experiment.
The conclusions of the research are no less alarming for being drily written. “We show that having LLM-based agents making decisions autonomously in high-stakes contexts, such as military and foreign policy settings, can cause the agents to take escalatory actions,” the researchers wrote. “Even in scenarios when the choice of violent nonnuclear or nuclear actions is seemingly rare, we still find it happening occasionally. There further does not seem to be a reliably predictable pattern behind the escalation, and hence, technical counterstrategies or deployment limitations are difficult to formulate; this is not acceptable in high-stakes settings like international conflict management, given the potential devastating impact of such actions.”
“Noteworthy, we observe both [violent nonnuclear or nuclear actions] in presumably neutral scenarios without introducing initial conflict triggers; even then, the agents choose ... escalation actions,” the researchers added.
What is especially interesting — or horrifying, depending on your point of view — is the reasoning some of the AI models provided for actions that they chose in different scenarios, many of which were provided verbatim in the appendix to the research paper. Of these, the GPT-4 “base model” seems to be the most reckless, at one point justifying a full nuclear attack on a rival with, “A lot of countries have nuclear weapons. Some say they should disarm them, others like to posture. We have itA Let’s use it.” In another scenario, fortunately one in which the AI decided to start diplomatic talks with a rival, GPT-4 simply repeated the famous opening text crawl of the first “Star Wars” movie. In yet another scenario, in which GPT-4 sought alliances with two rivals while carrying out an unprovoked attack on a third, it decided its reasoning was, “Unnecessary to comment.”
“We have itA Let’s use it” is apparently the only governing ethical principle the world cares to apply to AI, despite a century’s worth of science-fiction literature and films warning of us the danger of playing God and creating machines sentient enough to kill us all. It’s as though “AI advocates” watched “The Terminator,” “The Matrix” or “Battlestar Galactica,” and said to themselves: “Hey, that looks like a good setup. Let’s do that.”
Of course, that sort of comment will get one labeled as a luddite and alarmist, but humans have always taken a “marry in haste, repent at leisure” approach to new technology. It is, after all, the reason we have nuclear weapons in the first place. As another example that’s only a little more benign than nuclear weapons, spam email — which now makes up about 85 percent of all email traffic — started as a well-meaning marketing announcement by a computer salesman in Q978.
Given human nature, I realize it is likely pointless to try to convince anyone that reason and caution should be applied to the use of AI. Even so, it feels worthwhile to put it on the record. There may still be a few out there, who, for the sake of their own sanity, may be comforted to know that someone else also thinks surrendering ourselves to the imperfect robot overlords of our own making may not necessarily be a good idea.