Driverless cars come with moral dilemmas
Public needs to start talking about the choices vehicles will make, researchers say
Self-driving cars could save a million lives a year by eliminating the 90 per cent of car crashes caused by human error. But as autonomous vehicles proliferate on real-world roads, they will still inevitably face life-or-death dilemmas.
Should an autonomous vehicle’s algorithms be programmed to swerve and sacrifice its passengers if it means saving the lives of many pedestrians? Or should the car protect its occupants at all costs?
In a series of studies described in the journal Science, a trio of U.S. and French researchers tried to gauge the public’s response to this moral quandary — and discovered a typically human contradiction.
In six carefully designed online surveys, respondents voiced a strong moral preference for machines that would choose the greater good, sacrificing one or two passengers to save five or 10 pedestrians. But the respondents would not want to buy such a car themselves.
“You can kind of call this the tragedy of the algorithmic commons,” said Iyad Rahwan, a study co-author and a professor at MIT’s Media Lab.
“Even if you started off as one of the noble people who are willing to buy a self-sacrificing car, once you realize most people are buying self-protective ones, then you are really going to reconsider why you are putting yourself at risk to shoulder the burdens of the collective when no one else will.”
Even if these passenger-versus-pedestrian scenarios are rare — in fact, even if they never occur — these are the types of discussions the public needs to have, other researchers emphasized.
“We’re at the edge of an age where we’re programming behaviours into machines,” said Wendy Ju, executive director of interaction design at Stanford University’s Center for Design Research.
“One of the quandaries in this, and a moral dilemma in itself, is whether machines should necessarily be programmed the way everyone wants them to be. Because sometimes the crowds aren’t the most ethical decision-makers.”
The surveys found a consistent preference for self-sacrificing autonomous vehicles in general. For ex- ample, a car programmed to swerve and avoid hitting 10 or even two pedestrians while sacrificing its solo passenger received high moral approval. Even when participants were asked to imagine themselves and a family member as the car’s occupants, the approval of its morality dipped but remained above neutral.
Yet in these studies, the participants responded with a low likelihood of actually purchasing an autonomous vehicle that would sacrifice themselves and their family members for the greater good. They still thought the cars were programmed to do the right thing — they just didn’t want one for themselves. They also didn’t want the government to regulate autonomous vehicles to be self-sacrificing.
“That’s a big challenge to the widescale adoption of autonomous vehicles, especially when there’s already a basic fear about entrusting a computer program to zip us around at 60 miles an hour or more,” said Azim Shariff, a study co-author and direc- tor of the Culture and Morality Lab at the University of California Irvine.
Ju questioned whether these attitudes would truly hinder the adoption of autonomous vehicles or skew market forces: car manufacturers would not likely advertise how the vehicles would behave in these very rare scenarios. An accompanying perspective in the journal points out, however, that the greater ethical dilemma may be whether manufacturers even make the weighting of their algorithms transparent.
Moreover, many engineers believe autonomous vehicles will function more like a shared transit service. The public may accept a different risk profile for transportation that seems more like infrastructure than an individually owned commodity.
Either way, we have to ask the questions posed in the Science study, says Bertram Malle, co-director of the Humanity Centered Robotics Initiative at Brown University.
“These machines have to make decisions, and if we don’t think about it in advance, the machines will still do something,” Malle says. Humans generally dislike randomness, he points out. It is unlikely society will be comfortable putting these vehicles on the roads without any idea of how they will react in such scenarios, even if those scenarios turn out to be hypothetical.
Malle points out that autonomous vehicles are just the tip of the robothuman interaction iceberg. While the international community has extensively discussed the ethics of autonomous weapons, artificial intelligence in health care and robots that care for the sick and elderly are proliferating — yet there has been little discussion about the moral decisionmaking of robots in homes and hospitals. Interestingly, his own research has found that people have stronger moral expectations for robots than for humans, especially if the robots appear more machinelike than humanoid.