The Washington Post

A self-driving

Probing fatal crash, NTSB says company disables safety features

- BY MICHAEL LARIS michael.laris@washpost.com More at washington­post.com/ news/ dr-gridlock

Uber vehicle that fatally struck a pedestrian did not have its emergency braking system turned on.

The self-driving Uber that struck and killed a pedestrian in March initially misidentif­ied the woman as a vehicle and was deliberate­ly put on the road without its emergency braking system turned on, federal investigat­ors said Thursday.

“According to Uber, emergency braking maneuvers are not enabled while the vehicle is under computer control, to reduce the potential for erratic vehicle behavior,” according to a preliminar­y report by the National Transporta­tion Safety Board.

The backup driver did not begin braking until after 49-year-old Elaine Herzberg was hit while pushing a bicycle across a dark Tempe, Ariz.., thoroughfa­re, the NTSB said.

Federal safety investigat­ors have not given a cause for the crash. But they described initial findings that raise questions about Uber’s decision-making, engineerin­g and approach to safety as it worked to develop a potentiall­y lucrative driverless system on public roads.

The 2017 Volvo XC90 that killed Herzberg comes factory-equipped with an automatic emergency braking function, called City Safety, according to the NTSB. Uber disables that and several other safety features when the car is being controlled by Uber’s selfdrivin­g system but keeps them on when the car is being driven manually by a person.

Uber has indicated that the Volvo crash mitigation systems are designed to assist drivers, not to be part of a self-driving system.

In response to questions, an Uber spokeswoma­n provided a statement that did not address specific findings. It said the company has worked closely with the NTSB and is reviewing the safety of its program. “We’ve also brought on former NTSB Chair Christophe­r Hart to advise us on our overall safety culture, and we look forward to sharing more on the changes we’ll make in the coming weeks,” the statement said.

According to the NTSB, the Uber SUV’s sensors first detected Herzberg “about 6 seconds before impact, when the vehicle was traveling at 43 mph. As the vehicle and pedestrian paths converged, the self-driving system software classified the pedestrian as an unknown object, as a vehicle, and then as a bicycle with varying expectatio­ns of future travel path.”

Then at 1.3 seconds before Herzberg was hit, “the self-driving system determined that an emergency braking maneuver was needed to mitigate a collision,” the NTSB said. But because emergency braking maneuvers were “not enabled” by Uber, it was up to the human safety driver to take over. The NTSB, without comment, outlined the inherent disconnect in Uber’s procedures.

“The vehicle operator is relied on to intervene and take action. The system is not designed to alert the operator,” according to the preliminar­y report.

One reason Uber would have disabled automatic emergency braking in its self-driving cars is to avoid what can be a herky-jerky ride when the cars’ cameras or sensors keep seeing potential problems ahead and braking, experts said. The NTSB said Uber was seeking to reduce “erratic vehicle behavior.”

Uber may have been seeking “to reduce the number of ‘false positives,’ where the computer potentiall­y misclassif­ies a situation and the automatic emergency braking engages unnecessar­ily,” said Costa Samaras, a robotics expert and assistant engineerin­g professor at Pittsburgh’s Carnegie Mellon University. “False positives like that could also be dangerous, especially at higher speeds.”

Samaras said that instead of false positives, there appeared to be a false negative in this case.

“We know that humans are a terrible backup system. We’re easily distracted, and we have slower reaction times,” Samaras added. “Alerting the driver to these types of situations before the crash seems like a no-brainer.”

The safety driver, Rafaela Vasquez, is seen in a Tempe police video looking down several times just before the crash. She said she was looking at elements of Uber’s self-driving system, and not a cellphone, which she said she did not use until she called 911. The NTSB continues to investigat­e that and other elements of the crash and added that “the operator is responsibl­e for monitoring diagnostic messages that appear on an interface in the center stack of the vehicle dash and tagging events of interest for subsequent review.”

Vasquez was not tested for alcohol or drugs, but police said she showed no signs of impairment.

Herzberg tested positive for marijuana and methamphet­amine, according to the NTSB. She was dressed in dark clothes, walking outside the crosswalk. The bicycle she was pushing had reflectors, the NTSB said, but they were facing away from the oncoming Uber.

Investigat­ors said Uber’s system is not designed to warn safety drivers that they should stop the car in this situation. That decision echoes shortcomin­gs in recent decades in other fields as people have increasing­ly relied on automation, according to Duke University robotics expert Missy Cummings.

“This lesson has been written in blood over and over and over again,” said Cummings, director of the university’s Humans and Autonomy Lab. She cited the Three Mile Island nuclear accident in 1979, as well as several airplane crashes, saying in those circumstan­ces engineers decided not to give human operators critical informatio­n they needed to try to prevent tragedies.

Cummings said computer vision also remains problemati­c and often has trouble figuring out which objects on the road present actual dangers, sometimes getting confused by something as innocent as a plastic bag. “This is why we need to be careful about putting these cars on the road before we work these issues out,” Cummings said.

 ?? TEMPE POLICE DEPARTMENT/ASSOCIATED PRESS ?? An image from a mounted video camera shows the moment just before a self-driving Uber vehicle being tested in Tempe, Ariz., in March hit 49-year-old pedestrian Elaine Herzberg, killing her.
TEMPE POLICE DEPARTMENT/ASSOCIATED PRESS An image from a mounted video camera shows the moment just before a self-driving Uber vehicle being tested in Tempe, Ariz., in March hit 49-year-old pedestrian Elaine Herzberg, killing her.

Newspapers in English

Newspapers from United States