Failure to pick out pedestrian may have caused fatal Uber crash

The tragedy of the first person killed by an autonomous vehicle points to a potential vulnerability with the nascent technology now being tested on the open roads: While robo-cars, powered by sophisticated sensors and cameras, can reliably see their surroundings, the software doesn’t always understand what it detects.

New details about the Uber Technologies autonomous vehicle that struck and killed a woman in Tempe, Arizona, indicate that neither the self-driving system nor the human safety driver behind the wheel hit the brakes when she apparently stepped off a median and onto the roadway at around 10 p.m., according to an account the Tempe police chief gave to the San Francisco Chronicle. The human driver told police he didn’t see the pedestrian coming, and the autonomous system behaved as if it hadn’t either.

Experts say that the sophisticated sensors on the autonomous vehicle almost certainly detected the woman pushing her bicycle laden with bags along the median, close to the road. But it’s possible the car’s lidar and radar sensors, which scan the surroundings for objects, may not have realized it was detecting a person. (Uber did not immediately respond to a request for comment.)

“The real challenge is you need to distinguish the difference between people and cars and bushes and paper bags and anything else that could be out in the road environment,” said Matthew Johnson-Roberson, an engineering professor at the University of Michigan who works with Ford Motor Co. on autonomous vehicle research. “The detection algorithms may have failed to detect the person or distinguish her from a bush.”

Driverless cars “see” the world around them using data from cameras as well as radar and lidar sensors that bounce laser light off objects to assess shape and location. High-speed processors crunch the data to provide a 360-degree view of lanes, traffic, pedestrians, signs, stoplights and anything else in the vehicle’s path. That’s supposed to enable the vehicle to know, in real time, where to go and when to stop. But pedestrian identification remains a major challenge for self-driving systems.

Autonomous vehicles also struggle to master the elements. Snow, ice and even rain can obscure sensors and render the most advanced computing power useless. That’s one reason most self-driving cars are being tested in sunny climates like Arizona and Texas. Early autonomous cars have been knocked for creating hazards by rigidly following the rules and overly cautious driving.

“You need to distinguish the difference between people and cars and bushes and paper bags”

In the case of the fatal collision in Tempe, the dark conditions aren’t likely to have played a role.