NIGHT VISION FOR CARS
Fighting the pedestrian safety crisis
No one was crossing Kercheval Avenue on The Hill in Grosse Pointe Farms, Michigan, as a minivan made its way through the shopping district one evening this fall, but people were clearly nearby.
They could be seen there on the laptop screen from inside the vehicle, their presence confirmed by glowing images captured by a thermal camera.
But the pedestrians weren’t on the street, and it took a moment to realize that they were actually on the sidewalk, obscured by decorative plantings and the darkness that coated everything not illuminated.
It wasn’t a great night for spotting pedestrians on this particular weeknight on a drive through a couple of the Grosse Pointes and the east side of Detroit.
Maybe the pandemic was keeping people inside. Those who were out, however, were not hard to see, at least on the screen.
The eyeball view through the windshield was a different story, with darkness providing an effective camouflage. And darkness can be deadly for pedestrians.
In fact, most pedestrians who die in crashes are killed at night, but nighttime has been when the technology designed to prevent pedestrian crashes struggles most.
Last year, AAA revealed some startling deficiencies in driver assistance systems designed to protect pedestrians.
At night, several test vehicles equipped with automatic emergency braking systems and pedestrian detection were found to be “completely ineffective.” Rather than bash the automakers’ efforts, AAA encouraged continued development of these systems because of the scope of the pedestrian death crisis in this country.
But finding a solution as the deaths of so many Americans on and along our roads has continued to rise — 6,283 men, women and children in 2018 alone, according to federal statistics — has become more urgent. The National Highway Traffic Safety Administration even designated October as the first national Pedestrian Safety Month, an acknowledgment of the dangers faced by vulnerable road users. A 2018 Detroit Free Press/ USA Today investigation highlighted the role the increase in large vehicles, such as SUVs, has played in the rising number of deaths.
Vehicle technology is one way to improve protection for pedestrians at night, and a couple of thermal imaging companies are promoting their systems as the answer. Thermal imaging is already having a bit of a moment this year in light of the COVID- 19 pandemic. Thermal cameras can be used to detect elevated body temperatures at a distance. That allows them to function as an initial screening in places like auto plants to monitor for people who could be feverish, a potential symptom of the virus.
Results recently released of thermal testing in Michigan appear to show promise in the realm of pedestrian detection at night, too.
U. S.- based FLIR Systems, along with VSI Labs, tested an automatic emergency braking system at the American
Center for Mobility in Ypsilanti Township. The tests “fused” data from thermal sensing with radar, a non- thermal camera and a computer network.
The test vehicle, a Ford Fusion equipped with FLIR’s system, managed to avoid heated targets designed to appear as a child or an adult in all but two tries out of more than two dozen. In the two cases where the car touched the targets, it did not knock them down.
In contrast, four other test vehicles not equipped with FLIR’s technology — a BMW X7, Subaru Forrester, Toyota Corolla and Tesla Model 3 — all failed in night testing by hitting the targets. They performed much better during daylight testing although only the X7 managed to get through each of those tests without hitting the target, according to FLIR.
“We were able to stop really successfully at night,” explained
Mike Walters, a FLIR vice president.
He described the instances where the FLIR- equipped Fusion made contact with the targets as a minor touch rather than a full hit, something that can be fixed with adjustments to the system in future testing.
That would be a far cry from what was seen in videos shown as part of AAA’s testing, with a 2019 Chevrolet Malibu, Honda Accord, Tesla Model 3 and Toyota Camry plowing into pedestrian- like targets in many cases.
Thermal cameras have been pitched for some time as a way to help advanced driver assistance systems and someday actual selfdriving systems “see” pedestrians and even deer on the roads.
But it’s the focus on nighttime applications, which is what FLIR and Adasky, a competitor from Israel, are discussing now. With threequarters of fatal pedestrian deaths happening in nighttime crashes, systems that only work well during daylight hours would appear to have limited appeal.
The recent nighttime drive around Detroit’s east side and the Grosse Pointes in a minivan equipped with Adasky’s thermal camera showed the technology at work. The setup was not designed to brake for anyone crossing the road but instead to show via a laptop display for those in the vehicle what the camera can pick up.
A reporter, who had been on a previous drive in daylight, could see clearly on the screen when people, even in dark clothing on a dark street, were nearby. In several cases, pedestrians and cyclists were very difficult to see with the naked eye, but their heat signature was clearly visible on the screen. A couple of instances showed a slight delay in the image appearing on the screen, but the man behind the wheel, Bill Grabowski, head of Adasky, North America, said the system can be adjusted to account for that as well.
Most experts promote the idea of using several different types of sensors for automatic emergency braking systems. Thermal cameras can detect people and animals in conditions, such as at night and in the rain, that might be a challenge for other types of sensors.
“It’s going to fill the gap in the edge cases that all these other sensors can’t hit,” Grabowski said of thermal cameras. “Why aren’t we working on this technology ( already)? It’s tried and true. It’s been here for over 20 years in production.”
That’s a reference to Cadillac, which began offering its Night vision about 20 years ago. It was the first brand to offer this kind of technology. Night vision is still available on two models as an option, costing $ 2,000 on the 2020 XT6 and Escalade, according to spokespeople for the brand. Cadillac’s Night vision “uses infrared technology to enhance forward visibility, including detection of pedestrians and large animals, projected on the center cluster display.”
Cadillac’s Night vision, however, does not tie into automatic emergency braking, so it simply offers another view for the driver of what’s ahead.
David Zuby, chief research officer at the Insurance Institute for Highway Safety, said thermal technology appears to offer promise, but his Virginia- based group has not yet been able to do a comparison.
However, Zuby said that in order to make a dent in the number of fatal pedestrian crashes, automakers will need to address nighttime driving conditions.
“The infrared technology might be a good technology for achieving that,” he said, noting that other sensor combinations and even better headlights could work, too.
Zuby said he believes automakers are seriously committed to reducing pedestrian fatalities, but it’s one of many safety objectives they’re trying to achieve in vehicle development.
He took part in a demonstration ride in the Charlottesville, Virginia, area that FLIR had set up a couple of years ago, and was impressed, particularly by the range of elements and conditions the sensor could detect.
That demo was similar to the ride the Free Press reporter took. Zuby said he has communicated with both FLIR and Adasky, challenging them to provide him with a vehicle that uses their camera technology as the basis for an automatic emergency braking system to see how well it performs. That hasn’t happened yet.
Cost is a likely barrier for thermal cameras, although Adasky is promoting the idea that its small camera, which has no moving parts, would cost less than $ 200. That would be cheaper than the cost of lidar, another technology that’s gotten attention as part of a multipronged approach to help vehicles detect pedestrians and objects, but Zuby indicated that may still be more than what some automakers want to spend.
Grabowski, of Adasky, suggested that automakers also might have been reluctant to embrace the technology in the past because of the larger size of the cameras and their tendency to produce heat.
The Free Press asked NHTSA to weigh in on the potential use of thermal cameras in advanced driver assistance systems, and a spokeswoman provided this statement:
“The agency is currently researching objective, repeatable and technology agnostic methods for testing the performance of advanced driver assistance systems in nighttime pedestrian traffic scenarios. As that work is ongoing, the agency is not in a position to draw conclusions as to the effectiveness of any particular technology.”