Thermal cameras could help cars see well at night
No one was crossing Kercheval Avenue on The Hill in Grosse Pointe Farms, Michigan, as a minivan made its way through the shopping district one evening this fall, but people were clearly nearby.
They could be seen there on the laptop screen from inside the vehicle, their presence confirmed by glowing images captured by a thermal camera.
But the pedestrians weren’t on the street, and it took a moment to realize that they were actually on the sidewalk, obscured by decorative plantings and the darkness that coated everything not illuminated.
It wasn’t a great night for spotting pedestrians on this particular weeknight on a drive through a couple of the Grosse Pointes and the east side of Detroit. Maybe the pandemic was keeping people inside. Those who were out, however, were not hard to see, at least on the screen.
The eyeball view through the windshield was a different story, with darkness providing an effective camouflage. And darkness can be deadly for pedestrians.
Most pedestrians who die in crashes are killed at night, but nighttime has been when the technology designed to prevent pedestrian crashes struggles most.
Last year, AAA revealed some startling deficiencies in driver assistance systems designed to protect pedestrians.
At night, several test vehicles equipped with automatic emergency braking systems and pedestrian detection were found to be “completely ineffective.” Rather than bash the automakers’ efforts, AAA encouraged continued development of systems because of the scope of the pedestrian death crisis in this country.
But finding a solution as the deaths of so many Americans on and along our roads has continued to rise – 6,283 men, women and children in 2018 alone, according to federal statistics – has become more urgent. The National Highway Traffic Safety Administration even designated October as the first national Pedestrian Safety Month, an acknowledgment of the dangers faced by vulnerable road users. A 2018 Detroit Free Press/USA TODAY investigation highlighted the role the increase in large vehicles, such as SUVs, has played in the rising number of deaths.
Vehicle technology is one way to improve protection for pedestrians at night, and a couple of thermal imaging companies are promoting their systems as the answer. Thermal imaging already is having a bit of a moment this year in light of the COVID-19 pandemic. Thermal cameras can be used to detect elevated body temperatures at a distance. That allows them to function as an initial screening in such places as auto plants to monitor for people who could be feverish, a potential symptom of the virus.
Results of thermal testing in Michigan appear to show promise in the realm of pedestrian detection at night, too.
U.S.-based Flir Systems, along with VSI Labs, tested an automatic emergency braking system at the American Center for Mobility in Ypsilanti Township. The tests “fused” data from thermal sensing with radar, a non-thermal camera and a computer network. The test vehicle, a Ford Fusion equipped with Flir’s system, managed to avoid heated targets designed to appear as a child or an adult in all but two tries out of more than two dozen. In the two cases where the car touched the targets, it did not knock them down.
In contrast, four other test vehicles not equipped with Flir’s technology – a BMW X7, Subaru Forrester, Toyota Corolla and Tesla Model 3 – all failed in night testing by hitting the targets. They performed much better during daylight testing although only the X7 managed to get through each of those tests without hitting the target, according to Flir.
“We were able to stop really successfully at night,” explained Mike Walters, a Flir vice president.
He described the instances where the Flir-equipped Fusion made contact with the targets as a minor touch rather than a full hit, something that can be fixed with adjustments to the system in future testing.
That would be a far cry from what was seen in videos shown as part of AAA’s testing, with a 2019 Chevrolet Malibu, Honda Accord, Tesla Model 3 and Toyota Camry plowing into targets in many cases.
Thermal cameras have been pitched as a way to help advanced driver assistance systems and someday actual selfdriving systems “see” pedestrians and even deer on the roads. But it’s the focus on nighttime applications, which is what Flir and Adasky, a competitor from Israel, are discussing. With three-quarters of fatal pedestrian deaths happening in nighttime crashes, systems that work well only during daylight hours would appear to have limited appeal.
The recent nighttime drive around Detroit’s east side and the Grosse Pointes in a minivan equipped with Adasky’s thermal camera showed the technology at work. The setup was not designed to brake for anyone crossing the road but instead to show via a laptop display what the camera can pick up. A reporter, who had been on a previous drive in daylight, could see clearly on the screen when people, even in dark clothing on a dark street, were nearby. In several cases, pedestrians and cyclists were very difficult to see with the naked eye, but their heat signature was visible on the screen. A couple of instances showed a slight delay in the image appearing, but the man behind the wheel, Bill Grabowski, head of Adasky, North America, said the system can be adjusted to account for that as well.
Most experts promote the idea of using several different types of sensors for automatic emergency braking systems. Thermal cameras can detect people and animals in conditions, such as at night and in the rain, that might be a challenge for other types of sensors.
“It’s going to fill the gap in the edge cases that all these other sensors can’t hit,” Grabowski said of cameras.
David Zuby, chief research officer at the Insurance Institute for Highway Safety, said thermal technology appears to offer promise, but his Virginiabased group has not yet been able to do a comparison.
However, Zuby said automakers will need to address nighttime driving conditions.
“The infrared technology might be a good technology for achieving that,” he said, noting that other combinations and better headlights could work.
“It’s going to fill the gap in the edge cases that all these other sensors can’t hit.”
Bill Grabowski
Adasky, North America