Is it a bird? Is it a plane? Is it a pedestrian?

Future Merc tech doesn’t just look – it sees

You drive past a children’s playground. You’re half expecting a ball to come rolling out into the road. There’s a movement in the corner of your eye but you dismiss it instantly because the way it’s moving tells you it’s an empty McDonald’s bag caught by the breeze, not a ball being pursued by a child. A dog is secured by a lead to its owner, but the cat it’s growling at could spring in any direction.

And so on… You scroll through this sort of scenario many times on every journey without thinking about it. And now Daimler, working with Bosch, is teaching cars to think this way. The aim is to develop more advanced self-driving and car-sharing technology; to improve tra c flow in cities; and to enhance road safety. Daimler bo n Dr Uwe Franke spells out some of the limitations: ‘The system must be told what is wrong and what is right, otherwise it cannot learn. This is why we consciously refer to image interpretation as “machine learning” rather than “artificial intelligence”. Our engineers specify what is to be learned. It does not occur to the system itself to have a look at what lies behind the next hill.’

The key innovations are around ‘sensor fusion’ – the knitting together of inputs from sources including radar, lidar, stereo cameras and ultrasound sensors, all working at lightning speed, plus real-time tra c information about congestion, accidents and roadworks.

The sensor-fusion software is smart enough to analyse a pedestrian’s gait and infer from that their intended direction. Daimler is also working on an algorithm capable of detecting if a person is on their phone and not paying attention. Each type of sensor has weaknesses, but cross-checking with inputs from the other sensors builds up a more complete picture.

Later this year the project will step up a gear when a fleet of autonomous cars hits the road in California, and an automated shuttle service will go live in the San Francisco Bay area.