AI gone wrong
Here are five examples of AI incidents pulled from the database that prove the technology often needs more work before it’s used in the real world – though that’s already happening.
Incident #91: Vaccine allocations
Stanford Medicine used an algorithm to choose who would be the first 5,000 of its clinical staff to get the Pfizer vaccine. But only seven of the first wave were from staff who actually worked with Covid-19 patients, while the rest were doled out to higherranking employees at the medical school. Reports suggest Stanford’s leadership knew the algorithm hadn’t worked as hoped, but carried on with the decisions it had made regardless.
Incident #31: Driverless train crash
Back in 2017, driverless trains were set to open on Delhi’s metro system, but six days before the service was set to start, one train ploughed into a wall at the Kalindi Kunj station. No one was hurt and an inquiry was ordered. Although the trains were autonomous, drivers were expected to remain onboard.
Incident #35: Fired by algorithm
Ibrahim Diallo was fired by a smart system – but he wasn’t supposed to lose his job. Eight months into his three-year contract, Diallo’s logins stopped working and told the recruitment company he’d been fired. But he hadn’t been. After three weeks of investigation, his employer figured out that his manager had been laid off and sent home on garden leave. As the manager didn’t tick the box in the system to confirm Diallo’s contract was continuing, the system assumed he was fired and removed his access to the company for security reasons.
Incident #59: Google Translate forgets gender
Languages deal with gender in different ways, but the AI behind Google Translate lost some of that detail in translation. Researchers spotted that the system strips out female gendered phrases and changes them to male equivalents. For example, the German “Die Präsidentin”, meaning a female president, is outputted as “il presidente” in Italian instead of “la presidente”.
Incident #2: Robots with bear repellent problems
Workers in an Amazon warehouse in New Jersey were hospitalised in 2018 after a robot punctured a canister of bear repellent, sparking irritated eyes and breathing troubles. A similar incident, also with bear spray, happened the same year in Indiana – and in Texas in 2015. It’s unclear exactly why the
s ueezing such dangerous cans.