How reliable is forensic science?
Flawed scientific evidence is causing controversy in the United States. Is New Zealand immune, asks Nikki Macdonald.
Kiwi forensic researcher Niki Osborne keeps fielding calls from American media. ‘‘What about the bloodstain pattern analysis in that series
The Staircase?’’ they ask. Stuff is the first New Zealand media to call. You could read that two ways – maybe New Zealand is immune to the growing global disquiet about the reliability of forensic science. Of 350 wrongfully convicted Americans exonerated by DNA, 45 per cent had unvalidated or improper forensics as a contributing cause, according to The Innocence Project. Of those, 20 were sentenced to death.
Or maybe we’re only now catching up.
Osborne has just returned from 18 months doing postdoctoral research at the University of California, investigating how human fallibility can bias forensic results. All that stuff on CSI about forensics incontrovertibly fingering the killer – reality isn’t quite as clean-cut.
A good illustration is a University College London study, in which five fingerprint analysts were given print pairs to re-examine and told that the FBI had mistakenly deemed them a match. The samples were, in fact, pairs the experts had themselves judged as matching five years earlier, but only one expert reached the same conclusion as their earlier self.
Using crime scene blood spatter to reconstruct what happened is the latest forensic technique to come under fire. Osborne co-wrote a research paper led by Environmental Science and Research scientist Michael Taylor, which asked bloodstain pattern analysts to interpret crime scene spatter. Overall, 13 per cent of bloodstains were wrongly categorised.
However, that changed depending on the case information given, such as witness reports, the position and injuries of the victim and any weapons found.
When the case summary was deliberately misleading, the examiners’ error rate rose to 20 per cent. And when the case information pointed towards the correct pattern, the error rate fell to 8 per cent.
While Osborne says the error rates can’t be extrapolated to all blood spatter analysis, the results were still concerning: ‘‘It’s like whoa, what do we do about this?’’
However, she wasn’t surprised that contextual case information influenced analysts’ conclusions, because a similar bloodstain pattern on a wall could be caused by coughing, by someone being beaten, or by someone being shot.
‘‘The danger zones are when there’s ambiguity in the data, when there’s distortion, when there’s information missing within the evidence itself – really smudged fingerprints, or complex bloodstain patterns . . . When there is a rich contextual environment – at a crime scene, if you were analysing data with a whole lot of case information – if that context is coupled with the ambiguity, and the subjective methodology, that’s when you’re getting into the danger zone.’’
Osborne’s work follows on from the 2016 President’s Council of Advisers on Science and Technology (PCAST) report in the United States, which investigated the scientific validity of seven forensic methods, such as fingerprint and shoeprint analysis, that rely on matching patterns. It rejected bitemarks and concluded microscopic hair analysis, shoeprints and firearms analysis all lacked robust studies proving their reliability (see sidebar).
Behind those doubts are ruined lives. Santae Tribble spent 20 years in prison after an FBI expert found 13 crime scene hairs matched his ‘‘in all microscopic characteristics’’. DNA analysis later revealed none belonged to Tribble and one came from a dog.
There have been concerns, too, about just how expert forensic experts really are. A New York Times investigation found you could give blood pattern evidence in court in the United States after taking a 40-hour course.
So how does New Zealand shape up?
‘‘Assume nothing, check everything, believe nobody,’’ says forensic scientist Anna Sandiford, of The Forensic Group. She gave evidence for the defence in the David Bain and Mark Lundy murder cases and is part of the trust investigating potential miscarriages of justice in New Zealand.
It would be naive to think New Zealand was immune to flawed forensics, she says. ‘‘We’ve had some very difficult and very demonstrable miscarriages of justice, and I don’t necessarily think they are
exceptions . . . There are three main contributors to miscarriages of justice – poor police investigation, poor defence and poor forensic science. If you’ve got one or more of those in any particular case, the chances of a miscarriage increases.’’
Forensic science in New Zealand is generally good, Sandiford says. There is bias, but in a small country you know people’s strengths and weaknesses. She avoids using American experts because they have some of the worst – as well as the best – in the world.
And in spite of PCAST’s fears about some forensic methods, she wouldn’t rule anything out. Even the widely rubbished bitemarks could be useful, if someone had particularly distinctive teeth and the mark was in a rigid material, not skin.
It’s about transparency of process, and knowing what to question, Sandiford says. Take Europe’s phantom serial killer, whose DNA turned up at six
murders. It turned out the DNA belonged to a woman manufacturing cotton swabs used in crime scene examination kits.
‘‘Forensic science is good, but it can make mistakes. But if we don’t know where to look, then we won’t find them. And if we just accept everything at face value, we will just believe what we’re told.’’
The Australian and New Zealand Forensic Science Society said the PCAST report highlighted valid concerns about some forensic science, but was ‘‘overly simplistic’’ and specific to the US. However, it failed to specify why those concerns would not apply here.
ESR forensic programme manager Jill Vintiner can’t point to any changes ESR has made after the PCAST report, but says New Zealand is different from the US, both in its methods and the way experts present evidence in court. And of the methods
questioned, ESR uses only firearms analysis, shoeprints and analysis of DNA mixtures.
ESR is technically independent, although its experts work almost exclusively for the police.
Its service is internationally accredited and uses blind peer review to combat bias, Vintiner says. That means a second examiner reviews the same evidence, without knowing what the first scientist concluded. If they disagree, it’s recorded on the case file for a defence lawyer to see.
American firearms examiners choose between just three conclusions – identification, inconclusive, and elimination. ESR experts use a 13-point scale, from conclusively supporting the prosecution case, to completely excluding the prosecution’s scenario.
However, Vintiner maintains shoeprints can be linked to an individual shoe – an assertion PCAST found was ‘‘unsupported by any meaningful evidence’’.
If charged with a crime she didn’t commit, Vintiner would be sceptical of bitemark analysis being used in any attempt to exonerate her. But she’d be confident in the use of footprints, tyre marks, fingerprints and shoeprints.
Osborne has worked in labs around the world and says ESR is up there with the best in terms of culture and competence. Critically, its examiners are all scientists, which is not true in the US.
Like Sandiford, Osborne is wary of rejecting any methods outright. Shoeprints carry little weight if you’re looking for a size 10 Reebok sneaker of which The Warehouse has just sold 10,000. But if it’s a shoe that was never sold here, that evidence becomes more useful.
Police, who undertake fingerprint analysis, would not respond to Stuff’s questions about forensic reliability within the week given, saying they instead needed two months.