Extraordinary claims
ANDREW MAY argues that Carl Sagan’s famous quotation isn’t quite the sceptical mantra it’s usually thought to be...
In his 1980 TV series Cosmos, Carl Sagan famously said of UFOs that “extraordinary claims require extraordinary evidence.” The phrase has become something of a mantra for sceptics everywhere. That bastion of the status quo, Wikipedia – which handily refers to it by the acronym ECREE – has even elevated it to a fundamental philosophical principle, called the “Sagan Standard”. As a five-word soundbite, however, it’s pretty meaningless. In its original context it simply expresses Sagan’s view that UFOs don’t exist.
Yet if we trace it back to its roots, ECREE has a precise mathematical meaning – and it was first used to oppose dogmatic scepticism, not to support it. Whether Sagan knew it or not, he was paraphrasing a passage in A Philosophical Essay on Probabilities, published in 1814 by the brilliant polymath Pierre-Simon Laplace. Often referred to as the “French Newton”, Laplace was years ahead of his time, speculating on everything from black holes to the idea that mass extinctions are caused by cometary impacts. And unlike Newton, Laplace’s calculations showed the Solar System to be stable even without the intervention of an all-powerful deity. “I had no need of that hypothesis,” he’s reputed to have told Napoleon.
The Essay on Probabilities was ahead of its time too, dealing with the now-trendy topic of Bayesian inference. Using a theorem originally formulated in the mid-1700s by the Reverend Thomas Bayes, it was only when Laplace rediscovered it that its full potential was recognised
– and it wasn’t until the age of digital computers that it became a common tool in scientific circles.
Bayes’s formula shows how the odds of a theory being correct shorten as evidence is accumulated in its favour, or lengthen as evidence mounts up against it. In this context, Laplace’s essay refers to two of the hot fortean topics of his day: animal magnetism (a reiki-like healing therapy) and dowsing for metals or running water. While he acknowledges that the evidence for such phenomena may be equivocal, he has no time for knee-jerk sceptics who dismiss them out of hand: “It is natural to think that the action of these causes is very feeble, and that it may be easily disturbed by accidental circumstances; thus because in some cases it is not manifested at all its existence ought not to be denied. We are so far from recognising all the agents of nature and their diverse modes of action that it would be unphilosophical to deny the phenomena solely because they are inexplicable in the present state of our knowledge.”
This is where Laplace’s version of ECREE comes in: “We ought to examine them with an attention as much the more scrupulous as it appears the more difficult to admit them.” But unlike Carl Sagan (or Wikipedia), he doesn’t simply offer this as a blunt, take-it-or-leave-it aphorism. He proposes a rational, mathematical way to deal with such situations: “The calculation of probabilities becomes indispensable in determining to just what point it is necessary to multiply the observations, or the experiences, in order to obtain in favour of the agents which they indicate.”
In other words, Bayes’s theorem tells you how much evidence you need to accumulate before an “extraordinary claim” becomes the most viable explanation. Here’s a simple textbook example that shows how the principle works. A witness sees a night-time hit-and-run accident involving a taxi cab, which they say is blue. But there’s only one blue taxi in the city, the other 99 cabs being black. Without the witness statement, the odds against the guilty cab being blue are 99 to 1, making it an “extraordinary claim”. On the other hand, tests show the witness can correctly identify the colour of a car, under similar lighting conditions, nine times out of 10. Does this mean there’s a 90 per cent chance the cab really was blue, and only 10 per cent that it was black? No, because that ignores the original 99/1 odds against blue. Bayes’s theorem says that, in light of the witness testimony, you have to multiply these odds by the ratio of 10 per cent to 90 per cent – which still gives relatively long odds of 11/1.
So that single witness isn’t enough to confirm the “extraordinary claim” of a blue cab. On the other hand, if there were three independent witnesses, all equally reliable and all maintaining that the cab was blue, it would swing things the other way, making blue the 11/81 odds-on favourite. As far as Laplace was concerned, this would be sufficient “extraordinary evidence” to clinch the matter.
Unfortunately, you can’t apply analogous reasoning to the extraterrestrial visitors Carl Sagan was referring to, because you can’t assign meaningful odds to them. Either there’s evidence for them – evidence that stands up to scrutiny – or there isn’t. But if he’d put it that way, no one would have remembered the quotation.
NOTES 1 https://en.wikipedia.org/wiki/Sagan_ standard
2 For the full text, see http://www. gutenberg.org/ebooks/58881
3 Adapted from https://en.wikipedia. org/wiki/Representativeness_ heuristic#The_taxicab_problem
4 Just do the multiplication three times instead of once.
2 ANDREW MAY is a regular contributor to FT whose recent books include Rockets and Ray Guns and Astrobiology: The Search for Life Elsewhere in the Universe.