Fact-checking has an infinite-regress problem

The most highly trafficked post I ever wrote was about fake news on social media.

I dashed it off in perhaps 15 minutes, squatting on the floor of an airport between flights. I had seen a Martin Luther King Jr. quote going viral on Twitter after the death of Osama Bin Laden: “I mourn the loss of thousands of precious lives, but I will not rejoice in the death of one, not even an enemy.” I agreed with the sentiment, but I have a sideline in detecting fake quotes on the internet, and this felt … wrong. It didn’t appear on any of his quote pages. It also didn’t appear in any of his writing, as far as I could tell. I wrote a blog post pointing out that this seemed to be fake, and boarded my plane.

By the time I got settled at my destination, the post had started to blow up. Apparently, every time someone posted or tweeted the King quote, someone else would offer a rebuttal consisting of a link to my writing.

The story of the fake turned out to be completely innocent; someone had offered this, their own thought, followed by a Martin Luther King quote, and the magical power of the internet had somehow mashed them together. Much fake news is, of course, not so benign. Some people think Russia is creating fake news to destabilize Western governments. Other fake news is generated by scammers looking to cash in on gullible partisans. With people claiming that fake news may have helped get Donald Trump elected, there’s pressure on social media organizations to crack down on this phenomenon. And so Facebook has rolled out a new initiative to make it easier for people to report fake news, and has tapped a group of fact-checking organizations to help vet the stories people report.

I’m not against this initiative, exactly. But it does leave me with a version of the question asked by the Roman poet Juvenal: Quis custodiet ipsos custodes? Who will guard the guardians? Who will factcheck the fact-checkers?

The problem, as multiple critics of fact-checkers have pointed out, is that determining what constitutes a “fact” is often not so easy. If I say that 42 clowns showed up at my house in Washington D.C. this morning and danced the merengue in my front yard, we can be certain that no such thing occurred, because 42 clowns could not even stand in my tiny front yard.

But what if I say that my house has lost a great deal of value due to the poor urban management of Mayor Muriel Bowser? That’s a hard claim to evaluate. I have not attempted to sell my home during the administration of Muriel Bowser, nor did I during the previous administration, so we don’t know what has happened to the price. We could look at sale prices of similar homes, but those will not be an exact match. Moreover, I could retort that I am comparing my home’s value not to that of a few years ago, but to a counterfactual world in which Bowser wasn’t elected.

Think that’s a crazy thing to describe as a “fact”? Policy analysis traffics in these sort of counterfactuals all the time. And that’s the sort of thing fact-checkers are frequently called upon to evaluate.

The currency of politics is what we might call “dubious statements” -- things that have some basis in truth, but which, through sins of omission or commission, are spun into better support for one’s cause than the original material really offers. They are not as clearly false as, say, me claiming to be the Queen of Slovenia. But they are biased.

Numerous people have argued, correctly in my view, that fact-checking sites share the center-left slant of the mainstream media itself. Dubious statements need to be clarified by context, and fact-checkers often seem to be more generous in providing the context for liberal speakers than for conservative ones. Nor, as Juvenal wryly suggested millennia ago, is there any permanent solution for that problem. If you appoint guardians for the guardian class, then you have to worry about your superguardians.

Correcting for that bias is a tricky business, because of course, the fact-checkers themselves have biases.

Numerous people have argued, correctly in my view, that fact-checking sites share the center-left slant of the mainstream media itself. Dubious statements need to be clarified by context, and factcheckers often seem to be more generous in providing the context for liberal speakers than for conservative ones. Nor, as Juvenal wryly suggested millennia ago, is there any permanent solution for that problem. If you appoint guardians for the guardian class, then you have to worry about your superguardians.

You can, of course, limit the problem. The more narrow the powers of the guardians, the less you have to worry about abuse. Facebook seems to understand this, since they apparently intend to confine their fact-checks to “very clear-cut falsehoods.” But just how clearly can we cut?

Take the example I started with. I’m reasonably sure that those words were never recorded in King’s official works. Can I be certain he never said them? Of course not; the man didn’t live his life with a tape recorder running and a transcriptionist standing ready 24/7. It’s possible that I “debunked” a true statement. It is, to be sure, not likely, but it’s possible. Can we definitely say that viral meme was “fake news”? And might your opinion on that be influenced by your opinion on whether it’s moral or wise to celebrate the death of another human being who happens to be a loathsome terrorist?

As Vox’s Timothy B. Lee points out, the fact-checkers Facebook is working with “won’t have a ‘half true’ or ‘mostly false’ option. They’re going to have to decide which news stories to rate as fake -thereby branding them as ‘disputed’ in the Facebook newsfeed -- and which ones to leave alone.”

And here we reach the ultimate paradox of Juvenal’s epigram. The more narrow the questions to which fact-checkers confine themselves, the less room they will have for their own biases to matter -but the more fake news they will leave floating around social media. They can be trustworthy or they can be useful, but they may not be able to manage both. And there we are: We need guardians, and we cannot entirely trust them. It’s a high-tech version of a problem that has been with us for at least 2,000 years.