Google bias tweets are dubious

In a pair of early-morning tweets, President Donald Trump took aim at Google’s system of search results, saying it was biased against outlets that are friendly to him.

In two successive tweets on Aug. 28 (later deleted and reposted to correct a spelling error) Trump wrote:

“Google search results for ‘Trump News’ shows only the viewing/reporting of Fake New Media. In other words, they have it RIGGED, for me & others, so that almost all stories & news is BAD. Fake CNN is prominent. Republican/Conservative & Fair Media is shut out. Illegal? 96% of … results on ‘Trump News’ are from National Left-Wing Media, very dangerous. Google & others are suppressing voices of Conservatives and hiding information and news that is good. They are controlling what we can & cannot see. This is a very serious situation-will be addressed!”

The part where Trump said that 96 percent of “results on ‘Trump News’ are from National Left-Wing Media” caught our eye.

A close look at the source of the statistic suggests that Trump’s figure is dubious.

Where does the figure come from?

The White House didn’t respond to an inquiry.

The 96 percent figure appears to come from a post on PJ Media, a conservative site. The Aug. 25 story had attracted some attention in conservative media circles before getting picked up by Trump.

“Is Google manipulating its algorithm to prioritize left-leaning news outlets in their coverage of President Trump?” Paula Bolyard, PJ Media’s supervising editor, wrote in the article. “It sure looks that way based on recent search results for news on the president.”

To test the premise, Bolyard performed a Google search for “Trump“using the search engine’s “News” tab and analyzed the results using a media bias chart.

Of 76 total stories, most came from outlets such as CNN, the Washington Post and NBC. Only three came up from the Wall Street Journal, which is labeled right-leaning by the chart.

Bolyard wrote that she repeated her search “multiple times using different computers (registered to different users) and Google returned similar results.”

Bolyard acknowledged that while the study is “not scientific,” she did conclude that “the results suggest a pattern of bias against right-leaning content.”

But this conclusion is debatable, and Trump’s reporting of the results is more problematic still.

Factors in Google searches

There is broad agreement that the factors that go into Google’s algorithms for news searches are opaque. And the lack of transparency can allow theories about ideological bias to flourish.

They “depend on a lot of factors, including what other people are searching for, what they’re clicking on, what sites link to a given search result, whether the website is optimized for mobile, and even whether the site supports encrypted HTTPS,” said Jeremy Gillula, tech policy director at the Electronic Frontier Foundation. “But with all that said, Google’s search algorithms are still a black box, and we’d prefer if Google gave users more information and control over the factors that influence search results.”

In a statement, Google said that ideological factors do not play a role in its algorithms.

“When users type queries into the Google Search bar, our goal is to make sure they receive the most relevant answers in a matter of seconds,” the statement said. “Search is not used to set a political agenda, and we don’t bias our results toward any political ideology. Every year, we issue hundreds of improvements to our algorithms to ensure they surface high-quality content in response to users’ queries. We continually work to improve Google Search and we never rank search results to manipulate political sentiment.”

Even if one considers Google’s algorithmic factors to be “biases,” that doesn’t necessarily mean they are ideological biases. Searches, for instance, are widely believed to favor newer content and original sources rather than aggregations.

At the very least, there should be a fair amount of variability in search results over time. A study made during one snapshot in time, such as the one published in PJ Media, will not produce the same results a few hours later, as new articles come online.

Bolyard, in an email interview with PolitiFact, noted that she mentioned some of the caveats in her article. In the article, she cautioned that “factors such as the relevance of the topic, the design of the website, internal and external links, and the way articles are written and formatted all can affect a site’s Google traffic. Google is constantly tweaking their algorithm, and a website’s traffic prospects can rise or fall depending on the changes.”

However, Trump made no indication in his tweets that the findings could be chalked up to anything beyond ideological bias.

Rating news outlets’ ideology

A more problematic aspect of the study is the metric used to label websites.

To determine “left” and “right,” Bolyard used a chart of media outlets assembled by Sharyl Attkisson, a former CBS correspondent who hosts the Sunday morning program “Full Measure” for Sinclair Broadcasting, a network of local stations that has taken a pro-Trump editorial stance.

The chart is not neutral evidence supporting Trump’s point, and it labels anything not overtly conservative as “left.” In the “left” category are such rigorously mainstream outlets as the Associated Press and Reuters.

The three big broadcast networks — ABC, NBC, CBS — are considered “left,” as are the Washington Post and the New York Times. Other media outlets that produce a large amount of content every day, including CNN, NPR, Politico, USA Today, and CNBC, are labeled “left.”

This means that most of the media entities that produce large amounts of widely read and cited news content — factors that might well give them a leg up in the Google algorithm — are defined by the chart, and thus by the PJ Media study, as “left-wing.” So, by its design, the study guarantees that an enormous percentage of Trump news coverage would fall into what the chart defines as “left.”

Full disclosure: PolitiFact and its owner, the Poynter Institute, are both listed on the chart as left of center. We actually consider ourselves not to have an ideological leaning.

“It’s a ‘study’ designed to come up with the outcome it came up with, and it did just that,” Chip Stewart, a professor at Texas Christian University’s Bob Schieffer College of Communication.

Moreover, the “left” category lumps these large-staffed, comprehensive news outlets in with genuinely liberal-to-left outlets, such as the Daily Kos, Salon, the Nation, Huffington Post, Talking Points Memo, and Media Matters.

A more revealing comparison would be to compare search results of truly left-wing sites vs. truly right-wing sites.

In a written description of the chart, Attkisson acknowledged that the graphic is “subjective” and writes that “position on the chart doesn’t necessarily imply credibility or lack thereof. Sources on far right and far left have, in many instances, produced excellent, factually correct information at times.”

Bolyard agreed that “any chart that attempts to quantify media bias is going to be inherently subjective.” But she added that “while it may be fair to quibble with Attkisson’s assertion about how far left-of-center some major media outlets are, I think it’s entirely fair to say that outlets like NBC, the New York Times, and CNN skew to the left, especially in their coverage of the president.”

Stewart, however, rejected the notion that the algorithm has ideological bias.

“A far more plausible explanation is that the items that drive the algorithm — links, shares, etc. — are more common among the so-called liberal publications, which have a long track record of credible journalism and try to cater to a wider audience,” he said.

Either way, Trump used the 96 percent figure without taking into account any of these methodological concerns.

Our ruling

Trump tweeted that “96% of (Google News) results on ‘Trump News’ are from National Left-Wing Media.”

This figure is based on a nonscientific study from a conservative website that categorized any media outlet not expressly conservative as being part of the “left.” These outlets include wire services, broadcast networks and most major newspapers and collectively account for a large percentage of original news reports produced in the United States. The methodology essentially preordains that a large percentage of coverage captured by Google will be what the study defines as “left,” which is wrong.

We rate the statement False.