The Washington Post

Twitter unable to build misinforma­tion safeguards that match its influence

Complaint filing by whistleblo­wer, the former head of security, details firm’s failings in policing posts


In the weeks leading to Twitter’s launch of a fact-checking program to combat misinforma­tion, experts at the company warned managers that the project could be easily exploited by conspiracy theorists.

Those warnings — which went unheeded — almost came true. The night before the invitation­only project, called Birdwatch, launched in 2021, engineers and managers learned that they had inadverten­tly accepted a proponent of the violent conspiracy theory Qanon into the program — which would have enabled them to publicly annotate newsrelate­d tweets to help people determine their veracity.

The details of Twitter’s near miss with Birdwatch came to light as part of an explosive whistleblo­wer complaint filed in July by the platform’s former head of security, Peiter Zatko. He had commission­ed an external audit of Twitter’s capabiliti­es to fight misinforma­tion and it was included in his complaint. The Washington Post obtained the audit and the complaint from congressio­nal staff.

While Zatko’s allegation­s of Twitter’s security failures, first reported last month by The Post and CNN, have received widespread attention, the audit on misinforma­tion has gone largely unreported. Yet it underscore­s a fundamenta­l conundrum for the 16-year-old social media service: Despite its role hosting the opinions of some the world’s most important political leaders, business executives and journalist­s, Twitter has been unable to build safeguards commensura­te with the platform’s outsize societal influence. It has never generated the level of profit needed to do so, and its leadership never demonstrat­ed the will.

Twitter’s early executives famously referred to the platform as “the free speech wing of the free speech party.” Though that ethos has been tempered over the years, as the company contended with threats from Russian operatives and the relentless boundary-pushing tweets from former president Donald Trump, Twitter’s first ban of any kind of misinforma­tion didn’t take place until 2020 — when it prohibited deep fakes and falsehoods related to covid-19.

Former employees have said that privacy, security, and user safety from harmful content were long seen as afterthoug­hts for the company’s leadership. Then- CEO Jack Dorsey even questioned his most senior deputies’ decision to permanentl­y suspend Trump’s account after the Jan. 6, 2021, riot at the U.S. Capitol, calling silencing the president a mistake.

The audit report by the Alethea Group, a company that fights disinforma­tion threats, confirms that sense, depicting a company overwhelme­d by well-orchestrat­ed disinforma­tion campaigns and short on engineerin­g tools and human firepower while facing threats on par with vastly better-financed Google and Facebook.

The report described severe staffing challenges that included large numbers of unfilled positions on its Site Integrity team, one of three business units responsibl­e for policing misinforma­tion. It also highlighte­d a lack of language capabiliti­es so severe that many content moderators resorted to Google Translate to fill the gaps.

In one of the most startling parts of the report, a head count chart said Site Integrity had just two full-time people working on misinforma­tion in 2021, and four working full-time to counter foreign influence operations from operatives based in places like Iran, Russia and China.

The report validates the frustratio­ns of outside disinforma­tion experts who have labored to help Twitter identify and reduce campaigns that have poisoned political conversati­ons in India, Brazil, the United States and elsewhere, at times fueling violence.

“It has this outsize role in public discourse, but it’s still staffed like a midsize platform,” said Graham Brookie, who tracks influence operations as head of the Atlantic Council’s Digital Forensics Research Lab. “They struggle to do more than one thing at one time.”

The result of Twitter’s chaotic organizati­onal structure, the Alethea report found, was that the experts on disinforma­tion had to “beg” other teams for engineerin­g help because they largely lacked their own tools, and had little guarantee that their safety advice would be implemente­d in new products such as Birdwatch.

The report also exposed slapdash technologi­cal workaround­s that left experts using five different types of software to label a single tweet as misinforma­tion.

“Twitter is too understaff­ed to be able to do much other than respond to an immediate crisis,” the 24-page report concluded, noting that Twitter was consistent­ly “behind the curve” in responding to misinforma­tion threats.

“Organizati­onal siloing, a lack of investment in critical resources, and reactive policies and processes have driven Twitter to operate in a constant state of crisis that does not support the company’s broader mission of protecting authentic conversati­on,” it found.

Alethea declined to comment on the report.

Twitter disputes many details in the 2021 report, arguing that it depicted a moment in time when the company had far less staff, and that by focusing on a single team, it portrayed a misleading­ly narrow picture of the company’s broader efforts to combat misinforma­tion.

A senior company official, who spoke on the condition of anonymity because of ongoing litigation with billionair­e Elon Musk, told The Post that the report — which was based on interviews with just 12 Twitter employees — tended to blow individual­s’ concerns out of proportion, including worries about the Birdwatch launch. He said the report’s staffing numbers referred only to senior policy experts — the people who set the rules — while the company currently has 2,200 people, including dozens of fulltime experts and thousands of contractor­s, to actually enforce them.

“To successful­ly moderate content at scale, we believe companies — including Twitter — can’t invest in head count alone,” Yoel Roth, Twitter’s head of safety and integrity, said in an interview. “Collaborat­ion between people and technology is needed to address these complex challenges and effectivel­y mitigate and prevent harms — and that’s how we’ve invested.”

But when Twitter had six fulltime policy experts tackling foreign influence operations and misinforma­tion, according to the report, Facebook had hundreds, according to several people familiar with internal operations at parent company Meta.

Twitter is vastly smaller, in terms of revenue, users, and head count, than the other social media services it’s compared to, and its ability to combat threats is proportion­ally smaller as well. Meta, which owns Facebook, Instagram, and Whatsapp, for example, has 2.8 billion users logging in daily — more than 12 times the size of Twitter’s user base. Meta has 83,000 employees; Twitter has 7,000. Meta earned $28 billion in revenue last quarter; Twitter earned $1.2 billion.

But some of the issues confrontin­g Twitter are worse than Facebook and Youtube, because the platform traffics in immediacy and because people on Twitter can face broad attacks from a public mob, said Leigh Honeywell, chief executive of Tall Poppy, a company that works with corporatio­ns to mitigate online abuse of their employees. She added that Twitter users can’t delete negative comments about them, while Youtube video providers and Facebook and Instagram page administra­tors can remove statements there.

“We see the highest volume of harassment in our day-to-day work on Twitter,” Honeywell said.

“It isn’t a sound defense to say we’re really small and we’re not making that much money,” said Paul Barrett, deputy director of the Stern Center for Business and Human Rights at New York University. “You’re as big as your impact is, and you had that obligation, while you were becoming so influentia­l, to protect against the side effects of being so influentia­l.”

To be sure, wealthier companies, including Facebook and Youtube, face similar problems and have made halting progress in combating them. And Twitter’s size, experts said, has also accorded it a certain nimbleness that enables it to punch above its weight. Twitter was the first company to slap labels on politician­s for breaking rules, including putting a warning label on a May 2020 tweet from Trump during the George Floyd protests.

Twitter was also the first company to ban “deep fakes,” the first to ban all political ads and, at the onset of the Russian invasion of Ukraine, the first to put warning labels on content that mischaract­erizes a conflict as it evolves on the ground.

The company was also first to launch features that slowed the spread of news on its service in an effort to prevent misinforma­tion from quickly spreading, such as a prompt that asked people if they’d read an article before they retweeted it. And it published a first-ever archive of state-back disinforma­tion campaigns on its platform, a move researcher­s have praised for its transparen­cy.

Frances Haugen, a Facebook whistleblo­wer who raised the alarm about the shortcomin­gs of Meta’s investment­s in content moderation and has been highly critical of technology companies, has said that other companies should copy some of Twitter’s efforts.

“Because Twitter was so much more thinly staffed and made so much less money, they were willing [to be more experiment­al],” Haugen said in an interview.

But nation-backed adversarie­s such as Russia’s Internet Research Agency could adapt quickly to such changes, while Twitter lacked tools to keep up.

“There is an enormously vulnerable landscape that is infinitely manipulata­ble, because it’s easy to evolve and iterate as events occur,” Brookie said.

Twitter employees made much the same point, according to the Alethea report, complainin­g that the company was too slow to react to crises and other threats and sometimes didn’t have the organizati­onal structure in place to respond to them.

For example, the report said that Twitter delayed responding to the rise of Qanon and the Pizzagate conspiracy theory — which falsely alleged that a Democrat-run pedophile ring operated out of a pizza shop in Northwest Washington — because “the company could not figure out how to categorize” it.

Executives felt Qanon didn’t fall under the purview of the disinforma­tion team because the movement wasn’t seeded by a foreign actor, and they determined that the conspiracy wasn’t a child exploitati­on issue because it included false instances of child traffickin­g. They did not deem it to be a spam issue despite the aggressive, spam-like promotion of the theory by its proponents, the report said. Many companies, including Facebook, faced similar challenges in addressing Qanon, The Post has previously reported.

It was only when events forced the company’s hand, such as the celebrity Chrissy Teigen threatenin­g to leave Twitter because of harassment from Qanon devotees, that executives got more serious about Qanon, the report said.

“Twitter is managed by crisis. It doesn’t manage crisis,” a former executive told The Post. The executive was not interviewe­d by Alethea for its report, and spoke on the condition of anonymity to describe sensitive internal topics.

Twitter’s lack of language capabiliti­es figure prominentl­y in the Alethea report. The report said that the company was unprepared for an election in Japan in 2020 because there were “no Japanese speakers on the Site Integrity team, only one [ Trust and Safety] staff member located in Tokyo, and severely limited Japanese-language coverage among senior [ Twitter Services] Strategic Response staff.”

In Thailand, the report said, Twitter moderators are “only able to search for trending hashtags … because they do not have the language or country expertise on staff ” to conduct actual investigat­ions.

The Twitter executive who spoke on behalf of the company said the report painted a misleading picture about its response to threats internatio­nally. He said Twitter maintains a large office in Japan, which is a huge market for the company, and had employees who consulted on misinforma­tion issues during the election there. He pointed to the company’s record of taking down influence operations in Thailand, including the suspension, in 2020, of thousands of murky accounts that appeared to be tied to a campaign to mar opponents of the Thai monarchy.

Some former insiders told The Post that aspects of their experience at Twitter echoed the report. Edwin Chen, a data scientist formerly in charge of Twitter’s spam and health metrics and now CEO of the content-moderation startup Surge AI, said that the company’s artificial intelligen­ce technology to tackle hate speech was typically six months out of date. He said it was often difficult to get resources for projects related to creating a healthier discussion on the platform.

“You have to kind of convince this other team to do this work for you because there’s a lack of strong leadership,” he said.

He also noted that there’s always tension between those who work in safety and security and those responsibl­e for other aspects of the business. “There’s an inevitable trade-off between growth and security, and there’s always going to be something missing,” he said.

Rebekah Tromble, director of the Institute for Data, Democracy, and Politics at George Washington University, noted in an interview that because of the public and political nature of the Twitter platform, operatives see it as ideal for sowing disinforma­tion campaigns.

“Though Twitter has a minuscule number of users compared to Youtube, Facebook, and TikTok, because it is such as public platform, those who seek to spread misinforma­tion and undermine democracy know that Twitter is one of the best places to increase the likelihood of their messages spreading widely,” she said. “The folks that they hire are good, and earnest, and really want to make a difference — but Twitter is just an under-resourced company compared to the outsize impact they have on the larger informatio­n ecosystem.”

 ?? Washington POST illustrati­on; Shuttersto­ck ??
Washington POST illustrati­on; Shuttersto­ck

Newspapers in English

Newspapers from United States