Contagious & deadly
From Myanmar to Brazil, falsehoods spread insidiously and take a tragic toll.
INDIA
WhatsApp, the Facebook-owned encrypted messaging service, was held accountable by the Indian Government for 33 deaths in mob violence associated with false stories of child abduction.
MEXICO
It was also blamed for the spread of a similar falsehood in Mexico that led to two men being burnt to death by a mob. The following day, a mob pulled a man and a woman from their truck in a rural area and beat and burnt them, despite the pair’s pleas of innocence. The man died at the scene, and the woman in hospital.
BRAZIL
A mass yellow fever immunisation campaign in Brazil has been compromised by disinformation, including one post – shared 300,00 times – that side effects of the vaccine (used for decades with no serious issues) had killed a teenage girl. A total of 1257 confirmed cases and 394 deaths from yellow fever were reported in Brazil between July 2017 and June last year.
MYANMAR
Facebook accounts run by Myanmar military personnel targeted the Rohingya Muslim minority. Humanrights groups blame the anti-Rohingya disinformation campaign for inciting murders, rapes and the largest forced human migration in recent history. More than 700,000 people have fled Rakhine state and a UN mission estimated 10,000 Rohingya have died.
of getting to the root of the disinformation problem.
Although Facebook, Google and Twitter have moved to remove some of the false accounts used to plant disinformation – Twitter confirmed last June that it was conducting a mega purge and eliminating a million fake and suspicious accounts a day – the British government’s response to the Commons report in October was that the social-media companies were not doing enough.
Facebook and its kind earn little sympathy. They deserve to be accused of a form of third-party complicity by failing to build into their systems the checks and balances to prevent their misuse. However, it may be possible to grant them a thimbleful of understanding because the inherent characteristics of disinformation – and the fact that its form is changing so fast – mean its detection and suppression are becoming more difficult.
The false stories produced by bogus news sites and promulgated through Facebook and Twitter before the US presidential election now appear rather crude. Hindsight does provide us with insight but, even as they surfaced, there were ways in which the falsehoods could be outed. Some, after all, were a little too obvious: the Pope’s endorsement of presidential candidate Trump was a falsehood too far.
Newsrooms were provided with services that might be crudely but accurately described as bullshit detectors. Online services such as TinEye were developed to determine whether images were real or doctored – by doing a form of reverse engineering and checking back through image search engines such as Google to find full or partial matches. Services such as Storyful were set up to do verification checks on trending stories using tried-and-true journalistic techniques. Verification may be as simple as checking with the people mentioned in the story to ascertain whether they had actually said or done what was being attributed to them.
Most of the falsehoods that were produced in the run-up to the US election, and during European elections and the Brexit referendum, were detected and debunked. The British government’s response to the Commons report noted that it had not seen evidence of the successful use of disinformation by foreign actors, including Russia, to influence UK democratic processes. It did not, however, define “successful”.
Disinformation aimed at the UK may not have led to mass shifts of opinion, but it seldom seeks to achieve what Adolph Hitler
thought possible in Mein Kampf: that, by repetition and a clear understanding of psychology, you could prove to the masses that a square was in fact a circle. Rather, modern disinformation seeks to discredit right-angles among people already predisposed towards circles. This is not preaching to the converted, although they, too, will be willing recipients. It is a sophisticated targeting of what Indiana University researchers have identified as three different types of bias that are susceptible to manipulation. Giovanni
Luca Ciampaglia and Filippo Menczer, of the university’s observatory on social media, have developed tools to show people how cognitive, social and machine bias can aid the spread of disinformation. Cognitive bias emerges from the way the brain copes with information overload to prioritise some ideas over others.
“We have found that steep competition for users’ limited attention means that some ideas go viral despite their low quality – even when people prefer to share high-quality content,” they wrote. They added that the emotional connotations of a headline were a strong driver.
They found that when people connect directly with their peers via social media, the social biases that guided how they chose their friends also influenced the information they chose to see. It was also a significant factor in favourably evaluating information from within their own “echo chamber”. And these preferences are fed by the machine – the algorithms that determine what people see online. The internet is not simply a system of highways on which we may choose to drive. It is an organism that mines data to build a profile of every driver and passenger on the system and to feed on their wants and preferences. At the very least, its users are in semi-autonomous vehicles and, at worst, they have no control over the car whatsoever.
“These personalisation technologies are designed to select only the most engaging and relevant content for each individual user,” Ciampaglia and Menczer said. “But in doing so, it may end up reinforcing the cognitive and social biases of users, thus making them even more vulnerable to manipulation.”
Data is accessed by disinformation sources, and algorithms used to identify groups (say, people predisposed to circles) into which disinformation can be seeded and sent on its merry viral way. Algorithmic selection ensures the message reaches the “right people”. Disinformation campaigns may then be fed by social botnets that allow massive proliferation of messages. These automated social-media accounts create and move huge amounts of material that unsuspecting users believe is legitimate. This is how 45,000 tweets were created in the final two days of the Brexit campaign. A high percentage of the Twitter accounts elimi- nated after the US presidential election were operated by bots, and the FBI found new bot accounts were created before the recent midterm elections. The bureau believed many emanated from St Petersburg.
Milking bias is all the easier when the disinformation meets four criteria. According to Ben Nimmo, of the digital forensic lab at think tank Atlantic Council, a successful fake story has emotional appeal, a veneer of authority, an effective insertion point into the online space and an amplification network such as Twitter or Facebook.
And disinformation has something else working for it: we humans seem to prefer fake over fact when it is presented in ways that trigger those biases. Massachusetts Institute of Technology has studied rumour cascades. These are rumour-spreading patterns that have a single origin and an unbroken chain of retweeting or reposting. The study found that falsehood reached far more people than the truth. Whereas the truth rarely spread to more than 1000 people, the top 1% of false-news cascades routinely spread to between 1000 and 100,000 people. It took the truth about six times as long as falsehood to reach 1500 people. In other words, we now have scientific proof that Jonathan Swift was right: “Falsehood flies, and truth comes limping after it.”
QUANTUM LEAP FORWARD
To date, most disinformation has been believed by those who want to believe it. Others have rejected it, because they do not want to believe it and it has been relatively easy to discredit its content. That, however, is about to change.
Artificial intelligence and machine learning were able to do reasonably credible service creating and spreading disinformation, but there were telltale signs that these messages were created by bots. Tweets could be checked on Botometer, a joint project of the Network Science Institute and the Centre for Complex Networks and Systems Research at Indiana University, that used about 1200 features to characterise the suspected account’s profile, friends, social network structure, activity patterns, language and sentiment. Facebook posts often contained stilted, formulaic language. The use of bots to share disinformation was harder to detect.
But now, artificial intelligence has allowed disinformation to take a quantum leap forward to the point where it is no longer possible to tell whether what you see and hear is real. The threat these so-called deep fakes pose is so serious that the US Department of Defense has tasked one of its agencies with finding ways of detecting fake video and audio.
The threat stems from software that takes alarmingly small amounts of authentic material – as little as 3.7 seconds of audio and 300-2000 images from a short video clip – to create a visual message in which words have, quite literally, been put in someone else’s mouth. Facial and body movements will be indistinguishable from the real thing.
In August, a team led by the Max Planck Institute for Informatics in Germany revealed a system called Deep Video Portraits. In contrast to existing approaches
Humans seem to prefer fake over fact when it is presented in ways that trigger our biases.
restricted to manipulations of facial expressions only, it was the first to transfer the full 3D head position, head rotation, face expression, eye gaze, and eye blinking from a source actor to a portrait video of a target actor. In the first test of the technology, the team showed real and manipulated videos of Vladimir Putin, Theresa May and Barack Obama to two groups from North America and Europe. More than half thought the manipulated videos were real and 65% thought the altered images of Putin were authentic. Perhaps as a symptom of a posttruth age, only 80% thought the real videos were authentic.
In parallel with the German-led research, the University of Washington has been perfecting lip-sync software that allows a third party to script what a person will say in a deep fake video. Somewhat naively, the American researchers see “a range of important practical applications”, including allowing hearing-impaired people to lipread on their smartphones and providing new ways for Hollywood to seek box-office success.
One of the more worrying aspects of such research is the speed with which it is perfecting the software to create flawless fakes. The Defense Department’s Advanced Research Projects Agency has spent US$68 million but has so far found only limited ways to detect deep fakes. Matt Turek, head of the agency’s media forensics project, said in an ABC News interview – carried on the agency’s Facebook page that defensively labelled it “a real piece” – that deep-fake detection was a “bit of a cat-and-mouse game”.
“A lot of times there are some indica- tors that you can see, particularly if you are trained or used to looking at them. But it is going to get more and more challenging over time … We are looking at sophisticated indicators of manipulation, from low-level information about the pixels to metadata associated with the imagery, the physical information that is present in the images or media and then comparing it [to] information that we know about the outside world.”
One indicator identified by the agency was blinking. In many of the fakes it examined, manipulated images of people did not blink in a natural way. However, the German research is rapidly overcoming that anomaly. Its programme transfers not only eye movement but authentic blinking rates from the source to the deep fake. And it hasn’t finished. The research paper concludes: “We see our approach as a step towards highly realistic synthesis of full-frame video content under control of meaningful parameters. We hope that it will inspire future research in this very challenging field.”
It will almost certainly inspire ever more realistic deep fakes that may rob us of one of our most basic assumptions: that – in combination – we can believe our own eyes and ears. When we see a video of Obama, we expect it to be a captured version of what American philosopher John Searle calls “direct realism”: the camera as a surrogate for our own eyes. Perhaps it is inevitable that we are even less equipped to question the validity of a machine-created moving image than we are an AI-driven chatbot that can mimic human responses in text.
Yes, the word of the year 2019 will be disinformation. You just may not recognise it when you see it.
Researchers have been perfecting lip-sync software that allows a third party to script what a person will say in a deep-fake video.