Mass use of AI in fake web accounts a first

Facebook and Twitter have disabled a global network of hundreds of fake accounts that pushed pro-Donald Trump messages and covered its tracks using inauthentic photos generated with the aid of artificial intelligence.

The use of AI to trick social media companies, deceive unsuspecting users and essentially create people who do not exist marked a major, troubling new development, said disinformation researchers, who expressed fresh alarm that such tactics could have implications for the 2020 US presidential election.

The tech giants’ takedown targeted the BL, a US-based media company that Facebook linked to Epoch Media Group.

The organisation has criticised the Chinese government and has ties to Falun Gong, a Chinese spiritual movement. It also has vociferously supported the re-election of the US president.

In a report, the researchers with whom Facebook shared data about the BL and Epoch Media Group described the operation as ‘‘a largescale artificial amplification factory’’.

Facebook in particular said the BL was linked to hundreds of fake accounts spread across its services, which posted political messages at high frequencies.

The social media companies signalled that they took action not because of the content of those posts, but rather the tactics deployed by those who engaged in them, such as the use of AIgenerated images, which violated rules prohibiting spam, misrepresentation and coordinated inauthentic behaviour. Disinformation experts at Graphika and the Atlantic Council’s Digital Forensic Research Lab – which were given access to the Facebook data – said it was the first time they had seen AIgenerated pictures ‘‘deployed at scale to generate a mass collection of fake profile pictures deployed in a social media campaign’’.

Facebook disabled more than 600 accounts. Twitter suspended 700 accounts.

Some of the fake accounts relied on AI tools and technology to create pictures of people who do not exist. Others blatantly stole photos from real users. The fake accounts may have relied on publicly available technology to publish their posts, tweets and other content in batches, amplifying their reach.