Generative AI creates fake audio mimicking Kishida, other leaders

An app that uses generative AI to create fake audio clips imitating Prime Minister Fumio Kishida and other Japanese leaders is available online, it has been learned.

The app is free to use on smartphones and other devices.

It allows users to enter text and have it read out by voices imitating those of Kishida and former prime ministers Shinzo Abe and Yoshihide Suga.

Experts have called for countermeasures to prevent the app from being abused.

POSTED ON X

When a user inputs a sentence on a screen of the app displaying a picture of Kishida, Abe or Suga, the text is automatically read out by a voice that mimics one of the three.

The app’s URL has been shared on social media and other sites, making it instantly accessible on smartphones, computers and other devices.

Some fake audio clips that appear to have been made with this app have been posted on X, formerly known as Twitter.

One fabricated clip makes it sound like Abe is saying he and Russian President Vladimir Putin shared a sense of urgency about the so-called deep state and moves toward isolating Russia.

The app was created by a 25-year-old man from Hyogo Prefecture who has posted fake videos of Abe, Suga and others online.

He said he created the app by having an AI learn Kishida, Abe and Suga’s voices from online videos of speeches and other material.

Since autumn last year, the man has been producing and regularly posting videos with fabricated voices of the three leaders.

He created the app because he “wanted people to make parodies of politicians and others and make fun of them, just like I did,” he said.

A fake video of what appears to be Kishida making vulgar remarks was posted on X early this month. The fake video, which bears the logo of the Nippon Television Network Corp, went viral.

It was posted by a 25-yearold man from Osaka Prefecture. He taught the man from Hyogo Prefecture, who created the app, how to make fake voices.

TOUGHER MEASURES NEEDED

Shinichi Yamaguchi, an associate professor at the International University of Japan and a member of the government’s AI Strategy Council, said so-called deepfakes are increasingly popular as the rapid spread of generative AI makes it easy for anyone to create fake videos without specialized knowledge or significant amounts of money.

In a survey conducted this year, Yamaguchi and his research team listed six pieces of false or misleading information about politics spread online — such as the number of participants in a demonstration against Abe’s state funeral — and asked people whether they thought the pieces they had seen or heard were true or false.

Only small minority of the 2,018 respondents said they thought the information was false.

“It’s easy to imagine how fake videos and fake voices of politicians can cause confusion about things like elections, stock prices and disasters,” Yamaguchi said.

“Another threat is that such [fake videos and audio] could be spread across national borders via social media and could even become an international problem if used for propaganda purposes.”

“Stronger countermeasures are needed, such as requiring AI-generated video and audio to be clearly labeled as AI-generated when posted, and proactively deploying technology that can detect AI-generated video and audio,” he said.

FIGHTING DISINFORMATION

Authorities and tech companies overseas are working to clamp down on disinformation generated by AI.

The European Commission compiled an action plan to combat online disinformation, and it was signed by U.S. tech giants Google LLC and Meta

Platforms Inc., among other companies, in June.

Meta and Google announced their countermeasures earlier this month.

Meta said it would require advertisers to label AI-generated content in political ads on Facebook and Instagram.

Google also said it would require AI-generated content to be clearly labeled when posted on YouTube, and content that violates its policies will be removed.

“Spreading disinformation about the government would damage the foundation of democracy and should not be done,” Chief Cabinet Secretary Hirokazu Matsuno told reporters on Nov. 6.

“We will work with relevant ministries and agencies to come up with necessary measures based on international debate and discussions at the government’s council for strategy on AI.” (Nov. 23)