HOW FAR WILL AI GO?
Anyone with a laptop can make believable deepfakes, but most of the electorate are adopting a wait-and-watch approach. Meanwhile, experts warn against the prevalent menace of misinformation
Aday after Delhi Chief Minister Arvind Kejriwal was arrested by the Enforcement
Directorate in a liquor scam case, his wife Sunita read out a message from him on camera, in a video released by the Aam Aadmi Party online. But that was just one version of Kejriwal’s message that went viral — an Artificial Intelligencegenerated English translation of the incarcerated politician’s communication followed, and then another Hindi version. “I’m neither shocked nor worried; all my life, I’ve struggled for a better society,” these videos said, in Kejriwal’s voice.
As India gets into election season, more and more examples of synthetic and realistic deepfakes of politicians have started appearing. There’s one of the late M. Karunanidhi exhorting party cadre, as his son, Tamil Nadu Chief Minister M.K. Stalin, looks on; a Tamil dub of Prime Minister Narendra Modi addressing a gathering in Chennai; and videos of Madhya Pradesh leaders Shivraj Singh Chouhan and Kamal Nath with doctored remarks. Anxiety about AI deepfakes in campaigning has heightened largely because making synthetic images and videos has gotten far cheaper — and better — from four years ago, when the Bharatiya Janata Party’s Manoj Tiwari put out a lowresolution AIgenerated Haryanvi dub of a video message in the runup to the Delhi Assembly elections.
India’s neighbours too have lent some legitimacy to these anxieties: voter suppression deepfakes with a “don’t show up; the polls are rigged against us” theme have surfaced in the past year in both Pakistan and Bangladesh, where the targeted opposition politicians had to issue denials. With cheap mobile data and the highest smartphone penetration in India’s history, concerns are rife that similar crucially timed deepfakes circulated via platforms such as
WhatsApp could sour voters on candidates, or convince them that their vote could be meaningless.
‘Ethical’ AI creations
Not all synthetic AI creations take the form of sinister opponent slandering. Some implementations, like personalised interactive phone calls, seem more like interesting novelties than threats to electoral integrity. While onetoone calls are not yet a reality in India, socalled ‘blasters’ with a synthesised candidate’s voice speaking individual voters’ names in a prerecorded message have been sent out by the Congress party in Rajasthan and by the Aam Aadmi Party in Delhi.
Divyendra Singh Jadoun, one of an emerging crop of synthetic media creators, has assembled halfadozen staff to train voice and video AI models and distribute them through phone calls and video messages, on behalf of political parties. While Jadoun, who operates under the name The Indian Deepfaker, refused to name the organisations, he said that at least four of his current projects are on behalf of political parties, with at least two mainstream organisations in the fray.
Jadoun says he restricts his firm’s work to “ethical” AI creations such as authorised translations, revivals of deceased leaders (with the party’s blessings), and oneonone phone calls with chatbots synthesising tailored responses. He uses Mistral AI, which is based on open source models, to get around mainstream proprietary firms’ refusal to allow their tools to be used in electioneering.
He claims to have refused unethical requests by parties to depict opponents saying things they never have; but notes that increasingly, parties prefer to not outsource deepfakes of rivals to external firms, instead taking the task upon themselves. “Anyone with a laptop now can make this stuff,” said Karen Rebelo, a cofounder of the fact checking website BOOM, at a panel discussion this February. “You don’t need to go to a specialised agency, or even to somebody who knows code.” Rebelo noted a sharp increase in AI deepfakes already during recent Assembly elections in India.
Sagar Vishnoi who worked at The Ideaz Factory, the firm that made Tiwari’s Haryanvi video in 2020, agrees that the technology has become much cheaper. Tiwari’s lip sync back then was done over a dayandahalf, and his voice was not synthetic — a mimicry artist had dubbed over the MP’s video. “Now, the tech has changed. Voice training models have become available, not just lip syncing ones,” says Vishnoi, who has since left the company and is currently engaged as a political consultant for a “huge” clientele that he declines to name.
Tackling misinformation
But even those wary of the fairness of Indian elections in recent times are not entirely convinced that deepfakes will affect the integrity of the poll process any more than the conventional strategies already at play. Says Pratik Sinha, a cofounder of factchecking news website Alt News: “AI is an issue, but I don’t feel as bothered about a new way of creating disinformation; the existing methods are working quite well.”
Sinha, whose daily job involves flagging fake news and misinformation, is referring to the strategically clipped videos and inflammatory speeches that continue to be circulated by the dozen from ordinary citizens’ smartphones. “What has changed from 2019 to now is that the amount of hate speech has increased manifold,” he says.