Love Lives On(line)
Reviving departed loved ones in the digital realm, a stir of innovation and debate
Bao Xiaobai, a singer and actor from Taiwan Province, recently made headlines for using artificial i ntelligence (AI) to create a digital representation of his late daughter.
The process is known as “AI resurrection.” Chinese social media was abuzz with a viral short video featuring a digital avatar of Bao’s daughter singing Happy Birthday to her mother.
The young woman passed away in 2021 at the age of 22 due to a rare blood disease.
Struggling to come to terms with her untimely death, Bao in July 2022 turned to his friend Liu Yan for support. Liu, founder of livestreaming platform 6.cn and a skilled developer of virtual idols, was approached by the singer and actor with his desire to create a digital incarnation of his daughter.
Two months later, Liu came up with a plan to create a digital human—a 3D computergenerated representation of a person that can be animated to mimic the movements and behaviors of a real individual within a virtual environment.
Initially, Bao had little knowledge of the extent to which technology could replicate his daughter.
The turning point came with the introduction of ChatGPT, an AI chatbot launched in November 2022 by OpenAI, a U.S. research organization focused on AI.
This chatbot, based on a family of large language models (LLMs), can generate responses that are human-like and often accurate across a wide spectrum of inquiries.
This development reassured Liu that recreating humans in the digital realm was indeed achievable. LLMs serve as foundation models trained on vast datasets to understand and generate natural language and other forms of content.
Back to life
In January 2023, Liu approached XiaoIce, a popular social chatbot once under global tech giant Microsoft in China. XiaoIce is renowned for its LLM expertise.
The company’s avatar framework, based on natural language processing—a field of AI that enables computers to understand, interpret and generate human language, with LLMs being a key component, seemed promising for realizing Bao’s vision of developing a digital human capable of interacting with people.
But a significant challenge arose: They did not have sufficient data to train the digital human effectively. The crucial task of creating a digital replica of Bao’s daughter hinged on infusing the model with her unique traits, including her personality and voice.
To make this happen, Bao needed to provide clear voice recordings of his daughter. Despite scouring all available recordings, he was unable to find any that met the company’s strict requirements.
After discovering a recording of his daughter speaking English at school, Bao, a music producer skilled in audio processing, utilized techniques like noise reduction on the recording.
Four months and 1,000 rounds of finetuning later, Bao successfully recreated his daughter’s voiceprint, a mathematical representation of the distinct anatomical and acoustic factors that define a person’s voice.
Currently pursuing a Ph.D. in the field of AI, Bao is researching and writing a thesis on reconstructing voiceprints from damaged voice data.
The final phase involved online dialogue training and construction of the digital human’s memory. Bao engaged in conversations with the digital model while his wife provided input such as sharing their daughter’s favorite food and childhood best friend.
Eventually, Bao had a digital daughter who could talk with him and his wife, offering a sense of companionship and solace.
Like Bao, there are many others in China who have the need for AI resurrection.
Driven by grief
Zhang Zewei, who runs an AI studio in Nanjing, Jiangsu Province, has finished more than 600 orders to revive deceased relatives with the help of AI since 2022.
That year, Zhang accidentally stumbled into the field of AI resurrection. A friend approached him, revealing that his father had passed away and that the family had kept the death a secret from his 90-year-old grandmother to spare her the grief. The friend requested Zhang’s help to use AI to resurrect his father digitally, enabling him to communicate with his grandmother and maintain the illusion that he was still alive.
Zhang told Jimu News, a news portal based in Hubei Province, that he
was initially hesitant, grappling with ethical concerns about essentially deceiving the grandmother. Despite his reservations, he eventually agreed to help his friend. Witnessing the joy on the elderly woman’s face as she interacted with her son’s digital replica moved Zhang, affirming to him that he had achieved something meaningful.
Zhang explained to social media platform Yitiao that his studio primarily uses natural language processing, deep learning (a subset of machine learning employing multiple-layered artificial neural networks) and voice cloning technology. Voice cloning involves a deep learning system that synthesizes an individual’s voice recordings to closely resemble the original.
To create lifelike avatars, the studio requires photos, videos, recordings and personal information of the deceased. The more data, the more realistic the avatar.
For customers seeking more interactive experiences with the avatars, the studio can generate the image and voice of the deceased person. They then employ face-swapping technologies to enable a psychologist to communicate with customers via video chat, using the appearance and voice of the deceased individual for a personalized interaction.
At first, Zhang provided these services for free, but due to increasing demand, he introduced paid services to ensure the studio’s sustainability. The service is priced between several thousand and 10,000 yuan ($1,391).
Zhang further told Jimu News that his team had received over 1,500 orders for AI resurrection, but completed only 600-plus. The main reason for this backlog is usually insufficient data provided by customers.
Controversy
Although AI resurrection can provide some solace for the deceased’s loved ones, there are concerns that reliance on AI replicas may hinder the natural healing process by preventing people from fully confronting their loss.
Moreover, a commentary published on March 1 on news portal Redcn.net highlighted the pressing need to address the ethical issues and legal implications of AI resurrection.
One major concern is the potential for people to become addicted to virtual experiences, distancing t hemselves from reality. Moreover, questions arise regarding data ownership in AI resurrection: Should the living have the right to “resurrect,” or manipulate even, the data of the deceased?
And then there are concerns that the digital humans produced through AI resurrection could be exploited for fraudulent purposes.
Zhang explained on Yitiao that his studio follows a protocol of inquiring about the intended use of the digital human before accepting an order. Due to the absence of set industry standards or regulatory frameworks, the studio signs a contract with customers. This document explicitly states that the digital humans will not be used for illegal purposes, mitigating potential legal liabilities.
In his interview with Jimu News, Zhang expressed his surprise at the extensive demand for AI resurrection. He envisions a future where individuals can create digital replicas of themselves to provide companionship for loved ones after their passing.
He disclosed his studio’s plans to launch a service for generating digital humans for the living, allowing customers to upload their personal data to create their own digital counterpart. Dubbed “the digital immortality plan” by Zhang, this service aims for people to leave behind an eternal presence in the real world.