Interactive
See Me, Feel Me, Touch Screen
It’s like Siri if Siri were an actual human with a life story to tell.
the life-size image of pinchas
Gutter on a video screen, fidgeting, blinking and tapping his foot, seemed present and alive in the way portraits do in the magical world of Harry Potter. The Holocaust survivor, who lives in Toronto, was nowhere near the Museum of Jewish Heritage on the day I visited, but by stepping up to a podium, clicking on a mouse and speaking into a microphone, I was able to ask Gutter questions. His image responded with answers—speech quirks, pauses and gestures included. He spoke to me about religion and sports; he shared his favorite Yiddish joke; I hear he sometimes sings. Gutter also told me that he was a happy child until September 1, 1939, when Hitler’s armies invaded Poland and World War II began. Soon after, his father was taken away and beaten nearly to death. After that, he said, “I knew that life wouldn’t be the same.”
Eva Schloss, an Auschwitz survivor and the posthumous stepsister of Anne Frank, was a few feet from Gutter, on another screen. She had her own stories to tell. The effect of talking to these virtual people was startling and eerie. I didn’t for a minute forget that they were images, but I found myself deeply moved by both.
Gutter and Schloss are part of a project called New Dimensions in Testimony (NDT), a collaboration of the USC Shoah Foundation, the Institute for Creative Technologies (ICT, also at the University of Southern California) and Conscience Display. The two are providing “testimony” of their experience as survivors. When I asked Gutter’s image why he shares his story, he said “Number one, I would like people to know what can happen. And number two, I try to teach them tolerance.”
How did the images “know” what I was asking? A Google speech recognition algorithm identifies words that are then deciphered by a natural language processing editor, developed by ICT. A clip, or appropriate response, is then loaded. It’s like Siri if Siri were an actual human with a life story to tell. When the software doesn’t understand the question, the image might respond with “Can you ask me again?” Or, if there’s no relevant clip, “That’s a really good question. Unfortunately, I don’t have an answer for you.”
To accomplish this, survivors participating in NDT are filmed in high definition from 360 degrees with more than 100 cameras (the setup is so advanced that some of the data is captured in anticipation of future technological advancements). Simulating eye contact by staring directly into the camera, Gutter, the first survivor to participate, recorded about 1,900 answers to diverse questions, like “What was the scariest thing that ever happened to you?” and “What was your favorite movie?” In addition, he was filmed actively listening, to appear engaged between questions.
With NDT, the Shoah Foundation is looking to replicate what has been happening in classrooms for the past 40 years. “Teachers continually ask survivors to come and talk to students. They wouldn’t do that if it wasn’t affecting them,” says Heather Maio, the NDT concept creator who brought the idea to the foundation in 2010. In addition to museum installations, Maio says NDT could one day be available online, easy to use on laptops or Smart Boards in classrooms.
The intention is to create compassion via technology, an idea that continues to be furiously debated. The overwhelming response to the rise of technology has been to blame it for rises in narcissism and declines in empathy. Sara Konrath, an assistant professor of philanthropic studies at Indiana University and director of the Interdisciplinary Program for Empathy and Altruism Research, has been studying this connection for a decade, particularly younger generations. The thinking was that the internet, social media and countless other distractions were dehumanizing young people. Konrath wasn’t so certain. “They’re tools. We can hate or
love them,” she says, or “we can figure out how to use them.”
In recent years, her research has involved how Facebook, Twitter, texting and other technologies influence behavior intended to help others. Konrath cites one study that found people who experienced the wreckage of Aleppo in virtual reality were more likely to donate to an organization helping refugees than those who simply saw photos. With NDT, she saw the next logical step. It felt “personal,” she says of seeing an early demo of Gutter’s testimony. “I liked him, and I wanted to get to know him.” It’s a short leap from there to understanding and, perhaps, empathy.
a few months ago, my spry 89-year
old grandfather (who still goes to the gym three times a week) visited the U.S. from Israel. My younger brother and I taped his responses to questions about his childhood in Lithuania, about his middle-class Jewish family, banished by the Soviets to Siberia. As a teenager, he did hard labor in a frozen gulag as the Nazis killed 90 percent of the Jews in Lithuania. He spoke of near starvation and an arduous escape. We wanted to preserve his stories in his own voice. But what if we could record his answers as Gutter’s were, so that our grandchildren could ask him questions too?
NDT makes that seem possible. In addition to archiving the testimonies of 13 Holocaust survivors so far, the accounts of a survivor of the Nanjing massacre have been recorded in Mandarin, to launch in China at the Nanjing Massacre Memorial Hall. Stephen
D. Smith, executive director of the USC Shoah Foundation, sees the technology expanding to people who have survived cancer or catastrophic hurricanes. The possibilities are nearly endless—from the experiences of soldiers with post-traumatic stress disorder or survivors of sexual abuse, to those of presidents or great teachers. Imagine if a slave could have told her story to her grandchildren? “It’s not about the Holocaust,” says Smith. “It’s about how we communicate as human beings.”
Less than a week after I visited the Jewish Heritage installation, Gutter came to speak at the museum. He was introduced as the project’s rock star. For many years, he told the crowd, he never spoke about his experiences; even his children didn’t know the details. He gave his first testimony to a professor in Toronto in 1993, and a few years later sat to be interviewed for the Shoah Foundation’s more traditional archive of video testimony. Eventually, he was speaking to students of all ages and traveling to Germany and Poland to accompany educational tours. “There was no end to it. Once I started, they never let you go,” Gutter said with a laugh. His most recent VR experiment premiered at the Tribeca Film Festival in April: In the film The
Last Goodbye, he takes viewers to Majdanek, one of six Nazi concentrations camps that he survived, and where his parents and twin sister were killed.
“I’m old and tired,” said Gutter, but he keeps telling his story. He described standing off to the side at England’s Sheffield Doc/fest, watching people reacting emotionally to his screen self, what he calls his “alter ego,” then turning to discover the real him and continuing the questions. It comforts him to know that once he’s gone, children will be able to walk up to virtual Gutter and ask, “What was it like for you to go through that?”