DIGITAL DECISIONS

Does the shoe fit? Try it on with augmented reality

When Eden Chen wants to go sneaker shopping, he takes out his smartphone and points the camera at his feet. Shoes materialise. He turns his feet from side to side. How does the shoe look on his foot? How does the top of the sneaker meet the cuff of his pants leg?

“It’s one thing when a shoe looks great on display,” Chen said. “When it’s on your foot, it’s just different.”

Chen, who has founded start-ups focused on gaming and augmented reality, is one of a growing number of consumers who, stuck at home because of the pandemic, are shopping in augmented reality.

The technology was made ubiquitous by social media platform Snapchat, which used it to transform users’ faces into anime illustrations and add dancing hot dogs to their videos. But as the pandemic continues, retailers are increasingly relying on augmented reality to help customers try on products. It displays goods as a filter on what they see on their phones, stitching shoes onto customers’ feet, adding make-up to their faces and dropping furniture into their apartments.

The process isn’t foolproof, Chen said. Sometimes the shoes will flicker as the artificial intelligence powering them struggles to pinpoint where they ought to be. Other times, the shoes will layer unnaturally with pants legs, covering them up rather than vanishing underneath them.

But it’s better than not trying them on at all.

“The first time I did, it was when the Nike Diors came out,” said Jerry Lu, an investor at Maveron, a venture capital firm. The sneakers, a collaboration between one of the world’s bestknown sports apparel companies and the French luxury goods company, promised to blend “haute couture and high-performance sportswear”. They were released online but were not available in boutiques.

“I knew I wouldn’t be able to get it,” Lu said. “Why not just play around with this filter to imagine if I had it?”

Traditional retailers, struggling to stay afloat during the pandemic, are hoping augmented reality can help them re-create the real-world shopping experience in the virtual world. US retailers’ overall Black Friday sales fell 20% from last year, according to a

Morgan Stanley estimate, but online spending that day increased 21.6%, according to Adobe Analytics.

Retailers’ demand for augmented reality has led many of them to Snap, the parent company of Snapchat, which has raced to add shopping experiences to its array of filters. The company began adding shopping filters in June and now offers augmented reality try-on experiences for luxury brands like Gucci and Dior, and make-up tutorials from cosmetics manufacturer Too Faced. This month, Snap teamed up with Perfect, a company that creates makeup try-on experiences, to add more beauty filters and shopping experiences to Snapchat.

“The pandemic accelerated a lot of conversations that we were already having,” said Carolina Arguelles Navas, product strategy lead for augmented reality at Snap.

Traditional retailers like Home Depot and internet giants like Amazon have experimented with the technology as well, using augmented reality filters for displaying furniture in consumers’ homes.

Some companies that focus on augmented reality have created apps solely for trying things on, like Wanna Kicks, for sneakers. Others, like Marxent, have worked with retailers to help build augmented reality experiences focused on their products. In June, Snap also released a technical library of tools to help developers recognise and classify objects to create augmented reality filters for Snapchat.

The shopping filters have led to a flurry of flexing, or showing off, as users have rushed to share images of themselves “wearing” Gucci and other brands. Nearly 19 million Snapchat users have tried on Gucci products using the filter, Snap said.

But trying on high-end products in augmented reality doesn’t always translate into sales. Alex Sirota, a 21-year-old US student at Northwestern University, said she often played with new augmented reality shopping experiences on Snapchat but never bought anything.

She has tried on sunglasses from Dior, nail polish from Sally Hansen, sneakers from Nike. But sometimes the technology is glitchy or slow, and it can’t give users a sense of the way products feel. She still likes to shop in person, when possible.

“If I’m going to buy something, I want the experience of going into the store, trying things on and making sure it actually is for me,” Sirota said. “It’s not an automatic ‘I’ve got to have it’. It goes on the list of hopefully one day I’ll be able to afford it or convince my parents to buy it for me.”

Snap argues that turning an advertisement into something people want to share with their friends, rather than rush to skip over, should count as a win. And for Snapchat’s younger users who can’t afford luxury goods, augmented reality can be a way to connect with the social media influencers they see on YouTube and TikTok.

“You also feel like you’re part of that ‘in’ group,” Lu said.

While people who shop in augmented reality use those images to show off, retailers hope the technology can help prevent one of their major fears for the holiday season: returns.

Deborah Weinswig, chief executive of Coresight Research, an advisory and research firm that specialises in retail and technology, said US users stayed engaged with augmented reality experiences for three times as long as they did with traditional e-commerce websites, and retailers are hoping that will mean fewer exchanges and returns. Without augmented reality to simulate trying things on, Chen said, shopping during the pandemic has come with the usual online-shopping frustrations. “I just bought a couple things from Nike for Black Friday, and I’m returning them,” he said. “Your way to try stuff on is to buy it and return it.”

Returns have always been a problem for online retailers, especially after the holidays. But analysts worry the onslaught will be particularly bad this year, as consumers rely more heavily on online shopping and the economic pressure of the pandemic leads them to reconsider purchases.

“I think we’ll see returns unlike anything we’ve seen in the history of the US,” Weinswig said. “It’s going to be a bloodbath.”

Retailers hope augmented reality can help them re-create the real-world shopping experience