HOW AMAZON GO’S LINELESS GROCERY SERVICE MAY WORK
Question is whether company’s ‘just walk out’ technology is viable
The Amazon Go grocery store, now in the testing stage in Seattle, sounds like a dream come true for holiday shoppers waiting in long lines.
The underlying technology seems to be routed in terra firma, however, a mix of cameras, microphones and the massive servers Amazon uses to run its cloud computing service and power digital assistant Alexa.
The concept promises to let shoppers walk into a store, pick things up and walk out, thereby skipping the checkout line while everything acquired gets automatically charged to a credit card.
Amazon’s explanation on how it works in its video is heavy on buzzwords: computer vision, deep learning algorithms and sensor fusion.
The company declined to comment on the tech behind Amazon Go, but a patent filed by the company in 2014 and published in 2015 might shed some light on the process. It appears to rely on cameras and microphones — lots of them. The tech is similar to what’s used to allow self-driving cars to navigate the world.
According to the patent, each customer entering the store would be tagged as they entered. In Amazon’s video, they tag their smartphone, which contains the Amazon Go app, as they walk in.
That allows the store’s surveillance system to identify the customer so it can track them as they move throughout the space. Cameras pick up images of when they stop in front of shelves, what items they picked up and whether the item stayed in their hand or went back on the shelf.
As the patent puts it, “when the user’s hand is removed from the inventory location, one or more images may be captured of the user’s hand as it exits the inventory location. Those images may be compared to determine whether a user has picked an item from the inventory location or placed an item in the inventory location.”
The cameras would even know the customer’s skin tone, to determine if it was indeed the hand of the person the system thinks it is. As in, “image analysis may be performed on the first image to determine a skin tone color of the user’s hand and pixels including that color, or a range of colors similar to the identified skin tone color, may be identified to represent the user’s hand.”
That could potentially be used to distinguish between two people each reaching for things on adjacent shelves, as skin tone is very individual.
This is exactly what Amazon has become very good at with its Alexa voice-control system. That system sends sound files of spoken commands to Amazon’s cloud computing network, where they are identified, turned into digital commands, answered and then sent back to the Echo device.
In fact, some of that expertise may be explicit in the Amazon Go system, as microphones are also used in the stores to determine where users are by the noises they make. It even tracks them by noticing the time difference between the audio signals received by each microphone in the store, a kind of reverse echolocation.
The system uses infrared, pressure and load sensors on shelves that note when items have been picked up and whether they are put back. These sensors feed into the store’s continuous sense of where everything, and everyone, in it are at any moment.
“The challenge is whether you can make it cost-effective. Has Amazon come up with the secret sauce?” Forrester analyst Brendan Witcher said.
Many had thought something like this would happen through the use of tiny RFID, or radio-frequency ID, tags. These send out a constant “I am here!” message, allowing the system to track whatever they’re attached to.
“They’ve come down in price, they’re now only about 5 cents each. But it’s still pretty labor intensive to RFID tag everything,” Witcher said.
It has been suggested for years. In fact, IBM released an ad about a store where you just picked things up and walked out in 2006, he said. But looking at the patent, it seems Amazon has found a less expensive, more technologically intensive way to make that work.
Knowing customers from the moment they walk through the door, and then tracking everything they do once inside, could be hugely useful to a company, said David Hewitt, analyst with consulting firm Sapient Razorfish.
“Think about turning on a digital display that markets to the person who’s in the aisle right then. I know it’s a little Big Brother, but all this data is welcome if it makes the shopping process better,” he said.
Amazon could also license the technology to other supermarkets. That would be a win-win for Amazon, because it would be selling not only the technology but also the cloud computing power of its AWS cloud services.
“The challenge is whether you can make it costeffective. Has Amazon come up with the secret sauce?” Forrester analyst Brendan Witcher