Let’s play, without leaving homes

App lets kids remotely control robot toy

NEW HAVEN — When computer science students at Yale University were told not to return from spring break because of COVID-19 , their projects suddenly became difficult to finish.

At the same time, young schoolchildren found themselves unable to run outside and play or even to see their friends at school.

What the Yale students came up with was an app that lets two children play with a small robot together while each stay in their own homes.

“It started with an idea to help students connect during social distancing but still have a physical presence while they’re doing this, kind of the inperson play without being in person,” said Nathan Tsoi, a doctoral student in computer science.

Besides creating the app to control the robot, “we had to figure out what it meant to play remotely,” including designing a user interface, he said.

It all was accomplished after March, when people were told to stay home to prevent spreading the coronavirus. There were 10 to 12 students on the team, both undergraduates and graduate students.

The toy is aimed at children 5 to 12 years old.

“It’s a lot harder for them to just sit down in a Zoom call and start a conversation like the rest of us,” said Kaitlynn Pineda, a rising senior. “Those kids tend to play with each other. So we wanted to see how we can still have kids play with each other while stuck at home or in the remote setting.”

While one child has the robot in his or her home, the other can control it using the app on a smartphone or tablet. “It could be a different experience” for each child, said Pineda. “But at the end of the day you still have that interaction.”

The robots, called Vector, are off-the-shelf toys made by a company, Anki, that has gone out of business, said Brian Scassellati, a professor of computer science specializing in robotics. The students wrote the software that allows remote control of the robots, which also can be played with alone.

The robots, which retail for $250, are free to Connecticut residents who sign up on the Yale website, robotsforgood.yale.edu, but there are fewer than 50 left to give away.

“This comes from a donation from some Yale alumni,” Scassellati said. When they heard about the project, the alumni said, “this is fantastic but we’d love to see some New Haven-area kids get these robots and we’ll pay for them,” Scassellati said.

In addition to children playing with the robot, “We’ve also seen teachers using it with students, grandparents using it with a child,” Scassellati said. “There’s no limit to that in terms of who can actually use the device.

“And it’s amazing some of the things people come up with to use it with. We’ve heard of kids playing hide and seek basically where they hide from the robot and the remote child drives around and tries to find them.

Or they build little obstacle courses and racetracks and challenge each other to run around and see if they could beat a time or get over a big obstacle and all sorts of things,” he said.

“The point here is to connect those two sides into enjoying a common activity,” said Marynel Vázquez, assistant professor of computer science. “If they’re having fun, I think it’s going to be pretty engaging no matter which side you’re on.”

The robot has a display screen, like a digital watch face. “You can do very simple things like changing the eye color, which turns out is a lot of fun,” Vázquez said. “The eyes are one of the most attractive features of the robot.”

There is also a small forklift in front that lifts objects.

You can even train it to recognize you and say your name, Vázquez said.

“You can also use the speaker to say a couple of simple things to express emotion,” she said. Vector is “attractive and cute and has a little bit of a personality.”

The app is available free both in the Apple App Store and Google Play.

Because it can be used on numerous devices, the students tested the app on as many platforms as possible. “We tried to break it (the app, not the robot) in any way possible,” Pineda said.

Whether they can give away more robots will depend on whether more donations come in, Scassellati said. It appears the toy will continue to be available.

“Lucky for us … there was another company that later on took on … the intellectual property for the robot, and so there’s hope that they’ll keep working for a while longer,” Vázquez said. Any improvements to the app automatically will update it on the device.

Vázquez said she learned that, even as a computer scientist, “building robots’ mobile apps is hard.” She said the students “spent hours and hours and hours making the system and all the infrastructure necessary to make sure that things were simple to use.”

“That was actually a huge effort in my opinion so I hope that people who use it find it valuable, because their effort was a big deal this past spring,” she said.

Would it have happened if it weren’t for a pandemic? “Very unlikely,” she said.

“There’s two parts to this,” Scassellati said. “One is that these students would have been working on other course-related projects and they wouldn’t have turned their attention to this.

“But also, I don’t think they would have recognized the need. Normally, kids this age, they’d be in school and they’d be very happily getting a lot of this direct person-to-person play right there or in the neighborhoods or in after-school programs,” she said. “And it’s only because we see kids being isolated in this way do we realize how important that is.”

Anki / Courtesy of Yale University Vector, a toy robot, can be played with by two children in different homes by using an app designed by Yale University computer science students.