Zuckerberg wrestles with smart home
Facebook CEO encounters obstacles with own system
San Francisco — Artificial intelligence is moving into our homes, particularly Mark Zuckerberg’s. But this stuff of science fiction may still be more fantasy than utility for now.
The Facebook founder and CEO, who for the last year has been building a smart system inspired by the Iron Man character Jarvis as a virtual butler to run his household, updated followers on the project’s progress Monday.
His conclusion so far? Hooking up artificially intelligent homes may not be ready for prime time, at least for most of us.
With the Jarvis project, Zuckerberg wanted to use his voice to control everything in his house, from the music to the lights to the temperature. He also wanted Jarvis to swing open the front gate for friends by recognizing their faces. Essentially, the Jarvis project is like Zuckerberg’s homemade version of Amazon’s Alexa service or Google’s Home.
Nearly a year ago, the Facebook CEO said he planned to build an AI system as one of the personal growth challenges he gives himself each year. For Zuckerberg, this was a return to his programming roots. This isn’t the first time he has returned to coding. His personal growth challenge in 2012 was to code every day. But this challenge connected him to a new wave of computer science that is vital to his company’s growth.
“My goal was to learn about the state of artificial intelligence — where we’re further along than people realize and where we’re still a long ways off. These challenges always lead me to learn more than I expected, and this one also gave me a better sense of all the internal technology Facebook engineers get to use, as well as a thorough overview of home automation,” Zuckerberg said in a Facebook post.
“So far this year, I’ve built a simple AI that I can talk to on my phone and computer, that can control my home, including lights, temperature, appliances, music and security, that learns my tastes and patterns, that can learn new words and concepts, and that can even entertain Max.”
Artificial intelligence is increasingly climbing behind the wheel of driverless cars and nudging smartphones to get smarter by delivering relevant information unprompted. Amazon is even testing a supermarket system with no cash registers. But the technology is not yet up to the ambitions of technologists.
One of the biggest challenges Zuckerberg faced: Connecting all of his home devices. First he had to write code to connect all these different systems that speak different languages, he said.
“We use a Crestron system with our lights, thermostat and doors, a Sonos system with Spotify for music, a Samsung TV, a Nest cam for Max, and of course my work is connected to Facebook’s systems. I had to reverse engineer APIs for some of these to even get to the point where I could issue a command from my computer to turn the lights on or get a song to play,” Zuckerberg wrote.
A central problem: Most appliances aren’t connected to the Internet yet, he said.
“For example, one thing I learned is it’s hard to find a toaster that will let you push the bread down while it’s powered off so you can automatically start toasting when the power goes on. I ended up finding an old toaster from the 1950s and rigging it up with a connected switch.” He faced similar problems connecting a food dispenser for his dog, Beast, or another project — rigging a “cannon” to fire his trademark gray T-shirts. His conclusion? “For assistants like Jarvis to be able to control everything in homes for more people, we need more devices to be connected and the industry needs to develop common APIs and standards for the devices to talk to each other,” Zuckerberg said.
Creative Strategies analyst Carolina Milanesi says Zuckerberg is building what many people want: a digital personal assistant catering “to their needs in all kind of ways from understanding to executing the command.” But the technology is not there yet, she said.
“Zuck also clearly explains how the difficult part in building Jarvis was to have him understand his commands as well as his wife’s,” she said. “This is the point most people who have tried Alexa, Siri, Google Home would tell you to be the most annoying.”