Facebook’s Zuckerberg Builds Jarvis Home AI Assistant With Voice And Facebook Messenger Integration

"My personal challenge for 2016 is to build a simple AI to run my home and help me with my work. You can think of it kind of like Jarvis in Iron Man," said Zuckerberg at the time. "I'm going to start by exploring what technology is already out there. Then I'll start teaching it to understand my voice to control everything in our home -- music, lights, temperature and so on."

Nearly a year later, Zuckerberg is back with an update on how things are going. Not surprisingly, Zuckerberg has come a long way with coding, testing, and implementing Jarvis into the home that the shares with his wife and daughter.

For starters, Zuckerberg can communicate with Jarvis in one of two ways. He built a Facebook Messenger bot that can respond to text commands that he enters (and provide text feedback). If he doesn’t feel like lifting a finger, he built an iOS app that leverages iPhone hardware that is positioned throughout the house. These iPhones in essence act as a more expensive version of the Amazon Echo, Echo Dot or Google Home when it comes to responding to voice commands. Zuckerberg says that an Android app is also in the works.

“I've built a simple AI that I can talk to on my phone and computer, that can control my home, including lights, temperature, appliances, music and security, that learns my tastes and patterns, that can learn new words and concepts, and that can even entertain Max,” explains Zuckerberg. “It uses several artificial intelligence techniques, including natural language processing, speech recognition, face recognition, and reinforcement learning, written in Python, PHP and Objective C.”

Zuckerberg went on to explain that he had to teach Jarvis to understand synonyms (i.e. Family Room and Living Room) and to differentiate the sound of his voice with that of his wife. “For example, when I tell it to turn the AC up in ‘my office’, that means something completely different from when Priscilla tells it the exact same thing,” he explains. “That one caused some issues!”

Surprisingly though, Zuckerberg indicates that he much preferred interfacing with Jarvis through text (Facebook Messenger) versus voice:

When I have the choice of either speaking or texting, I text much more than I would have expected. This is for a number of reasons, but mostly it feels less disturbing to people around me. If I'm doing something that relates to them, like playing music for all of us, then speaking feels fine, but most of the time text feels more appropriate. Similarly, when Jarvis communicates with me, I'd much rather receive that over text message than voice. That's because voice can be disruptive and text gives you more control of when you want to look at it.

If you have the time, we highly suggest that you read the lengthy Facebook post which goes into great detail about some of the challenges that Zuckerberg ran into, features that he baked in like facial recognition, and his plans for the future of Jarvis. There’s no word yet if Zuckerberg did in fact get Robert Downey Jr. to provide voice feedback for Jarvis…