Today, you can ask Alexa to turn on the lights or Siri to tell you your bedroom’s temperature, and sometimes they will. Alternately, you might hear, “You have fifteen devices named lights; Which one do you want to have control over? or “The temperature is 53 degrees right now in Kathmandu.” But what if your voice assistant could respond to vague statements like “I’ve had a rough day;” in addition to being always accurate? What would be a good way to relax? with answers that are “intelligent”? For instance, by lowering the blinds, lowering the temperature, lowering the lights, and waiting in line for some Netflix goodies?
Alex Capecelatro, co-founder of the Josh.ai home automation system, asserts that this is the potential of voice assistants based on brand-new AI language models. Using OpenAI’s ChatGPT, Josh.ai has already begun working on a prototype integration. In this proof-of-concept video, Capecelatro asks Josh’s assistant to open the shades, turn off the music, and tell him the weather (Josh already has the ability to control three things simultaneously). He then uses voice commands for the smart home that sound more natural, like “I’m filming a video; The voice assistant responds clumsily by increasing the room’s lighting because “it’s kind of dark in here.”
It is intriguing that AI language models could be used to parse natural language to improve smart home control. Capecelatro considers it to be the future. He continues, “We’re trying to figure out how good we can get at controlling your environment in a way that feels more natural and intuitive.”
Today, voice assistants typically require precise language and frequently misinterpret simple smart home commands with information requests, resulting in irritated and sometimes ineffective responses. When Capecelatro and Quark founder Tim Gill founded Josh.ai in 2015, they set out to solve this issue. No matter how you phrase the request, its eponymous voice assistant aims to be excellent at controlling connected devices.
Josh is able to interpret when it hears “satellites” rather than “turn on the lights” using extensive knowledge graph models and take the appropriate action. Although “Open the drapes” sounds like “Get some grapes,” Josh is aware that you do not reside in a vineyard. According to Capecelatro, “we know what you mean even when you say “turn on the goddam lights” because we spend a lot of time working under the hood to fix mishearing, work with different accents, understand imperfect sentences, and the like.”
Josh is currently only available as a voice control layer for custom smart home installations that are powered by Crestron, Control4, or Josh’s own independent smart home control system. Josh has established a reputation for being a voice assistant that is more dependable and more private in that more secure setting, where the system is set up and largely controlled by a professional installer and makes use of Josh.ai’s proprietary hardware, despite the fact that Josh has a higher entry cost. Capecelatro says that even though Josh has a cloud component, the majority of requests are processed locally on the Josh Core or the Josh Micro, and when using cloud-based APIs, identifiable information is removed.)
The new generation of large language models (LLMs) utilized by ChatGPT and other chatbots is the focus of the business, which recently announced a partnership with Amazon. These systems, according to Capecelatro, will make today’s voice assistants much more useful. Even Josh, Alexa, and Google will no longer be able to function in the same way they did a year ago. Capecelatro asserts, “It’s just not going to be enough.” Businesses like mine won’t be around in a year if we don’t use ChatGPT technology. Anyone who uses voice control in the home will need it in the future.
The knowledge base that a ChatGPT integration brings to the voice assistant is a significant advancement for Josh.ai, which does not possess the breadth of general knowledge that its rivals do. Capecelatro states, “We’ve always wanted to make Josh as smart as possible, but we’re a small team.”
The promise, however, lies in combining the conversational capabilities of AI language models with the context that a home automation system can provide for the smart home in general. Josh could, for instance, convert natural language commands into actions in your home if he knew what smart devices you have in your home and how you use them. Say, “Hey Josh, it’s getting dark and the kids are about to come home.” Can you ensure that everything is prepared?” and the voice assistant could, for instance, turn on the lights in the porch, start preheating the oven, lower the blinds, and turn on the lights in the kitchen.
Josh has also worked on making use of ChatGPT to discover media in the smart home. something that has not yet been connected. According to Capecelatro, “if you don’t know what you want, voice control is not ideal.” You can use the Ava remote to browse the content you want to watch thanks to our integration. You can ask, “What are some really good shows on Netflix that maybe are romcoms and feature (this actor)?” by including ChatGPT in the mix. A list can be compiled by ChatGPT and displayed on the remote’s screen. That is family film night arranged, then.
Josh’s AI upgrade hasn’t gone live yet, and Capecelatro says that the company is keeping an eye on the growing technology of other companies in this field to see if they can provide a better model. In addition to ChatGPT’s current slowness (the video was edited to speed it up), there is the real problem of AI producing, well, crap. Additionally, the end of the dataset on which ChatGPT was trained in the middle of 2021. It is important to note that when Josh is asked in the demo video, “What are some shows to watch on Netflix?” the most recent show that is listed was released in 2019.) However, Capecelatro claims that the smart home will soon feature some kind of generative AI voice assistant.
There is absolutely a need for caution. Given examples of generative AI basically regurgitating content without a filter, no company wants a racist, homophobic, or homicidal voice assistant spewing its “opinions” into people’s homes through their hardware. We are exercising extreme caution. According to Capecelatro, “We could have gone live with the ChatGPT integration immediately.” That is not what we do. because we don’t want to give people data that isn’t very good. We won’t tell a lie.
It is reasonable to assume that Google, Apple, and Amazon are all considering ways to incorporate new AI language models into their voice assistants. The company intends to take its time to determine how to set up the appropriate safeguards, which will be necessary for this technology to be implemented in the smart home. According to Capecelatro, “I think Microsoft and Google jumped the gun a little bit with their search ChatBot models, and they are now seeing the consequences” of recent high-profile launches that quickly went wrong.
Smart home enthusiasts have already discovered ways to use Siri Shortcuts to integrate ChatGPT into their smart home, so it is reasonable to assume that Google, Apple, and Amazon are all examining ways to incorporate new AI language models into their voice assistants. Heck, maybe even Microsoft will bring back Cortana. Talking to a smart speaker is much simpler than typing into a web browser.
But is this kind of artificial intelligence really something we want in our homes? Are we so desperate for a voice assistant that “just works” that we would be content with one that could also attempt to instruct my eight-year-old about quantum physics? In my opinion, the Holy Grail here is a dependable, voice-controlled smart home system that understands what I mean when I say, “Turn off the goddam lights,” rather than an omniscient intelligence running my home.
Although the promise of a voice assistant that is naturally competent and extremely intuitive—a flawless butler for your home—is very appealing, I worry that the reality may be more like Space Odyssey than Downton Abbey. However, let’s see if I’m wrong.
