Welcome to Voice Chapter 11 🎉, our long-running sequence the place we share all the important thing developments in Open Voice. On this chapter, we’ll let you know how our assistant can now management extra issues within the residence, in a number of languages on the similar time, all whereas not speaking your ear off. What’s extra, our checklist of supported languages has grown once more with a number of languages that massive tech’s voice assistants gained’t help. Be part of us for a deeper take a look at this voice chapter in our livestream
Multilingual assistants
Our unique purpose for the 12 months of Voice again in 2023 was to “let customers management Residence Assistant in their very own language”. We’ve come a great distance in the direction of that purpose, and actually broadened our language help. We’ve additionally offered choices that permit customers to customise voice assistant pipelines with the companies that greatest help their language, whether or not run regionally or within the cloud of their alternative. However what should you converse two languages inside your property?
For a while, customers have been in a position to create Help voice assistant pipelines for various languages in Residence Assistant, however interacting with the completely different pipelines has both required a number of voice satellite tv for pc units (one per language) or some sort of automation set off to modify languages
Since even the tiniest voice satellite tv for pc {hardware} we help is able to working a number of wake phrases now, we’ve added help in 2025.10 for configuring as much as two wake phrases and voice assistant pipelines on every Help satellite tv for pc! This makes it easy to help twin language households by assigning completely different wake phrases to completely different languages. For instance, “Okay Nabu” may run an English voice assistant pipeline whereas “Hey Jarvis” is used for French.
A number of wake phrases and pipelines can be utilized for different functions as properly. Need to preserve your native and cloud-based voice assistants separate? Straightforward! Assign a wake phrase like “Okay Nabu” to a totally native pipeline utilizing our personal Speech-to-Phrase and Piper
We’d love to listen to suggestions on how you intend to make use of a number of wake phrases and voice assistants in your house!
Voice with out AI
The entire world is engulfed in hype about AI and including it to all of the issues — we’re not precisely quiet in regards to the cool stuff we’re doing with AI. Whereas powering your voice assistants with AI/LLMs makes them rather more versatile and highly effective, it comes at a value: paying to make use of cloud-based companies like OpenAI and Google, or expensive {hardware} and vitality to run native fashions by way of methods like Ollama. We began constructing our voice assistant earlier than AI was a factor, and thus it was designed with out requiring it. We proceed to make nice progress in the direction of delivering a strong voice expertise to customers who need to preserve their residence AI free — preserving AI opt-in solely and never required
Help, our built-in voice assistant, can do a number of cool issues with out the necessity for AI! This consists of a ton of voice instructions in dozens of languages for:
- Turning lights and different units on/off
- Opening/closing and locking/unlocking doorways, home windows, shades, and so on
- Adjusting the brightness and colour of lights
- Operating scripts and activating scenes
- Controlling media gamers and adjusting their quantity
- Taking part in music on supported media gamers by way of Music Assistant
- Beginning/stopping/pausing a number of timers, optionally with names
- Including/finishing objects on to-do lists
- Delaying a command for later (“flip off lights in 5 minutes”)…
- …and extra!
Need to embrace your personal voice instructions? You possibly can rapidly add customized sentences to an automation, permitting you to take any motion and tailor the response.
The simplest method to get began is with Residence Assistant Voice Preview Version, our small and easy-to-start with Voice Assistant {hardware}. This, mixed with a Residence Assistant Cloud subscription, permits any Residence Assistant system to rapidly deal with voice instructions, as our privacy-focused cloud processes the speech-to-text (turning your voice into textual content for Residence Assistant) and text-to-speech (turning Residence Assistant’s response again into voice). That is all with out the usage of LLMs, and helps the event of Residence Assistant 😎.
For customers wanting to maintain all voice processing native, we provide add-ons for each speech-to-text and text-to-speech:
All of this collectively reveals simply how a lot could be performed with no need to incorporate AI, regardless that it could actually do some fairly superb issues
Extra intents
Intents are what join a voice command to the appropriate actions in Residence Assistant to get one thing performed. Whereas the tip result’s typically easy, resembling turning on a light-weight, intents are designed as a “do what I imply” layer above the extent of primary actions. Within the earlier part, we listed the kinds of voice instructions that intents allow, from turning on lights to including objects to your to-do checklist. During the last three years, we’ve been progressively including new and extra advanced intents.
Lately, we’ve added three new intents to make Help even higher. To regulate media gamers, now you can set the relative quantity with voice instructions like “flip up the quantity” or “lower TV quantity by 25%”. This provides to the present quantity intent, which lets you set absolutely the quantity degree like “set TV quantity to 50%”.
Subsequent, it’s now attainable to set the velocity of a fan by share. For instance, “set desk fan velocity to 50%” and even “set followers to 50%” to focus on all followers within the present space. Be sure to expose the followers you need Help to have the ability to management.
Lastly, now you can inform the youngsters to “get off your garden” as a result of your robotic goes to mow it! Making use of the lawn_mower integration, your voice assistant can now perceive instructions like “mow the garden” and “cease the mower”. Paired with the present sensible vacuum instructions, chances are you’ll by no means have to elevate a finger once more to maintain issues clear and tidy.
Ask query
Image this: you come residence from work and, as you enter the lounge, your voice assistant asks what sort of music you’d like to listen to whereas making ready dinner. Because the music begins to play, it mentions you left the storage door open and needs to know should you’d prefer it closed. After dinner, as you’re hanging out on the sofa, your voice assistant informs you that the temperature exterior is decrease than your AC setting and asks for affirmation to show it off and open the home windows.
Certainly you’d want a strong LLM to carry out such wizardry, proper? With the Ask Query motion, this could all be performed regionally utilizing Help and some automations!
Inside an automation, the Ask Query motion permits you to announce a message on a voice satellite tv for pc, match the response in opposition to an inventory of attainable solutions, and take an motion relying on the person’s reply. Whereas solutions could be open-ended, resembling a musical artist or style, limiting the attainable solutions permits you to use the totally native Speech-to-Phrase for recognizing speech with out an web connection.
Improved sentence matching
Help was designed to run quick and totally offline on {hardware} just like the Raspberry Pi 4 for a lot of completely different languages. It really works by matching the textual content of your voice instructions in opposition to sentence templates, resembling “activate the {title}” or “flip off lights within the {space}”. Whereas that is very quick and simple to translate to many languages
Beginning in Residence Assistant 2025.9, we’ve included an improved “fuzzy matcher” that’s a lot better at dealing with further phrases or various phrasings of our supported voice instructions.
The fuzzy matcher is pre-trained on the present sentence templates, so we can use it for all of our supported languages. Nevertheless, that is initially solely out there for the English language and we’re working to find out one of the best ways to allow this for different languages.
Non-verbal confirmations
After a voice command, Help responds with a brief affirmation like “Turned on the lights” or “Brightness set”. This allows you to comprehend it understood your command and took the suitable actions. Nevertheless, should you’re in the identical room because the voice assistant, this affirmation is redundant; you possibly can see or hear that acceptable actions have been taken.
Beginning with Residence Assistant 2025.10, Help will detect if the voice command’s actions all came about inside the similar space because the satellite tv for pc system. In that case, a brief affirmation “beep” will likely be performed as an alternative of the complete verbal response. Apart from being much less verbose, this additionally serves as a reminder that your voice command solely affected the present space.
Non-verbal confirmations is not going to be utilized in voice assistant pipelines with LLMs, for the reason that person could have particular directions of their immediate, resembling “reply like a pirate”, and we wouldn’t need to deprive you of a enjoyable response, me mateys 🏴☠️.
Textual content-to-speech streaming
Giant language fashions (LLMs) could be particularly verbose of their responses, and we rapidly realized that this uncovered a weak spot in Residence Assistant’s text-to-speech (TTS) implementation. For many of its life, TTS in Residence Assistant has required the complete response to be generated earlier than any audio could be performed. This meant a number of ready for multi-paragraph LLM responses, particularly with native TTS methods like Piper.
Fixing this required an overhaul of the TTS structure to permit for streaming. As an alternative of ready for your complete audio message to be synthesized earlier than taking part in, we enabled TTS companies inside Residence Assistant to work with chunks of textual content (enter) and audio (output). As chunks of textual content are streamed in from an LLM, the TTS service can synthesize audio chunks and ship them out to be performed instantly.
To reveal the good thing about streaming, we requested an LLM to “inform me a protracted story a few frog” and timed how lengthy it took to begin talking the (multi-paragraph) response. With out streaming, each Residence Assistant Cloud and Piper took greater than 5 seconds to reply! That is lengthy sufficient to make you surprise in case your voice assistant heard you 😄 With streaming enabled, each TTS companies took about half a second to begin speaking again. A 10x enchancment in latency!
New Piper voices
Piper, our homegrown text-to-speech device, continues to develop with help for a number of new languages! These new voices have been educated from publicly out there voice datasets, and can be found now within the Piper add-on:
- Daniela (Argentinian Spanish)
- Pratham, Priyamvada, Rohan (Hindi)
- Information TTS (Indonesian)
- Maya, Padmavathi, Venkatesh (Telugu)
Need to know what the brand new voices sound like? You possibly can hearken to samples
In case your language is lacking from Piper, otherwise you don’t like the present voices to your language, we’re all the time on the lookout for volunteers to contribute their voices! Please contact us at [email protected]
Conclusion
Up to now three years, we’ve made nice strides with Residence Assistant Voice on each the {hardware} and software program fronts. Customers in the present day have all kinds of selections on the subject of voice: from totally native to utilizing the newest and best AI to energy their sensible houses. The beauty of our experimentation with AI is that there are not any buyers on the lookout for returns, faux cash, or “rug-pulls”. We do all the pieces for you, our group. We’re on this for the lengthy haul, and need this all to be your alternative, preserving you in full management of whether or not you need to use this know-how or keep away from the hype utterly.
A lot of the superior work performed on voice is simply attainable with the help of our group, particularly those that subscribe to Residence Assistant Cloud or anybody who has bought our Residence Assistant Voice Preview Version (each nice methods to get began with voice).
