Yes, you can replace Alexa with a local voice assistant in Home Assistant. For device control it is arguably the better tool. Speech-to-text with Whisper, text-to-speech with Piper and Home Assistant’s own intent matching all run on hardware you own, so a command like “turn on the kitchen lights” never leaves the house. Open-ended questions need a language model. We use a cheap cloud one that costs us about $2 a month. The hard part turned out not to be the software. It was the microphones.
This is how we built ours during a week-long Home Assistant build in late September 2026, with Claude Code doing the research, configuration and debugging alongside us. The assistant handles English and Polish, runs in Docker, and broke in several instructive ways before it settled down.
The problem we were actually solving
We did not start with an Echo to rip out. The house already had several Google Cast and Nest devices, five Sonos speakers and two TVs. By the end of the week it also had 24 lights, 9 motion sensors and 32 automations. Even so, our goal matched what sends most people searching for an Alexa replacement. We wanted the house to respond to voice without every utterance passing through a vendor’s cloud. We wanted it to work properly in two languages. And we wanted to see, and change, exactly how it decides what to do.
We also did not want to take away convenience the household already relied on. So we kept Google Home voice control running alongside, using the free manual cloud-to-cloud Google Assistant link rather than a paid subscription. Home Assistant’s own Assist pipeline became the private path. Running two voice systems over one set of entity names caused one of our odder bugs, which we come back to below.
The pipeline we built
Home Assistant’s voice control is a pipeline of swappable parts. A speech-to-text engine turns audio into text, a conversation agent works out the intent, and a text-to-speech engine speaks the reply. The speech components connect to Home Assistant over the Wyoming protocol, so each one can live in its own container. Our whole stack runs under Docker Compose on a Mac mini-class desktop with 16 GB of RAM and an 8 GB Docker VM. Claude Code added the Wyoming integrations through the Home Assistant API using a long-lived token, not by clicking through the UI.
Speech in and out stays on the box
Whisper, running as a Wyoming container, does speech-to-text. Piper does text-to-speech, with a British English voice and a Polish voice. Both run locally.
Our first English setup used a separate Parakeet model of roughly 575 MB. Testing then showed that the multilingual faster-whisper model transcribed English identically, so we retired Parakeet. One Whisper container now serves both languages. That looks like housekeeping, but it turned out to matter, for reasons covered in the memory section below.
Most commands never touch a language model
The single most useful setting is “prefer handling commands locally”. With it on, Home Assistant tries its built-in intent matching first. “Turn on X” is resolved directly against your entities and areas, instantly, for free, and without depending on any external API being up. Only requests the local matcher cannot handle fall through to the LLM.
If your goal is to replace Alexa for lights, plugs and scenes, this alone gets you most of the way with no cloud involved. The rest comes from clean structure. Our house is split into 12 areas, and devices carry both English and Polish aliases, so the matcher has good names to work with.
The LLM brain, and how we chose it
Some requests need reasoning over state, such as “is anyone in the living room?” or “what’s the weather doing?”. For those we use an LLM conversation agent routed through OpenRouter.
Our first pass was desk research, which pointed at Claude Sonnet 5 at an estimated $14 a month. We did not take that at face value. Instead, the agent benchmarked 10 tool-calling models on real Home Assistant-style commands in English and Polish. Gemini 2.5 Flash-Lite won on the numbers: 4/4 correct, around 0.5 seconds per answer, and roughly $0.00003 per call.
We did consider running the brain on our own hardware, since a local GPU box already runs the vision model that describes our CCTV snapshots. For voice we accepted the cloud trade-off. Device control already stays local, so the LLM only ever sees questions. If you want literally everything on-premises, our write-up on running a self-hosted LLM on your own GPU box covers what that really involves.
What broke, and what we changed
English questions answered in Polish
With a single conversation agent serving both languages, the model would occasionally answer an English question in Polish. The English Piper voice then read that Polish aloud, which is about as unhelpful as a voice assistant gets.
Prompting harder was not a dependable fix. A structural change was. We split it into two agents, one English-only and one Polish-only, sharing the same prompt body. Each sits in its own pipeline with its own speech-to-text language, its own agent and its own Piper voice. In testing, 4/4 answers came back in the right language. We cover the bilingual setup in more depth in Home Assistant Local Voice Assistant: Bilingual for $2 a Month.
Confident nonsense about presence and weather
The benchmark winner had flaws the benchmark did not catch. It hallucinated who was at home, and it mixed up today’s weather with tomorrow’s. Neither problem crashes anything. Both quietly destroy trust in the assistant.
We rewrote the prompt with three hard rules:
- Presence comes only from occupancy sensors.
- Only current weather is used.
- Replies are always in the user’s language.
We then moved to Gemini 3.1 Flash-Lite. It produced zero failures across 14 test questions, and its Polish reads as natural rather than translated.
The lesson reaches well beyond voice. A model that scores well on a short benchmark can still be wrong in exactly the ways your environment cares about. Explicit constraints on where facts may come from are how you close that gap. Our guide to LLM integration in production systems goes further into that pattern.
Voice models compete for memory
Voice did not run in isolation. The same host also runs Home Assistant, Frigate for CCTV, the bridges for three USB radio sticks, and some unrelated stacks.
When Frigate stalled, the Docker VM had about 83 MB free and swap was full. Turning off Frigate’s semantic search was the big fix, but stalls came back once more before we retired the extra English speech model. After that, available memory in the VM rose from 485 MB to 783 MB. Removing a roughly 575 MB model that turned out to be redundant was one of the cheapest stability wins of the week. If you are adding voice to an existing Home Assistant host, check your free memory before you add models, not after something else falls over.
Names that voice assistants grab
Our motion-light automations had toggle helpers named “… motion light”. When someone asked Google to turn off a light, it sometimes matched one of those helpers and switched the automation off instead.
Neither assistant was misbehaving; the names were ambiguous. We renamed the helpers so voice commands cannot collide with them. Whenever you add a voice layer, audit the names of helpers, groups and scripts, not just devices. Voice matching does not know which entities you meant to be user-facing.
The microphone problem nobody mentions first
This is where replacing Alexa gets genuinely hard. Smart speakers are cheap, decent far-field microphones in every room, and you may not be able to reuse them.
Nest speakers cannot act as Assist microphones, because Google blocks third-party AI on them. They are still useful as text-to-speech and announcement speakers, and that is how we use them. Home Assistant can talk through them; it just cannot listen through them.
For input, we set the Home Assistant phone app as the default Android assistant, so the usual assistant gesture on the phone goes to our pipeline. Dedicated room hardware, either ESP32 satellites or Home Assistant’s own Voice hardware, is a future option for us, not something we have deployed. If your plan is hands-free voice in every room, budget for that hardware from the start rather than discovering the gap after the software works.
What it cost
The running cost is small and easy to predict. We measured $0.0024 per question that reaches the LLM, which works out at about $2 a month at 30 questions a day. Device commands handled locally cost nothing. The API key has a hard $50 spend limit, so a runaway loop cannot become a surprise bill.
The key itself is generated and stored outside the agent’s context, in a self-hosted password vault. It is referenced by name and never printed. That is the same discipline we apply anywhere an agent touches credentials; our notes on data security and privacy in AI projects explain the reasoning.
The build cost was time, and the voice work was only part of a week-long project. The LLM benchmark ran on 28 September, and the two-pipeline language fix landed on 29 September. The agent also produced an 8-page household voice-command guide, as a PDF in English and Polish. We treat that as part of the build, not an extra, because a voice assistant the household does not know how to address will not get used.
We needed no new hardware for the software side. The real constraint was memory on a shared host, not money.
What we would do differently
We would skip the desk-research cost estimate and go straight to a benchmark on our own commands. The $14 estimate and the measured $2 are far enough apart that the estimate added nothing.
We would split pipelines per language on day one. A bilingual household is a known requirement, and one agent per language is simpler to reason about than a single agent you keep correcting.
We would also question every model before adding it. The extra English speech model cost memory and gave no accuracy gain, and we only found that out by testing.
We would name entities with voice in mind from the outset, before a second assistant starts grabbing helpers.
Finally, we would plan the host properly. A single 16 GB Docker Desktop machine running Home Assistant, Frigate and voice models is the bottleneck. That is why the stack is moving to Proxmox on two small-form-factor PCs, and why the compose files were written to move across with a folder copy and a few changed settings.
Build, buy or wait: a quick decision checklist
Build it now if most of what you ask Alexa is device control and your Home Assistant install already has clean areas and sensible names. Local intent matching will cover the bulk of your use for free, and an LLM can be added later for questions, with a hard spend cap.
Build it, but plan per-language pipelines from the start, if your household speaks more than one language.
Buy satellite or dedicated voice hardware alongside the software if you need far-field microphones in every room. Existing smart speakers may only be able to serve as outputs.
Wait if your Home Assistant base is not yet stable, or if the household depends on hands-free voice in every room and you cannot cover the microphone gap yet. A voice layer amplifies whatever naming and reliability problems already exist. In the meantime, there is nothing wrong with keeping your current assistant running alongside while the private path matures.
Where D3X Solutions LABS fits
The patterns in this build are the same ones that decide whether a business LLM integration works in production. We put deterministic handling in front of the model so it only sees what it must. We benchmark candidate models on your real inputs instead of headline scores. We set hard rules on where the model may get its facts, keep credentials out of the agent’s context, and cap spend. That is how we approach agent and automation work for teams in Ireland, the UK and the EU, and our breakdown of what it costs to build an AI agent shows how those choices affect budget. If you are weighing a voice interface, an internal assistant or an agent that needs to act on real systems, book a free 30-minute call and we will work through whether to build, buy or wait.
Frequently Asked Questions
Can Home Assistant fully replace Alexa?
For device control, largely yes. Whisper, Piper and Home Assistant's local intent matching handle commands like turning lights on and off without any cloud service. The main gap is hardware: you need microphones in each room, and some smart speakers cannot be reused as Home Assistant microphones.
Does a Home Assistant voice assistant work without the internet?
Device commands can, if you enable "prefer handling commands locally" and run Whisper and Piper on your own hardware. Open-ended questions only need the internet if you route them to a cloud LLM. If you want those offline too, you must run the language model locally, which takes considerably more hardware.
How much does a Home Assistant voice assistant cost to run?
Local speech-to-text, text-to-speech and intent matching cost nothing beyond your hardware. In our build, questions that reach a cloud LLM cost a measured $0.0024 each, about $2 a month at 30 questions a day. We also set a hard $50 spend limit on the API key.
Can I use Google Nest speakers as Home Assistant voice microphones?
No. Google blocks third-party AI on Nest speakers, so they cannot act as Assist microphones. They still work well as text-to-speech and announcement speakers. For input you can use the Home Assistant phone app or dedicated hardware such as ESP32 satellites.
How do I make a Home Assistant voice assistant work in two languages?
Use a separate pipeline for each language, each with its own speech-to-text language setting, conversation agent and text-to-speech voice. A single shared agent can reply in the wrong language. Splitting into one agent per language, with a shared prompt body, solved this reliably in our testing.
Want this working in your business?
D3X Solutions LABS builds AI agents and automation that pay back fast. Book a free 30-minute call and we'll map where this fits in your business.
Get in touch
