Home

Any experiences with a self-hosted assistant like the modern Google Assistant? Looking for something LLM-powered that is smarter than older assistants that would just try to call 3rd party tools directly and miss or misunderstand requests half of the time.

I’d like integration with a mobile app to use it from the phone and while driving. I see Home Assistant has an Android Auto integration. Has anyone used this, or another similar option? Any blatant limitations?

Multi-billion dollar companies like Google and Apple can’t even figure this shit out, doubt some nerd is gonna do it for free.

Home Assistant can absolutely do that. If you are ok with simple intent based phrasing it’ll do it out of the box. If you want complex understanding and reasoning you’ll have to run a local LLM, like Llama, on top of it

Home Assistant can do that, the quality will really depend on what hardware you have to run the LLM. If you only have a CPU you’ll be waiting 20 seconds for a response, which could also be pretty poor if you have to run a small quantized model

Maybe things have improved but the last time I tried the Home Assistant er- assistant, it was garbage at anything other than the most basic commands given perfectly.

1. They can and do; 2. LLMs can do tool calling just fine, even self-hosted ones.

yeah, that’s what I’m looking for. Do you know of a way to integrate ollama with HA?

LOL they can’t even reliably turn the lights on, WTF are you talking about?

I don’t think there’s a straightforward way like a HACS integration yet, but you can access it from the web and save the page to your homepage?

https://sh.itjust.works/pictrs/image/ae206b55-421f-4035-ac6a-a51774739fcd.jpeg

maybe last you tried it was over 6 months ago, maybe you’re using the old google assistant, or idk, but it definitely works for me

Everything I’ve read says Gemini is like 10x worse than Google Assistant.

You have to run an LLM of your own and link it, if you want quality even close to approaching Google, but the Home Assistant with the Nabu Casa “Home Assistant Voice Preview Edition” speakers are working well enough for me. I don’t use it for much beyond controlling my home automation components, though. But it’s still very early tech anf it doesn’t understand all that much unless you add a lot of your own configurations. I eventually plan to add an LLM, but even just running on the home assistant yellow hardware with a raspberry pi compute module 5 works ok for the basics though there is a slight delay.

I haven’t tried, but Nabu Casa also offers a subscription service for the voice processing if you want something more robust and can’t host your own LLM, but thst means sending your data out, even if they have good privacy policies, which I’m not interested in, because while I somewhat trust Nabu Casa’s current business model and policies, being hosted in the US means it’s susceptible to the current regime’s police-state policies. I’m waiting for hardware costs to recover from the AI bubble to self host an LLM, personally.

so you haven’t tried it recently

There’s an Ollama integration that adds it as a conversation agent.

And there’s another custom component, intrgrating all servers which offer an OpenAI-compatible API endpoint. That one also works well: https://github.com/jekalmin/extended_openai_conversation

Livekit can be used to build voice assistants. But it’s more a framework to program it yourself, not a ready-made solution.

Also https://github.com/acon96/home-llm/blob/90c0edc0907e4567eeca89a6f89b43c0b0b807eb/docs/Setup.md

Try ollama.com you can download and try whatever you want. Quality is mostly how much VRAM your video card has.

I don’t like the guy’s breathless over-enthusiasm, but NetworkChuck has a video on how to integrate LLM-based voice assistants with HomeAssistant using Whisper and Ollama.

ah yes, I stopped watching the guy because of that and the clickbait, but he does make some interesting content sometimes.

ah, this puts it together and it’s exactly what I was looking for, thanks

Gemini is a hot pile of garbage.

When I ask Gemini for directions it starts to give me a definition, as opposed to opening maps and showing me the way. If I ask to turn off the lights I get a conversation and I end up walking to the light switch myself.

idk what to tell you, because I just tried it and it works

Heh, I’ve actually moved away from using Google Home stuff because it’s shoving Gemini down my throat and it’s been worse in the last six months than it was a year ago.

You gotta hook it to a local LLM. Then it’s boss.

Using Home Assistant with Qwen locally. It functions better than any version of Google Home I’ve had. Understands me without having to think about how I say things. Can ask it for one or multiple things at the same time. Can even make it so that it pretends to be Santa Claus while responding. My wife was ecstatic when she heard the Ho-ho-ho while asking to turn the coffee machine on on Christmas.

HA with local LLM on Ollama. Can imtegrate the Android app as the default phone assistant. I don’t think it can use a wake word on the phone though. I invoke it by holding the power button, like a walkie.

He covers interesting subjects but he believes his audience is dumb and unknowledgeable which leads to this. He thinks he has to adapt to the regular youtube game to retain us, but he is just boring everyone

You are not wrong, I read an article a while (like several years) ago from an amazon engineer who revealed they coulnd’t wrap their minds around upgrading Alexa to the latest technologies. He told about them going in circles and people eventually leaving the company because of how bad of a mess it was. And it turns out they havn’t made any progress since then, Alexa is still absolute garbage. It can’t hear you when there is music, it can’t hear you when you cook, it can’t hear you from more than 5 meters, and even when it does hear you, it responds completely off the track and goes on and on, refusing to stop the nonsense

Any pointers where to begin?

Install Ollama on a machine with fast CPU or GPU and enough RAM. I currently use Qwen3 that takes 8GB RAM. Runs on an NVIDIA GPU. Running it on CPU is also fast enough. There’s a 4GB version which is also decent for device control. Add Ollama integration in Home Assistant. Connect it to the Ollama on the other machine. Add Ollama as conversation agent to the Home Assistant’s voice assistant. Expose HA devices to be controllable. That’s about it.

But what’s acting like the little box in your house that listens like an Amazon echo?