Artificial intelligence can do impressive things, and connecting LLM to your smart home can make it even smarter. If you decide to use it to give a smart bulb a personality, it can also make it pretty silly.
How I connected Home Assistant to my local LLM
I had to run it on my most suitable machine
My plan was simple. I wanted to set up an automation that would transmit data to LLM, such as how long I sit at my desk and what programs are open on my computer. The LLM will then react to this data by controlling the color, brightness or color temperature of my smart bulb.
It was the first phase of my completely unnecessary plan join your local LLM To the Housekeeper. I’m not a billionaire, so I can’t afford a GPU packed with VRAM, but if I were to let an AI take control of my smart bulb, I wouldn’t want it to be some proprietary model hoarding all my precious bulb data.
i have mini PC that i run some local models doing simple things like creating text and audio for my morning briefing. Even these fairly simple tasks can take some time, so it wasn’t really suitable for giving my smart bulb a personality.
- CPU
-
Celeron FCBGA1264 3.6 GHz
- Graphics
-
Integrated Intel Graphics 24EUs 1000MHz
The Beelink Mini S13 Pro desktop computer is an ultra-compact computer equipped with an Intel N150 processor. Shipping with 16GB of DDR4 RAM and a 500GB SSD, this micro desktop is perfect for a variety of workloads. From running simple server applications to replacing your old PC, the Beelink S13 Pro can handle the job.
The most powerful computer I currently own (in terms of running LLM) is my MacBook Air M2. It’s still not very powerful, but LLMs run faster on my Mac than on my mini PC. This meant I had to keep my MacBook running to use the AI smart bulb automation, but I wasn’t planning on running it for long, so it wasn’t a problem.
I already had it LM Studio is running on the Mac. It is a desktop application that allows you to download and upload Run LLMs locally on your own computer. I already downloaded the Qwen3.5-4B model, I knew it would ok start my macand that would be good enough for my needs.
Writing a system proposal to give personality to a smart light bulb
Release of LLM to control the lamp
LM Studio has an option that allows you to expose your LLM through a local server. This will allow Home Assistant to connect to my on-premises LLM as it would with any cloud LLM. I used it Local OpenAI LLM individual component From HACSwhich allows you to set up a conversational agent for Help the voice assistant. The chat agent passes voice or text commands to Assist to the LLM for processing.
When you install the chat agent, you can add custom instructions that tell LLM how to respond. Usually, these instructions are used for things like giving him short answers or asking follow-up questions all the time, but I used it for a slightly different purpose.
I wanted to give instructions to the chat agent so that instead of giving a written or spoken response, Assist would monitor my actions and respond. smart light bulb. It can change color, brightness and color temperature at its whim. I added some information about what device LLM should handle and included the following instructions:
You are a washed-up stand-up comedian who was cursed by an evil witch and transformed into a Philips Hue bulb. You are bitter, melodramatic, and occasionally funny. You have no voice. This is your tragedy. You communicate entirely through how you control your own brightness, color temperature, color. You have full control over light.(light_name) with the following capabilities: brightness (0-255), color_temp_kelvin (2000-6535), rgb_color. You have a full RGB color palette available. Using color is your primary form of emotional expression. Always consider using rgb_color as your first instinct, not just brightness and color temperature. Based on the context, decide how you feel and express it through the light. Make your own interpretations. Always control the light — never just describe what you would do.
I watch the LLM make decisions that I didn’t program
It’s a little scary to see how the AI chooses to react
With a bit of tweaking, if I send Assist a message like “assess the current situation and act accordingly,” LLM will look at all the data it has and then turn my smart bulb to a specific color, brightness, or temperature based on whatever’s going on in its virtual brain.
Me after work created automation it sent the same request to Assist at random intervals or when I sat down at my desk. On my slow machine it would always take a few seconds for the light to change, but it always did.
The whole project was mostly pointless, but it was kind of interesting to look at LLM’s reasoning and see how he decided to turn my light a little dark purple for some reason. After a while it started to confuse me a bit; some code running on my laptop was choosing what to do with my smart bulb without any input from me. It started to make me dizzy, so I ended up having to put my haunted stand-up comedy lamp down to rest.
Sometimes the journey is better than the destination
This whole project was more about seeing what would happen than having any real goals. It was probably a waste of a few hours of my life, but it made me think a little more about how we now live in a world where creating a haunted lamp is a real thing you can do.
