One of the best side effects of building this thing: the house stopped being a collection of apps and started being one system.

The heart of it is my Synology DS218+, which I’ve had for years and have been slowly turning into a proper media hub. Jellyfin for streaming, the full arr stack (Sonarr, Radarr, Lidarr, Bazarr) managing the library, Deluge for downloads, Pi-hole filtering DNS, Home Assistant tying the smart bits together.

What changed this month is that the assistant got access to all of it. Now I can ask what’s new on the server and get an answer instead of opening three apps. It can queue a stream to the Apple TV via pyatv, start something on Jellyfin, or tell me when the next episode of something lands. The lights, the Sonos, the Apple TV, the NAS: it’s all just things it can reach now.

The practical win: I used to run a media night like it was a small IT project. Now it’s one message.

The philosophical win: this is the part of self-hosting that’s hard to explain to people who’ve never done it. When everything lives on your own hardware, adding a new capability is a configuration change, not a subscription. My assistant doesn’t need permission to talk to my NAS. I built the permission. That’s the whole point.

Here’s what the assistant actually does while I’m not looking.

The 5:30pm takeaway deal. Every night it checks the Uber Eats offers hub for Hamilton, verifies the actual prices, and texts me the top picks under $20 after the discount. It has rules: max three food places, only deals that are actually cheaper, and Fusion Kebab’s two-for-one is always in the running because it’s been reliable for months. Last week it found a 50% off pizza for $11.50 that was $23 the day before. I don’t browse for dinner anymore. I get briefed.

The spend watchdog. I burn through AI tokens like a teenager with a credit card, so there’s a watchdog that watches my actual usage and only speaks up when something’s wrong: over $3 in a day, or five million tokens in a week. Silent unless it matters. That’s the whole design philosophy of these automations, coverage over noise.

The email to brain pipeline. Every hour, sent emails get captured into the knowledge graph. I mentioned this yesterday but it deserves repeating because it’s the most set-and-forget thing I built. It just runs.

The morning briefing. At 7am I get a briefing: what happened overnight, what needs a decision, what’s flagged. It reads the knowledge base, checks the monitors, looks at the kanban board, and gives me three sections: needs your decision, top priorities, recently shipped. It takes about 90 seconds to read and saves me an hour of context-switching.

The health checks. Overnight it checks the backup ran, the websites are up, the containers are healthy, the disk isn’t filling. If something’s broken it tells me in the morning instead of me finding out the hard way. There’s also an Uptime Kuma monitor doing the same job from the outside, because one monitor is an accident and two is a system.

The rule I stuck to the whole month: an automation that misses results is worse than one that runs slow. So everything is built for coverage first. If the deal cron takes an extra two minutes verifying prices, fine. Missing the deal because I optimised for speed would be a fail.

If you’ve ever told a chatbot something and then had it forget two hours later, you know the pain I was trying to solve. My assistant doesn’t forget. Well, it does, but it has a filing system.

Under the hood there’s something called a knowledge graph. Every conversation I have with it, every email that comes in or goes out, every interesting article it scans, gets captured and linked. People, companies, projects, decisions, dates, all connected. When I ask who I met last month or what we decided about the VPS, it doesn’t guess. It goes and looks.

The clever part is the dream cycle. Once a day the graph consolidates raw capture into proper facts, finds contradictions, and prunes the noise. It’s like a nightly filing run, except the filing clerk is also an analyst.

There’s a human-readable side too: everything also lands in my Obsidian vault on the NAS, so I can browse the same knowledge the assistant has. It’s genuinely weird opening a page in Obsidian and finding a summary of a conversation I’d completely forgotten having.

The email integration is the part that quietly became essential. Every sent email gets scanned and filed against the people involved, so the graph knows who I’ve talked to, about what, and when. My old self would have had to remember that. My new self asks.

Real talk: setting this up was the fiddliest week of the project. Embeddings, providers, sync jobs, a hundred small decisions about what deserves remembering. But it’s the difference between an assistant and a search bar. An assistant that remembers is a colleague. A search bar is just a search bar.

Everyone asks what I’m actually running, so here’s the honest wiring diagram, minus the boring parts.

The brain. The assistant runs on a framework called Hermes Agent, living on my main Windows 11 PC. It’s not a website and it’s not a box in the cloud. It’s an agent that sits on my desktop, connected to the platforms I already use: Telegram (my main chat), iMessage through a bridge, email, SMS, and Home Assistant.

The models. Heavy reasoning goes to cloud models. Anything cost-sensitive or private gets routed to local models running on my RTX 3090 via LM Studio. There’s a routing layer so the assistant picks the right model for the job instead of burning the expensive one on everything. The biggest surprise was how cheap this can be when you route properly.

The reach. This is the part commercial assistants can’t do. From the desktop it talks to my Synology NAS (media stack, backups) and my VPS in the cloud (websites, n8n, databases), all over SSH and Tailscale. It has Docker access on both hosts. It can deploy containers, restart services, check logs, run database queries. When something breaks at 2am, it’s already looking at the logs before I’m awake enough to ask.

The memory. I’ll do a whole post on this, but the short version is: every conversation, email and signal gets captured and organised into a knowledge graph I can search. The assistant remembers. That’s the feature I didn’t know I needed most.

The rules. We settled on a simple operating doctrine after a few early disasters: Python for scheduled jobs, PowerShell for Windows admin stuff, and never delete files without asking. The last one is non-negotiable. It’s the assistant’s equivalent of looking both ways before crossing.

That’s the skeleton. Tomorrow I’ll talk about the part that makes it feel like magic: the memory.

About a month ago I sat down and made a decision I’d been circling for a while: I was going to stop waiting for someone else to build the AI assistant I actually wanted, and build it myself.

I’ve spent the last 15 years in enterprise networking and infrastructure, and more recently I’ve been running Maintain AI, automating workflows for other people. But here’s the thing nobody tells you about doing automation for a living: your own life stays stubbornly manual. My to-do list lived in four different apps. My calendar didn’t talk to my NAS. My media server didn’t know what I wanted to watch. My inbox was a swamp.

The commercial assistants were getting good, don’t get me wrong. But they’re black boxes. My data goes to their servers, they can’t reach into my homelab, they can’t run a script at 5:30pm to check tonight’s takeaway deals, and they certainly can’t SSH into my Synology. I wanted something that lived in my infrastructure, not in someone else’s cloud.

So I built one. This series is the story of that first month: what it is, how it’s wired, what broke, and what genuinely surprised me.

A quick note on how this post (and the rest of this series) exists: it was drafted by my assistant, Hermes, from my notes and the actual logs of what we built, then published after I reviewed it. If you ever see the tag hermes-wrote-this on a post here, that’s what that means. I figure if I’m building an AI assistant, the least I can do is be upfront about when it’s doing the typing.

The short version of the last month: I now have something that reads me a briefing every morning, remembers conversations I forgot I had, texts me deals before I’m hungry, tells me off (politely) when my AI spend spikes, and controls half the house. It’s not perfect. It’s occasionally infuriating. And I wouldn’t go back.

Come for the build logs, stay for the part where the assistant starts correcting my spelling.