Last post of the series, and the one people will actually ask about: can you do this, and should you?

Short answer: yes, and probably, but manage your expectations.

What it costs. Time, mostly. I reused subscriptions I already had (chat model access, an existing VPS, a NAS I’d had for years). The only real spending was a few API keys and a bit of compute. If you’re starting from zero you can run a surprising amount of this on a $20 a month VPS plus whatever you already pay for a chat model. The expensive part is the hours, not the dollars.

What you need. A genuine itch. Not “AI is cool”, but a specific thing you’re sick of doing by hand. Mine was dinner decisions and context switching. Yours might be email triage, expense tracking, or remembering people’s names. Start there. The assistant that survives is the one that does one job you actually care about, and grows from that.

What you don’t need. A homelab. I have one because I already had one, but the same stack runs fine with nothing but a VPS and a laptop. What you do need is a tolerance for debugging, because the first month is mostly that, and the willingness to say the assistant wrote this when it did.

What surprised me most. That the ceiling isn’t the technology. It’s the trust. The moment I started letting it remember things and run things without me watching, everything changed. That’s a leap, and it’s a personal one. I can’t tell you when to make it. I can tell you it was worth it.

That’s the month. If you’ve read this far, you now know more about my assistant than most of my clients do. Ask me anything, or better yet, go build the one you’ve been thinking about. If you do, I’d genuinely love to hear how it goes.

Tags on this series: build-log, hermes-wrote-this. Every post in it was drafted by my assistant from my notes and logs, and reviewed by me before publishing. That’s the deal, and it’s a good one.

Nobody wants to read about boring infrastructure, so I’ll keep this short, but it’s the reason any of this works and it’s the bit people always skip.

My setup, in one breath: a Synology DS218+ NAS at home running the media stack, Home Assistant, and a reverse proxy (SWAG) behind Cloudflare. A Hostinger VPS in the cloud running Coolify, n8n, and the Mission Control stack. Tailscale stitching it all together into one private network. Docker everywhere, because Docker is how you stop fighting your own servers.

Three rules I kept the whole month:

No new firewall ports. Anything new gets exposed through the reverse proxy or over Tailscale. My perimeter is smaller now than when I started, which is the opposite of how these projects usually go.

Encrypted backups of everything that’s hard to rebuild. The assistant’s config, my Google tokens, the skills library. If the PC dies, the recovery path is documented and tested, not improvised.

Monitoring from both sides. Uptime Kuma watches from the internet, the assistant watches from inside. When a website went down last month, I knew before anyone else did, because two different systems both noticed and one of them texted me.

The boring truth about self-hosting: it’s not cheaper, it’s not easier, and it will occasionally ruin your evening. What it gives you is ownership. My assistant’s memory lives on my hardware. My automations don’t vanish when a startup pivots. My data is mine, my failure modes are mine, and my fixes are mine too.

I’d rather own my problems than rent someone else’s.

A month is long enough for the honeymoon to end and the real relationship to start. Here’s the honest report.

What broke. Plenty, and that was the point. A scheduled script had a timezone bug that was quietly producing wrong dates for months before we caught it (a fixed +12 offset forgot daylight saving, which is peak irony for a country that changes its clocks twice a year). A WordPress admin lockout ate a whole evening and taught me more about my own backup strategy than I wanted to know. A bridge between my desktop and the VPS refused to connect because a database driver was stricter about SSL than the last version. Every break was a lesson, but the stat that matters is that I spent roughly a third of the month fixing things and two thirds watching things work.

What stuck. The deal cron (never missed a night, and it’s bought my dinner twice). The spend watchdog (it caught a runaway job at 2am before it cost me real money). The morning briefing (I genuinely miss it when it doesn’t run). The knowledge graph (the more it knows, the more I rely on it). The things that stuck are the ones that removed a decision I used to make every day.

What surprised me. Three things. One: how much better a remembering assistant is than a clever one. Give me a model that remembers what I said last week over a model that’s 10% smarter, every time. Two: how quickly it became normal. The future is just a text message that knows your history. Three: how much of it is plumbing. The assistant is 90% infrastructure and 10% intelligence, and that’s not a bug, that’s the job. The magic is in the plumbing.

Would I do it again? Yes, and I’d start the same way: one automation that matters to you personally, then let it grow. More on that in the last post of this series.

Typing to an assistant is fine, but I’m a voice memo person, so I wanted the assistant to meet me halfway.

The main chat is Telegram, which I’m in all day anyway. From there it can do everything: run automations, answer from its knowledge graph, send me the deal list, whatever. But there are layers on top now.

Voice memos. I can dictate a note and it gets transcribed locally with faster-whisper and filed into the knowledge graph. No cloud, no third party hearing my half-formed thoughts. The transcription runs on my own GPU, which feels like the right trade: private stuff stays home.

Phone calls. There’s a voice agent (built on Vapi) so the assistant can actually handle a call. Still early days, but it’s wild hearing it hold a conversation. The plan is for it to take the calls I don’t want and brief me on what it sorted.

iMessage and SMS. Both bridged in, because half my family and most of my clients live on the other side of my chat apps. The iMessage bridge is the fiddliest piece of the whole setup and I’m not going to pretend otherwise, but it means the assistant can text me (and I can text it) from anywhere.

The morning briefing arrives as a voice-friendly summary too, which is the closest thing I’ve built to being read the paper.

Honest verdict on the voice layer: the transcription is 90% of the value, and it’s the part nobody sees. Talking to an assistant is a party trick. Being able to think out loud and have it actually captured, organised and remembered, that’s the real thing.

About three weeks in, I hit the point where the assistant had so many moving parts that I wanted to see it all in one place. So I built Mission Control.

It’s a dashboard running on my VPS, reachable at mission-control.vaughantaylor.com. Every important event in the system, from both my desktop and my servers, streams into a database over Tailscale, and the dashboard renders it as a live board. Deployments, bridge connections, cron runs, status changes. It’s my digital life as an event feed.

Signing in is with Google, because I refuse to maintain another password. The database sits on the VPS behind Tailscale so it’s not exposed to the public internet, and the whole thing routes through Traefik so I didn’t have to open a single new firewall port. That last bit was a deliberate design rule for the whole project: no new holes in the perimeter. Everything comes in through the existing front door or over the encrypted tailnet.

The dashboard itself is honestly a vanity project. The assistant doesn’t need it to function. But there’s something genuinely useful about watching your infrastructure talk to itself in real time, and it’s become my go-to is-everything-okay screen before bed.

Also, it gave the whole project a name. The dashboard is called Hermy HQ. My assistant has a home now.

One of the best side effects of building this thing: the house stopped being a collection of apps and started being one system.

The heart of it is my Synology DS218+, which I’ve had for years and have been slowly turning into a proper media hub. Jellyfin for streaming, the full arr stack (Sonarr, Radarr, Lidarr, Bazarr) managing the library, Deluge for downloads, Pi-hole filtering DNS, Home Assistant tying the smart bits together.

What changed this month is that the assistant got access to all of it. Now I can ask what’s new on the server and get an answer instead of opening three apps. It can queue a stream to the Apple TV via pyatv, start something on Jellyfin, or tell me when the next episode of something lands. The lights, the Sonos, the Apple TV, the NAS: it’s all just things it can reach now.

The practical win: I used to run a media night like it was a small IT project. Now it’s one message.

The philosophical win: this is the part of self-hosting that’s hard to explain to people who’ve never done it. When everything lives on your own hardware, adding a new capability is a configuration change, not a subscription. My assistant doesn’t need permission to talk to my NAS. I built the permission. That’s the whole point.

Here’s what the assistant actually does while I’m not looking.

The 5:30pm takeaway deal. Every night it checks the Uber Eats offers hub for Hamilton, verifies the actual prices, and texts me the top picks under $20 after the discount. It has rules: max three food places, only deals that are actually cheaper, and Fusion Kebab’s two-for-one is always in the running because it’s been reliable for months. Last week it found a 50% off pizza for $11.50 that was $23 the day before. I don’t browse for dinner anymore. I get briefed.

The spend watchdog. I burn through AI tokens like a teenager with a credit card, so there’s a watchdog that watches my actual usage and only speaks up when something’s wrong: over $3 in a day, or five million tokens in a week. Silent unless it matters. That’s the whole design philosophy of these automations, coverage over noise.

The email to brain pipeline. Every hour, sent emails get captured into the knowledge graph. I mentioned this yesterday but it deserves repeating because it’s the most set-and-forget thing I built. It just runs.

The morning briefing. At 7am I get a briefing: what happened overnight, what needs a decision, what’s flagged. It reads the knowledge base, checks the monitors, looks at the kanban board, and gives me three sections: needs your decision, top priorities, recently shipped. It takes about 90 seconds to read and saves me an hour of context-switching.

The health checks. Overnight it checks the backup ran, the websites are up, the containers are healthy, the disk isn’t filling. If something’s broken it tells me in the morning instead of me finding out the hard way. There’s also an Uptime Kuma monitor doing the same job from the outside, because one monitor is an accident and two is a system.

The rule I stuck to the whole month: an automation that misses results is worse than one that runs slow. So everything is built for coverage first. If the deal cron takes an extra two minutes verifying prices, fine. Missing the deal because I optimised for speed would be a fail.

If you’ve ever told a chatbot something and then had it forget two hours later, you know the pain I was trying to solve. My assistant doesn’t forget. Well, it does, but it has a filing system.

Under the hood there’s something called a knowledge graph. Every conversation I have with it, every email that comes in or goes out, every interesting article it scans, gets captured and linked. People, companies, projects, decisions, dates, all connected. When I ask who I met last month or what we decided about the VPS, it doesn’t guess. It goes and looks.

The clever part is the dream cycle. Once a day the graph consolidates raw capture into proper facts, finds contradictions, and prunes the noise. It’s like a nightly filing run, except the filing clerk is also an analyst.

There’s a human-readable side too: everything also lands in my Obsidian vault on the NAS, so I can browse the same knowledge the assistant has. It’s genuinely weird opening a page in Obsidian and finding a summary of a conversation I’d completely forgotten having.

The email integration is the part that quietly became essential. Every sent email gets scanned and filed against the people involved, so the graph knows who I’ve talked to, about what, and when. My old self would have had to remember that. My new self asks.

Real talk: setting this up was the fiddliest week of the project. Embeddings, providers, sync jobs, a hundred small decisions about what deserves remembering. But it’s the difference between an assistant and a search bar. An assistant that remembers is a colleague. A search bar is just a search bar.

Everyone asks what I’m actually running, so here’s the honest wiring diagram, minus the boring parts.

The brain. The assistant runs on a framework called Hermes Agent, living on my main Windows 11 PC. It’s not a website and it’s not a box in the cloud. It’s an agent that sits on my desktop, connected to the platforms I already use: Telegram (my main chat), iMessage through a bridge, email, SMS, and Home Assistant.

The models. Heavy reasoning goes to cloud models. Anything cost-sensitive or private gets routed to local models running on my RTX 3090 via LM Studio. There’s a routing layer so the assistant picks the right model for the job instead of burning the expensive one on everything. The biggest surprise was how cheap this can be when you route properly.

The reach. This is the part commercial assistants can’t do. From the desktop it talks to my Synology NAS (media stack, backups) and my VPS in the cloud (websites, n8n, databases), all over SSH and Tailscale. It has Docker access on both hosts. It can deploy containers, restart services, check logs, run database queries. When something breaks at 2am, it’s already looking at the logs before I’m awake enough to ask.

The memory. I’ll do a whole post on this, but the short version is: every conversation, email and signal gets captured and organised into a knowledge graph I can search. The assistant remembers. That’s the feature I didn’t know I needed most.

The rules. We settled on a simple operating doctrine after a few early disasters: Python for scheduled jobs, PowerShell for Windows admin stuff, and never delete files without asking. The last one is non-negotiable. It’s the assistant’s equivalent of looking both ways before crossing.

That’s the skeleton. Tomorrow I’ll talk about the part that makes it feel like magic: the memory.

About a month ago I sat down and made a decision I’d been circling for a while: I was going to stop waiting for someone else to build the AI assistant I actually wanted, and build it myself.

I’ve spent the last 15 years in enterprise networking and infrastructure, and more recently I’ve been running Maintain AI, automating workflows for other people. But here’s the thing nobody tells you about doing automation for a living: your own life stays stubbornly manual. My to-do list lived in four different apps. My calendar didn’t talk to my NAS. My media server didn’t know what I wanted to watch. My inbox was a swamp.

The commercial assistants were getting good, don’t get me wrong. But they’re black boxes. My data goes to their servers, they can’t reach into my homelab, they can’t run a script at 5:30pm to check tonight’s takeaway deals, and they certainly can’t SSH into my Synology. I wanted something that lived in my infrastructure, not in someone else’s cloud.

So I built one. This series is the story of that first month: what it is, how it’s wired, what broke, and what genuinely surprised me.

A quick note on how this post (and the rest of this series) exists: it was drafted by my assistant, Hermes, from my notes and the actual logs of what we built, then published after I reviewed it. If you ever see the tag hermes-wrote-this on a post here, that’s what that means. I figure if I’m building an AI assistant, the least I can do is be upfront about when it’s doing the typing.

The short version of the last month: I now have something that reads me a briefing every morning, remembers conversations I forgot I had, texts me deals before I’m hungry, tells me off (politely) when my AI spend spikes, and controls half the house. It’s not perfect. It’s occasionally infuriating. And I wouldn’t go back.

Come for the build logs, stay for the part where the assistant starts correcting my spelling.