
The loose end I leave most often isn’t a bug. It’s “check this next week.”
I ship a fix, and the real test is next week’s numbers. I send a partner a question, and I need to chase it on Friday if nothing comes back. A coding agent can do the work today, but in my setup the session didn’t leave a durable follow-up behind, and I’ve learned not to assume the next session has this one’s context.
My calendar notes didn’t fix that either. They held a title, “check the fix”, not a ready-to-run check, and a week later I had to rebuild everything else: what exactly to look at, why it mattered, and what a normal result would look like. By then the context was gone, so the check got skipped, or done badly.
This is part 7 of a series on what I learned putting AI agents to work in my own job. It’s about the smallest agent I run, and the design choice that made it useful.
What a follow-up actually needs
When I finish a task, I know four things that I won’t know next week:
- What to check. The query, the log, the page.
- Why. What the change was supposed to do.
- What normal looks like. The number or result that means “it worked”.
- The request to run. The exact words to give an agent so it can do the check without me explaining it again.
The fourth is the one my calendar notes never had, and it’s the one that saves the most time. So the question became: where do I put a follow-up that carries all four, and who reminds me?
The long way round
I went through three designs before one stuck.
First, a plain reminder from my personal assistant. Hermes is my personal assistant, a profile on the open-source Hermes Agent runtime that I talk to in a Slack DM. It reminded me on the day, but it can’t see any company data, so I still had to do the whole check myself. I dropped it.
Second, a scheduled job on the team agent. kenny, the agent that answers my team’s questions, can read production data through a read-only view. So I had my coding agent register a one-off scheduled job on kenny: on the day, kenny would run the check itself and post the result. It worked: a test run queried the data once and reported back.
It was also heavy. Each follow-up meant logging in to the agent host, registering the job, test-running a copy, re-baselining the integrity check that pins every scheduled job, committing, and syncing. Twice, two of my coding-agent sessions did it at once: once registering the same job forty seconds apart, once committing new baselines at the same time and having to be untangled. And I couldn’t close or postpone a follow-up from Slack; that needed another round on the host.
Third, back to the assistant, on purpose. The obvious fix was to let kenny create its own jobs from a Slack conversation. I decided against it. kenny touches production data, and its guardrails deliberately stop it from changing its own configuration through chat. Giving an agent like that the power to create persistent jobs from a conversation is exactly the kind of widening I don’t want. So I gave up the version that could run the check by itself, and took the smaller blast radius instead: the assistant only reminds, and I run the check.
How it works now
At the end of a task, my coding agent posts one message to Hermes in Slack. It always has the same fields: a title, when to remind me first, how (once, or until I answer), and then the important part: what to check and why, what normal looks like, and a request I can paste straight back into a coding agent.
Hermes turns that into a reminder. By default it reminds me twice a day from the chosen date, up to seven times over four days, stopping sooner when I answer. When it fires, I paste the request into a local agent, which has the access Hermes doesn’t, and judge the result. Then I reply in the thread with one word (in my setup the replies are Korean words; these are their translations):
- done (완료): it’s finished, and the reminder is removed.
- again (다시) with a time: remind me later, in the same mode as before.
- cancel (취소): stop all reminders for it.
That’s the whole interface. Registering, finishing and postponing all happen in Slack. Nothing changes on the team agent, and nothing gets committed.
The message itself is also the record. Weeks later, the registration post says why I wanted to check, which is half of what part 5 is about.
Moving over, and the bug the test caught
I wrote the new rules on 8 November and moved everything the same day: four pending follow-ups and one daily reminder came off the team agent and became Hermes reminders.
Testing the reply paths that day caught a bug in “again”. Replying “again” did recreate the reminder, but in the wrong mode: an “until I answer” reminder came back as a one-off, which would have fired once and then gone quiet. Hermes now has an explicit rule to keep the original mode. The lesson I keep relearning: the creation path is the one everyone tests; the reply paths are where reminders quietly die.
Speed was never the problem. When I first connected a coding agent to Hermes over Slack, the round trip from posting to a reply in the thread was 4.4 seconds. Registering a follow-up is one message my coding agent writes for me.
The other half: an advisor with a role, not a history
Hermes does one more job: I ask it for advice on decisions. One recommendation, the reason, and the cost if it’s wrong. The decision stays mine.
I wanted it to reason like an experienced founder. The tempting way is to write that history into its instructions: “you’ve run three companies”. I didn’t, because a model given a fake history will produce fake anecdotes, “at my second company we…”, and those read like evidence. When I’m the one making a real decision from the advice, invented evidence is the most dangerous thing it can produce.
So its instructions describe a perspective, not a past: what it looks at first, what it weighs heavily, what it’s suspicious of. It speaks in patterns and judgments, never in made-up stories, and it never hides that it’s an AI. The advice is a little less impressive. It’s also something I can actually check.
How I use it now
Two follow-ups I registered on 15 and 16 November, both written by my coding agent at the end of the task:
- A reminder to check, a week after this series started, how the first post did on each channel, with the request that pulls the numbers together and records them.
- One reminder a day while this series is publishing: check that the day’s post went live, then prepare the cross-posts. The request even says what to do if the post didn’t appear.
Neither needed me to remember anything except to read a Slack message.
What it costs
- No automatic check. In the second design, the agent looked at the data itself. Now I do it, with a local agent, when the reminder fires. That’s real work I took back on purpose.
- It depends on me answering. Seven reminders over four days is persistent, not infinite.
- A reminder is only as good as the request written at the time. If the request is vague, the check will be too. The fields force me to write what normal looks like while I still know it.
What I took from it
- Write the check while you still have the context. Next week’s you won’t.
- Make the reminder carry the request, not just the title. The paste-ready request lets me start the check without rebuilding the context.
- Keep the reminder away from company systems. An assistant that has no access to them and stores only the reminder and the check instructions can’t do much damage, even if its own instructions are wrong. The message itself still follows the same rules as anything I post.
- Make closing it one word. If finishing or postponing takes effort, reminders pile up and you start ignoring them.
- Test the reply paths, not just creation.
- Give an advisor a role, not a history, if you want advice you can check.
The series
- From doing the work to delegating it
- Let code decide, let the LLM explain
- The rules I never break
- Context is the job
- Agents forget, so I write it down
- kenny: an agent that does my job
- Hermes: reminders that carry the check (this post)
- How to start in two weeks
What’s the follow-up you most often forget, and what would the reminder need to say for you to do it in five minutes?