I’d been looking at several LLM powered “agent” frameworks recently. OpenClaw got a lot of buzz several months ago, spiked the cost of Mac Mini’s, and it seems reality doesn’t quite live up to the hype. Peter Steinberger’s description of his OpenClaw1 creation in multiple videos, podcasts, and posts was he would wake up each day to discovery the agent had been working tirelessly throughout the night on projects, building things, discover things, bring him new opportunities, and learning and growing as it went. Reddit and Youtube exploded with people snatching up Mac Mini’s for their high unified RAM / small form factor / modest cost, implementing their own OpenClaw’s, and telling everyone about their successes breathlessly.
I think my favorite “review” was this:
In the span of a few weeks ClawdBot went through several name changes, Anthropic blocked it from consumer level plans, Steinberger got a job with OpenAI, while people started to post their runaway token costs and concerns the agent was escaping containment, grabbing personal information, publishing personal information, and just generally running amok.
I have never tried OpenClaw because it seemed like an AI vibe coded slop privacy / security nightmare with a thick frosting of marketing slathered all over. I understand the appeal of OpenClaw. It would be so nice to have a little digital butler, looking at my cluttered desktop, helping tidy up, organize me, make some connections and surprising me in the morning with some new developments. After all, where would Tony Stark be without Pepper Potts and Jarvis to keep him on the rails?2 I also understand how some semi-technical people would be dazzled by the promise of something that would work around the clock to make them money, all for the cost of a Mac Mini.
A short interlude about Chik-Fil-A. I don’t like their politics and will not support them. I’ve never once given them a single penny. But, I have eaten food from Chik-Fil-A… When a new located opened not far from my former workplace, they came by and left a HUGE platter of delicious chicken nuggets and sauces. They have historically donated to our local Maker Faire, providing lunch for the makers. Hide their branding and of course their food is going to be delicious – that’s the entire point of fast food. The first year they provided lunch, I didn’t eat it – because I don’t like Chik-Fil-A. The second year… I was weaker – but also more practical? I was hungry, the kids were hungry, and although I had no problem paying for food at one of the many vendors, we’d been there a while, it was hot, the lines were long, and well, the food was right there. I suppose the most subversive thing to do would be to take their food and toss it in the trash – or maybe shuttle their food to the kinds of people they despise. But, I’m not in the habit of wasting or delivering food. My own subversive little act was to eat the food – and resolve to never, ever, advertise for them or give them any money.
Back to LLM’s. I have tried Hermes Agent, running in a docker container in a machine with very little personal information about me. I’ve only been using it for a few days – and I think it has some real promise with some real caveats.
- The first caveat is that I won’t use any frontier LLM services because I can’t justify giving OpenAI/Anthropic/Google/Meta money for such things. Whatever Google once was, whatever their motto might have been, they’re so far removed from that as to be a different animal entirely. 3 The obvious downside is that without access to a very fast, very big, and very powerful LLM, my Hermes Agent will absolutely be less useful than someone else’s instance.
- It will run in a Docker container on an isolated, budget machine, or not at all.
In practical terms, this means I’ve limited myself to either smaller and less capable local LLM’s that my machine can run or free tier API’s for various LLM providers. I’m still fumbling my way around Linux / Ubuntu and already using LLM’s to get my bearings, so these resources for providing the LLM reasoning power for Hermes feels within bounds too.
As always, I simply don’t trust any LLM’s outputs. I don’t trust their purported facts, claims, summaries, etc. Even so, there’s no arguing they are capable of building systems. Some family recently “fact checked” me using ChatGPT and I pointed out its merely a machine designed to provide them with the very next token of what they want to hear – not a system for giving them facts. I feel like it’s probably impossible to build an AI / AGI without training it on facts, but relying on a token generator for facts seems foolhardy. That said, I don’t have a problem relying on a system / set of processes / code base that I can inspect and verify works to arrive at a solution.
These “agent” systems are usually better called “agent harnesses,” since their best feature is offloading the work of monitoring, guiding, and nudging an LLM to perform a task. The Hermes agent adds a lot of extra features – keeping things organized, keeping certain chats available to revisit later on, switching LLM’s between local / API based, but at it’s core it is saving me the time of just babysitting an LLM.
As understand these agents, the first thing they do is take a query, ask the LLM to break it into pieces, and then get to work on discrete elements. It used to be that on long tasks ChatGPT would keep popping up a “Continue” button. About two years ago I wrote a simple Chrome extension that was active on ChatGPT and would simply auto-click the “Continue” button whenever it appeared. I think this is less a thing these days and ChatGPT will either truncate a reply or simply stop. But, keeping the LLM on track for each small sub-task and spurring it to complete appears to be part of what the agent is doing under the hood.
I’ve tried several LLM’s to assist with writing code, with varying degrees of success. The best OpenRouter free model I’ve used recently has been Poolside’s Laguna S2.1 with 118B parameters, a “mixture of experts” model with only 8B active at any given time. I’ve been very impressed with it’s coherence on long tasks and overall quality of work – and it has been extremely helpful in setting up Hermes. I haven’t tried a recent Qwen or Deepseek model on my very much underpowered system, but I plan on doing so soon.
My first goal with this system is to get it to just run a bunch of python scrips on a cron job. I may need to build up a separate system entirely that only runs cron jobs, but which can have it’s roster of tasks manipulated by Hermes. Too often the LLM gets involved and wants to interpret a query rather than run a static script, which just leads to more hassles. Perhaps a frontier LLM wouldn’t have this problem – but I don’t plan on using them.
Project Jarvis- Building a Jarvis-inspired voice activated LLM powered virtual assistant
- Building a Home PC Lab
- Secret Agent Man
- Well, Clawdbot -> Moltbot -> OpenClaw [↩]
- NGL, I really hated Gwenth Paltrow’s Pepper Potts and would have liked nearly anyone else in that role. Maybe it was the direction – or maybe just my dislike for Paltrow – but I found her bumbling, flustered character exhausting. The comic book Potts was a capable, intelligent person in their own right. Maybe there’s hope for a better person in a future reboot? Hell, if they can recast Roddy, they can recast Potts. [↩]
- I won’t even bore you with the full story of how Google promised a remediation for a Pixel 6 battery problem, bait and switched, then slowplayed it until everything expired. [↩]