Until recently I could talk to my agent from exactly two places: the GX10’s terminal, or VS Code sitting in front of one of the two machines. An agent that only answers when you’re at your desk is an agent with office hours.

So I built gx-telegram: a Python gateway - a single process, with SQLite as its only heavyweight dependency - that connects a Telegram bot (@inferont) to the Hermes agents running on the GX10 and the SCAR16, and that has meanwhile become the log of everything I do with the agent. Including this blog: the Monday-morning drafts start there.

The architecture in thirty seconds

                        ┌────────────────────────── GX10 ────────────────────────────┐
 Telegram ◄──────────►  │  telegram-gateway (one process)                            │
  (long polling)        │  ├── Telegram poller ──► router ──► TaskManager ──► Hermes │
                        │  ├── HTTP API :30100 ◄────────────── agent REST            │
                        │  └── SQLite (tasks, agents, conversations, notifications)  │
                        └──────────────────────────────┬─────────────────────────────┘
                                                       │ LAN HTTP + shared token
                                                       ▼
                                     SCAR16: telegram-gateway agent --agent-id scar16
                                     (registers, heartbeats, polls tasks, local Hermes)

The design choice that holds everything up: the gateway doesn’t know which machines exist. No hostname is hardcoded anywhere. Every machine that wants to run tasks introduces itself - it registers with hostname, OS, model and capabilities, then heartbeats every 30 seconds. Silent for 90? It’s offline. Tomorrow I add a NAS with --agent-id nas-01 and the gateway doesn’t change a line.

No webhooks, not even for Telegram: long polling. The GX10 sits behind a home NAT, with no public TLS and no open ports - polling works everywhere and the Bot API supports it natively.

Hermes: what the agent actually runs

The gateway implements no agent at all: it simply shells out to the Hermes CLI in one-shot mode, hermes -z "<prompt>" --yolo, and reads stdout. No modifications to Hermes sources or configuration - the only requirement is that the binary exists and is already configured on the executing machine. The gateway is an orchestrator, not a brain - and that separation has saved me from reinventing the whole agent layer every time the model changes.

The lifecycle of a task

A Telegram message is not an HTTP request: a task can take twenty minutes. So no synchronous calls - instead a queue with an explicit state machine:

queued → running → completed | failed | cancelled

On the GX10 the task runs immediately as a subprocess, with a timeout and a concurrency limit. For remote agents, the runner on the SCAR16 polls GET /agent/tasks, executes locally and reports back. Cancellation propagates in both directions: /cancel 42 shows up in the next heartbeat response, and the runner kills the local Hermes process.

The Telegram interface acts as the UI: the ⏳ Task running… message is edited in place (editMessageText) until it becomes ✅ Done or ❌ Failed. No spam of intermediate messages.

Check GPU temperature          → agent gx10 (default)
/scar16 summarize the logs     → queued for the SCAR16
/status                        → gateway state + map of online agents
/tasks                         → recent tasks
/cancel [id]                   → cancel queued or running task

Notifications: not everything deserves to ring

A chatty agent is worse than a mute one. The notification system crosses two dimensions:

  • type - success | info | warning | error | progress | task_started | task_completed | task_failed | scheduled_task | startup | shutdown
  • priority - low | normal | high | critical

Plus duplicate suppression (identical notifications within a few seconds arrive once) and an ignore mode for low priority: redundant progress stays in the logs, not on the phone. That’s the difference between a tool you keep installed and one you mute after two days.

The VS Code proxy: a mirror for chats

The last piece built and the one that ended up the most useful. Between VS Code and llama-server (Qwen on port 30000) sits a transparent OpenAI proxy on port 30001: Copilot Chat traffic passes through it, and every turn

  • my message and the model’s reply - is pushed to Telegram and appended to a JSONL feed.
Strix Scar 16 ─┐                                  ┌─► llama-server :30000
               ├─POST /v1/chat/completions─► proxy :30001
GX10 (local) ──┘        (LAN bind)                ├─ notify ─► gateway API ─► Telegram
                                                  └─ append ─► activity.jsonl

Three details worth more than the idea itself:

  1. Fail-open. If the gateway is down or Telegram is slow, the VS Code chat keeps working anyway: mirror errors only go to the logs. An observer must never be able to break the observed.
  2. Agent mode handled. In agent mode the model emits dozens of intermediate replies with tool calls: those stay silent, only the final answer reaches Telegram. And internal service calls (title generation with tiny max_tokens) are discarded wholesale.
  3. Machine identification was a surprise. The plan was to tell GX10 and SCAR16 apart through the apiKey VS Code declares - except the Copilot custom client does not forward the key: it sends an empty Authorization: Bearer. Fell back to the source IP: the proxy binds the LAN and an ip → machine mapping resolves identity. It works as long as DHCP is good, of course - a trade-off I knew I was making, written in black and white in the README.

The feed that writes the posts

The activity.jsonl didn’t end up in a corner: it powers the automatic Monday drafts. A systemd timer (blog-draft.timer) reads the week’s feed, compacts it by day and project, feeds it to local Qwen with reasoning disabled, and drops a draft into this blog’s content/posts/. The post you are reading, too, was born from material that passed through that feed: the loop closes.

flowchart LR
    V[VS Code ×2] -->|chat| P[proxy :30001]
    P -->|every turn| A[(activity.jsonl)]
    A -->|Mon 09:30 timer| G[generate-draft.py]
    G -->|local Qwen| B[(blog draft)]
    B -->|draft: false| S[(Hugo site)]

Security: a yard, not a fortress

Everything runs on the home LAN, but “home” doesn’t mean “unguarded”:

  • Strict allowlist: explicit Telegram user IDs and chat IDs - the gateway refuses to start without a token and an allowlist. Anyone not in the list gets logged and ignored.
  • Rate limiting per user on the Telegram ingress.
  • Per-agent shared tokens (X-Agent-Token) on the REST endpoints; admin endpoints are localhost-only unless an admin token is configured - the secure default coincides with the lazy default.
  • Secret redaction: every error string passing from the gateway to logs or Telegram is scrubbed of known secrets. A traceback that lands on your phone must not carry the API key with it.
  • SQLite in WAL mode: tasks and conversations survive reboots, and tasks stuck running after a crash are marked failed at startup.

What I’d repeat (and what I wouldn’t)

The choices that paid off: the generic agent registry (zero friction to add machines), long polling (no TLS, no DDNS, no ports), and the proxy’s fail-open behavior. The single-process Python with SQLite handles - without batting an eye - a load that is, realistically, “one person and their agents”.

What cost me time was Telegram itself: the 409 Conflict when a second poller is active (the gateway now deletes any webhook at startup, but I fell for it anyway), and the 4096 characters per message that forced truncating long replies. And the apiKey vanishing into an empty Bearer remains the lesson of the day: never trust what a client declares - verify what it delivers.

The natural next step: reaching the gateway from outside the house via Tailscale, so my phone talks to my agents while traveling - without exposing anything to the public network. But that, eventually, is another draft.