Jarvis (Hermes Agent)¶

Jarvis is the homelab's autonomous AI assistant — the Nous Research Hermes Agent running on its own dedicated VM so it can be isolated (or killed) instantly.

Why a dedicated VM¶

The agent runs shell commands as part of its job, so the VM boundary is the sandbox — not a container. Containerizing an infra-ops agent forces a docker-socket mount or SSH-back-to-host, which is host-root with extra steps.

ssh root@192.168.1.199 "qm stop 121"    # pull the plug — severs everything
ssh root@192.168.1.199 "qm start 121"   # self-resumes via systemd

VS Code ACP bridge¶

Hermes exposes the Agent Client Protocol (ACP) natively through hermes acp. Its transport is stdio JSON-RPC only: there is no socket, listening service or open port. The VS Code client starts that process through SSH, so Hermes runs on VM 121 and JSON-RPC rides the existing SSH channel. The earlier design for an MCP service on the VM was rejected in favour of this path because ACP adds no network attack surface to the VM.

The client configuration lives on the operator's laptop, not in this repo. It uses Jun Han's formulahendry.acp-client extension ("ACP Client for VS Code") and this SSH alias:

Host jarvis-vm
    HostName 192.168.1.252
    User jarvis
    IdentityFile ~/.ssh/claude-code-lan
    IdentitiesOnly yes
    RequestTTY no
    ServerAliveInterval 30
    ServerAliveCountMax 6

The laptop's VS Code settings.json has a separate acp.agents entry with these exact values:

"acp.agents": {
    "Hermes (Jarvis VM)": {
        "command": "ssh",
        "args": ["-T", "jarvis-vm", "hermes", "acp"],
        "env": {}
    }
}

RequestTTY no and -T are load-bearing. A PTY can inject terminal control bytes into stdout and corrupt the JSON-RPC framing. The extension's shipped Hermes Agent entry was deliberately left in place beside the VM entry; it invokes a bare local hermes and cannot start anything because Hermes is not installed on the laptop. Do not repurpose that default entry as the VM bridge.

Behaviour recorded on 2026-08-28¶

• hermes acp --check returned Hermes ACP check OK on VM 121.
• A JSON-RPC initialize over ssh -T jarvis-vm hermes acp returned cleanly, with pure JSON-RPC on stdout and Hermes logs on stderr.
• The handshake reported agent name hermes-agent, version 0.20.0, protocol version 1, loadSession: true, session capabilities fork, list and resume, and auth methods openai-codex ("openai-codex runtime credentials") plus the terminal-based hermes-setup method.
• Hermes' documentation says ACP sessions are in-memory only and server restart clears session history, while the live agent advertised loadSession, fork, list and resume. This is an observed discrepancy, not a resolved statement about persistence.
• End-to-end tool execution and filesystem access worked. The agent wrote /home/jarvis/acp-connection-test.txt, proving that its filesystem context is the VM rather than the laptop workspace. Durable work must still cross the established GitHub boundary; no completed work may exist only on the VM.
• A session took roughly 10 seconds to start while plugin discovery found 55 plugins and enabled 48.

Phase 13 is complete: Stage D passed 2026-09-01. An audit of 17 laptop configuration surfaces found no critical-path dependency on VM 121, and a live test with the VM stopped confirmed the bridge fails closed (ssh -T jarvis-vm hermes acp times out cleanly) while git and gh to github.com keep working; Jarvis recovered ~20s after restart. The bridge is non-essential to laptop Git work by design.

VM maintenance recorded with the bridge¶

The same session ran hermes doctor --fix, which migrated _config_version from 33 to 34. hermes acp --setup-browser installed agent-browser and Chrome for Testing 152.0.7977.64 under /home/jarvis/.agent-browser/browsers/; the binary launched, so the installer's suggestion to run agent-browser install --with-deps did not apply. The remaining hermes doctor findings are npm advisories in the web and ui-tui build workspaces (3-4 high in each). These are build-tooling advisories, not a runtime risk.

Only openai-codex was authenticated, using gpt-5.6-sol; Nous Portal, MiniMax, xAI and all other providers were logged out. The live profile was default with its gateway running. The retired neteng profile still existed on disk in a stopped state; it has not been live since 2026-08-09.

Memory cap repair¶

Hermes injects ~/.hermes/memories/MEMORY.md in full into the system prompt at session start. memory_char_limit is a hard cap: a write that would exceed it returns an error rather than silently dropping entries. The 2026-08-09 neteng-to-default consolidation appended the old profile's memory to the base file, growing it from 1,328 to 24,964 characters against a 2,200-character cap (11.3 times over). It then went unwritten for 19 days and 80 sessions, consistent with memory writes returning errors.

The 2026-08-28 fix set and read back the larger cap through the harness with hermes config set memory.memory_char_limit 40000, then rebuilt MEMORY.md to 30,088 characters. Seven entries were removed after each was found to be false or obsolete: the claude-opus-4-8/Anthropic OAuth fallback chain with Nous fallbacks; the claim that pve2 had no node exporter or Alloy; a nonexistent 02:00 user-systemd timer said to restart both gateways; three point-in-time PBS backup-freshness JSON snapshots; and a duplicated Ubuntu VM entry.

Three facts were corrected in place: pve1 SSD life is 14% as of 2026-08-28; the pve1 HighMemoryUsage condition was resolved on 2026-08-11 by capping ZFS ARC at 2 GiB; and the expected-state table now records VM 123 haos as parked (stopped, onboot=0). The embedded "MDH Media Homelab - Network Engineer Knowledge Pack" seed was preserved intact.

The trade-off was explicit: all 30,088 characters are injected on every session, roughly 11,000 tokens against the intended budget of about 800. The operator chose that cost, and it is reversible by moving the knowledge pack back into HOMELAB.md. The standalone ~/.hermes/HOMELAB.md is not referenced by config.yaml or .env, so it is not auto-injected. SOUL.md tells Hermes to read it without giving its location; that is why the agent looked in /home/jarvis and incorrectly reported the file missing.

Security posture¶

• Command approval manual (never YOLO mode); Telegram allowlist, deny-by-default gateway auth; SSRF protection on.
• Hardened systemd unit: NoNewPrivileges, ProtectSystem=strict, PrivateTmp.
• The CT 101 Arcane manager can create and destroy containers on VM 121. A separate Arcane headless agent has been live there since 2026-08-10 as the Jarvis VM environment, beside the manager's Local Docker environment 0; its environment id is a UUID, not 1. The agent mounts /var/run/docker.sock, giving the manager root-equivalent container control on VM 121. This is an accepted, deliberate trade-off for managing a genuinely separate docker host, not a reversal of the 2026-07-21 retirement of the redundant same-host agent. Exposure is limited to the LAN-only 192.168.1.252:3553 bind, with no Caddy route. The bind limits the interface, not the source, and Docker's NAT bypasses the VM's ufw, so a boot-persistent DOCKER-USER guard (arcane-agent-fw timer on VM 121, installed from stacks/management/arcane-agent/deploy/) restricts :3553 to the CT 101 manager (192.168.1.241) only — verified live 2026-08-23 (#302). The agent is capped at mem_limit: 512m. Its image is distroless, so inspect it with docker inspect and docker logs, not docker exec ... sh. The agent is a separate container; Hermes itself does not receive the docker socket.
• The jarvis user retains passwordless sudo (NOPASSWD:ALL) and docker-group membership by design: the VM boundary plus qm stop 121 is the accepted containment, so intra-VM privilege reduction was deliberately not pursued. The security audit's two "CRITICAL" findings (docker-group defeating systemd hardening; NOPASSWD sudoers) are accepted trade-offs, not open items.
• Provider credentials are held in Hermes' local auth store, never in git.

Model authentication¶

The 2026-08-28 session recorded openai-codex/gpt-5.6-sol as the only authenticated provider/model. Nous Portal, MiniMax, xAI and every other provider were logged out, so no provider fallback was available. The full provider configuration runbook is hermes-vm/agent/MULTI_PROVIDER_SETUP.md.

Base default profile — homelab network engineer¶

Since 2026-08-09, Jarvis has run one profile: base default. Its gateway is running. The retired neteng profile remains on disk but is stopped; its crons, environment, SOUL.md, HOMELAB.md and workspace were ported to default, with their model pins intact. The homelab network/SRE capabilities below therefore remain live even though that profile is retired.

• Taught the homelab. The network-engineer knowledge pack - topology, host inventory, Loki/PromQL cookbooks, deployment model and quirks - is embedded in MEMORY.md and therefore injected with that file. The standalone ~/.hermes/HOMELAB.md is not auto-injected; see the memory-cap finding above.
• Monitors. The grafana-stack MCP gives it live Loki + Prometheus queries; the ALERTS metric is its source of truth for what is firing.
• Fixes within a tight scope. Its one write capability is restarting existing containers through a scoped, revocable Arcane API token. Everything structural (Caddy, Proxmox, DNS, data) is proposed via Telegram and routed through the GitOps repo.
• Acts on the smart home — scoped (2026-08-01). Jarvis has API access to the rebuilt Home Assistant (the HAOS VM, built but parked since 2026-08-27 with onboot=0; the old .240 instance owns the route until the VM is started and set onboot=1 at cutover): it can read any entity but act only on lights, switches and scenes (HA_URL/HA_TOKEN live in the base profile env). It is one leg of the future voice story — HA Assist with an AI conversation agent and/or the Jarvis Hermes HA skill.
• Keeps the pane of glass honest. It annotates Grafana for every incident and action; the "Jarvis Ops" dashboard renders those as an incident timeline.
• Alerts two ways. A Grafana-native Telegram contact point fires straight from Grafana (independent of the agent — the dumb-pipe safety net). The triage layer that once sat on top — neteng-health-sweep, neteng-daily-digest and neteng-weekly-review — is present but disabled (verified in the live cron list, 2026-08-12), and carries no model pin, so re-enabling one without pinning it would fail closed. Grafana's own alerting is what actually pages today. The jobs kept their neteng-* names when ported, so the retired profile name still appears in logs; it does not identify a live profile.
• Protects the data. Three data-protection cron jobs — all pinned to openai-codex/gpt-5.6-sol (Hermes fail-closes an unpinned cron whose inference config has drifted — "#44585") — run under the SOUL.md "Reclaim and malware protocol", the only sanctioned auto-deletes:
  • neteng-malware-sweep (daily 07:30) auto-removes fake video torrents (video category + an executable file extension + no real video file; capped at 5 per run). It prefers the Radarr/Sonarr queue delete with blocklist=true — the bad release is blocklisted and re-searched — and falls back to a direct qBittorrent delete only for orphans. Its first run removed three fakes (an Odyssey .exe and two House of the Dragon .scr).
  • neteng-reclaim-scan (Mon 09:00) cross-references *arr imported history against qBittorrent and writes workspace/reclaim-pending.tsv; a Telegram reply (reclaim delete / list / cancel) acts on it — nothing is deleted without that reply.
  • neteng-backup-watchdog (daily 20:05 — moved from 06:00, when the wake-on-LAN PBS box is asleep) alerts if a guest's newest PBS snapshot is over 30 h old or missing, or the datastore passes 90%. It also tracks the age of Holly's host/holly appdata backup (uploaded daily at 20:35, so ~23.5 h old at the 20:05 check is healthy). An unreachable PBS is expected (it sleeps between backup windows) and stays silent.

The retired profile folder, source and deploy script are archived in archive/neteng-profile/. See hermes-vm/agent/README.md for the full write-up.