agents/ is a Bun workspace. Use Bun commands inside it.

From repo root:

Validation:

test:trigger boots the real daemon against a local stand-in for hyper.media's /api/ListEvents, creates an agent and a user-mention trigger over the signed API, and asserts that a comment mention fires exactly one session. It guards the comment and citation sibling race. See triggers.

The desktop app:

Build the deployment image from the repo root:

Run the image with persistent state mounted at /data:

The image records what it was built from and exposes it at runtime. /api/version (and /agents/api/version) returns:

The commit, branch, and date fields mirror the daemon /debug/version and web /hm/api/version shape. version (the image tag) is specific to agents. /api/health also includes the version field for quick checks. To see what a host is running:

The values come from agents/src/build-info.ts, populated by Docker build args (VERSION, COMMIT_HASH, BRANCH, DATE; the web and daemon images use the same COMMIT_HASH, BRANCH, and DATE names). agents/Dockerfile maps them to namespaced SEED_AGENTS_* runtime ENV. Without build args (for example local bun dev or a plain docker build) they fall back to dev or unknown.

CI build and push

Manual build and push from a workstation

agents/scripts/build-and-push.sh dev      # or: latest

Deploy onto the agent host

# Force an update now, with compose (the reliable way):
ssh ubuntu@agentic.seed.hyper.media \
  'cd /opt/agentic && docker compose pull agents-stable agents-staging agents-dev && docker compose up -d'

# Or a one-shot Watchtower. Note: only works when the resident Watchtower is stopped — it treats a
# second instance as an "excess Watchtower container" and SIGTERMs it mid-run.
ssh ubuntu@agentic.seed.hyper.media \
  'docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
     nickfedor/watchtower:1.18.1 --run-once --label-enable --cleanup'

Config source: agents/src/config.ts.

CLI flags override env vars and defaults:

Every environment variable in the table has a matching flag (--searxng-url, --exec-backend, --exec-ts-image, --subscription-auth, --session-title-generation, …); the full list is the Flags type in config.ts. An unknown flag or a flag with an empty value is a startup error, so a typo fails loudly and never runs with a default.

Web research is fully self-hosted and uses no third-party API keys. It covers the web_search callable tool and the tiered reader behind read https://… (MediaWiki, then in-process static Readability+Turndown, then Crawl4AI). See tools. Both backends are optional. Without configuration, web_search errors and web reads fall back to the MediaWiki and static tiers. To enable them, run the backends as sidecar containers on the same internal network and point the agents service at them.

For local development a ready-to-run compose and SearXNG config live at agents/dev/web-backends/. Start the containers, then run the service. bun run dev already points at these dev endpoints by default:

The dev script defaults SEED_AGENTS_SEARXNG_URL (http://127.0.0.1:8899), SEED_AGENTS_CRAWLER_URL (http://127.0.0.1:11235), and SEED_AGENTS_CRAWLER_TOKEN (dev-crawl-token) to the compose values. Export those env vars to override them. The same script also turns SEED_AGENTS_SUBSCRIPTION_AUTH on for local development. The startup log prints Web tools: search=on reader=static+crawl4ai when wired. The reader's MediaWiki and static tiers still work if the containers are not running. Only web_search and the browser escalation need them. Stop the backends with docker compose down in agents/dev/web-backends/.

For production the same two containers run as internal sidecars:

Required SearXNG settings.yml (the JSON format is off by default and the limiter blocks API calls):

Then wire the agents container:

Capacity note: Crawl4AI runs a headless Chromium and documents a >=4 GB RAM minimum plus 1 GB shared memory. Size the host accordingly. The SearXNG and in-process static reader path is lightweight. Crawl4AI is the heavy escalation tier.

The health endpoints (/api/health, /agents/api/health) report what this server can actually do:

ipfsServerUrl defaults to hmServerUrl, which is correct for hosted all-in-one origins. Local desktop environments set them separately because the desktop bridge owns /api/* while the daemon owns gateway reads under /ipfs/*. IPFS publication itself chunks UnixFS blocks and uses PublishBlobs on hmServerUrl. webTools derives from SEED_AGENTS_SEARXNG_URL and SEED_AGENTS_CRAWLER_URL. codeExec derives from the exec backend probe, and codeExecReason and codeExecReasonCode explain an unavailable sandbox. codeExecRuntimes says which runtimes this host offers. ts needs an image with bun, so an operator can see at a glance whether TypeScript execution is on here. Clients read these fields to grey out what the server cannot run.

The execute tool, and every authored lambda tool (which runs in the same sandbox), uses the embedded microsandbox npm runtime: hardware-isolated microVMs with no separate server process. Host requirements: Apple Silicon on macOS, KVM on Linux, WHP (Windows Hypervisor Platform) on Windows. The runtime and native binaries install with the package. The first execution pulls the configured OCI image (default python), which takes tens of seconds. After that, sandboxes boot in well under a second. The ts runtime comes from a second image (SEED_AGENTS_EXEC_TS_IMAGE, default oven/bun); Setting it empty leaves TypeScript unavailable, and codeExecRuntimes reports that.

On a host that cannot run sandboxes, the tool is withheld so it never fails at call time. enabledCallableTools drops execute from the agent's callable set when the availability probe says no, so the model never sees a tool that can only error. SEED_AGENTS_EXEC_BACKEND=off disables the backend outright.

When the agents service runs inside a container, the container needs access to the host virtualization device (on Linux, --device /dev/kvm). Without it, executions fail with the backend-unavailable error, and the rest of the service keeps working.

Sandbox networking is on by default so agents can install packages and fetch data. The runtime gives each sandbox an explicit DNS resolver (SEED_AGENTS_EXEC_DNS, default 1.1.1.1,8.8.8.8; a guest has no resolver otherwise) and a non-local egress policy. The sandbox reaches the public internet but not the host's private network or cloud-metadata endpoints (for example 169.254.169.254). Set SEED_AGENTS_EXEC_ALLOW_NETWORK=false to cut the sandbox off entirely. Installed packages do not survive between calls, because each call runs in a fresh microVM. The tool prompt tells agents to pip install --target /workspace/pylibs <pkg> so packages persist in memory and can be re-imported later.

Production deployment of these sidecars on the hosted agent server is handled in the mintterteam/infrastructure repo (the seed_infra/agentic Terraform stack adds the searxng and crawl4ai containers and wires the env vars). It is not in this repo. Security covers the sandbox threat model.

Default local files:

SQLite is authoritative for everything except bytes on disk. An agent's state directory is created with the agent and holds its memory/ filesystem (the ~/memory/ half of its Space) plus session-private attachments. It is removed with the agent. See persistence. Staged chunked uploads live under the data dir until they are committed or expire.

Everything else is a 404. The server has no browser UI and no unauthenticated data routes. Account-owned state (agents, triggers, sessions, events, memory) is reachable only through signed envelopes on /api/message and /agents/ws. Use the desktop app, or the signed API (ListAgents, ListSessions, GetSession, GetAgentTrigger, ListRuns), to inspect a server.

The server starts a background ActivityFeed monitor for enabled HM activity triggers. It polls on SEED_AGENTS_ACTIVITY_POLL_INTERVAL_MS (default 5 seconds) and does not contact the HM activity feed unless an account has at least one enabled non-schedule trigger.

Each polled event is offered to two consumers: trigger matching, and any run parked on ctx.waitForEvent({eventType, resource, author}) for that account (#deliverActivityToRunWaits). A wait belongs to one run, and a trigger is the agent's standing configuration, but both use the same feed. An account with no enabled activity trigger is never polled, so an activity-shaped wait on such an account only resolves by timeout. Signal-shaped waits do not depend on polling at all. The first poll for an account establishes a baseline watermark and only processes activity observed after the earliest enabled trigger was created. Later polls process new ActivityFeed events through trigger matching. If a persisted watermark is older because the server was down, the monitor backfills unseen events for up to 1 hour and then advances the watermark.

The server also starts a background schedule monitor. It evaluates enabled schedule triggers on the same poll interval, records durable trigger_firings rows with schedule:<triggerId>:<scheduledAt> idempotency keys, creates sessions for due occurrences, and disables one-time triggers after a successful run.

Every agent execution is a durable row in the runs table, which is also the dispatch queue (agents/src/runs.ts). See Runs:

Queues: interactive user turns are claimed inline by MessageSession (unchanged latency). Everything else (trigger firings, agent-started sessions, delegated model children, script children) dispatches on the background queue. Agent runs are capped at 8 concurrent provider streams. Script runs get their own pool of 32 (they hold no provider stream), so a script awaiting its children can never starve them.

One live agent run per session is enforced at claim time (lease-based). This replaced the old racy status-column 409.

Boot sweep: on service construction, runs a dead process left claimed or running are requeued, and any queued backlog resumes. Agent runs resume by replaying their durable session events. A crash between a persisted tool_call and its tool_result gets a synthesized "interrupted by service restart" result so the provider request is well-formed and the model decides whether to retry. Script runs resume by journal replay.

Retry classification: provider 5xx and network failures are retryable with exponential backoff (base 5s, cap 5min). Validation and config errors fail immediately. maxAttempts is 3 (AGENT_RUN_MAX_ATTEMPTS) for background agent runs: trigger firings, agent-started sessions, delegated model children. It is 1 for interactive turns, which have a person waiting, and for script runs, which resume from their journal and never restart.

Parking: a run parks for one of four reasons: its children; a timer (ctx.sleep of 60s or more parks with not_before, and the dispatcher's 1-second interval wakes due timers); an event (ctx.waitForEvent, woken by a SignalRun, a trigger wake continuation, an activity event, or its timeout); or a budget pause a person resumes. Parked runs hold no resources.

Leases expire after 60 seconds. The boot sweep uses them to tell a live claim from a dead one.

Tests and shutdown wait for the queue via Service.drainTriggerSessions(). awaitQueueIdle() is an alias for it under the name the runs feature documents.

Script execution bounds (see agents/src/workflow-host.ts): QuickJS-WASM realm per run, 64 MiB memory cap, 2s pure-compute fuel between awaits, 256 KiB source cap, and journal caps of 5,000 entries / 8 MiB per run (WORKFLOW_JOURNAL_MAX_ENTRIES / WORKFLOW_JOURNAL_MAX_BYTES in api-service.ts). A long-lived loop escapes the journal cap with ctx.continueAsNew, which ends the run and starts a successor carrying only the declared state.

The cassettes are stale and the replay gate is currently skipping. agents/e2e/recordings/STALE.md records why:

every cassette was recorded against the pre-harness tool surface (sub_session, run_workflow, update_plan,

memory_*, execute_code), and the verb collapse changed both the tool names and the system prompt, so every replay

fingerprint is invalid. While that file exists, bun e2e/run.ts prints recordings are STALE … skipping replay,

reports 0 pass, 0 fail (skipped), and exits 0. That keeps src/e2e-replay.test.ts green without pretending

coverage exists. Do not read a passing bun test as evidence that these gates ran. Restoring them means

re-recording (below), verifying every scenario, then deleting STALE.md and any cassette the re-record did not

overwrite.

agents/e2e/run.ts drives the real Service through six behavioral gates. By default it replays recorded gpt-5-mini responses from agents/e2e/recordings/: full service and tool loop, no network, no API key. agents/src/e2e-replay.test.ts runs that replay as part of the regular bun test suite. Pass --record to hit the live OpenAI API and refresh the cassettes. That spends real tokens, so run it manually and never in CI:

Recording proxies provider traffic through a local server that captures each response keyed by a request fingerprint (model + system-prompt head + last user message + tool names + tool-result count). Replay matches the same fingerprints, so cassettes survive tool-description edits but need re-recording when system prompts or conversation flow change. Only a passing run overwrites a scenario's cassette. The live key comes from OPENAI_API_KEY or the repo-root .keys file (never committed). Every scenario asserts on durable state (runs, journals, session events) and dumps full transcripts to agents/e2e-artifacts/<timestamp>/ for prompt autopsies. Scenarios: chat-smoke, sub-basic, sub-typed, sub-restraint, wf-hello, todo-adoption.

Status: the cassettes on disk were last recorded from live gpt-5-mini before the verb collapse and are now stale (see the note at the top of this section). The scenario code itself still exercises the current Service. The last live recording pass found and fixed three real-model defects: top-level-array output schemas now rejected at spawn time with a self-correctable message, functions.-prefixed tool names normalized in scripts' ctx.call, and the standard minItems/maxItems/maxLength/maximum keywords added to the schema subset.

agents/e2e/live-gate.ts is the other half. It is a pure client that signs envelopes with the desktop daemon's key over grpc-web, exactly as the desktop app does, and talks CBOR to an already-running server. So it gates the server an operator actually uses, and not an in-process Service. It creates a throwaway NightGate agent mirroring an existing agent's provider/model/tools, runs the scenarios, writes full transcripts to an artifact directory, and deletes the agent again.

Flags: --base (agents server), --daemon (Seed daemon grpc-web, default http://localhost:56001), --account (uid to sign as), --provider / --model, --like (agent id to mirror), --out, --timeout (per-scenario ms, default 600000), --keep (leave the NightGate agent behind).

Scenarios: trivial (restraint plus self-naming: arithmetic must be answered directly, with no delegation, and the session still ends up model-titled), memory (a fact written to ~/memory in one session, read back by a second), parallel-delegate, script-parallel, script-narration. Enforced checks are deterministic properties. Model behavior is reported and not asserted.

This gate spends real tokens and needs a running server with a working provider. It is never part of bun test.

Two focused companions live beside it. Both are model-free, so they gate a mechanism and not a model:

e2e/narration-check.ts drives the script VM directly (no model, no server, no credentials) and asserts that a ctx.call(tool, input, {description}) label reaches both the durable journal entry and the tool-call adapter. The UI reads the first, and the run transcript reads the second.

e2e/obligations-live-check.ts drives a real server over HTTP with a scripted provider (e2e/scripted-provider.ts) through the whole unified-obligations loop: a plan-shaped task, a step left open at turn end, the continuation the runtime hands itself, and the settled checklist afterwards.

Simulated-model gates (no API key required)

On startup:

config.create(config.parseArgs()) builds config.

sqlite.open(cfg.dbPath) validates or initializes the DB.

If schema is valid, Service (which boot-sweeps the run queue), the activity trigger monitor, and the schedule trigger monitor are created, and the Bun server starts.

If the schema is mismatched, the server starts in schema-mismatch mode and returns a JSON error.

The schema mismatch log includes the stored and expected version. For local throwaway data, delete the SQLite files and restart.

The service handles SIGINT and SIGTERM:

stop repeated shutdown handling;

stop the activity trigger monitor and the schedule monitor;

close WebSocket clients with code 1001;

clear client set;

stop Bun server;

drain in-flight background runs, bounded at 5 seconds so one stuck session cannot block shutdown, then stop the run-queue timers;

close SQLite DB;

exit.

Health and CBOR API routes return permissive CORS headers. Security comes from signatures and account authorization. Browser origin plays no part.

Logs include IDs, counts, statuses, sizes, timings, trigger sources, and compact activity metadata, on purpose. They never include secrets or full message or session content.

Logging is leveled (agents/src/log.ts, SEED_AGENTS_LOG_LEVEL or --log-level, default info). The per-event hot paths log at debug and are silent by default: every WebSocket partial (publish partial, send partial, skip partial) and every no-op activity poll (Polling feed, Feed page received, Processing feed events with nothing new). Measured in production, they were ~90% of ~440k lines/hour. Set the level to debug when tracing streaming or trigger delivery.

Activity trigger diagnostics use:

[Agents Activity] Polling feed

[Agents Activity] Feed page received

[Agents Activity] First poll processing events

[Agents Activity] Processing feed events

[Agents Activity] Poll failed

[Agents Trigger] Skipping activity without stable key

[Agents Trigger] Checked activity against trigger

[Agents Trigger] Skipping duplicate trigger firing

[Agents Trigger] Skipping run-completed trigger already in this chain

[Agents Trigger] Fired trigger and created session

[Agents Trigger] Trigger session run enqueued

[Agents Trigger] Trigger session dispatch failed

[Agents Trigger] Trigger woke a parked run

[Agents Trigger] Trigger fired with no run listening

[Agents Trigger] Trigger firing failed

Run, runtime, and script diagnostics:

[agents/runs] boot sweep requeued interrupted runs

[agents/runs] boot reconcile: replaying finished child into parked parent

[agents/runs] activity woke a waiting run

[agents/runtime] sending provider request

[agents/runtime] ending turn after tool batch

[agents/runtime] sub-session spawned / [agents/runtime] sub-session resolved

[agents/runtime] plan step settled from children

[agents/runtime] agent started session / [agents/runtime] agent-started session run failed

[agents/runtime] session titled by model / [agents/runtime] session title generation failed

[agents/workflow] workflow spawned from chat

[agents/workflow] continued as new run

[agents/workflow] run paused on its time budget

Server model execution goes through the Pi SDK. The old manual OpenAI stream logs are not emitted on the primary Pi-backed path. Use durable session events (via GetSession or the desktop session page), WebSocket partial logs, and mocked tests for current runtime diagnosis. Add Seed-level Pi runtime diagnostics before production if real-provider troubleshooting needs more visibility.

Server WebSocket logs:

[agents/ws] open

[agents/ws] subscribed

[agents/ws] publish partial

[agents/ws] send partial

[agents/ws] skip partial; no subscription

[agents/ws] close

Desktop WebSocket and UI logs:

[agents/ws] connecting

[agents/ws] open; signing subscribe

[agents/ws] subscribe sent

[agents/ws] subscribed event

[agents/ws] partial event

[agents/ws] partial state updated

[agents/ws] partial marked done; keeping visible until durable append

[agents/ui] sending session message

[agents/ui] rendering streaming assistant partial

Desktop says server offline

curl http://localhost:3051/agents/api/health

direnv exec . bash -lc 'cd agents && bun src/main.ts'

WebSocket subscription says Invalid signature

No live streaming appears

API key save fails

Schema mismatch

rm -f agents/data/agents.sqlite agents/data/agents.sqlite-shm agents/data/agents.sqlite-wal

Do not add logs that include:

plaintext secrets;

decrypted API keys;

signed request bodies;

full model request or response bodies;

full session messages;

large tool outputs.

Prefer logging IDs, lengths, counts, timings, statuses, and booleans.