gib/spoon
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: rewrite for box-first model + webhooks + notifications

Browse Source
This commit is contained in:
Gabriel Brown
2026-07-11 18:23:49 -04:00
parent 1fbb2dd0a1
commit a255a06a35
5 changed files with 332 additions and 166 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/agent-terminal.md
+44 -30
View File
@@ -1,34 +1,44 @@
# Workspace interactive terminal
# Interactive terminal
A real shell inside the agent workspace, shown as the **Terminal** tab in the
workspace UI. It's an xterm.js front end bridged to a bash/tmux PTY running in a
persistent per-job container (the agent job image), mounting the same workspace
the editor and agent use.
A real shell inside the user's persistent box (`spoon-box-{username}`). There are
two entry points, both bridging to the **same** box:
- The workspace **Terminal** tab — a shell opened at the thread's checkout
(`~/Code/{spoon}/{branch}`).
- The **My Machine** page (`/machine`) — a `~`-rooted shell into the same box.
Both are an xterm.js front end bridged to a bash/tmux PTY via `docker exec` into
the box. There is no per-job `spoon-agent-term-*` container anymore — the box is
long-running and shared by the agent, the editor, and both terminals.
## Architecture
```
browser (xterm.js)
│ 1. GET /api/agent-jobs/:id/terminal-token (Convex-auth'd, owner only)
│ → { url: "wss://worker…/jobs/:id/terminal?token=…", expiresAt }
│
│ 2. WebSocket wss://worker.<domain>/jobs/:id/terminal?token=…
│ 1a. GET /api/agent-jobs/:id/terminal-token (Convex-auth'd, owner only)
│ → { url: "wss://worker…/jobs/:id/terminal?token=…", expiresAt }
│ 1b. GET /api/box/terminal-token (Convex-auth'd; username
│ → { url: "wss://worker…/box/terminal?token=…", expiresAt } derived
│ server-side)
│ 2. WebSocket wss://worker.<domain>/{jobs/:id|box}/terminal?token=…
▼
nginx ── upgrade ──► spoon-agent-worker :3921
│ verifyTerminalToken(token, jobId, secret)
│ dockerode exec -t → bash/tmux PTY
│ verify HMAC token (job-scoped or username-scoped)
│ docker exec -it → bash/tmux PTY
▼
spoon-agent-term-<jobId> (job image, mounts the workspace)
spoon-box-{username} (Fedora box; persistent home mounted)
```
- The browser **never** holds the worker secret. The Next app (which has already
verified job ownership) mints a short-lived HMAC token; the worker verifies it.
- The browser **never** holds the worker secret. The Next app mints a short-lived
HMAC token; the worker verifies it. The job terminal token is minted only after
a Convex ownership check; the box terminal token carries the username derived
**server-side** from the authed user (never from the request).
- Frames: **binary** = stdin/stdout bytes; **text JSON** `{type:"resize",cols,rows}`
= resize. The token's 2-minute expiry is a _connect_ window; an established
session persists.
- The shell runs `tmux new-session -A -s spoon` (falls back to `bash -l`), so
reconnecting reattaches the same session. Idle containers are removed after
`SPOON_AGENT_TERMINAL_IDLE_MS` (default 30m).
- The shell prefers `tmux new-session -A -s spoon` (falls back to `bash -l`), so
reconnecting reattaches the same session. The box itself is idle-reaped after
`SPOON_AGENT_BOX_IDLE_MS` (default 30m) once no job or terminal holds it.
## Configuration
@@ -37,8 +47,7 @@ nginx ── upgrade ──► spoon-agent-worker :3921
| Next app | `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL` | **Yes** | Browser-facing worker WS base, e.g. `wss://worker.spoon.gbrown.org` (prod) or `ws://localhost:3921` (dev). **Build-time** (`NEXT_PUBLIC`): for the Docker image it must be passed as a build arg (wired in `docker/Dockerfile` + `docker/compose.yml`, sourced from the build env file), not a runtime env. Unset → the Terminal tab shows "not configured". |
| Next app | `SPOON_AGENT_TERMINAL_SECRET` | No | HMAC secret for signing tokens. Falls back to `SPOON_AGENT_WORKER_INTERNAL_TOKEN`. |
| Worker | `SPOON_AGENT_TERMINAL_SECRET` | No | Must match the Next app's. Falls back to `SPOON_AGENT_WORKER_INTERNAL_TOKEN` (already shared), so by default **no new secret is needed**. |
| Worker | `SPOON_AGENT_TERMINAL_IMAGE` | No | Shell container image. Defaults to `SPOON_AGENT_JOB_IMAGE`. |
| Worker | `SPOON_AGENT_TERMINAL_IDLE_MS` | No | Idle-container reap delay (default `1800000`). |
| Worker | `SPOON_AGENT_BOX_IDLE_MS` | No | Idle-box reap delay, shared with agent jobs (default `1800000`). The terminal holds the box open while connected. |
Because the secret defaults to the already-shared worker token, the **only**
required step is setting `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL` and exposing the
@@ -70,9 +79,10 @@ server {
Then set on the Next app: `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL=wss://worker.spoon.gbrown.org`.
> The worker's HTTP routes (`/jobs/:id/tree` etc.) require the internal bearer
> token, so exposing the worker host only usefully exposes the token-gated
> `/jobs/:id/terminal` upgrade. Still, restrict the server block to TLS.
> The worker's HTTP routes (`/jobs/:id/tree`, `/box/*`, etc.) require the
> internal bearer token, so exposing the worker host only usefully exposes the
> token-gated `/jobs/:id/terminal` and `/box/terminal` upgrades. Still, restrict
> the server block to TLS.
## Dev testing (no nginx)
@@ -90,15 +100,19 @@ Docker mode. Prod (Docker socket mounted) works as-is.
## Security
- Owner-only: the token route uses Convex auth + `assertOwned`.
- Tokens are short-lived (2m connect window), job-scoped, HMAC-signed.
- A shell in the workspace can reach the network and the repo's git credentials.
This is intended for the single-user self-hosted deployment; do not expose the
- Owner-only: the job token route uses Convex auth + ownership assertion; the
box token derives the username server-side from the authed user.
- Tokens are short-lived (2m connect window), scoped (job-scoped or
username-scoped), and HMAC-signed.
- A shell in the box can reach the network and the repo's git credentials. This
is intended for the single-user self-hosted deployment; do not expose the
worker domain without TLS, and keep the deployment single-tenant.
## Tools in the shell
The job image ships `bash`, `tmux`, `neovim`, `git`, `ripgrep`, `jq`, `python3`,
`node`, `bun`, `pnpm`, `yarn`, `curl`/`wget`, `unzip`. Bring your own dotfiles by
cloning them in-session (e.g. `git clone <dotfiles> ~/.config/...`); persistent
auto-cloning of a dotfiles repo is a planned follow-up.
The box image ships `bash`, `tmux`, `neovim`, `git`, `ripgrep`, `jq`, `python3`,
`node`, `bun`, `pnpm`, `yarn`, `curl`/`wget`, `unzip`, plus QoL tooling (`fzf`,
`fd`, `bat`, `eza`, `zoxide`, `gh`, `gum`, `oh-my-posh`) and the agent CLIs
(`codex`, `opencode`, `claude`). Personalize the shell in **Settings → Dotfiles**
(overlay files + an optional dotfiles repo), applied to the persistent home; see
[dotfiles.md](dotfiles.md).
docs/compose.prod.yml
+13 -13
View File
@@ -7,18 +7,18 @@
# If you change them, read the comments first:
#
# 1. AGENT WORKDIR (spoon-agent-worker): the worker is containerized but
# launches the Codex job container by talking to the HOST Docker daemon.
# The host can only bind-mount real HOST paths, so the work directory MUST
# be a bind mount whose path is IDENTICAL inside and outside the container,
# and SPOON_AGENT_HOST_WORKDIR must match it. A named volume does NOT work
# here because its real host path is hidden from the worker. All three
# references to /var/lib/spoon-agent/work below must stay in sync; change
# them together if you want the data somewhere else.
# creates and `exec`s into the per-user box (spoon-box-{username}) by talking
# to the HOST Docker daemon. The host can only bind-mount real HOST paths, so
# the work directory MUST be a bind mount whose path is IDENTICAL inside and
# outside the container, and SPOON_AGENT_HOST_WORKDIR must match it. A named
# volume does NOT work here because its real host path is hidden from the
# worker. All three references to /var/lib/spoon-agent/work below must stay
# in sync; change them together if you want the data somewhere else.
#
# 2. IMAGE FRESHNESS: services use `pull_policy: always` + Watchtower labels so
# a redeploy / new push always lands. The Codex *job* image is pulled by the
# worker itself on startup (see SPOON_AGENT_JOB_IMAGE); restarting the worker
# (which Watchtower does on a new image) re-pulls a fresh job image.
# a redeploy / new push always lands. The *box* image is pulled by the worker
# itself on startup (see SPOON_AGENT_JOB_IMAGE); restarting the worker (which
# Watchtower does on a new image) re-pulls a fresh box image.
networks:
nginx-bridge: # Change to network you plan to use
@@ -83,9 +83,9 @@ services:
- SPOON_WORKER_TOKEN=${SPOON_WORKER_TOKEN}
volumes:
- /var/run/docker.sock:/var/run/docker.sock
# Identical host:container path so the sibling Codex job containers can
# bind-mount the workspace via the host daemon. Do NOT switch this to a
# named volume. See header note (1).
# Identical host:container path so the sibling per-user box can bind-mount
# the persistent home (homes/{username}) via the host daemon. Do NOT switch
# this to a named volume. See header note (1).
- /var/lib/spoon-agent/work:/var/lib/spoon-agent/work
labels: ['com.centurylinklabs.watchtower.enable=true']
tty: true
docs/dotfiles.md
+14 -10
View File
@@ -6,10 +6,11 @@ preloaded with QoL CLI tooling, a persistent per-user home, and user dotfiles.
## The model
- **Persistent per-user home.** Each user gets a home directory on the worker
host at `${SPOON_AGENT_WORKDIR}/homes/{username}`, bind-mounted into every
job/terminal container at `/home/{username}` (`HOME`). It survives across
sessions, so dotfiles, installed tools, nvim plugins, and shell history persist.
`username` is derived from the user's profile first name (sanitized).
host at `${SPOON_AGENT_WORKDIR}/homes/{username}`, bind-mounted into the
per-user box (`spoon-box-{username}`) at `/home/{username}` (`HOME`). It
survives across sessions, so dotfiles, installed tools, nvim plugins, and shell
history persist. `username` is derived from the user's profile first name
(sanitized).
- **Threads as folders.** Each thread's checkout lives at
`~/Code/{spoon}/{branch}` inside that home, so every thread shows up as a
folder in one home. The agent (`codex --cd …`) and the terminal both open there.
@@ -61,18 +62,21 @@ Secrets: dotfiles are encrypted, but real API keys/tokens belong in a Spoon's
No new required env. The home is a host directory under the existing workdir, so
the prod bind-mount + `SPOON_AGENT_HOST_WORKDIR` translation already covers it.
## Notes / limits (Phase 1)
## Notes / limits
- **Repo auth:** public repos only. Private/self-hosted (e.g. Gitea) dotfiles
repos are a follow-up (store a token/deploy key).
- **Binary files:** the overlay is text-first.
- **Cleanup:** `~/Code/{spoon}/{branch}` checkouts persist (threads as folders);
a per-thread "delete checkout" action is a follow-up.
- **Concurrency:** jobs share one home; fine at the default
- **Concurrency:** jobs and terminals share one box/home; fine at the default
`SPOON_AGENT_MAX_CONCURRENT_JOBS=1`.
## Phase 2 north star
## The box
A single long-running per-user container that every thread `exec`s into (agent
via `docker exec`, not `docker run --rm`). The per-user home + `~/Code/{spoon}/
{branch}` layout built here is its foundation.
The per-user home + `~/Code/{spoon}/{branch}` layout is the foundation of the
box model: a single long-running per-user container (`spoon-box-{username}`) that
every thread's agent turn, terminal, and command `exec`s into (`docker exec`, not
`docker run --rm`). Home materialization runs once per box start (and again when
the dotfiles config hash changes), so the overlay and repo setup apply to every
session in that box.
docs/server-deploy-changes.md
+36 -13
View File
@@ -1,7 +1,7 @@
# Server deploy changes (terminal + dotfiles + Fedora + Phase 2)
# Server deploy changes (box + terminal + dotfiles + webhooks + notifications)
Everything the production host / compose / `.env` needs for the workspace
terminal, personalized dev environment, Nerd Font, and the per-user container.
Everything the production host / compose / `.env` needs for the per-user box, the
terminal, the personalized dev environment, GitHub webhooks, and notifications.
Most items have safe defaults; the **Required** ones are the only must-dos.
## Required
@@ -44,9 +44,10 @@ Most items have safe defaults; the **Required** ones are the only must-dos.
restart picks up the new Fedora job image. Make sure the prod registry has the
new `spoon-agent-job:latest`.
4. **Deploy Convex functions** (new tables `userDotfiles`, `userEnvironment`).
`SPOON_ENCRYPTION_KEY` (or `INSTANCE_SECRET`) is already required and is what
encrypts dotfiles at rest — no change, just confirm it's set.
4. **Deploy Convex functions** (new tables `userDotfiles`, `userEnvironment`,
`notifications`, `notificationPreferences`). `SPOON_ENCRYPTION_KEY` (or
`INSTANCE_SECRET`) is already required and is what encrypts dotfiles at rest —
no change, just confirm it's set.
5. **Confirm `SPOON_AGENT_HOST_WORKDIR`** on the `spoon-agent-worker` service is
the absolute host path backing `SPOON_AGENT_WORKDIR` (the fix from the terminal
@@ -54,17 +55,36 @@ Most items have safe defaults; the **Required** ones are the only must-dos.
and are bind-mounted into the box via the host daemon — this only resolves if
the host-workdir translation is correct. (No new var; just verify.)
6. **Configure the GitHub App webhook.** Set the App's webhook URL to
`<CONVEX_SITE_URL>/webhooks/github` and set `GITHUB_APP_WEBHOOK_SECRET` in the
**Convex** production env (the signature-verified `httpAction` runs in Convex,
not the worker). Without the secret the endpoint fails closed (HTTP 503) and
drift only refreshes via the hourly cron. `push` refreshes affected Spoons;
`installation` / `installation_repositories` update connection status.
7. **Notifications email (optional).** In-app notifications work with no config.
To also send email, set `USESEND_API_KEY`, `USESEND_URL`, and
`USESEND_FROM_EMAIL` in the Convex env. Per-user, per-kind email toggles live
in `Settings → Notifications`; unset preferences default to on.
8. **Claude Code runtime.** The box image pins `@anthropic-ai/claude-code`, so
Anthropic provider profiles (API key or `anthropic_oauth_json` credential
snapshot) can drive the `claude` runtime with no extra host env. Anthropic
API-key auth flows through `ANTHROPIC_API_KEY`; OAuth snapshots are written to
`~/.claude/.credentials.json` in the box.
## Optional (safe defaults — only set to override)
On the `spoon-agent-worker` service:
| Var | Default | Purpose |
| ------------------------------ | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `SPOON_AGENT_TERMINAL_SECRET` | falls back to `SPOON_AGENT_WORKER_INTERNAL_TOKEN` | HMAC secret for terminal tokens (must match the Next app's, which also falls back). Leave unset to use the shared token. |
| `SPOON_AGENT_BOX_IDLE_MS` | `1800000` (30m) | How long a per-user box survives idle before being reaped. |
| `SPOON_AGENT_TERMINAL_IDLE_MS` | `1800000` | (Legacy; box idle now governs cleanup.) |
| Var | Default | Purpose |
| ----------------------------- | ------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `SPOON_AGENT_TERMINAL_SECRET` | falls back to `SPOON_AGENT_WORKER_INTERNAL_TOKEN` | HMAC secret for terminal/box tokens (must match the Next app's, which also falls back). Leave unset to use the shared token. |
| `SPOON_AGENT_BOX_IDLE_MS` | `1800000` (30m) | How long a per-user box survives idle before being reaped. |
No new env is needed for dotfiles, the per-user home, or the Nerd Font.
`SPOON_AGENT_TERMINAL_IMAGE` and `SPOON_AGENT_TERMINAL_IDLE_MS` were removed —
the box uses `SPOON_AGENT_JOB_IMAGE` and `SPOON_AGENT_BOX_IDLE_MS`.
## Notes / one-time cleanup
@@ -84,7 +104,10 @@ No new env is needed for dotfiles, the per-user home, or the Nerd Font.
```bash
docker exec spoon-agent-worker docker --version # CLI present (29.x)
docker run --rm git.gbrown.org/gib/spoon-agent-job:latest codex --version # 0.142
docker run --rm git.gbrown.org/gib/spoon-agent-job:latest codex --version # 0.142
docker run --rm git.gbrown.org/gib/spoon-agent-job:latest claude --version # 2.1.207
docker run --rm git.gbrown.org/gib/spoon-agent-job:latest opencode --version
docker run --rm git.gbrown.org/gib/spoon-agent-job:latest bash -lc 'eza --version; zoxide --version; oh-my-posh --version'
# then: open a thread → Terminal tab; Settings → Dotfiles add a .bashrc alias.
# then: /machine → box status + terminal; open a thread → Terminal tab;
# Settings → Dotfiles add a .bashrc alias; Settings → Notifications toggles.
```