gib/spoon
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: rewrite for box-first model + webhooks + notifications

Browse Source
This commit is contained in:
Gabriel Brown
2026-07-11 18:23:49 -04:00
parent 1fbb2dd0a1
commit a255a06a35
5 changed files with 332 additions and 166 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+225 -100
View File
@@ -10,7 +10,8 @@
<p align="center">
Spoon is a self-hostable fork maintenance cockpit built around managed forks,
durable maintenance threads, and OpenCode-powered workspaces.
durable maintenance threads, and a persistent per-user dev box that agents,
terminals, and project commands all run inside.
</p>
<p align="center">
@@ -61,14 +62,38 @@ in the data model, but GitHub is the active automation surface today.
Spoon keeps raw GitHub drift visible while also tracking ignored upstream
changes so irrelevant commits do not keep a fork permanently actionable.
- **OpenCode workspaces**
Agent work happens in an isolated workspace with a file tree, browser editor,
diff viewer, command panel, logs, artifacts, and draft PR actions.
- **One persistent per-user box**
Every user owns a single long-running Fedora container `spoon-box-{username}`
with a persistent home. Every thread's agent turn, terminal session, and
project command `docker exec`s into that same box — there is no per-job
throwaway container.
- **Three agent runtimes**
Codex, OpenCode, and Claude Code all run inside the box behind one adapter
interface, selected per thread and validated at queue time against the AI
provider profile.
- **Workspaces**
Agent work happens in a workspace with a file tree, browser editor, diff
viewer, command panel, logs, artifacts, an interactive terminal, and draft PR
actions.
- **My Machine**
A `/machine` surface to start/stop/restart the box, open a `~`-rooted
terminal, and browse the persistent home.
- **GitHub webhooks**
A signature-verified webhook keeps drift fresh on `push` and flips connection
status on `installation` changes; the hourly cron is a fallback.
- **Notifications**
An in-app bell plus preference-gated transactional email for maintenance
threads, agent turns, needed input, sync failures, and connection re-auth.
- **User-owned providers and secrets**
AI provider profiles, Codex/OpenCode auth, and per-Spoon project secrets are
encrypted. Secrets are redacted from logs and refused from commits when
materialized into env files.
AI provider profiles, Codex/OpenCode/Anthropic auth, and per-Spoon project
secrets are encrypted. Secrets are written `0600`, redacted from logs, and
refused from commits when materialized into env files.
- **Draft PR handoff**
Code changes become branches and draft pull requests. Spoon does not
@@ -138,12 +163,23 @@ custom work exists.
</details>
<details>
<summary><strong>OpenCode workspaces</strong></summary>
<summary><strong>The box and workspaces</strong></summary>
Spoon's optional agent worker is designed to run outside Convex actions. The
worker claims queued jobs, clones the current GitHub fork, creates a branch,
starts an isolated workspace, and exposes workspace operations to the Next app
through server-only API proxies.
worker claims queued jobs, clones the current GitHub fork into the owner's
persistent home, creates a branch, and exposes workspace operations to the Next
app through server-only API proxies.
Everything runs inside **one long-running container per user**,
`spoon-box-{username}`. The box is a Fedora image (`docker/agent-job.Dockerfile`)
started with `sleep infinity` and a `--memory 4g` / `--cpus 2` cap. Each
thread's agent turn, terminal, and project commands `docker exec` into that same
box; the box's `/home/{username}` is a bind-mounted persistent home so dotfiles,
installed tools, shell history, and thread checkouts under `~/Code/{spoon}/{branch}`
survive across sessions. The box is reference-counted by the worker and reaped
after `SPOON_AGENT_BOX_IDLE_MS` idle (default 30m). There is no per-job
`docker run --rm` container and no separate `opencode serve` container — that
path was removed.
Workspace capabilities:
@@ -154,16 +190,18 @@ Workspace capabilities:
- inspect diffs
- send thread messages to the agent
- run configured commands
- use an interactive terminal (xterm.js → box PTY; see
[docs/agent-terminal.md](docs/agent-terminal.md))
- store logs and artifacts
- push a branch
- open a draft PR
The browser never receives worker tokens and never talks directly to the worker
or job container.
or the box.
Worker cleanup is available in `Settings -> Worker`. It can delete old terminal
workspace records and ask the active worker to remove orphaned job containers
and inactive work directories.
Worker cleanup is available in `Settings -> Worker`. It can delete stale
workspace records and ask the active worker to remove orphaned containers and
inactive work directories (persistent per-user homes are preserved).
Local worker development:
@@ -174,20 +212,53 @@ bun dev:next:worker
bun dev:next:worker:staging
```
Local host-run worker commands still load env through Infisical, then
`scripts/dev-agent-worker` selects Podman when available, falls back to Docker,
and publishes the OpenCode server on a localhost port so the host worker can
reach the job container. Override with:
Local host-run worker commands load env through Infisical, then
`scripts/dev-agent-worker` selects Podman when available and falls back to
Docker. Override the container CLI with:
```env
SPOON_AGENT_CONTAINER_RUNTIME=podman
SPOON_AGENT_CONTAINER_ACCESS=host_port
```
</details>
<details>
<summary><strong>Production agent runtime images</strong></summary>
<summary><strong>Agent runtimes</strong></summary>
Three agent CLIs run inside the box behind one `AgentRuntime` adapter interface
(`apps/agent-worker/src/runtime/*-adapter.ts`), selected by `job.runtime`:
| Runtime | CLI | Selected for |
| ---------- | ----------------------------------- | -------------------------------------------------- |
| `codex` | `@openai/codex` | OpenAI providers and Codex ChatGPT-login snapshots |
| `opencode` | `opencode-ai` | OpenAI-compatible API-key providers (default) |
| `claude` | `@anthropic-ai/claude-code@2.1.207` | Anthropic providers |
The runtime is validated at queue time (`runtimeSupport.ts` +
`agentJobs.insertJob`): a job is rejected unless the resolved runtime is one the
AI provider profile supports. `runtimesForProfile` maps profiles to runtimes —
ChatGPT-login snapshots → `codex` only; `openai` API keys → `opencode`/`codex`;
Anthropic API keys → `claude`/`opencode`; Anthropic `anthropic_oauth_json`
credential snapshots → `claude` only; every other OpenAI-compatible API key →
`opencode`.
Auth is materialized into the box before a turn:
- **Codex ChatGPT-login** profiles get the encrypted `auth.json` written to
`CODEX_HOME/.codex/auth.json`.
- **Anthropic API-key** profiles authenticate via `ANTHROPIC_API_KEY` in the
environment (no file needed).
- **Anthropic `anthropic_oauth_json`** credential-snapshot profiles get the
encrypted OAuth blob written to `~/.claude/.credentials.json`.
- **API-key** profiles run through OpenCode.
All materialized auth files are written `0600`. Treat those saved auth files
like a password and only use them on trusted workers.
</details>
<details>
<summary><strong>Production runtime images</strong></summary>
Gitea CI builds and pushes three production images:
@@ -197,32 +268,36 @@ git.gbrown.org/gib/spoon-agent-worker:latest
git.gbrown.org/gib/spoon-agent-job:latest
```
The worker image is the long-running service that polls Convex. The job image is
the isolated workbench that the worker launches for each agent job. For the MVP,
production should use the repo-provided JS/TS workbench image:
The worker image is the long-running service that polls Convex. The
`spoon-agent-job` image is the **box** image — the per-user Fedora dev box the
worker `exec`s into (it is not launched fresh per job). Point the worker at it
with:
```env
SPOON_AGENT_JOB_IMAGE="git.gbrown.org/gib/spoon-agent-job:latest"
```
The job image includes Node 22, Bun, pnpm and yarn through Corepack, npm, git,
ripgrep, Python, build tools, OpenCode, the Codex CLI, and the Claude Code CLI.
It is not the forked project's production runtime; it is the agent execution
environment.
The box image is Fedora 41 with Node, Bun, pnpm and yarn, npm, git, ripgrep,
Python, build tools, plus interactive tooling (neovim, tmux, fzf, fd, bat, eza,
zoxide, gh, gum, oh-my-posh) and the pinned agent CLIs OpenCode, Codex, and
Claude Code. It is not the forked project's production runtime; it is the agent
execution environment.
Production worker runtime requirements:
- `spoon-agent-worker` must run as a separate service.
- The worker needs `/var/run/docker.sock` mounted so it can launch job
containers.
- Production should keep `SPOON_AGENT_CONTAINER_RUNTIME=docker` and
`SPOON_AGENT_CONTAINER_ACCESS=network`.
- The production Docker host must be logged into `git.gbrown.org` so worker jobs
can pull the private `spoon-agent-job` image.
- The worker needs `/var/run/docker.sock` mounted so it can create and `exec`
into per-user boxes on the host daemon.
- Production should keep `SPOON_AGENT_CONTAINER_RUNTIME=docker`.
- `SPOON_AGENT_HOST_WORKDIR` must equal the absolute host path backing
`SPOON_AGENT_WORKDIR` (identical inside and outside the worker container) so
the host daemon can bind-mount per-user homes under `${WORKDIR}/homes/{username}`.
- The production Docker host must be logged into `git.gbrown.org` so the worker
can pull the private `spoon-agent-job` box image.
- `SPOON_WORKER_TOKEN` must match the value stored in Convex production env.
- `spoon-next` needs `SPOON_AGENT_WORKER_URL=http://spoon-agent-worker:3921` and
`SPOON_AGENT_WORKER_INTERNAL_TOKEN` so Next API routes can proxy workspace
file, diff, message, command, and draft PR actions.
`SPOON_AGENT_WORKER_INTERNAL_TOKEN` so Next API routes can proxy workspace and
box file, diff, message, command, terminal, and draft PR actions.
- `spoon-agent-worker` also needs `GITHUB_APP_ID` and `GITHUB_APP_PRIVATE_KEY`.
If the private key is stored in a single-line dotenv value, encode newlines as
literal `\n` characters so the worker can restore the PEM before using it.
@@ -241,26 +316,57 @@ curl -H "Authorization: Bearer $SPOON_AGENT_WORKER_INTERNAL_TOKEN" \
Deployment readiness checklist:
1. Production Convex env has `SPOON_WORKER_TOKEN`, `SPOON_ENCRYPTION_KEY`,
GitHub App env, and Convex Auth signing keys.
GitHub App env (including `GITHUB_APP_WEBHOOK_SECRET`), and Convex Auth
signing keys.
2. Compose env has `SPOON_AGENT_WORKER_URL`,
`SPOON_AGENT_WORKER_INTERNAL_TOKEN`, `SPOON_AGENT_JOB_IMAGE`, and the GitHub
App private key.
App private key; the `spoon-next` image is built with
`NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL`.
3. The production Docker host can pull private images from `git.gbrown.org`.
4. `Settings -> Worker` reports the expected job image, runtime, network, and
active workspace count.
4. `Settings -> Worker` reports the expected box image, runtime, network, and
active box count.
5. The first test thread uses a configured API-key provider or a trusted Codex
login profile.
6. If a worker restart leaves stale workspace state, use the workspace recovery
panel or `Settings -> Worker` cleanup.
6. If a worker restart leaves stale state, use the workspace recovery panel or
`Settings -> Worker` cleanup.
API-key based AI provider profiles run through OpenCode. Codex ChatGPT login
profiles run through the Codex CLI: Spoon writes the encrypted `auth.json` into
the isolated job workspace as `CODEX_HOME/.codex/auth.json` before execution.
Anthropic profiles run through the Claude Code CLI: API-key profiles authenticate
via `ANTHROPIC_API_KEY`, while `anthropic_oauth_json` credential-snapshot profiles
have their encrypted OAuth blob written into the isolated job workspace as
`~/.claude/.credentials.json` before execution. Treat those saved auth files like
a password and only use them on trusted workers.
</details>
<details>
<summary><strong>GitHub webhooks</strong></summary>
Spoon exposes a signature-verified webhook as a Convex `httpAction` at
`POST /webhooks/github` (`packages/backend/convex/githubWebhooks.ts`,
`http.ts`). Configure the GitHub App webhook URL as
`<CONVEX_SITE_URL>/webhooks/github` and set `GITHUB_APP_WEBHOOK_SECRET`; the
endpoint fails closed (HTTP 503) when no secret is configured and rejects any
payload whose `x-hub-signature-256` does not verify.
- `push` → a targeted drift refresh for every Spoon tracking that repository.
- `installation` / `installation_repositories` → the matching connection's
status flips to `revoked` (deleted/suspended), `active` (created/unsuspended),
or otherwise `needs_reauth`, surfaced in `Settings -> Integrations`.
The hourly `refreshDueSpoons` cron remains a fallback, so drift still refreshes
even if a webhook is missed.
</details>
<details>
<summary><strong>Notifications</strong></summary>
Spoon delivers an in-app bell plus preference-gated transactional email
(`packages/backend/convex/notifications.ts`). Five kinds are emitted:
`maintenance_thread`, `agent_turn_finished`, `agent_needs_input`, `sync_failed`,
and `connection_needs_reauth`.
- Every event inserts an in-app notification (bell badge via `unreadCount`,
list via `listMine`, `markRead` / `markAllRead`).
- Email is sent through UseSend (`USESEND_*`) only when the user has an email
and the matching per-kind preference is not disabled. Unset preferences
default to enabled.
- Per-kind email toggles live in `Settings -> Notifications`. Web push is out of
scope.
</details>
@@ -289,45 +395,54 @@ a password and only use them on trusted workers.
<details>
<summary><strong>Core tables</strong></summary>
| Table | Purpose |
| ------------------------ | --------------------------------------------------------- |
| `spoons` | Managed fork records |
| `threads` | Durable maintenance and work conversations |
| `threadMessages` | Messages inside threads |
| `syncRuns` | Upstream checks, sync attempts, and maintenance decisions |
| `ignoredUpstreamChanges` | Intentional ignore records that affect effective drift |
| `gitConnections` | Git provider connection metadata |
| `spoonRepositoryStates` | Latest cached upstream/fork state |
| `spoonCommits` | Cached upstream and fork-only commits |
| `spoonPullRequests` | Cached fork/upstream pull requests |
| `spoonSecrets` | Encrypted per-Spoon environment variables |
| `spoonAgentSettings` | Per-Spoon runtime, branch, command, and env-file settings |
| `aiProviderProfiles` | Encrypted provider/auth profiles used by OpenCode |
| `agentJobs` | Worker-executed workspace jobs and PR lifecycle |
| `agentJobEvents` | Append-only worker event log |
| `agentJobArtifacts` | Diffs, summaries, command output, PR body drafts |
| `agentWorkspaceChanges` | Recorded user, agent, and command file changes |
| Table | Purpose |
| ------------------------- | ----------------------------------------------------------- |
| `spoons` | Managed fork records |
| `threads` | Durable maintenance and work conversations |
| `threadMessages` | Messages inside threads |
| `syncRuns` | Upstream checks, sync attempts, and maintenance decisions |
| `ignoredUpstreamChanges` | Intentional ignore records that affect effective drift |
| `gitConnections` | Git provider connection metadata |
| `spoonRepositoryStates` | Latest cached upstream/fork state |
| `spoonCommits` | Cached upstream and fork-only commits |
| `spoonPullRequests` | Cached fork/upstream pull requests |
| `spoonSecrets` | Encrypted per-Spoon environment variables |
| `spoonAgentSettings` | Per-Spoon runtime, branch, command, and env-file settings |
| `aiProviderProfiles` | Encrypted provider/auth profiles (Codex/OpenCode/Anthropic) |
| `agentJobs` | Worker-executed workspace jobs and PR lifecycle |
| `agentJobEvents` | Append-only worker event log |
| `agentJobArtifacts` | Diffs, summaries, command output, PR body drafts |
| `agentWorkspaceChanges` | Recorded user, agent, and command file changes |
| `userDotfiles` | Encrypted per-user dotfiles overlay + repo/setup config |
| `userEnvironment` | Per-user box/home environment config |
| `notifications` | In-app notification rows (bell) |
| `notificationPreferences` | Per-user email notification toggles |
</details>
<details>
<summary><strong>Important routes</strong></summary>
| Route | Purpose |
| --------------------------------- | --------------------------------------- |
| `/` | Public product landing page |
| `/dashboard` | Maintenance overview |
| `/spoons` | Managed fork list |
| `/spoons/new` | Manual/GitHub Spoon creation |
| `/spoons/[spoonId]` | Spoon detail dashboard |
| `/spoons/[spoonId]/agent/[jobId]` | Interactive workspace |
| `/threads` | Global thread queue |
| `/threads/[threadId]` | Thread detail |
| `/settings/profile` | User profile settings |
| `/settings/integrations` | GitHub and service integration settings |
| `/settings/ai-providers` | AI/OpenCode provider profiles |
| Route | Purpose |
| --------------------------------- | ----------------------------------------------- |
| `/` | Public product landing page |
| `/dashboard` | Maintenance overview |
| `/spoons` | Managed fork list |
| `/spoons/new` | Manual/GitHub Spoon creation |
| `/spoons/[spoonId]` | Spoon detail dashboard |
| `/spoons/[spoonId]/agent/[jobId]` | Interactive workspace |
| `/threads` | Global thread queue |
| `/threads/[threadId]` | Thread detail |
| `/machine` | My Machine: box status, terminal, home |
| `/settings/profile` | User profile settings |
| `/settings/integrations` | GitHub and service integration settings |
| `/settings/ai-providers` | AI provider profiles (Codex/OpenCode/Anthropic) |
| `/settings/dotfiles` | Per-user dotfiles overlay + repo |
| `/settings/notifications` | Email notification preferences |
Legacy `/updates` and `/agents` routes redirect into `/threads`.
The Convex `httpAction` `POST /webhooks/github` (mounted at
`<CONVEX_SITE_URL>/webhooks/github`) handles GitHub App webhooks. Legacy
`/updates` and `/agents` routes redirect into `/threads`.
</details>
@@ -428,16 +543,17 @@ not call Infisical.
<details open>
<summary><strong>Public Next variables</strong></summary>
| Variable | Used for |
| --------------------------------- | ------------------------------------------- |
| `NEXT_PUBLIC_SITE_URL` | Canonical Spoon web URL |
| `NEXT_PUBLIC_CONVEX_URL` | Convex client URL |
| `NEXT_PUBLIC_DEPLOYMENT_URL` | Convex dashboard/deployment URL when needed |
| `NEXT_PUBLIC_PLAUSIBLE_URL` | Plausible analytics endpoint |
| `NEXT_PUBLIC_SENTRY_DSN` | Browser Sentry DSN |
| `NEXT_PUBLIC_SENTRY_URL` | Sentry instance URL |
| `NEXT_PUBLIC_SENTRY_ORG` | Sentry organization |
| `NEXT_PUBLIC_SENTRY_PROJECT_NAME` | Sentry project name |
| Variable | Used for |
| --------------------------------------- | ------------------------------------------------------------------- |
| `NEXT_PUBLIC_SITE_URL` | Canonical Spoon web URL |
| `NEXT_PUBLIC_CONVEX_URL` | Convex client URL |
| `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL` | Browser-facing worker WS base for the terminal (**build-time** var) |
| `NEXT_PUBLIC_DEPLOYMENT_URL` | Convex dashboard/deployment URL when needed |
| `NEXT_PUBLIC_PLAUSIBLE_URL` | Plausible analytics endpoint |
| `NEXT_PUBLIC_SENTRY_DSN` | Browser Sentry DSN |
| `NEXT_PUBLIC_SENTRY_URL` | Sentry instance URL |
| `NEXT_PUBLIC_SENTRY_ORG` | Sentry organization |
| `NEXT_PUBLIC_SENTRY_PROJECT_NAME` | Sentry project name |
</details>
@@ -492,15 +608,16 @@ not call Infisical.
| `SPOON_AGENT_WORKER_URL` | Internal worker HTTP URL used by Next |
| `SPOON_AGENT_WORKER_HTTP_PORT` | Worker HTTP port |
| `SPOON_AGENT_WORKER_INTERNAL_TOKEN` | Server-only token for Next-to-worker proxy |
| `SPOON_AGENT_JOB_IMAGE` | Agent job container image |
| `SPOON_AGENT_TERMINAL_SECRET` | HMAC secret for terminal/box tokens (falls back to the internal/worker token) |
| `SPOON_AGENT_JOB_IMAGE` | Per-user box (Fedora) image the worker `exec`s into |
| `SPOON_AGENT_RUNTIME` | Runtime mode, currently Docker/Podman-oriented |
| `SPOON_AGENT_CONTAINER_RUNTIME` | Container CLI used by worker, `docker`/`podman` |
| `SPOON_AGENT_CONTAINER_ACCESS` | `network` in prod, `host_port` for host dev |
| `SPOON_AGENT_BOX_IDLE_MS` | Idle time before a per-user box is reaped (default `1800000`) |
| `SPOON_AGENT_MAX_CONCURRENT_JOBS` | Worker concurrency limit |
| `SPOON_AGENT_JOB_TIMEOUT_MS` | Job timeout |
| `SPOON_AGENT_WORKDIR` | Worker work directory |
| `SPOON_AGENT_JOB_TIMEOUT_MS` | Agent turn timeout |
| `SPOON_AGENT_WORKDIR` | Worker work directory; per-user homes live under `homes/{username}` |
| `SPOON_AGENT_HOST_WORKDIR` | Host path matching `SPOON_AGENT_WORKDIR` when the worker runs in Docker and controls the host Docker socket |
| `SPOON_AGENT_NETWORK` | Optional job container network |
| `SPOON_AGENT_NETWORK` | Optional box container network |
</details>
@@ -528,12 +645,20 @@ not call Infisical.
- GitHub drift refresh, commit cache, PR cache, and sync-run history
- Effective drift and ignored upstream change records
- Global Threads page and Spoon-scoped Threads tab
- OpenCode/Codex-oriented agent worker and browser workspace foundation
- Persistent per-user box the worker `exec`s into for agent turns, terminal, and
commands
- Three agent runtimes (Codex, OpenCode, Claude Code) behind one adapter, with
queue-time runtime validation
- My Machine surface (box status/start/stop/restart, terminal, home browser)
- Interactive workspace terminal (xterm.js → box PTY)
- Per-user dotfiles overlay + optional dotfiles repo
- Signature-verified GitHub webhook with hourly cron fallback
- In-app + email notifications with per-user preferences
- Monaco editor with optional Vim mode
- Diff viewer, command panel, worker logs, and artifacts
- Encrypted Spoon secrets and bulk `.env` import
- Encrypted AI provider profiles, including Codex auth JSON and API-key
provider support
- Encrypted AI provider profiles, including Codex/Anthropic auth JSON and
API-key provider support
- Authentik, GitHub, and password auth through Convex Auth
- Self-hosted Convex/Postgres deployment model
docs/agent-terminal.md
+44 -30
View File
@@ -1,34 +1,44 @@
# Workspace interactive terminal
# Interactive terminal
A real shell inside the agent workspace, shown as the **Terminal** tab in the
workspace UI. It's an xterm.js front end bridged to a bash/tmux PTY running in a
persistent per-job container (the agent job image), mounting the same workspace
the editor and agent use.
A real shell inside the user's persistent box (`spoon-box-{username}`). There are
two entry points, both bridging to the **same** box:
- The workspace **Terminal** tab — a shell opened at the thread's checkout
(`~/Code/{spoon}/{branch}`).
- The **My Machine** page (`/machine`) — a `~`-rooted shell into the same box.
Both are an xterm.js front end bridged to a bash/tmux PTY via `docker exec` into
the box. There is no per-job `spoon-agent-term-*` container anymore — the box is
long-running and shared by the agent, the editor, and both terminals.
## Architecture
```
browser (xterm.js)
│ 1. GET /api/agent-jobs/:id/terminal-token (Convex-auth'd, owner only)
│ → { url: "wss://worker…/jobs/:id/terminal?token=…", expiresAt }
│
│ 2. WebSocket wss://worker.<domain>/jobs/:id/terminal?token=…
│ 1a. GET /api/agent-jobs/:id/terminal-token (Convex-auth'd, owner only)
│ → { url: "wss://worker…/jobs/:id/terminal?token=…", expiresAt }
│ 1b. GET /api/box/terminal-token (Convex-auth'd; username
│ → { url: "wss://worker…/box/terminal?token=…", expiresAt } derived
│ server-side)
│ 2. WebSocket wss://worker.<domain>/{jobs/:id|box}/terminal?token=…
▼
nginx ── upgrade ──► spoon-agent-worker :3921
│ verifyTerminalToken(token, jobId, secret)
│ dockerode exec -t → bash/tmux PTY
│ verify HMAC token (job-scoped or username-scoped)
│ docker exec -it → bash/tmux PTY
▼
spoon-agent-term-<jobId> (job image, mounts the workspace)
spoon-box-{username} (Fedora box; persistent home mounted)
```
- The browser **never** holds the worker secret. The Next app (which has already
verified job ownership) mints a short-lived HMAC token; the worker verifies it.
- The browser **never** holds the worker secret. The Next app mints a short-lived
HMAC token; the worker verifies it. The job terminal token is minted only after
a Convex ownership check; the box terminal token carries the username derived
**server-side** from the authed user (never from the request).
- Frames: **binary** = stdin/stdout bytes; **text JSON** `{type:"resize",cols,rows}`
= resize. The token's 2-minute expiry is a _connect_ window; an established
session persists.
- The shell runs `tmux new-session -A -s spoon` (falls back to `bash -l`), so
reconnecting reattaches the same session. Idle containers are removed after
`SPOON_AGENT_TERMINAL_IDLE_MS` (default 30m).
- The shell prefers `tmux new-session -A -s spoon` (falls back to `bash -l`), so
reconnecting reattaches the same session. The box itself is idle-reaped after
`SPOON_AGENT_BOX_IDLE_MS` (default 30m) once no job or terminal holds it.
## Configuration
@@ -37,8 +47,7 @@ nginx ── upgrade ──► spoon-agent-worker :3921
| Next app | `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL` | **Yes** | Browser-facing worker WS base, e.g. `wss://worker.spoon.gbrown.org` (prod) or `ws://localhost:3921` (dev). **Build-time** (`NEXT_PUBLIC`): for the Docker image it must be passed as a build arg (wired in `docker/Dockerfile` + `docker/compose.yml`, sourced from the build env file), not a runtime env. Unset → the Terminal tab shows "not configured". |
| Next app | `SPOON_AGENT_TERMINAL_SECRET` | No | HMAC secret for signing tokens. Falls back to `SPOON_AGENT_WORKER_INTERNAL_TOKEN`. |
| Worker | `SPOON_AGENT_TERMINAL_SECRET` | No | Must match the Next app's. Falls back to `SPOON_AGENT_WORKER_INTERNAL_TOKEN` (already shared), so by default **no new secret is needed**. |
| Worker | `SPOON_AGENT_TERMINAL_IMAGE` | No | Shell container image. Defaults to `SPOON_AGENT_JOB_IMAGE`. |
| Worker | `SPOON_AGENT_TERMINAL_IDLE_MS` | No | Idle-container reap delay (default `1800000`). |
| Worker | `SPOON_AGENT_BOX_IDLE_MS` | No | Idle-box reap delay, shared with agent jobs (default `1800000`). The terminal holds the box open while connected. |
Because the secret defaults to the already-shared worker token, the **only**
required step is setting `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL` and exposing the
@@ -70,9 +79,10 @@ server {
Then set on the Next app: `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL=wss://worker.spoon.gbrown.org`.
> The worker's HTTP routes (`/jobs/:id/tree` etc.) require the internal bearer
> token, so exposing the worker host only usefully exposes the token-gated
> `/jobs/:id/terminal` upgrade. Still, restrict the server block to TLS.
> The worker's HTTP routes (`/jobs/:id/tree`, `/box/*`, etc.) require the
> internal bearer token, so exposing the worker host only usefully exposes the
> token-gated `/jobs/:id/terminal` and `/box/terminal` upgrades. Still, restrict
> the server block to TLS.
## Dev testing (no nginx)
@@ -90,15 +100,19 @@ Docker mode. Prod (Docker socket mounted) works as-is.
## Security
- Owner-only: the token route uses Convex auth + `assertOwned`.
- Tokens are short-lived (2m connect window), job-scoped, HMAC-signed.
- A shell in the workspace can reach the network and the repo's git credentials.
This is intended for the single-user self-hosted deployment; do not expose the
- Owner-only: the job token route uses Convex auth + ownership assertion; the
box token derives the username server-side from the authed user.
- Tokens are short-lived (2m connect window), scoped (job-scoped or
username-scoped), and HMAC-signed.
- A shell in the box can reach the network and the repo's git credentials. This
is intended for the single-user self-hosted deployment; do not expose the
worker domain without TLS, and keep the deployment single-tenant.
## Tools in the shell
The job image ships `bash`, `tmux`, `neovim`, `git`, `ripgrep`, `jq`, `python3`,
`node`, `bun`, `pnpm`, `yarn`, `curl`/`wget`, `unzip`. Bring your own dotfiles by
cloning them in-session (e.g. `git clone <dotfiles> ~/.config/...`); persistent
auto-cloning of a dotfiles repo is a planned follow-up.
The box image ships `bash`, `tmux`, `neovim`, `git`, `ripgrep`, `jq`, `python3`,
`node`, `bun`, `pnpm`, `yarn`, `curl`/`wget`, `unzip`, plus QoL tooling (`fzf`,
`fd`, `bat`, `eza`, `zoxide`, `gh`, `gum`, `oh-my-posh`) and the agent CLIs
(`codex`, `opencode`, `claude`). Personalize the shell in **Settings → Dotfiles**
(overlay files + an optional dotfiles repo), applied to the persistent home; see
[dotfiles.md](dotfiles.md).
docs/compose.prod.yml
+13 -13
View File
@@ -7,18 +7,18 @@
# If you change them, read the comments first:
#
# 1. AGENT WORKDIR (spoon-agent-worker): the worker is containerized but
# launches the Codex job container by talking to the HOST Docker daemon.
# The host can only bind-mount real HOST paths, so the work directory MUST
# be a bind mount whose path is IDENTICAL inside and outside the container,
# and SPOON_AGENT_HOST_WORKDIR must match it. A named volume does NOT work
# here because its real host path is hidden from the worker. All three
# references to /var/lib/spoon-agent/work below must stay in sync; change
# them together if you want the data somewhere else.
# creates and `exec`s into the per-user box (spoon-box-{username}) by talking
# to the HOST Docker daemon. The host can only bind-mount real HOST paths, so
# the work directory MUST be a bind mount whose path is IDENTICAL inside and
# outside the container, and SPOON_AGENT_HOST_WORKDIR must match it. A named
# volume does NOT work here because its real host path is hidden from the
# worker. All three references to /var/lib/spoon-agent/work below must stay
# in sync; change them together if you want the data somewhere else.
#
# 2. IMAGE FRESHNESS: services use `pull_policy: always` + Watchtower labels so
# a redeploy / new push always lands. The Codex *job* image is pulled by the
# worker itself on startup (see SPOON_AGENT_JOB_IMAGE); restarting the worker
# (which Watchtower does on a new image) re-pulls a fresh job image.
# a redeploy / new push always lands. The *box* image is pulled by the worker
# itself on startup (see SPOON_AGENT_JOB_IMAGE); restarting the worker (which
# Watchtower does on a new image) re-pulls a fresh box image.
networks:
nginx-bridge: # Change to network you plan to use
@@ -83,9 +83,9 @@ services:
- SPOON_WORKER_TOKEN=${SPOON_WORKER_TOKEN}
volumes:
- /var/run/docker.sock:/var/run/docker.sock
# Identical host:container path so the sibling Codex job containers can
# bind-mount the workspace via the host daemon. Do NOT switch this to a
# named volume. See header note (1).
# Identical host:container path so the sibling per-user box can bind-mount
# the persistent home (homes/{username}) via the host daemon. Do NOT switch
# this to a named volume. See header note (1).
- /var/lib/spoon-agent/work:/var/lib/spoon-agent/work
labels: ['com.centurylinklabs.watchtower.enable=true']
tty: true
docs/dotfiles.md
+14 -10
View File
@@ -6,10 +6,11 @@ preloaded with QoL CLI tooling, a persistent per-user home, and user dotfiles.
## The model
- **Persistent per-user home.** Each user gets a home directory on the worker
host at `${SPOON_AGENT_WORKDIR}/homes/{username}`, bind-mounted into every
job/terminal container at `/home/{username}` (`HOME`). It survives across
sessions, so dotfiles, installed tools, nvim plugins, and shell history persist.
`username` is derived from the user's profile first name (sanitized).
host at `${SPOON_AGENT_WORKDIR}/homes/{username}`, bind-mounted into the
per-user box (`spoon-box-{username}`) at `/home/{username}` (`HOME`). It
survives across sessions, so dotfiles, installed tools, nvim plugins, and shell
history persist. `username` is derived from the user's profile first name
(sanitized).
- **Threads as folders.** Each thread's checkout lives at
`~/Code/{spoon}/{branch}` inside that home, so every thread shows up as a
folder in one home. The agent (`codex --cd …`) and the terminal both open there.
@@ -61,18 +62,21 @@ Secrets: dotfiles are encrypted, but real API keys/tokens belong in a Spoon's
No new required env. The home is a host directory under the existing workdir, so
the prod bind-mount + `SPOON_AGENT_HOST_WORKDIR` translation already covers it.
## Notes / limits (Phase 1)
## Notes / limits
- **Repo auth:** public repos only. Private/self-hosted (e.g. Gitea) dotfiles
repos are a follow-up (store a token/deploy key).
- **Binary files:** the overlay is text-first.
- **Cleanup:** `~/Code/{spoon}/{branch}` checkouts persist (threads as folders);
a per-thread "delete checkout" action is a follow-up.
- **Concurrency:** jobs share one home; fine at the default
- **Concurrency:** jobs and terminals share one box/home; fine at the default
`SPOON_AGENT_MAX_CONCURRENT_JOBS=1`.
## Phase 2 north star
## The box
A single long-running per-user container that every thread `exec`s into (agent
via `docker exec`, not `docker run --rm`). The per-user home + `~/Code/{spoon}/
{branch}` layout built here is its foundation.
The per-user home + `~/Code/{spoon}/{branch}` layout is the foundation of the
box model: a single long-running per-user container (`spoon-box-{username}`) that
every thread's agent turn, terminal, and command `exec`s into (`docker exec`, not
`docker run --rm`). Home materialization runs once per box start (and again when
the dotfiles config hash changes), so the overlay and repo setup apply to every
session in that box.
docs/server-deploy-changes.md
+36 -13
View File
@@ -1,7 +1,7 @@
# Server deploy changes (terminal + dotfiles + Fedora + Phase 2)
# Server deploy changes (box + terminal + dotfiles + webhooks + notifications)
Everything the production host / compose / `.env` needs for the workspace
terminal, personalized dev environment, Nerd Font, and the per-user container.
Everything the production host / compose / `.env` needs for the per-user box, the
terminal, the personalized dev environment, GitHub webhooks, and notifications.
Most items have safe defaults; the **Required** ones are the only must-dos.
## Required
@@ -44,9 +44,10 @@ Most items have safe defaults; the **Required** ones are the only must-dos.
restart picks up the new Fedora job image. Make sure the prod registry has the
new `spoon-agent-job:latest`.
4. **Deploy Convex functions** (new tables `userDotfiles`, `userEnvironment`).
`SPOON_ENCRYPTION_KEY` (or `INSTANCE_SECRET`) is already required and is what
encrypts dotfiles at rest — no change, just confirm it's set.
4. **Deploy Convex functions** (new tables `userDotfiles`, `userEnvironment`,
`notifications`, `notificationPreferences`). `SPOON_ENCRYPTION_KEY` (or
`INSTANCE_SECRET`) is already required and is what encrypts dotfiles at rest —
no change, just confirm it's set.
5. **Confirm `SPOON_AGENT_HOST_WORKDIR`** on the `spoon-agent-worker` service is
the absolute host path backing `SPOON_AGENT_WORKDIR` (the fix from the terminal
@@ -54,17 +55,36 @@ Most items have safe defaults; the **Required** ones are the only must-dos.
and are bind-mounted into the box via the host daemon — this only resolves if
the host-workdir translation is correct. (No new var; just verify.)
6. **Configure the GitHub App webhook.** Set the App's webhook URL to
`<CONVEX_SITE_URL>/webhooks/github` and set `GITHUB_APP_WEBHOOK_SECRET` in the
**Convex** production env (the signature-verified `httpAction` runs in Convex,
not the worker). Without the secret the endpoint fails closed (HTTP 503) and
drift only refreshes via the hourly cron. `push` refreshes affected Spoons;
`installation` / `installation_repositories` update connection status.
7. **Notifications email (optional).** In-app notifications work with no config.
To also send email, set `USESEND_API_KEY`, `USESEND_URL`, and
`USESEND_FROM_EMAIL` in the Convex env. Per-user, per-kind email toggles live
in `Settings → Notifications`; unset preferences default to on.
8. **Claude Code runtime.** The box image pins `@anthropic-ai/claude-code`, so
Anthropic provider profiles (API key or `anthropic_oauth_json` credential
snapshot) can drive the `claude` runtime with no extra host env. Anthropic
API-key auth flows through `ANTHROPIC_API_KEY`; OAuth snapshots are written to
`~/.claude/.credentials.json` in the box.
## Optional (safe defaults — only set to override)
On the `spoon-agent-worker` service:
| Var | Default | Purpose |
| ------------------------------ | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `SPOON_AGENT_TERMINAL_SECRET` | falls back to `SPOON_AGENT_WORKER_INTERNAL_TOKEN` | HMAC secret for terminal tokens (must match the Next app's, which also falls back). Leave unset to use the shared token. |
| `SPOON_AGENT_BOX_IDLE_MS` | `1800000` (30m) | How long a per-user box survives idle before being reaped. |
| `SPOON_AGENT_TERMINAL_IDLE_MS` | `1800000` | (Legacy; box idle now governs cleanup.) |
| Var | Default | Purpose |
| ----------------------------- | ------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `SPOON_AGENT_TERMINAL_SECRET` | falls back to `SPOON_AGENT_WORKER_INTERNAL_TOKEN` | HMAC secret for terminal/box tokens (must match the Next app's, which also falls back). Leave unset to use the shared token. |
| `SPOON_AGENT_BOX_IDLE_MS` | `1800000` (30m) | How long a per-user box survives idle before being reaped. |
No new env is needed for dotfiles, the per-user home, or the Nerd Font.
`SPOON_AGENT_TERMINAL_IMAGE` and `SPOON_AGENT_TERMINAL_IDLE_MS` were removed —
the box uses `SPOON_AGENT_JOB_IMAGE` and `SPOON_AGENT_BOX_IDLE_MS`.
## Notes / one-time cleanup
@@ -84,7 +104,10 @@ No new env is needed for dotfiles, the per-user home, or the Nerd Font.
```bash
docker exec spoon-agent-worker docker --version # CLI present (29.x)
docker run --rm git.gbrown.org/gib/spoon-agent-job:latest codex --version # 0.142
docker run --rm git.gbrown.org/gib/spoon-agent-job:latest codex --version # 0.142
docker run --rm git.gbrown.org/gib/spoon-agent-job:latest claude --version # 2.1.207
docker run --rm git.gbrown.org/gib/spoon-agent-job:latest opencode --version
docker run --rm git.gbrown.org/gib/spoon-agent-job:latest bash -lc 'eza --version; zoxide --version; oh-my-posh --version'
# then: open a thread → Terminal tab; Settings → Dotfiles add a .bashrc alias.
# then: /machine → box status + terminal; open a thread → Terminal tab;
# Settings → Dotfiles add a .bashrc alias; Settings → Notifications toggles.
```