gib/spoon
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: rewrite for box-first model + webhooks + notifications

Browse Source
This commit is contained in:
Gabriel Brown
2026-07-11 18:23:49 -04:00
parent 1fbb2dd0a1
commit a255a06a35
5 changed files with 332 additions and 166 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+225 -100
View File
@@ -10,7 +10,8 @@
<p align="center"> <p align="center">
Spoon is a self-hostable fork maintenance cockpit built around managed forks, Spoon is a self-hostable fork maintenance cockpit built around managed forks,
durable maintenance threads, and OpenCode-powered workspaces. durable maintenance threads, and a persistent per-user dev box that agents,
terminals, and project commands all run inside.
</p> </p>
<p align="center"> <p align="center">
@@ -61,14 +62,38 @@ in the data model, but GitHub is the active automation surface today.
Spoon keeps raw GitHub drift visible while also tracking ignored upstream Spoon keeps raw GitHub drift visible while also tracking ignored upstream
changes so irrelevant commits do not keep a fork permanently actionable. changes so irrelevant commits do not keep a fork permanently actionable.
- **OpenCode workspaces** - **One persistent per-user box**
Agent work happens in an isolated workspace with a file tree, browser editor, Every user owns a single long-running Fedora container `spoon-box-{username}`
diff viewer, command panel, logs, artifacts, and draft PR actions. with a persistent home. Every thread's agent turn, terminal session, and
project command `docker exec`s into that same box — there is no per-job
throwaway container.
- **Three agent runtimes**
Codex, OpenCode, and Claude Code all run inside the box behind one adapter
interface, selected per thread and validated at queue time against the AI
provider profile.
- **Workspaces**
Agent work happens in a workspace with a file tree, browser editor, diff
viewer, command panel, logs, artifacts, an interactive terminal, and draft PR
actions.
- **My Machine**
A `/machine` surface to start/stop/restart the box, open a `~`-rooted
terminal, and browse the persistent home.
- **GitHub webhooks**
A signature-verified webhook keeps drift fresh on `push` and flips connection
status on `installation` changes; the hourly cron is a fallback.
- **Notifications**
An in-app bell plus preference-gated transactional email for maintenance
threads, agent turns, needed input, sync failures, and connection re-auth.
- **User-owned providers and secrets** - **User-owned providers and secrets**
AI provider profiles, Codex/OpenCode auth, and per-Spoon project secrets are AI provider profiles, Codex/OpenCode/Anthropic auth, and per-Spoon project
encrypted. Secrets are redacted from logs and refused from commits when secrets are encrypted. Secrets are written `0600`, redacted from logs, and
materialized into env files. refused from commits when materialized into env files.
- **Draft PR handoff** - **Draft PR handoff**
Code changes become branches and draft pull requests. Spoon does not Code changes become branches and draft pull requests. Spoon does not
@@ -138,12 +163,23 @@ custom work exists.
</details> </details>
<details> <details>
<summary><strong>OpenCode workspaces</strong></summary> <summary><strong>The box and workspaces</strong></summary>
Spoon's optional agent worker is designed to run outside Convex actions. The Spoon's optional agent worker is designed to run outside Convex actions. The
worker claims queued jobs, clones the current GitHub fork, creates a branch, worker claims queued jobs, clones the current GitHub fork into the owner's
starts an isolated workspace, and exposes workspace operations to the Next app persistent home, creates a branch, and exposes workspace operations to the Next
through server-only API proxies. app through server-only API proxies.
Everything runs inside **one long-running container per user**,
`spoon-box-{username}`. The box is a Fedora image (`docker/agent-job.Dockerfile`)
started with `sleep infinity` and a `--memory 4g` / `--cpus 2` cap. Each
thread's agent turn, terminal, and project commands `docker exec` into that same
box; the box's `/home/{username}` is a bind-mounted persistent home so dotfiles,
installed tools, shell history, and thread checkouts under `~/Code/{spoon}/{branch}`
survive across sessions. The box is reference-counted by the worker and reaped
after `SPOON_AGENT_BOX_IDLE_MS` idle (default 30m). There is no per-job
`docker run --rm` container and no separate `opencode serve` container — that
path was removed.
Workspace capabilities: Workspace capabilities:
@@ -154,16 +190,18 @@ Workspace capabilities:
- inspect diffs - inspect diffs
- send thread messages to the agent - send thread messages to the agent
- run configured commands - run configured commands
- use an interactive terminal (xterm.js → box PTY; see
[docs/agent-terminal.md](docs/agent-terminal.md))
- store logs and artifacts - store logs and artifacts
- push a branch - push a branch
- open a draft PR - open a draft PR
The browser never receives worker tokens and never talks directly to the worker The browser never receives worker tokens and never talks directly to the worker
or job container. or the box.
Worker cleanup is available in `Settings -> Worker`. It can delete old terminal Worker cleanup is available in `Settings -> Worker`. It can delete stale
workspace records and ask the active worker to remove orphaned job containers workspace records and ask the active worker to remove orphaned containers and
and inactive work directories. inactive work directories (persistent per-user homes are preserved).
Local worker development: Local worker development:
@@ -174,20 +212,53 @@ bun dev:next:worker
bun dev:next:worker:staging bun dev:next:worker:staging
``` ```
Local host-run worker commands still load env through Infisical, then Local host-run worker commands load env through Infisical, then
`scripts/dev-agent-worker` selects Podman when available, falls back to Docker, `scripts/dev-agent-worker` selects Podman when available and falls back to
and publishes the OpenCode server on a localhost port so the host worker can Docker. Override the container CLI with:
reach the job container. Override with:
```env ```env
SPOON_AGENT_CONTAINER_RUNTIME=podman SPOON_AGENT_CONTAINER_RUNTIME=podman
SPOON_AGENT_CONTAINER_ACCESS=host_port
``` ```
</details> </details>
<details> <details>
<summary><strong>Production agent runtime images</strong></summary> <summary><strong>Agent runtimes</strong></summary>
Three agent CLIs run inside the box behind one `AgentRuntime` adapter interface
(`apps/agent-worker/src/runtime/*-adapter.ts`), selected by `job.runtime`:
| Runtime | CLI | Selected for |
| ---------- | ----------------------------------- | -------------------------------------------------- |
| `codex` | `@openai/codex` | OpenAI providers and Codex ChatGPT-login snapshots |
| `opencode` | `opencode-ai` | OpenAI-compatible API-key providers (default) |
| `claude` | `@anthropic-ai/claude-code@2.1.207` | Anthropic providers |
The runtime is validated at queue time (`runtimeSupport.ts` +
`agentJobs.insertJob`): a job is rejected unless the resolved runtime is one the
AI provider profile supports. `runtimesForProfile` maps profiles to runtimes —
ChatGPT-login snapshots → `codex` only; `openai` API keys → `opencode`/`codex`;
Anthropic API keys → `claude`/`opencode`; Anthropic `anthropic_oauth_json`
credential snapshots → `claude` only; every other OpenAI-compatible API key →
`opencode`.
Auth is materialized into the box before a turn:
- **Codex ChatGPT-login** profiles get the encrypted `auth.json` written to
`CODEX_HOME/.codex/auth.json`.
- **Anthropic API-key** profiles authenticate via `ANTHROPIC_API_KEY` in the
environment (no file needed).
- **Anthropic `anthropic_oauth_json`** credential-snapshot profiles get the
encrypted OAuth blob written to `~/.claude/.credentials.json`.
- **API-key** profiles run through OpenCode.
All materialized auth files are written `0600`. Treat those saved auth files
like a password and only use them on trusted workers.
</details>
<details>
<summary><strong>Production runtime images</strong></summary>
Gitea CI builds and pushes three production images: Gitea CI builds and pushes three production images:
@@ -197,32 +268,36 @@ git.gbrown.org/gib/spoon-agent-worker:latest
git.gbrown.org/gib/spoon-agent-job:latest git.gbrown.org/gib/spoon-agent-job:latest
``` ```
The worker image is the long-running service that polls Convex. The job image is The worker image is the long-running service that polls Convex. The
the isolated workbench that the worker launches for each agent job. For the MVP, `spoon-agent-job` image is the **box** image — the per-user Fedora dev box the
production should use the repo-provided JS/TS workbench image: worker `exec`s into (it is not launched fresh per job). Point the worker at it
with:
```env ```env
SPOON_AGENT_JOB_IMAGE="git.gbrown.org/gib/spoon-agent-job:latest" SPOON_AGENT_JOB_IMAGE="git.gbrown.org/gib/spoon-agent-job:latest"
``` ```
The job image includes Node 22, Bun, pnpm and yarn through Corepack, npm, git, The box image is Fedora 41 with Node, Bun, pnpm and yarn, npm, git, ripgrep,
ripgrep, Python, build tools, OpenCode, the Codex CLI, and the Claude Code CLI. Python, build tools, plus interactive tooling (neovim, tmux, fzf, fd, bat, eza,
It is not the forked project's production runtime; it is the agent execution zoxide, gh, gum, oh-my-posh) and the pinned agent CLIs OpenCode, Codex, and
environment. Claude Code. It is not the forked project's production runtime; it is the agent
execution environment.
Production worker runtime requirements: Production worker runtime requirements:
- `spoon-agent-worker` must run as a separate service. - `spoon-agent-worker` must run as a separate service.
- The worker needs `/var/run/docker.sock` mounted so it can launch job - The worker needs `/var/run/docker.sock` mounted so it can create and `exec`
containers. into per-user boxes on the host daemon.
- Production should keep `SPOON_AGENT_CONTAINER_RUNTIME=docker` and - Production should keep `SPOON_AGENT_CONTAINER_RUNTIME=docker`.
`SPOON_AGENT_CONTAINER_ACCESS=network`. - `SPOON_AGENT_HOST_WORKDIR` must equal the absolute host path backing
- The production Docker host must be logged into `git.gbrown.org` so worker jobs `SPOON_AGENT_WORKDIR` (identical inside and outside the worker container) so
can pull the private `spoon-agent-job` image. the host daemon can bind-mount per-user homes under `${WORKDIR}/homes/{username}`.
- The production Docker host must be logged into `git.gbrown.org` so the worker
can pull the private `spoon-agent-job` box image.
- `SPOON_WORKER_TOKEN` must match the value stored in Convex production env. - `SPOON_WORKER_TOKEN` must match the value stored in Convex production env.
- `spoon-next` needs `SPOON_AGENT_WORKER_URL=http://spoon-agent-worker:3921` and - `spoon-next` needs `SPOON_AGENT_WORKER_URL=http://spoon-agent-worker:3921` and
`SPOON_AGENT_WORKER_INTERNAL_TOKEN` so Next API routes can proxy workspace `SPOON_AGENT_WORKER_INTERNAL_TOKEN` so Next API routes can proxy workspace and
file, diff, message, command, and draft PR actions. box file, diff, message, command, terminal, and draft PR actions.
- `spoon-agent-worker` also needs `GITHUB_APP_ID` and `GITHUB_APP_PRIVATE_KEY`. - `spoon-agent-worker` also needs `GITHUB_APP_ID` and `GITHUB_APP_PRIVATE_KEY`.
If the private key is stored in a single-line dotenv value, encode newlines as If the private key is stored in a single-line dotenv value, encode newlines as
literal `\n` characters so the worker can restore the PEM before using it. literal `\n` characters so the worker can restore the PEM before using it.
@@ -241,26 +316,57 @@ curl -H "Authorization: Bearer $SPOON_AGENT_WORKER_INTERNAL_TOKEN" \
Deployment readiness checklist: Deployment readiness checklist:
1. Production Convex env has `SPOON_WORKER_TOKEN`, `SPOON_ENCRYPTION_KEY`, 1. Production Convex env has `SPOON_WORKER_TOKEN`, `SPOON_ENCRYPTION_KEY`,
GitHub App env, and Convex Auth signing keys. GitHub App env (including `GITHUB_APP_WEBHOOK_SECRET`), and Convex Auth
signing keys.
2. Compose env has `SPOON_AGENT_WORKER_URL`, 2. Compose env has `SPOON_AGENT_WORKER_URL`,
`SPOON_AGENT_WORKER_INTERNAL_TOKEN`, `SPOON_AGENT_JOB_IMAGE`, and the GitHub `SPOON_AGENT_WORKER_INTERNAL_TOKEN`, `SPOON_AGENT_JOB_IMAGE`, and the GitHub
App private key. App private key; the `spoon-next` image is built with
`NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL`.
3. The production Docker host can pull private images from `git.gbrown.org`. 3. The production Docker host can pull private images from `git.gbrown.org`.
4. `Settings -> Worker` reports the expected job image, runtime, network, and 4. `Settings -> Worker` reports the expected box image, runtime, network, and
active workspace count. active box count.
5. The first test thread uses a configured API-key provider or a trusted Codex 5. The first test thread uses a configured API-key provider or a trusted Codex
login profile. login profile.
6. If a worker restart leaves stale workspace state, use the workspace recovery 6. If a worker restart leaves stale state, use the workspace recovery panel or
panel or `Settings -> Worker` cleanup. `Settings -> Worker` cleanup.
API-key based AI provider profiles run through OpenCode. Codex ChatGPT login </details>
profiles run through the Codex CLI: Spoon writes the encrypted `auth.json` into
the isolated job workspace as `CODEX_HOME/.codex/auth.json` before execution. <details>
Anthropic profiles run through the Claude Code CLI: API-key profiles authenticate <summary><strong>GitHub webhooks</strong></summary>
via `ANTHROPIC_API_KEY`, while `anthropic_oauth_json` credential-snapshot profiles
have their encrypted OAuth blob written into the isolated job workspace as Spoon exposes a signature-verified webhook as a Convex `httpAction` at
`~/.claude/.credentials.json` before execution. Treat those saved auth files like `POST /webhooks/github` (`packages/backend/convex/githubWebhooks.ts`,
a password and only use them on trusted workers. `http.ts`). Configure the GitHub App webhook URL as
`<CONVEX_SITE_URL>/webhooks/github` and set `GITHUB_APP_WEBHOOK_SECRET`; the
endpoint fails closed (HTTP 503) when no secret is configured and rejects any
payload whose `x-hub-signature-256` does not verify.
- `push` → a targeted drift refresh for every Spoon tracking that repository.
- `installation` / `installation_repositories` → the matching connection's
status flips to `revoked` (deleted/suspended), `active` (created/unsuspended),
or otherwise `needs_reauth`, surfaced in `Settings -> Integrations`.
The hourly `refreshDueSpoons` cron remains a fallback, so drift still refreshes
even if a webhook is missed.
</details>
<details>
<summary><strong>Notifications</strong></summary>
Spoon delivers an in-app bell plus preference-gated transactional email
(`packages/backend/convex/notifications.ts`). Five kinds are emitted:
`maintenance_thread`, `agent_turn_finished`, `agent_needs_input`, `sync_failed`,
and `connection_needs_reauth`.
- Every event inserts an in-app notification (bell badge via `unreadCount`,
list via `listMine`, `markRead` / `markAllRead`).
- Email is sent through UseSend (`USESEND_*`) only when the user has an email
and the matching per-kind preference is not disabled. Unset preferences
default to enabled.
- Per-kind email toggles live in `Settings -> Notifications`. Web push is out of
scope.
</details> </details>
@@ -289,45 +395,54 @@ a password and only use them on trusted workers.
<details> <details>
<summary><strong>Core tables</strong></summary> <summary><strong>Core tables</strong></summary>
| Table | Purpose | | Table | Purpose |
| ------------------------ | --------------------------------------------------------- | | ------------------------- | ----------------------------------------------------------- |
| `spoons` | Managed fork records | | `spoons` | Managed fork records |
| `threads` | Durable maintenance and work conversations | | `threads` | Durable maintenance and work conversations |
| `threadMessages` | Messages inside threads | | `threadMessages` | Messages inside threads |
| `syncRuns` | Upstream checks, sync attempts, and maintenance decisions | | `syncRuns` | Upstream checks, sync attempts, and maintenance decisions |
| `ignoredUpstreamChanges` | Intentional ignore records that affect effective drift | | `ignoredUpstreamChanges` | Intentional ignore records that affect effective drift |
| `gitConnections` | Git provider connection metadata | | `gitConnections` | Git provider connection metadata |
| `spoonRepositoryStates` | Latest cached upstream/fork state | | `spoonRepositoryStates` | Latest cached upstream/fork state |
| `spoonCommits` | Cached upstream and fork-only commits | | `spoonCommits` | Cached upstream and fork-only commits |
| `spoonPullRequests` | Cached fork/upstream pull requests | | `spoonPullRequests` | Cached fork/upstream pull requests |
| `spoonSecrets` | Encrypted per-Spoon environment variables | | `spoonSecrets` | Encrypted per-Spoon environment variables |
| `spoonAgentSettings` | Per-Spoon runtime, branch, command, and env-file settings | | `spoonAgentSettings` | Per-Spoon runtime, branch, command, and env-file settings |
| `aiProviderProfiles` | Encrypted provider/auth profiles used by OpenCode | | `aiProviderProfiles` | Encrypted provider/auth profiles (Codex/OpenCode/Anthropic) |
| `agentJobs` | Worker-executed workspace jobs and PR lifecycle | | `agentJobs` | Worker-executed workspace jobs and PR lifecycle |
| `agentJobEvents` | Append-only worker event log | | `agentJobEvents` | Append-only worker event log |
| `agentJobArtifacts` | Diffs, summaries, command output, PR body drafts | | `agentJobArtifacts` | Diffs, summaries, command output, PR body drafts |
| `agentWorkspaceChanges` | Recorded user, agent, and command file changes | | `agentWorkspaceChanges` | Recorded user, agent, and command file changes |
| `userDotfiles` | Encrypted per-user dotfiles overlay + repo/setup config |
| `userEnvironment` | Per-user box/home environment config |
| `notifications` | In-app notification rows (bell) |
| `notificationPreferences` | Per-user email notification toggles |
</details> </details>
<details> <details>
<summary><strong>Important routes</strong></summary> <summary><strong>Important routes</strong></summary>
| Route | Purpose | | Route | Purpose |
| --------------------------------- | --------------------------------------- | | --------------------------------- | ----------------------------------------------- |
| `/` | Public product landing page | | `/` | Public product landing page |
| `/dashboard` | Maintenance overview | | `/dashboard` | Maintenance overview |
| `/spoons` | Managed fork list | | `/spoons` | Managed fork list |
| `/spoons/new` | Manual/GitHub Spoon creation | | `/spoons/new` | Manual/GitHub Spoon creation |
| `/spoons/[spoonId]` | Spoon detail dashboard | | `/spoons/[spoonId]` | Spoon detail dashboard |
| `/spoons/[spoonId]/agent/[jobId]` | Interactive workspace | | `/spoons/[spoonId]/agent/[jobId]` | Interactive workspace |
| `/threads` | Global thread queue | | `/threads` | Global thread queue |
| `/threads/[threadId]` | Thread detail | | `/threads/[threadId]` | Thread detail |
| `/settings/profile` | User profile settings | | `/machine` | My Machine: box status, terminal, home |
| `/settings/integrations` | GitHub and service integration settings | | `/settings/profile` | User profile settings |
| `/settings/ai-providers` | AI/OpenCode provider profiles | | `/settings/integrations` | GitHub and service integration settings |
| `/settings/ai-providers` | AI provider profiles (Codex/OpenCode/Anthropic) |
| `/settings/dotfiles` | Per-user dotfiles overlay + repo |
| `/settings/notifications` | Email notification preferences |
Legacy `/updates` and `/agents` routes redirect into `/threads`. The Convex `httpAction` `POST /webhooks/github` (mounted at
`<CONVEX_SITE_URL>/webhooks/github`) handles GitHub App webhooks. Legacy
`/updates` and `/agents` routes redirect into `/threads`.
</details> </details>
@@ -428,16 +543,17 @@ not call Infisical.
<details open> <details open>
<summary><strong>Public Next variables</strong></summary> <summary><strong>Public Next variables</strong></summary>
| Variable | Used for | | Variable | Used for |
| --------------------------------- | ------------------------------------------- | | --------------------------------------- | ------------------------------------------------------------------- |
| `NEXT_PUBLIC_SITE_URL` | Canonical Spoon web URL | | `NEXT_PUBLIC_SITE_URL` | Canonical Spoon web URL |
| `NEXT_PUBLIC_CONVEX_URL` | Convex client URL | | `NEXT_PUBLIC_CONVEX_URL` | Convex client URL |
| `NEXT_PUBLIC_DEPLOYMENT_URL` | Convex dashboard/deployment URL when needed | | `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL` | Browser-facing worker WS base for the terminal (**build-time** var) |
| `NEXT_PUBLIC_PLAUSIBLE_URL` | Plausible analytics endpoint | | `NEXT_PUBLIC_DEPLOYMENT_URL` | Convex dashboard/deployment URL when needed |
| `NEXT_PUBLIC_SENTRY_DSN` | Browser Sentry DSN | | `NEXT_PUBLIC_PLAUSIBLE_URL` | Plausible analytics endpoint |
| `NEXT_PUBLIC_SENTRY_URL` | Sentry instance URL | | `NEXT_PUBLIC_SENTRY_DSN` | Browser Sentry DSN |
| `NEXT_PUBLIC_SENTRY_ORG` | Sentry organization | | `NEXT_PUBLIC_SENTRY_URL` | Sentry instance URL |
| `NEXT_PUBLIC_SENTRY_PROJECT_NAME` | Sentry project name | | `NEXT_PUBLIC_SENTRY_ORG` | Sentry organization |
| `NEXT_PUBLIC_SENTRY_PROJECT_NAME` | Sentry project name |
</details> </details>
@@ -492,15 +608,16 @@ not call Infisical.
| `SPOON_AGENT_WORKER_URL` | Internal worker HTTP URL used by Next | | `SPOON_AGENT_WORKER_URL` | Internal worker HTTP URL used by Next |
| `SPOON_AGENT_WORKER_HTTP_PORT` | Worker HTTP port | | `SPOON_AGENT_WORKER_HTTP_PORT` | Worker HTTP port |
| `SPOON_AGENT_WORKER_INTERNAL_TOKEN` | Server-only token for Next-to-worker proxy | | `SPOON_AGENT_WORKER_INTERNAL_TOKEN` | Server-only token for Next-to-worker proxy |
| `SPOON_AGENT_JOB_IMAGE` | Agent job container image | | `SPOON_AGENT_TERMINAL_SECRET` | HMAC secret for terminal/box tokens (falls back to the internal/worker token) |
| `SPOON_AGENT_JOB_IMAGE` | Per-user box (Fedora) image the worker `exec`s into |
| `SPOON_AGENT_RUNTIME` | Runtime mode, currently Docker/Podman-oriented | | `SPOON_AGENT_RUNTIME` | Runtime mode, currently Docker/Podman-oriented |
| `SPOON_AGENT_CONTAINER_RUNTIME` | Container CLI used by worker, `docker`/`podman` | | `SPOON_AGENT_CONTAINER_RUNTIME` | Container CLI used by worker, `docker`/`podman` |
| `SPOON_AGENT_CONTAINER_ACCESS` | `network` in prod, `host_port` for host dev | | `SPOON_AGENT_BOX_IDLE_MS` | Idle time before a per-user box is reaped (default `1800000`) |
| `SPOON_AGENT_MAX_CONCURRENT_JOBS` | Worker concurrency limit | | `SPOON_AGENT_MAX_CONCURRENT_JOBS` | Worker concurrency limit |
| `SPOON_AGENT_JOB_TIMEOUT_MS` | Job timeout | | `SPOON_AGENT_JOB_TIMEOUT_MS` | Agent turn timeout |
| `SPOON_AGENT_WORKDIR` | Worker work directory | | `SPOON_AGENT_WORKDIR` | Worker work directory; per-user homes live under `homes/{username}` |
| `SPOON_AGENT_HOST_WORKDIR` | Host path matching `SPOON_AGENT_WORKDIR` when the worker runs in Docker and controls the host Docker socket | | `SPOON_AGENT_HOST_WORKDIR` | Host path matching `SPOON_AGENT_WORKDIR` when the worker runs in Docker and controls the host Docker socket |
| `SPOON_AGENT_NETWORK` | Optional job container network | | `SPOON_AGENT_NETWORK` | Optional box container network |
</details> </details>
@@ -528,12 +645,20 @@ not call Infisical.
- GitHub drift refresh, commit cache, PR cache, and sync-run history - GitHub drift refresh, commit cache, PR cache, and sync-run history
- Effective drift and ignored upstream change records - Effective drift and ignored upstream change records
- Global Threads page and Spoon-scoped Threads tab - Global Threads page and Spoon-scoped Threads tab
- OpenCode/Codex-oriented agent worker and browser workspace foundation - Persistent per-user box the worker `exec`s into for agent turns, terminal, and
commands
- Three agent runtimes (Codex, OpenCode, Claude Code) behind one adapter, with
queue-time runtime validation
- My Machine surface (box status/start/stop/restart, terminal, home browser)
- Interactive workspace terminal (xterm.js → box PTY)
- Per-user dotfiles overlay + optional dotfiles repo
- Signature-verified GitHub webhook with hourly cron fallback
- In-app + email notifications with per-user preferences
- Monaco editor with optional Vim mode - Monaco editor with optional Vim mode
- Diff viewer, command panel, worker logs, and artifacts - Diff viewer, command panel, worker logs, and artifacts
- Encrypted Spoon secrets and bulk `.env` import - Encrypted Spoon secrets and bulk `.env` import
- Encrypted AI provider profiles, including Codex auth JSON and API-key - Encrypted AI provider profiles, including Codex/Anthropic auth JSON and
provider support API-key provider support
- Authentik, GitHub, and password auth through Convex Auth - Authentik, GitHub, and password auth through Convex Auth
- Self-hosted Convex/Postgres deployment model - Self-hosted Convex/Postgres deployment model
docs/agent-terminal.md
+44 -30
View File
@@ -1,34 +1,44 @@
# Workspace interactive terminal # Interactive terminal
A real shell inside the agent workspace, shown as the **Terminal** tab in the A real shell inside the user's persistent box (`spoon-box-{username}`). There are
workspace UI. It's an xterm.js front end bridged to a bash/tmux PTY running in a two entry points, both bridging to the **same** box:
persistent per-job container (the agent job image), mounting the same workspace
the editor and agent use. - The workspace **Terminal** tab — a shell opened at the thread's checkout
(`~/Code/{spoon}/{branch}`).
- The **My Machine** page (`/machine`) — a `~`-rooted shell into the same box.
Both are an xterm.js front end bridged to a bash/tmux PTY via `docker exec` into
the box. There is no per-job `spoon-agent-term-*` container anymore — the box is
long-running and shared by the agent, the editor, and both terminals.
## Architecture ## Architecture
``` ```
browser (xterm.js) browser (xterm.js)
│ 1. GET /api/agent-jobs/:id/terminal-token (Convex-auth'd, owner only) │ 1a. GET /api/agent-jobs/:id/terminal-token (Convex-auth'd, owner only)
│ → { url: "wss://worker…/jobs/:id/terminal?token=…", expiresAt } │ → { url: "wss://worker…/jobs/:id/terminal?token=…", expiresAt }
│ │ 1b. GET /api/box/terminal-token (Convex-auth'd; username
│ 2. WebSocket wss://worker.<domain>/jobs/:id/terminal?token=… │ → { url: "wss://worker…/box/terminal?token=…", expiresAt } derived
│ server-side)
│ 2. WebSocket wss://worker.<domain>/{jobs/:id|box}/terminal?token=…
▼ ▼
nginx ── upgrade ──► spoon-agent-worker :3921 nginx ── upgrade ──► spoon-agent-worker :3921
│ verifyTerminalToken(token, jobId, secret) │ verify HMAC token (job-scoped or username-scoped)
│ dockerode exec -t → bash/tmux PTY │ docker exec -it → bash/tmux PTY
▼ ▼
spoon-agent-term-<jobId> (job image, mounts the workspace) spoon-box-{username} (Fedora box; persistent home mounted)
``` ```
- The browser **never** holds the worker secret. The Next app (which has already - The browser **never** holds the worker secret. The Next app mints a short-lived
verified job ownership) mints a short-lived HMAC token; the worker verifies it. HMAC token; the worker verifies it. The job terminal token is minted only after
a Convex ownership check; the box terminal token carries the username derived
**server-side** from the authed user (never from the request).
- Frames: **binary** = stdin/stdout bytes; **text JSON** `{type:"resize",cols,rows}` - Frames: **binary** = stdin/stdout bytes; **text JSON** `{type:"resize",cols,rows}`
= resize. The token's 2-minute expiry is a _connect_ window; an established = resize. The token's 2-minute expiry is a _connect_ window; an established
session persists. session persists.
- The shell runs `tmux new-session -A -s spoon` (falls back to `bash -l`), so - The shell prefers `tmux new-session -A -s spoon` (falls back to `bash -l`), so
reconnecting reattaches the same session. Idle containers are removed after reconnecting reattaches the same session. The box itself is idle-reaped after
`SPOON_AGENT_TERMINAL_IDLE_MS` (default 30m). `SPOON_AGENT_BOX_IDLE_MS` (default 30m) once no job or terminal holds it.
## Configuration ## Configuration
@@ -37,8 +47,7 @@ nginx ── upgrade ──► spoon-agent-worker :3921
| Next app | `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL` | **Yes** | Browser-facing worker WS base, e.g. `wss://worker.spoon.gbrown.org` (prod) or `ws://localhost:3921` (dev). **Build-time** (`NEXT_PUBLIC`): for the Docker image it must be passed as a build arg (wired in `docker/Dockerfile` + `docker/compose.yml`, sourced from the build env file), not a runtime env. Unset → the Terminal tab shows "not configured". | | Next app | `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL` | **Yes** | Browser-facing worker WS base, e.g. `wss://worker.spoon.gbrown.org` (prod) or `ws://localhost:3921` (dev). **Build-time** (`NEXT_PUBLIC`): for the Docker image it must be passed as a build arg (wired in `docker/Dockerfile` + `docker/compose.yml`, sourced from the build env file), not a runtime env. Unset → the Terminal tab shows "not configured". |
| Next app | `SPOON_AGENT_TERMINAL_SECRET` | No | HMAC secret for signing tokens. Falls back to `SPOON_AGENT_WORKER_INTERNAL_TOKEN`. | | Next app | `SPOON_AGENT_TERMINAL_SECRET` | No | HMAC secret for signing tokens. Falls back to `SPOON_AGENT_WORKER_INTERNAL_TOKEN`. |
| Worker | `SPOON_AGENT_TERMINAL_SECRET` | No | Must match the Next app's. Falls back to `SPOON_AGENT_WORKER_INTERNAL_TOKEN` (already shared), so by default **no new secret is needed**. | | Worker | `SPOON_AGENT_TERMINAL_SECRET` | No | Must match the Next app's. Falls back to `SPOON_AGENT_WORKER_INTERNAL_TOKEN` (already shared), so by default **no new secret is needed**. |
| Worker | `SPOON_AGENT_TERMINAL_IMAGE` | No | Shell container image. Defaults to `SPOON_AGENT_JOB_IMAGE`. | | Worker | `SPOON_AGENT_BOX_IDLE_MS` | No | Idle-box reap delay, shared with agent jobs (default `1800000`). The terminal holds the box open while connected. |
| Worker | `SPOON_AGENT_TERMINAL_IDLE_MS` | No | Idle-container reap delay (default `1800000`). |
Because the secret defaults to the already-shared worker token, the **only** Because the secret defaults to the already-shared worker token, the **only**
required step is setting `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL` and exposing the required step is setting `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL` and exposing the
@@ -70,9 +79,10 @@ server {
Then set on the Next app: `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL=wss://worker.spoon.gbrown.org`. Then set on the Next app: `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL=wss://worker.spoon.gbrown.org`.
> The worker's HTTP routes (`/jobs/:id/tree` etc.) require the internal bearer > The worker's HTTP routes (`/jobs/:id/tree`, `/box/*`, etc.) require the
> token, so exposing the worker host only usefully exposes the token-gated > internal bearer token, so exposing the worker host only usefully exposes the
> `/jobs/:id/terminal` upgrade. Still, restrict the server block to TLS. > token-gated `/jobs/:id/terminal` and `/box/terminal` upgrades. Still, restrict
> the server block to TLS.
## Dev testing (no nginx) ## Dev testing (no nginx)
@@ -90,15 +100,19 @@ Docker mode. Prod (Docker socket mounted) works as-is.
## Security ## Security
- Owner-only: the token route uses Convex auth + `assertOwned`. - Owner-only: the job token route uses Convex auth + ownership assertion; the
- Tokens are short-lived (2m connect window), job-scoped, HMAC-signed. box token derives the username server-side from the authed user.
- A shell in the workspace can reach the network and the repo's git credentials. - Tokens are short-lived (2m connect window), scoped (job-scoped or
This is intended for the single-user self-hosted deployment; do not expose the username-scoped), and HMAC-signed.
- A shell in the box can reach the network and the repo's git credentials. This
is intended for the single-user self-hosted deployment; do not expose the
worker domain without TLS, and keep the deployment single-tenant. worker domain without TLS, and keep the deployment single-tenant.
## Tools in the shell ## Tools in the shell
The job image ships `bash`, `tmux`, `neovim`, `git`, `ripgrep`, `jq`, `python3`, The box image ships `bash`, `tmux`, `neovim`, `git`, `ripgrep`, `jq`, `python3`,
`node`, `bun`, `pnpm`, `yarn`, `curl`/`wget`, `unzip`. Bring your own dotfiles by `node`, `bun`, `pnpm`, `yarn`, `curl`/`wget`, `unzip`, plus QoL tooling (`fzf`,
cloning them in-session (e.g. `git clone <dotfiles> ~/.config/...`); persistent `fd`, `bat`, `eza`, `zoxide`, `gh`, `gum`, `oh-my-posh`) and the agent CLIs
auto-cloning of a dotfiles repo is a planned follow-up. (`codex`, `opencode`, `claude`). Personalize the shell in **Settings → Dotfiles**
(overlay files + an optional dotfiles repo), applied to the persistent home; see
[dotfiles.md](dotfiles.md).
docs/compose.prod.yml
+13 -13
View File
@@ -7,18 +7,18 @@
# If you change them, read the comments first: # If you change them, read the comments first:
# #
# 1. AGENT WORKDIR (spoon-agent-worker): the worker is containerized but # 1. AGENT WORKDIR (spoon-agent-worker): the worker is containerized but
# launches the Codex job container by talking to the HOST Docker daemon. # creates and `exec`s into the per-user box (spoon-box-{username}) by talking
# The host can only bind-mount real HOST paths, so the work directory MUST # to the HOST Docker daemon. The host can only bind-mount real HOST paths, so
# be a bind mount whose path is IDENTICAL inside and outside the container, # the work directory MUST be a bind mount whose path is IDENTICAL inside and
# and SPOON_AGENT_HOST_WORKDIR must match it. A named volume does NOT work # outside the container, and SPOON_AGENT_HOST_WORKDIR must match it. A named
# here because its real host path is hidden from the worker. All three # volume does NOT work here because its real host path is hidden from the
# references to /var/lib/spoon-agent/work below must stay in sync; change # worker. All three references to /var/lib/spoon-agent/work below must stay
# them together if you want the data somewhere else. # in sync; change them together if you want the data somewhere else.
# #
# 2. IMAGE FRESHNESS: services use `pull_policy: always` + Watchtower labels so # 2. IMAGE FRESHNESS: services use `pull_policy: always` + Watchtower labels so
# a redeploy / new push always lands. The Codex *job* image is pulled by the # a redeploy / new push always lands. The *box* image is pulled by the worker
# worker itself on startup (see SPOON_AGENT_JOB_IMAGE); restarting the worker # itself on startup (see SPOON_AGENT_JOB_IMAGE); restarting the worker (which
# (which Watchtower does on a new image) re-pulls a fresh job image. # Watchtower does on a new image) re-pulls a fresh box image.
networks: networks:
nginx-bridge: # Change to network you plan to use nginx-bridge: # Change to network you plan to use
@@ -83,9 +83,9 @@ services:
- SPOON_WORKER_TOKEN=${SPOON_WORKER_TOKEN} - SPOON_WORKER_TOKEN=${SPOON_WORKER_TOKEN}
volumes: volumes:
- /var/run/docker.sock:/var/run/docker.sock - /var/run/docker.sock:/var/run/docker.sock
# Identical host:container path so the sibling Codex job containers can # Identical host:container path so the sibling per-user box can bind-mount
# bind-mount the workspace via the host daemon. Do NOT switch this to a # the persistent home (homes/{username}) via the host daemon. Do NOT switch
# named volume. See header note (1). # this to a named volume. See header note (1).
- /var/lib/spoon-agent/work:/var/lib/spoon-agent/work - /var/lib/spoon-agent/work:/var/lib/spoon-agent/work
labels: ['com.centurylinklabs.watchtower.enable=true'] labels: ['com.centurylinklabs.watchtower.enable=true']
tty: true tty: true
docs/dotfiles.md
+14 -10
View File
@@ -6,10 +6,11 @@ preloaded with QoL CLI tooling, a persistent per-user home, and user dotfiles.
## The model ## The model
- **Persistent per-user home.** Each user gets a home directory on the worker - **Persistent per-user home.** Each user gets a home directory on the worker
host at `${SPOON_AGENT_WORKDIR}/homes/{username}`, bind-mounted into every host at `${SPOON_AGENT_WORKDIR}/homes/{username}`, bind-mounted into the
job/terminal container at `/home/{username}` (`HOME`). It survives across per-user box (`spoon-box-{username}`) at `/home/{username}` (`HOME`). It
sessions, so dotfiles, installed tools, nvim plugins, and shell history persist. survives across sessions, so dotfiles, installed tools, nvim plugins, and shell
`username` is derived from the user's profile first name (sanitized). history persist. `username` is derived from the user's profile first name
(sanitized).
- **Threads as folders.** Each thread's checkout lives at - **Threads as folders.** Each thread's checkout lives at
`~/Code/{spoon}/{branch}` inside that home, so every thread shows up as a `~/Code/{spoon}/{branch}` inside that home, so every thread shows up as a
folder in one home. The agent (`codex --cd …`) and the terminal both open there. folder in one home. The agent (`codex --cd …`) and the terminal both open there.
@@ -61,18 +62,21 @@ Secrets: dotfiles are encrypted, but real API keys/tokens belong in a Spoon's
No new required env. The home is a host directory under the existing workdir, so No new required env. The home is a host directory under the existing workdir, so
the prod bind-mount + `SPOON_AGENT_HOST_WORKDIR` translation already covers it. the prod bind-mount + `SPOON_AGENT_HOST_WORKDIR` translation already covers it.
## Notes / limits (Phase 1) ## Notes / limits
- **Repo auth:** public repos only. Private/self-hosted (e.g. Gitea) dotfiles - **Repo auth:** public repos only. Private/self-hosted (e.g. Gitea) dotfiles
repos are a follow-up (store a token/deploy key). repos are a follow-up (store a token/deploy key).
- **Binary files:** the overlay is text-first. - **Binary files:** the overlay is text-first.
- **Cleanup:** `~/Code/{spoon}/{branch}` checkouts persist (threads as folders); - **Cleanup:** `~/Code/{spoon}/{branch}` checkouts persist (threads as folders);
a per-thread "delete checkout" action is a follow-up. a per-thread "delete checkout" action is a follow-up.
- **Concurrency:** jobs share one home; fine at the default - **Concurrency:** jobs and terminals share one box/home; fine at the default
`SPOON_AGENT_MAX_CONCURRENT_JOBS=1`. `SPOON_AGENT_MAX_CONCURRENT_JOBS=1`.
## Phase 2 north star ## The box
A single long-running per-user container that every thread `exec`s into (agent The per-user home + `~/Code/{spoon}/{branch}` layout is the foundation of the
via `docker exec`, not `docker run --rm`). The per-user home + `~/Code/{spoon}/ box model: a single long-running per-user container (`spoon-box-{username}`) that
{branch}` layout built here is its foundation. every thread's agent turn, terminal, and command `exec`s into (`docker exec`, not
`docker run --rm`). Home materialization runs once per box start (and again when
the dotfiles config hash changes), so the overlay and repo setup apply to every
session in that box.
docs/server-deploy-changes.md
+36 -13
View File
@@ -1,7 +1,7 @@
# Server deploy changes (terminal + dotfiles + Fedora + Phase 2) # Server deploy changes (box + terminal + dotfiles + webhooks + notifications)
Everything the production host / compose / `.env` needs for the workspace Everything the production host / compose / `.env` needs for the per-user box, the
terminal, personalized dev environment, Nerd Font, and the per-user container. terminal, the personalized dev environment, GitHub webhooks, and notifications.
Most items have safe defaults; the **Required** ones are the only must-dos. Most items have safe defaults; the **Required** ones are the only must-dos.
## Required ## Required
@@ -44,9 +44,10 @@ Most items have safe defaults; the **Required** ones are the only must-dos.
restart picks up the new Fedora job image. Make sure the prod registry has the restart picks up the new Fedora job image. Make sure the prod registry has the
new `spoon-agent-job:latest`. new `spoon-agent-job:latest`.
4. **Deploy Convex functions** (new tables `userDotfiles`, `userEnvironment`). 4. **Deploy Convex functions** (new tables `userDotfiles`, `userEnvironment`,
`SPOON_ENCRYPTION_KEY` (or `INSTANCE_SECRET`) is already required and is what `notifications`, `notificationPreferences`). `SPOON_ENCRYPTION_KEY` (or
encrypts dotfiles at rest — no change, just confirm it's set. `INSTANCE_SECRET`) is already required and is what encrypts dotfiles at rest —
no change, just confirm it's set.
5. **Confirm `SPOON_AGENT_HOST_WORKDIR`** on the `spoon-agent-worker` service is 5. **Confirm `SPOON_AGENT_HOST_WORKDIR`** on the `spoon-agent-worker` service is
the absolute host path backing `SPOON_AGENT_WORKDIR` (the fix from the terminal the absolute host path backing `SPOON_AGENT_WORKDIR` (the fix from the terminal
@@ -54,17 +55,36 @@ Most items have safe defaults; the **Required** ones are the only must-dos.
and are bind-mounted into the box via the host daemon — this only resolves if and are bind-mounted into the box via the host daemon — this only resolves if
the host-workdir translation is correct. (No new var; just verify.) the host-workdir translation is correct. (No new var; just verify.)
6. **Configure the GitHub App webhook.** Set the App's webhook URL to
`<CONVEX_SITE_URL>/webhooks/github` and set `GITHUB_APP_WEBHOOK_SECRET` in the
**Convex** production env (the signature-verified `httpAction` runs in Convex,
not the worker). Without the secret the endpoint fails closed (HTTP 503) and
drift only refreshes via the hourly cron. `push` refreshes affected Spoons;
`installation` / `installation_repositories` update connection status.
7. **Notifications email (optional).** In-app notifications work with no config.
To also send email, set `USESEND_API_KEY`, `USESEND_URL`, and
`USESEND_FROM_EMAIL` in the Convex env. Per-user, per-kind email toggles live
in `Settings → Notifications`; unset preferences default to on.
8. **Claude Code runtime.** The box image pins `@anthropic-ai/claude-code`, so
Anthropic provider profiles (API key or `anthropic_oauth_json` credential
snapshot) can drive the `claude` runtime with no extra host env. Anthropic
API-key auth flows through `ANTHROPIC_API_KEY`; OAuth snapshots are written to
`~/.claude/.credentials.json` in the box.
## Optional (safe defaults — only set to override) ## Optional (safe defaults — only set to override)
On the `spoon-agent-worker` service: On the `spoon-agent-worker` service:
| Var | Default | Purpose | | Var | Default | Purpose |
| ------------------------------ | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | | ----------------------------- | ------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `SPOON_AGENT_TERMINAL_SECRET` | falls back to `SPOON_AGENT_WORKER_INTERNAL_TOKEN` | HMAC secret for terminal tokens (must match the Next app's, which also falls back). Leave unset to use the shared token. | | `SPOON_AGENT_TERMINAL_SECRET` | falls back to `SPOON_AGENT_WORKER_INTERNAL_TOKEN` | HMAC secret for terminal/box tokens (must match the Next app's, which also falls back). Leave unset to use the shared token. |
| `SPOON_AGENT_BOX_IDLE_MS` | `1800000` (30m) | How long a per-user box survives idle before being reaped. | | `SPOON_AGENT_BOX_IDLE_MS` | `1800000` (30m) | How long a per-user box survives idle before being reaped. |
| `SPOON_AGENT_TERMINAL_IDLE_MS` | `1800000` | (Legacy; box idle now governs cleanup.) |
No new env is needed for dotfiles, the per-user home, or the Nerd Font. No new env is needed for dotfiles, the per-user home, or the Nerd Font.
`SPOON_AGENT_TERMINAL_IMAGE` and `SPOON_AGENT_TERMINAL_IDLE_MS` were removed —
the box uses `SPOON_AGENT_JOB_IMAGE` and `SPOON_AGENT_BOX_IDLE_MS`.
## Notes / one-time cleanup ## Notes / one-time cleanup
@@ -84,7 +104,10 @@ No new env is needed for dotfiles, the per-user home, or the Nerd Font.
```bash ```bash
docker exec spoon-agent-worker docker --version # CLI present (29.x) docker exec spoon-agent-worker docker --version # CLI present (29.x)
docker run --rm git.gbrown.org/gib/spoon-agent-job:latest codex --version # 0.142 docker run --rm git.gbrown.org/gib/spoon-agent-job:latest codex --version # 0.142
docker run --rm git.gbrown.org/gib/spoon-agent-job:latest claude --version # 2.1.207
docker run --rm git.gbrown.org/gib/spoon-agent-job:latest opencode --version
docker run --rm git.gbrown.org/gib/spoon-agent-job:latest bash -lc 'eza --version; zoxide --version; oh-my-posh --version' docker run --rm git.gbrown.org/gib/spoon-agent-job:latest bash -lc 'eza --version; zoxide --version; oh-my-posh --version'
# then: open a thread → Terminal tab; Settings → Dotfiles add a .bashrc alias. # then: /machine → box status + terminal; open a thread → Terminal tab;
# Settings → Dotfiles add a .bashrc alias; Settings → Notifications toggles.
``` ```