docs: rewrite for box-first model + webhooks + notifications
This commit is contained in:
Gabriel Brown
@@ -10,7 +10,8 @@
|
||||
|
||||
<p align="center">
|
||||
Spoon is a self-hostable fork maintenance cockpit built around managed forks,
|
||||
durable maintenance threads, and OpenCode-powered workspaces.
|
||||
durable maintenance threads, and a persistent per-user dev box that agents,
|
||||
terminals, and project commands all run inside.
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
@@ -61,14 +62,38 @@ in the data model, but GitHub is the active automation surface today.
|
||||
Spoon keeps raw GitHub drift visible while also tracking ignored upstream
|
||||
changes so irrelevant commits do not keep a fork permanently actionable.
|
||||
|
||||
- **OpenCode workspaces**
|
||||
Agent work happens in an isolated workspace with a file tree, browser editor,
|
||||
diff viewer, command panel, logs, artifacts, and draft PR actions.
|
||||
- **One persistent per-user box**
|
||||
Every user owns a single long-running Fedora container `spoon-box-{username}`
|
||||
with a persistent home. Every thread's agent turn, terminal session, and
|
||||
project command `docker exec`s into that same box — there is no per-job
|
||||
throwaway container.
|
||||
|
||||
- **Three agent runtimes**
|
||||
Codex, OpenCode, and Claude Code all run inside the box behind one adapter
|
||||
interface, selected per thread and validated at queue time against the AI
|
||||
provider profile.
|
||||
|
||||
- **Workspaces**
|
||||
Agent work happens in a workspace with a file tree, browser editor, diff
|
||||
viewer, command panel, logs, artifacts, an interactive terminal, and draft PR
|
||||
actions.
|
||||
|
||||
- **My Machine**
|
||||
A `/machine` surface to start/stop/restart the box, open a `~`-rooted
|
||||
terminal, and browse the persistent home.
|
||||
|
||||
- **GitHub webhooks**
|
||||
A signature-verified webhook keeps drift fresh on `push` and flips connection
|
||||
status on `installation` changes; the hourly cron is a fallback.
|
||||
|
||||
- **Notifications**
|
||||
An in-app bell plus preference-gated transactional email for maintenance
|
||||
threads, agent turns, needed input, sync failures, and connection re-auth.
|
||||
|
||||
- **User-owned providers and secrets**
|
||||
AI provider profiles, Codex/OpenCode auth, and per-Spoon project secrets are
|
||||
encrypted. Secrets are redacted from logs and refused from commits when
|
||||
materialized into env files.
|
||||
AI provider profiles, Codex/OpenCode/Anthropic auth, and per-Spoon project
|
||||
secrets are encrypted. Secrets are written `0600`, redacted from logs, and
|
||||
refused from commits when materialized into env files.
|
||||
|
||||
- **Draft PR handoff**
|
||||
Code changes become branches and draft pull requests. Spoon does not
|
||||
@@ -138,12 +163,23 @@ custom work exists.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>OpenCode workspaces</strong></summary>
|
||||
<summary><strong>The box and workspaces</strong></summary>
|
||||
|
||||
Spoon's optional agent worker is designed to run outside Convex actions. The
|
||||
worker claims queued jobs, clones the current GitHub fork, creates a branch,
|
||||
starts an isolated workspace, and exposes workspace operations to the Next app
|
||||
through server-only API proxies.
|
||||
worker claims queued jobs, clones the current GitHub fork into the owner's
|
||||
persistent home, creates a branch, and exposes workspace operations to the Next
|
||||
app through server-only API proxies.
|
||||
|
||||
Everything runs inside **one long-running container per user**,
|
||||
`spoon-box-{username}`. The box is a Fedora image (`docker/agent-job.Dockerfile`)
|
||||
started with `sleep infinity` and a `--memory 4g` / `--cpus 2` cap. Each
|
||||
thread's agent turn, terminal, and project commands `docker exec` into that same
|
||||
box; the box's `/home/{username}` is a bind-mounted persistent home so dotfiles,
|
||||
installed tools, shell history, and thread checkouts under `~/Code/{spoon}/{branch}`
|
||||
survive across sessions. The box is reference-counted by the worker and reaped
|
||||
after `SPOON_AGENT_BOX_IDLE_MS` idle (default 30m). There is no per-job
|
||||
`docker run --rm` container and no separate `opencode serve` container — that
|
||||
path was removed.
|
||||
|
||||
Workspace capabilities:
|
||||
|
||||
@@ -154,16 +190,18 @@ Workspace capabilities:
|
||||
- inspect diffs
|
||||
- send thread messages to the agent
|
||||
- run configured commands
|
||||
- use an interactive terminal (xterm.js → box PTY; see
|
||||
[docs/agent-terminal.md](docs/agent-terminal.md))
|
||||
- store logs and artifacts
|
||||
- push a branch
|
||||
- open a draft PR
|
||||
|
||||
The browser never receives worker tokens and never talks directly to the worker
|
||||
or job container.
|
||||
or the box.
|
||||
|
||||
Worker cleanup is available in `Settings -> Worker`. It can delete old terminal
|
||||
workspace records and ask the active worker to remove orphaned job containers
|
||||
and inactive work directories.
|
||||
Worker cleanup is available in `Settings -> Worker`. It can delete stale
|
||||
workspace records and ask the active worker to remove orphaned containers and
|
||||
inactive work directories (persistent per-user homes are preserved).
|
||||
|
||||
Local worker development:
|
||||
|
||||
@@ -174,20 +212,53 @@ bun dev:next:worker
|
||||
bun dev:next:worker:staging
|
||||
```
|
||||
|
||||
Local host-run worker commands still load env through Infisical, then
|
||||
`scripts/dev-agent-worker` selects Podman when available, falls back to Docker,
|
||||
and publishes the OpenCode server on a localhost port so the host worker can
|
||||
reach the job container. Override with:
|
||||
Local host-run worker commands load env through Infisical, then
|
||||
`scripts/dev-agent-worker` selects Podman when available and falls back to
|
||||
Docker. Override the container CLI with:
|
||||
|
||||
```env
|
||||
SPOON_AGENT_CONTAINER_RUNTIME=podman
|
||||
SPOON_AGENT_CONTAINER_ACCESS=host_port
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Production agent runtime images</strong></summary>
|
||||
<summary><strong>Agent runtimes</strong></summary>
|
||||
|
||||
Three agent CLIs run inside the box behind one `AgentRuntime` adapter interface
|
||||
(`apps/agent-worker/src/runtime/*-adapter.ts`), selected by `job.runtime`:
|
||||
|
||||
| Runtime | CLI | Selected for |
|
||||
| ---------- | ----------------------------------- | -------------------------------------------------- |
|
||||
| `codex` | `@openai/codex` | OpenAI providers and Codex ChatGPT-login snapshots |
|
||||
| `opencode` | `opencode-ai` | OpenAI-compatible API-key providers (default) |
|
||||
| `claude` | `@anthropic-ai/claude-code@2.1.207` | Anthropic providers |
|
||||
|
||||
The runtime is validated at queue time (`runtimeSupport.ts` +
|
||||
`agentJobs.insertJob`): a job is rejected unless the resolved runtime is one the
|
||||
AI provider profile supports. `runtimesForProfile` maps profiles to runtimes —
|
||||
ChatGPT-login snapshots → `codex` only; `openai` API keys → `opencode`/`codex`;
|
||||
Anthropic API keys → `claude`/`opencode`; Anthropic `anthropic_oauth_json`
|
||||
credential snapshots → `claude` only; every other OpenAI-compatible API key →
|
||||
`opencode`.
|
||||
|
||||
Auth is materialized into the box before a turn:
|
||||
|
||||
- **Codex ChatGPT-login** profiles get the encrypted `auth.json` written to
|
||||
`CODEX_HOME/.codex/auth.json`.
|
||||
- **Anthropic API-key** profiles authenticate via `ANTHROPIC_API_KEY` in the
|
||||
environment (no file needed).
|
||||
- **Anthropic `anthropic_oauth_json`** credential-snapshot profiles get the
|
||||
encrypted OAuth blob written to `~/.claude/.credentials.json`.
|
||||
- **API-key** profiles run through OpenCode.
|
||||
|
||||
All materialized auth files are written `0600`. Treat those saved auth files
|
||||
like a password and only use them on trusted workers.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Production runtime images</strong></summary>
|
||||
|
||||
Gitea CI builds and pushes three production images:
|
||||
|
||||
@@ -197,32 +268,36 @@ git.gbrown.org/gib/spoon-agent-worker:latest
|
||||
git.gbrown.org/gib/spoon-agent-job:latest
|
||||
```
|
||||
|
||||
The worker image is the long-running service that polls Convex. The job image is
|
||||
the isolated workbench that the worker launches for each agent job. For the MVP,
|
||||
production should use the repo-provided JS/TS workbench image:
|
||||
The worker image is the long-running service that polls Convex. The
|
||||
`spoon-agent-job` image is the **box** image — the per-user Fedora dev box the
|
||||
worker `exec`s into (it is not launched fresh per job). Point the worker at it
|
||||
with:
|
||||
|
||||
```env
|
||||
SPOON_AGENT_JOB_IMAGE="git.gbrown.org/gib/spoon-agent-job:latest"
|
||||
```
|
||||
|
||||
The job image includes Node 22, Bun, pnpm and yarn through Corepack, npm, git,
|
||||
ripgrep, Python, build tools, OpenCode, the Codex CLI, and the Claude Code CLI.
|
||||
It is not the forked project's production runtime; it is the agent execution
|
||||
environment.
|
||||
The box image is Fedora 41 with Node, Bun, pnpm and yarn, npm, git, ripgrep,
|
||||
Python, build tools, plus interactive tooling (neovim, tmux, fzf, fd, bat, eza,
|
||||
zoxide, gh, gum, oh-my-posh) and the pinned agent CLIs OpenCode, Codex, and
|
||||
Claude Code. It is not the forked project's production runtime; it is the agent
|
||||
execution environment.
|
||||
|
||||
Production worker runtime requirements:
|
||||
|
||||
- `spoon-agent-worker` must run as a separate service.
|
||||
- The worker needs `/var/run/docker.sock` mounted so it can launch job
|
||||
containers.
|
||||
- Production should keep `SPOON_AGENT_CONTAINER_RUNTIME=docker` and
|
||||
`SPOON_AGENT_CONTAINER_ACCESS=network`.
|
||||
- The production Docker host must be logged into `git.gbrown.org` so worker jobs
|
||||
can pull the private `spoon-agent-job` image.
|
||||
- The worker needs `/var/run/docker.sock` mounted so it can create and `exec`
|
||||
into per-user boxes on the host daemon.
|
||||
- Production should keep `SPOON_AGENT_CONTAINER_RUNTIME=docker`.
|
||||
- `SPOON_AGENT_HOST_WORKDIR` must equal the absolute host path backing
|
||||
`SPOON_AGENT_WORKDIR` (identical inside and outside the worker container) so
|
||||
the host daemon can bind-mount per-user homes under `${WORKDIR}/homes/{username}`.
|
||||
- The production Docker host must be logged into `git.gbrown.org` so the worker
|
||||
can pull the private `spoon-agent-job` box image.
|
||||
- `SPOON_WORKER_TOKEN` must match the value stored in Convex production env.
|
||||
- `spoon-next` needs `SPOON_AGENT_WORKER_URL=http://spoon-agent-worker:3921` and
|
||||
`SPOON_AGENT_WORKER_INTERNAL_TOKEN` so Next API routes can proxy workspace
|
||||
file, diff, message, command, and draft PR actions.
|
||||
`SPOON_AGENT_WORKER_INTERNAL_TOKEN` so Next API routes can proxy workspace and
|
||||
box file, diff, message, command, terminal, and draft PR actions.
|
||||
- `spoon-agent-worker` also needs `GITHUB_APP_ID` and `GITHUB_APP_PRIVATE_KEY`.
|
||||
If the private key is stored in a single-line dotenv value, encode newlines as
|
||||
literal `\n` characters so the worker can restore the PEM before using it.
|
||||
@@ -241,26 +316,57 @@ curl -H "Authorization: Bearer $SPOON_AGENT_WORKER_INTERNAL_TOKEN" \
|
||||
Deployment readiness checklist:
|
||||
|
||||
1. Production Convex env has `SPOON_WORKER_TOKEN`, `SPOON_ENCRYPTION_KEY`,
|
||||
GitHub App env, and Convex Auth signing keys.
|
||||
GitHub App env (including `GITHUB_APP_WEBHOOK_SECRET`), and Convex Auth
|
||||
signing keys.
|
||||
2. Compose env has `SPOON_AGENT_WORKER_URL`,
|
||||
`SPOON_AGENT_WORKER_INTERNAL_TOKEN`, `SPOON_AGENT_JOB_IMAGE`, and the GitHub
|
||||
App private key.
|
||||
App private key; the `spoon-next` image is built with
|
||||
`NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL`.
|
||||
3. The production Docker host can pull private images from `git.gbrown.org`.
|
||||
4. `Settings -> Worker` reports the expected job image, runtime, network, and
|
||||
active workspace count.
|
||||
4. `Settings -> Worker` reports the expected box image, runtime, network, and
|
||||
active box count.
|
||||
5. The first test thread uses a configured API-key provider or a trusted Codex
|
||||
login profile.
|
||||
6. If a worker restart leaves stale workspace state, use the workspace recovery
|
||||
panel or `Settings -> Worker` cleanup.
|
||||
6. If a worker restart leaves stale state, use the workspace recovery panel or
|
||||
`Settings -> Worker` cleanup.
|
||||
|
||||
API-key based AI provider profiles run through OpenCode. Codex ChatGPT login
|
||||
profiles run through the Codex CLI: Spoon writes the encrypted `auth.json` into
|
||||
the isolated job workspace as `CODEX_HOME/.codex/auth.json` before execution.
|
||||
Anthropic profiles run through the Claude Code CLI: API-key profiles authenticate
|
||||
via `ANTHROPIC_API_KEY`, while `anthropic_oauth_json` credential-snapshot profiles
|
||||
have their encrypted OAuth blob written into the isolated job workspace as
|
||||
`~/.claude/.credentials.json` before execution. Treat those saved auth files like
|
||||
a password and only use them on trusted workers.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>GitHub webhooks</strong></summary>
|
||||
|
||||
Spoon exposes a signature-verified webhook as a Convex `httpAction` at
|
||||
`POST /webhooks/github` (`packages/backend/convex/githubWebhooks.ts`,
|
||||
`http.ts`). Configure the GitHub App webhook URL as
|
||||
`<CONVEX_SITE_URL>/webhooks/github` and set `GITHUB_APP_WEBHOOK_SECRET`; the
|
||||
endpoint fails closed (HTTP 503) when no secret is configured and rejects any
|
||||
payload whose `x-hub-signature-256` does not verify.
|
||||
|
||||
- `push` → a targeted drift refresh for every Spoon tracking that repository.
|
||||
- `installation` / `installation_repositories` → the matching connection's
|
||||
status flips to `revoked` (deleted/suspended), `active` (created/unsuspended),
|
||||
or otherwise `needs_reauth`, surfaced in `Settings -> Integrations`.
|
||||
|
||||
The hourly `refreshDueSpoons` cron remains a fallback, so drift still refreshes
|
||||
even if a webhook is missed.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Notifications</strong></summary>
|
||||
|
||||
Spoon delivers an in-app bell plus preference-gated transactional email
|
||||
(`packages/backend/convex/notifications.ts`). Five kinds are emitted:
|
||||
`maintenance_thread`, `agent_turn_finished`, `agent_needs_input`, `sync_failed`,
|
||||
and `connection_needs_reauth`.
|
||||
|
||||
- Every event inserts an in-app notification (bell badge via `unreadCount`,
|
||||
list via `listMine`, `markRead` / `markAllRead`).
|
||||
- Email is sent through UseSend (`USESEND_*`) only when the user has an email
|
||||
and the matching per-kind preference is not disabled. Unset preferences
|
||||
default to enabled.
|
||||
- Per-kind email toggles live in `Settings -> Notifications`. Web push is out of
|
||||
scope.
|
||||
|
||||
</details>
|
||||
|
||||
@@ -290,7 +396,7 @@ a password and only use them on trusted workers.
|
||||
<summary><strong>Core tables</strong></summary>
|
||||
|
||||
| Table | Purpose |
|
||||
| ------------------------ | --------------------------------------------------------- |
|
||||
| ------------------------- | ----------------------------------------------------------- |
|
||||
| `spoons` | Managed fork records |
|
||||
| `threads` | Durable maintenance and work conversations |
|
||||
| `threadMessages` | Messages inside threads |
|
||||
@@ -302,11 +408,15 @@ a password and only use them on trusted workers.
|
||||
| `spoonPullRequests` | Cached fork/upstream pull requests |
|
||||
| `spoonSecrets` | Encrypted per-Spoon environment variables |
|
||||
| `spoonAgentSettings` | Per-Spoon runtime, branch, command, and env-file settings |
|
||||
| `aiProviderProfiles` | Encrypted provider/auth profiles used by OpenCode |
|
||||
| `aiProviderProfiles` | Encrypted provider/auth profiles (Codex/OpenCode/Anthropic) |
|
||||
| `agentJobs` | Worker-executed workspace jobs and PR lifecycle |
|
||||
| `agentJobEvents` | Append-only worker event log |
|
||||
| `agentJobArtifacts` | Diffs, summaries, command output, PR body drafts |
|
||||
| `agentWorkspaceChanges` | Recorded user, agent, and command file changes |
|
||||
| `userDotfiles` | Encrypted per-user dotfiles overlay + repo/setup config |
|
||||
| `userEnvironment` | Per-user box/home environment config |
|
||||
| `notifications` | In-app notification rows (bell) |
|
||||
| `notificationPreferences` | Per-user email notification toggles |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -314,7 +424,7 @@ a password and only use them on trusted workers.
|
||||
<summary><strong>Important routes</strong></summary>
|
||||
|
||||
| Route | Purpose |
|
||||
| --------------------------------- | --------------------------------------- |
|
||||
| --------------------------------- | ----------------------------------------------- |
|
||||
| `/` | Public product landing page |
|
||||
| `/dashboard` | Maintenance overview |
|
||||
| `/spoons` | Managed fork list |
|
||||
@@ -323,11 +433,16 @@ a password and only use them on trusted workers.
|
||||
| `/spoons/[spoonId]/agent/[jobId]` | Interactive workspace |
|
||||
| `/threads` | Global thread queue |
|
||||
| `/threads/[threadId]` | Thread detail |
|
||||
| `/machine` | My Machine: box status, terminal, home |
|
||||
| `/settings/profile` | User profile settings |
|
||||
| `/settings/integrations` | GitHub and service integration settings |
|
||||
| `/settings/ai-providers` | AI/OpenCode provider profiles |
|
||||
| `/settings/ai-providers` | AI provider profiles (Codex/OpenCode/Anthropic) |
|
||||
| `/settings/dotfiles` | Per-user dotfiles overlay + repo |
|
||||
| `/settings/notifications` | Email notification preferences |
|
||||
|
||||
Legacy `/updates` and `/agents` routes redirect into `/threads`.
|
||||
The Convex `httpAction` `POST /webhooks/github` (mounted at
|
||||
`<CONVEX_SITE_URL>/webhooks/github`) handles GitHub App webhooks. Legacy
|
||||
`/updates` and `/agents` routes redirect into `/threads`.
|
||||
|
||||
</details>
|
||||
|
||||
@@ -429,9 +544,10 @@ not call Infisical.
|
||||
<summary><strong>Public Next variables</strong></summary>
|
||||
|
||||
| Variable | Used for |
|
||||
| --------------------------------- | ------------------------------------------- |
|
||||
| --------------------------------------- | ------------------------------------------------------------------- |
|
||||
| `NEXT_PUBLIC_SITE_URL` | Canonical Spoon web URL |
|
||||
| `NEXT_PUBLIC_CONVEX_URL` | Convex client URL |
|
||||
| `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL` | Browser-facing worker WS base for the terminal (**build-time** var) |
|
||||
| `NEXT_PUBLIC_DEPLOYMENT_URL` | Convex dashboard/deployment URL when needed |
|
||||
| `NEXT_PUBLIC_PLAUSIBLE_URL` | Plausible analytics endpoint |
|
||||
| `NEXT_PUBLIC_SENTRY_DSN` | Browser Sentry DSN |
|
||||
@@ -492,15 +608,16 @@ not call Infisical.
|
||||
| `SPOON_AGENT_WORKER_URL` | Internal worker HTTP URL used by Next |
|
||||
| `SPOON_AGENT_WORKER_HTTP_PORT` | Worker HTTP port |
|
||||
| `SPOON_AGENT_WORKER_INTERNAL_TOKEN` | Server-only token for Next-to-worker proxy |
|
||||
| `SPOON_AGENT_JOB_IMAGE` | Agent job container image |
|
||||
| `SPOON_AGENT_TERMINAL_SECRET` | HMAC secret for terminal/box tokens (falls back to the internal/worker token) |
|
||||
| `SPOON_AGENT_JOB_IMAGE` | Per-user box (Fedora) image the worker `exec`s into |
|
||||
| `SPOON_AGENT_RUNTIME` | Runtime mode, currently Docker/Podman-oriented |
|
||||
| `SPOON_AGENT_CONTAINER_RUNTIME` | Container CLI used by worker, `docker`/`podman` |
|
||||
| `SPOON_AGENT_CONTAINER_ACCESS` | `network` in prod, `host_port` for host dev |
|
||||
| `SPOON_AGENT_BOX_IDLE_MS` | Idle time before a per-user box is reaped (default `1800000`) |
|
||||
| `SPOON_AGENT_MAX_CONCURRENT_JOBS` | Worker concurrency limit |
|
||||
| `SPOON_AGENT_JOB_TIMEOUT_MS` | Job timeout |
|
||||
| `SPOON_AGENT_WORKDIR` | Worker work directory |
|
||||
| `SPOON_AGENT_JOB_TIMEOUT_MS` | Agent turn timeout |
|
||||
| `SPOON_AGENT_WORKDIR` | Worker work directory; per-user homes live under `homes/{username}` |
|
||||
| `SPOON_AGENT_HOST_WORKDIR` | Host path matching `SPOON_AGENT_WORKDIR` when the worker runs in Docker and controls the host Docker socket |
|
||||
| `SPOON_AGENT_NETWORK` | Optional job container network |
|
||||
| `SPOON_AGENT_NETWORK` | Optional box container network |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -528,12 +645,20 @@ not call Infisical.
|
||||
- GitHub drift refresh, commit cache, PR cache, and sync-run history
|
||||
- Effective drift and ignored upstream change records
|
||||
- Global Threads page and Spoon-scoped Threads tab
|
||||
- OpenCode/Codex-oriented agent worker and browser workspace foundation
|
||||
- Persistent per-user box the worker `exec`s into for agent turns, terminal, and
|
||||
commands
|
||||
- Three agent runtimes (Codex, OpenCode, Claude Code) behind one adapter, with
|
||||
queue-time runtime validation
|
||||
- My Machine surface (box status/start/stop/restart, terminal, home browser)
|
||||
- Interactive workspace terminal (xterm.js → box PTY)
|
||||
- Per-user dotfiles overlay + optional dotfiles repo
|
||||
- Signature-verified GitHub webhook with hourly cron fallback
|
||||
- In-app + email notifications with per-user preferences
|
||||
- Monaco editor with optional Vim mode
|
||||
- Diff viewer, command panel, worker logs, and artifacts
|
||||
- Encrypted Spoon secrets and bulk `.env` import
|
||||
- Encrypted AI provider profiles, including Codex auth JSON and API-key
|
||||
provider support
|
||||
- Encrypted AI provider profiles, including Codex/Anthropic auth JSON and
|
||||
API-key provider support
|
||||
- Authentik, GitHub, and password auth through Convex Auth
|
||||
- Self-hosted Convex/Postgres deployment model
|
||||
|
||||
|
||||
+43
-29
@@ -1,34 +1,44 @@
|
||||
# Workspace interactive terminal
|
||||
# Interactive terminal
|
||||
|
||||
A real shell inside the agent workspace, shown as the **Terminal** tab in the
|
||||
workspace UI. It's an xterm.js front end bridged to a bash/tmux PTY running in a
|
||||
persistent per-job container (the agent job image), mounting the same workspace
|
||||
the editor and agent use.
|
||||
A real shell inside the user's persistent box (`spoon-box-{username}`). There are
|
||||
two entry points, both bridging to the **same** box:
|
||||
|
||||
- The workspace **Terminal** tab — a shell opened at the thread's checkout
|
||||
(`~/Code/{spoon}/{branch}`).
|
||||
- The **My Machine** page (`/machine`) — a `~`-rooted shell into the same box.
|
||||
|
||||
Both are an xterm.js front end bridged to a bash/tmux PTY via `docker exec` into
|
||||
the box. There is no per-job `spoon-agent-term-*` container anymore — the box is
|
||||
long-running and shared by the agent, the editor, and both terminals.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
browser (xterm.js)
|
||||
│ 1. GET /api/agent-jobs/:id/terminal-token (Convex-auth'd, owner only)
|
||||
│ 1a. GET /api/agent-jobs/:id/terminal-token (Convex-auth'd, owner only)
|
||||
│ → { url: "wss://worker…/jobs/:id/terminal?token=…", expiresAt }
|
||||
│
|
||||
│ 2. WebSocket wss://worker.<domain>/jobs/:id/terminal?token=…
|
||||
│ 1b. GET /api/box/terminal-token (Convex-auth'd; username
|
||||
│ → { url: "wss://worker…/box/terminal?token=…", expiresAt } derived
|
||||
│ server-side)
|
||||
│ 2. WebSocket wss://worker.<domain>/{jobs/:id|box}/terminal?token=…
|
||||
▼
|
||||
nginx ── upgrade ──► spoon-agent-worker :3921
|
||||
│ verifyTerminalToken(token, jobId, secret)
|
||||
│ dockerode exec -t → bash/tmux PTY
|
||||
│ verify HMAC token (job-scoped or username-scoped)
|
||||
│ docker exec -it → bash/tmux PTY
|
||||
▼
|
||||
spoon-agent-term-<jobId> (job image, mounts the workspace)
|
||||
spoon-box-{username} (Fedora box; persistent home mounted)
|
||||
```
|
||||
|
||||
- The browser **never** holds the worker secret. The Next app (which has already
|
||||
verified job ownership) mints a short-lived HMAC token; the worker verifies it.
|
||||
- The browser **never** holds the worker secret. The Next app mints a short-lived
|
||||
HMAC token; the worker verifies it. The job terminal token is minted only after
|
||||
a Convex ownership check; the box terminal token carries the username derived
|
||||
**server-side** from the authed user (never from the request).
|
||||
- Frames: **binary** = stdin/stdout bytes; **text JSON** `{type:"resize",cols,rows}`
|
||||
= resize. The token's 2-minute expiry is a _connect_ window; an established
|
||||
session persists.
|
||||
- The shell runs `tmux new-session -A -s spoon` (falls back to `bash -l`), so
|
||||
reconnecting reattaches the same session. Idle containers are removed after
|
||||
`SPOON_AGENT_TERMINAL_IDLE_MS` (default 30m).
|
||||
- The shell prefers `tmux new-session -A -s spoon` (falls back to `bash -l`), so
|
||||
reconnecting reattaches the same session. The box itself is idle-reaped after
|
||||
`SPOON_AGENT_BOX_IDLE_MS` (default 30m) once no job or terminal holds it.
|
||||
|
||||
## Configuration
|
||||
|
||||
@@ -37,8 +47,7 @@ nginx ── upgrade ──► spoon-agent-worker :3921
|
||||
| Next app | `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL` | **Yes** | Browser-facing worker WS base, e.g. `wss://worker.spoon.gbrown.org` (prod) or `ws://localhost:3921` (dev). **Build-time** (`NEXT_PUBLIC`): for the Docker image it must be passed as a build arg (wired in `docker/Dockerfile` + `docker/compose.yml`, sourced from the build env file), not a runtime env. Unset → the Terminal tab shows "not configured". |
|
||||
| Next app | `SPOON_AGENT_TERMINAL_SECRET` | No | HMAC secret for signing tokens. Falls back to `SPOON_AGENT_WORKER_INTERNAL_TOKEN`. |
|
||||
| Worker | `SPOON_AGENT_TERMINAL_SECRET` | No | Must match the Next app's. Falls back to `SPOON_AGENT_WORKER_INTERNAL_TOKEN` (already shared), so by default **no new secret is needed**. |
|
||||
| Worker | `SPOON_AGENT_TERMINAL_IMAGE` | No | Shell container image. Defaults to `SPOON_AGENT_JOB_IMAGE`. |
|
||||
| Worker | `SPOON_AGENT_TERMINAL_IDLE_MS` | No | Idle-container reap delay (default `1800000`). |
|
||||
| Worker | `SPOON_AGENT_BOX_IDLE_MS` | No | Idle-box reap delay, shared with agent jobs (default `1800000`). The terminal holds the box open while connected. |
|
||||
|
||||
Because the secret defaults to the already-shared worker token, the **only**
|
||||
required step is setting `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL` and exposing the
|
||||
@@ -70,9 +79,10 @@ server {
|
||||
|
||||
Then set on the Next app: `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL=wss://worker.spoon.gbrown.org`.
|
||||
|
||||
> The worker's HTTP routes (`/jobs/:id/tree` etc.) require the internal bearer
|
||||
> token, so exposing the worker host only usefully exposes the token-gated
|
||||
> `/jobs/:id/terminal` upgrade. Still, restrict the server block to TLS.
|
||||
> The worker's HTTP routes (`/jobs/:id/tree`, `/box/*`, etc.) require the
|
||||
> internal bearer token, so exposing the worker host only usefully exposes the
|
||||
> token-gated `/jobs/:id/terminal` and `/box/terminal` upgrades. Still, restrict
|
||||
> the server block to TLS.
|
||||
|
||||
## Dev testing (no nginx)
|
||||
|
||||
@@ -90,15 +100,19 @@ Docker mode. Prod (Docker socket mounted) works as-is.
|
||||
|
||||
## Security
|
||||
|
||||
- Owner-only: the token route uses Convex auth + `assertOwned`.
|
||||
- Tokens are short-lived (2m connect window), job-scoped, HMAC-signed.
|
||||
- A shell in the workspace can reach the network and the repo's git credentials.
|
||||
This is intended for the single-user self-hosted deployment; do not expose the
|
||||
- Owner-only: the job token route uses Convex auth + ownership assertion; the
|
||||
box token derives the username server-side from the authed user.
|
||||
- Tokens are short-lived (2m connect window), scoped (job-scoped or
|
||||
username-scoped), and HMAC-signed.
|
||||
- A shell in the box can reach the network and the repo's git credentials. This
|
||||
is intended for the single-user self-hosted deployment; do not expose the
|
||||
worker domain without TLS, and keep the deployment single-tenant.
|
||||
|
||||
## Tools in the shell
|
||||
|
||||
The job image ships `bash`, `tmux`, `neovim`, `git`, `ripgrep`, `jq`, `python3`,
|
||||
`node`, `bun`, `pnpm`, `yarn`, `curl`/`wget`, `unzip`. Bring your own dotfiles by
|
||||
cloning them in-session (e.g. `git clone <dotfiles> ~/.config/...`); persistent
|
||||
auto-cloning of a dotfiles repo is a planned follow-up.
|
||||
The box image ships `bash`, `tmux`, `neovim`, `git`, `ripgrep`, `jq`, `python3`,
|
||||
`node`, `bun`, `pnpm`, `yarn`, `curl`/`wget`, `unzip`, plus QoL tooling (`fzf`,
|
||||
`fd`, `bat`, `eza`, `zoxide`, `gh`, `gum`, `oh-my-posh`) and the agent CLIs
|
||||
(`codex`, `opencode`, `claude`). Personalize the shell in **Settings → Dotfiles**
|
||||
(overlay files + an optional dotfiles repo), applied to the persistent home; see
|
||||
[dotfiles.md](dotfiles.md).
|
||||
|
||||
+13
-13
@@ -7,18 +7,18 @@
|
||||
# If you change them, read the comments first:
|
||||
#
|
||||
# 1. AGENT WORKDIR (spoon-agent-worker): the worker is containerized but
|
||||
# launches the Codex job container by talking to the HOST Docker daemon.
|
||||
# The host can only bind-mount real HOST paths, so the work directory MUST
|
||||
# be a bind mount whose path is IDENTICAL inside and outside the container,
|
||||
# and SPOON_AGENT_HOST_WORKDIR must match it. A named volume does NOT work
|
||||
# here because its real host path is hidden from the worker. All three
|
||||
# references to /var/lib/spoon-agent/work below must stay in sync; change
|
||||
# them together if you want the data somewhere else.
|
||||
# creates and `exec`s into the per-user box (spoon-box-{username}) by talking
|
||||
# to the HOST Docker daemon. The host can only bind-mount real HOST paths, so
|
||||
# the work directory MUST be a bind mount whose path is IDENTICAL inside and
|
||||
# outside the container, and SPOON_AGENT_HOST_WORKDIR must match it. A named
|
||||
# volume does NOT work here because its real host path is hidden from the
|
||||
# worker. All three references to /var/lib/spoon-agent/work below must stay
|
||||
# in sync; change them together if you want the data somewhere else.
|
||||
#
|
||||
# 2. IMAGE FRESHNESS: services use `pull_policy: always` + Watchtower labels so
|
||||
# a redeploy / new push always lands. The Codex *job* image is pulled by the
|
||||
# worker itself on startup (see SPOON_AGENT_JOB_IMAGE); restarting the worker
|
||||
# (which Watchtower does on a new image) re-pulls a fresh job image.
|
||||
# a redeploy / new push always lands. The *box* image is pulled by the worker
|
||||
# itself on startup (see SPOON_AGENT_JOB_IMAGE); restarting the worker (which
|
||||
# Watchtower does on a new image) re-pulls a fresh box image.
|
||||
|
||||
networks:
|
||||
nginx-bridge: # Change to network you plan to use
|
||||
@@ -83,9 +83,9 @@ services:
|
||||
- SPOON_WORKER_TOKEN=${SPOON_WORKER_TOKEN}
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock
|
||||
# Identical host:container path so the sibling Codex job containers can
|
||||
# bind-mount the workspace via the host daemon. Do NOT switch this to a
|
||||
# named volume. See header note (1).
|
||||
# Identical host:container path so the sibling per-user box can bind-mount
|
||||
# the persistent home (homes/{username}) via the host daemon. Do NOT switch
|
||||
# this to a named volume. See header note (1).
|
||||
- /var/lib/spoon-agent/work:/var/lib/spoon-agent/work
|
||||
labels: ['com.centurylinklabs.watchtower.enable=true']
|
||||
tty: true
|
||||
|
||||
+14
-10
@@ -6,10 +6,11 @@ preloaded with QoL CLI tooling, a persistent per-user home, and user dotfiles.
|
||||
## The model
|
||||
|
||||
- **Persistent per-user home.** Each user gets a home directory on the worker
|
||||
host at `${SPOON_AGENT_WORKDIR}/homes/{username}`, bind-mounted into every
|
||||
job/terminal container at `/home/{username}` (`HOME`). It survives across
|
||||
sessions, so dotfiles, installed tools, nvim plugins, and shell history persist.
|
||||
`username` is derived from the user's profile first name (sanitized).
|
||||
host at `${SPOON_AGENT_WORKDIR}/homes/{username}`, bind-mounted into the
|
||||
per-user box (`spoon-box-{username}`) at `/home/{username}` (`HOME`). It
|
||||
survives across sessions, so dotfiles, installed tools, nvim plugins, and shell
|
||||
history persist. `username` is derived from the user's profile first name
|
||||
(sanitized).
|
||||
- **Threads as folders.** Each thread's checkout lives at
|
||||
`~/Code/{spoon}/{branch}` inside that home, so every thread shows up as a
|
||||
folder in one home. The agent (`codex --cd …`) and the terminal both open there.
|
||||
@@ -61,18 +62,21 @@ Secrets: dotfiles are encrypted, but real API keys/tokens belong in a Spoon's
|
||||
No new required env. The home is a host directory under the existing workdir, so
|
||||
the prod bind-mount + `SPOON_AGENT_HOST_WORKDIR` translation already covers it.
|
||||
|
||||
## Notes / limits (Phase 1)
|
||||
## Notes / limits
|
||||
|
||||
- **Repo auth:** public repos only. Private/self-hosted (e.g. Gitea) dotfiles
|
||||
repos are a follow-up (store a token/deploy key).
|
||||
- **Binary files:** the overlay is text-first.
|
||||
- **Cleanup:** `~/Code/{spoon}/{branch}` checkouts persist (threads as folders);
|
||||
a per-thread "delete checkout" action is a follow-up.
|
||||
- **Concurrency:** jobs share one home; fine at the default
|
||||
- **Concurrency:** jobs and terminals share one box/home; fine at the default
|
||||
`SPOON_AGENT_MAX_CONCURRENT_JOBS=1`.
|
||||
|
||||
## Phase 2 north star
|
||||
## The box
|
||||
|
||||
A single long-running per-user container that every thread `exec`s into (agent
|
||||
via `docker exec`, not `docker run --rm`). The per-user home + `~/Code/{spoon}/
|
||||
{branch}` layout built here is its foundation.
|
||||
The per-user home + `~/Code/{spoon}/{branch}` layout is the foundation of the
|
||||
box model: a single long-running per-user container (`spoon-box-{username}`) that
|
||||
every thread's agent turn, terminal, and command `exec`s into (`docker exec`, not
|
||||
`docker run --rm`). Home materialization runs once per box start (and again when
|
||||
the dotfiles config hash changes), so the overlay and repo setup apply to every
|
||||
session in that box.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Server deploy changes (terminal + dotfiles + Fedora + Phase 2)
|
||||
# Server deploy changes (box + terminal + dotfiles + webhooks + notifications)
|
||||
|
||||
Everything the production host / compose / `.env` needs for the workspace
|
||||
terminal, personalized dev environment, Nerd Font, and the per-user container.
|
||||
Everything the production host / compose / `.env` needs for the per-user box, the
|
||||
terminal, the personalized dev environment, GitHub webhooks, and notifications.
|
||||
Most items have safe defaults; the **Required** ones are the only must-dos.
|
||||
|
||||
## Required
|
||||
@@ -44,9 +44,10 @@ Most items have safe defaults; the **Required** ones are the only must-dos.
|
||||
restart picks up the new Fedora job image. Make sure the prod registry has the
|
||||
new `spoon-agent-job:latest`.
|
||||
|
||||
4. **Deploy Convex functions** (new tables `userDotfiles`, `userEnvironment`).
|
||||
`SPOON_ENCRYPTION_KEY` (or `INSTANCE_SECRET`) is already required and is what
|
||||
encrypts dotfiles at rest — no change, just confirm it's set.
|
||||
4. **Deploy Convex functions** (new tables `userDotfiles`, `userEnvironment`,
|
||||
`notifications`, `notificationPreferences`). `SPOON_ENCRYPTION_KEY` (or
|
||||
`INSTANCE_SECRET`) is already required and is what encrypts dotfiles at rest —
|
||||
no change, just confirm it's set.
|
||||
|
||||
5. **Confirm `SPOON_AGENT_HOST_WORKDIR`** on the `spoon-agent-worker` service is
|
||||
the absolute host path backing `SPOON_AGENT_WORKDIR` (the fix from the terminal
|
||||
@@ -54,17 +55,36 @@ Most items have safe defaults; the **Required** ones are the only must-dos.
|
||||
and are bind-mounted into the box via the host daemon — this only resolves if
|
||||
the host-workdir translation is correct. (No new var; just verify.)
|
||||
|
||||
6. **Configure the GitHub App webhook.** Set the App's webhook URL to
|
||||
`<CONVEX_SITE_URL>/webhooks/github` and set `GITHUB_APP_WEBHOOK_SECRET` in the
|
||||
**Convex** production env (the signature-verified `httpAction` runs in Convex,
|
||||
not the worker). Without the secret the endpoint fails closed (HTTP 503) and
|
||||
drift only refreshes via the hourly cron. `push` refreshes affected Spoons;
|
||||
`installation` / `installation_repositories` update connection status.
|
||||
|
||||
7. **Notifications email (optional).** In-app notifications work with no config.
|
||||
To also send email, set `USESEND_API_KEY`, `USESEND_URL`, and
|
||||
`USESEND_FROM_EMAIL` in the Convex env. Per-user, per-kind email toggles live
|
||||
in `Settings → Notifications`; unset preferences default to on.
|
||||
|
||||
8. **Claude Code runtime.** The box image pins `@anthropic-ai/claude-code`, so
|
||||
Anthropic provider profiles (API key or `anthropic_oauth_json` credential
|
||||
snapshot) can drive the `claude` runtime with no extra host env. Anthropic
|
||||
API-key auth flows through `ANTHROPIC_API_KEY`; OAuth snapshots are written to
|
||||
`~/.claude/.credentials.json` in the box.
|
||||
|
||||
## Optional (safe defaults — only set to override)
|
||||
|
||||
On the `spoon-agent-worker` service:
|
||||
|
||||
| Var | Default | Purpose |
|
||||
| ------------------------------ | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `SPOON_AGENT_TERMINAL_SECRET` | falls back to `SPOON_AGENT_WORKER_INTERNAL_TOKEN` | HMAC secret for terminal tokens (must match the Next app's, which also falls back). Leave unset to use the shared token. |
|
||||
| ----------------------------- | ------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `SPOON_AGENT_TERMINAL_SECRET` | falls back to `SPOON_AGENT_WORKER_INTERNAL_TOKEN` | HMAC secret for terminal/box tokens (must match the Next app's, which also falls back). Leave unset to use the shared token. |
|
||||
| `SPOON_AGENT_BOX_IDLE_MS` | `1800000` (30m) | How long a per-user box survives idle before being reaped. |
|
||||
| `SPOON_AGENT_TERMINAL_IDLE_MS` | `1800000` | (Legacy; box idle now governs cleanup.) |
|
||||
|
||||
No new env is needed for dotfiles, the per-user home, or the Nerd Font.
|
||||
`SPOON_AGENT_TERMINAL_IMAGE` and `SPOON_AGENT_TERMINAL_IDLE_MS` were removed —
|
||||
the box uses `SPOON_AGENT_JOB_IMAGE` and `SPOON_AGENT_BOX_IDLE_MS`.
|
||||
|
||||
## Notes / one-time cleanup
|
||||
|
||||
@@ -85,6 +105,9 @@ No new env is needed for dotfiles, the per-user home, or the Nerd Font.
|
||||
```bash
|
||||
docker exec spoon-agent-worker docker --version # CLI present (29.x)
|
||||
docker run --rm git.gbrown.org/gib/spoon-agent-job:latest codex --version # 0.142
|
||||
docker run --rm git.gbrown.org/gib/spoon-agent-job:latest claude --version # 2.1.207
|
||||
docker run --rm git.gbrown.org/gib/spoon-agent-job:latest opencode --version
|
||||
docker run --rm git.gbrown.org/gib/spoon-agent-job:latest bash -lc 'eza --version; zoxide --version; oh-my-posh --version'
|
||||
# then: open a thread → Terminal tab; Settings → Dotfiles add a .bashrc alias.
|
||||
# then: /machine → box status + terminal; open a thread → Terminal tab;
|
||||
# Settings → Dotfiles add a .bashrc alias; Settings → Notifications toggles.
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user