gib/spoon

Files

T

Gabriel Brown 8aa9140191 docs: multi-phase implementation plan (63 tasks across 5 phases)

Phase 1 Stabilize (13), Phase 2 Runtime unification (12),
Phase 3 Dev-box surface (11), Phase 4 Sync correctness (12),
Phase 5 Notifications & polish (15), plus index.

2026-07-10 15:18:51 -04:00

40 KiB

Raw Blame History

Spoon Phase 3 — Dev Box as a First-Class Surface: Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Give every user a standalone "My Machine" page at /machine that exposes their persistent per-user Fedora box directly: live status + start/stop/restart controls, a full-page terminal rooted at ~, and a home-scoped file browser (reuse FileTree/CodeEditor). Make the workspace terminal survive tab switches, and add terminal QoL (web-links, copy-on-select). The box is USER-scoped (there is no jobId): the browser reaches it only through Next routes that mint short-lived HMAC tokens after Convex auth resolves the caller's own box username server-side — the browser never sees a worker secret and can only ever reach its OWN box.

Architecture: Same ownership-proxy pattern already used for /jobs/:id/*, replicated for a user-scoped /box/* surface:

Worker (Bun/dockerode/ws): new user-scoped HMAC terminal token (terminal-token.ts), a box helper module (box.ts) for home-path derivation + home-scoped file access + status, HTTP routes GET /box/status, POST /box/lifecycle, GET /box/tree, GET|PUT /box/file, and a WS route GET /box/terminal. All authed by the internal bearer token (HTTP) or the signed user token (WS), which embeds the username.
Next (App Router): agent-worker-proxy.ts gains mintBoxTerminalToken(username), resolveBoxUsername(), and proxyBox(...); /api/box/* route handlers derive the caller's username from Convex auth and proxy to the worker; a client /machine page reuses an extracted XtermSession component plus FileTree + CodeEditor.

Tech Stack: Next.js 16 (React 19), Convex, agent-worker (Bun, dockerode, ws, xterm).

Depends on: Phase 1 (terminal sizing fix + box handle/mutex lifecycle). Phase 3 assumes the Phase-1 acquireUserBox handle API (see "Phase-1 assumptions" below) and the Phase-1 realpath-based path containment. It benefits from Phase 2 (one-box-three-agents) but does not require it: the /box surface only needs the persistent box to exist, not any particular agent runtime.

Phase-1 assumptions (read before starting)

Phase 1 reworks apps/agent-worker/src/user-container.ts to a per-username mutex + handle API. This plan is written against:

// apps/agent-worker/src/user-container.ts (Phase-1 shape)
export type BoxHandle = { boxName: string; release: () => void }; // release is idempotent
export const acquireUserBox: (args: {
  username: string; workdir: string; containerHome: string;
}) => Promise<BoxHandle>;

If Phase 1 kept the pre-Phase-1 signature (acquireUserBox(...) => Promise<string> + releaseUserBox(username), as in commit b092955), adapt every handle.release() in this plan to releaseUserBox(username) and treat the returned string as boxName. Check the real signature in user-container.ts before writing Task 5, and follow it.

Global Constraints

Next tests: jsdom + @testing-library/react. Component tests live in apps/next/tests/component/*.test.tsx, unit tests in apps/next/tests/unit/*.test.ts. Run cd apps/next && bun run test:component and bun run test:unit. Mock convex/react, next/navigation, and sonner exactly as apps/next/tests/component/render.test.tsx:10-38 does. Monaco (CodeEditor) and xterm do not render under jsdom — mock those child components in page tests (see Task 9).
Worker tests: apps/agent-worker/tests/unit/*.test.ts (vitest). Run cd apps/agent-worker && bun run test:unit. Match apps/agent-worker/tests/unit/terminal-token.test.ts for token tests. Docker-dependent code (status/lifecycle/exec) is not unit-testable without a daemon — document manual verification instead; unit-test only the pure logic (path containment, token parsing, tree building against a real tmpdir).
Conventional commits, one per task (e.g. feat(worker): user-scoped box terminal token). Commit only after the task's tests pass.
Browser NEVER receives a worker token. Tokens are minted server-side in Next route handlers after Convex auth. The username is derived server-side from the authed user (api.userEnvironment.getMine), never taken from the request body/query — so a user can only ever reach their own box.
Reuse, don't duplicate. Extract the xterm wiring from workspace-terminal.tsx into a shared XtermSession (Task 8) before building the box terminal; reuse FileTree and CodeEditor verbatim on /machine. Reuse safeHomeJoin-style realpath containment for /box/file.

Reference facts (verified against the codebase)

Job terminal token (apps/agent-worker/src/terminal-token.ts): format ${expiresAtMs}.${jobId}.${hmacHex}, HMAC over ${expiresAtMs}.${jobId}. Minted in apps/next/src/lib/agent-worker-proxy.ts:20 (mintTerminalToken). Secret precedence (both sides): SPOON_AGENT_TERMINAL_SECRET → SPOON_AGENT_WORKER_INTERNAL_TOKEN → SPOON_WORKER_TOKEN. Worker reads it as env.terminalSecret (apps/agent-worker/src/env.ts:43-47); Next builds it in terminalSecret() (agent-worker-proxy.ts:12-15).
Worker HTTP auth: requireAuth (apps/agent-worker/src/server.ts:45-51) compares the Authorization: Bearer header to env.internalToken. Routes are dispatched in startWorkerServer (server.ts:59-186); only /health, /cleanup, and /jobs/:id/* exist today. The jobRoute regex (server.ts:53-57) is the pattern to mirror for /box/*.
Worker WS: attachTerminalServer (terminal.ts:157-181) handles server.on('upgrade'), matches /^\/jobs\/([^/]+)\/terminal$/, verifies the token, then calls bridge(ws, jobId). Only enabled when env.runtime === 'docker'.
bridge() (terminal.ts:21-150): resolves the workspace via getTerminalWorkspace(jobId), acquires the box via acquireUserBox, then spawns <runtime> exec -i -e TERM=... -e HOME=... -w <containerRepo> <box> /bin/bash -lc 'exec script -qfc <launcher> /dev/null'. The launcher does stty rows.. cols.. then exec tmux new-session -A -s spoon (or bash -il). Input/resize buffered until the exec is ready; cleanup() kills the proc and releases the box.
Box home layout (apps/agent-worker/src/worker.ts:1315-1334): username = userEnv?.username ?? 'user'; homeDir = path.resolve(env.workdir, 'homes', username); containerHome = path.posix.join('/home', username). The box is acquired with { username, workdir: homeDir, containerHome }.
Box container (apps/agent-worker/src/runtime/docker.ts): userContainerName(username) = spoon-box-<sanitized> (docker.ts:338-339); ensureUserContainer({username,workdir,containerHome}) starts spoon-box-* with --memory 4g --cpus 2 mounting the home (docker.ts:341-383); stopWorkspaceContainer(name) = rm -f (docker.ts:438-442); inspectWorkspaceContainer(name) runs inspect (docker.ts:444-453). Container runtime binary = containerRuntime().
Username resolution (server): api.userEnvironment.getMine (packages/backend/convex/userEnvironment.ts:15-34) returns { enabled, username, firstName, dotfilesRepoUrl, dotfilesRepoRef, setupCommand } where username = settings?.homeUsername ?? deriveHomeUsername(user?.name) — the exact value the worker uses. This is the authoritative source for the caller's box username on the Next side.
Existing file/tree worker fns (worker.ts): listWorkspaceTree (1440), readWorkspaceFile (1477), writeWorkspaceFile (1483) — model the box variants on these but root at homeDir and use realpath containment.
Reusable UI: FileTree (apps/next/src/components/agent-workspace/file-tree.tsx) props { tree: FileTreeNode|null; selectedPath?; expandedPaths: string[]; onSelect; onToggleDirectory }. CodeEditor (code-editor.tsx:33-51) props { path?; content; savedContent; readOnly; vimEnabled; onSave; onChange; onVimEnabledChange }. FileTreeNode, FileResponse, DiffResponse live in agent-workspace/types.ts. Nav items are declared in apps/next/src/components/layout/header/index.tsx:23-45 (NavItem = { href; icon; label; external? }).

Task 1: Worker — user-scoped box terminal token (mint helper is Next-side; verify is worker-side)

Add a user-scoped token alongside the existing job-scoped one. It must be unambiguous from the 3-part job token, so it embeds a literal box segment.

Files:

apps/agent-worker/src/terminal-token.ts — add verifyBoxTerminalToken; keep verifyTerminalToken unchanged.
apps/agent-worker/tests/unit/box-terminal-token.test.ts — new test.

Interfaces:

Token format (Produces/Consumes): ${expiresAtMs}.box.${username}.${hmacSha256Hex}, HMAC over the string ${expiresAtMs}.box.${username}. Four dot-separated parts (job token has three), so the two can never collide. Username is [a-z0-9_-]+ (already sanitized by deriveHomeUsername) and therefore never contains ..
Produces:
```
export const verifyBoxTerminalToken = (
  token: string,
  username: string,
  secret: string,
): boolean;
```
Returns true only when: token has exactly 4 parts, part[1] === 'box', part[2] === username, expiry not passed, and the HMAC (over ${parts[0]}.box.${parts[2]}) matches via timingSafeEqual on equal-length buffers. Empty token or empty secret → false.

Steps:

Write apps/agent-worker/tests/unit/box-terminal-token.test.ts mirroring terminal-token.test.ts: a local mintBox(username, expiresAt, secret) helper producing the 4-part token; cases: accepts valid unexpired username-matched token; rejects expired; rejects token minted for another username; rejects wrong secret; rejects malformed/empty; rejects a 3-part job token passed to verifyBoxTerminalToken (cross-scheme confusion guard).
Run cd apps/agent-worker && bun run test:unit → FAIL (function missing).
Implement verifyBoxTerminalToken in terminal-token.ts reusing the existing signature() helper. Split on ., require parts.length === 4 && parts[1] === 'box', check parts[2] === username, parse+check expiry, timingSafeEqual.
Run tests → PASS.
Commit: feat(worker): user-scoped box terminal token verification.

Task 2: Worker — box helper module (home paths, status, lifecycle by username)

Centralize per-user box path derivation, status inspection, and lifecycle so both HTTP routes and the WS bridge share one source of truth.

Files:

apps/agent-worker/src/box.ts — new module.
apps/agent-worker/src/runtime/docker.ts — add inspectUserBoxStatus.
apps/agent-worker/tests/unit/box-paths.test.ts — new test (pure path logic only).

Interfaces:

Produces (box.ts):

export const boxHomePaths = (username: string) => ({
  homeDir: string,        // path.resolve(env.workdir, 'homes', username)
  containerHome: string,  // path.posix.join('/home', username)
});

export type BoxStatus = {
  running: boolean;
  image: string | null;
  startedAt: string | null;      // ISO from container State.StartedAt, null if stopped/absent
  memoryLimitBytes: number | null;
  containerName: string;         // userContainerName(username)
};
export const getBoxStatus = (username: string) => Promise<BoxStatus>;

export const startBox = (username: string) => Promise<void>;   // ensureUserContainer(boxHomePaths)
export const stopBox = (username: string) => Promise<void>;    // stop container + drop registry entry
export const restartBox = (username: string) => Promise<void>; // stopBox then startBox

Produces (docker.ts):
```
export const inspectUserBoxStatus = (username: string) => Promise<{
  running: boolean; image: string | null; startedAt: string | null; memoryLimitBytes: number | null;
}>;
```
Implementation: execa(containerRuntime(), ['inspect','--format','{{.State.Running}}|{{.Config.Image}}|{{.State.StartedAt}}|{{.HostConfig.Memory}}', userContainerName(username)], { reject:false, stdin:'ignore' }). On non-zero exit (no such container) return { running:false, image:null, startedAt:null, memoryLimitBytes:null }. Parse the pipe-split line; running = field==='true'; memoryLimitBytes = Number(field) || null (0 means unlimited → null); startedAt null when not running.
Consumes: ensureUserContainer, stopWorkspaceContainer, userContainerName (docker.ts); env.workdir (env.ts). stopBox must also clear the in-memory registry so a stopped box isn't considered "held" — add export const resetBox = (username: string) => void to user-container.ts (clears the idle timer and deletes the map entry) and call it from stopBox.

Steps:

Add resetBox(username) to user-container.ts (clear box.idleTimer, boxes.delete(username)).
Write apps/agent-worker/tests/unit/box-paths.test.ts: assert boxHomePaths('gib') returns homeDir ending in homes/gib and containerHome === '/home/gib'; assert two usernames produce distinct homeDirs. (Do NOT test docker-touching fns here.)
Run cd apps/agent-worker && bun run test:unit → FAIL.
Implement inspectUserBoxStatus in docker.ts, then box.ts (boxHomePaths, getBoxStatus wrapping inspectUserBoxStatus + userContainerName, startBox/stopBox/restartBox).
Run tests → PASS.
Manual verification (docker required), record in the commit body: with the worker running and a box up, curl -H "Authorization: Bearer $TOKEN" localhost:3921/box/status?user=<you> (after Task 4) shows running:true + image + startedAt; after POST /box/lifecycle {"action":"stop"} status flips to running:false; start brings it back.
Commit: feat(worker): per-user box status and lifecycle helpers.

Task 3: Worker — home-scoped file access (tree/read/write with realpath containment)

Home-scoped equivalents of listWorkspaceTree/readWorkspaceFile/writeWorkspaceFile, rooted at the user's homeDir, hardened against symlink escape (reuse the Phase-1 containment approach: resolve fs.realpath of the parent and re-check containment; reject symlinked write targets).

Files:

apps/agent-worker/src/box.ts — add tree/read/write.
apps/agent-worker/tests/unit/box-files.test.ts — new test against a real tmpdir.

Interfaces:

Produces:
```
export const listBoxTree = (username: string) => Promise<FileTreeNode>;
export const readBoxFile = (username: string, relPath: string) => Promise<string>;
export const writeBoxFile = (username: string, relPath: string, content: string) => Promise<{ success: true }>;
// internal: safeHomePath(homeDir, relPath) => Promise<string>  (realpath-checked absolute path)
```
- listBoxTree: walk homeDir (root node name: '~', path: ''), skipping a small ignore set: .git, node_modules, .cache, .local/share/Trash, .npm, .bun, .cargo, dist, build, .next. Cap traversal (e.g. skip directories once total node count exceeds ~5000) so a huge home can't hang the request. Node shape matches FileTreeNode from agent-workspace/types.ts ({ name; path; type: 'file'|'directory'; children? }).
- readBoxFile/writeBoxFile: resolve via safeHomePath. writeBoxFile mkdir -p the parent, then write. No job/diff/event recording (there is no job) — this is the key difference from writeWorkspaceFile.
- safeHomePath: lexical resolve, then realpath the existing ancestor and re-check startsWith(homeDir + sep); throw Refusing to access path outside home: <relPath> on escape. For writes, if the target itself exists and is a symlink, reject.
Consumes: boxHomePaths, node:fs/promises (stat, readdir, readFile, writeFile, mkdir, realpath), FileTreeNode type (import from where the worker declares it — check worker.ts for the local FileTreeNode type; if not exported, define a matching local type in box.ts).

Steps:

Write apps/agent-worker/tests/unit/box-files.test.ts: create a tmp home dir (os.tmpdir() + mkdtemp), stub env.workdir so boxHomePaths('t').homeDir points at it (either set SPOON_AGENT_WORKDIR before importing, or test safeHomePath directly by exporting it). Cases: writeBoxFile then readBoxFile round-trips; readBoxFile('../../etc/passwd') rejects; a symlink inside home pointing outside is rejected on read AND write; listBoxTree includes a created file and excludes node_modules.
Run cd apps/agent-worker && bun run test:unit → FAIL.
Implement safeHomePath, listBoxTree, readBoxFile, writeBoxFile in box.ts.
Run tests → PASS.
Commit: feat(worker): home-scoped box file tree/read/write with realpath containment.

Task 4: Worker — `/box/*` HTTP routes

Wire the box helpers into the HTTP server, authed by the internal bearer token like every other route.

Files:

apps/agent-worker/src/server.ts — add box route dispatch (import from ./box).

Interfaces (route → response shape):

GET /box/status?user=<username> → 200 { status: BoxStatus }.
POST /box/lifecycle?user=<username> body { action: 'start' | 'stop' | 'restart' } → 200 { status: BoxStatus } (return the post-action status; unknown action → 400 { error }).
GET /box/tree?user=<username> → 200 { tree: FileTreeNode }.
GET /box/file?user=<username>&path=<rel> → 200 { path, content }.
PUT /box/file?user=<username> body { path, content } → 200 { success: true }.
Missing/empty user → 400 { error: 'Missing user' }.

Notes: the user query param is trusted here because this route is only reachable with the internal bearer token, and the only caller (Next, Task 7) derives it from Convex auth. Add a boxRoute(pathname) matcher (mirror jobRoute, server.ts:53-57) matching /^\/box\/(status|lifecycle|tree|file)$/, and dispatch inside the existing try/catch in startWorkerServer before the jobRoute block. Read user via url.searchParams.get('user').

Steps:

(No new unit test — this is HTTP glue over already-tested helpers; verify manually.) Add the boxRoute matcher and the five handlers in server.ts, importing getBoxStatus, startBox, stopBox, restartBox, listBoxTree, readBoxFile, writeBoxFile from ./box. Reuse sendJson/parseJson.
cd apps/agent-worker && bun run build (or bun run typecheck) → passes.
Manual verification (record in commit body): with a box for your username, curl -H "Authorization: Bearer $SPOON_WORKER_TOKEN" 'localhost:3921/box/status?user=<you>' returns status JSON; curl -X PUT ... 'localhost:3921/box/file?user=<you>' -d '{"path":"spoon-test.txt","content":"hi"}' then GET .../box/file?...&path=spoon-test.txt round-trips; GET .../box/tree?user=<you> lists the home.
Commit: feat(worker): /box status, lifecycle, tree, and file HTTP routes.

Task 5: Worker — `/box/terminal` WebSocket bridge

Add a user-scoped terminal WS that acquires the box and opens a login shell rooted at ~. Extract the shared PTY-bridging logic so the job and box bridges don't duplicate the exec/buffer/cleanup dance.

Files:

apps/agent-worker/src/terminal.ts — add bridgeBox, refactor shared exec bridging into a helper, add the /box/terminal upgrade match.

Interfaces:

WS route: GET /box/terminal?token=<boxToken>. Verify with verifyBoxTerminalToken(token, username, env.terminalSecret) — but the username comes from the token itself (parse part[2] after a successful format check), since the token is what authorizes the connection. Concretely: parse the token, extract the candidate username, then if (!verifyBoxTerminalToken(token, candidateUsername, env.terminalSecret)) reject. On success call bridgeBox(ws, candidateUsername).
bridgeBox(ws, username): derive { homeDir, containerHome } = boxHomePaths(username); const handle = await acquireUserBox({ username, workdir: homeDir, containerHome }); ensure a login shell works even for a brand-new home by writing a minimal .bash_profile if absent (reuse the snippet from user-environment.ts:62-68, or extract that into a shared ensureBashProfile(homeDir) and call it here); spawn the same exec -i shell as bridge() but with -w <containerHome> (root at ~, not a repo) and env TERM + HOME=<containerHome> only (no per-job secrets — the box terminal is not job-scoped). Reuse the buffered-input + resize + cleanup logic; cleanup() calls handle.release() (or releaseUserBox(username) per the Phase-1 assumption).

Refactor detail: extract from the current bridge() (terminal.ts:21-150) a helper like:

const runShellBridge = (ws, boxName, opts: { cwd: string; envFlags: string[]; getSize: () => {cols:number;rows:number} }) => { /* buffered input, spawn, forward, exit */ };

so both bridge(ws, jobId) and bridgeBox(ws, username) call it. Keep behavior identical for the job path (regression-guard by leaving its manual smoke test intact).

Steps:

In attachTerminalServer (terminal.ts:157-181), add a second upgrade match /^\/box\/terminal$/: read token, parse username, verify with verifyBoxTerminalToken, on failure 401 + destroy, on success wss.handleUpgrade(... => bridgeBox(ws, username)). Keep the existing /jobs/:id/terminal branch working.
Implement bridgeBox and the shared runShellBridge (refactor bridge to use it). Add ensureBashProfile (shared with user-environment.ts or duplicated minimal).
cd apps/agent-worker && bun run test:unit → existing tests still PASS; bun run build/typecheck passes.
Manual verification (record in commit body): after Task 8/9 the browser flow covers this; for now, mint a box token by hand and connect a websocat/wscat client to ws://localhost:3921/box/terminal?token=..., confirm you land in ~ (pwd shows /home/<you>), typing works, resize works (stty size reflects the client), and disconnect releases the box (status idle-reaps after boxIdleMs).
Commit: feat(worker): user-scoped /box/terminal websocket bridge.

Task 6: Next — box proxy helpers (mint token, resolve username, proxy)

Server-only helpers that mint the box token and proxy /box/* to the worker, deriving the username from Convex auth so the caller can only reach their own box.

Files:

apps/next/src/lib/agent-worker-proxy.ts — add mintBoxTerminalToken, resolveBoxUsername, proxyBox, withBox.
apps/next/tests/unit/box-terminal-token.test.ts — new unit test for the mint format.

Interfaces:

Produces:

// Mirrors mintTerminalToken (agent-worker-proxy.ts:20). Returns null if secret/WS base unset.
export const mintBoxTerminalToken = (username: string): { url: string; expiresAt: number } | null;
//   payload = `${expiresAt}.box.${username}`; token = `${payload}.${hmacHex}`;
//   url = `${wsBase}/box/terminal?token=${encodeURIComponent(token)}`  (NOTE: /box/terminal, no jobId)

// Resolves the authed caller's own box username, or an unauthorized/again response.
export const resolveBoxUsername = (): Promise<
  | { ok: true; username: string }
  | { ok: false; response: NextResponse }
>;
//   uses convexAuthNextjsToken() then fetchQuery(api.userEnvironment.getMine, {}, { token }) -> .username

// Proxies to the worker with the internal token, always appending user=<derived username>.
export const proxyBox = (
  username: string, action: 'status'|'lifecycle'|'tree'|'file',
  init?: RequestInit, search?: URLSearchParams,
): Promise<NextResponse>;

// Convenience wrapper: resolve username -> run handler, 401/500 on failure (mirror withOwnedJob).
export const withBox = (
  handler: (username: string) => Promise<Response>,
): Promise<Response>;

Consumes: terminalSecret() and workerToken() (existing, agent-worker-proxy.ts:12-15,41-42), env.NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL, env.SPOON_AGENT_WORKER_URL, convexAuthNextjsToken, fetchQuery, api.userEnvironment.getMine.

Steps:

Write apps/next/tests/unit/box-terminal-token.test.ts: import createHmac, replicate the expected token string for a fixed secret+username+expiry, and assert mintBoxTerminalToken's url contains /box/terminal?token= and the token has 4 dot-parts with parts[1]==='box'. To make the secret deterministic, set process.env.SPOON_AGENT_TERMINAL_SECRET and NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL before import (match how apps/next/tests/unit/environment.test.ts handles env). If importing agent-worker-proxy.ts pulls in server-only/Convex and fails under vitest, extract the pure token-building into a tiny buildBoxToken(username, secret, wsBase, now) and unit-test that instead (keep mintBoxTerminalToken a thin wrapper). Prefer the extraction — it keeps the test hermetic.
Run cd apps/next && bun run test:unit → FAIL.
Implement the four helpers (+ buildBoxToken if extracted). resolveBoxUsername returns { ok:false, response: 401 } when no auth token; proxyBox mirrors proxyWorker (agent-worker-proxy.ts:95-131) but targets /box/${action} and force-sets user in the search params from the derived username.
Run tests → PASS.
Commit: feat(web): box terminal token mint + ownership proxy helpers.

Task 7: Next — `/api/box/*` route handlers

App-Router handlers that derive the caller's username and proxy to the worker.

Files (all new):

apps/next/src/app/api/box/status/route.ts — GET.
apps/next/src/app/api/box/lifecycle/route.ts — POST.
apps/next/src/app/api/box/tree/route.ts — GET.
apps/next/src/app/api/box/file/route.ts — GET + PUT.
apps/next/src/app/api/box/terminal-token/route.ts — GET.

Interfaces (client contract):

GET /api/box/status → { status: BoxStatus }.
POST /api/box/lifecycle body { action:'start'|'stop'|'restart' } → { status: BoxStatus }.
GET /api/box/tree → { tree: FileTreeNode }.
GET /api/box/file?path=<rel> → { path, content }; PUT /api/box/file body { path, content } → { success:true }.
GET /api/box/terminal-token → { url, expiresAt } or 503 { error } when unconfigured (mirror agent-jobs/[jobId]/terminal-token/route.ts).
All return 401 when unauthenticated (via resolveBoxUsername/withBox).

Implementation: each handler uses withBox/resolveBoxUsername. Example (tree):

export const GET = async () => await withBox(async (username) => await proxyBox(username, 'tree', { method: 'GET' }));

terminal-token uses resolveBoxUsername then mintBoxTerminalToken(username) (503 if null). file forwards path and request body like agent-jobs/[jobId]/file/route.ts:1-28. lifecycle forwards the JSON body.

Steps:

Create the five route files following the agent-jobs/[jobId]/* handlers as templates but with withBox/resolveBoxUsername instead of withOwnedJob.
cd apps/next && bun run typecheck (or the repo's lint/build) → passes.
Manual verification (record in commit body): signed in, hit /api/box/status in the browser/devtools → 200 with your box status; signed out → 401. PUT+GET /api/box/file round-trips a file into your home.
Commit: feat(web): /api/box status, lifecycle, tree, file, terminal-token routes.

Task 8: Next — extract a reusable `XtermSession` component

Factor the xterm wiring out of workspace-terminal.tsx so /machine and the workspace both use one implementation (DRY the WS/fit/fonts/theme logic). This also sets up Task 10's persistence work.

Files:

apps/next/src/components/agent-workspace/xterm-session.tsx — new shared component (move the themes + effect from workspace-terminal.tsx:16-209).
apps/next/src/components/agent-workspace/workspace-terminal.tsx — becomes a thin wrapper over XtermSession.
apps/next/tests/component/xterm-session.test.tsx — new (behavioral, jsdom-limited).

Interfaces:

Produces:

type XtermStatus = 'idle' | 'connecting' | 'connected' | 'closed' | 'error' | 'unconfigured';
export const XtermSession = (props: {
  active: boolean;                 // mount/connect gate
  tokenUrl: string;                // e.g. `/api/box/terminal-token` or `/api/agent-jobs/${jobId}/terminal-token`
  sessionKey: string;              // identity for the connection effect deps (jobId or username)
  waitingLabel?: string;           // shown when active is false-but-expected (see Task 10)
  copyOnSelect?: boolean;          // Task 11
}) => JSX.Element;

Preserve current behavior: lazy-import @xterm/xterm, @xterm/addon-fit, @xterm/addon-web-links; fetch tokenUrl for { url }; open WS; fit() after term.open(); refit on document.fonts.ready / Nerd-Font load (Phase-1 sizing fix — keep it); theme swap without teardown; reconnect button. Move the two ITheme objects into this file.

WorkspaceTerminal keeps its { jobId, active } signature and renders <XtermSession active={active} tokenUrl={/api/agent-jobs/${jobId}/terminal-token} sessionKey={jobId} />.

Steps:

Write apps/next/tests/component/xterm-session.test.tsx. Since xterm can't render in jsdom, mock the three @xterm/* dynamic imports (vi.mock('@xterm/xterm', ...) returning a fake Terminal class with open/loadAddon/onData/onResize/dispose/refresh spies, etc.) and stub global.WebSocket. Assert: with active:false it renders the waiting label and does NOT construct a WebSocket; with active:true it fetches tokenUrl and opens a WebSocket to the returned url. (Mock fetch to return { url: 'ws://x' }.) Keep assertions coarse — this is a smoke/behavior guard, not pixel-level.
Run cd apps/next && bun run test:component → FAIL.
Extract XtermSession, rewrite WorkspaceTerminal as the wrapper. Ensure the existing terminal tab still works (agent-workspace-shell.tsx:664-667 passes active).
Run tests → PASS.
Commit: refactor(web): extract reusable XtermSession from WorkspaceTerminal.

Task 9: Next — `/machine` page + nav entry ("My Machine")

The standalone dev-box surface: status card + lifecycle controls, home file browser (FileTree + CodeEditor), and a full-page ~-rooted terminal.

Files (new unless noted):

apps/next/src/app/(app)/machine/page.tsx — the page.
apps/next/src/components/machine/machine-shell.tsx — client shell (status card, actions, split file-browser/editor + terminal). Mirrors the load/save/tree logic from agent-workspace-shell.tsx but against /api/box/* and with no job concepts.
apps/next/src/components/machine/box-status-card.tsx — status + start/stop/restart buttons.
apps/next/src/components/layout/header/index.tsx — add the /machine nav item (edit, not new).
apps/next/tests/component/machine-shell.test.tsx — new component test.

Interfaces / behavior:

Nav: add { href: '/machine', icon: <lucide icon, e.g. ServerorHardDrive>, label: 'Machine' } to the authenticated navItems array (index.tsx:23-45), placed after Threads.
Status card: fetch GET /api/box/status on mount + after each lifecycle action; poll every ~10s. Show running/stopped badge, image, uptime (derive from startedAt), memory cap (format memoryLimitBytes → GiB, or "unlimited" when null). Buttons call POST /api/box/lifecycle with start/stop/restart; disable while a request is in flight; toast.error on failure and only toast.success after the awaited response resolves (per the repo's error-handling principle). Refresh status from the response.
File browser: reuse FileTree (props exactly as agent-workspace-shell.tsx:574-580) fed by GET /api/box/tree; open files via GET /api/box/file?path=; edit in CodeEditor; save via PUT /api/box/file. Reuse the OpenFileState/openFile/loadFile/writeFileContent shapes from agent-workspace-shell.tsx (copy the minimal subset — no diff/UI-state persistence needed). readOnly={false}, vimEnabled local state.
Terminal: <XtermSession active tokenUrl="/api/box/terminal-token" sessionKey={username} waitingLabel="Starting your machine…" />, laid out full-page (own tab or a resizable pane — a Radix Tabs with "Files" and "Terminal", or a two-pane layout). The page is ~-rooted by construction (worker bridgeBox uses containerHome).
Username: the page reads api.userEnvironment.getMine via useQuery for display (firstName/username), but all box access is server-derived — the client never sends the username.

Steps:

Write apps/next/tests/component/machine-shell.test.tsx: mock convex/react (useQuery → { username:'gib', firstName:'Gib', enabled:true }), next/navigation, sonner, and @/components/agent-workspace/xterm-session (render <div>terminal</div>) and @/components/agent-workspace/code-editor (Monaco won't run in jsdom). Mock global.fetch so /api/box/status returns a stopped box and /api/box/tree returns a small tree. Assert: the status card renders "Stopped" and a "Start" button; clicking "Start" POSTs /api/box/lifecycle with {action:'start'} (assert on the fetch mock); the file tree renders a known file; the terminal placeholder renders.
Run cd apps/next && bun run test:component → FAIL.
Build box-status-card.tsx, machine-shell.tsx, machine/page.tsx, and add the nav item. The page is a server component that renders the client MachineShell (like other (app) pages); auth-gating follows the existing (app) layout — no special guard needed beyond what the layout provides.
Run tests → PASS.
Manual verification (record in commit body): sign in, open /machine. (1) Status card shows correct running/stopped + image + uptime + mem cap; Start/Stop/Restart each change the status within a few seconds. (2) Terminal drops you into ~ (pwd = /home/<you>), typing + resize work, output survives idle. (3) File browser lists your home; open+edit+save a file, confirm it persists (re-open shows the change; cat in the terminal confirms).
Commit: feat(web): /machine dev-box page with status, terminal, and home file browser.

Task 10: Next — persistent workspace terminal (survive tab switches)

Make the workspace Terminal tab keep its PTY/scrollback when switching Radix tabs, refit + resize-send when it becomes visible, and distinguish "waiting for workspace" from "connecting".

Files:

apps/next/src/components/agent-workspace/agent-workspace-shell.tsx — the Terminal TabsContent (currently 660-668).
apps/next/src/components/agent-workspace/xterm-session.tsx — add refit-on-visible + waiting handling.
apps/next/tests/component/xterm-session.test.tsx — extend from Task 8.

Behavior / interfaces:

Persistence: give the Terminal TabsContent forceMount and hide it with CSS when inactive instead of unmounting (Radix TabsContent unmounts by default — that destroys the PTY, audit item #15). Pattern: <TabsContent value='terminal' forceMount className={cn('m-0 min-h-0 flex-1 overflow-hidden', activeWorkspaceTab !== 'terminal' && 'hidden')}>. The other tabs stay as-is. Keep <XtermSession active={workspaceReady} ... /> mounted regardless of tab so the WS/PTY persists; use a separate visible prop for fit behavior.
Refit on visible: add a visible: boolean prop to XtermSession. When it transitions to true, call fitAddon.fit() inside requestAnimationFrame and send a resize frame (the Phase-1 pattern). Keep active as the connect gate (workspaceReady), visible as the CSS-visibility gate (activeWorkspaceTab === 'terminal'). In the shell, pass active={workspaceReady} and visible={activeWorkspaceTab === 'terminal'}.
Waiting vs connecting: XtermSession status starts 'idle'; when active is false but the workspace is expected (job pending), show waitingLabel ("Waiting for workspace…") — distinct from 'connecting' (token fetched, WS opening). The shell already computes workspacePending/workspaceReady (agent-workspace-shell.tsx:109-122); pass an appropriate waitingLabel.

Steps:

Extend xterm-session.test.tsx: assert that toggling visible false→true triggers a fit() call on the mocked fit addon and sends a resize frame over the mocked WebSocket; assert that with active:false + a waitingLabel, the waiting label renders and no WebSocket is constructed; assert the component stays mounted (WS not closed) when visible goes true→false while active stays true.
Run cd apps/next && bun run test:component → FAIL.
Add the visible prop + refit effect to XtermSession; update WorkspaceTerminal to forward visible; change the Terminal TabsContent in agent-workspace-shell.tsx to forceMount + CSS-hide, and mount <WorkspaceTerminal active={workspaceReady} visible={activeWorkspaceTab==='terminal'} .../> (update WorkspaceTerminal props to accept visible).
Run tests → PASS.
Manual verification (record in commit body): open a running thread workspace, go to Terminal, run top (or type a long-running output), switch to Editor then back — the session and scrollback are intact (not a fresh shell), and the terminal is correctly sized (not quarter-size) on return. Before a worker claims the job, the Terminal tab shows "Waiting for workspace…", not "Connecting…".
Commit: feat(web): persistent workspace terminal across tab switches.

Task 11: Next — terminal QoL (copy-on-select; confirm web-links)

Web-links is already loaded (workspace-terminal.tsx:144 / carried into XtermSession). Add a copy-on-select toggle available in both terminals.

Files:

apps/next/src/components/agent-workspace/xterm-session.tsx — copy-on-select via term.onSelectionChange.
apps/next/tests/component/xterm-session.test.tsx — extend.

Interfaces / behavior:

Add a copyOnSelect?: boolean prop (already declared in Task 8) with a small toggle control in the terminal header (a button/checkbox). When enabled, on term.onSelectionChange copy term.getSelection() to the clipboard (navigator.clipboard.writeText, guarded — clipboard may be unavailable/denied; swallow errors). Persist the toggle in localStorage (spoon.terminal.copyOnSelect) so it sticks across sessions. Confirm WebLinksAddon is loaded in XtermSession (it is — keep it; it makes URLs clickable).

Steps:

Extend xterm-session.test.tsx: mock navigator.clipboard.writeText; with copyOnSelect on, simulate the mocked terminal's selection-change callback with a non-empty selection and assert writeText was called with it; with it off, assert not called. Assert the header toggle renders and flipping it updates behavior.
Run cd apps/next && bun run test:component → FAIL.
Implement the toggle + onSelectionChange handler + localStorage persistence in XtermSession; ensure WorkspaceTerminal and the /machine terminal both expose it.
Run tests → PASS.
Manual verification (record in commit body): in /machine and a workspace terminal, enable copy-on-select, select text → it's on the clipboard; a printed URL is clickable (web-links) and opens in a new tab.
Commit: feat(web): terminal copy-on-select toggle and web-links polish.

Final verification (run before declaring the phase done)

cd apps/agent-worker && bun run test:unit — all green (token, box-paths, box-files).
cd apps/next && bun run test:unit && bun run test:component — all green.
Typecheck/lint per repo scripts for both apps.
End-to-end manual smoke on a docker-enabled deployment: /machine status+lifecycle+terminal+file-edit all work; workspace terminal persists across tab switches and is correctly sized; a user signed in as A cannot reach user B's box (all /api/box/* derive the username server-side — confirm by inspecting that no request carries a username).

Self-review against the Phase 3 spec

✅ /machine page: status (running/stopped, image, uptime, mem cap), start/stop/restart, full-page ~-rooted terminal, home file browser reusing FileTree/CodeEditor — Tasks 9 (+2,3,4 backing).
✅ Worker user-scoped HMAC tokens + /box/tree|file|terminal + status/lifecycle endpoints — Tasks 1–5.
✅ Same ownership-proxy pattern (Next mints after Convex auth, browser never sees worker secret) — Tasks 6–7; username derived server-side (a user can only reach their own box).
✅ Persistent workspace terminal (forceMount + CSS hide, refit-on-visible, waiting-vs-connecting) — Task 10.
✅ Terminal QoL (web-links, copy-on-select) — Task 11.
✅ Secret precedence + payload around username+expiry — Task 1/6.

40 KiB Raw Blame History Unescape Escape

Spoon Phase 3 — Dev Box as a First-Class Surface: Implementation Plan

Phase-1 assumptions (read before starting)

Global Constraints

Reference facts (verified against the codebase)

Task 1: Worker — user-scoped box terminal token (mint helper is Next-side; verify is worker-side)

Task 2: Worker — box helper module (home paths, status, lifecycle by username)

Task 3: Worker — home-scoped file access (tree/read/write with realpath containment)

Task 4: Worker — /box/* HTTP routes

Task 5: Worker — /box/terminal WebSocket bridge

Task 6: Next — box proxy helpers (mint token, resolve username, proxy)

Task 7: Next — /api/box/* route handlers

Task 8: Next — extract a reusable XtermSession component

Task 9: Next — /machine page + nav entry ("My Machine")

Task 10: Next — persistent workspace terminal (survive tab switches)

Task 11: Next — terminal QoL (copy-on-select; confirm web-links)

Final verification (run before declaring the phase done)

Self-review against the Phase 3 spec

40 KiB

Raw Blame History

Task 4: Worker — `/box/*` HTTP routes

Task 5: Worker — `/box/terminal` WebSocket bridge

Task 7: Next — `/api/box/*` route handlers

Task 8: Next — extract a reusable `XtermSession` component

Task 9: Next — `/machine` page + nav entry ("My Machine")