diff --git a/README.md b/README.md index adbf62a..1f10691 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,8 @@

Spoon is a self-hostable fork maintenance cockpit built around managed forks, - durable maintenance threads, and OpenCode-powered workspaces. + durable maintenance threads, and a persistent per-user dev box that agents, + terminals, and project commands all run inside.

@@ -61,14 +62,38 @@ in the data model, but GitHub is the active automation surface today. Spoon keeps raw GitHub drift visible while also tracking ignored upstream changes so irrelevant commits do not keep a fork permanently actionable. -- **OpenCode workspaces** - Agent work happens in an isolated workspace with a file tree, browser editor, - diff viewer, command panel, logs, artifacts, and draft PR actions. +- **One persistent per-user box** + Every user owns a single long-running Fedora container `spoon-box-{username}` + with a persistent home. Every thread's agent turn, terminal session, and + project command `docker exec`s into that same box — there is no per-job + throwaway container. + +- **Three agent runtimes** + Codex, OpenCode, and Claude Code all run inside the box behind one adapter + interface, selected per thread and validated at queue time against the AI + provider profile. + +- **Workspaces** + Agent work happens in a workspace with a file tree, browser editor, diff + viewer, command panel, logs, artifacts, an interactive terminal, and draft PR + actions. + +- **My Machine** + A `/machine` surface to start/stop/restart the box, open a `~`-rooted + terminal, and browse the persistent home. + +- **GitHub webhooks** + A signature-verified webhook keeps drift fresh on `push` and flips connection + status on `installation` changes; the hourly cron is a fallback. + +- **Notifications** + An in-app bell plus preference-gated transactional email for maintenance + threads, agent turns, needed input, sync failures, and connection re-auth. - **User-owned providers and secrets** - AI provider profiles, Codex/OpenCode auth, and per-Spoon project secrets are - encrypted. Secrets are redacted from logs and refused from commits when - materialized into env files. + AI provider profiles, Codex/OpenCode/Anthropic auth, and per-Spoon project + secrets are encrypted. Secrets are written `0600`, redacted from logs, and + refused from commits when materialized into env files. - **Draft PR handoff** Code changes become branches and draft pull requests. Spoon does not @@ -138,12 +163,23 @@ custom work exists.

-OpenCode workspaces +The box and workspaces Spoon's optional agent worker is designed to run outside Convex actions. The -worker claims queued jobs, clones the current GitHub fork, creates a branch, -starts an isolated workspace, and exposes workspace operations to the Next app -through server-only API proxies. +worker claims queued jobs, clones the current GitHub fork into the owner's +persistent home, creates a branch, and exposes workspace operations to the Next +app through server-only API proxies. + +Everything runs inside **one long-running container per user**, +`spoon-box-{username}`. The box is a Fedora image (`docker/agent-job.Dockerfile`) +started with `sleep infinity` and a `--memory 4g` / `--cpus 2` cap. Each +thread's agent turn, terminal, and project commands `docker exec` into that same +box; the box's `/home/{username}` is a bind-mounted persistent home so dotfiles, +installed tools, shell history, and thread checkouts under `~/Code/{spoon}/{branch}` +survive across sessions. The box is reference-counted by the worker and reaped +after `SPOON_AGENT_BOX_IDLE_MS` idle (default 30m). There is no per-job +`docker run --rm` container and no separate `opencode serve` container — that +path was removed. Workspace capabilities: @@ -154,16 +190,18 @@ Workspace capabilities: - inspect diffs - send thread messages to the agent - run configured commands +- use an interactive terminal (xterm.js → box PTY; see + [docs/agent-terminal.md](docs/agent-terminal.md)) - store logs and artifacts - push a branch - open a draft PR The browser never receives worker tokens and never talks directly to the worker -or job container. +or the box. -Worker cleanup is available in `Settings -> Worker`. It can delete old terminal -workspace records and ask the active worker to remove orphaned job containers -and inactive work directories. +Worker cleanup is available in `Settings -> Worker`. It can delete stale +workspace records and ask the active worker to remove orphaned containers and +inactive work directories (persistent per-user homes are preserved). Local worker development: @@ -174,20 +212,53 @@ bun dev:next:worker bun dev:next:worker:staging ``` -Local host-run worker commands still load env through Infisical, then -`scripts/dev-agent-worker` selects Podman when available, falls back to Docker, -and publishes the OpenCode server on a localhost port so the host worker can -reach the job container. Override with: +Local host-run worker commands load env through Infisical, then +`scripts/dev-agent-worker` selects Podman when available and falls back to +Docker. Override the container CLI with: ```env SPOON_AGENT_CONTAINER_RUNTIME=podman -SPOON_AGENT_CONTAINER_ACCESS=host_port ```
-Production agent runtime images +Agent runtimes + +Three agent CLIs run inside the box behind one `AgentRuntime` adapter interface +(`apps/agent-worker/src/runtime/*-adapter.ts`), selected by `job.runtime`: + +| Runtime | CLI | Selected for | +| ---------- | ----------------------------------- | -------------------------------------------------- | +| `codex` | `@openai/codex` | OpenAI providers and Codex ChatGPT-login snapshots | +| `opencode` | `opencode-ai` | OpenAI-compatible API-key providers (default) | +| `claude` | `@anthropic-ai/claude-code@2.1.207` | Anthropic providers | + +The runtime is validated at queue time (`runtimeSupport.ts` + +`agentJobs.insertJob`): a job is rejected unless the resolved runtime is one the +AI provider profile supports. `runtimesForProfile` maps profiles to runtimes — +ChatGPT-login snapshots → `codex` only; `openai` API keys → `opencode`/`codex`; +Anthropic API keys → `claude`/`opencode`; Anthropic `anthropic_oauth_json` +credential snapshots → `claude` only; every other OpenAI-compatible API key → +`opencode`. + +Auth is materialized into the box before a turn: + +- **Codex ChatGPT-login** profiles get the encrypted `auth.json` written to + `CODEX_HOME/.codex/auth.json`. +- **Anthropic API-key** profiles authenticate via `ANTHROPIC_API_KEY` in the + environment (no file needed). +- **Anthropic `anthropic_oauth_json`** credential-snapshot profiles get the + encrypted OAuth blob written to `~/.claude/.credentials.json`. +- **API-key** profiles run through OpenCode. + +All materialized auth files are written `0600`. Treat those saved auth files +like a password and only use them on trusted workers. + +
+ +
+Production runtime images Gitea CI builds and pushes three production images: @@ -197,31 +268,36 @@ git.gbrown.org/gib/spoon-agent-worker:latest git.gbrown.org/gib/spoon-agent-job:latest ``` -The worker image is the long-running service that polls Convex. The job image is -the isolated workbench that the worker launches for each agent job. For the MVP, -production should use the repo-provided JS/TS workbench image: +The worker image is the long-running service that polls Convex. The +`spoon-agent-job` image is the **box** image — the per-user Fedora dev box the +worker `exec`s into (it is not launched fresh per job). Point the worker at it +with: ```env SPOON_AGENT_JOB_IMAGE="git.gbrown.org/gib/spoon-agent-job:latest" ``` -The job image includes Node 22, Bun, pnpm and yarn through Corepack, npm, git, -ripgrep, Python, build tools, OpenCode, and the Codex CLI. It is not the forked -project's production runtime; it is the agent execution environment. +The box image is Fedora 41 with Node, Bun, pnpm and yarn, npm, git, ripgrep, +Python, build tools, plus interactive tooling (neovim, tmux, fzf, fd, bat, eza, +zoxide, gh, gum, oh-my-posh) and the pinned agent CLIs OpenCode, Codex, and +Claude Code. It is not the forked project's production runtime; it is the agent +execution environment. Production worker runtime requirements: - `spoon-agent-worker` must run as a separate service. -- The worker needs `/var/run/docker.sock` mounted so it can launch job - containers. -- Production should keep `SPOON_AGENT_CONTAINER_RUNTIME=docker` and - `SPOON_AGENT_CONTAINER_ACCESS=network`. -- The production Docker host must be logged into `git.gbrown.org` so worker jobs - can pull the private `spoon-agent-job` image. +- The worker needs `/var/run/docker.sock` mounted so it can create and `exec` + into per-user boxes on the host daemon. +- Production should keep `SPOON_AGENT_CONTAINER_RUNTIME=docker`. +- `SPOON_AGENT_HOST_WORKDIR` must equal the absolute host path backing + `SPOON_AGENT_WORKDIR` (identical inside and outside the worker container) so + the host daemon can bind-mount per-user homes under `${WORKDIR}/homes/{username}`. +- The production Docker host must be logged into `git.gbrown.org` so the worker + can pull the private `spoon-agent-job` box image. - `SPOON_WORKER_TOKEN` must match the value stored in Convex production env. - `spoon-next` needs `SPOON_AGENT_WORKER_URL=http://spoon-agent-worker:3921` and - `SPOON_AGENT_WORKER_INTERNAL_TOKEN` so Next API routes can proxy workspace - file, diff, message, command, and draft PR actions. + `SPOON_AGENT_WORKER_INTERNAL_TOKEN` so Next API routes can proxy workspace and + box file, diff, message, command, terminal, and draft PR actions. - `spoon-agent-worker` also needs `GITHUB_APP_ID` and `GITHUB_APP_PRIVATE_KEY`. If the private key is stored in a single-line dotenv value, encode newlines as literal `\n` characters so the worker can restore the PEM before using it. @@ -240,22 +316,57 @@ curl -H "Authorization: Bearer $SPOON_AGENT_WORKER_INTERNAL_TOKEN" \ Deployment readiness checklist: 1. Production Convex env has `SPOON_WORKER_TOKEN`, `SPOON_ENCRYPTION_KEY`, - GitHub App env, and Convex Auth signing keys. + GitHub App env (including `GITHUB_APP_WEBHOOK_SECRET`), and Convex Auth + signing keys. 2. Compose env has `SPOON_AGENT_WORKER_URL`, `SPOON_AGENT_WORKER_INTERNAL_TOKEN`, `SPOON_AGENT_JOB_IMAGE`, and the GitHub - App private key. + App private key; the `spoon-next` image is built with + `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL`. 3. The production Docker host can pull private images from `git.gbrown.org`. -4. `Settings -> Worker` reports the expected job image, runtime, network, and - active workspace count. +4. `Settings -> Worker` reports the expected box image, runtime, network, and + active box count. 5. The first test thread uses a configured API-key provider or a trusted Codex login profile. -6. If a worker restart leaves stale workspace state, use the workspace recovery - panel or `Settings -> Worker` cleanup. +6. If a worker restart leaves stale state, use the workspace recovery panel or + `Settings -> Worker` cleanup. -API-key based AI provider profiles run through OpenCode. Codex ChatGPT login -profiles run through the Codex CLI: Spoon writes the encrypted `auth.json` into -the isolated job workspace as `CODEX_HOME/.codex/auth.json` before execution. -Treat that saved auth file like a password and only use it on trusted workers. +
+ +
+GitHub webhooks + +Spoon exposes a signature-verified webhook as a Convex `httpAction` at +`POST /webhooks/github` (`packages/backend/convex/githubWebhooks.ts`, +`http.ts`). Configure the GitHub App webhook URL as +`/webhooks/github` and set `GITHUB_APP_WEBHOOK_SECRET`; the +endpoint fails closed (HTTP 503) when no secret is configured and rejects any +payload whose `x-hub-signature-256` does not verify. + +- `push` → a targeted drift refresh for every Spoon tracking that repository. +- `installation` / `installation_repositories` → the matching connection's + status flips to `revoked` (deleted/suspended), `active` (created/unsuspended), + or otherwise `needs_reauth`, surfaced in `Settings -> Integrations`. + +The hourly `refreshDueSpoons` cron remains a fallback, so drift still refreshes +even if a webhook is missed. + +
+ +
+Notifications + +Spoon delivers an in-app bell plus preference-gated transactional email +(`packages/backend/convex/notifications.ts`). Five kinds are emitted: +`maintenance_thread`, `agent_turn_finished`, `agent_needs_input`, `sync_failed`, +and `connection_needs_reauth`. + +- Every event inserts an in-app notification (bell badge via `unreadCount`, + list via `listMine`, `markRead` / `markAllRead`). +- Email is sent through UseSend (`USESEND_*`) only when the user has an email + and the matching per-kind preference is not disabled. Unset preferences + default to enabled. +- Per-kind email toggles live in `Settings -> Notifications`. Web push is out of + scope.
@@ -284,45 +395,54 @@ Treat that saved auth file like a password and only use it on trusted workers.
Core tables -| Table | Purpose | -| ------------------------ | --------------------------------------------------------- | -| `spoons` | Managed fork records | -| `threads` | Durable maintenance and work conversations | -| `threadMessages` | Messages inside threads | -| `syncRuns` | Upstream checks, sync attempts, and maintenance decisions | -| `ignoredUpstreamChanges` | Intentional ignore records that affect effective drift | -| `gitConnections` | Git provider connection metadata | -| `spoonRepositoryStates` | Latest cached upstream/fork state | -| `spoonCommits` | Cached upstream and fork-only commits | -| `spoonPullRequests` | Cached fork/upstream pull requests | -| `spoonSecrets` | Encrypted per-Spoon environment variables | -| `spoonAgentSettings` | Per-Spoon runtime, branch, command, and env-file settings | -| `aiProviderProfiles` | Encrypted provider/auth profiles used by OpenCode | -| `agentJobs` | Worker-executed workspace jobs and PR lifecycle | -| `agentJobEvents` | Append-only worker event log | -| `agentJobArtifacts` | Diffs, summaries, command output, PR body drafts | -| `agentWorkspaceChanges` | Recorded user, agent, and command file changes | +| Table | Purpose | +| ------------------------- | ----------------------------------------------------------- | +| `spoons` | Managed fork records | +| `threads` | Durable maintenance and work conversations | +| `threadMessages` | Messages inside threads | +| `syncRuns` | Upstream checks, sync attempts, and maintenance decisions | +| `ignoredUpstreamChanges` | Intentional ignore records that affect effective drift | +| `gitConnections` | Git provider connection metadata | +| `spoonRepositoryStates` | Latest cached upstream/fork state | +| `spoonCommits` | Cached upstream and fork-only commits | +| `spoonPullRequests` | Cached fork/upstream pull requests | +| `spoonSecrets` | Encrypted per-Spoon environment variables | +| `spoonAgentSettings` | Per-Spoon runtime, branch, command, and env-file settings | +| `aiProviderProfiles` | Encrypted provider/auth profiles (Codex/OpenCode/Anthropic) | +| `agentJobs` | Worker-executed workspace jobs and PR lifecycle | +| `agentJobEvents` | Append-only worker event log | +| `agentJobArtifacts` | Diffs, summaries, command output, PR body drafts | +| `agentWorkspaceChanges` | Recorded user, agent, and command file changes | +| `userDotfiles` | Encrypted per-user dotfiles overlay + repo/setup config | +| `userEnvironment` | Per-user box/home environment config | +| `notifications` | In-app notification rows (bell) | +| `notificationPreferences` | Per-user email notification toggles |
Important routes -| Route | Purpose | -| --------------------------------- | --------------------------------------- | -| `/` | Public product landing page | -| `/dashboard` | Maintenance overview | -| `/spoons` | Managed fork list | -| `/spoons/new` | Manual/GitHub Spoon creation | -| `/spoons/[spoonId]` | Spoon detail dashboard | -| `/spoons/[spoonId]/agent/[jobId]` | Interactive workspace | -| `/threads` | Global thread queue | -| `/threads/[threadId]` | Thread detail | -| `/settings/profile` | User profile settings | -| `/settings/integrations` | GitHub and service integration settings | -| `/settings/ai-providers` | AI/OpenCode provider profiles | +| Route | Purpose | +| --------------------------------- | ----------------------------------------------- | +| `/` | Public product landing page | +| `/dashboard` | Maintenance overview | +| `/spoons` | Managed fork list | +| `/spoons/new` | Manual/GitHub Spoon creation | +| `/spoons/[spoonId]` | Spoon detail dashboard | +| `/spoons/[spoonId]/agent/[jobId]` | Interactive workspace | +| `/threads` | Global thread queue | +| `/threads/[threadId]` | Thread detail | +| `/machine` | My Machine: box status, terminal, home | +| `/settings/profile` | User profile settings | +| `/settings/integrations` | GitHub and service integration settings | +| `/settings/ai-providers` | AI provider profiles (Codex/OpenCode/Anthropic) | +| `/settings/dotfiles` | Per-user dotfiles overlay + repo | +| `/settings/notifications` | Email notification preferences | -Legacy `/updates` and `/agents` routes redirect into `/threads`. +The Convex `httpAction` `POST /webhooks/github` (mounted at +`/webhooks/github`) handles GitHub App webhooks. Legacy +`/updates` and `/agents` routes redirect into `/threads`.
@@ -423,16 +543,17 @@ not call Infisical.
Public Next variables -| Variable | Used for | -| --------------------------------- | ------------------------------------------- | -| `NEXT_PUBLIC_SITE_URL` | Canonical Spoon web URL | -| `NEXT_PUBLIC_CONVEX_URL` | Convex client URL | -| `NEXT_PUBLIC_DEPLOYMENT_URL` | Convex dashboard/deployment URL when needed | -| `NEXT_PUBLIC_PLAUSIBLE_URL` | Plausible analytics endpoint | -| `NEXT_PUBLIC_SENTRY_DSN` | Browser Sentry DSN | -| `NEXT_PUBLIC_SENTRY_URL` | Sentry instance URL | -| `NEXT_PUBLIC_SENTRY_ORG` | Sentry organization | -| `NEXT_PUBLIC_SENTRY_PROJECT_NAME` | Sentry project name | +| Variable | Used for | +| --------------------------------------- | ------------------------------------------------------------------- | +| `NEXT_PUBLIC_SITE_URL` | Canonical Spoon web URL | +| `NEXT_PUBLIC_CONVEX_URL` | Convex client URL | +| `NEXT_PUBLIC_SPOON_AGENT_WORKER_WS_URL` | Browser-facing worker WS base for the terminal (**build-time** var) | +| `NEXT_PUBLIC_DEPLOYMENT_URL` | Convex dashboard/deployment URL when needed | +| `NEXT_PUBLIC_PLAUSIBLE_URL` | Plausible analytics endpoint | +| `NEXT_PUBLIC_SENTRY_DSN` | Browser Sentry DSN | +| `NEXT_PUBLIC_SENTRY_URL` | Sentry instance URL | +| `NEXT_PUBLIC_SENTRY_ORG` | Sentry organization | +| `NEXT_PUBLIC_SENTRY_PROJECT_NAME` | Sentry project name |
@@ -487,15 +608,16 @@ not call Infisical. | `SPOON_AGENT_WORKER_URL` | Internal worker HTTP URL used by Next | | `SPOON_AGENT_WORKER_HTTP_PORT` | Worker HTTP port | | `SPOON_AGENT_WORKER_INTERNAL_TOKEN` | Server-only token for Next-to-worker proxy | -| `SPOON_AGENT_JOB_IMAGE` | Agent job container image | +| `SPOON_AGENT_TERMINAL_SECRET` | HMAC secret for terminal/box tokens (falls back to the internal/worker token) | +| `SPOON_AGENT_JOB_IMAGE` | Per-user box (Fedora) image the worker `exec`s into | | `SPOON_AGENT_RUNTIME` | Runtime mode, currently Docker/Podman-oriented | | `SPOON_AGENT_CONTAINER_RUNTIME` | Container CLI used by worker, `docker`/`podman` | -| `SPOON_AGENT_CONTAINER_ACCESS` | `network` in prod, `host_port` for host dev | +| `SPOON_AGENT_BOX_IDLE_MS` | Idle time before a per-user box is reaped (default `1800000`) | | `SPOON_AGENT_MAX_CONCURRENT_JOBS` | Worker concurrency limit | -| `SPOON_AGENT_JOB_TIMEOUT_MS` | Job timeout | -| `SPOON_AGENT_WORKDIR` | Worker work directory | +| `SPOON_AGENT_JOB_TIMEOUT_MS` | Agent turn timeout | +| `SPOON_AGENT_WORKDIR` | Worker work directory; per-user homes live under `homes/{username}` | | `SPOON_AGENT_HOST_WORKDIR` | Host path matching `SPOON_AGENT_WORKDIR` when the worker runs in Docker and controls the host Docker socket | -| `SPOON_AGENT_NETWORK` | Optional job container network | +| `SPOON_AGENT_NETWORK` | Optional box container network | @@ -523,12 +645,20 @@ not call Infisical. - GitHub drift refresh, commit cache, PR cache, and sync-run history - Effective drift and ignored upstream change records - Global Threads page and Spoon-scoped Threads tab -- OpenCode/Codex-oriented agent worker and browser workspace foundation +- Persistent per-user box the worker `exec`s into for agent turns, terminal, and + commands +- Three agent runtimes (Codex, OpenCode, Claude Code) behind one adapter, with + queue-time runtime validation +- My Machine surface (box status/start/stop/restart, terminal, home browser) +- Interactive workspace terminal (xterm.js → box PTY) +- Per-user dotfiles overlay + optional dotfiles repo +- Signature-verified GitHub webhook with hourly cron fallback +- In-app + email notifications with per-user preferences - Monaco editor with optional Vim mode - Diff viewer, command panel, worker logs, and artifacts - Encrypted Spoon secrets and bulk `.env` import -- Encrypted AI provider profiles, including Codex auth JSON and API-key - provider support +- Encrypted AI provider profiles, including Codex/Anthropic auth JSON and + API-key provider support - Authentik, GitHub, and password auth through Convex Auth - Self-hosted Convex/Postgres deployment model diff --git a/apps/agent-worker/src/agent-events.ts b/apps/agent-worker/src/agent-events.ts index d6f07a6..e5663fd 100644 --- a/apps/agent-worker/src/agent-events.ts +++ b/apps/agent-worker/src/agent-events.ts @@ -357,6 +357,114 @@ export const normalizeCodexJsonLine = ( return events; }; +// Claude Code `--output-format stream-json` emits one JSON object per line: +// `system:init` announces the session id, `assistant` messages carry text and +// tool_use blocks, `user` messages carry tool_result blocks, and `result` +// terminates the turn (success with final text, or an `error_*` subtype). +export const normalizeClaudeJsonLine = ( + line: string, +): NormalizedAgentEvent[] => { + if (!line.trim()) return []; + let parsed: unknown; + try { + parsed = JSON.parse(line) as unknown; + } catch { + return [{ kind: 'status', status: line }]; + } + const event = asRecord(parsed); + if (!event) return [{ kind: 'status', status: line }]; + const type = stringify(event.type); + const subtype = stringify(event.subtype); + const events: NormalizedAgentEvent[] = []; + + if (type === 'system') { + const sessionId = stringify(event.session_id); + if (subtype === 'init' && sessionId) { + events.push({ kind: 'session', sessionId }); + } + if (events.length === 0) { + events.push({ kind: 'status', status: subtype || 'system' }); + } + return events; + } + + if (type === 'assistant') { + const message = asRecord(event.message); + const content = message?.content; + let text = ''; + if (Array.isArray(content)) { + for (const block of content) { + const record = asRecord(block); + if (!record) continue; + const blockType = stringify(record.type); + if (blockType === 'text') { + text += textFromPart(record); + } else if (blockType === 'tool_use') { + events.push({ + kind: 'tool_started', + name: stringify(record.name ?? 'tool'), + input: JSON.stringify(record.input ?? {}), + }); + } + } + } + if (text) events.unshift({ kind: 'assistant_delta', content: text }); + if (events.length === 0) { + events.push({ kind: 'status', status: 'assistant' }); + } + return events; + } + + if (type === 'content_block_delta') { + const delta = asRecord(event.delta); + const text = delta ? textFromPart(delta) : ''; + if (text) events.push({ kind: 'assistant_delta', content: text }); + if (events.length === 0) { + events.push({ kind: 'status', status: type }); + } + return events; + } + + if (type === 'user') { + const message = asRecord(event.message); + const content = message?.content; + if (Array.isArray(content)) { + for (const block of content) { + const record = asRecord(block); + if (!record || stringify(record.type) !== 'tool_result') continue; + events.push({ + kind: 'tool_completed', + name: 'tool', + output: toolOutputFromRecord(record), + }); + } + } + if (events.length === 0) { + events.push({ kind: 'status', status: 'user' }); + } + return events; + } + + if (type === 'result') { + const sessionId = stringify(event.session_id); + if (sessionId) events.push({ kind: 'session', sessionId }); + if (subtype === 'success') { + events.push({ + kind: 'assistant_completed', + content: stringify(event.result), + }); + } else { + events.push({ + kind: 'error', + message: stringify(event.result ?? event.error ?? subtype), + }); + } + return events; + } + + return [{ kind: 'status', status: type || line }]; +}; + export const normalizeOpenCodeEvent = ( input: unknown, ): NormalizedAgentEvent[] => { @@ -464,3 +572,56 @@ export const normalizeOpenCodeEvent = ( } return events; }; + +// `opencode run --format json` emits newline-delimited JSON: streamed events share +// the server's `{ type, properties }` SSE envelope, plus a final result object that +// carries the session id and the assistant's completed text. +export const normalizeOpenCodeRunLine = ( + line: string, +): NormalizedAgentEvent[] => { + if (!line.trim()) return []; + let parsed: unknown; + try { + parsed = JSON.parse(line) as unknown; + } catch { + return [{ kind: 'status', status: line }]; + } + const record = asRecord(parsed); + if (!record) return [{ kind: 'status', status: line }]; + if ('type' in record && 'properties' in record) { + return normalizeOpenCodeEvent(record); + } + const events: NormalizedAgentEvent[] = []; + const sessionId = record.sessionID ?? record.sessionId; + if (typeof sessionId === 'string') { + events.push({ kind: 'session', sessionId }); + } + const parts = record.parts; + let text = ''; + if (Array.isArray(parts)) { + text = parts + .map((part) => { + const record = asRecord(part); + return record ? textFromPart(record) : ''; + }) + .join(''); + } + if (!text) { + const message = asRecord(record.message); + text = + textFromPart(record) || + (message ? stringify(message.content ?? message.text) : ''); + } + if (text) { + const externalMessageId = stringify(record.messageID ?? record.id); + events.push({ + kind: 'assistant_completed', + content: text, + ...(externalMessageId ? { externalMessageId } : {}), + }); + } + if (events.length === 0) { + events.push({ kind: 'status', status: line }); + } + return events; +}; diff --git a/apps/agent-worker/src/box.ts b/apps/agent-worker/src/box.ts new file mode 100644 index 0000000..3a3e854 --- /dev/null +++ b/apps/agent-worker/src/box.ts @@ -0,0 +1,153 @@ +import { mkdir, readFile, readdir, stat, writeFile } from 'node:fs/promises'; +import path from 'node:path'; + +import { env } from './env'; +import { assertContainedRealPath } from './path-containment'; +import { + ensureUserContainer, + inspectUserBoxStatus, + stopWorkspaceContainer, + userContainerName, +} from './runtime/docker'; +import { resetBox } from './user-container'; + +// Matches FileTreeNode in apps/next/src/components/agent-workspace/types.ts. +type FileTreeNode = { + name: string; + path: string; + type: 'file' | 'directory'; + children?: FileTreeNode[]; +}; + +// Per-user box home layout, shared by the HTTP routes and the terminal WS bridge +// so path derivation lives in exactly one place (must match worker.ts). +export const boxHomePaths = (username: string) => ({ + homeDir: path.resolve(env.workdir, 'homes', username), + containerHome: path.posix.join('/home', username), +}); + +export type BoxStatus = { + running: boolean; + image: string | null; + startedAt: string | null; + memoryLimitBytes: number | null; + containerName: string; +}; + +export const getBoxStatus = async (username: string): Promise => { + const status = await inspectUserBoxStatus(username); + return { ...status, containerName: userContainerName(username) }; +}; + +export const startBox = async (username: string): Promise => { + const { homeDir, containerHome } = boxHomePaths(username); + await ensureUserContainer({ username, workdir: homeDir, containerHome }); +}; + +export const stopBox = async (username: string): Promise => { + await stopWorkspaceContainer(userContainerName(username)); + resetBox(username); +}; + +export const restartBox = async (username: string): Promise => { + await stopBox(username); + await startBox(username); +}; + +// Directories never surfaced in the box file tree — VCS internals, package +// caches and build output that would bloat (or hang) the walk. +const BOX_TREE_IGNORE = new Set([ + '.git', + 'node_modules', + '.cache', + '.npm', + '.bun', + '.cargo', + 'dist', + 'build', + '.next', +]); + +// Home-relative paths (POSIX) that are always skipped regardless of depth. +const BOX_TREE_IGNORE_PATHS = new Set(['.local/share/Trash']); + +// Guard against a huge home hanging the request; stop descending past this. +const BOX_TREE_MAX_NODES = 5000; + +// Realpath-checked absolute path inside the user's home. Rejects lexical `..` +// escapes and symlinks that resolve outside home (reuses the Phase-1 +// containment helper); for writes, also rejects a symlinked final target. +const safeHomePath = ( + homeDir: string, + relPath: string, + opts: { forWrite?: boolean } = {}, +): Promise => + assertContainedRealPath(homeDir, relPath, { ...opts, label: 'home' }); + +export const listBoxTree = async (username: string): Promise => { + const { homeDir } = boxHomePaths(username); + let count = 0; + + const buildChildren = async ( + absoluteDir: string, + relativeDir: string, + ): Promise => { + const entries = await readdir(absoluteDir); + const nodes = await Promise.all( + entries + .sort((a, b) => a.localeCompare(b)) + .map(async (entry): Promise => { + const relativePath = relativeDir ? `${relativeDir}/${entry}` : entry; + if ( + BOX_TREE_IGNORE.has(entry) || + BOX_TREE_IGNORE_PATHS.has(relativePath) + ) { + return null; + } + if (count >= BOX_TREE_MAX_NODES) return null; + count += 1; + const absolutePath = path.join(absoluteDir, entry); + const stats = await stat(absolutePath).catch(() => null); + if (!stats) return null; + if (stats.isDirectory()) { + return { + name: entry, + path: relativePath, + type: 'directory', + children: await buildChildren(absolutePath, relativePath), + }; + } + return { name: entry, path: relativePath, type: 'file' }; + }), + ); + return nodes.filter((node): node is FileTreeNode => Boolean(node)); + }; + + return { + name: '~', + path: '', + type: 'directory', + children: await buildChildren(homeDir, ''), + }; +}; + +export const readBoxFile = async ( + username: string, + relPath: string, +): Promise => { + const { homeDir } = boxHomePaths(username); + const target = await safeHomePath(homeDir, relPath); + return await readFile(target, 'utf8'); +}; + +export const writeBoxFile = async ( + username: string, + relPath: string, + content: string, +): Promise<{ success: true }> => { + const { homeDir } = boxHomePaths(username); + const target = await safeHomePath(homeDir, relPath, { forWrite: true }); + await mkdir(path.dirname(target), { recursive: true }); + await writeFile(target, content); + return { success: true }; +}; diff --git a/apps/agent-worker/src/codex-process.ts b/apps/agent-worker/src/codex-process.ts new file mode 100644 index 0000000..c6223a2 --- /dev/null +++ b/apps/agent-worker/src/codex-process.ts @@ -0,0 +1,88 @@ +type CommandResult = { exitCode: number; output: string }; + +const launchScript = ` +import os +import sys + +pid_file, job_id, *command = sys.argv[1:] +if os.getpgrp() != os.getpid(): + os.setsid() +os.environ["SPOON_CODEX_JOB_ID"] = job_id +with open(pid_file, "w", encoding="utf-8") as handle: + handle.write(str(os.getpid())) + handle.flush() + os.fsync(handle.fileno()) +os.execvpe(command[0], command, os.environ) +`; + +const abortScript = ` +import os +import signal +import sys +import time + +pid_file, job_id = sys.argv[1:] +pid = None +for _ in range(20): + try: + with open(pid_file, encoding="utf-8") as handle: + pid = int(handle.read().strip()) + break + except FileNotFoundError: + time.sleep(0.1) +if pid is None: + sys.exit(0) + +try: + with open(f"/proc/{pid}/environ", "rb") as handle: + environment = handle.read().split(b"\\0") + marker = f"SPOON_CODEX_JOB_ID={job_id}".encode() + if marker not in environment or os.getpgid(pid) != pid: + raise RuntimeError("Codex process identity did not match the requested job") + os.killpg(pid, signal.SIGTERM) + for _ in range(20): + time.sleep(0.1) + try: + os.killpg(pid, 0) + except ProcessLookupError: + break + else: + os.killpg(pid, signal.SIGKILL) +except ProcessLookupError: + pass +finally: + try: + os.unlink(pid_file) + except FileNotFoundError: + pass +`; + +export const managedCodexCommand = (args: { + jobId: string; + pidFile: string; + command: string[]; +}) => [ + 'python3', + '-c', + launchScript, + args.pidFile, + args.jobId, + ...args.command, +]; + +export const abortManagedCodexProcess = async (args: { + jobId: string; + pidFile: string; + execute: (command: string[]) => Promise; +}) => { + const result = await args.execute([ + 'python3', + '-c', + abortScript, + args.pidFile, + args.jobId, + ]); + if (result.exitCode !== 0) { + throw new Error(`Failed to abort Codex process: ${result.output}`); + } +}; diff --git a/apps/agent-worker/src/env.ts b/apps/agent-worker/src/env.ts index 86b1689..07e79fb 100644 --- a/apps/agent-worker/src/env.ts +++ b/apps/agent-worker/src/env.ts @@ -27,25 +27,14 @@ export const env = { 'docker', containerVolumeOptions: process.env.SPOON_AGENT_CONTAINER_VOLUME_OPTIONS?.trim(), - containerAccess: - process.env.SPOON_AGENT_CONTAINER_ACCESS?.trim() === 'host_port' - ? 'host_port' - : 'network', jobImage: process.env.SPOON_AGENT_JOB_IMAGE?.trim() ?? 'spoon-agent-job:latest', - // Interactive terminal: image for the persistent shell container (defaults to - // the job image), the secret shared with the Next app for verifying terminal - // tokens, and how long an idle terminal container survives before cleanup. - terminalImage: - process.env.SPOON_AGENT_TERMINAL_IMAGE?.trim() ?? - process.env.SPOON_AGENT_JOB_IMAGE?.trim() ?? - 'spoon-agent-job:latest', + // Secret shared with the Next app for verifying interactive terminal tokens. terminalSecret: process.env.SPOON_AGENT_TERMINAL_SECRET?.trim() ?? process.env.SPOON_AGENT_WORKER_INTERNAL_TOKEN?.trim() ?? process.env.SPOON_WORKER_TOKEN?.trim() ?? '', - terminalIdleMs: intEnv('SPOON_AGENT_TERMINAL_IDLE_MS', 1_800_000), // How long a per-user box container survives with no active jobs/terminals. boxIdleMs: intEnv('SPOON_AGENT_BOX_IDLE_MS', 1_800_000), // Dev-only: exit if the parent dev runner dies, so the worker never orphans @@ -60,7 +49,6 @@ export const env = { process.env.SPOON_AGENT_WORKER_INTERNAL_TOKEN?.trim() ?? process.env.SPOON_WORKER_TOKEN?.trim() ?? '', - maxConcurrentJobs: intEnv('SPOON_AGENT_MAX_CONCURRENT_JOBS', 1), jobTimeoutMs: intEnv('SPOON_AGENT_JOB_TIMEOUT_MS', 1_800_000), githubAppId: requiredEnv('GITHUB_APP_ID'), githubPrivateKey: requiredEnv('GITHUB_APP_PRIVATE_KEY').replaceAll( diff --git a/apps/agent-worker/src/heartbeat-controller.ts b/apps/agent-worker/src/heartbeat-controller.ts new file mode 100644 index 0000000..c6e8380 --- /dev/null +++ b/apps/agent-worker/src/heartbeat-controller.ts @@ -0,0 +1,42 @@ +export const createHeartbeatController = (args: { + intervalMs: number; + heartbeat: (jobId: string) => Promise<{ cancelRequested: boolean }>; + onCancel: (jobId: string) => Promise; + onError: (error: unknown) => void; +}) => { + const timers = new Map>(); + const inFlight = new Set(); + + const stop = (jobId: string) => { + const timer = timers.get(jobId); + if (timer) clearInterval(timer); + timers.delete(jobId); + }; + + const tick = async (jobId: string) => { + if (inFlight.has(jobId) || !timers.has(jobId)) return; + inFlight.add(jobId); + try { + const result = await args.heartbeat(jobId); + if (result.cancelRequested) { + stop(jobId); + await args.onCancel(jobId); + } + } catch (error) { + args.onError(error); + } finally { + inFlight.delete(jobId); + } + }; + + return { + start: (jobId: string) => { + stop(jobId); + const timer = setInterval(() => void tick(jobId), args.intervalMs); + timer.unref(); + timers.set(jobId, timer); + }, + stop, + has: (jobId: string) => timers.has(jobId), + }; +}; diff --git a/apps/agent-worker/src/index.ts b/apps/agent-worker/src/index.ts index 2237756..14d5841 100644 --- a/apps/agent-worker/src/index.ts +++ b/apps/agent-worker/src/index.ts @@ -1,5 +1,6 @@ import { env } from './env'; import { startWorkerServer } from './server'; +import { reconcileExistingBoxes } from './user-container'; import { startWorker } from './worker'; // Dev-only watchdog: the dev runner chain (turbo → with-env → dotenv → bash) @@ -26,4 +27,5 @@ if (env.devWatchdog) { } startWorkerServer(); +await reconcileExistingBoxes(); await startWorker(); diff --git a/apps/agent-worker/src/materialize-env.ts b/apps/agent-worker/src/materialize-env.ts new file mode 100644 index 0000000..6dd8106 --- /dev/null +++ b/apps/agent-worker/src/materialize-env.ts @@ -0,0 +1,38 @@ +import { chmod, mkdir, rm, writeFile } from 'node:fs/promises'; +import path from 'node:path'; + +import { assertContainedRealPath } from './path-containment'; + +export type MaterializedSecret = { name: string; value: string }; + +// Format matches the historical `NAME="value"` env-file layout the agent CLIs +// source; values are JSON-quoted so newlines/quotes survive. +export const formatMaterializedEnv = (secrets: MaterializedSecret[]) => + `${secrets + .map((secret) => `${secret.name}=${JSON.stringify(secret.value)}`) + .join('\n')}\n`; + +// Resolves the contained env-file path inside the checkout and writes it 0600 +// so plaintext secrets never sit world/group-readable in the persistent home. +export const writeMaterializedEnv = async ( + repoDir: string, + envFilePath: string, + secrets: MaterializedSecret[], +): Promise => { + const envPath = await assertContainedRealPath(repoDir, envFilePath, { + forWrite: true, + }); + await mkdir(path.dirname(envPath), { recursive: true }); + await writeFile(envPath, formatMaterializedEnv(secrets), { mode: 0o600 }); + // writeFile's mode only applies on create; chmod unconditionally tightens an + // already-existing (possibly 0644) file so plaintext secrets never leak. + await chmod(envPath, 0o600); + return envPath; +}; + +// Removes a previously materialized env file; no-ops when the path is unset or +// already gone so it is safe on every teardown path. +export const removeMaterializedEnv = async (envPath: string | undefined) => { + if (!envPath) return; + await rm(envPath, { force: true }); +}; diff --git a/apps/agent-worker/src/opencode-session.ts b/apps/agent-worker/src/opencode-session.ts deleted file mode 100644 index 7b8ff1c..0000000 --- a/apps/agent-worker/src/opencode-session.ts +++ /dev/null @@ -1,128 +0,0 @@ -import type { OpencodeClient } from '@opencode-ai/sdk'; -import { createOpencodeClient } from '@opencode-ai/sdk'; - -import type { NormalizedAgentEvent } from './agent-events'; -import { normalizeOpenCodeEvent } from './agent-events'; - -export type OpenCodeSession = { - client: OpencodeClient; - sessionId: string; - close: () => void; -}; - -const basicAuth = (username: string, password: string) => - `Basic ${Buffer.from(`${username}:${password}`).toString('base64')}`; - -const modelParts = (model: string) => { - const [rawProviderId, ...rest] = model.split('/'); - const providerID = - rawProviderId && rawProviderId.length > 0 ? rawProviderId : 'openai'; - const modelID = rest.length > 0 ? rest.join('/') : model; - return { - providerID, - modelID, - }; -}; - -export const createOpenCodeSession = async (args: { - baseUrl: string; - password: string; - directory: string; - title: string; - onEvent: (event: NormalizedAgentEvent) => Promise; -}) => { - const abortController = new AbortController(); - const client = createOpencodeClient({ - baseUrl: args.baseUrl, - directory: args.directory, - headers: { - authorization: basicAuth('opencode', args.password), - }, - }); - const created = await client.session.create({ - query: { directory: args.directory }, - body: { title: args.title }, - }); - if (!created.data) { - throw new Error('OpenCode session could not be created.'); - } - const sessionId = created.data.id; - void (async () => { - const events = await client.event.subscribe({ - signal: abortController.signal, - query: { directory: args.directory }, - onSseEvent: (event) => { - for (const normalized of normalizeOpenCodeEvent(event.data)) { - void args.onEvent(normalized); - } - }, - onSseError: (error) => { - void args.onEvent({ - kind: 'error', - message: error instanceof Error ? error.message : String(error), - }); - }, - }); - for await (const event of events.stream) { - for (const normalized of normalizeOpenCodeEvent(event)) { - await args.onEvent(normalized); - } - } - })().catch((error: unknown) => { - if (!abortController.signal.aborted) { - void args.onEvent({ - kind: 'error', - message: error instanceof Error ? error.message : String(error), - }); - } - }); - return { - client, - sessionId, - close: () => abortController.abort(), - } satisfies OpenCodeSession; -}; - -export const promptOpenCodeSession = async (args: { - session: OpenCodeSession; - prompt: string; - model: string; - directory: string; -}) => { - const model = modelParts(args.model); - const result = await args.session.client.session.promptAsync({ - path: { id: args.session.sessionId }, - query: { directory: args.directory }, - body: { - model, - parts: [{ type: 'text', text: args.prompt }], - }, - }); - if (result.error) { - throw new Error('OpenCode prompt was rejected.'); - } -}; - -export const abortOpenCodeSession = async (session: OpenCodeSession) => { - await session.client.session.abort({ - path: { id: session.sessionId }, - }); -}; - -export const replyOpenCodePermission = async (args: { - session: OpenCodeSession; - permissionId: string; - response: 'once' | 'always' | 'reject'; - directory: string; -}) => { - const result = await args.session.client.postSessionIdPermissionsPermissionId( - { - path: { id: args.session.sessionId, permissionID: args.permissionId }, - query: { directory: args.directory }, - body: { response: args.response }, - }, - ); - if (result.error) { - throw new Error('OpenCode permission response was rejected.'); - } -}; diff --git a/apps/agent-worker/src/path-containment.ts b/apps/agent-worker/src/path-containment.ts new file mode 100644 index 0000000..2154c8f --- /dev/null +++ b/apps/agent-worker/src/path-containment.ts @@ -0,0 +1,41 @@ +import { lstat, realpath } from 'node:fs/promises'; +import path from 'node:path'; + +const realpathOrDeepestExisting = async (target: string): Promise => { + let current = target; + const tail: string[] = []; + for (;;) { + try { + const real = await realpath(current); + return path.join(real, ...tail.reverse()); + } catch (error) { + if ((error as { code?: string }).code !== 'ENOENT') throw error; + const parent = path.dirname(current); + if (parent === current) throw error; + tail.push(path.basename(current)); + current = parent; + } + } +}; + +export const assertContainedRealPath = async ( + root: string, + requested: string, + opts: { forWrite?: boolean; label?: string } = {}, +): Promise => { + const lexical = path.resolve(root, requested); + const realRoot = await realpath(root); + const resolved = await realpathOrDeepestExisting(lexical); + if (resolved !== realRoot && !resolved.startsWith(`${realRoot}${path.sep}`)) { + throw new Error( + `Refusing to access path outside ${opts.label ?? 'root'}: ${requested}`, + ); + } + if (opts.forWrite) { + const info = await lstat(lexical).catch(() => null); + if (info?.isSymbolicLink()) { + throw new Error(`Refusing to write through a symlink: ${requested}`); + } + } + return resolved; +}; diff --git a/apps/agent-worker/src/redact.ts b/apps/agent-worker/src/redact.ts index b2e6756..30c8323 100644 --- a/apps/agent-worker/src/redact.ts +++ b/apps/agent-worker/src/redact.ts @@ -1,8 +1,11 @@ -const secretPatterns = [ - /ghs_[A-Za-z0-9_]+/g, - /github_pat_[A-Za-z0-9_]+/g, - /sk-[A-Za-z0-9_-]+/g, - /(client_secret|auth_secret|api_key|token|password)=([^ \n\r]+)/gi, +const secretPatterns: { pattern: RegExp; replacement: string }[] = [ + { pattern: /ghs_[A-Za-z0-9_]+/g, replacement: '[redacted]' }, + { pattern: /github_pat_[A-Za-z0-9_]+/g, replacement: '[redacted]' }, + { pattern: /sk-[A-Za-z0-9_-]+/g, replacement: '[redacted]' }, + { + pattern: /(client_secret|auth_secret|api_key|token|password)=([^ \n\r]+)/gi, + replacement: '$1=[redacted]', + }, ]; export const createRedactor = (values: string[]) => { @@ -12,8 +15,8 @@ export const createRedactor = (values: string[]) => { for (const secret of secrets) { output = output.split(secret).join('[redacted]'); } - for (const pattern of secretPatterns) { - output = output.replace(pattern, '$1=[redacted]'); + for (const { pattern, replacement } of secretPatterns) { + output = output.replace(pattern, replacement); } return output; }; diff --git a/apps/agent-worker/src/runtime/agent-runtime.ts b/apps/agent-worker/src/runtime/agent-runtime.ts new file mode 100644 index 0000000..039dab9 --- /dev/null +++ b/apps/agent-worker/src/runtime/agent-runtime.ts @@ -0,0 +1,49 @@ +import type { NormalizedAgentEvent } from '../agent-events'; +import type { streamExecInContainer } from './docker'; +import type { AdapterWorkspace } from './provider'; + +export type AgentRuntimeName = 'codex' | 'opencode' | 'claude'; + +export type ExecStreamFn = typeof streamExecInContainer; + +export type TurnResult = { + // Assistant text the adapter captured out-of-band (Codex --output-last-message + // file, Claude `result` event). Empty/undefined when everything streamed via onEvent. + finalMessage?: string; + // Runtime session id to persist for continuity (resume/--session/--resume). + sessionId?: string; + // Non-empty when the runtime reported a hard failure the caller must surface. + error?: string; +}; + +export type AgentRuntime = { + readonly name: AgentRuntimeName; + prepareAuth(workspace: AdapterWorkspace): Promise; + runTurn( + workspace: AdapterWorkspace, + prompt: string, + onEvent: (event: NormalizedAgentEvent) => Promise, + ): Promise; + abort(workspace: AdapterWorkspace): Promise; +}; + +export type AdapterFactory = (deps?: { + execStream?: ExecStreamFn; +}) => AgentRuntime; + +const factories = new Map(); + +export const registerAdapter = ( + name: AgentRuntimeName, + factory: AdapterFactory, +): void => { + factories.set(name, factory); +}; + +export const getAdapter = (name: AgentRuntimeName): AgentRuntime => { + const factory = factories.get(name); + if (!factory) { + throw new Error(`No agent runtime adapter registered for "${name}".`); + } + return factory(); +}; diff --git a/apps/agent-worker/src/runtime/auth.ts b/apps/agent-worker/src/runtime/auth.ts new file mode 100644 index 0000000..cec04b5 --- /dev/null +++ b/apps/agent-worker/src/runtime/auth.ts @@ -0,0 +1,20 @@ +import { mkdir, writeFile } from 'node:fs/promises'; +import path from 'node:path'; + +// Normalize + pretty-print an auth JSON blob before writing it with owner-only +// permissions. Shared by the credential-writing adapters (Claude, and available +// to Codex/OpenCode). `label` names the source in the parse-error message. +export const writeJsonFile = async ( + filePath: string, + content: string, + label = 'auth', +) => { + let normalized = content.trim(); + try { + normalized = `${JSON.stringify(JSON.parse(normalized), null, 2)}\n`; + } catch { + throw new Error(`${label} JSON is not valid JSON.`); + } + await mkdir(path.dirname(filePath), { recursive: true }); + await writeFile(filePath, normalized, { mode: 0o600 }); +}; diff --git a/apps/agent-worker/src/runtime/claude-adapter.ts b/apps/agent-worker/src/runtime/claude-adapter.ts new file mode 100644 index 0000000..eb90b2d --- /dev/null +++ b/apps/agent-worker/src/runtime/claude-adapter.ts @@ -0,0 +1,127 @@ +import path from 'node:path'; +import { randomUUID } from 'node:crypto'; + +import type { NormalizedAgentEvent } from '../agent-events'; +import { normalizeClaudeJsonLine } from '../agent-events'; +import { env } from '../env'; +import type { AdapterFactory, TurnResult } from './agent-runtime'; +import { writeJsonFile } from './auth'; +import { + buildMarkedCommand, + killBoxProcessesByMarker, + streamExecInContainer, +} from './docker'; +import type { AdapterWorkspace } from './provider'; +import { claudeEnv, claudeModel, isClaudeOAuthProfile } from './provider'; + +export const createClaudeAdapter: AdapterFactory = (deps) => { + const execStream = deps?.execStream ?? streamExecInContainer; + + const prepareAuth = async (workspace: AdapterWorkspace) => { + // API-key profiles authenticate purely via ANTHROPIC_API_KEY in the + // environment; only OAuth-json profiles need `.credentials.json` on disk. + if (!isClaudeOAuthProfile(workspace.claim)) return; + const secret = workspace.claim.aiProviderProfile?.secret; + if (!secret) { + throw new Error('Claude auth profile is missing credentials.json contents.'); + } + const credentialsPath = path.join( + workspace.homeDir, + '.claude', + '.credentials.json', + ); + await writeJsonFile(credentialsPath, secret, 'Claude auth'); + }; + + const runTurn = async ( + workspace: AdapterWorkspace, + prompt: string, + onEvent: (event: NormalizedAgentEvent) => Promise, + ): Promise => { + workspace.runtime = 'claude'; + workspace.turnMarker = `spoon-turn-${randomUUID()}`; + const marker = workspace.turnMarker; + + // Flags verified against `claude --help` (@anthropic-ai/claude-code@2.1.207): + // `-p`, `--output-format stream-json`, `--verbose`, `--dangerously-skip-permissions`, + // `--model`, and `--resume ` (continuity) all exist. + const argv = [ + 'claude', + '-p', + prompt, + '--output-format', + 'stream-json', + '--verbose', + '--dangerously-skip-permissions', + '--model', + claudeModel(workspace.claim), + ...(workspace.claudeSessionId + ? ['--resume', workspace.claudeSessionId] + : []), + ]; + const command = buildMarkedCommand(marker, argv); + + const aiEnv = claudeEnv(workspace.claim, workspace.containerHome); + const secretEnv = Object.fromEntries( + workspace.claim.secrets.map((secret) => [secret.name, secret.value]), + ); + + let capturedSessionId: string | undefined; + let capturedError: string | undefined; + let finalMessage: string | undefined; + + // execa's own `timeout` only kills the local docker-exec client, not the + // in-box process group. Schedule an explicit marker kill on timeout and clear + // it once the exec resolves. + const killTimer = setTimeout(() => { + void killBoxProcessesByMarker({ + containerName: workspace.boxName, + marker, + }); + }, env.jobTimeoutMs); + + let result; + try { + result = await execStream({ + containerName: workspace.boxName, + containerCwd: workspace.containerRepo, + command, + environment: { ...aiEnv, ...secretEnv }, + redact: workspace.redact, + timeoutMs: env.jobTimeoutMs, + onStdoutLine: async (line) => { + for (const event of normalizeClaudeJsonLine(line)) { + if (event.kind === 'session') capturedSessionId = event.sessionId; + if (event.kind === 'error') capturedError = event.message; + if (event.kind === 'assistant_completed' && event.content) { + finalMessage = event.content; + } + await onEvent(event); + } + }, + onStderrLine: async (line) => { + if (!line.trim()) return; + await onEvent({ kind: 'status', status: line }); + }, + }); + } finally { + clearTimeout(killTimer); + } + + if (result.exitCode !== 0) { + capturedError = result.output; + } + + return { finalMessage, sessionId: capturedSessionId, error: capturedError }; + }; + + const abort = async (workspace: AdapterWorkspace) => { + if (!workspace.turnMarker) return; + await killBoxProcessesByMarker({ + containerName: workspace.boxName, + marker: workspace.turnMarker, + }); + }; + + return { name: 'claude', prepareAuth, runTurn, abort }; +}; diff --git a/apps/agent-worker/src/runtime/codex-adapter.ts b/apps/agent-worker/src/runtime/codex-adapter.ts new file mode 100644 index 0000000..4f93b5f --- /dev/null +++ b/apps/agent-worker/src/runtime/codex-adapter.ts @@ -0,0 +1,193 @@ +import { mkdir, readFile, writeFile } from 'node:fs/promises'; +import path from 'node:path'; +import { randomUUID } from 'node:crypto'; + +import type { NormalizedAgentEvent } from '../agent-events'; +import { normalizeCodexJsonLine } from '../agent-events'; +import { prepareCodexWorkspaceFiles } from '../codex-runtime'; +import { env } from '../env'; +import type { AdapterFactory, TurnResult } from './agent-runtime'; +import { + buildMarkedCommand, + killBoxProcessesByMarker, + streamExecInContainer, +} from './docker'; +import type { AdapterWorkspace } from './provider'; +import { + codexModelArgs, + isCodexLoginProfile, + providerEnvironment, +} from './provider'; + +// Reused verbatim from the worker: normalize + pretty-print an auth JSON blob +// before writing it with owner-only permissions. +const writeJsonFile = async (filePath: string, content: string) => { + let normalized = content.trim(); + try { + normalized = `${JSON.stringify(JSON.parse(normalized), null, 2)}\n`; + } catch { + throw new Error('Codex auth JSON is not valid JSON.'); + } + await mkdir(path.dirname(filePath), { recursive: true }); + await writeFile(filePath, normalized, { mode: 0o600 }); +}; + +const isNoise = (line: string) => { + const trimmed = line.trim(); + return ( + !trimmed || + trimmed === 'Reading additional input from stdin...' || + trimmed.includes('`[features].codex_hooks` is deprecated') + ); +}; + +export const createCodexAdapter: AdapterFactory = (deps) => { + const execStream = deps?.execStream ?? streamExecInContainer; + + const prepareAuth = async (workspace: AdapterWorkspace) => { + // API-key profiles (e.g. OpenAI api_key) authenticate purely via + // OPENAI_API_KEY from providerEnvironment; only ChatGPT-login/JSON-snapshot + // profiles need a Codex `auth.json` on disk. Mirror the Claude adapter, + // which early-returns for non-OAuth profiles. + if (!isCodexLoginProfile(workspace.claim)) return; + const secret = workspace.claim.aiProviderProfile?.secret; + if (!secret) { + throw new Error('Codex auth profile is missing auth.json contents.'); + } + await prepareCodexWorkspaceFiles({ + workdir: workspace.workdir, + repoDir: workspace.repoDir, + }); + const codexAuthPath = path.join(workspace.workdir, '.codex', 'auth.json'); + await writeJsonFile(codexAuthPath, secret); + + // Also seed OpenCode's auth location with the saved JSON for forward + // compatibility if this profile later runs through OpenCode directly. + const openCodeAuthPath = path.join( + workspace.workdir, + '.local', + 'share', + 'opencode', + 'auth.json', + ); + await writeJsonFile(openCodeAuthPath, secret); + }; + + const runTurn = async ( + workspace: AdapterWorkspace, + prompt: string, + onEvent: (event: NormalizedAgentEvent) => Promise, + ): Promise => { + workspace.runtime = 'codex'; + workspace.turnMarker = `spoon-turn-${randomUUID()}`; + const marker = workspace.turnMarker; + + const outputFileName = `last-message-${workspace.claim.job._id}.txt`; + const outputFileHostPath = path.join( + workspace.workdir, + '.codex', + outputFileName, + ); + const outputFileContainerPath = path.posix.join( + workspace.containerHome, + '.codex', + outputFileName, + ); + const codexArgv = workspace.codexSessionId + ? [ + 'codex', + 'exec', + 'resume', + '--json', + ...codexModelArgs(workspace.claim), + '--dangerously-bypass-approvals-and-sandbox', + '--output-last-message', + outputFileContainerPath, + workspace.codexSessionId, + prompt, + ] + : [ + 'codex', + 'exec', + '--json', + ...codexModelArgs(workspace.claim), + '--dangerously-bypass-approvals-and-sandbox', + '--output-last-message', + outputFileContainerPath, + '--cd', + workspace.containerRepo, + prompt, + ]; + const command = buildMarkedCommand(marker, codexArgv); + + const aiEnv = providerEnvironment(workspace.claim, workspace.containerHome); + const secretEnv = Object.fromEntries( + workspace.claim.secrets.map((secret) => [secret.name, secret.value]), + ); + + let capturedSessionId: string | undefined; + let capturedError: string | undefined; + + // execa's own `timeout` only kills the local docker-exec client, not the + // in-box process group. Schedule an explicit marker kill on timeout and + // clear it once the exec resolves. + const killTimer = setTimeout(() => { + void killBoxProcessesByMarker({ + containerName: workspace.boxName, + marker, + }); + }, env.jobTimeoutMs); + + let result; + try { + result = await execStream({ + containerName: workspace.boxName, + containerCwd: workspace.containerRepo, + command, + environment: { ...aiEnv, ...secretEnv }, + redact: workspace.redact, + timeoutMs: env.jobTimeoutMs, + onStdoutLine: async (line) => { + for (const event of normalizeCodexJsonLine(line)) { + if (event.kind === 'session') capturedSessionId = event.sessionId; + if (event.kind === 'error') capturedError = event.message; + await onEvent(event); + } + }, + onStderrLine: async (line) => { + if (isNoise(line)) return; + await onEvent({ kind: 'status', status: line }); + }, + }); + } finally { + clearTimeout(killTimer); + } + + if (result.exitCode !== 0) { + capturedError = result.output; + } + + let finalMessage: string | undefined; + try { + const lastMessage = await readFile(outputFileHostPath, 'utf8'); + if (lastMessage.trim()) finalMessage = lastMessage.trim(); + } catch (error) { + const code = error && typeof error === 'object' ? 'code' in error : false; + if (!code || (error as { code?: string }).code !== 'ENOENT') { + throw error; + } + } + + return { finalMessage, sessionId: capturedSessionId, error: capturedError }; + }; + + const abort = async (workspace: AdapterWorkspace) => { + if (!workspace.turnMarker) return; + await killBoxProcessesByMarker({ + containerName: workspace.boxName, + marker: workspace.turnMarker, + }); + }; + + return { name: 'codex', prepareAuth, runTurn, abort }; +}; diff --git a/apps/agent-worker/src/runtime/docker.ts b/apps/agent-worker/src/runtime/docker.ts index 9d79176..a4b3d75 100644 --- a/apps/agent-worker/src/runtime/docker.ts +++ b/apps/agent-worker/src/runtime/docker.ts @@ -1,6 +1,7 @@ import { mkdir } from 'node:fs/promises'; import path from 'node:path'; import type { Readable } from 'node:stream'; +import Docker from 'dockerode'; import { execa } from 'execa'; import { env } from '../env'; @@ -10,6 +11,13 @@ type CommandResult = { output: string; }; +let dockerClient: Docker | undefined; + +export const getDockerClient = () => { + dockerClient ??= new Docker(); + return dockerClient; +}; + const environmentArgs = (environment: Record) => Object.entries(environment).flatMap(([name, value]) => [ '-e', @@ -95,138 +103,6 @@ export const jobWorkspaceVolumeSpec = ( : `${source}:${containerHome}`; }; -export const runInJobContainer = async (args: { - workdir: string; - containerHome?: string; - containerCwd?: string; - command: string[]; - environment: Record; - redact: (value: string) => string; - timeoutMs: number; -}): Promise => { - await ensureJobImagePulled(); - const result = await execa( - containerRuntime(), - [ - 'run', - '--rm', - '--memory', - '4g', - '--cpus', - '2', - ...networkArgs(), - ...environmentArgs(args.environment), - '-v', - jobWorkspaceVolumeSpec(args.workdir, args.containerHome), - '-w', - args.containerCwd ?? '/workspace/repo', - env.jobImage, - ...args.command, - ], - { - all: true, - reject: false, - stdin: 'ignore', - timeout: args.timeoutMs, - }, - ); - return normalizeRunResult(result, result.all, args.redact); -}; - -export const startWorkspaceContainer = async (args: { - workdir: string; - containerHome?: string; - containerCwd?: string; - containerName: string; - environment: Record; - command?: string[]; - publishTcpPort?: number; -}) => { - await ensureJobImagePulled(); - await execa(containerRuntime(), ['rm', '-f', args.containerName], { - reject: false, - }); - const result = await execa( - containerRuntime(), - [ - 'run', - '-d', - '--name', - args.containerName, - '--memory', - '4g', - '--cpus', - '2', - ...networkArgs(), - ...(args.publishTcpPort - ? ['-p', `127.0.0.1::${args.publishTcpPort}`] - : []), - ...environmentArgs(args.environment), - '-v', - jobWorkspaceVolumeSpec(args.workdir, args.containerHome), - '-w', - args.containerCwd ?? '/workspace/repo', - env.jobImage, - ...(args.command ?? ['sleep', 'infinity']), - ], - { all: true, stdin: 'ignore' }, - ); - return { - containerId: result.stdout.trim(), - containerName: args.containerName, - hostPort: args.publishTcpPort - ? await getPublishedPort(args.containerName, args.publishTcpPort) - : undefined, - }; -}; - -const getPublishedPort = async ( - containerName: string, - containerPort: number, -) => { - const result = await execa( - containerRuntime(), - ['port', containerName, `${containerPort}/tcp`], - { all: true, reject: false, stdin: 'ignore' }, - ); - const output = result.all.trim(); - const match = /:(\d+)\s*$/.exec(output); - if (!match?.[1]) { - throw new Error( - `Could not determine published port for ${containerName}:${containerPort}.`, - ); - } - return match[1]; -}; - -export const execInWorkspaceContainer = async (args: { - containerName: string; - command: string[]; - environment?: Record; - redact: (value: string) => string; - timeoutMs: number; -}): Promise => { - const result = await execa( - containerRuntime(), - [ - 'exec', - ...(args.environment ? environmentArgs(args.environment) : []), - args.containerName, - ...args.command, - ], - { - all: true, - reject: false, - stdin: 'ignore', - timeout: args.timeoutMs, - }, - ); - return { - exitCode: result.exitCode ?? 0, - output: args.redact(result.all), - }; -}; - // Shared line-streaming + result normalization for a started subprocess // (used by both `docker run` and `docker exec` paths). type StreamingSubprocess = { @@ -288,51 +164,6 @@ const streamSubprocess = async ( return normalizeRunResult(result, output.join(''), redact); }; -export const streamInJobContainer = async (args: { - workdir: string; - containerHome?: string; - containerCwd?: string; - command: string[]; - environment: Record; - redact: (value: string) => string; - timeoutMs: number; - onStdoutLine?: (line: string) => Promise; - onStderrLine?: (line: string) => Promise; -}): Promise => { - await ensureJobImagePulled(); - const subprocess = execa( - containerRuntime(), - [ - 'run', - '--rm', - '--memory', - '4g', - '--cpus', - '2', - ...networkArgs(), - ...environmentArgs(args.environment), - '-v', - jobWorkspaceVolumeSpec(args.workdir, args.containerHome), - '-w', - args.containerCwd ?? '/workspace/repo', - env.jobImage, - ...args.command, - ], - { - all: true, - reject: false, - stdin: 'ignore', - timeout: args.timeoutMs, - }, - ); - return streamSubprocess( - subprocess, - args.redact, - args.onStdoutLine, - args.onStderrLine, - ); -}; - // Per-user persistent "box" container that all of a user's threads exec into // (Phase 2). Started once, reused; the home volume persists state across stops. export const userContainerName = (username: string) => @@ -362,6 +193,7 @@ export const ensureUserContainer = async (args: { [ 'run', '-d', + '--init', '--name', name, '--memory', @@ -382,6 +214,127 @@ export const ensureUserContainer = async (args: { return name; }; +// Inspect the per-user box in one shot. Non-zero exit (no such container) yields +// an all-null/false status. `--memory 0` means unlimited, which the Go template +// prints as `0`; we normalize that (and any unparseable value) to null. +export const inspectUserBoxStatus = async ( + username: string, +): Promise<{ + running: boolean; + image: string | null; + startedAt: string | null; + memoryLimitBytes: number | null; +}> => { + const result = await execa( + containerRuntime(), + [ + 'inspect', + '--format', + '{{.State.Running}}|{{.Config.Image}}|{{.State.StartedAt}}|{{.HostConfig.Memory}}', + userContainerName(username), + ], + { reject: false, stdin: 'ignore' }, + ); + if (result.exitCode !== 0) { + return { running: false, image: null, startedAt: null, memoryLimitBytes: null }; + } + const [runningField, imageField, startedAtField, memoryField] = result.stdout + .trim() + .split('|'); + const running = runningField === 'true'; + return { + running, + // eslint-disable-next-line @typescript-eslint/prefer-nullish-coalescing -- empty field means "no image" + image: imageField || null, + // eslint-disable-next-line @typescript-eslint/prefer-nullish-coalescing -- empty field means "not started" + startedAt: running ? startedAtField || null : null, + memoryLimitBytes: Number(memoryField) || null, + }; +}; + +const shellQuote = (value: string) => `'${value.replaceAll("'", "'\\''")}'`; + +// The marker is embedded literally into a `# comment` line and into a `pgrep` +// pattern, so it must not contain shell metacharacters or a newline (either could +// break out of the comment or the pattern). Callers generate it, but validate +// defensively so a malformed marker fails loudly instead of silently corrupting +// the script. +const MARKER_PATTERN = /^[A-Za-z0-9_-]+$/; +const assertMarker = (marker: string) => { + if (!MARKER_PATTERN.test(marker)) { + throw new Error( + `Invalid process marker ${JSON.stringify(marker)}: must match ${String(MARKER_PATTERN)}.`, + ); + } +}; + +// Wraps a CLI argv so the process runs as a new session/process-group leader whose +// bash parent carries the marker in its argv (matchable by `pgrep -f`). We do NOT +// `exec` the CLI: bash tail-exec's a final simple command, which would replace the +// marker-carrying bash with the CLI and make `pgrep -f ` miss the running +// turn. Keeping a trailing `rc=$?; exit "$rc"` after the CLI means the CLI is no +// longer the script's final statement, so bash survives as the group leader while +// still propagating the CLI's exit code (downstream `normalizeRunResult` relies on +// it — a bare trailing `wait` would mask a non-zero CLI failure as 0). The CLI runs +// in the same process group as this bash, so the kill helper's group signal reaches +// it. stdout/stderr fds are inherited by the CLI so streaming still works. +export const buildMarkedCommand = ( + marker: string, + command: string[], +): string[] => { + assertMarker(marker); + const script = [ + `# ${marker}`, + 'exec 0`, whose OWN argv +// contains the marker literal, so `pgrep -f ` matches the kill script (and +// its command-substitution subshells) too. Left unguarded it would `kill -TERM` its +// own process group and die on iteration one, leaking the real target. So we compute +// this script's pgid up front and skip any match sharing it, killing only the +// target's process group. +export const buildKillScript = (marker: string): string => { + assertMarker(marker); + return [ + `self_pgid=$(ps -o pgid= -p $$ | tr -d ' ')`, + `pids=$(pgrep -f ${shellQuote(marker)} || true)`, + `for pid in $pids; do`, + ` pgid=$(ps -o pgid= -p "$pid" | tr -d ' ')`, + ` [ -z "$pgid" ] && continue`, + ` [ "$pgid" = "$self_pgid" ] && continue`, + ` kill -TERM -"$pgid" 2>/dev/null || true`, + `done`, + `sleep 2`, + `for pid in $pids; do`, + ` pgid=$(ps -o pgid= -p "$pid" | tr -d ' ')`, + ` [ -z "$pgid" ] && continue`, + ` [ "$pgid" = "$self_pgid" ] && continue`, + ` kill -KILL -"$pgid" 2>/dev/null || true`, + `done`, + ].join('\n'); +}; + +export const killBoxProcessesByMarker = async (args: { + containerName: string; + marker: string; +}): Promise => { + await execa( + containerRuntime(), + ['exec', args.containerName, 'bash', '-lc', buildKillScript(args.marker)], + { reject: false, stdin: 'ignore' }, + ); +}; + export const streamExecInContainer = async (args: { containerName: string; command: string[]; @@ -441,17 +394,6 @@ export const stopWorkspaceContainer = async (containerName: string) => { }); }; -export const inspectWorkspaceContainer = async (containerName: string) => { - const result = await execa(containerRuntime(), ['inspect', containerName], { - all: true, - reject: false, - }); - return { - exists: result.exitCode === 0, - output: result.all, - }; -}; - export const listWorkspaceContainerNames = async (prefix: string) => { const result = await execa( containerRuntime(), diff --git a/apps/agent-worker/src/runtime/opencode-adapter.ts b/apps/agent-worker/src/runtime/opencode-adapter.ts new file mode 100644 index 0000000..5bf4fc2 --- /dev/null +++ b/apps/agent-worker/src/runtime/opencode-adapter.ts @@ -0,0 +1,140 @@ +import { mkdir, writeFile } from 'node:fs/promises'; +import path from 'node:path'; +import { randomUUID } from 'node:crypto'; + +import type { NormalizedAgentEvent } from '../agent-events'; +import { normalizeOpenCodeRunLine } from '../agent-events'; +import { env } from '../env'; +import type { AdapterFactory, TurnResult } from './agent-runtime'; +import { + buildMarkedCommand, + killBoxProcessesByMarker, + streamExecInContainer, +} from './docker'; +import type { AdapterWorkspace } from './provider'; +import { opencodeModel, providerEnvironment } from './provider'; + +// Normalize + pretty-print an auth JSON blob before writing it with owner-only +// permissions (mirrors the CodexAdapter helper). +const writeJsonFile = async (filePath: string, content: string) => { + let normalized = content.trim(); + try { + normalized = `${JSON.stringify(JSON.parse(normalized), null, 2)}\n`; + } catch { + throw new Error('OpenCode auth JSON is not valid JSON.'); + } + await mkdir(path.dirname(filePath), { recursive: true }); + await writeFile(filePath, normalized, { mode: 0o600 }); +}; + +export const createOpenCodeAdapter: AdapterFactory = (deps) => { + const execStream = deps?.execStream ?? streamExecInContainer; + + const prepareAuth = async (workspace: AdapterWorkspace) => { + // API-key profiles authenticate purely via ANTHROPIC_API_KEY/OPENAI_API_KEY + // in the environment; only `opencode_auth_json` profiles need auth.json on disk. + const profile = workspace.claim.aiProviderProfile; + if (profile?.authType !== 'opencode_auth_json') return; + if (!profile.secret) { + throw new Error('OpenCode auth profile is missing auth.json contents.'); + } + const authPath = path.join( + workspace.workdir, + '.local', + 'share', + 'opencode', + 'auth.json', + ); + await writeJsonFile(authPath, profile.secret); + }; + + const runTurn = async ( + workspace: AdapterWorkspace, + prompt: string, + onEvent: (event: NormalizedAgentEvent) => Promise, + ): Promise => { + workspace.runtime = 'opencode'; + workspace.turnMarker = `spoon-turn-${randomUUID()}`; + const marker = workspace.turnMarker; + + // Flags verified against `opencode run --help` (opencode-ai@1.17.9): + // `--format json`, `--model`, and `--session ` all exist. + const argv = [ + 'opencode', + 'run', + '--format', + 'json', + '--model', + opencodeModel(workspace.claim), + ...(workspace.opencodeSessionId + ? ['--session', workspace.opencodeSessionId] + : []), + '--', + prompt, + ]; + const command = buildMarkedCommand(marker, argv); + + const aiEnv = providerEnvironment(workspace.claim, workspace.containerHome); + const secretEnv = Object.fromEntries( + workspace.claim.secrets.map((secret) => [secret.name, secret.value]), + ); + + let capturedSessionId: string | undefined; + let capturedError: string | undefined; + let finalMessage: string | undefined; + + // execa's own `timeout` only kills the local docker-exec client, not the + // in-box process group. Schedule an explicit marker kill on timeout and clear + // it once the exec resolves. + const killTimer = setTimeout(() => { + void killBoxProcessesByMarker({ + containerName: workspace.boxName, + marker, + }); + }, env.jobTimeoutMs); + + let result; + try { + result = await execStream({ + containerName: workspace.boxName, + containerCwd: workspace.containerRepo, + command, + environment: { ...aiEnv, ...secretEnv }, + redact: workspace.redact, + timeoutMs: env.jobTimeoutMs, + onStdoutLine: async (line) => { + for (const event of normalizeOpenCodeRunLine(line)) { + if (event.kind === 'session') capturedSessionId = event.sessionId; + if (event.kind === 'error') capturedError = event.message; + if (event.kind === 'assistant_completed' && event.content) { + finalMessage = event.content; + } + await onEvent(event); + } + }, + onStderrLine: async (line) => { + if (!line.trim()) return; + await onEvent({ kind: 'status', status: line }); + }, + }); + } finally { + clearTimeout(killTimer); + } + + if (result.exitCode !== 0) { + capturedError = result.output; + } + + return { finalMessage, sessionId: capturedSessionId, error: capturedError }; + }; + + const abort = async (workspace: AdapterWorkspace) => { + if (!workspace.turnMarker) return; + await killBoxProcessesByMarker({ + containerName: workspace.boxName, + marker: workspace.turnMarker, + }); + }; + + return { name: 'opencode', prepareAuth, runTurn, abort }; +}; diff --git a/apps/agent-worker/src/runtime/provider.ts b/apps/agent-worker/src/runtime/provider.ts new file mode 100644 index 0000000..2dd3ddf --- /dev/null +++ b/apps/agent-worker/src/runtime/provider.ts @@ -0,0 +1,199 @@ +import path from 'node:path'; + +import type { Id } from '@spoon/backend/convex/_generated/dataModel.js'; + +import type { AgentRuntimeName } from './agent-runtime'; + +export type Claim = { + job: { + _id: Id<'agentJobs'>; + prompt: string; + runtime?: 'codex' | 'opencode' | 'claude'; + jobType?: 'user_change' | 'maintenance_review' | 'conflict_resolution'; + envFilePath?: string; + materializeEnvFile?: boolean; + baseBranch: string; + workBranch: string; + forkOwner: string; + forkRepo: string; + upstreamOwner: string; + upstreamRepo: string; + }; + spoon: { name: string }; + openai: { + apiKey?: string; + model: string; + reasoningEffort: 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh'; + }; + aiProviderProfile?: { + id: string; + name: string; + provider: + | 'openai' + | 'anthropic' + | 'google' + | 'openrouter' + | 'requesty' + | 'litellm' + | 'cloudflare_ai_gateway' + | 'custom_openai_compatible' + | 'opencode_openai_login'; + authType: 'api_key' | 'opencode_auth_json' | 'anthropic_oauth_json' | 'none'; + secret?: string; + baseUrl?: string; + model: string; + reasoningEffort: 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh'; + }; + github: { installationId?: string }; + agentSettings?: { + installCommand?: string; + checkCommand?: string; + testCommand?: string; + autoDetectCommands?: boolean; + } | null; + secrets: { name: string; value: string }[]; +}; + +// The subset of the worker's active workspace that agent adapters read. +export type AdapterWorkspace = { + claim: Claim; + workdir: string; + homeDir: string; + username: string; + containerHome: string; + containerRepo: string; + repoDir: string; + boxName: string; + redact: (value: string) => string; + runtime: AgentRuntimeName; + codexSessionId?: string; + opencodeSessionId?: string; + claudeSessionId?: string; + turnMarker?: string; +}; + +export const isCodexLoginProfile = (claim: Claim) => + claim.aiProviderProfile?.provider === 'opencode_openai_login' || + claim.aiProviderProfile?.authType === 'opencode_auth_json'; + +export const collectJsonStringValues = (value?: string): string[] => { + if (!value) return []; + try { + const parsed = JSON.parse(value) as unknown; + const values: string[] = []; + const visit = (item: unknown) => { + if (typeof item === 'string') { + if (item.length >= 12) values.push(item); + return; + } + if (Array.isArray(item)) { + item.forEach(visit); + return; + } + if (item && typeof item === 'object') { + Object.values(item).forEach(visit); + } + }; + visit(parsed); + return values; + } catch { + return []; + } +}; + +export const providerEnvironment = ( + claim: Claim, + workspaceRoot?: string, +): Record => { + if (isCodexLoginProfile(claim)) { + if (!workspaceRoot) { + throw new Error('Codex auth profiles require a prepared workspace.'); + } + return { + CODEX_HOME: path.join(workspaceRoot, '.codex'), + HOME: workspaceRoot, + XDG_DATA_HOME: path.join(workspaceRoot, '.local', 'share'), + XDG_CONFIG_HOME: path.join(workspaceRoot, '.config'), + }; + } + const profile = claim.aiProviderProfile; + const secret = profile?.secret ?? claim.openai.apiKey; + if (!secret) { + throw new Error('No AI provider credential is configured for this job.'); + } + const baseUrl: Record = profile?.baseUrl + ? { OPENAI_BASE_URL: profile.baseUrl } + : {}; + if (!profile || profile.provider === 'openai') { + return { OPENAI_API_KEY: secret, ...baseUrl }; + } + if (profile.provider === 'anthropic') return { ANTHROPIC_API_KEY: secret }; + if (profile.provider === 'google') return { GOOGLE_API_KEY: secret }; + if (profile.provider === 'openrouter') { + return { OPENROUTER_API_KEY: secret, ...baseUrl }; + } + if (profile.provider === 'requesty') { + return { REQUESTY_API_KEY: secret, ...baseUrl }; + } + if (profile.provider === 'cloudflare_ai_gateway') { + return { CLOUDFLARE_API_KEY: secret, ...baseUrl }; + } + if ( + profile.provider === 'litellm' || + profile.provider === 'custom_openai_compatible' + ) { + return { OPENAI_API_KEY: secret, ...baseUrl }; + } + throw new Error('Unsupported AI provider profile.'); +}; + +export const opencodeModel = (claim: Claim) => { + const profile = claim.aiProviderProfile; + const model = profile?.model ?? claim.openai.model; + if (model.includes('/')) return model; + if (!profile) return `openai/${model}`; + if ( + profile.provider === 'custom_openai_compatible' || + profile.provider === 'cloudflare_ai_gateway' + ) { + return model; + } + if (profile.provider === 'opencode_openai_login') return `openai/${model}`; + return `${profile.provider}/${model}`; +}; + +export const codexModel = (claim: Claim) => { + const model = claim.aiProviderProfile?.model ?? claim.openai.model; + return model.includes('/') ? (model.split('/').at(-1) ?? model) : model; +}; + +export const codexModelArgs = (claim: Claim) => + isCodexLoginProfile(claim) ? [] : ['--model', codexModel(claim)]; + +// Claude Code OAuth profiles store their credentials as a JSON blob written to +// `~/.claude/.credentials.json`; API-key profiles authenticate via +// ANTHROPIC_API_KEY. Keyed on the Anthropic OAuth snapshot type. +export const isClaudeOAuthProfile = (claim: Claim) => + claim.aiProviderProfile?.authType === 'anthropic_oauth_json'; + +// The `claude --model` flag expects a bare model id (e.g. `claude-sonnet-4`), +// so strip any `provider/` prefix like codexModel does. +export const claudeModel = (claim: Claim) => { + const model = claim.aiProviderProfile?.model ?? claim.openai.model; + return model.includes('/') ? (model.split('/').at(-1) ?? model) : model; +}; + +// Environment for a `claude -p` turn. OAuth-json profiles authenticate from the +// on-disk credentials file (only HOME is needed); API-key profiles export +// ANTHROPIC_API_KEY. +export const claudeEnv = ( + claim: Claim, + containerHome: string, +): Record => { + if (isClaudeOAuthProfile(claim)) return { HOME: containerHome }; + const secret = claim.aiProviderProfile?.secret ?? claim.openai.apiKey; + if (!secret) { + throw new Error('No Anthropic API key is configured for this Claude job.'); + } + return { ANTHROPIC_API_KEY: secret, HOME: containerHome }; +}; diff --git a/apps/agent-worker/src/runtime/register.ts b/apps/agent-worker/src/runtime/register.ts new file mode 100644 index 0000000..7a71ac7 --- /dev/null +++ b/apps/agent-worker/src/runtime/register.ts @@ -0,0 +1,8 @@ +import { registerAdapter } from './agent-runtime'; +import { createClaudeAdapter } from './claude-adapter'; +import { createCodexAdapter } from './codex-adapter'; +import { createOpenCodeAdapter } from './opencode-adapter'; + +registerAdapter('codex', createCodexAdapter); +registerAdapter('opencode', createOpenCodeAdapter); +registerAdapter('claude', createClaudeAdapter); diff --git a/apps/agent-worker/src/runtime/turn-outcome.ts b/apps/agent-worker/src/runtime/turn-outcome.ts new file mode 100644 index 0000000..f7b45ce --- /dev/null +++ b/apps/agent-worker/src/runtime/turn-outcome.ts @@ -0,0 +1,60 @@ +import type { AgentRuntimeName } from './agent-runtime'; + +// Pure, side-effect-free decision for how a completed runtime turn should be +// surfaced. Kept out of worker.ts so it is unit-testable without loading the +// worker's Convex/Docker dependencies. +// +// Inputs: +// - assistantText: text already streamed to the user (may be partial). +// - recoveredText: text recovered from `TurnResult.finalMessage` (already +// redacted+truncated by the caller), used only when nothing streamed. +// - error: `TurnResult.error` — non-empty when the runtime reported a hard +// failure (nonzero exit / failure event). +// - runtime: runtime name, for the failure message prefix. +// +// Output: +// - text: the assistant text to persist (streamed text, else recovered text). +// - failure: when set, the caller must surface the turn as FAILED (throw). The +// partial text in `text` is still preserved so the user can see it. +export const resolveTurnOutcome = (args: { + assistantText: string; + recoveredText?: string; + error?: string; + runtime: AgentRuntimeName; +}): { text: string; failure?: string } => { + const { assistantText, recoveredText, error, runtime } = args; + let text = assistantText; + if (!text.trim() && recoveredText) { + text = recoveredText; + } + if (!text.trim()) { + return { + text, + failure: error + ? `${runtime} failed:\n${error}` + : 'Codex completed without producing an assistant response.', + }; + } + // A hard failure that streamed partial text must still be surfaced as failed; + // the streamed text is preserved above so the user sees the truncated answer. + if (error) { + return { text, failure: `${runtime} failed:\n${error}` }; + } + return { text }; +}; + +// Pure decision for whether the `content` on an `assistant_completed` event +// should be folded into the accumulated assistant text. +// +// Some runtimes (Claude `-p --output-format stream-json`, OpenCode) emit the +// final answer TWICE: once as streamed `assistant_delta` chunks and again as +// the `content` on the terminal `assistant_completed`/`result` event. Appending +// both yields a doubled "". So only fold in the completed +// content when nothing has streamed yet (a runtime that emits only a final +// result). When deltas already carried the answer, skip it. The truly-empty +// case is still covered by resolveTurnOutcome's recoveredText/finalMessage +// fallback, so nothing is lost. +export const shouldAppendCompletedContent = ( + currentText: string, + completedContent?: string, +): boolean => Boolean(completedContent) && !currentText.trim(); diff --git a/apps/agent-worker/src/server.ts b/apps/agent-worker/src/server.ts index 89574dc..a65ea78 100644 --- a/apps/agent-worker/src/server.ts +++ b/apps/agent-worker/src/server.ts @@ -3,6 +3,15 @@ import type { IncomingMessage, ServerResponse } from 'node:http'; import type { Id } from '@spoon/backend/convex/_generated/dataModel.js'; +import { + getBoxStatus, + listBoxTree, + readBoxFile, + restartBox, + startBox, + stopBox, + writeBoxFile, +} from './box'; import { env } from './env'; import { attachTerminalServer } from './terminal'; import { @@ -56,6 +65,11 @@ const jobRoute = (pathname: string) => { return { jobId: decodeURIComponent(match[1]), action: match[2] }; }; +export const boxRoute = (pathname: string) => { + const match = /^\/box\/(status|lifecycle|tree|file)$/.exec(pathname); + return match?.[1] ? { action: match[1] } : null; +}; + export const startWorkerServer = () => { const server = createServer((request, response) => { void (async () => { @@ -73,6 +87,59 @@ export const startWorkerServer = () => { sendJson(response, 200, await cleanupOrphanedWorkspaces()); return; } + const box = boxRoute(url.pathname); + if (box) { + const user = url.searchParams.get('user') ?? ''; + if (!user) { + sendJson(response, 400, { error: 'Missing user' }); + return; + } + if (request.method === 'GET' && box.action === 'status') { + sendJson(response, 200, { status: await getBoxStatus(user) }); + return; + } + if (request.method === 'POST' && box.action === 'lifecycle') { + const body = await parseJson<{ action?: string }>(request); + if (body.action === 'start') { + await startBox(user); + } else if (body.action === 'stop') { + await stopBox(user); + } else if (body.action === 'restart') { + await restartBox(user); + } else { + sendJson(response, 400, { error: 'Unknown action' }); + return; + } + sendJson(response, 200, { status: await getBoxStatus(user) }); + return; + } + if (request.method === 'GET' && box.action === 'tree') { + sendJson(response, 200, { tree: await listBoxTree(user) }); + return; + } + if (request.method === 'GET' && box.action === 'file') { + const filePath = url.searchParams.get('path') ?? ''; + sendJson(response, 200, { + path: filePath, + content: await readBoxFile(user, filePath), + }); + return; + } + if (request.method === 'PUT' && box.action === 'file') { + const body = await parseJson<{ path?: string; content?: string }>( + request, + ); + sendJson( + response, + 200, + await writeBoxFile(user, body.path ?? '', body.content ?? ''), + ); + return; + } + sendJson(response, 404, { error: 'Not found' }); + return; + } + const route = jobRoute(url.pathname); if (!route) { sendJson(response, 404, { error: 'Not found' }); @@ -175,9 +242,11 @@ export const startWorkerServer = () => { const status = message === 'Unauthorized' ? 401 - : message.includes('not supported') - ? 409 - : 500; + : message.startsWith('Refusing to') + ? 400 + : message.includes('not supported') + ? 409 + : 500; sendJson(response, status, { error: message, }); diff --git a/apps/agent-worker/src/terminal-token.ts b/apps/agent-worker/src/terminal-token.ts index 29a70fb..2fe8cfd 100644 --- a/apps/agent-worker/src/terminal-token.ts +++ b/apps/agent-worker/src/terminal-token.ts @@ -27,3 +27,41 @@ export const verifyTerminalToken = ( timingSafeEqual(providedBuf, expectedBuf) ); }; + +// Extract the username a box token authorizes without verifying its signature. +// The token itself authorizes the connection, so the upgrade handler reads the +// username from it (rather than a query param) and then verifies the whole +// token against that username. Returns null unless the format matches +// `${expiresAtMs}.box.${username}.${hmacSha256Hex}`. +export const parseBoxTokenUsername = (token: string): string | null => { + const parts = token.split('.'); + if (parts.length !== 4 || parts[1] !== 'box') return null; + const username = parts[2]; + if (!username) return null; + return username; +}; + +// User-scoped variant authorizing a terminal connection to a user's box. +// Embeds a literal `box` segment so its four parts never collide with the +// three-part job token above. Format: +// `${expiresAtMs}.box.${username}.${hmacSha256Hex}` +export const verifyBoxTerminalToken = ( + token: string, + username: string, + secret: string, +): boolean => { + if (!token || !secret) return false; + const parts = token.split('.'); + if (parts.length !== 4 || parts[1] !== 'box') return false; + const [expRaw, , tokenUsername, provided] = parts; + if (tokenUsername !== username) return false; + const exp = Number.parseInt(expRaw ?? '', 10); + if (!Number.isFinite(exp) || Date.now() > exp) return false; + const expected = signature(`${expRaw}.box.${tokenUsername}`, secret); + const providedBuf = Buffer.from(provided ?? '', 'hex'); + const expectedBuf = Buffer.from(expected, 'hex'); + return ( + providedBuf.length === expectedBuf.length && + timingSafeEqual(providedBuf, expectedBuf) + ); +}; diff --git a/apps/agent-worker/src/terminal.ts b/apps/agent-worker/src/terminal.ts index 1a8d3be..b3e9a19 100644 --- a/apps/agent-worker/src/terminal.ts +++ b/apps/agent-worker/src/terminal.ts @@ -1,12 +1,19 @@ -import { spawn } from 'node:child_process'; -import type { ChildProcessWithoutNullStreams } from 'node:child_process'; import type { Server } from 'node:http'; +import type { Duplex } from 'node:stream'; import type { WebSocket } from 'ws'; import { WebSocketServer } from 'ws'; +import type { BoxHandle } from './user-container'; +import { boxHomePaths } from './box'; import { env } from './env'; -import { verifyTerminalToken } from './terminal-token'; -import { acquireUserBox, releaseUserBox } from './user-container'; +import { getDockerClient } from './runtime/docker'; +import { + parseBoxTokenUsername, + verifyBoxTerminalToken, + verifyTerminalToken, +} from './terminal-token'; +import { acquireUserBox } from './user-container'; +import { ensureBashProfile } from './user-environment'; import { getTerminalWorkspace } from './worker'; const clampDimension = (value: unknown) => { @@ -15,55 +22,79 @@ const clampDimension = (value: unknown) => { return Math.min(Math.max(n, 1), 1000); }; -// Single-quote a string for a POSIX shell. -const shellQuote = (value: string) => `'${value.replaceAll("'", `'\\''`)}'`; - -const bridge = async (ws: WebSocket, jobId: string) => { - const workspace = getTerminalWorkspace(jobId); - if (!workspace) { - ws.close(1011, 'Workspace is not active.'); - return; +export const parseResizeMessage = ( + data: Buffer, + isBinary: boolean, +): { cols: number; rows: number } | null => { + if (isBinary) return null; + try { + const message = JSON.parse(data.toString('utf8')) as { + type?: string; + cols?: number; + rows?: number; + }; + if (message.type !== 'resize') return null; + const cols = clampDimension(message.cols); + const rows = clampDimension(message.rows); + if (!cols || !rows) return null; + return { cols, rows }; + } catch { + return null; } +}; - // bun can't load node-pty (native ABI mismatch) and dockerode can't attach to - // podman, so we drive the runtime CLI (` exec -i`) and allocate the PTY - // *inside* the container with `script`, bridging the plain pipes to the socket. - // - // Register the message handler immediately and buffer input/size until the exec - // is ready (acquiring the box can take seconds on first connect), so the initial - // resize and early keystrokes aren't dropped. - const procHolder: { current?: ChildProcessWithoutNullStreams } = {}; +// The login-shell command shared by the job and box terminals: prefer a +// resumable tmux session, else fall back to a plain interactive login shell. +const SHELL_CMD = [ + '/bin/bash', + '-lc', + 'if command -v tmux >/dev/null 2>&1; then exec tmux new-session -A -s spoon; else exec bash -il; fi', +]; + +type ShellBridgeOptions = { + // Acquire the per-user box (reference-counted); its handle is released on + // cleanup. Called after the input handler is wired so early keystrokes buffer. + acquire: () => Promise; + // Host-side preparation before the shell starts (e.g. seed .bash_profile). + prepare?: () => Promise; + // Directory the login shell starts in (container path). + cwd: string; + // Env entries appended after TERM (e.g. HOME, and job secrets for job terms). + envFlags: string[]; +}; + +/** + * Shared PTY bridge: wire a WebSocket to an interactive login shell running via + * a real TTY (dockerode `exec` with `Tty:true` + `exec.resize`) inside the + * user's box. Registers the input handler immediately and buffers input/size + * until the exec is ready (acquiring the box can take seconds on first connect), + * so the initial resize and early keystrokes aren't dropped. Used by both the + * job terminal (`bridge`) and the box terminal (`bridgeBox`); only the box to + * acquire, the working directory, and the env differ. + */ +const runShellBridge = async (ws: WebSocket, opts: ShellBridgeOptions) => { + const execHolder: { + current?: { exec: import('dockerode').Exec; stream: Duplex }; + } = {}; const pendingInput: Buffer[] = []; let cols = 80; let rows = 24; ws.on('message', (data: Buffer, isBinary: boolean) => { - if (!isBinary) { - // Text frames are control messages (resize); anything else is raw input. - try { - const message = JSON.parse(data.toString('utf8')) as { - type?: string; - cols?: number; - rows?: number; - }; - if (message.type === 'resize') { - const c = clampDimension(message.cols); - const r = clampDimension(message.rows); - if (c && r) { - cols = c; - rows = r; - } - return; - } - } catch { - // fall through: treat as raw input - } + const resize = parseResizeMessage(data, isBinary); + if (resize) { + cols = resize.cols; + rows = resize.rows; + void execHolder.current?.exec + .resize({ h: rows, w: cols }) + .catch(() => undefined); + return; } - if (procHolder.current) procHolder.current.stdin.write(data); + if (execHolder.current) execHolder.current.stream.write(data); else pendingInput.push(data); }); - let acquired = false; + const handleHolder: { current?: BoxHandle } = {}; let released = false; // Read through a function so TS doesn't narrow `released` to a constant — the // cleanup handler flips it asynchronously when the socket closes. @@ -71,8 +102,9 @@ const bridge = async (ws: WebSocket, jobId: string) => { const cleanup = () => { if (released) return; released = true; - procHolder.current?.kill(); - if (acquired) releaseUserBox(workspace.username); + execHolder.current?.stream.end(); + execHolder.current?.stream.destroy(); + handleHolder.current?.release(); }; ws.on('close', cleanup); ws.on('error', cleanup); @@ -81,12 +113,10 @@ const bridge = async (ws: WebSocket, jobId: string) => { // the terminal share the exact same container (Phase 2). let boxName: string; try { - boxName = await acquireUserBox({ - username: workspace.username, - workdir: workspace.workdir, - containerHome: workspace.containerHome, - }); - acquired = true; + if (opts.prepare) await opts.prepare(); + const handle = await opts.acquire(); + handleHolder.current = handle; + boxName = handle.boxName; } catch (error) { ws.close( 1011, @@ -95,64 +125,90 @@ const bridge = async (ws: WebSocket, jobId: string) => { return; } - if (isReleased()) return; // client disconnected during startup; cleanup ran + // Cleanup may have run before the awaited handle existed. + if (isReleased()) { + handleHolder.current.release(); + return; + } - // Reattach a persistent tmux session across reconnects when available, else a - // plain login shell. `stty` sizes the PTY to the client's viewport up front. - const launcher = - `stty rows ${rows} cols ${cols} 2>/dev/null; ` + - // Reattach a persistent tmux session when tmux is present; otherwise fall back - // to an interactive login shell (`-i` so it prints a prompt and line-edits). - // Check with `command -v` rather than `exec tmux || …`: a failed `exec` makes a - // non-interactive shell exit before the `||`, so the fallback never runs. - 'if command -v tmux >/dev/null 2>&1; then exec tmux new-session -A -s spoon; ' + - 'else exec bash -il; fi'; - const envFlags = [ - '-e', - 'TERM=xterm-256color', - '-e', - `HOME=${workspace.containerHome}`, - ...workspace.secrets.flatMap((s) => ['-e', `${s.name}=${s.value}`]), - ]; + const docker = getDockerClient(); + const container = docker.getContainer(boxName); + const exec = await container.exec({ + AttachStdin: true, + AttachStdout: true, + AttachStderr: true, + Tty: true, + Cmd: SHELL_CMD, + Env: ['TERM=xterm-256color', ...opts.envFlags], + WorkingDir: opts.cwd, + }); + const stream = await exec.start({ hijack: true, stdin: true, Tty: true }); + execHolder.current = { exec, stream }; - const proc = spawn( - env.containerRuntime, - [ - 'exec', - '-i', - ...envFlags, - '-w', - workspace.containerRepo, - boxName, - '/bin/bash', - '-lc', - `exec script -qfc ${shellQuote(launcher)} /dev/null`, - ], - { stdio: ['pipe', 'pipe', 'pipe'] }, - ); - procHolder.current = proc; + if (isReleased()) { + stream.end(); + stream.destroy(); + return; + } + + await exec.resize({ h: rows, w: cols }).catch(() => undefined); // Replay any keystrokes the client sent before the process was ready. - for (const buffered of pendingInput) proc.stdin.write(buffered); + for (const buffered of pendingInput) stream.write(buffered); pendingInput.length = 0; - const forward = (chunk: Buffer) => { + stream.on('data', (chunk: Buffer) => { if (ws.readyState === ws.OPEN) ws.send(chunk, { binary: true }); - }; - proc.stdout.on('data', forward); - proc.stderr.on('data', forward); - proc.on('exit', () => { + }); + stream.on('end', () => { if (ws.readyState === ws.OPEN) ws.close(); }); - proc.on('error', () => { + stream.on('error', () => { if (ws.readyState === ws.OPEN) ws.close(); }); }; +// Job terminal: opens the shell at the repo checkout, with the job's selected +// secrets in the environment (it is scoped to that job's active workspace). +const bridge = async (ws: WebSocket, jobId: string) => { + const workspace = getTerminalWorkspace(jobId); + if (!workspace) { + ws.close(1011, 'Workspace is not active.'); + return; + } + await runShellBridge(ws, { + acquire: () => + acquireUserBox({ + username: workspace.username, + workdir: workspace.workdir, + containerHome: workspace.containerHome, + }), + cwd: workspace.containerRepo, + envFlags: [ + `HOME=${workspace.containerHome}`, + ...workspace.secrets.map((secret) => `${secret.name}=${secret.value}`), + ], + }); +}; + +// Box terminal: opens a login shell rooted at the user's home (`~`), with no +// per-job secrets — it is user-scoped, not tied to any job's workspace. +const bridgeBox = async (ws: WebSocket, username: string) => { + const { homeDir, containerHome } = boxHomePaths(username); + await runShellBridge(ws, { + prepare: () => ensureBashProfile(homeDir), + acquire: () => + acquireUserBox({ username, workdir: homeDir, containerHome }), + cwd: containerHome, + envFlags: [`HOME=${containerHome}`], + }); +}; + /** - * Attaches the interactive-terminal WebSocket endpoint to the worker's HTTP - * server. Browser connects to `/jobs/:jobId/terminal?token=…` with a short-lived - * token minted by the Next app (which has already verified job ownership). + * Attaches the interactive-terminal WebSocket endpoints to the worker's HTTP + * server. Two routes, each authorized by a short-lived token minted by the Next + * app: `/jobs/:jobId/terminal?token=…` (job-scoped, after ownership check) and + * `/box/terminal?token=…` (user-scoped, username read from the token itself). */ export const attachTerminalServer = (server: Server) => { if (env.runtime !== 'docker') return; @@ -160,20 +216,43 @@ export const attachTerminalServer = (server: Server) => { server.on('upgrade', (request, socket, head) => { const url = new URL(request.url ?? '', `http://localhost:${env.httpPort}`); + const token = url.searchParams.get('token') ?? ''; + + // User-scoped box terminal: the token authorizes the connection, so the + // username is read FROM the token, then verified against the whole token. + if (url.pathname === '/box/terminal') { + const username = parseBoxTokenUsername(token); + if ( + !username || + !verifyBoxTerminalToken(token, username, env.terminalSecret) + ) { + socket.write('HTTP/1.1 401 Unauthorized\r\n\r\n'); + socket.destroy(); + return; + } + wss.handleUpgrade(request, socket, head, (ws) => { + void bridgeBox(ws, username).catch(() => { + if (ws.readyState === ws.OPEN) ws.close(); + }); + }); + return; + } + const match = /^\/jobs\/([^/]+)\/terminal$/.exec(url.pathname); if (!match?.[1]) { socket.destroy(); return; } const jobId = decodeURIComponent(match[1]); - const token = url.searchParams.get('token') ?? ''; if (!verifyTerminalToken(token, jobId, env.terminalSecret)) { socket.write('HTTP/1.1 401 Unauthorized\r\n\r\n'); socket.destroy(); return; } wss.handleUpgrade(request, socket, head, (ws) => { - void bridge(ws, jobId); + void bridge(ws, jobId).catch(() => { + if (ws.readyState === ws.OPEN) ws.close(); + }); }); }); diff --git a/apps/agent-worker/src/user-container.ts b/apps/agent-worker/src/user-container.ts index c4bb374..f5ac6d5 100644 --- a/apps/agent-worker/src/user-container.ts +++ b/apps/agent-worker/src/user-container.ts @@ -1,40 +1,135 @@ import { env } from './env'; import { ensureUserContainer, + listWorkspaceContainerNames, stopWorkspaceContainer, userContainerName, } from './runtime/docker'; -// Phase 2: one persistent "box" container per user that all of their threads -// (agent turns + terminal + commands) exec into. Reference-counted so it stays -// up while any thread workspace is active or a terminal is connected, and is -// reaped after an idle period once nothing holds it. -type Box = { refs: number; idleTimer?: NodeJS.Timeout }; -const boxes = new Map(); +// Phase 2: one persistent "box" container per user. Reference-counted by opaque +// handles (not a shared integer), serialized per-username with an async mutex, +// and idle-reaped once no handle is held. +export type BoxHandle = { boxName: string; release: () => void }; -export const acquireUserBox = async (args: { +type Box = { + name: string; + initialized: boolean; + refs: Set; + idleTimer?: ReturnType; +}; +const boxes = new Map(); +const locks = new Map>(); + +// Per-username async mutex: chain each operation after the previous one. +const withLock = (username: string, fn: () => Promise): Promise => { + const previous = locks.get(username) ?? Promise.resolve(); + const next = previous.then(fn, fn); + locks.set( + username, + next.then( + () => undefined, + () => undefined, + ), + ); + return next; +}; + +const scheduleReapIfIdle = (username: string) => { + const box = boxes.get(username); + if (!box || box.refs.size > 0) return; + if (box.idleTimer) clearTimeout(box.idleTimer); + box.idleTimer = setTimeout(() => { + void withLock(username, async () => { + const current = boxes.get(username); + if (current !== box || current.refs.size > 0) return; + await stopWorkspaceContainer(current.name); + if (boxes.get(username) === current && current.refs.size === 0) { + boxes.delete(username); + } + }); + }, env.boxIdleMs); +}; + +const makeHandle = (username: string, box: Box): BoxHandle => { + const token = Symbol('box-ref'); + box.refs.add(token); + let released = false; + return { + boxName: box.name, + release: () => { + if (released) return; + released = true; + box.refs.delete(token); + scheduleReapIfIdle(username); + }, + }; +}; + +export const acquireUserBox = (args: { username: string; workdir: string; containerHome: string; -}): Promise => { - const name = await ensureUserContainer(args); - const box = boxes.get(args.username) ?? { refs: 0 }; - if (box.idleTimer) { - clearTimeout(box.idleTimer); - box.idleTimer = undefined; +}): Promise => + withLock(args.username, async () => { + let box = boxes.get(args.username); + if (box?.idleTimer) { + clearTimeout(box.idleTimer); + box.idleTimer = undefined; + } + if (!box) { + box = { + name: userContainerName(args.username), + initialized: false, + refs: new Set(), + }; + boxes.set(args.username, box); + } + // Register the ref before the slow Docker call so a failed acquire can + // release it without leaking registry state. + const handle = makeHandle(args.username, box); + try { + if (!box.initialized) { + box.name = await ensureUserContainer(args); + box.initialized = true; + } + return handle; + } catch (error) { + handle.release(); + if (boxes.get(args.username) === box && box.refs.size === 0) { + if (box.idleTimer) clearTimeout(box.idleTimer); + boxes.delete(args.username); + } + throw error; + } + }); + +// Adopt boxes left by a prior worker process so the idle reaper cleans them up. +export const reconcileExistingBoxes = async (): Promise => { + const names = await listWorkspaceContainerNames('spoon-box-'); + for (const name of names) { + const username = name.replace(/^spoon-box-/, ''); + if (boxes.has(username)) continue; + const box: Box = { name, initialized: false, refs: new Set() }; + boxes.set(username, box); + scheduleReapIfIdle(username); } - box.refs += 1; - boxes.set(args.username, box); - return name; }; -export const releaseUserBox = (username: string) => { +export const runningBoxUsernames = (): Set => new Set(boxes.keys()); + +// Drop a box from the in-memory registry (used after an explicit stop so a +// stopped box isn't treated as "held" by the idle reaper or reconcile logic). +export const resetBox = (username: string): void => { const box = boxes.get(username); if (!box) return; - box.refs = Math.max(0, box.refs - 1); - if (box.refs > 0) return; - box.idleTimer = setTimeout(() => { - void stopWorkspaceContainer(userContainerName(username)); - boxes.delete(username); - }, env.boxIdleMs); + if (box.idleTimer) clearTimeout(box.idleTimer); + boxes.delete(username); +}; + +export const _resetBoxRegistryForTests = () => { + for (const box of boxes.values()) { + if (box.idleTimer) clearTimeout(box.idleTimer); + } + boxes.clear(); + locks.clear(); }; diff --git a/apps/agent-worker/src/user-environment.ts b/apps/agent-worker/src/user-environment.ts index 19f6a30..921fb36 100644 --- a/apps/agent-worker/src/user-environment.ts +++ b/apps/agent-worker/src/user-environment.ts @@ -7,6 +7,7 @@ import type { Id } from '@spoon/backend/convex/_generated/dataModel.js'; import { api } from '@spoon/backend/convex/_generated/api.js'; import { env } from './env'; +import { assertContainedRealPath } from './path-containment'; import { runExecInContainer } from './runtime/docker'; const client = new ConvexHttpClient(env.convexUrl); @@ -31,14 +32,21 @@ export const fetchUserEnvironment = async ( const shellQuote = (value: string) => `'${value.replaceAll("'", "'\\''")}'`; -// Keep a written path inside the home directory. -const safeHomeJoin = (homeDir: string, relPath: string) => { - const target = path.resolve(homeDir, relPath); - const root = path.resolve(homeDir); - if (target !== root && !target.startsWith(`${root}${path.sep}`)) { - throw new Error(`Refusing to write dotfile outside home: ${relPath}`); - } - return target; +/** + * A mounted home has no /etc/skel, so login shells wouldn't source ~/.bashrc. + * Write a minimal `.bash_profile` (only if absent) so an interactive login + * shell — the agent's, or the user's box terminal — loads their environment. + * Shared by `materializeUserHome` and the box terminal bridge. + */ +export const ensureBashProfile = async (homeDir: string): Promise => { + await mkdir(homeDir, { recursive: true }); + const bashProfile = path.join(homeDir, '.bash_profile'); + await readFile(bashProfile, 'utf8').catch(async () => { + await writeFile( + bashProfile, + '# Spoon: load ~/.bashrc for login shells.\n[ -f ~/.bashrc ] && . ~/.bashrc\n', + ); + }); }; /** @@ -56,16 +64,7 @@ export const materializeUserHome = async (args: { redact: (value: string) => string; }): Promise => { const { homeDir, containerHome, boxName, userEnv, redact } = args; - await mkdir(homeDir, { recursive: true }); - - // A mounted home has no /etc/skel, so ensure login shells source ~/.bashrc. - const bashProfile = path.join(homeDir, '.bash_profile'); - await readFile(bashProfile, 'utf8').catch(async () => { - await writeFile( - bashProfile, - '# Spoon: load ~/.bashrc for login shells.\n[ -f ~/.bashrc ] && . ~/.bashrc\n', - ); - }); + await ensureBashProfile(homeDir); if (!userEnv.enabled) return; @@ -111,7 +110,9 @@ export const materializeUserHome = async (args: { // Editable overlay tree (wins over the repo/setup output). for (const file of userEnv.files) { - const target = safeHomeJoin(homeDir, file.path); + const target = await assertContainedRealPath(homeDir, file.path, { + forWrite: true, + }); await mkdir(path.dirname(target), { recursive: true }); await writeFile(target, file.content); if (file.isExecutable) await chmod(target, 0o755); diff --git a/apps/agent-worker/src/worker.ts b/apps/agent-worker/src/worker.ts index d5025ad..279e4cc 100644 --- a/apps/agent-worker/src/worker.ts +++ b/apps/agent-worker/src/worker.ts @@ -1,4 +1,3 @@ -import { randomBytes } from 'node:crypto'; import { access, mkdir, @@ -15,9 +14,15 @@ import type { Id } from '@spoon/backend/convex/_generated/dataModel.js'; import { api } from '@spoon/backend/convex/_generated/api.js'; import type { NormalizedAgentEvent } from './agent-events'; -import type { OpenCodeSession } from './opencode-session'; -import { normalizeCodexJsonLine } from './agent-events'; -import { prepareCodexWorkspaceFiles } from './codex-runtime'; +import type { AdapterWorkspace, Claim } from './runtime/provider'; +import type { BoxHandle } from './user-container'; +import type { AgentRuntimeName } from './runtime/agent-runtime'; +import { getAdapter } from './runtime/agent-runtime'; +import { + resolveTurnOutcome, + shouldAppendCompletedContent, +} from './runtime/turn-outcome'; +import './runtime/register'; import { env } from './env'; import { cloneRepository, @@ -27,101 +32,30 @@ import { run, } from './git'; import { getInstallationToken, openDraftPullRequest } from './github'; -import { - abortOpenCodeSession, - createOpenCodeSession, - promptOpenCodeSession, - replyOpenCodePermission, -} from './opencode-session'; +import { createHeartbeatController } from './heartbeat-controller'; +import { removeMaterializedEnv, writeMaterializedEnv } from './materialize-env'; +import { assertContainedRealPath } from './path-containment'; import { createRedactor, truncate } from './redact'; import { listWorkspaceContainerNames, runExecInContainer, - startWorkspaceContainer, - stopWorkspaceContainer, - streamExecInContainer, } from './runtime/docker'; -import { acquireUserBox, releaseUserBox } from './user-container'; +import { collectJsonStringValues } from './runtime/provider'; +import { acquireUserBox, runningBoxUsernames } from './user-container'; import { fetchUserEnvironment, materializeUserHome } from './user-environment'; +import { teardownWorkspace } from './workspace-teardown'; -type Claim = { - job: { - _id: Id<'agentJobs'>; - prompt: string; - runtime?: 'openai_direct' | 'opencode'; - jobType?: 'user_change' | 'maintenance_review' | 'conflict_resolution'; - envFilePath?: string; - materializeEnvFile?: boolean; - baseBranch: string; - workBranch: string; - forkOwner: string; - forkRepo: string; - upstreamOwner: string; - upstreamRepo: string; - }; - spoon: { name: string }; - openai: { - apiKey?: string; - model: string; - reasoningEffort: 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh'; - }; - aiProviderProfile?: { - id: string; - name: string; - provider: - | 'openai' - | 'anthropic' - | 'google' - | 'openrouter' - | 'requesty' - | 'litellm' - | 'cloudflare_ai_gateway' - | 'custom_openai_compatible' - | 'opencode_openai_login'; - authType: 'api_key' | 'opencode_auth_json' | 'none'; - secret?: string; - baseUrl?: string; - model: string; - reasoningEffort: 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh'; - }; - github: { installationId?: string }; - agentSettings?: { - installCommand?: string; - checkCommand?: string; - testCommand?: string; - autoDetectCommands?: boolean; - } | null; - secrets: { name: string; value: string }[]; -}; - -type ActiveWorkspace = { - claim: Claim; - // Host path of the persistent per-user home (mounted at `containerHome`). - // Equal to `homeDir`; kept as `workdir` for the container mount source. - workdir: string; - homeDir: string; - username: string; - // In-container paths: HOME and the thread's checkout (~/Code/{spoon}/{branch}). - containerHome: string; - containerRepo: string; - repoDir: string; +type ActiveWorkspace = AdapterWorkspace & { // Phase 2: the per-user box container this thread execs into. - boxName: string; + boxHandle: BoxHandle; githubToken: string; - redact: (value: string) => string; runtimeMode?: 'opencode_server' | 'codex_exec' | 'legacy_cli'; - containerName?: string; - containerId?: string; - opencodePassword?: string; - opencodeSession?: OpenCodeSession; - codexSessionId?: string; agentTurnActive?: boolean; + cancelRequested?: boolean; resolveTurn?: () => void; lastRecordedDiffSignature?: string; - // Captures the most recent Codex `error`/`turn.failed` event for the active - // turn so the failure surfaces the real reason instead of a generic - // "no assistant response" message. - codexTurnError?: string; + // Absolute path of the last materialized env file so teardown can delete it. + materializedEnvPath?: string; }; type FileTreeNode = { @@ -136,6 +70,13 @@ const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms)); const client = new ConvexHttpClient(env.convexUrl); const activeWorkspaces = new Map(); +export const _setActiveWorkspaceForTests = ( + jobId: string, + workspace: ActiveWorkspace, +) => activeWorkspaces.set(jobId, workspace); + +export const _resetActiveWorkspacesForTests = () => activeWorkspaces.clear(); + const appendEvent = async ( jobId: Id<'agentJobs'>, level: 'debug' | 'info' | 'warn' | 'error', @@ -252,6 +193,25 @@ const markWorkspaceStopped = async ( workspaceStatus, }); +const heartbeatWorkspaceMutation = async (jobId: Id<'agentJobs'>) => + (await client.mutation(api.agentJobs.heartbeatWorkspace, { + workerToken: env.workerToken, + workerId: env.workerId, + jobId, + })) as { success: true; cancelRequested: boolean }; + +const heartbeatController = createHeartbeatController({ + intervalMs: 30_000, + heartbeat: async (jobId) => + await heartbeatWorkspaceMutation(jobId as Id<'agentJobs'>), + onCancel: async (jobId) => await cancelWorkspace(jobId), + onError: (error) => console.error(error), +}); + +const stopHeartbeat = (jobId: string) => heartbeatController.stop(jobId); +const startHeartbeat = (jobId: Id<'agentJobs'>) => + heartbeatController.start(jobId); + const appendMessage = async (args: { jobId: Id<'agentJobs'>; role: 'user' | 'assistant' | 'system' | 'tool'; @@ -301,6 +261,22 @@ const setCodexSessionId = async ( codexSessionId, }); +// Persist the runtime session id for continuity, keyed on the workspace runtime +// (replaces the per-runtime setter call sites so dispatch stays generic). +const persistRuntimeSession = async ( + workspace: ActiveWorkspace, + sessionId: string, +) => { + if (workspace.runtime === 'codex') { + workspace.codexSessionId = sessionId; + await setCodexSessionId(workspace.claim.job._id, sessionId); + } else if (workspace.runtime === 'opencode') { + workspace.opencodeSessionId = sessionId; + } else { + workspace.claudeSessionId = sessionId; + } +}; + const createInteractionRequest = async (args: { jobId: Id<'agentJobs'>; runtime: 'opencode' | 'codex'; @@ -317,7 +293,7 @@ const createInteractionRequest = async (args: { ...args, }); -const patchInteractionRequest = async (args: { +const _patchInteractionRequest = async (args: { interactionId: Id<'agentInteractionRequests'>; status: 'pending' | 'answered' | 'approved' | 'rejected' | 'expired'; response?: string; @@ -344,152 +320,6 @@ const recordWorkspaceChange = async (args: { const commandToShell = (command: string) => ['bash', '-lc', command]; -const workspaceContainerName = (jobId: string) => - `spoon-agent-job-${jobId.replace(/[^a-zA-Z0-9_.-]/g, '-')}`; - -const isCodexLoginProfile = (claim: Claim) => - claim.aiProviderProfile?.provider === 'opencode_openai_login' || - claim.aiProviderProfile?.authType === 'opencode_auth_json'; - -const collectJsonStringValues = (value?: string): string[] => { - if (!value) return []; - try { - const parsed = JSON.parse(value) as unknown; - const values: string[] = []; - const visit = (item: unknown) => { - if (typeof item === 'string') { - if (item.length >= 12) values.push(item); - return; - } - if (Array.isArray(item)) { - item.forEach(visit); - return; - } - if (item && typeof item === 'object') { - Object.values(item).forEach(visit); - } - }; - visit(parsed); - return values; - } catch { - return []; - } -}; - -const providerEnvironment = ( - claim: Claim, - workspaceRoot?: string, -): Record => { - if (isCodexLoginProfile(claim)) { - if (!workspaceRoot) { - throw new Error('Codex auth profiles require a prepared workspace.'); - } - return { - CODEX_HOME: path.join(workspaceRoot, '.codex'), - HOME: workspaceRoot, - XDG_DATA_HOME: path.join(workspaceRoot, '.local', 'share'), - XDG_CONFIG_HOME: path.join(workspaceRoot, '.config'), - }; - } - const profile = claim.aiProviderProfile; - const secret = profile?.secret ?? claim.openai.apiKey; - if (!secret) { - throw new Error('No AI provider credential is configured for this job.'); - } - const baseUrl: Record = profile?.baseUrl - ? { OPENAI_BASE_URL: profile.baseUrl } - : {}; - if (!profile || profile.provider === 'openai') { - return { OPENAI_API_KEY: secret, ...baseUrl }; - } - if (profile.provider === 'anthropic') return { ANTHROPIC_API_KEY: secret }; - if (profile.provider === 'google') return { GOOGLE_API_KEY: secret }; - if (profile.provider === 'openrouter') { - return { OPENROUTER_API_KEY: secret, ...baseUrl }; - } - if (profile.provider === 'requesty') { - return { REQUESTY_API_KEY: secret, ...baseUrl }; - } - if (profile.provider === 'cloudflare_ai_gateway') { - return { CLOUDFLARE_API_KEY: secret, ...baseUrl }; - } - if ( - profile.provider === 'litellm' || - profile.provider === 'custom_openai_compatible' - ) { - return { OPENAI_API_KEY: secret, ...baseUrl }; - } - throw new Error('Unsupported AI provider profile.'); -}; - -const opencodeModel = (claim: Claim) => { - const profile = claim.aiProviderProfile; - const model = profile?.model ?? claim.openai.model; - if (model.includes('/')) return model; - if (!profile) return `openai/${model}`; - if ( - profile.provider === 'custom_openai_compatible' || - profile.provider === 'cloudflare_ai_gateway' - ) { - return model; - } - if (profile.provider === 'opencode_openai_login') return `openai/${model}`; - return `${profile.provider}/${model}`; -}; - -const codexModel = (claim: Claim) => { - const model = claim.aiProviderProfile?.model ?? claim.openai.model; - return model.includes('/') ? (model.split('/').at(-1) ?? model) : model; -}; - -const codexModelArgs = (claim: Claim) => - isCodexLoginProfile(claim) ? [] : ['--model', codexModel(claim)]; - -const writeJsonFile = async (filePath: string, content: string) => { - let normalized = content.trim(); - try { - normalized = `${JSON.stringify(JSON.parse(normalized), null, 2)}\n`; - } catch { - throw new Error('Codex auth JSON is not valid JSON.'); - } - await mkdir(path.dirname(filePath), { recursive: true }); - await writeFile(filePath, normalized, { mode: 0o600 }); -}; - -const prepareCodexAuth = async (workspace: ActiveWorkspace) => { - const secret = workspace.claim.aiProviderProfile?.secret; - if (!secret) { - throw new Error('Codex auth profile is missing auth.json contents.'); - } - await prepareCodexWorkspaceFiles({ - workdir: workspace.workdir, - repoDir: workspace.repoDir, - }); - const codexAuthPath = path.join(workspace.workdir, '.codex', 'auth.json'); - await writeJsonFile(codexAuthPath, secret); - - // Also seed OpenCode's auth location with the saved JSON for forward - // compatibility if this profile later runs through OpenCode directly. - const openCodeAuthPath = path.join( - workspace.workdir, - '.local', - 'share', - 'opencode', - 'auth.json', - ); - await writeJsonFile(openCodeAuthPath, secret); - - await appendEvent( - workspace.claim.job._id, - 'info', - 'clone', - 'Prepared Codex auth JSON for the isolated workspace.', - ); -}; - -const agentFailurePrefix = (claim: Claim) => - isCodexLoginProfile(claim) ? 'codex failed' : 'opencode failed'; - const handleAgentEvent = async (args: { workspace: ActiveWorkspace; event: NormalizedAgentEvent; @@ -517,7 +347,7 @@ const handleAgentEvent = async (args: { workspace.agentTurnActive = false; workspace.resolveTurn?.(); workspace.resolveTurn = undefined; - if (event.content) { + if (shouldAppendCompletedContent(assistantContent.value, event.content)) { assistantContent.value = truncate( `${assistantContent.value}${event.content}`, 40_000, @@ -531,9 +361,11 @@ const handleAgentEvent = async (args: { return; } if (event.kind === 'session') { + // Session persistence is centralized in persistRuntimeSession (called from + // the dispatch after the turn); here we only track it in memory so a later + // turn can resume. if (workspace.runtimeMode === 'codex_exec') { workspace.codexSessionId = event.sessionId; - await setCodexSessionId(jobId, event.sessionId); } return; } @@ -610,277 +442,12 @@ const handleAgentEvent = async (args: { return; } // event.kind === 'error' - // Record the real Codex failure reason on the workspace so the turn can - // surface it (Codex can emit `error`/`turn.failed` events and still exit 0 - // in some versions, which otherwise looks like an empty response). - workspace.codexTurnError = event.message; + // The adapter captures the real Codex failure reason (Codex can emit + // `error`/`turn.failed` events and still exit 0) and surfaces it as + // TurnResult.error; here we just log it. await appendEvent(jobId, 'error', 'plan', truncate(event.message, 20_000)); }; -const ensureOpenCodeSession = async (workspace: ActiveWorkspace) => { - if (workspace.opencodeSession) return workspace.opencodeSession; - const containerName = workspaceContainerName(workspace.claim.job._id); - const password = randomBytes(24).toString('hex'); - const aiEnv = providerEnvironment(workspace.claim); - const secretEnv = Object.fromEntries( - workspace.claim.secrets.map((secret) => [secret.name, secret.value]), - ); - const container = await startWorkspaceContainer({ - workdir: workspace.workdir, - containerHome: workspace.containerHome, - containerCwd: workspace.containerRepo, - containerName, - environment: { - ...aiEnv, - ...secretEnv, - OPENCODE_SERVER_PASSWORD: password, - OPENCODE_SERVER_USERNAME: 'opencode', - }, - command: ['opencode', 'serve', '--hostname', '0.0.0.0', '--port', '4096'], - publishTcpPort: env.containerAccess === 'host_port' ? 4096 : undefined, - }); - const baseUrl = - env.containerAccess === 'host_port' - ? `http://127.0.0.1:${container.hostPort}` - : `http://${containerName}:4096`; - workspace.containerName = container.containerName; - workspace.containerId = container.containerId; - workspace.opencodePassword = password; - workspace.runtimeMode = 'opencode_server'; - await setRuntimeSession({ - jobId: workspace.claim.job._id, - agentRuntimeMode: 'opencode_server', - containerId: container.containerId, - }); - let lastError: unknown; - for (let attempt = 0; attempt < 20; attempt += 1) { - try { - const session = await createOpenCodeSession({ - baseUrl, - password, - directory: workspace.containerRepo, - title: workspace.claim.job.prompt.slice(0, 80) || 'Spoon workspace', - onEvent: async (event) => { - const messageId = workspaceCurrentMessage.get( - workspace.claim.job._id, - ); - if (!messageId) return; - await handleAgentEvent({ - workspace, - event, - assistantMessageId: messageId, - assistantContent: workspaceCurrentContent.get( - workspace.claim.job._id, - ) ?? { - value: '', - }, - }); - }, - }); - workspace.opencodeSession = session; - await setRuntimeSession({ - jobId: workspace.claim.job._id, - agentRuntimeMode: 'opencode_server', - opencodeSessionId: session.sessionId, - containerId: container.containerId, - }); - return session; - } catch (error) { - lastError = error; - await sleep(500); - } - } - throw lastError instanceof Error - ? lastError - : new Error('OpenCode server did not become ready.'); -}; - -const workspaceCurrentMessage = new Map>(); -const workspaceCurrentContent = new Map< - string, - { - value: string; - } ->(); - -// Reading through a function boundary prevents TypeScript from narrowing the -// field to `undefined` after the synchronous reset in `runCodexTurn`; it is set -// asynchronously by the stream event handler. -const readCodexTurnError = (workspace: ActiveWorkspace) => - workspace.codexTurnError; - -const runCodexTurn = async (args: { - workspace: ActiveWorkspace; - prompt: string; - assistantMessageId: Id<'agentJobMessages'>; - assistantContent: { value: string }; -}) => { - const { workspace, prompt, assistantMessageId, assistantContent } = args; - workspace.runtimeMode = 'codex_exec'; - workspace.codexTurnError = undefined; - await setRuntimeSession({ - jobId: workspace.claim.job._id, - agentRuntimeMode: 'codex_exec', - codexSessionId: workspace.codexSessionId, - }); - const outputFileName = `last-message-${workspace.claim.job._id}.txt`; - const outputFileHostPath = path.join( - workspace.workdir, - '.codex', - outputFileName, - ); - const outputFileContainerPath = path.posix.join( - workspace.containerHome, - '.codex', - outputFileName, - ); - const command = workspace.codexSessionId - ? [ - 'codex', - 'exec', - 'resume', - '--json', - ...codexModelArgs(workspace.claim), - '--dangerously-bypass-approvals-and-sandbox', - '--output-last-message', - outputFileContainerPath, - workspace.codexSessionId, - prompt, - ] - : [ - 'codex', - 'exec', - '--json', - ...codexModelArgs(workspace.claim), - '--dangerously-bypass-approvals-and-sandbox', - '--output-last-message', - outputFileContainerPath, - '--cd', - workspace.containerRepo, - prompt, - ]; - const aiEnv = providerEnvironment(workspace.claim, workspace.containerHome); - const secretEnv = Object.fromEntries( - workspace.claim.secrets.map((secret) => [secret.name, secret.value]), - ); - const result = await streamExecInContainer({ - containerName: workspace.boxName, - containerCwd: workspace.containerRepo, - command, - environment: { - ...aiEnv, - ...secretEnv, - }, - redact: workspace.redact, - timeoutMs: env.jobTimeoutMs, - onStdoutLine: async (line) => { - for (const event of normalizeCodexJsonLine(line)) { - await handleAgentEvent({ - workspace, - event, - assistantMessageId, - assistantContent, - }); - } - }, - onStderrLine: async (line) => { - const trimmed = line.trim(); - if ( - trimmed && - trimmed !== 'Reading additional input from stdin...' && - !trimmed.includes('`[features].codex_hooks` is deprecated') - ) { - await appendEvent( - workspace.claim.job._id, - 'info', - 'plan', - truncate(trimmed, 10_000), - ); - } - }, - }); - await appendEvent( - workspace.claim.job._id, - 'info', - 'plan', - `Codex CLI exited with code ${result.exitCode}; captured output length ${result.output.length}; assistant length ${assistantContent.value.length}.`, - ); - if (result.exitCode !== 0) { - throw new Error(`codex failed:\n${result.output}`); - } - if (!assistantContent.value.trim()) { - try { - const lastMessage = await readFile(outputFileHostPath, 'utf8'); - if (lastMessage.trim()) { - assistantContent.value = truncate( - workspace.redact(lastMessage.trim()), - 40_000, - ); - await updateMessage({ - messageId: assistantMessageId, - content: assistantContent.value, - status: 'streaming', - }); - await appendEvent( - workspace.claim.job._id, - 'info', - 'plan', - `Recovered assistant response from Codex output file ${outputFileName}.`, - ); - } - } catch (error) { - const code = error && typeof error === 'object' ? 'code' in error : false; - if (!code || (error as { code?: string }).code !== 'ENOENT') { - throw error; - } - await appendEvent( - workspace.claim.job._id, - 'warn', - 'plan', - `Codex output file ${outputFileName} was not created.`, - ); - } - } - // Codex can report a failure via a JSON `error`/`turn.failed` event while - // still exiting 0. If the turn produced no assistant text but did report an - // error, surface that real reason rather than a generic empty response. - // Read through a helper so it is not narrowed away by the reset above (the - // field is mutated asynchronously inside the stream handler). - const codexTurnError = readCodexTurnError(workspace); - if (!assistantContent.value.trim() && codexTurnError) { - throw new Error(`codex failed:\n${codexTurnError}`); - } -}; - -const runOpenCodeTurn = async (args: { - workspace: ActiveWorkspace; - prompt: string; - assistantMessageId: Id<'agentJobMessages'>; - assistantContent: { value: string }; -}) => { - const { workspace, prompt, assistantMessageId, assistantContent } = args; - workspaceCurrentMessage.set(workspace.claim.job._id, assistantMessageId); - workspaceCurrentContent.set(workspace.claim.job._id, assistantContent); - const session = await ensureOpenCodeSession(workspace); - const turnDone = new Promise((resolve, reject) => { - const timeout = setTimeout(() => { - workspace.resolveTurn = undefined; - reject(new Error('OpenCode turn timed out.')); - }, env.jobTimeoutMs); - workspace.resolveTurn = () => { - clearTimeout(timeout); - resolve(); - }; - }); - await promptOpenCodeSession({ - session, - prompt, - model: opencodeModel(workspace.claim), - directory: workspace.containerRepo, - }); - await turnDone; -}; - const systemPromptForJob = (claim: Claim) => { const base = [ `Spoon: ${claim.spoon.name}`, @@ -1046,8 +613,6 @@ const runProjectCommand = async (args: { } }; -const quoteShell = (value: string) => `'${value.replaceAll("'", "'\\''")}'`; - const resolveWorkspace = (jobId: string) => { const workspace = activeWorkspaces.get(jobId); if (!workspace) { @@ -1056,15 +621,6 @@ const resolveWorkspace = (jobId: string) => { return workspace; }; -const safeWorkspacePath = (repoDir: string, filePath: string) => { - const resolved = path.resolve(repoDir, filePath); - const root = path.resolve(repoDir); - if (resolved !== root && !resolved.startsWith(`${root}${path.sep}`)) { - throw new Error(`Refusing to access path outside repository: ${filePath}`); - } - return resolved; -}; - const fileChangedType = async (repoDir: string, filePath: string) => { const status = await run('git', ['status', '--short', '--', filePath], { cwd: repoDir, @@ -1167,12 +723,11 @@ const recordChangedFiles = async ( const materializeEnvFile = async (workspace: ActiveWorkspace) => { const { claim, repoDir } = workspace; if (!claim.job.materializeEnvFile || !claim.job.envFilePath) return; - const envPath = safeWorkspacePath(repoDir, claim.job.envFilePath); - await mkdir(path.dirname(envPath), { recursive: true }); - const content = `${claim.secrets - .map((secret) => `${secret.name}=${JSON.stringify(secret.value)}`) - .join('\n')}\n`; - await writeFile(envPath, content); + workspace.materializedEnvPath = await writeMaterializedEnv( + repoDir, + claim.job.envFilePath, + claim.secrets, + ); await appendEvent( claim.job._id, 'info', @@ -1287,6 +842,29 @@ const slugify = (value: string) => .replace(/^-+|-+$/g, '') .replace(/-{2,}/g, '-') || 'x'; +export const planWorkdirCleanup = (args: { + root: string; + rootEntries: { name: string; isDirectory: boolean }[]; + homesEntries: Record; + codeLeaves: Record; + activeWorkdirs: Set; + runningBoxUsernames: Set; +}): { removeDirs: string[] } => { + const removeDirs: string[] = []; + for (const entry of args.rootEntries) { + if (!entry.isDirectory || entry.name.startsWith('.')) continue; + if (entry.name === 'homes') continue; + const abs = path.resolve(args.root, entry.name); + if (args.activeWorkdirs.has(abs)) continue; + removeDirs.push(abs); + } + for (const [username, leaves] of Object.entries(args.codeLeaves)) { + if (args.runningBoxUsernames.has(username)) continue; + for (const leaf of leaves) removeDirs.push(leaf); + } + return { removeDirs }; +}; + const runClaim = async (claim: Claim) => { const jobId = claim.job._id; const secretValues = [ @@ -1296,11 +874,10 @@ const runClaim = async (claim: Claim) => { ...claim.secrets.map((secret) => secret.value), ].filter(Boolean); const redact = createRedactor(secretValues); - let acquiredBoxUser: string | undefined; + let acquiredBoxHandle: BoxHandle | undefined; + // job.runtime is authoritative (validated against the profile at queue time). + const runtime = (claim.job.runtime ?? 'opencode') as AgentRuntimeName; try { - if ((claim.job.runtime ?? 'opencode') !== 'opencode') { - throw new Error('Legacy OpenAI direct jobs are no longer supported.'); - } await updateStatus(jobId, 'preparing'); await appendEvent(jobId, 'info', 'clone', 'Creating installation token.'); if (!claim.github.installationId) { @@ -1327,12 +904,13 @@ const runClaim = async (claim: Claim) => { // Start (or reuse) the persistent per-user box that this thread — and the // terminal — exec into. It mounts the home, so the clone below is visible. - const boxName = await acquireUserBox({ + const boxHandle = await acquireUserBox({ username, workdir: homeDir, containerHome, }); - acquiredBoxUser = username; + acquiredBoxHandle = boxHandle; + const boxName = boxHandle.boxName; const repoDir = await cloneRepository({ workdir: checkoutParent, @@ -1354,8 +932,10 @@ const runClaim = async (claim: Claim) => { containerRepo, repoDir, boxName, + boxHandle, githubToken, redact, + runtime, }; if (userEnv) { await appendEvent( @@ -1372,9 +952,7 @@ const runClaim = async (claim: Claim) => { redact, }); } - if (isCodexLoginProfile(claim)) { - await prepareCodexAuth(workspace); - } + await getAdapter(runtime).prepareAuth(workspace); await materializeEnvFile(workspace); const detected = await detectPackageCommands(repoDir); await addArtifact({ @@ -1386,6 +964,7 @@ const runClaim = async (claim: Claim) => { }); activeWorkspaces.set(jobId, workspace); await markWorkspaceActive({ jobId }); + startHeartbeat(jobId); await updateStatus(jobId, 'running', { summary: 'Workspace is active.', }); @@ -1409,31 +988,46 @@ const runClaim = async (claim: Claim) => { }); } catch (error) { const message = error instanceof Error ? error.message : String(error); - await appendEvent( - jobId, - 'error', - 'cleanup', - truncate(redact(message), 20_000), - ); - await addArtifact({ - jobId, - kind: 'error', - title: 'Failure', - content: truncate(redact(message), 50_000), - contentType: 'text/plain', - }); - await updateStatus( - jobId, - message.toLowerCase().includes('timed out') ? 'timed_out' : 'failed', - { error: truncate(redact(message), 10_000) }, - ); - await markWorkspaceStopped( - jobId, - message.toLowerCase().includes('timed out') ? 'expired' : 'failed', - ).catch((stopError: unknown) => { - console.error(stopError); - }); - if (acquiredBoxUser) releaseUserBox(acquiredBoxUser); + const workspaceStatus = message.toLowerCase().includes('timed out') + ? 'expired' + : 'failed'; + try { + await appendEvent( + jobId, + 'error', + 'cleanup', + truncate(redact(message), 20_000), + ); + await addArtifact({ + jobId, + kind: 'error', + title: 'Failure', + content: truncate(redact(message), 50_000), + contentType: 'text/plain', + }); + await updateStatus( + jobId, + workspaceStatus === 'expired' ? 'timed_out' : 'failed', + { error: truncate(redact(message), 10_000) }, + ); + } finally { + const workspace = activeWorkspaces.get(jobId); + if (workspace) { + const teardown = await teardownActiveWorkspace({ + jobId, + workspace, + abortAgent: true, + workspaceStatus, + }); + reportTeardownErrors(teardown.errors); + } else { + stopHeartbeat(jobId); + await markWorkspaceStopped(jobId, workspaceStatus).catch( + (stopError: unknown) => console.error(stopError), + ); + acquiredBoxHandle?.release(); + } + } } }; @@ -1476,7 +1070,7 @@ export const listWorkspaceTree = async (jobId: string) => { export const readWorkspaceFile = async (jobId: string, filePath: string) => { const workspace = resolveWorkspace(jobId); - const target = safeWorkspacePath(workspace.repoDir, filePath); + const target = await assertContainedRealPath(workspace.repoDir, filePath); return await readFile(target, 'utf8'); }; @@ -1486,7 +1080,9 @@ export const writeWorkspaceFile = async ( content: string, ) => { const workspace = resolveWorkspace(jobId); - const target = safeWorkspacePath(workspace.repoDir, filePath); + const target = await assertContainedRealPath(workspace.repoDir, filePath, { + forWrite: true, + }); await mkdir(path.dirname(target), { recursive: true }); await writeFile(target, content); const diff = await getWorktreeDiff(workspace.repoDir, workspace.redact); @@ -1551,74 +1147,43 @@ export const getTerminalWorkspace = (jobId: string) => { export const getWorkspaceAgentStatus = (jobId: string) => { const workspace = resolveWorkspace(jobId); return { - runtimeMode: workspace.runtimeMode ?? 'legacy_cli', - opencodeSessionId: workspace.opencodeSession?.sessionId, + runtimeMode: workspace.runtime, codexSessionId: workspace.codexSessionId, - containerId: workspace.containerId, + opencodeSessionId: workspace.opencodeSessionId, + claudeSessionId: workspace.claudeSessionId, active: Boolean(workspace.agentTurnActive), }; }; -export const abortWorkspaceAgent = async (jobId: string) => { - const workspace = resolveWorkspace(jobId); - if (workspace.opencodeSession) { - await abortOpenCodeSession(workspace.opencodeSession); - workspace.agentTurnActive = false; - workspace.resolveTurn?.(); - workspace.resolveTurn = undefined; - await appendEvent( - workspace.claim.job._id, - 'warn', - 'cleanup', - 'Agent turn aborted.', - ); - return { success: true }; - } - if (workspace.runtimeMode === 'codex_exec') { - throw new Error('Codex agent turns cannot be aborted from Spoon yet.'); - } +const abortWorkspaceAgentForWorkspace = async (workspace: ActiveWorkspace) => { + await getAdapter(workspace.runtime).abort(workspace); + workspace.agentTurnActive = false; + workspace.resolveTurn?.(); + workspace.resolveTurn = undefined; + await appendEvent( + workspace.claim.job._id, + 'warn', + 'cleanup', + 'Agent turn aborted.', + ); return { success: true }; }; +export const abortWorkspaceAgent = async (jobId: string) => + await abortWorkspaceAgentForWorkspace(resolveWorkspace(jobId)); + export const replyToInteraction = async ( - jobId: string, - args: { + _jobId: string, + _args: { interactionId: Id<'agentInteractionRequests'>; externalRequestId: string; response: string; }, + // eslint-disable-next-line @typescript-eslint/require-await -- keep async so callers receive a rejected promise ) => { - const workspace = resolveWorkspace(jobId); - if (workspace.runtimeMode === 'codex_exec') { - throw new Error('Codex interaction replies are not supported yet.'); - } - if (!workspace.opencodeSession) { - throw new Error('OpenCode session is not active.'); - } - const mapped = - args.response === 'reject' - ? 'reject' - : args.response === 'always' - ? 'always' - : 'once'; - await replyOpenCodePermission({ - session: workspace.opencodeSession, - permissionId: args.externalRequestId, - response: mapped, - directory: workspace.containerRepo, - }); - await patchInteractionRequest({ - interactionId: args.interactionId, - status: mapped === 'reject' ? 'rejected' : 'approved', - response: mapped, - }); - await appendMessage({ - jobId: workspace.claim.job._id, - role: 'system', - status: 'completed', - content: `Interaction ${mapped === 'reject' ? 'rejected' : 'approved'}.`, - }); - return { success: true }; + // Every runtime now runs its CLI non-interactively inside the box, so no + // runtime emits an interaction request that expects a reply. + throw new Error('Interactive replies are not supported by exec-based runtimes.'); }; export const sendWorkspaceMessage = async ( @@ -1631,6 +1196,9 @@ export const sendWorkspaceMessage = async ( if (workspace.agentTurnActive) { throw new Error('Wait for the current agent turn to finish or abort it.'); } + // Re-materialize the env file fresh each turn so it is present for the + // adapter and can be removed between turns / on teardown. + await materializeEnvFile(workspace); if (options.recordUserMessage ?? true) { await appendMessage({ jobId: claim.job._id, @@ -1651,84 +1219,80 @@ export const sendWorkspaceMessage = async ( content: '', }); const assistantContent = { value: '' }; - if (isCodexLoginProfile(claim)) { - await appendEvent( - claim.job._id, - 'info', - 'plan', - 'Starting Codex CLI turn with the configured login profile.', - ); - await runCodexTurn({ - workspace, - prompt, - assistantMessageId, - assistantContent, - }); - console.log( - `Codex turn completed for job ${claim.job._id}; response length=${assistantContent.value.length}`, - ); - } else if (env.runtime === 'docker') { - await appendEvent( - claim.job._id, - 'info', - 'plan', - 'Starting OpenCode server turn with the configured API provider.', - ); - await runOpenCodeTurn({ - workspace, - prompt, - assistantMessageId, - assistantContent, - }); - } else { - const aiEnv = providerEnvironment(claim); - const secretEnv = Object.fromEntries( - claim.secrets.map((secret) => [secret.name, secret.value]), - ); - const result = await run( - 'bash', - [ - '-lc', - `opencode run --format json --model ${quoteShell(opencodeModel(claim))} ${quoteShell(prompt)}`, - ], - { - cwd: workspace.repoDir, - env: { - ...aiEnv, - ...secretEnv, - }, - redact, - timeoutMs: env.jobTimeoutMs, - }, - ); - await updateMessage({ - messageId: assistantMessageId, - status: result.exitCode === 0 ? 'completed' : 'failed', - content: truncate(result.output, 40_000), - }); - if (result.exitCode !== 0) { - throw new Error(`${agentFailurePrefix(claim)}:\n${result.output}`); - } + // Every runtime now runs its CLI inside the per-user box via its adapter + // (Codex `codex exec`, OpenCode `opencode run`); there is no per-job side + // container. `runtimeMode` keeps its legacy label per runtime for the UI. + const messageId = assistantMessageId; + const runtimeMode = + workspace.runtime === 'codex' ? 'codex_exec' : 'opencode_server'; + workspace.runtimeMode = runtimeMode; + await appendEvent( + claim.job._id, + 'info', + 'plan', + `Starting ${workspace.runtime} turn in the workspace box.`, + ); + await setRuntimeSession({ + jobId: claim.job._id, + agentRuntimeMode: runtimeMode, + codexSessionId: workspace.codexSessionId, + opencodeSessionId: workspace.opencodeSessionId, + }); + if (workspace.cancelRequested) { + throw new Error('Workspace cancellation requested.'); } - if (isCodexLoginProfile(claim)) { + const onEvent = (event: NormalizedAgentEvent) => + handleAgentEvent({ + workspace, + event, + assistantMessageId: messageId, + assistantContent, + }); + const adapter = getAdapter(workspace.runtime); + const result = await adapter.runTurn(workspace, prompt, onEvent); + if (result.sessionId) { + await persistRuntimeSession(workspace, result.sessionId); + } + const recoveredText = result.finalMessage + ? truncate(redact(result.finalMessage), 40_000) + : undefined; + const outcome = resolveTurnOutcome({ + assistantText: assistantContent.value, + recoveredText, + error: result.error, + runtime: workspace.runtime, + }); + // Persist any recovered finalMessage text so the user still sees it, even + // when the turn ultimately failed with partial output. + if (outcome.text !== assistantContent.value) { + assistantContent.value = outcome.text; + await updateMessage({ + messageId, + content: assistantContent.value, + status: 'streaming', + }); + } + // A hard failure (nonzero exit / failure event) surfaces the turn as failed + // even when partial assistant text streamed — the catch path marks the + // message failed and appends the error event. This applies to every runtime, + // so a nonzero-exit OpenCode turn with partial text also surfaces as failed. + if (outcome.failure) { if (!assistantContent.value.trim()) { console.error( - `Codex completed without producing an assistant response for job ${claim.job._id}.`, - ); - throw new Error( - workspace.codexTurnError - ? `Codex failed: ${workspace.codexTurnError}` - : 'Codex completed without producing an assistant response.', + `${workspace.runtime} completed without producing an assistant response for job ${claim.job._id}.`, ); } - await updateMessage({ - messageId: assistantMessageId, - status: 'completed', - content: assistantContent.value, - }); - workspace.agentTurnActive = false; + throw new Error(outcome.failure); } + await updateMessage({ + messageId, + status: 'completed', + content: assistantContent.value, + }); workspace.agentTurnActive = false; + console.log( + `${workspace.runtime} turn completed for job ${claim.job._id}; response length=${assistantContent.value.length}`, + ); if (claim.job.jobType === 'maintenance_review') { const decision = parseMaintenanceDecision(assistantContent.value); if (decision) { @@ -1740,6 +1304,10 @@ export const sendWorkspaceMessage = async ( contentType: 'application/json', }); await applyMaintenanceDecision(claim.job._id, decision); + await stopWorkspace(claim.job._id).catch((error: unknown) => { + console.error(error); + }); + return; } else { await updateStatus(claim.job._id, 'changes_ready', { summary: @@ -1786,6 +1354,60 @@ export const sendWorkspaceMessage = async ( } }; +const teardownActiveWorkspace = async (args: { + jobId: string; + workspace: ActiveWorkspace; + abortAgent: boolean; + appendCancellationEvent?: boolean; + workspaceStatus?: 'stopped' | 'expired' | 'failed'; +}) => { + const { jobId, workspace } = args; + if (args.abortAgent) workspace.cancelRequested = true; + // The per-user box is shared across a user's threads and reference-counted by + // the box registry, so teardown only aborts the turn and releases the box + // handle — there is no per-job container to stop or agent session to close. + return await teardownWorkspace({ + stopHeartbeat: () => stopHeartbeat(jobId), + removeActive: () => activeWorkspaces.delete(jobId), + appendEvent: args.appendCancellationEvent + ? async () => + await appendEvent( + workspace.claim.job._id, + 'warn', + 'cleanup', + 'Cancellation requested; stopping workspace.', + ) + : undefined, + abortAgent: args.abortAgent + ? async () => await abortWorkspaceAgentForWorkspace(workspace) + : undefined, + removeMaterializedEnv: async () => + await removeMaterializedEnv(workspace.materializedEnvPath), + markStopped: async () => + await markWorkspaceStopped(workspace.claim.job._id, args.workspaceStatus), + release: () => workspace.boxHandle.release(), + }); +}; + +const reportTeardownErrors = (errors: unknown[]) => { + for (const error of errors) console.error(error); +}; + +const cancelWorkspace = async (jobId: string) => { + const workspace = activeWorkspaces.get(jobId); + if (!workspace) { + stopHeartbeat(jobId); + return; + } + const result = await teardownActiveWorkspace({ + jobId, + workspace, + abortAgent: true, + appendCancellationEvent: true, + }); + reportTeardownErrors(result.errors); +}; + export const openWorkspacePullRequest = async (jobId: string) => { const workspace = resolveWorkspace(jobId); const { claim, repoDir, redact } = workspace; @@ -1848,15 +1470,14 @@ export const openWorkspacePullRequest = async (jobId: string) => { pullRequestNumber: pullRequest.number, summary: 'Draft PR opened from interactive workspace.', }); - await markWorkspaceStopped(claim.job._id); - workspace.opencodeSession?.close(); - if (workspace.containerName) { - await stopWorkspaceContainer(workspace.containerName); + const teardown = await teardownActiveWorkspace({ + jobId, + workspace, + abortAgent: false, + }); + if (teardown.errors.length > 0) { + throw new AggregateError(teardown.errors, 'Workspace teardown failed.'); } - activeWorkspaces.delete(jobId); - // The persistent per-user home + ~/Code checkouts survive across sessions; - // release the box (reaped once no other thread/terminal holds it). - releaseUserBox(workspace.username); return { pullRequestUrl: pullRequest.html_url, pullRequestNumber: pullRequest.number, @@ -1865,27 +1486,26 @@ export const openWorkspacePullRequest = async (jobId: string) => { export const stopWorkspace = async (jobId: string) => { const workspace = resolveWorkspace(jobId); - await markWorkspaceStopped(workspace.claim.job._id); - workspace.opencodeSession?.close(); - if (workspace.containerName) { - await stopWorkspaceContainer(workspace.containerName); + const teardown = await teardownActiveWorkspace({ + jobId, + workspace, + abortAgent: true, + }); + if (teardown.errors.length > 0) { + throw new AggregateError(teardown.errors, 'Workspace teardown failed.'); } - activeWorkspaces.delete(jobId); - // The persistent per-user home + ~/Code checkouts survive across sessions; - // release the box (reaped once no other thread/terminal holds it). - releaseUserBox(workspace.username); return { success: true }; }; export const getWorkerHealth = async () => { const active = [...activeWorkspaces.entries()].map(([jobId, workspace]) => ({ jobId, - runtimeMode: workspace.runtimeMode ?? 'legacy_cli', - containerName: workspace.containerName, + runtimeMode: workspace.runtime, + boxName: workspace.boxName, workdir: workspace.workdir, agentTurnActive: Boolean(workspace.agentTurnActive), })); - const containerNames = await listWorkspaceContainerNames('spoon-agent-job-'); + const boxContainers = await listWorkspaceContainerNames('spoon-box-'); return { ok: true, buildSha: env.buildSha, @@ -1894,48 +1514,77 @@ export const getWorkerHealth = async () => { convexUrl: env.convexUrl, runtime: env.runtime, containerRuntime: env.containerRuntime, - containerAccess: env.containerAccess, jobImage: env.jobImage, workdir: env.workdir, hostWorkdir: env.hostWorkdir, network: env.network, httpPort: env.httpPort, - maxConcurrentJobs: env.maxConcurrentJobs, jobTimeoutMs: env.jobTimeoutMs, activeWorkspaceCount: active.length, activeWorkspaces: active, - workspaceContainers: containerNames, + boxContainers, }; }; export const cleanupOrphanedWorkspaces = async () => { - const activeContainers = new Set( - [...activeWorkspaces.values()] - .map((workspace) => workspace.containerName) - .filter((value): value is string => Boolean(value)), - ); + // Per-user boxes are shared and reference-counted, so cleanup no longer force + // removes containers here — it only reclaims orphaned workdirs and reports the + // live boxes. (Job containers no longer exist.) const activeWorkdirs = new Set( [...activeWorkspaces.values()].map((workspace) => path.resolve(workspace.workdir), ), ); - const removedContainers: string[] = []; - for (const containerName of await listWorkspaceContainerNames( - 'spoon-agent-job-', - )) { - if (activeContainers.has(containerName)) continue; - await stopWorkspaceContainer(containerName); - removedContainers.push(containerName); - } - const removedWorkdirs: string[] = []; const root = path.resolve(env.workdir); + const removedWorkdirs: string[] = []; try { - const entries = await readdir(root, { withFileTypes: true }); - for (const entry of entries) { - if (!entry.isDirectory() || entry.name.startsWith('.')) continue; - const target = path.resolve(root, entry.name); - if (activeWorkdirs.has(target)) continue; + const rootDirents = await readdir(root, { withFileTypes: true }); + const rootEntries = rootDirents.map((d) => ({ + name: d.name, + isDirectory: d.isDirectory(), + })); + const homesRoot = path.join(root, 'homes'); + const homesEntries: Record< + string, + { name: string; isDirectory: boolean }[] + > = {}; + const codeLeaves: Record = {}; + const homesDirents = await readdir(homesRoot, { + withFileTypes: true, + }).catch(() => []); + for (const userDir of homesDirents) { + if (!userDir.isDirectory()) continue; + const username = userDir.name; + const codeRoot = path.join(homesRoot, username, 'Code'); + homesEntries[username] = []; + const leaves: string[] = []; + const spoons = await readdir(codeRoot, { withFileTypes: true }).catch( + () => [], + ); + for (const spoonDir of spoons) { + if (!spoonDir.isDirectory()) continue; + const spoonRoot = path.join(codeRoot, spoonDir.name); + const branches = await readdir(spoonRoot, { + withFileTypes: true, + }).catch(() => []); + for (const branchDir of branches) { + if (branchDir.isDirectory()) { + leaves.push(path.join(spoonRoot, branchDir.name)); + } + } + } + codeLeaves[username] = leaves; + } + const { removeDirs } = planWorkdirCleanup({ + root, + rootEntries, + homesEntries, + codeLeaves, + activeWorkdirs, + runningBoxUsernames: runningBoxUsernames(), + }); + for (const target of removeDirs) { await rm(target, { recursive: true, force: true }); removedWorkdirs.push(target); } @@ -1946,11 +1595,8 @@ export const cleanupOrphanedWorkspaces = async () => { } } - return { - success: true, - removedContainers, - removedWorkdirs, - }; + const boxContainers = await listWorkspaceContainerNames('spoon-box-'); + return { success: true, removedWorkdirs, boxContainers }; }; export const startWorker = async () => { @@ -1965,7 +1611,10 @@ export const startWorker = async () => { await sleep(env.pollMs); continue; } - await runClaim(claim); + // The generated `Doc<'agentJobs'>.runtime` still includes the legacy + // `openai_direct` literal (queue-time validation now excludes it), so + // narrow to the worker's codex/opencode/claude `Claim` shape here. + await runClaim(claim as Claim); } catch (error) { console.error(error); await sleep(env.pollMs); diff --git a/apps/agent-worker/src/workspace-teardown.ts b/apps/agent-worker/src/workspace-teardown.ts new file mode 100644 index 0000000..72c116d --- /dev/null +++ b/apps/agent-worker/src/workspace-teardown.ts @@ -0,0 +1,37 @@ +type TeardownStep = () => unknown; + +export const teardownWorkspace = async (steps: { + stopHeartbeat: TeardownStep; + removeActive: TeardownStep; + appendEvent?: TeardownStep; + abortAgent?: TeardownStep; + removeMaterializedEnv?: TeardownStep; + markStopped?: TeardownStep; + closeAgent?: TeardownStep; + stopContainer?: TeardownStep; + release: TeardownStep; +}) => { + const errors: unknown[] = []; + const call = (step: TeardownStep | undefined) => { + if (!step) return Promise.resolve(); + try { + return Promise.resolve(step()).catch((error: unknown) => { + errors.push(error); + }); + } catch (error) { + errors.push(error); + return Promise.resolve(); + } + }; + + const immediate = [call(steps.stopHeartbeat), call(steps.removeActive)]; + await Promise.all(immediate); + await call(steps.appendEvent); + await call(steps.abortAgent); + await call(steps.removeMaterializedEnv); + await call(steps.markStopped); + await call(steps.closeAgent); + await call(steps.stopContainer); + await call(steps.release); + return { errors }; +}; diff --git a/apps/agent-worker/tests/unit/agent-events.test.ts b/apps/agent-worker/tests/unit/agent-events.test.ts index da4baae..075e9e7 100644 --- a/apps/agent-worker/tests/unit/agent-events.test.ts +++ b/apps/agent-worker/tests/unit/agent-events.test.ts @@ -1,8 +1,10 @@ import { describe, expect, test } from 'vitest'; import { + normalizeClaudeJsonLine, normalizeCodexJsonLine, normalizeOpenCodeEvent, + normalizeOpenCodeRunLine, } from '../../src/agent-events'; describe('agent event normalization', () => { @@ -291,4 +293,110 @@ describe('agent event normalization', () => { externalMessageId: 'message-2', }); }); + + test('normalizes opencode run --format json output lines', () => { + expect( + normalizeOpenCodeRunLine( + JSON.stringify({ + type: 'message.part.delta', + properties: { part: { text: 'hi' }, messageID: 'm1' }, + }), + ), + ).toContainEqual({ + kind: 'assistant_delta', + content: 'hi', + externalMessageId: 'm1', + }); + + expect( + normalizeOpenCodeRunLine( + JSON.stringify({ + sessionID: 'ses_abc', + parts: [{ text: 'final answer' }], + }), + ), + ).toEqual( + expect.arrayContaining([ + { kind: 'session', sessionId: 'ses_abc' }, + expect.objectContaining({ + kind: 'assistant_completed', + content: 'final answer', + }), + ]), + ); + + expect(normalizeOpenCodeRunLine('not json at all')).toContainEqual({ + kind: 'status', + status: 'not json at all', + }); + }); + + test('normalizes Claude Code stream-json output lines', () => { + expect( + normalizeClaudeJsonLine( + JSON.stringify({ + type: 'system', + subtype: 'init', + session_id: 'abc', + }), + ), + ).toContainEqual({ kind: 'session', sessionId: 'abc' }); + + expect( + normalizeClaudeJsonLine( + JSON.stringify({ + type: 'assistant', + message: { + content: [ + { type: 'text', text: 'hello ' }, + { type: 'text', text: 'world' }, + ], + }, + }), + ), + ).toContainEqual({ kind: 'assistant_delta', content: 'hello world' }); + + expect( + normalizeClaudeJsonLine( + JSON.stringify({ + type: 'assistant', + message: { + content: [ + { type: 'tool_use', name: 'Bash', input: { command: 'ls' } }, + ], + }, + }), + ), + ).toContainEqual({ + kind: 'tool_started', + name: 'Bash', + input: JSON.stringify({ command: 'ls' }), + }); + + expect( + normalizeClaudeJsonLine( + JSON.stringify({ + type: 'result', + subtype: 'success', + result: 'final text', + session_id: 'abc', + }), + ), + ).toContainEqual({ kind: 'assistant_completed', content: 'final text' }); + + expect( + normalizeClaudeJsonLine( + JSON.stringify({ + type: 'result', + subtype: 'error_max_turns', + result: 'hit the turn limit', + }), + ), + ).toContainEqual({ kind: 'error', message: 'hit the turn limit' }); + + expect(normalizeClaudeJsonLine('not json at all')).toContainEqual({ + kind: 'status', + status: 'not json at all', + }); + }); }); diff --git a/apps/agent-worker/tests/unit/agent-runtime.test.ts b/apps/agent-worker/tests/unit/agent-runtime.test.ts new file mode 100644 index 0000000..c37238f --- /dev/null +++ b/apps/agent-worker/tests/unit/agent-runtime.test.ts @@ -0,0 +1,42 @@ +import { describe, expect, test } from 'vitest'; + +import type { AgentRuntime } from '../../src/runtime/agent-runtime'; +import type { Claim } from '../../src/runtime/provider'; +import { getAdapter, registerAdapter } from '../../src/runtime/agent-runtime'; +import { isCodexLoginProfile, opencodeModel } from '../../src/runtime/provider'; + +describe('agent runtime registry', () => { + test('registerAdapter/getAdapter resolves a registered runtime', () => { + const fake: AgentRuntime = { + name: 'codex', + prepareAuth: () => Promise.resolve(), + runTurn: () => Promise.resolve({}), + abort: () => Promise.resolve(), + }; + registerAdapter('codex', () => fake); + expect(getAdapter('codex').name).toBe('codex'); + }); + + test('getAdapter throws for an unregistered runtime', () => { + expect(() => getAdapter('opencode')).toThrow(/No agent runtime adapter/); + }); +}); + +describe('shared provider helpers re-exported from runtime/provider', () => { + test('isCodexLoginProfile detects opencode login profiles', () => { + expect( + isCodexLoginProfile({ + aiProviderProfile: { provider: 'opencode_openai_login' }, + } as Claim), + ).toBe(true); + expect(isCodexLoginProfile({} as Claim)).toBe(false); + }); + + test('opencodeModel prefixes the provider', () => { + expect( + opencodeModel({ + aiProviderProfile: { provider: 'anthropic', model: 'claude-x' }, + } as Claim), + ).toBe('anthropic/claude-x'); + }); +}); diff --git a/apps/agent-worker/tests/unit/box-files.test.ts b/apps/agent-worker/tests/unit/box-files.test.ts new file mode 100644 index 0000000..e01a046 --- /dev/null +++ b/apps/agent-worker/tests/unit/box-files.test.ts @@ -0,0 +1,84 @@ +import { + mkdir, + mkdtemp, + rm, + symlink, + writeFile, +} from 'node:fs/promises'; +import os from 'node:os'; +import path from 'node:path'; +import { afterEach, describe, expect, test, vi } from 'vitest'; + +const tempDirs: string[] = []; + +// Point boxHomePaths('t').homeDir at a real tmp home, then import box fresh so +// env.workdir (captured at import) reflects the tmp workdir for this test. +const loadWithHome = async () => { + const workdir = await mkdtemp(path.join(os.tmpdir(), 'spoon-box-')); + tempDirs.push(workdir); + vi.resetModules(); + process.env.SPOON_WORKER_TOKEN = 'test-worker-token'; + process.env.GITHUB_APP_ID = '123'; + process.env.GITHUB_APP_PRIVATE_KEY = + '-----BEGIN PRIVATE KEY-----\ntest\n-----END PRIVATE KEY-----'; + process.env.SPOON_AGENT_WORKDIR = workdir; + const box = await import('../../src/box'); + const homeDir = box.boxHomePaths('t').homeDir; + await mkdir(homeDir, { recursive: true }); + return { box, homeDir, workdir }; +}; + +describe('box file helpers', () => { + afterEach(async () => { + await Promise.all( + tempDirs.map((dir) => rm(dir, { force: true, recursive: true })), + ); + tempDirs.length = 0; + vi.resetModules(); + }); + + test('writeBoxFile then readBoxFile round-trips', async () => { + const { box } = await loadWithHome(); + await expect( + box.writeBoxFile('t', 'notes/todo.txt', 'hello box'), + ).resolves.toEqual({ success: true }); + await expect(box.readBoxFile('t', 'notes/todo.txt')).resolves.toBe( + 'hello box', + ); + }); + + test('readBoxFile rejects a lexical .. escape', async () => { + const { box } = await loadWithHome(); + await expect( + box.readBoxFile('t', '../../etc/passwd'), + ).rejects.toThrow(/outside home/); + }); + + test('rejects a symlink inside home that points outside on read and write', async () => { + const { box, homeDir, workdir } = await loadWithHome(); + const outside = path.join(workdir, 'outside'); + await mkdir(outside, { recursive: true }); + await writeFile(path.join(outside, 'secret'), 'S'); + await symlink(path.join(outside, 'secret'), path.join(homeDir, 'link')); + + await expect(box.readBoxFile('t', 'link')).rejects.toThrow(/outside home/); + await expect( + box.writeBoxFile('t', 'link', 'nope'), + ).rejects.toThrow(); + }); + + test('listBoxTree includes a created file and excludes node_modules', async () => { + const { box, homeDir } = await loadWithHome(); + await writeFile(path.join(homeDir, 'readme.md'), '# hi'); + await mkdir(path.join(homeDir, 'node_modules', 'pkg'), { recursive: true }); + await writeFile(path.join(homeDir, 'node_modules', 'pkg', 'x.js'), '1'); + + const tree = await box.listBoxTree('t'); + expect(tree.name).toBe('~'); + expect(tree.path).toBe(''); + expect(tree.type).toBe('directory'); + const names = (tree.children ?? []).map((child) => child.name); + expect(names).toContain('readme.md'); + expect(names).not.toContain('node_modules'); + }); +}); diff --git a/apps/agent-worker/tests/unit/box-paths.test.ts b/apps/agent-worker/tests/unit/box-paths.test.ts new file mode 100644 index 0000000..ed524f6 --- /dev/null +++ b/apps/agent-worker/tests/unit/box-paths.test.ts @@ -0,0 +1,29 @@ +import { afterEach, describe, expect, test, vi } from 'vitest'; + +const load = async () => { + vi.resetModules(); + process.env.SPOON_WORKER_TOKEN = 'test-worker-token'; + process.env.GITHUB_APP_ID = '123'; + process.env.GITHUB_APP_PRIVATE_KEY = + '-----BEGIN PRIVATE KEY-----\ntest\n-----END PRIVATE KEY-----'; + process.env.SPOON_AGENT_WORKDIR = '/work'; + return await import('../../src/box'); +}; + +describe('boxHomePaths', () => { + afterEach(() => vi.resetModules()); + + test('derives the per-user home dir and container home', async () => { + const { boxHomePaths } = await load(); + const paths = boxHomePaths('gib'); + expect(paths.homeDir.endsWith('homes/gib')).toBe(true); + expect(paths.containerHome).toBe('/home/gib'); + }); + + test('distinct usernames produce distinct home dirs', async () => { + const { boxHomePaths } = await load(); + expect(boxHomePaths('gib').homeDir).not.toBe( + boxHomePaths('other').homeDir, + ); + }); +}); diff --git a/apps/agent-worker/tests/unit/box-process-kill.test.ts b/apps/agent-worker/tests/unit/box-process-kill.test.ts new file mode 100644 index 0000000..0d67a43 --- /dev/null +++ b/apps/agent-worker/tests/unit/box-process-kill.test.ts @@ -0,0 +1,75 @@ +import { describe, expect, test } from 'vitest'; + +const loadDocker = async () => { + process.env.SPOON_WORKER_TOKEN = 'test-worker-token'; + process.env.GITHUB_APP_ID = '123'; + process.env.GITHUB_APP_PRIVATE_KEY = + '-----BEGIN PRIVATE KEY-----\\ntest\\n-----END PRIVATE KEY-----'; + return await import('../../src/runtime/docker'); +}; + +describe('in-box process-group marker + kill helpers', () => { + test('buildMarkedCommand wraps argv as a setsid bash session carrying the marker', async () => { + const { buildMarkedCommand } = await loadDocker(); + const argv = buildMarkedCommand('spoon-turn-abc', [ + 'codex', + 'exec', + '--json', + 'hi', + ]); + expect(argv[0]).toBe('setsid'); + expect(argv[1]).toBe('bash'); + expect(argv[2]).toBe('-lc'); + const script = argv[3] ?? ''; + expect(script).toContain('# spoon-turn-abc'); + expect(script).toContain("'codex' 'exec' '--json' 'hi'"); + }); + + test('buildMarkedCommand keeps a trailing exit after the CLI so bash survives tail-exec while propagating the exit code (C1 guard)', async () => { + const { buildMarkedCommand } = await loadDocker(); + const script = buildMarkedCommand('spoon-turn-abc', ['codex', 'run'])[3] ?? ''; + // The CLI must NOT be the final simple command (bash would tail-exec it, + // replacing the marker-carrying bash). The exit code must still propagate. + expect(script).toContain("'codex' 'run'\nrc=$?\nexit \"$rc\""); + expect(script.split('\n').at(-1)).toBe('exit "$rc"'); + expect(script.split('\n').at(-1)).not.toBe("'codex' 'run'"); + }); + + test('buildMarkedCommand rejects a marker that could break out of the comment', async () => { + const { buildMarkedCommand } = await loadDocker(); + expect(() => buildMarkedCommand('bad\nmarker', ['codex'])).toThrow(); + }); + + test('buildKillScript TERM-then-KILLs every group matching the marker', async () => { + const { buildKillScript } = await loadDocker(); + const script = buildKillScript('spoon-turn-abc'); + expect(script).toContain("pgrep -f 'spoon-turn-abc'"); + expect(script).toContain('kill -TERM -"$pgid"'); + expect(script).toContain('kill -KILL -"$pgid"'); + }); + + test('buildKillScript excludes its own process group so it does not kill itself (C2 guard)', async () => { + const { buildKillScript } = await loadDocker(); + const script = buildKillScript('spoon-turn-abc'); + // Computes its own pgid and skips any match sharing it (the kill script's + // own bash + its command-substitution subshells all carry the marker). + expect(script).toContain('self_pgid=$(ps -o pgid= -p $$ | tr -d \' \')'); + expect(script).toContain('[ "$pgid" = "$self_pgid" ] && continue'); + }); + + test('buildKillScript escalates TERM -> KILL with a sleep 2 in between (order matters)', async () => { + const { buildKillScript } = await loadDocker(); + const script = buildKillScript('spoon-turn-abc'); + const termIdx = script.indexOf('kill -TERM -"$pgid"'); + const sleepIdx = script.indexOf('\nsleep 2\n'); + const killIdx = script.indexOf('kill -KILL -"$pgid"'); + expect(termIdx).toBeGreaterThanOrEqual(0); + expect(sleepIdx).toBeGreaterThan(termIdx); + expect(killIdx).toBeGreaterThan(sleepIdx); + }); + + test('buildKillScript rejects a marker that could break out of the pgrep pattern', async () => { + const { buildKillScript } = await loadDocker(); + expect(() => buildKillScript('bad\nmarker')).toThrow(); + }); +}); diff --git a/apps/agent-worker/tests/unit/box-route.test.ts b/apps/agent-worker/tests/unit/box-route.test.ts new file mode 100644 index 0000000..09c1a9e --- /dev/null +++ b/apps/agent-worker/tests/unit/box-route.test.ts @@ -0,0 +1,32 @@ +import { afterEach, describe, expect, test, vi } from 'vitest'; + +const load = async () => { + vi.resetModules(); + process.env.SPOON_WORKER_TOKEN = 'test-worker-token'; + process.env.GITHUB_APP_ID = '123'; + process.env.GITHUB_APP_PRIVATE_KEY = + '-----BEGIN PRIVATE KEY-----\ntest\n-----END PRIVATE KEY-----'; + process.env.SPOON_AGENT_WORKDIR = '/work'; + return await import('../../src/server'); +}; + +describe('boxRoute', () => { + afterEach(() => vi.resetModules()); + + test('matches the four box actions', async () => { + const { boxRoute } = await load(); + expect(boxRoute('/box/status')).toEqual({ action: 'status' }); + expect(boxRoute('/box/lifecycle')).toEqual({ action: 'lifecycle' }); + expect(boxRoute('/box/tree')).toEqual({ action: 'tree' }); + expect(boxRoute('/box/file')).toEqual({ action: 'file' }); + }); + + test('rejects unknown, nested, and trailing paths', async () => { + const { boxRoute } = await load(); + expect(boxRoute('/box/other')).toBeNull(); + expect(boxRoute('/box/file/extra')).toBeNull(); + expect(boxRoute('/box/status/')).toBeNull(); + expect(boxRoute('/box')).toBeNull(); + expect(boxRoute('/jobs/x/file')).toBeNull(); + }); +}); diff --git a/apps/agent-worker/tests/unit/box-terminal-token.test.ts b/apps/agent-worker/tests/unit/box-terminal-token.test.ts new file mode 100644 index 0000000..2f1b48c --- /dev/null +++ b/apps/agent-worker/tests/unit/box-terminal-token.test.ts @@ -0,0 +1,76 @@ +import { createHmac } from 'node:crypto'; +import { describe, expect, test } from 'vitest'; + +import { + parseBoxTokenUsername, + verifyBoxTerminalToken, +} from '../../src/terminal-token'; + +const mintBox = (username: string, expiresAt: number, secret: string) => { + const payload = `${expiresAt}.box.${username}`; + const sig = createHmac('sha256', secret).update(payload).digest('hex'); + return `${payload}.${sig}`; +}; + +describe('verifyBoxTerminalToken', () => { + const secret = 'test-secret'; + + test('accepts a valid, unexpired, username-matched token', () => { + const token = mintBox('alice', Date.now() + 60_000, secret); + expect(verifyBoxTerminalToken(token, 'alice', secret)).toBe(true); + }); + + test('rejects an expired token', () => { + const token = mintBox('alice', Date.now() - 1, secret); + expect(verifyBoxTerminalToken(token, 'alice', secret)).toBe(false); + }); + + test('rejects a token minted for another username', () => { + const token = mintBox('alice', Date.now() + 60_000, secret); + expect(verifyBoxTerminalToken(token, 'bob', secret)).toBe(false); + }); + + test('rejects a token signed with a different secret', () => { + const token = mintBox('alice', Date.now() + 60_000, 'other-secret'); + expect(verifyBoxTerminalToken(token, 'alice', secret)).toBe(false); + }); + + test('rejects malformed input and an empty secret', () => { + expect(verifyBoxTerminalToken('garbage', 'alice', secret)).toBe(false); + expect(verifyBoxTerminalToken('', 'alice', secret)).toBe(false); + expect( + verifyBoxTerminalToken(mintBox('alice', Date.now() + 1000, ''), 'alice', ''), + ).toBe(false); + }); + + test('rejects a 3-part job token (cross-scheme confusion guard)', () => { + const payload = `${Date.now() + 60_000}.alice`; + const sig = createHmac('sha256', secret).update(payload).digest('hex'); + const jobToken = `${payload}.${sig}`; + expect(verifyBoxTerminalToken(jobToken, 'alice', secret)).toBe(false); + }); +}); + +describe('parseBoxTokenUsername', () => { + test('extracts the username from a well-formed box token', () => { + const token = mintBox('alice', Date.now() + 60_000, 'test-secret'); + expect(parseBoxTokenUsername(token)).toBe('alice'); + }); + + test('returns null for a 3-part job token', () => { + expect(parseBoxTokenUsername('123.alice.sig')).toBeNull(); + }); + + test('returns null when the literal box segment is missing', () => { + expect(parseBoxTokenUsername('123.notbox.alice.sig')).toBeNull(); + }); + + test('returns null for an empty username segment', () => { + expect(parseBoxTokenUsername('123.box..sig')).toBeNull(); + }); + + test('returns null for garbage input', () => { + expect(parseBoxTokenUsername('garbage')).toBeNull(); + expect(parseBoxTokenUsername('')).toBeNull(); + }); +}); diff --git a/apps/agent-worker/tests/unit/claude-adapter.test.ts b/apps/agent-worker/tests/unit/claude-adapter.test.ts new file mode 100644 index 0000000..c60d938 --- /dev/null +++ b/apps/agent-worker/tests/unit/claude-adapter.test.ts @@ -0,0 +1,177 @@ +import { mkdtemp, readFile, rm, stat } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import path from 'node:path'; + +import { afterEach, describe, expect, test, vi } from 'vitest'; + +import type { NormalizedAgentEvent } from '../../src/agent-events'; +import type { ExecStreamFn } from '../../src/runtime/agent-runtime'; +import type { AdapterWorkspace, Claim } from '../../src/runtime/provider'; + +const loadAdapter = async () => { + process.env.SPOON_WORKER_TOKEN = 'test-worker-token'; + process.env.GITHUB_APP_ID = '123'; + process.env.GITHUB_APP_PRIVATE_KEY = + '-----BEGIN PRIVATE KEY-----\\ntest\\n-----END PRIVATE KEY-----'; + return await import('../../src/runtime/claude-adapter'); +}; + +const claim = { + job: { + _id: 'job-1', + prompt: 'hi', + baseBranch: 'main', + workBranch: 'spoon/x', + forkOwner: 'o', + forkRepo: 'r', + upstreamOwner: 'u', + upstreamRepo: 'r', + }, + spoon: { name: 'Spoon' }, + openai: { model: 'claude-sonnet-4', reasoningEffort: 'medium' }, + aiProviderProfile: { + id: 'p', + name: 'Anthropic', + provider: 'anthropic', + authType: 'api_key', + secret: 'sk-ant-test', + model: 'claude-sonnet-4', + reasoningEffort: 'medium', + }, + github: {}, + secrets: [], +} as unknown as Claim; + +const makeWorkspace = (): AdapterWorkspace => ({ + claim, + workdir: '/tmp/spoon-claude-adapter-test-missing', + homeDir: '/tmp/spoon-claude-adapter-test-missing', + username: 'tester', + containerHome: '/home/tester', + containerRepo: '/home/tester/Code/spoon', + repoDir: '/tmp/spoon-claude-adapter-test-missing/Code/spoon', + boxName: 'spoon-box-tester', + redact: (value: string) => value, + runtime: 'claude', +}); + +const makeOAuthWorkspace = ( + homeDir: string, + authType: 'anthropic_oauth_json' | 'opencode_auth_json', +): AdapterWorkspace => ({ + claim: { + ...claim, + aiProviderProfile: { + id: 'p', + name: 'Anthropic', + provider: 'anthropic', + authType, + secret: JSON.stringify({ access_token: 'oauth-token' }), + model: 'claude-sonnet-4', + reasoningEffort: 'medium', + }, + } as unknown as Claim, + workdir: homeDir, + homeDir, + username: 'tester', + containerHome: '/home/tester', + containerRepo: '/home/tester/Code/spoon', + repoDir: path.join(homeDir, 'Code/spoon'), + boxName: 'spoon-box-tester', + redact: (value: string) => value, + runtime: 'claude', +}); + +const tempDirs: string[] = []; + +afterEach(async () => { + while (tempDirs.length) { + const dir = tempDirs.pop(); + if (dir) await rm(dir, { recursive: true, force: true }); + } +}); + +describe('ClaudeCodeAdapter prepareAuth', () => { + test('writes credentials.json for an anthropic_oauth_json profile', async () => { + const { createClaudeAdapter } = await loadAdapter(); + const homeDir = await mkdtemp(path.join(tmpdir(), 'spoon-claude-oauth-')); + tempDirs.push(homeDir); + + const adapter = createClaudeAdapter(); + await adapter.prepareAuth(makeOAuthWorkspace(homeDir, 'anthropic_oauth_json')); + + const credentialsPath = path.join(homeDir, '.claude', '.credentials.json'); + const contents = await readFile(credentialsPath, 'utf8'); + expect(contents).toContain('oauth-token'); + }); + + test('does not write credentials.json for an opencode_auth_json profile', async () => { + const { createClaudeAdapter } = await loadAdapter(); + const homeDir = await mkdtemp(path.join(tmpdir(), 'spoon-claude-oauth-')); + tempDirs.push(homeDir); + + const adapter = createClaudeAdapter(); + await adapter.prepareAuth(makeOAuthWorkspace(homeDir, 'opencode_auth_json')); + + const credentialsPath = path.join(homeDir, '.claude', '.credentials.json'); + await expect(stat(credentialsPath)).rejects.toThrow(); + }); +}); + +describe('ClaudeCodeAdapter', () => { + test('streams normalized events and marks the turn via buildMarkedCommand', async () => { + const { createClaudeAdapter } = await loadAdapter(); + const captured: unknown[] = []; + const execStream = vi.fn(async (args) => { + captured.push(args.command); + await args.onStdoutLine?.( + JSON.stringify({ + type: 'system', + subtype: 'init', + session_id: 'abc', + }), + ); + await args.onStdoutLine?.( + JSON.stringify({ + type: 'assistant', + message: { content: [{ type: 'text', text: 'hi there' }] }, + }), + ); + await args.onStdoutLine?.( + JSON.stringify({ + type: 'result', + subtype: 'success', + result: 'final text', + session_id: 'abc', + }), + ); + return { exitCode: 0, output: '' }; + }) as unknown as ExecStreamFn; + + const adapter = createClaudeAdapter({ execStream }); + const events: NormalizedAgentEvent[] = []; + const onEvent = (event: NormalizedAgentEvent): Promise => { + events.push(event); + return Promise.resolve(); + }; + + const result = await adapter.runTurn(makeWorkspace(), 'do it', onEvent); + + expect(events).toContainEqual({ kind: 'session', sessionId: 'abc' }); + expect(events).toContainEqual({ + kind: 'assistant_delta', + content: 'hi there', + }); + expect(result.finalMessage).toBe('final text'); + expect(result.sessionId).toBe('abc'); + expect(result.error).toBeUndefined(); + + // The turn ran through buildMarkedCommand (setsid bash session) carrying the + // `claude -p` argv in the script. + const command = captured[0] as string[]; + expect(command[0]).toBe('setsid'); + expect(command[1]).toBe('bash'); + expect(command[3]).toContain("'claude' '-p'"); + expect(command[3]).toContain("'--output-format' 'stream-json'"); + }); +}); diff --git a/apps/agent-worker/tests/unit/cleanup-layout.test.ts b/apps/agent-worker/tests/unit/cleanup-layout.test.ts new file mode 100644 index 0000000..d30053c --- /dev/null +++ b/apps/agent-worker/tests/unit/cleanup-layout.test.ts @@ -0,0 +1,73 @@ +import { afterEach, describe, expect, test, vi } from 'vitest'; + +const load = async () => { + vi.resetModules(); + process.env.SPOON_WORKER_TOKEN = 'test-worker-token'; + process.env.GITHUB_APP_ID = '123'; + process.env.GITHUB_APP_PRIVATE_KEY = + '-----BEGIN PRIVATE KEY-----\ntest\n-----END PRIVATE KEY-----'; + return await import('../../src/worker'); +}; + +describe('planWorkdirCleanup', () => { + afterEach(() => vi.resetModules()); + + test('never removes the homes/ root or any home directory', async () => { + const { planWorkdirCleanup } = await load(); + const plan = planWorkdirCleanup({ + root: '/work', + rootEntries: [{ name: 'homes', isDirectory: true }], + homesEntries: { alice: [{ name: 'Code', isDirectory: true }] }, + codeLeaves: {}, + activeWorkdirs: new Set(), + runningBoxUsernames: new Set(), + }); + expect(plan.removeDirs).not.toContain('/work/homes'); + expect(plan.removeDirs).not.toContain('/work/homes/alice'); + }); + + test('removes legacy top-level job dirs not in the active set', async () => { + const { planWorkdirCleanup } = await load(); + const plan = planWorkdirCleanup({ + root: '/work', + rootEntries: [ + { name: 'homes', isDirectory: true }, + { name: 'legacy-job-123', isDirectory: true }, + { name: 'dev', isDirectory: true }, + ], + homesEntries: {}, + codeLeaves: {}, + activeWorkdirs: new Set(['/work/dev']), + runningBoxUsernames: new Set(), + }); + expect(plan.removeDirs).toContain('/work/legacy-job-123'); + expect(plan.removeDirs).not.toContain('/work/dev'); + expect(plan.removeDirs).not.toContain('/work/homes'); + }); + + test('removes per-thread checkouts only for users with no running box', async () => { + const { planWorkdirCleanup } = await load(); + const plan = planWorkdirCleanup({ + root: '/work', + rootEntries: [{ name: 'homes', isDirectory: true }], + homesEntries: { + alice: [{ name: 'Code', isDirectory: true }], + bob: [{ name: 'Code', isDirectory: true }], + }, + codeLeaves: { + alice: ['/work/homes/alice/Code/spoon-a/branch-x'], + bob: ['/work/homes/bob/Code/spoon-b/branch-y'], + }, + activeWorkdirs: new Set(), + runningBoxUsernames: new Set(['bob']), + }); + expect(plan.removeDirs).toContain( + '/work/homes/alice/Code/spoon-a/branch-x', + ); + expect(plan.removeDirs).not.toContain( + '/work/homes/bob/Code/spoon-b/branch-y', + ); + expect(plan.removeDirs).not.toContain('/work/homes/alice'); + expect(plan.removeDirs).not.toContain('/work/homes/alice/Code'); + }); +}); diff --git a/apps/agent-worker/tests/unit/codex-adapter.test.ts b/apps/agent-worker/tests/unit/codex-adapter.test.ts new file mode 100644 index 0000000..a5b7761 --- /dev/null +++ b/apps/agent-worker/tests/unit/codex-adapter.test.ts @@ -0,0 +1,207 @@ +import { mkdir, mkdtemp, readFile, rm, stat } from 'node:fs/promises'; +import os from 'node:os'; +import path from 'node:path'; + +import { afterEach, describe, expect, test, vi } from 'vitest'; + +import type { NormalizedAgentEvent } from '../../src/agent-events'; +import type { ExecStreamFn } from '../../src/runtime/agent-runtime'; +import type { AdapterWorkspace, Claim } from '../../src/runtime/provider'; + +const tempDirs: string[] = []; + +const pathExists = async (filePath: string): Promise => { + try { + await stat(filePath); + return true; + } catch { + return false; + } +}; + +const loadAdapter = async () => { + process.env.SPOON_WORKER_TOKEN = 'test-worker-token'; + process.env.GITHUB_APP_ID = '123'; + process.env.GITHUB_APP_PRIVATE_KEY = + '-----BEGIN PRIVATE KEY-----\\ntest\\n-----END PRIVATE KEY-----'; + return await import('../../src/runtime/codex-adapter'); +}; + +const claim = { + job: { + _id: 'job-1', + prompt: 'hi', + baseBranch: 'main', + workBranch: 'spoon/x', + forkOwner: 'o', + forkRepo: 'r', + upstreamOwner: 'u', + upstreamRepo: 'r', + }, + spoon: { name: 'Spoon' }, + openai: { model: 'gpt-5', reasoningEffort: 'medium' }, + aiProviderProfile: { + id: 'p', + name: 'Codex', + provider: 'opencode_openai_login', + authType: 'opencode_auth_json', + model: 'gpt-5', + reasoningEffort: 'medium', + }, + github: {}, + secrets: [], +} as unknown as Claim; + +const makeWorkspace = (): AdapterWorkspace => ({ + claim, + // A non-existent workdir so the --output-last-message read is a clean ENOENT. + workdir: '/tmp/spoon-codex-adapter-test-missing', + homeDir: '/tmp/spoon-codex-adapter-test-missing', + username: 'tester', + containerHome: '/home/tester', + containerRepo: '/home/tester/Code/spoon', + repoDir: '/tmp/spoon-codex-adapter-test-missing/Code/spoon', + boxName: 'spoon-box-tester', + redact: (value: string) => value, + runtime: 'codex', +}); + +const makeAuthWorkspace = async ( + profile: Record, +): Promise => { + const workdir = await mkdtemp(path.join(os.tmpdir(), 'spoon-codex-auth-')); + tempDirs.push(workdir); + const repoDir = path.join(workdir, 'Code', 'spoon'); + await mkdir(repoDir, { recursive: true }); + return { + ...makeWorkspace(), + claim: { ...claim, aiProviderProfile: profile } as unknown as Claim, + workdir, + homeDir: workdir, + repoDir, + }; +}; + +describe('CodexAdapter', () => { + afterEach(async () => { + await Promise.all( + tempDirs.map((dir) => rm(dir, { force: true, recursive: true })), + ); + tempDirs.length = 0; + }); + + test('prepareAuth skips auth.json for an api_key profile', async () => { + const { createCodexAdapter } = await loadAdapter(); + const adapter = createCodexAdapter(); + const workspace = await makeAuthWorkspace({ + id: 'p', + name: 'OpenAI', + provider: 'openai', + authType: 'api_key', + model: 'gpt-5', + // A raw API key, which is deliberately NOT valid JSON. + secret: 'sk-test', + }); + + await expect(adapter.prepareAuth?.(workspace)).resolves.toBeUndefined(); + + const codexAuthPath = path.join(workspace.workdir, '.codex', 'auth.json'); + const openCodeAuthPath = path.join( + workspace.workdir, + '.local', + 'share', + 'opencode', + 'auth.json', + ); + await expect(pathExists(codexAuthPath)).resolves.toBe(false); + await expect(pathExists(openCodeAuthPath)).resolves.toBe(false); + }); + + test('prepareAuth writes auth.json for a ChatGPT-login profile', async () => { + const { createCodexAdapter } = await loadAdapter(); + const adapter = createCodexAdapter(); + const secret = JSON.stringify({ tokens: { access: 'abc' } }); + const workspace = await makeAuthWorkspace({ + id: 'p', + name: 'Codex', + provider: 'opencode_openai_login', + authType: 'opencode_auth_json', + model: 'gpt-5', + secret, + }); + + await adapter.prepareAuth?.(workspace); + + const codexAuthPath = path.join(workspace.workdir, '.codex', 'auth.json'); + await expect(readFile(codexAuthPath, 'utf8')).resolves.toBe( + `${JSON.stringify(JSON.parse(secret), null, 2)}\n`, + ); + const openCodeAuthPath = path.join( + workspace.workdir, + '.local', + 'share', + 'opencode', + 'auth.json', + ); + await expect(readFile(openCodeAuthPath, 'utf8')).resolves.toBe( + `${JSON.stringify(JSON.parse(secret), null, 2)}\n`, + ); + }); + + test('streams normalized events and marks the turn via buildMarkedCommand', async () => { + const { createCodexAdapter } = await loadAdapter(); + const captured: unknown[] = []; + const execStream = vi.fn(async (args) => { + captured.push(args.command); + await args.onStdoutLine?.( + JSON.stringify({ + type: 'item.completed', + item: { id: 'item-1', type: 'agent_message', text: 'done' }, + }), + ); + await args.onStdoutLine?.(JSON.stringify({ type: 'turn.completed' })); + return { exitCode: 0, output: '' }; + }) as unknown as ExecStreamFn; + + const adapter = createCodexAdapter({ execStream }); + const events: NormalizedAgentEvent[] = []; + const onEvent = (event: NormalizedAgentEvent): Promise => { + events.push(event); + return Promise.resolve(); + }; + + const result = await adapter.runTurn(makeWorkspace(), 'do it', onEvent); + + expect(events).toContainEqual({ + kind: 'assistant_delta', + content: 'done\n\n', + externalMessageId: 'item-1', + }); + expect(events).toContainEqual({ kind: 'assistant_completed' }); + expect(result).toBeTypeOf('object'); + // The turn ran through buildMarkedCommand (setsid bash session). + const command = captured[0] as string[]; + expect(command[0]).toBe('setsid'); + expect(command[1]).toBe('bash'); + }); + + test('surfaces a turn.failed error while exiting zero', async () => { + const { createCodexAdapter } = await loadAdapter(); + const execStream = vi.fn(async (args) => { + await args.onStdoutLine?.( + JSON.stringify({ + type: 'turn.failed', + error: { message: 'boom' }, + }), + ); + return { exitCode: 0, output: '' }; + }) as unknown as ExecStreamFn; + + const adapter = createCodexAdapter({ execStream }); + const result = await adapter.runTurn(makeWorkspace(), 'do it', () => + Promise.resolve(), + ); + + expect(result.error).toContain('boom'); + }); +}); diff --git a/apps/agent-worker/tests/unit/codex-process.test.ts b/apps/agent-worker/tests/unit/codex-process.test.ts new file mode 100644 index 0000000..27cb15f --- /dev/null +++ b/apps/agent-worker/tests/unit/codex-process.test.ts @@ -0,0 +1,37 @@ +import { describe, expect, test, vi } from 'vitest'; + +import { + abortManagedCodexProcess, + managedCodexCommand, +} from '../../src/codex-process'; + +describe('managed Codex process', () => { + test('launches Codex in a job-specific process session', () => { + const command = managedCodexCommand({ + jobId: 'job-1', + pidFile: '/home/alice/.codex/job-1.pid', + command: ['codex', 'exec', '--json', 'fix it'], + }); + + expect(command.slice(0, 2)).toEqual(['python3', '-c']); + expect(command).toContain('/home/alice/.codex/job-1.pid'); + expect(command).toContain('job-1'); + expect(command.slice(-4)).toEqual(['codex', 'exec', '--json', 'fix it']); + }); + + test('invokes an isolated in-box kill for the matching job', async () => { + const execute = vi.fn().mockResolvedValue({ exitCode: 0, output: '' }); + + await abortManagedCodexProcess({ + jobId: 'job-1', + pidFile: '/home/alice/.codex/job-1.pid', + execute, + }); + + expect(execute).toHaveBeenCalledOnce(); + const [command] = execute.mock.calls[0] as [string[]]; + expect(command.slice(0, 2)).toEqual(['python3', '-c']); + expect(command).toContain('/home/alice/.codex/job-1.pid'); + expect(command).toContain('job-1'); + }); +}); diff --git a/apps/agent-worker/tests/unit/heartbeat-controller.test.ts b/apps/agent-worker/tests/unit/heartbeat-controller.test.ts new file mode 100644 index 0000000..d933bee --- /dev/null +++ b/apps/agent-worker/tests/unit/heartbeat-controller.test.ts @@ -0,0 +1,60 @@ +import { afterEach, beforeEach, describe, expect, test, vi } from 'vitest'; + +import { createHeartbeatController } from '../../src/heartbeat-controller'; + +describe('heartbeat controller', () => { + beforeEach(() => vi.useFakeTimers()); + afterEach(() => vi.useRealTimers()); + + test('does not overlap ticks while a heartbeat is in flight', async () => { + let resolveHeartbeat!: (value: { cancelRequested: boolean }) => void; + const heartbeat = vi.fn( + () => + new Promise<{ cancelRequested: boolean }>((resolve) => { + resolveHeartbeat = resolve; + }), + ); + const controller = createHeartbeatController({ + intervalMs: 30_000, + heartbeat, + onCancel: vi.fn(), + onError: vi.fn(), + }); + + controller.start('job-1'); + await vi.advanceTimersByTimeAsync(90_000); + expect(heartbeat).toHaveBeenCalledOnce(); + + resolveHeartbeat({ cancelRequested: false }); + await Promise.resolve(); + await vi.advanceTimersByTimeAsync(30_000); + expect(heartbeat).toHaveBeenCalledTimes(2); + controller.stop('job-1'); + }); + + test('removes the timer before awaiting cancellation teardown', async () => { + let finishCancel!: () => void; + const onCancel = vi.fn( + () => + new Promise((resolve) => { + finishCancel = resolve; + }), + ); + const heartbeat = vi.fn().mockResolvedValue({ cancelRequested: true }); + const controller = createHeartbeatController({ + intervalMs: 30_000, + heartbeat, + onCancel, + onError: vi.fn(), + }); + + controller.start('job-1'); + await vi.advanceTimersByTimeAsync(30_000); + expect(onCancel).toHaveBeenCalledOnce(); + expect(controller.has('job-1')).toBe(false); + + await vi.advanceTimersByTimeAsync(90_000); + expect(heartbeat).toHaveBeenCalledOnce(); + finishCancel(); + }); +}); diff --git a/apps/agent-worker/tests/unit/maintenance-workspace-cleanup.test.ts b/apps/agent-worker/tests/unit/maintenance-workspace-cleanup.test.ts new file mode 100644 index 0000000..dcf180c --- /dev/null +++ b/apps/agent-worker/tests/unit/maintenance-workspace-cleanup.test.ts @@ -0,0 +1,160 @@ +import { getFunctionName } from 'convex/server'; +import { afterEach, describe, expect, test, vi } from 'vitest'; + +import type { Id } from '@spoon/backend/convex/_generated/dataModel.js'; + +const mocks = vi.hoisted(() => ({ + getWorktreeDiff: vi.fn(() => + Promise.resolve({ exitCode: 0, output: 'unexpected diff' }), + ), + mutation: vi.fn(), + streamExecInContainer: vi.fn(), +})); + +vi.mock('convex/browser', () => ({ + ConvexHttpClient: vi.fn( + class ConvexHttpClient { + mutation = mocks.mutation; + }, + ), +})); +vi.mock('../../src/env', () => ({ + env: { + buildCreatedAt: 'test', + buildSha: 'test', + convexUrl: 'http://convex.test', + jobTimeoutMs: 1000, + runtime: 'docker', + workerId: 'worker-1', + workerToken: 'token', + }, +})); +vi.mock('../../src/git', () => ({ + cloneRepository: vi.fn(), + commitAndPush: vi.fn(), + getStatus: vi.fn(), + getWorktreeDiff: mocks.getWorktreeDiff, + run: vi.fn(), +})); +vi.mock('../../src/github', () => ({ + getInstallationToken: vi.fn(), + openDraftPullRequest: vi.fn(), +})); +vi.mock('../../src/runtime/docker', () => ({ + buildMarkedCommand: vi.fn(() => ['setsid', 'bash', '-lc', '# marker']), + killBoxProcessesByMarker: vi.fn(() => Promise.resolve()), + listWorkspaceContainerNames: vi.fn(() => Promise.resolve([])), + runExecInContainer: vi.fn(), + streamExecInContainer: mocks.streamExecInContainer, +})); +vi.mock('../../src/user-container', () => ({ + acquireUserBox: vi.fn(), + runningBoxUsernames: vi.fn(() => new Set()), +})); +vi.mock('../../src/user-environment', () => ({ + fetchUserEnvironment: vi.fn(), + materializeUserHome: vi.fn(), +})); + +const decision = { + decision: 'sync', + risk: 'low', + summary: 'Safe to sync.', + ignoredCommitShas: [], + ignoredReason: '', + recommendedAction: 'Sync upstream.', + requiresUserApproval: false, +}; + +const load = async () => await import('../../src/worker'); + +afterEach(async () => { + const worker = await load(); + worker._resetActiveWorkspacesForTests(); + vi.restoreAllMocks(); + vi.clearAllMocks(); +}); + +describe('maintenance workspace cleanup', () => { + test('releases and removes the workspace when status teardown fails', async () => { + const worker = await load(); + const release = vi.fn(); + const jobId = 'maintenance-mark'; + mocks.mutation.mockImplementation((reference) => { + if (getFunctionName(reference) === 'agentJobs:markWorkspaceStopped') { + return Promise.reject(new Error('status teardown failed')); + } + return Promise.resolve('message-1'); + }); + mocks.streamExecInContainer.mockImplementation(async (args) => { + await args.onStdoutLine( + JSON.stringify({ + type: 'item.completed', + item: { + type: 'agent_message', + text: JSON.stringify(decision), + }, + }), + ); + return { exitCode: 0, output: '' }; + }); + worker._setActiveWorkspaceForTests(jobId, { + claim: { + job: { + _id: jobId as Id<'agentJobs'>, + prompt: 'Review maintenance changes', + jobType: 'maintenance_review', + baseBranch: 'main', + workBranch: 'spoon/maintenance', + forkOwner: 'team', + forkRepo: 'spoon', + upstreamOwner: 'upstream', + upstreamRepo: 'spoon', + }, + spoon: { name: 'Spoon' }, + openai: { + model: 'gpt-5', + reasoningEffort: 'medium', + }, + aiProviderProfile: { + id: 'profile-1', + name: 'Codex', + provider: 'opencode_openai_login', + authType: 'opencode_auth_json', + model: 'gpt-5', + reasoningEffort: 'medium', + }, + github: {}, + secrets: [], + }, + workdir: '/tmp/workspace', + homeDir: '/tmp/workspace', + username: 'tester', + containerHome: '/home/tester', + containerRepo: '/home/tester/Code/spoon', + repoDir: '/tmp/workspace/Code/spoon', + boxName: 'spoon-box-tester', + boxHandle: { boxName: 'spoon-box-tester', release }, + githubToken: 'github-token', + redact: (value: string) => value, + runtime: 'codex', + }); + const consoleError = vi + .spyOn(console, 'error') + .mockImplementation(() => undefined); + + await expect( + worker.sendWorkspaceMessage(jobId, 'Review maintenance changes'), + ).resolves.toBeUndefined(); + + expect(release).toHaveBeenCalledTimes(1); + expect(mocks.getWorktreeDiff).not.toHaveBeenCalled(); + expect( + mocks.mutation.mock.calls.some(([, args]) => args.kind === 'diff'), + ).toBe(false); + await expect(worker.stopWorkspace(jobId)).rejects.toThrow( + 'Agent workspace is not active on this worker.', + ); + expect(consoleError).toHaveBeenCalledTimes(1); + }); +}); diff --git a/apps/agent-worker/tests/unit/materialize-env.test.ts b/apps/agent-worker/tests/unit/materialize-env.test.ts new file mode 100644 index 0000000..f4f15bd --- /dev/null +++ b/apps/agent-worker/tests/unit/materialize-env.test.ts @@ -0,0 +1,86 @@ +import { + access, + chmod, + mkdtemp, + readFile, + rm, + stat, + writeFile, +} from 'node:fs/promises'; +import os from 'node:os'; +import path from 'node:path'; +import { afterEach, describe, expect, test } from 'vitest'; + +import { + removeMaterializedEnv, + writeMaterializedEnv, +} from '../../src/materialize-env'; + +const tempDirs: string[] = []; + +const mode = async (filePath: string) => (await stat(filePath)).mode & 0o777; + +const exists = async (filePath: string) => + access(filePath).then( + () => true, + () => false, + ); + +describe('materialize env', () => { + afterEach(async () => { + await Promise.all( + tempDirs.map((dir) => rm(dir, { force: true, recursive: true })), + ); + tempDirs.length = 0; + }); + + test('writes the env file 0600 at the returned path with quoted values', async () => { + const repoDir = await mkdtemp(path.join(os.tmpdir(), 'spoon-env-')); + tempDirs.push(repoDir); + + const envPath = await writeMaterializedEnv(repoDir, '.env.local', [ + { name: 'A', value: 'x' }, + ]); + + await expect(exists(envPath)).resolves.toBe(true); + await expect(mode(envPath)).resolves.toBe(0o600); + await expect(readFile(envPath, 'utf8')).resolves.toBe('A="x"\n'); + }); + + test('tightens perms to 0600 and overwrites content when the env file pre-exists 0644', async () => { + const repoDir = await mkdtemp(path.join(os.tmpdir(), 'spoon-env-')); + tempDirs.push(repoDir); + + const preExisting = path.join(repoDir, '.env.local'); + // Pre-create the target with loose perms. umask may mask the writeFile + // mode, so chmod explicitly and assert the precondition really is 0644. + await writeFile(preExisting, 'OLD', { mode: 0o644 }); + await chmod(preExisting, 0o644); + await expect(mode(preExisting)).resolves.toBe(0o644); + + const envPath = await writeMaterializedEnv(repoDir, '.env.local', [ + { name: 'A', value: 'x' }, + ]); + + // Overwrote the plaintext content AND tightened perms unconditionally. + await expect(mode(envPath)).resolves.toBe(0o600); + await expect(readFile(envPath, 'utf8')).resolves.toBe('A="x"\n'); + }); + + test('removeMaterializedEnv deletes an existing file and no-ops when absent', async () => { + const repoDir = await mkdtemp(path.join(os.tmpdir(), 'spoon-env-')); + tempDirs.push(repoDir); + + const envPath = await writeMaterializedEnv(repoDir, '.env.local', [ + { name: 'A', value: 'x' }, + ]); + await expect(exists(envPath)).resolves.toBe(true); + + await removeMaterializedEnv(envPath); + await expect(exists(envPath)).resolves.toBe(false); + + // No-ops when the file is already gone or the path is undefined. + await expect(removeMaterializedEnv(envPath)).resolves.toBeUndefined(); + await expect(removeMaterializedEnv(undefined)).resolves.toBeUndefined(); + }); +}); diff --git a/apps/agent-worker/tests/unit/opencode-adapter.test.ts b/apps/agent-worker/tests/unit/opencode-adapter.test.ts new file mode 100644 index 0000000..df2afeb --- /dev/null +++ b/apps/agent-worker/tests/unit/opencode-adapter.test.ts @@ -0,0 +1,100 @@ +import { describe, expect, test, vi } from 'vitest'; + +import type { NormalizedAgentEvent } from '../../src/agent-events'; +import type { ExecStreamFn } from '../../src/runtime/agent-runtime'; +import type { AdapterWorkspace, Claim } from '../../src/runtime/provider'; + +const loadAdapter = async () => { + process.env.SPOON_WORKER_TOKEN = 'test-worker-token'; + process.env.GITHUB_APP_ID = '123'; + process.env.GITHUB_APP_PRIVATE_KEY = + '-----BEGIN PRIVATE KEY-----\\ntest\\n-----END PRIVATE KEY-----'; + return await import('../../src/runtime/opencode-adapter'); +}; + +const claim = { + job: { + _id: 'job-1', + prompt: 'hi', + baseBranch: 'main', + workBranch: 'spoon/x', + forkOwner: 'o', + forkRepo: 'r', + upstreamOwner: 'u', + upstreamRepo: 'r', + }, + spoon: { name: 'Spoon' }, + openai: { model: 'gpt-5', reasoningEffort: 'medium' }, + aiProviderProfile: { + id: 'p', + name: 'OpenAI', + provider: 'openai', + authType: 'api_key', + secret: 'sk-test', + model: 'gpt-5', + reasoningEffort: 'medium', + }, + github: {}, + secrets: [], +} as unknown as Claim; + +const makeWorkspace = (): AdapterWorkspace => ({ + claim, + workdir: '/tmp/spoon-opencode-adapter-test-missing', + homeDir: '/tmp/spoon-opencode-adapter-test-missing', + username: 'tester', + containerHome: '/home/tester', + containerRepo: '/home/tester/Code/spoon', + repoDir: '/tmp/spoon-opencode-adapter-test-missing/Code/spoon', + boxName: 'spoon-box-tester', + redact: (value: string) => value, + runtime: 'opencode', +}); + +describe('OpenCodeAdapter', () => { + test('streams normalized events and marks the turn via buildMarkedCommand', async () => { + const { createOpenCodeAdapter } = await loadAdapter(); + const captured: unknown[] = []; + const execStream = vi.fn(async (args) => { + captured.push(args.command); + await args.onStdoutLine?.( + JSON.stringify({ + type: 'message.part.delta', + properties: { part: { text: 'hi' }, messageID: 'm1' }, + }), + ); + await args.onStdoutLine?.( + JSON.stringify({ + sessionID: 'ses_abc', + parts: [{ text: 'final answer' }], + }), + ); + return { exitCode: 0, output: '' }; + }) as unknown as ExecStreamFn; + + const adapter = createOpenCodeAdapter({ execStream }); + const events: NormalizedAgentEvent[] = []; + const onEvent = (event: NormalizedAgentEvent): Promise => { + events.push(event); + return Promise.resolve(); + }; + + const result = await adapter.runTurn(makeWorkspace(), 'do it', onEvent); + + expect(events).toContainEqual({ + kind: 'assistant_delta', + content: 'hi', + externalMessageId: 'm1', + }); + expect(result).toBeTypeOf('object'); + expect(result.sessionId).toBe('ses_abc'); + expect(result.finalMessage).toBe('final answer'); + + // The turn ran through buildMarkedCommand (setsid bash session) carrying the + // opencode run argv in the script. + const command = captured[0] as string[]; + expect(command[0]).toBe('setsid'); + expect(command[1]).toBe('bash'); + expect(command[3]).toContain("'opencode' 'run' '--format' 'json'"); + }); +}); diff --git a/apps/agent-worker/tests/unit/path-containment.test.ts b/apps/agent-worker/tests/unit/path-containment.test.ts new file mode 100644 index 0000000..02112a3 --- /dev/null +++ b/apps/agent-worker/tests/unit/path-containment.test.ts @@ -0,0 +1,54 @@ +import { mkdir, mkdtemp, rm, symlink, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import path from 'node:path'; +import { afterEach, describe, expect, test } from 'vitest'; + +import { assertContainedRealPath } from '../../src/path-containment'; + +let dir: string; + +afterEach(async () => { + if (dir) await rm(dir, { recursive: true, force: true }); +}); + +describe('assertContainedRealPath', () => { + test('allows a normal file inside the root', async () => { + dir = await mkdtemp(path.join(tmpdir(), 'spoon-')); + const root = path.join(dir, 'repo'); + await mkdir(root, { recursive: true }); + await writeFile(path.join(root, 'a.txt'), 'hi'); + await expect(assertContainedRealPath(root, 'a.txt')).resolves.toBe( + path.join(root, 'a.txt'), + ); + }); + + test('rejects a lexical .. escape', async () => { + dir = await mkdtemp(path.join(tmpdir(), 'spoon-')); + const root = path.join(dir, 'repo'); + await mkdir(root, { recursive: true }); + await expect(assertContainedRealPath(root, '../secret')).rejects.toThrow(); + }); + + test('rejects reading through a symlink that escapes the root', async () => { + dir = await mkdtemp(path.join(tmpdir(), 'spoon-')); + const root = path.join(dir, 'repo'); + const outside = path.join(dir, 'outside'); + await mkdir(root, { recursive: true }); + await mkdir(outside, { recursive: true }); + await writeFile(path.join(outside, 'secret'), 'S'); + await symlink(outside, path.join(root, 'link')); + await expect( + assertContainedRealPath(root, 'link/secret'), + ).rejects.toThrow(); + }); + + test('rejects a write whose final target is a symlink', async () => { + dir = await mkdtemp(path.join(tmpdir(), 'spoon-')); + const root = path.join(dir, 'repo'); + await mkdir(root, { recursive: true }); + await symlink(path.join(dir, 'evil'), path.join(root, 'out')); + await expect( + assertContainedRealPath(root, 'out', { forWrite: true }), + ).rejects.toThrow(); + }); +}); diff --git a/apps/agent-worker/tests/unit/redact.test.ts b/apps/agent-worker/tests/unit/redact.test.ts new file mode 100644 index 0000000..c282aab --- /dev/null +++ b/apps/agent-worker/tests/unit/redact.test.ts @@ -0,0 +1,38 @@ +import { describe, expect, test } from 'vitest'; + +import { createRedactor } from '../../src/redact'; + +describe('createRedactor', () => { + test('redacts group-less token patterns without emitting a literal $1', () => { + const redact = createRedactor([]); + const output = redact('token ghs_ABC123 and sk-xyz789 here'); + + expect(output).toContain('[redacted]'); + expect(output).not.toContain('$1'); + expect(output).not.toContain('ghs_ABC123'); + expect(output).not.toContain('sk-xyz789'); + }); + + test('redacts github_pat_ tokens to [redacted]', () => { + const redact = createRedactor([]); + const output = redact('github_pat_ABCDEF123456'); + + expect(output).toBe('[redacted]'); + expect(output).not.toContain('$1'); + }); + + test('keeps the key= prefix for key=value secrets', () => { + const redact = createRedactor([]); + + expect(redact('api_key=secret')).toBe('api_key=[redacted]'); + expect(redact('password=hunter2')).toBe('password=[redacted]'); + }); + + test('redacts caller-supplied secret values verbatim', () => { + const redact = createRedactor(['super-secret-value']); + + expect(redact('the value is super-secret-value')).toBe( + 'the value is [redacted]', + ); + }); +}); diff --git a/apps/agent-worker/tests/unit/terminal-resize.test.ts b/apps/agent-worker/tests/unit/terminal-resize.test.ts new file mode 100644 index 0000000..4a7d1fe --- /dev/null +++ b/apps/agent-worker/tests/unit/terminal-resize.test.ts @@ -0,0 +1,46 @@ +import { afterEach, describe, expect, test, vi } from 'vitest'; + +const load = async () => { + vi.resetModules(); + process.env.SPOON_WORKER_TOKEN = 'test-worker-token'; + process.env.GITHUB_APP_ID = '123'; + process.env.GITHUB_APP_PRIVATE_KEY = + '-----BEGIN PRIVATE KEY-----\ntest\n-----END PRIVATE KEY-----'; + vi.doMock('../../src/worker', () => ({ getTerminalWorkspace: () => null })); + vi.doMock('../../src/user-container', () => ({ + acquireUserBox: vi.fn(), + })); + return await import('../../src/terminal'); +}; + +describe('parseResizeMessage', () => { + afterEach(() => vi.resetModules()); + + test('parses and clamps a resize control frame', async () => { + const { parseResizeMessage } = await load(); + const msg = Buffer.from( + JSON.stringify({ type: 'resize', cols: 120, rows: 40 }), + ); + expect(parseResizeMessage(msg, false)).toEqual({ cols: 120, rows: 40 }); + }); + + test('returns null for binary input (raw keystrokes)', async () => { + const { parseResizeMessage } = await load(); + expect(parseResizeMessage(Buffer.from([1, 2, 3]), true)).toBeNull(); + }); + + test('returns null for non-resize JSON', async () => { + const { parseResizeMessage } = await load(); + expect( + parseResizeMessage(Buffer.from('{"type":"other"}'), false), + ).toBeNull(); + }); + + test('clamps out-of-range dimensions into [1,1000]', async () => { + const { parseResizeMessage } = await load(); + const msg = Buffer.from( + JSON.stringify({ type: 'resize', cols: 99999, rows: 0 }), + ); + expect(parseResizeMessage(msg, false)).toEqual({ cols: 1000, rows: 1 }); + }); +}); diff --git a/apps/agent-worker/tests/unit/turn-outcome.test.ts b/apps/agent-worker/tests/unit/turn-outcome.test.ts new file mode 100644 index 0000000..72892bb --- /dev/null +++ b/apps/agent-worker/tests/unit/turn-outcome.test.ts @@ -0,0 +1,94 @@ +import { describe, expect, test } from 'vitest'; + +import { + resolveTurnOutcome, + shouldAppendCompletedContent, +} from '../../src/runtime/turn-outcome'; + +describe('shouldAppendCompletedContent', () => { + test('does not re-append when deltas already streamed the answer', () => { + // Claude/OpenCode emit the final answer as assistant_delta AND as the + // content-bearing assistant_completed event. Folding the completed + // content in again would produce "". + expect(shouldAppendCompletedContent('the answer', 'the answer')).toBe( + false, + ); + }); + + test('appends completed content when nothing has streamed yet', () => { + expect(shouldAppendCompletedContent('', 'the final result')).toBe(true); + }); + + test('treats whitespace-only streamed text as empty', () => { + expect(shouldAppendCompletedContent(' \n', 'the final result')).toBe( + true, + ); + }); + + test('does not append when the completed event carries no content', () => { + expect(shouldAppendCompletedContent('the answer', undefined)).toBe(false); + expect(shouldAppendCompletedContent('', undefined)).toBe(false); + expect(shouldAppendCompletedContent('', '')).toBe(false); + }); +}); + +describe('resolveTurnOutcome', () => { + test('partial streamed text + error is surfaced as failed and text preserved', () => { + const outcome = resolveTurnOutcome({ + assistantText: 'partial answer before crash', + error: 'exit code 1: rate limited', + runtime: 'codex', + }); + expect(outcome.text).toBe('partial answer before crash'); + expect(outcome.failure).toBe('codex failed:\nexit code 1: rate limited'); + }); + + test('successful turn with streamed text and no error has no failure', () => { + const outcome = resolveTurnOutcome({ + assistantText: 'complete answer', + runtime: 'codex', + }); + expect(outcome.text).toBe('complete answer'); + expect(outcome.failure).toBeUndefined(); + }); + + test('empty turn with error is surfaced as failed', () => { + const outcome = resolveTurnOutcome({ + assistantText: ' ', + error: 'exit code 137', + runtime: 'claude', + }); + expect(outcome.failure).toBe('claude failed:\nexit code 137'); + }); + + test('empty turn with no error reports the no-response failure', () => { + const outcome = resolveTurnOutcome({ + assistantText: '', + runtime: 'codex', + }); + expect(outcome.failure).toBe( + 'Codex completed without producing an assistant response.', + ); + }); + + test('recovers finalMessage text when nothing streamed and no error', () => { + const outcome = resolveTurnOutcome({ + assistantText: '', + recoveredText: 'recovered from last-message file', + runtime: 'codex', + }); + expect(outcome.text).toBe('recovered from last-message file'); + expect(outcome.failure).toBeUndefined(); + }); + + test('recovered finalMessage text + error is still surfaced as failed', () => { + const outcome = resolveTurnOutcome({ + assistantText: '', + recoveredText: 'recovered partial', + error: 'network drop', + runtime: 'codex', + }); + expect(outcome.text).toBe('recovered partial'); + expect(outcome.failure).toBe('codex failed:\nnetwork drop'); + }); +}); diff --git a/apps/agent-worker/tests/unit/user-container.test.ts b/apps/agent-worker/tests/unit/user-container.test.ts new file mode 100644 index 0000000..7f6a7a3 --- /dev/null +++ b/apps/agent-worker/tests/unit/user-container.test.ts @@ -0,0 +1,178 @@ +import { afterEach, beforeEach, describe, expect, test, vi } from 'vitest'; + +vi.mock('../../src/env', () => ({ env: { boxIdleMs: 1000 } })); +vi.mock('../../src/runtime/docker', () => ({ + ensureUserContainer: vi.fn((args: { username: string }) => + Promise.resolve(`spoon-box-${args.username}`), + ), + stopWorkspaceContainer: vi.fn(() => Promise.resolve()), + userContainerName: (username: string) => `spoon-box-${username}`, + listWorkspaceContainerNames: vi.fn(() => Promise.resolve([])), +})); + +const load = async () => await import('../../src/user-container'); + +describe('box registry', () => { + beforeEach(() => vi.useFakeTimers()); + afterEach(async () => { + const module = await load(); + module._resetBoxRegistryForTests(); + vi.useRealTimers(); + vi.resetModules(); + vi.clearAllMocks(); + }); + + test('serializes concurrent acquire into a single docker run', async () => { + const module = await load(); + const docker = await import('../../src/runtime/docker'); + const [a, b] = await Promise.all([ + module.acquireUserBox({ + username: 'alice', + workdir: '/w', + containerHome: '/home/alice', + }), + module.acquireUserBox({ + username: 'alice', + workdir: '/w', + containerHome: '/home/alice', + }), + ]); + expect(docker.ensureUserContainer).toHaveBeenCalledTimes(1); + expect(a.boxName).toBe('spoon-box-alice'); + a.release(); + await vi.advanceTimersByTimeAsync(2000); + expect(docker.stopWorkspaceContainer).not.toHaveBeenCalled(); + b.release(); + await vi.advanceTimersByTimeAsync(2000); + expect(docker.stopWorkspaceContainer).toHaveBeenCalledTimes(1); + }); + + test('double-release of one handle does not reap a box held elsewhere', async () => { + const module = await load(); + const docker = await import('../../src/runtime/docker'); + const a = await module.acquireUserBox({ + username: 'bob', + workdir: '/w', + containerHome: '/home/bob', + }); + const b = await module.acquireUserBox({ + username: 'bob', + workdir: '/w', + containerHome: '/home/bob', + }); + a.release(); + a.release(); + await vi.advanceTimersByTimeAsync(2000); + expect(docker.stopWorkspaceContainer).not.toHaveBeenCalled(); + b.release(); + await vi.advanceTimersByTimeAsync(2000); + expect(docker.stopWorkspaceContainer).toHaveBeenCalledTimes(1); + }); + + test('reconcileExistingBoxes adopts orphans with zero refs and reaps them', async () => { + const module = await load(); + const docker = await import('../../src/runtime/docker'); + ( + docker.listWorkspaceContainerNames as unknown as ReturnType + ).mockResolvedValueOnce(['spoon-box-carol']); + await module.reconcileExistingBoxes(); + expect(module.runningBoxUsernames().has('carol')).toBe(true); + await vi.advanceTimersByTimeAsync(2000); + expect(docker.stopWorkspaceContainer).toHaveBeenCalledWith( + 'spoon-box-carol', + ); + }); + + test('ensures an adopted box once before returning acquire handles', async () => { + const module = await load(); + const docker = await import('../../src/runtime/docker'); + vi.mocked(docker.listWorkspaceContainerNames).mockResolvedValueOnce([ + 'spoon-box-casey', + ]); + await module.reconcileExistingBoxes(); + + const first = await module.acquireUserBox({ + username: 'casey', + workdir: '/w', + containerHome: '/home/casey', + }); + const second = await module.acquireUserBox({ + username: 'casey', + workdir: '/w', + containerHome: '/home/casey', + }); + + expect(docker.ensureUserContainer).toHaveBeenCalledTimes(1); + expect(first.boxName).toBe('spoon-box-casey'); + expect(second.boxName).toBe('spoon-box-casey'); + }); + + test('retries ensure after the first acquire fails', async () => { + const module = await load(); + const docker = await import('../../src/runtime/docker'); + const ensure = vi.mocked(docker.ensureUserContainer); + ensure + .mockRejectedValueOnce(new Error('docker failed')) + .mockResolvedValueOnce('spoon-box-dana'); + + await expect( + module.acquireUserBox({ + username: 'dana', + workdir: '/w', + containerHome: '/home/dana', + }), + ).rejects.toThrow('docker failed'); + + const handle = await module.acquireUserBox({ + username: 'dana', + workdir: '/w', + containerHome: '/home/dana', + }); + expect(ensure).toHaveBeenCalledTimes(2); + expect(handle.boxName).toBe('spoon-box-dana'); + }); + + test('finishes an old idle reap before a new acquire can ensure the box', async () => { + const module = await load(); + const docker = await import('../../src/runtime/docker'); + const ensure = vi.mocked(docker.ensureUserContainer); + const stop = vi.mocked(docker.stopWorkspaceContainer); + const events: string[] = []; + let finishStop!: () => void; + const stopDone = new Promise((resolve) => { + finishStop = resolve; + }); + ensure.mockReset().mockImplementation((args: { username: string }) => { + events.push('ensure'); + return Promise.resolve(`spoon-box-${args.username}`); + }); + stop.mockReset().mockImplementationOnce(() => { + events.push('stop:start'); + return stopDone.then(() => { + events.push('stop:end'); + }); + }); + + const first = await module.acquireUserBox({ + username: 'erin', + workdir: '/w', + containerHome: '/home/erin', + }); + first.release(); + await vi.advanceTimersByTimeAsync(1000); + + const secondPromise = module.acquireUserBox({ + username: 'erin', + workdir: '/w', + containerHome: '/home/erin', + }); + await Promise.resolve(); + await Promise.resolve(); + expect(events).toEqual(['ensure', 'stop:start']); + + finishStop(); + const second = await secondPromise; + expect(second.boxName).toBe('spoon-box-erin'); + expect(events).toEqual(['ensure', 'stop:start', 'stop:end', 'ensure']); + }); +}); diff --git a/apps/agent-worker/tests/unit/workspace-teardown.test.ts b/apps/agent-worker/tests/unit/workspace-teardown.test.ts new file mode 100644 index 0000000..3bc1fc2 --- /dev/null +++ b/apps/agent-worker/tests/unit/workspace-teardown.test.ts @@ -0,0 +1,62 @@ +import { describe, expect, test, vi } from 'vitest'; + +import { abortManagedCodexProcess } from '../../src/codex-process'; +import { teardownWorkspace } from '../../src/workspace-teardown'; + +describe('workspace teardown', () => { + test('event failure does not skip abort, stop, removal, or release', async () => { + const calls: string[] = []; + const record = (name: string) => vi.fn(() => calls.push(name)); + + const result = await teardownWorkspace({ + stopHeartbeat: record('heartbeat'), + removeActive: record('remove'), + appendEvent: vi.fn().mockRejectedValue(new Error('event failed')), + abortAgent: record('abort'), + markStopped: vi.fn().mockRejectedValue(new Error('mutation failed')), + closeAgent: record('close'), + stopContainer: vi.fn().mockRejectedValue(new Error('container failed')), + release: record('release'), + }); + + expect(calls).toEqual(['heartbeat', 'remove', 'abort', 'close', 'release']); + expect(result.errors).toHaveLength(3); + }); + + test('removes active state before asynchronous cancellation work', async () => { + const active = new Set(['job-1']); + let activeDuringAbort = true; + + await teardownWorkspace({ + stopHeartbeat: vi.fn(), + removeActive: () => active.delete('job-1'), + abortAgent: () => { + activeDuringAbort = active.has('job-1'); + }, + release: vi.fn(), + }); + + expect(activeDuringAbort).toBe(false); + expect(active.has('job-1')).toBe(false); + }); + + test('Codex cancellation invokes isolated kill and leaves no active workspace', async () => { + const active = new Set(['job-1']); + const execute = vi.fn().mockResolvedValue({ exitCode: 0, output: '' }); + + await teardownWorkspace({ + stopHeartbeat: vi.fn(), + removeActive: () => active.delete('job-1'), + abortAgent: async () => + await abortManagedCodexProcess({ + jobId: 'job-1', + pidFile: '/home/alice/.codex/job-1.pid', + execute, + }), + release: vi.fn(), + }); + + expect(execute).toHaveBeenCalledOnce(); + expect(active.has('job-1')).toBe(false); + }); +}); diff --git a/apps/next/src/app/(app)/agents/page.tsx b/apps/next/src/app/(app)/agents/page.tsx deleted file mode 100644 index cff9752..0000000 --- a/apps/next/src/app/(app)/agents/page.tsx +++ /dev/null @@ -1,7 +0,0 @@ -import { redirect } from 'next/navigation'; - -const AgentsRedirectPage = () => { - redirect('/threads?source=user_request'); -}; - -export default AgentsRedirectPage; diff --git a/apps/next/src/app/(app)/dashboard/page.tsx b/apps/next/src/app/(app)/dashboard/page.tsx index 1edaf22..73baa58 100644 --- a/apps/next/src/app/(app)/dashboard/page.tsx +++ b/apps/next/src/app/(app)/dashboard/page.tsx @@ -3,27 +3,38 @@ import Link from 'next/link'; import { MetricCard } from '@/components/dashboard/metric-card'; import { SpoonCard } from '@/components/spoons/spoon-card'; +import { ListSkeleton } from '@/components/ui/list-skeleton'; import { MaintenanceQueue } from '@/components/threads/maintenance-queue'; import { useQuery } from 'convex/react'; import { GitBranch, MessageSquare, RefreshCw, ShieldCheck } from 'lucide-react'; import { api } from '@spoon/backend/convex/_generated/api.js'; -import { Button, Card, CardContent, CardHeader, CardTitle } from '@spoon/ui'; +import { + Button, + Card, + CardContent, + CardHeader, + CardTitle, + Skeleton, +} from '@spoon/ui'; const DashboardPage = () => { - const spoons = useQuery(api.spoons.listMineWithState, {}) ?? []; - const syncRuns = useQuery(api.syncRuns.listRecent, { limit: 5 }) ?? []; - const threads = useQuery(api.threads.listMine, { limit: 25 }) ?? []; - const activeSpoons = spoons.filter( + const spoons = useQuery(api.spoons.listMineWithState, {}); + const syncRuns = useQuery(api.syncRuns.listRecent, { limit: 5 }); + const threads = useQuery(api.threads.listMine, { limit: 25 }); + const spoonList = spoons ?? []; + const threadList = threads ?? []; + const metricsLoading = spoons === undefined || threads === undefined; + const activeSpoons = spoonList.filter( (spoon) => spoon.status === 'active', ).length; - const behind = spoons.filter( + const behind = spoonList.filter( (spoon) => spoon.effectiveUpstreamAheadBy > 0 && spoon.forkAheadBy === 0, ).length; - const diverged = spoons.filter( + const diverged = spoonList.filter( (spoon) => spoon.effectiveUpstreamAheadBy > 0 && spoon.forkAheadBy > 0, ).length; - const openPullRequests = spoons.reduce( + const openPullRequests = spoonList.reduce( (total, spoon) => total + spoon.effectiveUpstreamAheadBy, 0, ); @@ -43,49 +54,68 @@ const DashboardPage = () => { -
- - - - !['resolved', 'ignored', 'failed', 'cancelled'].includes( - thread.status, - ), - ).length - } - note='Across all Spoons' - icon={MessageSquare} - /> - -
+ {metricsLoading ? ( +
+ {Array.from({ length: 4 }).map((_, index) => ( + + ))} + Loading +
+ ) : ( +
+ + + + !['resolved', 'ignored', 'failed', 'cancelled'].includes( + thread.status, + ), + ).length + } + note='Across all Spoons' + icon={MessageSquare} + /> + +
+ )}

Maintenance queue

- + {threads === undefined ? ( + + ) : ( + + )}

Recent Spoons

- {spoons.length ? ( + {spoons === undefined ? ( + + ) : spoons.length ? ( spoons .slice(0, 3) .map((spoon) => ) @@ -108,7 +138,9 @@ const DashboardPage = () => { Upstream checks - {syncRuns.length ? ( + {syncRuns === undefined ? ( + + ) : syncRuns.length ? (
{syncRuns.map((run) => (
{ Recent threads - {threads.length ? ( + {threads === undefined ? ( + + ) : threads.length ? (
{threads.slice(0, 5).map((thread) => (
void; +}) => { + useEffect(() => { + Sentry.captureException(error); + }, [error]); + + return ( +
+

Something went wrong

+

+ This page hit an unexpected error. It has been reported. +

+ +
+ ); +}; + +export default AppError; diff --git a/apps/next/src/app/(app)/machine/page.tsx b/apps/next/src/app/(app)/machine/page.tsx new file mode 100644 index 0000000..0a854e3 --- /dev/null +++ b/apps/next/src/app/(app)/machine/page.tsx @@ -0,0 +1,5 @@ +import { MachineShell } from '@/components/machine/machine-shell'; + +const MachinePage = () => ; + +export default MachinePage; diff --git a/apps/next/src/app/(app)/not-found.tsx b/apps/next/src/app/(app)/not-found.tsx new file mode 100644 index 0000000..0e9a171 --- /dev/null +++ b/apps/next/src/app/(app)/not-found.tsx @@ -0,0 +1,17 @@ +import Link from 'next/link'; + +import { Button } from '@spoon/ui'; + +const NotFound = () => ( +
+

Not found

+

+ This item does not exist, or you do not have access to it. +

+ +
+); + +export default NotFound; diff --git a/apps/next/src/app/(app)/settings/ai/page.tsx b/apps/next/src/app/(app)/settings/ai/page.tsx deleted file mode 100644 index 35a52e3..0000000 --- a/apps/next/src/app/(app)/settings/ai/page.tsx +++ /dev/null @@ -1,5 +0,0 @@ -import { redirect } from 'next/navigation'; - -const AiSettingsPage = () => redirect('/settings/ai-providers'); - -export default AiSettingsPage; diff --git a/apps/next/src/app/(app)/settings/layout.tsx b/apps/next/src/app/(app)/settings/layout.tsx index 218d154..639fb03 100644 --- a/apps/next/src/app/(app)/settings/layout.tsx +++ b/apps/next/src/app/(app)/settings/layout.tsx @@ -3,7 +3,15 @@ import type { ReactNode } from 'react'; import Link from 'next/link'; import { usePathname } from 'next/navigation'; -import { Brain, FileCog, Github, ServerCog, Shield, User } from 'lucide-react'; +import { + Bell, + Brain, + FileCog, + Github, + ServerCog, + Shield, + User, +} from 'lucide-react'; import { cn } from '@spoon/ui'; @@ -12,6 +20,7 @@ const settingsItems = [ { href: '/settings/integrations', label: 'Integrations', icon: Github }, { href: '/settings/ai-providers', label: 'AI providers', icon: Brain }, { href: '/settings/dotfiles', label: 'Dotfiles', icon: FileCog }, + { href: '/settings/notifications', label: 'Notifications', icon: Bell }, { href: '/settings/worker', label: 'Worker', icon: ServerCog }, { href: '/settings/security', label: 'Security', icon: Shield }, ]; diff --git a/apps/next/src/app/(app)/settings/notifications/page.tsx b/apps/next/src/app/(app)/settings/notifications/page.tsx new file mode 100644 index 0000000..42732cc --- /dev/null +++ b/apps/next/src/app/(app)/settings/notifications/page.tsx @@ -0,0 +1,19 @@ +import { NotificationPreferencesPanel } from '@/components/settings/notification-preferences-panel'; + +const SettingsNotificationsPage = () => { + return ( +
+
+

Notifications

+

+ Choose which notifications also reach you by email. In-app + notifications always show in the bell; these toggles only control the + email copies. +

+
+ +
+ ); +}; + +export default SettingsNotificationsPage; diff --git a/apps/next/src/app/(app)/settings/profile/page.tsx b/apps/next/src/app/(app)/settings/profile/page.tsx index d6e80ca..a585525 100644 --- a/apps/next/src/app/(app)/settings/profile/page.tsx +++ b/apps/next/src/app/(app)/settings/profile/page.tsx @@ -1,5 +1,3 @@ -'use server'; - import { AvatarUpload, ProfileHeader, diff --git a/apps/next/src/app/(app)/spoons/[spoonId]/agent/[jobId]/page.tsx b/apps/next/src/app/(app)/spoons/[spoonId]/agent/[jobId]/page.tsx index 06a8883..f37e085 100644 --- a/apps/next/src/app/(app)/spoons/[spoonId]/agent/[jobId]/page.tsx +++ b/apps/next/src/app/(app)/spoons/[spoonId]/agent/[jobId]/page.tsx @@ -21,7 +21,24 @@ const AgentWorkspacePage = () => { if (job?.threadId) router.replace(`/threads/${job.threadId}`); }, [job?.threadId, router]); - if (job?.threadId) { + if (job === undefined) { + return ( +
Loading workspace...
+ ); + } + + if (job === null) { + return ( +
+

This workspace was not found.

+ +
+ ); + } + + if (job.threadId) { return (
Opening thread workspace... @@ -37,7 +54,7 @@ const AgentWorkspacePage = () => { Back to Spoon - +
); }; diff --git a/apps/next/src/app/(app)/spoons/page.tsx b/apps/next/src/app/(app)/spoons/page.tsx index 0358e14..e126143 100644 --- a/apps/next/src/app/(app)/spoons/page.tsx +++ b/apps/next/src/app/(app)/spoons/page.tsx @@ -1,8 +1,8 @@ 'use client'; import Link from 'next/link'; -import { useRouter } from 'next/navigation'; import { SpoonStatusBadge } from '@/components/spoons/spoon-status-badge'; +import { ListSkeleton } from '@/components/ui/list-skeleton'; import { useQuery } from 'convex/react'; import { ArrowUpRight, @@ -17,6 +17,7 @@ import { Button, Card, CardContent, + Skeleton, Table, TableBody, TableCell, @@ -31,16 +32,19 @@ const formatDate = (value?: number) => : 'Never'; const SpoonsPage = () => { - const router = useRouter(); - const spoons = useQuery(api.spoons.listMineWithState, {}) ?? []; - const threads = useQuery(api.threads.listMine, { limit: 100 }) ?? []; - const active = spoons.filter((spoon) => spoon.status === 'active').length; - const needsReview = threads.filter( + const spoons = useQuery(api.spoons.listMineWithState, {}); + const threads = useQuery(api.threads.listMine, { limit: 100 }); + const spoonList = spoons ?? []; + const threadList = threads ?? []; + const spoonsLoading = spoons === undefined; + const metricsLoading = spoons === undefined || threads === undefined; + const active = spoonList.filter((spoon) => spoon.status === 'active').length; + const needsReview = threadList.filter( (thread) => thread.spoonId && !['resolved', 'ignored', 'failed', 'cancelled'].includes(thread.status), ).length; - const upstreamWaiting = spoons.reduce( + const upstreamWaiting = spoonList.reduce( (total, spoon) => total + spoon.effectiveUpstreamAheadBy, 0, ); @@ -65,7 +69,11 @@ const SpoonsPage = () => {

Managed

-

{spoons.length}

+ {spoonsLoading ? ( + + ) : ( +

{spoonList.length}

+ )}
@@ -74,7 +82,11 @@ const SpoonsPage = () => {

Active

-

{active}

+ {spoonsLoading ? ( + + ) : ( +

{active}

+ )}
@@ -83,14 +95,20 @@ const SpoonsPage = () => {

Open threads

-

{needsReview}

+ {metricsLoading ? ( + + ) : ( +

{needsReview}

+ )}
- {spoons.length ? ( + {spoons === undefined ? ( + + ) : spoons.length ? ( @@ -111,22 +129,12 @@ const SpoonsPage = () => { return ( router.push(href)} - onKeyDown={(event) => { - if (event.key === 'Enter' || event.key === ' ') { - event.preventDefault(); - router.push(href); - } - }} + className='hover:bg-muted/50 relative cursor-pointer' > event.stopPropagation()} + className='group inline-flex min-w-0 flex-col after:absolute after:inset-0 after:content-[""]' > {spoon.name} @@ -170,10 +178,7 @@ const SpoonsPage = () => { {formatDate(spoon.lastCheckedAt)} - + ); } diff --git a/apps/next/src/app/(app)/threads/page.tsx b/apps/next/src/app/(app)/threads/page.tsx index a11127c..6f994b1 100644 --- a/apps/next/src/app/(app)/threads/page.tsx +++ b/apps/next/src/app/(app)/threads/page.tsx @@ -4,6 +4,7 @@ import { useState } from 'react'; import Link from 'next/link'; import { useRouter, useSearchParams } from 'next/navigation'; import { DeleteThreadButton } from '@/components/threads/delete-thread-button'; +import { ListSkeleton } from '@/components/ui/list-skeleton'; import { useMutation, useQuery } from 'convex/react'; import { MessageSquare, Plus } from 'lucide-react'; import { toast } from 'sonner'; @@ -48,8 +49,7 @@ const ThreadsPage = () => { const spoons = useQuery(api.spoons.listMineWithState, {}) ?? []; const profiles = useQuery(api.aiProviderProfiles.listMine, {}) ?? []; const defaultProfile = profiles.find((profile) => profile.isDefault); - const threads = - useQuery(api.threads.listMine, { + const threads = useQuery(api.threads.listMine, { source: source as | 'all' | 'user_request' @@ -69,9 +69,10 @@ const ThreadsPage = () => { | 'ignored' | 'failed' | 'cancelled', - limit: 100, - }) ?? []; - const visibleThreads = threads.filter((thread) => { + limit: 100, + }); + const threadsLoading = threads === undefined; + const visibleThreads = (threads ?? []).filter((thread) => { if (spoonFilter !== 'all' && thread.spoonId !== spoonFilter) return false; if (priorityFilter !== 'all' && thread.priority !== priorityFilter) { return false; @@ -312,25 +313,25 @@ const ThreadsPage = () => {
- {visibleThreads.length ? ( + {threadsLoading ? ( + + ) : visibleThreads.length ? ( visibleThreads.map((thread) => ( router.push(threadTarget(thread))} - onKeyDown={(event) => { - if (event.key === 'Enter' || event.key === ' ') { - event.preventDefault(); - router.push(threadTarget(thread)); - } - }} + className='hover:border-primary/50 relative cursor-pointer shadow-none transition-colors' >
-

{thread.title}

+

+ + {thread.title} + +

{thread.spoonName ? ( {thread.spoonName} ) : null} @@ -361,15 +362,10 @@ const ThreadsPage = () => { {thread.latestJobWorkspaceStatus.replaceAll('_', ' ')}

) : null} -
+
{thread.latestAgentJobId ? ( ) : null} {thread.latestJobPullRequestUrl ? ( @@ -378,7 +374,6 @@ const ThreadsPage = () => { href={thread.latestJobPullRequestUrl} target='_blank' rel='noreferrer' - onClick={(event) => event.stopPropagation()} > PR diff --git a/apps/next/src/app/(app)/updates/page.tsx b/apps/next/src/app/(app)/updates/page.tsx deleted file mode 100644 index 02d0364..0000000 --- a/apps/next/src/app/(app)/updates/page.tsx +++ /dev/null @@ -1,7 +0,0 @@ -import { redirect } from 'next/navigation'; - -const UpdatesRedirectPage = () => { - redirect('/threads?source=upstream_update'); -}; - -export default UpdatesRedirectPage; diff --git a/apps/next/src/app/(auth)/forgot-password/page.tsx b/apps/next/src/app/(auth)/forgot-password/page.tsx index ec9c61f..fe56805 100644 --- a/apps/next/src/app/(auth)/forgot-password/page.tsx +++ b/apps/next/src/app/(auth)/forgot-password/page.tsx @@ -100,12 +100,14 @@ const ForgotPassword = () => { await signIn('password', formData).then(() => { setEmail(values.email); setFlow('reset-verification'); + // Only clear the form once the request succeeded; a failed request + // keeps the typed email so the user can retry without re-entering it. + forgotPasswordForm.reset(); }); } catch (error) { console.error('Error resetting password: ', error); toast.error('Error resetting password.'); } finally { - forgotPasswordForm.reset(); setLoading(false); } }; @@ -121,12 +123,14 @@ const ForgotPassword = () => { setLoading(true); try { await signIn('password', formData); + // Only clear on success (navigation follows); a failed submit keeps the + // entered password so the user can correct and retry. + resetVerificationForm.reset(); router.push('/'); } catch (error) { console.error('Error resetting password: ', error); toast.error('Error resetting password.'); } finally { - resetVerificationForm.reset(); setLoading(false); } }; @@ -230,7 +234,7 @@ const ForgotPassword = () => { Please enter the one-time password sent to your - phone. + email.
@@ -265,7 +269,7 @@ const ForgotPassword = () => { render={({ field }) => ( - Confirm Passsword + Confirm Password { - return { - title: 'Profile', - robots: { - index: false, - follow: false, - googleBot: { - index: false, - follow: false, - }, - }, - }; -}; - -const ProfileLayout = async ({ - children, -}: Readonly<{ children: React.ReactNode }>) => { - if (!(await isAuthenticatedNextjs())) redirect('/sign-in'); - return <>{children}; -}; -export default ProfileLayout; diff --git a/apps/next/src/app/(auth)/profile/page.tsx b/apps/next/src/app/(auth)/profile/page.tsx deleted file mode 100644 index e38982b..0000000 --- a/apps/next/src/app/(auth)/profile/page.tsx +++ /dev/null @@ -1,6 +0,0 @@ -import { redirect } from 'next/navigation'; - -const Profile = () => { - redirect('/settings/profile'); -}; -export default Profile; diff --git a/apps/next/src/app/(auth)/sign-in/page.tsx b/apps/next/src/app/(auth)/sign-in/page.tsx index 42d6eac..8cc49ff 100644 --- a/apps/next/src/app/(auth)/sign-in/page.tsx +++ b/apps/next/src/app/(auth)/sign-in/page.tsx @@ -132,12 +132,14 @@ const SignIn = () => { formData.append('flow', flow); setLoading(true); try { - await signIn('password', formData).then(() => router.push('/dashboard')); + await signIn('password', formData).then(() => { + signInForm.reset(); + router.push('/dashboard'); + }); } catch (error) { console.error('Error signing in:', error); toast.error('Error signing in.'); } finally { - signInForm.reset(); setLoading(false); } }; @@ -153,6 +155,7 @@ const SignIn = () => { if (values.confirmPassword !== values.password) throw new ConvexError('Passwords do not match.'); await signIn('password', formData).then(() => { + signUpForm.reset(); setEmail(values.email); setFlow('email-verification'); }); @@ -160,7 +163,6 @@ const SignIn = () => { console.error('Error signing up:', error); toast.error('Error signing up.'); } finally { - signUpForm.reset(); setLoading(false); } }; @@ -174,12 +176,14 @@ const SignIn = () => { formData.append('email', email); setLoading(true); try { - await signIn('password', formData).then(() => router.push('/dashboard')); + await signIn('password', formData).then(() => { + verifyEmailForm.reset(); + router.push('/dashboard'); + }); } catch (error) { console.error('Error verifying email:', error); toast.error('Error verifying email.'); } finally { - verifyEmailForm.reset(); setLoading(false); } }; @@ -232,7 +236,7 @@ const SignIn = () => { /> Verify Email @@ -423,7 +427,7 @@ const SignIn = () => { render={({ field }) => ( - Confirm Passsword + Confirm Password + await withBox(async (username) => { + const url = new URL(request.url); + return await proxyBox( + username, + 'file', + { method: 'GET' }, + new URLSearchParams({ path: url.searchParams.get('path') ?? '' }), + ); + }); + +export const PUT = async (request: Request) => + await withBox( + async (username) => + await proxyBox(username, 'file', { + method: 'PUT', + body: await request.text(), + }), + ); diff --git a/apps/next/src/app/api/box/lifecycle/route.ts b/apps/next/src/app/api/box/lifecycle/route.ts new file mode 100644 index 0000000..365aab6 --- /dev/null +++ b/apps/next/src/app/api/box/lifecycle/route.ts @@ -0,0 +1,10 @@ +import { proxyBox, withBox } from '@/lib/agent-worker-proxy'; + +export const POST = async (request: Request) => + await withBox( + async (username) => + await proxyBox(username, 'lifecycle', { + method: 'POST', + body: await request.text(), + }), + ); diff --git a/apps/next/src/app/api/box/status/route.ts b/apps/next/src/app/api/box/status/route.ts new file mode 100644 index 0000000..838a103 --- /dev/null +++ b/apps/next/src/app/api/box/status/route.ts @@ -0,0 +1,6 @@ +import { proxyBox, withBox } from '@/lib/agent-worker-proxy'; + +export const GET = async () => + await withBox( + async (username) => await proxyBox(username, 'status', { method: 'GET' }), + ); diff --git a/apps/next/src/app/api/box/terminal-token/route.ts b/apps/next/src/app/api/box/terminal-token/route.ts new file mode 100644 index 0000000..855e5a7 --- /dev/null +++ b/apps/next/src/app/api/box/terminal-token/route.ts @@ -0,0 +1,14 @@ +import { NextResponse } from 'next/server'; +import { mintBoxTerminalToken, resolveBoxUsername } from '@/lib/agent-worker-proxy'; + +export const GET = async () => { + const resolved = await resolveBoxUsername(); + if (!resolved.ok) return resolved.response; + const minted = mintBoxTerminalToken(resolved.username); + return minted + ? NextResponse.json(minted) + : NextResponse.json( + { error: 'Terminal is not configured on this deployment.' }, + { status: 503 }, + ); +}; diff --git a/apps/next/src/app/api/box/tree/route.ts b/apps/next/src/app/api/box/tree/route.ts new file mode 100644 index 0000000..77a27f5 --- /dev/null +++ b/apps/next/src/app/api/box/tree/route.ts @@ -0,0 +1,6 @@ +import { proxyBox, withBox } from '@/lib/agent-worker-proxy'; + +export const GET = async () => + await withBox( + async (username) => await proxyBox(username, 'tree', { method: 'GET' }), + ); diff --git a/apps/next/src/app/api/threads/[threadId]/message/route.ts b/apps/next/src/app/api/threads/[threadId]/message/route.ts index 2b0626d..3934585 100644 --- a/apps/next/src/app/api/threads/[threadId]/message/route.ts +++ b/apps/next/src/app/api/threads/[threadId]/message/route.ts @@ -35,7 +35,14 @@ export const POST = async ( { status: 400 }, ); } - const details = await fetchQuery(api.threads.get, { threadId }, { token }); + const details = (await fetchQuery( + api.threads.get, + { threadId }, + { token }, + )); + if (details === null) { + return NextResponse.json({ error: 'Thread not found.' }, { status: 404 }); + } const latestJob = details.latestJob; const canSendToWorker = latestJob && diff --git a/apps/next/src/components/agent-workspace/agent-thread.tsx b/apps/next/src/components/agent-workspace/agent-thread.tsx index 9a7bff5..b6da874 100644 --- a/apps/next/src/components/agent-workspace/agent-thread.tsx +++ b/apps/next/src/components/agent-workspace/agent-thread.tsx @@ -2,6 +2,7 @@ import { useEffect, useMemo, useRef, useState } from 'react'; import { + ArrowDown, Ban, FilePenLine, MessagesSquare, @@ -16,6 +17,7 @@ import { Badge, Button, Textarea } from '@spoon/ui'; import { DiffFileView, useDiffTheme } from './diff-file-view'; import { parseDiffFileForPath } from './diff-utils'; +import { isNearBottom } from './workspace-helpers'; type ActivityFilter = 'all' | 'chat' | 'activity' | 'files' | 'errors'; @@ -70,6 +72,7 @@ export const AgentThread = ({ const [filter, setFilter] = useState('all'); const diffTheme = useDiffTheme(); const scrollRef = useRef(null); + const [showJumpToLatest, setShowJumpToLatest] = useState(false); const chatMessages = useMemo( () => messages.filter((message) => { @@ -111,20 +114,35 @@ export const AgentThread = ({ ? [] : workspaceChanges; + const scrollToLatest = () => { + const node = scrollRef.current; + if (!node) return; + if (typeof node.scrollTo === 'function') { + node.scrollTo({ top: node.scrollHeight, behavior: 'smooth' }); + } else { + node.scrollTop = node.scrollHeight; + } + setShowJumpToLatest(false); + }; + + // Only auto-scroll to the newest content when the user is already near the + // bottom. If they have scrolled up to read, don't yank them — surface a + // "jump to latest" affordance instead. (The agent streaming no longer forces + // a scroll; that was fighting the reader.) useEffect(() => { const node = scrollRef.current; if (!node) return; - const distanceFromBottom = - node.scrollHeight - node.scrollTop - node.clientHeight; - if (distanceFromBottom < 160 || agentTurnActive) { + if (isNearBottom(node)) { if (typeof node.scrollTo === 'function') { node.scrollTo({ top: node.scrollHeight, behavior: 'smooth' }); } else { node.scrollTop = node.scrollHeight; } + setShowJumpToLatest(false); + } else { + setShowJumpToLatest(true); } }, [ - agentTurnActive, events.length, interactions.length, messages.length, @@ -227,8 +245,13 @@ export const AgentThread = ({ ))}
+
{ + const node = scrollRef.current; + if (node && isNearBottom(node)) setShowJumpToLatest(false); + }} className='min-h-0 flex-1 space-y-3 overflow-y-auto overscroll-contain p-3' > {(filter === 'all' || filter === 'chat') && interactions.length > 0 @@ -430,6 +453,19 @@ export const AgentThread = ({ yet.

) : null} +
+ {showJumpToLatest ? ( + + ) : null}