Files
spoon/docs/superpowers/plans/2026-07-10-spoon-phase2-runtime-unification.md
T

46 KiB
Raw Blame History

Spoon Phase 2 — Runtime Unification: Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Collapse the three agent runtimes (Codex, OpenCode, Claude Code) behind one AgentRuntime adapter interface in the worker, run all three inside the per-user spoon-box-{username} container (no per-job containers), and let users pick the runtime per thread / per spoon with queue-time validation against their AI provider profile.

Architecture: The worker (apps/agent-worker) polls Convex, claims a job, acquires the user's persistent box (Phase 1), clones ~/Code/{spoon}/{branch} into the mounted home, then runs agent turns by docker exec-ing a CLI into the box. Today dispatch is implicit (Codex if the profile is a ChatGPT-login profile, otherwise a separate opencode serve container). Phase 2 makes job.runtime (codex|opencode|claude) authoritative: a registry maps it to an adapter; every adapter execs into the box via streamExecInContainer, tags its process group with a unique marker, and kills that group in-box on timeout/abort. The OpenCode server-in-a-side-container path is deleted. The Codex auth.json encrypted-snapshot pattern is generalized to Claude OAuth credentials.

Tech Stack: TypeScript, Bun, execa (docker/podman CLI), Convex (packages/backend/convex), Vitest (worker unit tests + convex-test backend tests), Next.js 16 (apps/next), Fedora 41 job image (docker/agent-job.Dockerfile) shipping opencode, @openai/codex, and (new) @anthropic-ai/claude-code.

Depends on: Phase 1 (box lifecycle: per-username mutex, idempotent acquire/release handles, --init on the box, heartbeat + cancelRequested teardown, startup reconcile of spoon-box-*). This plan assumes acquireUserBox/releaseUserBox are already concurrency-safe and the box runs with --init so orphaned in-box children are reaped. Where Phase-1 handle semantics are referenced, use whatever acquireUserBox returns after Phase 1; if Phase 1 kept the string box-name return used in worker.ts:1330, keep using it.

Global Constraints

  • Worker unit tests live in apps/agent-worker/tests/unit/*.test.ts; run cd apps/agent-worker && bun run test:unit (vitest run --project unit). Worker typecheck: cd apps/agent-worker && bun run typecheck.
  • Backend tests live in packages/backend/tests/unit/*.test.ts using convex-test (see packages/backend/tests/unit/harness.test.ts for the exact fixture style: convexTest(schema, modules), authed(t, userId), spoonInput/githubSpoonInput). Run cd packages/backend && bun run test:unit. Always cd packages/backend && bun run codegen before bun run typecheck (codegen regenerates _generated/ after schema/function changes).
  • Match the existing fixture/test style: expect(normalizeX(...)).toContainEqual({ kind: ... }) for event normalizers (see apps/agent-worker/tests/unit/agent-events.test.ts); mkdtemp + afterEach cleanup for filesystem tests (see apps/agent-worker/tests/unit/codex-runtime.test.ts).
  • Conventional commits; one commit per task. Run the relevant test:unit + typecheck (and codegen for backend) before committing. Do not commit with a failing suite.
  • Three runtimes: codex, opencode, claude — all docker exec into spoon-box-{username}. No per-job containers. Never leave a spoon-agent-job-* code path behind.
  • A real turn against a live CLI cannot run in unit tests. Adapters therefore take an injectable exec function so runTurn can be driven with fixture stream lines; live behavior is covered by the extended scripts/smoke-agent-container manual step. Event parsing is covered by pure normalizer fixtures.

Task 1: Define the AgentRuntime adapter interface, shared types, and registry

Files:

  • Create apps/agent-worker/src/runtime/agent-runtime.ts (new).
  • Create apps/agent-worker/src/runtime/provider.ts (new — pure helpers moved out of worker.ts so adapters don't import worker.ts at runtime).
  • Edit apps/agent-worker/src/worker.ts (extract the Claim type and pure helpers; make ActiveWorkspace extend the shared workspace shape; add runtime/turnMarker fields).
  • Create apps/agent-worker/tests/unit/agent-runtime.test.ts (new).

Interfaces:

Produces (runtime/agent-runtime.ts):

import type { NormalizedAgentEvent } from '../agent-events';
import type { streamExecInContainer } from './docker';
import type { AdapterWorkspace } from './provider';

export type AgentRuntimeName = 'codex' | 'opencode' | 'claude';

export type ExecStreamFn = typeof streamExecInContainer;

export type TurnResult = {
  // Assistant text the adapter captured out-of-band (Codex --output-last-message
  // file, Claude `result` event). Empty/undefined when everything streamed via onEvent.
  finalMessage?: string;
  // Runtime session id to persist for continuity (resume/--session/--resume).
  sessionId?: string;
  // Non-empty when the runtime reported a hard failure the caller must surface.
  error?: string;
};

export type AgentRuntime = {
  readonly name: AgentRuntimeName;
  prepareAuth(workspace: AdapterWorkspace): Promise<void>;
  runTurn(
    workspace: AdapterWorkspace,
    prompt: string,
    onEvent: (event: NormalizedAgentEvent) => Promise<void>,
  ): Promise<TurnResult>;
  abort(workspace: AdapterWorkspace): Promise<void>;
};

export type AdapterFactory = (deps?: { execStream?: ExecStreamFn }) => AgentRuntime;

Produces (runtime/provider.ts) — AdapterWorkspace is the subset of ActiveWorkspace adapters read, plus the pure helpers currently in worker.ts:47-95, 350-446:

export type Claim = { /* moved verbatim from worker.ts:47-95, but runtime widened */ };
// job.runtime union becomes: runtime?: 'codex' | 'opencode' | 'claude';

export type AdapterWorkspace = {
  claim: Claim;
  workdir: string;
  homeDir: string;
  username: string;
  containerHome: string;   // e.g. /home/{username}
  containerRepo: string;   // e.g. /home/{username}/Code/{spoon}/{branch}
  repoDir: string;         // host path to the checkout
  boxName: string;         // spoon-box-{username}
  redact: (value: string) => string;
  runtime: AgentRuntimeName;
  codexSessionId?: string;
  opencodeSessionId?: string;
  claudeSessionId?: string;
  turnMarker?: string;     // set per turn; used for in-box process-group kill
};

export const isCodexLoginProfile: (claim: Claim) => boolean;           // moved from worker.ts:350
export const collectJsonStringValues: (value?: string) => string[];    // moved from worker.ts:354
export const providerEnvironment: (claim: Claim, workspaceRoot?: string) => Record<string, string>; // moved from worker.ts:379
export const opencodeModel: (claim: Claim) => string;                  // moved from worker.ts:425
export const codexModel: (claim: Claim) => string;                     // moved from worker.ts:440
export const codexModelArgs: (claim: Claim) => string[];               // moved from worker.ts:445

Consumes: NormalizedAgentEvent from agent-events.ts; streamExecInContainer from runtime/docker.ts.

Registry (runtime/agent-runtime.ts): a lazy map. Adapters register in later tasks; until then getAdapter throws for unknown names.

const factories = new Map<AgentRuntimeName, AdapterFactory>();
export const registerAdapter = (name: AgentRuntimeName, factory: AdapterFactory): void => { factories.set(name, factory); };
export const getAdapter = (name: AgentRuntimeName): AgentRuntime => {
  const factory = factories.get(name);
  if (!factory) throw new Error(`No agent runtime adapter registered for "${name}".`);
  return factory();
};

Steps:

  • Write a failing test apps/agent-worker/tests/unit/agent-runtime.test.ts: registerAdapter('codex', () => fake) then expect(getAdapter('codex').name).toBe('codex'), and expect(() => getAdapter('opencode')).toThrow(/No agent runtime adapter/). Also assert isCodexLoginProfile/opencodeModel re-exported from runtime/provider.ts behave as before (copy one existing expectation, e.g. opencodeModel({ aiProviderProfile: { provider: 'anthropic', model: 'claude-x', ... } } as Claim)'anthropic/claude-x').
  • Run cd apps/agent-worker && bun run test:unit → FAIL (module/exports missing).
  • Create runtime/provider.ts: move Claim (worker.ts:47-95), isCodexLoginProfile, collectJsonStringValues, providerEnvironment, opencodeModel, codexModel, codexModelArgs (worker.ts:350-446) verbatim; widen Claim['job']['runtime'] to 'codex' | 'opencode' | 'claude'; add and export AdapterWorkspace.
  • Create runtime/agent-runtime.ts with the interface, TurnResult, ExecStreamFn, AdapterFactory, and the registry above.
  • Edit worker.ts: import Claim, the helpers, and AdapterWorkspace from runtime/provider.ts (delete the now-moved definitions); change ActiveWorkspace (worker.ts:97) to type ActiveWorkspace = AdapterWorkspace & { githubToken: string; runtimeMode?: ...; agentTurnActive?: boolean; resolveTurn?: () => void; lastRecordedDiffSignature?: string; codexTurnError?: string; ... }; set runtime when the workspace is built in runClaim (worker.ts:1348 — temporarily runtime: 'codex' placeholder; real resolution lands in Task 3).
  • Run cd apps/agent-worker && bun run test:unit → PASS. Run bun run typecheck → PASS.
  • Commit: refactor(worker): extract AgentRuntime interface, shared provider helpers, and adapter registry.

Task 2: In-box process-group kill + marked-command helpers

Files:

  • Edit apps/agent-worker/src/runtime/docker.ts (add two exported helpers near streamExecInContainer at line 385).
  • Create apps/agent-worker/tests/unit/box-process-kill.test.ts (new).

Interfaces:

Produces (runtime/docker.ts):

// Wraps a CLI argv so the process runs as a new session/process-group leader whose
// bash parent carries the marker in its argv (matchable by `pgrep -f`). We do NOT
// `exec` the CLI: bash tail-exec's a final simple command, so an `exec`-or-trailing
// CLI would replace the marker-carrying bash and `pgrep -f <marker>` would miss the
// running turn. Keep a trailing `rc=$?; exit "$rc"` so the CLI is no longer the final
// statement (bash survives as group leader) while still propagating its exit code
// (downstream `normalizeRunResult` relies on it; a bare `wait` would mask failures).
export const buildMarkedCommand = (marker: string, command: string[]): string[] => {
  // assertMarker(marker) — reject anything but /^[A-Za-z0-9_-]+$/ so a newline can't
  // break out of the `# comment` line.
  const script = [
    `# ${marker}`,
    'exec 0</dev/null',
    command.map(shellQuote).join(' '),
    'rc=$?',
    'exit "$rc"',
  ].join('\n');
  return ['setsid', 'bash', '-lc', script];
};

// Kills every process group whose bash parent matches the marker, TERM then KILL.
// pgrep self-match: the kill script runs via `bash -lc <script>` whose own argv holds
// the marker literal, so pgrep matches the kill script (and its command-substitution
// subshells) too. Exclude its own pgid or it SIGTERMs itself on iteration one and
// leaks the target; kill the TARGET's pgid instead.
export const killBoxProcessesByMarker = async (args: {
  containerName: string;
  marker: string;
}): Promise<void> => {
  // assertMarker(args.marker) — same /^[A-Za-z0-9_-]+$/ guard for the pgrep pattern.
  const script = [
    `self_pgid=$(ps -o pgid= -p $$ | tr -d ' ')`,
    `pids=$(pgrep -f ${shellQuote(args.marker)} || true)`,
    `for pid in $pids; do`,
    `  pgid=$(ps -o pgid= -p "$pid" | tr -d ' ')`,
    `  [ -z "$pgid" ] && continue`,
    `  [ "$pgid" = "$self_pgid" ] && continue`,
    `  kill -TERM -"$pgid" 2>/dev/null || true`,
    `done`,
    `sleep 2`,
    `for pid in $pids; do`,
    `  pgid=$(ps -o pgid= -p "$pid" | tr -d ' ')`,
    `  [ -z "$pgid" ] && continue`,
    `  [ "$pgid" = "$self_pgid" ] && continue`,
    `  kill -KILL -"$pgid" 2>/dev/null || true`,
    `done`,
  ].join('\n');
  await execa(containerRuntime(), ['exec', args.containerName, 'bash', '-lc', script], {
    reject: false, stdin: 'ignore',
  });
};

Add a local const shellQuote = (v: string) => '${v.replaceAll("'", "'\''")}'; (mirror worker.ts:1049).

Consumes: execa, containerRuntime() (already in file).

Steps:

  • Write failing apps/agent-worker/tests/unit/box-process-kill.test.ts: import buildMarkedCommand. Assert buildMarkedCommand('spoon-turn-abc', ['codex', 'exec', '--json', 'hi']) returns ['setsid', 'bash', '-lc', expect.stringContaining('# spoon-turn-abc')] and the script's last line contains codex exec --json 'hi'. (Do not unit-test killBoxProcessesByMarker end-to-end — it shells out; assert only the argv via a spy if you refactor it to a pure buildKillScript(marker) helper. Recommended: extract export const buildKillScript = (marker: string): string and test it contains pgrep -f 'spoon-turn-abc', kill -TERM -"$pgid", and kill -KILL -"$pgid", plus the self-pgid exclusion ([ "$pgid" = "$self_pgid" ] && continue).)
  • Run bun run test:unit → FAIL.
  • Implement buildMarkedCommand, buildKillScript, and killBoxProcessesByMarker in docker.ts.
  • Run bun run test:unit → PASS. bun run typecheck → PASS.
  • Commit: feat(worker): in-box process-group marker + kill helpers for agent turns.

Task 3: CodexAdapter — refactor runCodexTurn behind the interface, dispatch codex through the registry

Files:

  • Create apps/agent-worker/src/runtime/codex-adapter.ts (new).
  • Edit apps/agent-worker/src/worker.ts (replace runCodexTurn call path in sendWorkspaceMessage; resolve workspace.runtime; route abort).
  • Create apps/agent-worker/tests/unit/codex-adapter.test.ts (new).

Interfaces:

Produces (runtime/codex-adapter.ts):

export const createCodexAdapter: AdapterFactory = (deps) => {
  const execStream = deps?.execStream ?? streamExecInContainer;
  return {
    name: 'codex',
    prepareAuth,      // moves prepareCodexAuth (worker.ts:459) + prepareCodexWorkspaceFiles + writeJsonFile here
    runTurn,          // moves runCodexTurn body (worker.ts:712-853), using buildMarkedCommand + execStream
    abort,            // killBoxProcessesByMarker({ containerName: ws.boxName, marker: ws.turnMarker })
  };
};

runTurn(ws, prompt, onEvent):

  1. ws.runtime = 'codex'; ws.turnMarker = 'spoon-turn-' + randomUUID(); ws.codexSessionId continuity preserved.
  2. Build the same codex argv as worker.ts:737-761 (codex exec [resume <id>] --json ...codexModelArgs --dangerously-bypass-approvals-and-sandbox --output-last-message <containerPath> [--cd <repo>] <prompt>), then const command = buildMarkedCommand(ws.turnMarker, codexArgv).
  3. await execStream({ containerName: ws.boxName, containerCwd: ws.containerRepo, command, environment: { ...providerEnvironment(ws.claim, ws.containerHome), ...secretEnv }, redact: ws.redact, timeoutMs: env.jobTimeoutMs, onStdoutLine: (line) => forEach normalizeCodexJsonLine(line) → onEvent(event), onStderrLine: (line) => onEvent({ kind: 'status', status: line }) if not noise }).
  4. On timeout: register setTimeout(() => void killBoxProcessesByMarker(...), env.jobTimeoutMs) and clear it when execStream resolves (execa's own timeout only kills the local client — the marker kill is what stops the in-box process).
  5. Read --output-last-message file (worker.ts:811-843) → finalMessage. Capture error events into a local var → TurnResult.error. Return { finalMessage, sessionId: capturedSessionId, error }.

Consumes: normalizeCodexJsonLine (agent-events.ts), providerEnvironment/codexModelArgs/isCodexLoginProfile (runtime/provider.ts), buildMarkedCommand/killBoxProcessesByMarker/streamExecInContainer (runtime/docker.ts), prepareCodexWorkspaceFiles (codex-runtime.ts), env.

Worker dispatch change (sendWorkspaceMessage, worker.ts:1624): replace the isCodexLoginProfile(claim) ? runCodexTurn : env.runtime==='docker' ? runOpenCodeTurn : localRun branching with:

const adapter = getAdapter(workspace.runtime);
const onEvent = (event: NormalizedAgentEvent) => handleAgentEvent({ workspace, event, assistantMessageId, assistantContent });
const result = await adapter.runTurn(workspace, prompt, onEvent);
if (result.sessionId) await persistRuntimeSession(workspace, result.sessionId); // generic setter, replaces setCodexSessionId call site
if (!assistantContent.value.trim() && result.finalMessage) { assistantContent.value = truncate(workspace.redact(result.finalMessage), 40_000); await updateMessage(...); }
if (!assistantContent.value.trim() && result.error) throw new Error(`${workspace.runtime} failed:\n${result.error}`);

Resolve workspace.runtime in runClaim (worker.ts:1348): for now set runtime: isCodexLoginProfile(claim) ? 'codex' : 'opencode' so behavior is byte-identical to today (job.runtime becomes authoritative in Task 8). Route abortWorkspaceAgent (worker.ts:1562) through getAdapter(workspace.runtime).abort(workspace).

Steps:

  • Write failing apps/agent-worker/tests/unit/codex-adapter.test.ts: build the adapter with a fake execStream that calls onStdoutLine with fixture lines (reuse the exact JSON shapes from agent-events.test.ts:110-172, e.g. {"type":"item.completed","item":{"id":"item-1","type":"agent_message","text":"done"}} and {"type":"turn.completed"}) then resolves { exitCode: 0, output: '' }. Assert onEvent received { kind: 'assistant_delta', content: 'done\n\n', externalMessageId: 'item-1' } and { kind: 'assistant_completed' }, and that runTurn returns a TurnResult. Add a second case: fake execStream that emits {"type":"turn.failed","error":{"message":"boom"}} and returns exitCode 0 → result.error contains boom. Assert buildMarkedCommand was used (spy the fake to capture args.command[0] === 'setsid').
  • Run bun run test:unit → FAIL.
  • Create runtime/codex-adapter.ts; move Codex logic out of worker.ts (prepareCodexAuth, runCodexTurn, writeJsonFile, readCodexTurnError). Register it: add registerAdapter('codex', createCodexAdapter) in a new runtime/register.ts imported once from worker.ts top-level.
  • Rewire sendWorkspaceMessage and abortWorkspaceAgent per above; add persistRuntimeSession (writes setCodexSessionId/opencodeSessionId/claudeSessionId based on workspace.runtime). Keep the maintenance-decision parse, diff artifact, recordChangedFiles tail (worker.ts:1731-1758) unchanged.
  • Run bun run test:unit (all existing + new) → PASS. bun run typecheck → PASS.
  • Commit: refactor(worker): move Codex turn logic into CodexAdapter behind AgentRuntime.

Task 4: OpenCodeAdapter — run opencode run --format json inside the box

Files:

  • Create apps/agent-worker/src/runtime/opencode-adapter.ts (new).
  • Edit apps/agent-worker/src/agent-events.ts (add normalizeOpenCodeRunLine for opencode run --format json line output; may delegate to normalizeOpenCodeEvent).
  • Edit apps/agent-worker/src/runtime/register.ts (register opencode).
  • Create apps/agent-worker/tests/unit/opencode-adapter.test.ts (new).
  • Edit apps/agent-worker/tests/unit/agent-events.test.ts (add normalizeOpenCodeRunLine cases).

Interfaces:

Produces (runtime/opencode-adapter.ts):

export const createOpenCodeAdapter: AdapterFactory = (deps) => {
  const execStream = deps?.execStream ?? streamExecInContainer;
  return {
    name: 'opencode',
    prepareAuth: async (ws) => { /* opencode reads ANTHROPIC_API_KEY/OPENAI_API_KEY from env; for opencode_auth_json profiles, write ~/.local/share/opencode/auth.json 0600 (reuse logic from worker.ts:472-480) */ },
    runTurn,
    abort: (ws) => killBoxProcessesByMarker({ containerName: ws.boxName, marker: ws.turnMarker ?? '' }),
  };
};

runTurn(ws, prompt, onEvent):

  1. ws.runtime = 'opencode'; ws.turnMarker = 'spoon-turn-' + randomUUID().
  2. argv: ['opencode', 'run', '--format', 'json', '--model', opencodeModel(ws.claim), ...(ws.opencodeSessionId ? ['--session', ws.opencodeSessionId] : []), '--', prompt]. Verify the continuity flag against opencode run --help during the smoke step (Task 11); if --session is unsupported, drop it and note that OpenCode turns are stateless (acceptable — Codex/Claude carry continuity).
  3. command = buildMarkedCommand(ws.turnMarker, argv); execStream({ containerName: ws.boxName, containerCwd: ws.containerRepo, command, environment: { ...providerEnvironment(ws.claim), ...secretEnv }, redact, timeoutMs, onStdoutLine: (line) => normalizeOpenCodeRunLine(line).forEach(onEvent), onStderrLine }).
  4. Return { finalMessage, sessionId: capturedSessionId, error } (session id parsed from the run output if present).

Produces (agent-events.ts): export const normalizeOpenCodeRunLine = (line: string): NormalizedAgentEvent[]JSON.parse each line; if it matches the SSE-style { type, properties } shape, delegate to normalizeOpenCodeEvent(parsed); otherwise map opencode run's result shape (capture sessionID, final text). Unparseable lines → [{ kind: 'status', status: line }].

Consumes: opencodeModel/providerEnvironment (runtime/provider.ts), buildMarkedCommand/killBoxProcessesByMarker/streamExecInContainer (runtime/docker.ts).

Steps:

  • Write failing agent-events.test.ts case: normalizeOpenCodeRunLine(JSON.stringify({ type: 'message.part.delta', properties: { part: { text: 'hi' }, messageID: 'm1' } }))toContainEqual({ kind: 'assistant_delta', content: 'hi', externalMessageId: 'm1' }); and a non-JSON line → { kind: 'status', status: '<line>' }.
  • Write failing opencode-adapter.test.ts: fake execStream emits the delta line + a final line, returns { exitCode: 0, output: '' }; assert onEvent got the delta, runTurn resolves, and the captured args.command[0] === 'setsid' and args.command[3] (script) contains opencode run --format json.
  • Run bun run test:unit → FAIL.
  • Implement normalizeOpenCodeRunLine, createOpenCodeAdapter, and register it.
  • Run bun run test:unit → PASS. bun run typecheck → PASS.
  • Commit: feat(worker): OpenCodeAdapter runs opencode in the per-user box (no side container).

Task 5: Delete the per-job OpenCode container path and dead docker helpers

Files:

  • Edit apps/agent-worker/src/worker.ts (remove ensureOpenCodeSession, runOpenCodeTurn, workspaceContainerName, startWorkspaceContainer import + usage, stopWorkspaceContainer calls, containerName/containerId/opencodePassword/opencodeSession workspace fields, the OpenCode-server branches of abortWorkspaceAgent/replyToInteraction/getWorkspaceAgentStatus; retarget health/cleanup to spoon-box-).
  • Delete apps/agent-worker/src/opencode-session.ts.
  • Edit apps/agent-worker/src/runtime/docker.ts (delete runInJobContainer, streamInJobContainer, execInWorkspaceContainer, inspectWorkspaceContainer, startWorkspaceContainer, getPublishedPort; keep ensureUserContainer, streamExecInContainer, runExecInContainer, stopWorkspaceContainer, listWorkspaceContainerNames, jobWorkspaceVolumeSpec, normalizeRunResult, ensureJobImagePulled).
  • Edit apps/agent-worker/src/env.ts (remove containerAccess; remove now-dead terminalImage/terminalIdleMs/maxConcurrentJobs only if not read anywhere else — grep first).
  • Edit packages/backend/convex/* — none here.
  • Edit apps/agent-worker/tests/unit/* as needed (no test imports the deleted symbols; if any do, update).

Interfaces: Consumes: now only box-scoped docker helpers. getWorkerHealth (worker.ts:1880) enumerates listWorkspaceContainerNames('spoon-box-'); cleanupOrphanedWorkspaces (worker.ts:1911) enumerates spoon-box- for the container list (do not touch the homes/ workdir removal here — that is Phase 1's layout-aware rewrite; leave whatever Phase 1 produced).

Steps:

  • grep -rn "spoon-agent-job\|startWorkspaceContainer\|ensureOpenCodeSession\|opencode-session\|containerAccess\|host_port\|runInJobContainer\|streamInJobContainer\|execInWorkspaceContainer\|inspectWorkspaceContainer" apps/agent-worker/src → build the deletion checklist from the hits.
  • grep -rn "env.terminalImage\|env.terminalIdleMs\|env.maxConcurrentJobs" apps/agent-worker/src → only remove the env var if zero hits remain after Phase 1 (maxConcurrentJobs is surfaced in getWorkerHealth at worker.ts:1903 — drop that line too, or keep the field; note it in the commit).
  • Remove the symbols above. In abortWorkspaceAgent/replyToInteraction/getWorkspaceAgentStatus, drop the opencodeSession branches: abortWorkspaceAgentawait getAdapter(workspace.runtime).abort(workspace); replyToInteractionthrow new Error('Interactive replies are not supported by exec-based runtimes.') (no runtime generates interaction requests after this refactor); getWorkspaceAgentStatus → return { runtimeMode: workspace.runtime, codexSessionId, opencodeSessionId, claudeSessionId, active }.
  • In openWorkspacePullRequest (worker.ts:1852-1855) and stopWorkspace (worker.ts:1869-1872), delete the workspace.opencodeSession?.close() and stopWorkspaceContainer(workspace.containerName) lines (box lifecycle is now purely releaseUserBox).
  • Delete apps/agent-worker/src/opencode-session.ts and its imports (worker.ts:31-35).
  • Run cd apps/agent-worker && bun run typecheck → PASS (fix any dangling refs). Run bun run test:unit → PASS.
  • Commit: refactor(worker): delete per-job OpenCode container path, host_port split, and dead docker helpers.

Task 6: ClaudeCodeAdapter — claude -p --output-format stream-json in the box

Files:

  • Create apps/agent-worker/src/runtime/claude-adapter.ts (new).
  • Edit apps/agent-worker/src/agent-events.ts (add normalizeClaudeJsonLine).
  • Edit apps/agent-worker/src/runtime/register.ts (register claude).
  • Create apps/agent-worker/tests/unit/claude-adapter.test.ts (new).
  • Edit apps/agent-worker/tests/unit/agent-events.test.ts (add Claude normalizer cases).

Interfaces:

Produces (agent-events.ts): export const normalizeClaudeJsonLine = (line: string): NormalizedAgentEvent[]. Claude Code stream-json emits one JSON object per line:

  • {"type":"system","subtype":"init","session_id":"abc",...}{ kind: 'session', sessionId: 'abc' }.
  • {"type":"assistant","message":{"content":[{"type":"text","text":"..."}]}}{ kind: 'assistant_delta', content: '...' } (concatenate text blocks); {"type":"content_block_delta",...} if present → assistant_delta.
  • {"type":"assistant","message":{"content":[{"type":"tool_use","name":"Bash","input":{...}}]}}{ kind: 'tool_started', name, input: JSON.stringify(input) }.
  • {"type":"user","message":{"content":[{"type":"tool_result",...}]}}{ kind: 'tool_completed', name: 'tool', output }.
  • {"type":"result","subtype":"success","result":"final text","session_id":"abc"}{ kind: 'assistant_completed', content: 'final text' } (+ a session event if the id is new).
  • {"type":"result","subtype":"error_max_turns"|"error_during_execution",...}{ kind: 'error', message }.
  • Unparseable → [{ kind: 'status', status: line }].

Produces (runtime/claude-adapter.ts):

export const createClaudeAdapter: AdapterFactory = (deps) => {
  const execStream = deps?.execStream ?? streamExecInContainer;
  return {
    name: 'claude',
    prepareAuth,   // API-key kind: nothing on disk (ANTHROPIC_API_KEY comes from providerEnvironment). OAuth-json kind: write <homeDir>/.claude/.credentials.json 0600 from claim.aiProviderProfile.secret (reuse writeJsonFile).
    runTurn,
    abort: (ws) => killBoxProcessesByMarker({ containerName: ws.boxName, marker: ws.turnMarker ?? '' }),
  };
};

runTurn(ws, prompt, onEvent):

  1. ws.runtime = 'claude'; ws.turnMarker = 'spoon-turn-' + randomUUID().
  2. argv: ['claude', '-p', prompt, '--output-format', 'stream-json', '--verbose', '--dangerously-skip-permissions', '--model', claudeModel(ws.claim), ...(ws.claudeSessionId ? ['--resume', ws.claudeSessionId] : [])] (cwd = containerRepo). --verbose is required for stream-json. Confirm exact flags in the smoke step against the pinned CLI (claude --help).
  3. command = buildMarkedCommand(ws.turnMarker, argv); execStream({ containerName: ws.boxName, containerCwd: ws.containerRepo, command, environment: { ...claudeEnv(ws.claim, ws.containerHome), ...secretEnv }, redact, timeoutMs, onStdoutLine: (line) => normalizeClaudeJsonLine(line).forEach(onEvent), onStderrLine }). claudeEnv: for API-key kind { ANTHROPIC_API_KEY: secret, HOME: ws.containerHome }; for OAuth-json kind { HOME: ws.containerHome } (CLI reads ~/.claude/.credentials.json).
  4. Return { finalMessage, sessionId, error } (finalMessage from the result event; sessionId from system:init/result).

Consumes: normalizeClaudeJsonLine (agent-events.ts), providerEnvironment (runtime/provider.ts), box helpers (runtime/docker.ts), writeJsonFile (move to runtime/provider.ts or a small runtime/auth.ts shared by Codex + Claude adapters).

Steps:

  • Write failing agent-events.test.ts Claude cases: system:init → session; assistant text block → assistant_delta; result:successassistant_completed with content; result:error_max_turnserror.
  • Write failing claude-adapter.test.ts: fake execStream emits system:init + assistant text + result:success lines, returns { exitCode: 0, output: '' }; assert onEvent received the mapped events, runTurn returns { finalMessage: 'final text', sessionId: 'abc' }, and args.command[0] === 'setsid' with the script containing claude -p.
  • Run bun run test:unit → FAIL.
  • Implement normalizeClaudeJsonLine, createClaudeAdapter, register it. Move writeJsonFile to a shared runtime/auth.ts.
  • Run bun run test:unit → PASS. bun run typecheck → PASS.
  • Commit: feat(worker): ClaudeCodeAdapter runs claude -p stream-json in the per-user box.

Task 7: Backend schema — add claude runtime, Anthropic OAuth auth kind, and profile→runtimes derivation

Files:

  • Edit packages/backend/convex/schema.ts (runtime unions + authType union).
  • Create packages/backend/convex/runtimeSupport.ts (new — pure derivation helper, no 'use node').
  • Edit packages/backend/convex/agentJobs.ts and packages/backend/convex/spoonAgentSettings.ts and packages/backend/convex/aiProviderProfiles.ts/aiProviderProfilesNode.ts (union constants).
  • Create packages/backend/tests/unit/runtime-support.test.ts (new).

Interfaces:

Schema changes:

  • agentJobs.runtime (schema.ts:536): v.optional(v.union(v.literal('openai_direct'), v.literal('opencode'), v.literal('codex'), v.literal('claude'))) — keep openai_direct for legacy reads (deprecated, never written).
  • spoonAgentSettings.runtime (schema.ts:472): same widened union.
  • aiProviderProfiles.authType (schema.ts:389): add v.literal('anthropic_oauth_json').

Produces (runtimeSupport.ts):

import type { Doc } from './_generated/dataModel';
export type AgentRuntimeName = 'codex' | 'opencode' | 'claude';

// Which runtimes a provider profile can drive.
export const runtimesForProfile = (
  profile: Pick<Doc<'aiProviderProfiles'>, 'provider' | 'authType'>,
): AgentRuntimeName[] => {
  if (profile.provider === 'opencode_openai_login' || profile.authType === 'opencode_auth_json') {
    return ['codex'];                              // ChatGPT-login snapshot → Codex CLI only
  }
  if (profile.provider === 'anthropic') {
    return profile.authType === 'anthropic_oauth_json' ? ['claude'] : ['claude', 'opencode'];
  }
  if (profile.provider === 'openai') return ['opencode', 'codex'];
  // Every remaining API-key provider is OpenAI-compatible → OpenCode.
  return ['opencode'];
};

Steps:

  • Write failing packages/backend/tests/unit/runtime-support.test.ts: runtimesForProfile({ provider: 'anthropic', authType: 'api_key' })['claude','opencode']; { provider: 'opencode_openai_login', authType: 'opencode_auth_json' }['codex']; { provider: 'openrouter', authType: 'api_key' }['opencode']; { provider: 'openai', authType: 'api_key' }['opencode','codex'].
  • Run cd packages/backend && bun run test:unit → FAIL.
  • Edit schema.ts unions; create runtimeSupport.ts; widen the authType union constant in aiProviderProfiles.ts:24, aiProviderProfilesNode.ts:24.
  • Run cd packages/backend && bun run codegen then bun run typecheck → PASS. bun run test:unit → PASS.
  • Commit: feat(backend): add claude runtime, anthropic OAuth auth kind, and profile→runtime derivation.

Task 8: Backend queue-time runtime validation + claim payload carries runtime

Files:

  • Edit packages/backend/convex/agentJobs.ts (insertJob at line 379, the three runtime union constants at line 21, createFromRequest/createForThread/createForThreadInternal args, claimNextInternal return).
  • Edit packages/backend/convex/agentJobsNode.ts (map job.runtime into the WorkerClaim).
  • Edit packages/backend/convex/threads.ts (createUserThread passes runtime).
  • Edit packages/backend/convex/aiProviderProfiles.ts (publicProfile exposes supportedRuntimes).
  • Edit apps/agent-worker/src/runtime/provider.ts (Claim.job.runtime already widened in Task 1 — confirm) and worker.ts runClaim (use claim.job.runtime as authoritative).
  • Edit packages/backend/tests/unit/harness.test.ts or add packages/backend/tests/unit/runtime-validation.test.ts (new).

Interfaces:

  • const runtime = v.union(v.literal('codex'), v.literal('opencode'), v.literal('claude')) (agentJobs.ts:21). The three create mutations gain runtime: v.optional(runtime).
  • insertJob resolves and validates:
const profile = await getJobProfile(ctx, ownerId, aiProviderProfileId);
const supported = runtimesForProfile(profile);
const resolvedRuntime =
  requestedRuntime ?? (settings.runtime as AgentRuntimeName | undefined) ?? supported[0];
if (!supported.includes(resolvedRuntime)) {
  throw new ConvexError(
    `Provider "${profile.name}" cannot run the "${resolvedRuntime}" runtime. Supported: ${supported.join(', ')}.`,
  );
}
// insert agentJobs with runtime: resolvedRuntime
  • claimNextInternal already returns the full job doc (schema.ts row includes runtime), so no change there; agentJobsNode.claimNextForWorker must forward job.runtime — it already returns claimed.job verbatim in WorkerClaim.job, so job.runtime flows through. Confirm the worker's Claim.job.runtime type (widened in Task 1) matches.
  • publicProfile (aiProviderProfiles.ts:45): add supportedRuntimes: runtimesForProfile(profile) so the UI can filter.

Worker change: runClaim (worker.ts:1301) currently throws for non-opencode. Replace with: resolve const runtime = (claim.job.runtime ?? 'opencode') as AgentRuntimeName; set workspace.runtime = runtime; delete the legacy openai_direct throw (queue-time guard now prevents bad runtimes). prepareAuth for the resolved adapter runs in place of the isCodexLoginProfile special-case (worker.ts:1375): await getAdapter(runtime).prepareAuth(workspace).

Steps:

  • Write failing packages/backend/tests/unit/runtime-validation.test.ts (convex-test style from harness.test.ts): create a user + GitHub spoon + an anthropic/api_key profile set default; call createForThread/createUserThread equivalent with runtime: 'codex' → expect throw /cannot run the "codex" runtime/; with runtime: 'claude' → resolves; with no runtime and an opencode_openai_login profile → job.runtime === 'codex'.
  • Run cd packages/backend && bun run test:unit → FAIL.
  • Implement the union widening, insertJob validation, thread/request arg plumbing, and publicProfile.supportedRuntimes.
  • Update worker.ts runClaim to set workspace.runtime from claim.job.runtime and call getAdapter(runtime).prepareAuth(workspace); delete the legacy throw.
  • Run cd packages/backend && bun run codegen && bun run typecheck && bun run test:unit → PASS. Run cd apps/agent-worker && bun run typecheck && bun run test:unit → PASS.
  • Commit: feat(backend): queue-time runtime/profile validation; job.runtime authoritative in worker.

Task 9: Migrate/deprecate openai_direct

Files:

  • Create packages/backend/convex/migrations.ts (new internalMutation) OR add to an existing maintenance module — grep for an existing pattern first (grep -rn "internalMutation" packages/backend/convex | grep -i migrat; there is none, so create migrations.ts).
  • Edit packages/backend/tests/unit/ (new test or extend).

Interfaces:

export const migrateOpenAiDirectRuntime = internalMutation({
  args: {},
  handler: async (ctx) => {
    let patched = 0;
    for (const job of await ctx.db.query('agentJobs').collect()) {
      if (job.runtime === 'openai_direct') { await ctx.db.patch(job._id, { runtime: 'opencode', updatedAt: Date.now() }); patched++; }
    }
    for (const s of await ctx.db.query('spoonAgentSettings').collect()) {
      if (s.runtime === 'openai_direct') { await ctx.db.patch(s._id, { runtime: 'opencode', updatedAt: Date.now() }); patched++; }
    }
    return { patched };
  },
});

Run once post-deploy via npx convex run migrations:migrateOpenAiDirectRuntime (document in the smoke/deploy notes). openai_direct stays in the schema unions as a legacy-read literal; nothing writes it.

Steps:

  • Write failing test: insert a spoonAgentSettings row with runtime: 'openai_direct' (via t.run/raw ctx), run the migration, assert it becomes 'opencode' and patched >= 1.
  • Run cd packages/backend && bun run test:unit → FAIL.
  • Implement migrations.ts.
  • Run cd packages/backend && bun run codegen && bun run typecheck && bun run test:unit → PASS.
  • Commit: feat(backend): migration to retire openai_direct runtime.

Task 10: Next UI — runtime picker (thread create + spoon agent settings) and Anthropic OAuth profile kind

Files:

  • Edit apps/next/src/components/threads/thread-workspace-form.tsx (replace the disabled Input value='OpenCode workspace' at line 142-144 with a runtime Select; pass runtime to createUserThread).
  • Edit apps/next/src/components/spoons/spoon-agent-settings-form.tsx (replace the disabled Runtime input at line 187-190 with a Select; pass runtime to spoonAgentSettings.update).
  • Edit apps/next/src/components/integrations/ai-provider-profiles-panel.tsx (add an Anthropic OAuth JSON option to providerOptions / auth-type handling at line 71-97).
  • Edit packages/backend/convex/threads.ts createUserThread args + spoonAgentSettings.ts update args + agentJobs.ts create mutations to accept runtime (done in Task 8 for agentJobs; add to createUserThread/update).

Interfaces:

  • createUserThread gains runtime: v.optional(v.union(v.literal('codex'), v.literal('opencode'), v.literal('claude'))), forwarded to createForThreadInternalinsertJob.
  • spoonAgentSettings.update runtime arg union widens to codex|opencode|claude and patch.runtime = args.runtime (drop the = 'opencode' hardcode at spoonAgentSettings.ts:126).
  • UI: both forms read selectedProfile.supportedRuntimes (new field from Task 8) and render only those SelectItems; default the value to the settings runtime or supportedRuntimes[0]; disable submit if the chosen runtime ∉ supportedRuntimes.
  • Profiles panel: add { value: 'anthropic', label: 'Anthropic OAuth (paste credentials)', authType: 'anthropic_oauth_json' } alongside the existing anthropic API-key option; when anthropic_oauth_json, render a Textarea for the pasted ~/.claude/.credentials.json (same UX as the Codex opencode_auth_json textarea).

Steps:

  • Add runtime to createUserThread and spoonAgentSettings.update args + handlers; cd packages/backend && bun run codegen && bun run typecheck.
  • Edit the three Next components: runtime Selects wired to supportedRuntimes; Anthropic OAuth option in the profiles panel.
  • Typecheck Next: cd apps/next && bun run typecheck (or the repo's Next typecheck script — confirm via apps/next/package.json). If component tests exist for these forms, extend them; otherwise cover behavior in the Task 11 manual checklist (note: no jsdom test is required by the spec for the picker, but add one if a sibling form test already exists).
  • Commit: feat(web): per-thread/per-spoon runtime picker and Anthropic OAuth provider profile.

Task 11: Add Claude Code CLI to the job image + extend the smoke test

Files:

  • Edit docker/agent-job.Dockerfile (line 50 npm install line).
  • Edit scripts/smoke-agent-container (version checks + one turn per runtime).
  • Edit README.md (the "Production agent runtime images" <details> at line 189-260: mention Claude Code CLI + the anthropic_oauth_json credential snapshot pattern, mirroring the Codex auth.json paragraph at line 255-258).

Interfaces / concrete commands:

  • Dockerfile line 50 becomes:
RUN npm install -g pnpm yarn bun@1.3.10 opencode-ai@1.17.9 @openai/codex@0.142.0 @anthropic-ai/claude-code@<PIN> \
  && npm cache clean --force

Pin <PIN> to the latest published @anthropic-ai/claude-code version at implementation time (npm view @anthropic-ai/claude-code version); record the exact version in the commit message.

  • scripts/smoke-agent-container: add to the in-container check block (after codex --version at line 30): claude --version. Then add a documented manual per-runtime turn section (commented, runnable by hand) — build the image, start a throwaway box, exec one turn per runtime:
# Manual per-runtime smoke (requires creds in your shell env):
#   docker build -f docker/agent-job.Dockerfile -t spoon-agent-job:latest .
#   docker run -d --name spoon-box-smoke --init spoon-agent-job:latest sleep infinity
#   docker exec -e OPENAI_API_KEY -w /workspace spoon-box-smoke \
#     opencode run --format json --model openai/gpt-5.5 -- 'print hello'
#   docker exec -e ANTHROPIC_API_KEY -w /workspace spoon-box-smoke \
#     claude -p 'print hello' --output-format stream-json --verbose --dangerously-skip-permissions
#   docker exec -e CODEX_HOME=/root/.codex -w /workspace spoon-box-smoke \
#     codex exec --json --dangerously-bypass-approvals-and-sandbox 'print hello'
#   docker rm -f spoon-box-smoke

Use these commands to also confirm the real flag names assumed in Tasks 4/6 (opencode run --help, claude --help) and fix the adapters if they differ.

Steps:

  • npm view @anthropic-ai/claude-code version → choose the pin. Edit the Dockerfile.
  • Build the image and run SPOON_AGENT_JOB_IMAGE=spoon-agent-job:latest ./scripts/smoke-agent-container → all --version checks pass (claude --version included).
  • Run the manual per-runtime turns; reconcile adapter flags with reality (patch Task 4/6 code if needed, keeping unit tests green).
  • Update README.md.
  • Commit: build(agent-job): pin Claude Code CLI; smoke one turn per runtime.

Task 12: Secrets hygiene — 0600 env files, deleted on stop, re-materialized per run

Files:

  • Edit apps/agent-worker/src/worker.ts (materializeEnvFile at line 1167; stopWorkspace/openWorkspacePullRequest teardown; call materializeEnvFile per run).
  • Create apps/agent-worker/tests/unit/materialize-env.test.ts (new).

Interfaces:

  • materializeEnvFile (worker.ts:1167): write with { mode: 0o600 } (currently no mode). Extract the file path resolution + write into a pure-ish helper writeMaterializedEnv(repoDir, envFilePath, secrets): Promise<string> returning the absolute path, so it is unit-testable with mkdtemp.
  • Track the materialized path on the workspace (workspace.materializedEnvPath?: string).
  • On stopWorkspace (worker.ts:1866) and openWorkspacePullRequest (worker.ts:1851) and the runClaim catch (worker.ts:1410) teardown: if workspace.materializedEnvPath exists, await rm(workspace.materializedEnvPath, { force: true }).
  • Re-materialize per run: call materializeEnvFile(workspace) at the top of sendWorkspaceMessage (worker.ts:1624) when claim.job.materializeEnvFile is set, before the adapter turn — so the env file is fresh each turn and can be deleted between turns. Keep the existing ensureNoEnvFilesStaged staging guard (worker.ts:1269) unchanged.

Steps:

  • Write failing materialize-env.test.ts: mkdtemp a repo dir, call writeMaterializedEnv(dir, '.env.local', [{ name: 'A', value: 'x' }]), assert the file exists, (await stat(path)).mode & 0o777 === 0o600, and content is A="x"\n (matches worker.ts:1172-1174 formatting). Mirror the codex-runtime.test.ts mode() helper + afterEach cleanup.
  • Run cd apps/agent-worker && bun run test:unit → FAIL.
  • Implement writeMaterializedEnv (0600), wire deletion on stop/PR/failure, re-materialize per run.
  • Run bun run test:unit → PASS. bun run typecheck → PASS.
  • Commit: fix(worker): materialize env files 0600, re-materialize per run, delete on stop.

Self-review against the Phase 2 spec

  • Adapter interface, all exec into the box, process-group kill → Tasks 12 (buildMarkedCommand = setsid + marker; killBoxProcessesByMarker = in-box pgrep+kill -TERM/-KILL of the group; never the local execa client).
  • Codex refactored behind the interface → Task 3.
  • OpenCode inside the box, per-job path deleted → Tasks 45 (opencode run --format json in the box; startWorkspaceContainer/spoon-agent-job-*/containerAccess host_port|network/dead helpers all removed).
  • Claude Code adapter + new provider kind + pinned CLI + prepareAuth → Tasks 6, 7, 10, 11 (anthropic_oauth_json snapshot mirrors Codex auth.json; API-key path uses ANTHROPIC_API_KEY).
  • Runtime picker + profiles declare supported runtimes + queue-time validation replacing run-time openai_direct discovery + migrate/deprecate → Tasks 710.
  • Secrets hygiene (0600, delete on stop, re-materialize; staging guard stays) → Task 12.
  • Testing approach → adapter-level fixture stream tests (injectable execStream), normalizer fixtures matching the existing style, convex-test backend tests, extended manual smoke.