docs: multi-phase implementation plan (63 tasks across 5 phases)
Phase 1 Stabilize (13), Phase 2 Runtime unification (12), Phase 3 Dev-box surface (11), Phase 4 Sync correctness (12), Phase 5 Notifications & polish (15), plus index.
This commit is contained in:
Gabriel Brown
@@ -0,0 +1,545 @@
|
||||
# Spoon Phase 2 — Runtime Unification: Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Collapse the three agent runtimes (Codex, OpenCode, Claude Code) behind one `AgentRuntime` adapter interface in the worker, run all three *inside* the per-user `spoon-box-{username}` container (no per-job containers), and let users pick the runtime per thread / per spoon with queue-time validation against their AI provider profile.
|
||||
|
||||
**Architecture:** The worker (`apps/agent-worker`) polls Convex, claims a job, acquires the user's persistent box (Phase 1), clones `~/Code/{spoon}/{branch}` into the mounted home, then runs agent turns by `docker exec`-ing a CLI into the box. Today dispatch is implicit (Codex if the profile is a ChatGPT-login profile, otherwise a *separate* `opencode serve` container). Phase 2 makes `job.runtime` (`codex|opencode|claude`) authoritative: a registry maps it to an adapter; every adapter execs into the box via `streamExecInContainer`, tags its process group with a unique marker, and kills that group in-box on timeout/abort. The OpenCode server-in-a-side-container path is deleted. The Codex `auth.json` encrypted-snapshot pattern is generalized to Claude OAuth credentials.
|
||||
|
||||
**Tech Stack:** TypeScript, Bun, execa (docker/podman CLI), Convex (`packages/backend/convex`), Vitest (worker unit tests + convex-test backend tests), Next.js 16 (`apps/next`), Fedora 41 job image (`docker/agent-job.Dockerfile`) shipping `opencode`, `@openai/codex`, and (new) `@anthropic-ai/claude-code`.
|
||||
|
||||
**Depends on:** Phase 1 (box lifecycle: per-username mutex, idempotent acquire/release handles, `--init` on the box, heartbeat + `cancelRequested` teardown, startup reconcile of `spoon-box-*`). This plan assumes `acquireUserBox`/`releaseUserBox` are already concurrency-safe and the box runs with `--init` so orphaned in-box children are reaped. Where Phase-1 handle semantics are referenced, use whatever `acquireUserBox` returns after Phase 1; if Phase 1 kept the `string` box-name return used in `worker.ts:1330`, keep using it.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- **Worker unit tests** live in `apps/agent-worker/tests/unit/*.test.ts`; run `cd apps/agent-worker && bun run test:unit` (`vitest run --project unit`). **Worker typecheck:** `cd apps/agent-worker && bun run typecheck`.
|
||||
- **Backend tests** live in `packages/backend/tests/unit/*.test.ts` using `convex-test` (see `packages/backend/tests/unit/harness.test.ts` for the exact fixture style: `convexTest(schema, modules)`, `authed(t, userId)`, `spoonInput`/`githubSpoonInput`). Run `cd packages/backend && bun run test:unit`. **Always `cd packages/backend && bun run codegen` before `bun run typecheck`** (codegen regenerates `_generated/` after schema/function changes).
|
||||
- Match the existing fixture/test style: `expect(normalizeX(...)).toContainEqual({ kind: ... })` for event normalizers (see `apps/agent-worker/tests/unit/agent-events.test.ts`); `mkdtemp` + `afterEach` cleanup for filesystem tests (see `apps/agent-worker/tests/unit/codex-runtime.test.ts`).
|
||||
- **Conventional commits; one commit per task.** Run the relevant `test:unit` + `typecheck` (and `codegen` for backend) before committing. Do not commit with a failing suite.
|
||||
- **Three runtimes: `codex`, `opencode`, `claude` — all `docker exec` into `spoon-box-{username}`. No per-job containers.** Never leave a `spoon-agent-job-*` code path behind.
|
||||
- A real turn against a live CLI cannot run in unit tests. Adapters therefore take an **injectable exec function** so `runTurn` can be driven with fixture stream lines; live behavior is covered by the extended `scripts/smoke-agent-container` manual step. Event parsing is covered by pure normalizer fixtures.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Define the `AgentRuntime` adapter interface, shared types, and registry
|
||||
|
||||
**Files:**
|
||||
- Create `apps/agent-worker/src/runtime/agent-runtime.ts` (new).
|
||||
- Create `apps/agent-worker/src/runtime/provider.ts` (new — pure helpers moved out of `worker.ts` so adapters don't import `worker.ts` at runtime).
|
||||
- Edit `apps/agent-worker/src/worker.ts` (extract the `Claim` type and pure helpers; make `ActiveWorkspace` extend the shared workspace shape; add `runtime`/`turnMarker` fields).
|
||||
- Create `apps/agent-worker/tests/unit/agent-runtime.test.ts` (new).
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
*Produces* (`runtime/agent-runtime.ts`):
|
||||
```ts
|
||||
import type { NormalizedAgentEvent } from '../agent-events';
|
||||
import type { streamExecInContainer } from './docker';
|
||||
import type { AdapterWorkspace } from './provider';
|
||||
|
||||
export type AgentRuntimeName = 'codex' | 'opencode' | 'claude';
|
||||
|
||||
export type ExecStreamFn = typeof streamExecInContainer;
|
||||
|
||||
export type TurnResult = {
|
||||
// Assistant text the adapter captured out-of-band (Codex --output-last-message
|
||||
// file, Claude `result` event). Empty/undefined when everything streamed via onEvent.
|
||||
finalMessage?: string;
|
||||
// Runtime session id to persist for continuity (resume/--session/--resume).
|
||||
sessionId?: string;
|
||||
// Non-empty when the runtime reported a hard failure the caller must surface.
|
||||
error?: string;
|
||||
};
|
||||
|
||||
export type AgentRuntime = {
|
||||
readonly name: AgentRuntimeName;
|
||||
prepareAuth(workspace: AdapterWorkspace): Promise<void>;
|
||||
runTurn(
|
||||
workspace: AdapterWorkspace,
|
||||
prompt: string,
|
||||
onEvent: (event: NormalizedAgentEvent) => Promise<void>,
|
||||
): Promise<TurnResult>;
|
||||
abort(workspace: AdapterWorkspace): Promise<void>;
|
||||
};
|
||||
|
||||
export type AdapterFactory = (deps?: { execStream?: ExecStreamFn }) => AgentRuntime;
|
||||
```
|
||||
|
||||
*Produces* (`runtime/provider.ts`) — `AdapterWorkspace` is the subset of `ActiveWorkspace` adapters read, plus the pure helpers currently in `worker.ts:47-95, 350-446`:
|
||||
```ts
|
||||
export type Claim = { /* moved verbatim from worker.ts:47-95, but runtime widened */ };
|
||||
// job.runtime union becomes: runtime?: 'codex' | 'opencode' | 'claude';
|
||||
|
||||
export type AdapterWorkspace = {
|
||||
claim: Claim;
|
||||
workdir: string;
|
||||
homeDir: string;
|
||||
username: string;
|
||||
containerHome: string; // e.g. /home/{username}
|
||||
containerRepo: string; // e.g. /home/{username}/Code/{spoon}/{branch}
|
||||
repoDir: string; // host path to the checkout
|
||||
boxName: string; // spoon-box-{username}
|
||||
redact: (value: string) => string;
|
||||
runtime: AgentRuntimeName;
|
||||
codexSessionId?: string;
|
||||
opencodeSessionId?: string;
|
||||
claudeSessionId?: string;
|
||||
turnMarker?: string; // set per turn; used for in-box process-group kill
|
||||
};
|
||||
|
||||
export const isCodexLoginProfile: (claim: Claim) => boolean; // moved from worker.ts:350
|
||||
export const collectJsonStringValues: (value?: string) => string[]; // moved from worker.ts:354
|
||||
export const providerEnvironment: (claim: Claim, workspaceRoot?: string) => Record<string, string>; // moved from worker.ts:379
|
||||
export const opencodeModel: (claim: Claim) => string; // moved from worker.ts:425
|
||||
export const codexModel: (claim: Claim) => string; // moved from worker.ts:440
|
||||
export const codexModelArgs: (claim: Claim) => string[]; // moved from worker.ts:445
|
||||
```
|
||||
|
||||
*Consumes:* `NormalizedAgentEvent` from `agent-events.ts`; `streamExecInContainer` from `runtime/docker.ts`.
|
||||
|
||||
**Registry** (`runtime/agent-runtime.ts`): a lazy map. Adapters register in later tasks; until then `getAdapter` throws for unknown names.
|
||||
```ts
|
||||
const factories = new Map<AgentRuntimeName, AdapterFactory>();
|
||||
export const registerAdapter = (name: AgentRuntimeName, factory: AdapterFactory): void => { factories.set(name, factory); };
|
||||
export const getAdapter = (name: AgentRuntimeName): AgentRuntime => {
|
||||
const factory = factories.get(name);
|
||||
if (!factory) throw new Error(`No agent runtime adapter registered for "${name}".`);
|
||||
return factory();
|
||||
};
|
||||
```
|
||||
|
||||
**Steps:**
|
||||
- [ ] Write a failing test `apps/agent-worker/tests/unit/agent-runtime.test.ts`: `registerAdapter('codex', () => fake)` then `expect(getAdapter('codex').name).toBe('codex')`, and `expect(() => getAdapter('opencode')).toThrow(/No agent runtime adapter/)`. Also assert `isCodexLoginProfile`/`opencodeModel` re-exported from `runtime/provider.ts` behave as before (copy one existing expectation, e.g. `opencodeModel({ aiProviderProfile: { provider: 'anthropic', model: 'claude-x', ... } } as Claim)` → `'anthropic/claude-x'`).
|
||||
- [ ] Run `cd apps/agent-worker && bun run test:unit` → FAIL (module/exports missing).
|
||||
- [ ] Create `runtime/provider.ts`: move `Claim` (worker.ts:47-95), `isCodexLoginProfile`, `collectJsonStringValues`, `providerEnvironment`, `opencodeModel`, `codexModel`, `codexModelArgs` (worker.ts:350-446) verbatim; widen `Claim['job']['runtime']` to `'codex' | 'opencode' | 'claude'`; add and export `AdapterWorkspace`.
|
||||
- [ ] Create `runtime/agent-runtime.ts` with the interface, `TurnResult`, `ExecStreamFn`, `AdapterFactory`, and the registry above.
|
||||
- [ ] Edit `worker.ts`: import `Claim`, the helpers, and `AdapterWorkspace` from `runtime/provider.ts` (delete the now-moved definitions); change `ActiveWorkspace` (worker.ts:97) to `type ActiveWorkspace = AdapterWorkspace & { githubToken: string; runtimeMode?: ...; agentTurnActive?: boolean; resolveTurn?: () => void; lastRecordedDiffSignature?: string; codexTurnError?: string; ... }`; set `runtime` when the workspace is built in `runClaim` (worker.ts:1348 — temporarily `runtime: 'codex'` placeholder; real resolution lands in Task 3).
|
||||
- [ ] Run `cd apps/agent-worker && bun run test:unit` → PASS. Run `bun run typecheck` → PASS.
|
||||
- [ ] Commit: `refactor(worker): extract AgentRuntime interface, shared provider helpers, and adapter registry`.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: In-box process-group kill + marked-command helpers
|
||||
|
||||
**Files:**
|
||||
- Edit `apps/agent-worker/src/runtime/docker.ts` (add two exported helpers near `streamExecInContainer` at line 385).
|
||||
- Create `apps/agent-worker/tests/unit/box-process-kill.test.ts` (new).
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
*Produces* (`runtime/docker.ts`):
|
||||
```ts
|
||||
// Wraps a CLI argv so the process runs as a new session/process-group leader whose
|
||||
// bash parent carries the marker in its argv (matchable by `pgrep -f`). We do NOT
|
||||
// `exec` (bash must survive so the marker stays visible); stdout/stderr fds are
|
||||
// inherited by the CLI so streaming still works.
|
||||
export const buildMarkedCommand = (marker: string, command: string[]): string[] => {
|
||||
const script = `# ${marker}\nexec 0</dev/null\n${command.map(shellQuote).join(' ')}`;
|
||||
return ['setsid', 'bash', '-lc', script];
|
||||
};
|
||||
|
||||
// Kills every process group whose bash parent matches the marker, TERM then KILL.
|
||||
export const killBoxProcessesByMarker = async (args: {
|
||||
containerName: string;
|
||||
marker: string;
|
||||
}): Promise<void> => {
|
||||
const script = [
|
||||
`pids=$(pgrep -f ${shellQuote(args.marker)} || true)`,
|
||||
`for pid in $pids; do kill -TERM -"$pid" 2>/dev/null || true; done`,
|
||||
`sleep 2`,
|
||||
`for pid in $pids; do kill -KILL -"$pid" 2>/dev/null || true; done`,
|
||||
].join('\n');
|
||||
await execa(containerRuntime(), ['exec', args.containerName, 'bash', '-lc', script], {
|
||||
reject: false, stdin: 'ignore',
|
||||
});
|
||||
};
|
||||
```
|
||||
Add a local `const shellQuote = (v: string) => `'${v.replaceAll("'", "'\\''")}'`;` (mirror `worker.ts:1049`).
|
||||
|
||||
*Consumes:* `execa`, `containerRuntime()` (already in file).
|
||||
|
||||
**Steps:**
|
||||
- [ ] Write failing `apps/agent-worker/tests/unit/box-process-kill.test.ts`: import `buildMarkedCommand`. Assert `buildMarkedCommand('spoon-turn-abc', ['codex', 'exec', '--json', 'hi'])` returns `['setsid', 'bash', '-lc', expect.stringContaining('# spoon-turn-abc')]` and the script's last line contains `codex exec --json 'hi'`. (Do not unit-test `killBoxProcessesByMarker` end-to-end — it shells out; assert only the argv via a spy if you refactor it to a pure `buildKillScript(marker)` helper. **Recommended:** extract `export const buildKillScript = (marker: string): string` and test it contains `pgrep -f 'spoon-turn-abc'`, `kill -TERM -"$pid"`, and `kill -KILL -"$pid"`.)
|
||||
- [ ] Run `bun run test:unit` → FAIL.
|
||||
- [ ] Implement `buildMarkedCommand`, `buildKillScript`, and `killBoxProcessesByMarker` in `docker.ts`.
|
||||
- [ ] Run `bun run test:unit` → PASS. `bun run typecheck` → PASS.
|
||||
- [ ] Commit: `feat(worker): in-box process-group marker + kill helpers for agent turns`.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: CodexAdapter — refactor `runCodexTurn` behind the interface, dispatch codex through the registry
|
||||
|
||||
**Files:**
|
||||
- Create `apps/agent-worker/src/runtime/codex-adapter.ts` (new).
|
||||
- Edit `apps/agent-worker/src/worker.ts` (replace `runCodexTurn` call path in `sendWorkspaceMessage`; resolve `workspace.runtime`; route abort).
|
||||
- Create `apps/agent-worker/tests/unit/codex-adapter.test.ts` (new).
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
*Produces* (`runtime/codex-adapter.ts`):
|
||||
```ts
|
||||
export const createCodexAdapter: AdapterFactory = (deps) => {
|
||||
const execStream = deps?.execStream ?? streamExecInContainer;
|
||||
return {
|
||||
name: 'codex',
|
||||
prepareAuth, // moves prepareCodexAuth (worker.ts:459) + prepareCodexWorkspaceFiles + writeJsonFile here
|
||||
runTurn, // moves runCodexTurn body (worker.ts:712-853), using buildMarkedCommand + execStream
|
||||
abort, // killBoxProcessesByMarker({ containerName: ws.boxName, marker: ws.turnMarker })
|
||||
};
|
||||
};
|
||||
```
|
||||
`runTurn(ws, prompt, onEvent)`:
|
||||
1. `ws.runtime = 'codex'`; `ws.turnMarker = 'spoon-turn-' + randomUUID()`; `ws.codexSessionId` continuity preserved.
|
||||
2. Build the same codex argv as `worker.ts:737-761` (`codex exec [resume <id>] --json ...codexModelArgs --dangerously-bypass-approvals-and-sandbox --output-last-message <containerPath> [--cd <repo>] <prompt>`), then `const command = buildMarkedCommand(ws.turnMarker, codexArgv)`.
|
||||
3. `await execStream({ containerName: ws.boxName, containerCwd: ws.containerRepo, command, environment: { ...providerEnvironment(ws.claim, ws.containerHome), ...secretEnv }, redact: ws.redact, timeoutMs: env.jobTimeoutMs, onStdoutLine: (line) => forEach normalizeCodexJsonLine(line) → onEvent(event), onStderrLine: (line) => onEvent({ kind: 'status', status: line }) if not noise })`.
|
||||
4. On timeout: register `setTimeout(() => void killBoxProcessesByMarker(...), env.jobTimeoutMs)` and clear it when execStream resolves (execa's own `timeout` only kills the local client — the marker kill is what stops the in-box process).
|
||||
5. Read `--output-last-message` file (worker.ts:811-843) → `finalMessage`. Capture `error` events into a local var → `TurnResult.error`. Return `{ finalMessage, sessionId: capturedSessionId, error }`.
|
||||
|
||||
*Consumes:* `normalizeCodexJsonLine` (`agent-events.ts`), `providerEnvironment`/`codexModelArgs`/`isCodexLoginProfile` (`runtime/provider.ts`), `buildMarkedCommand`/`killBoxProcessesByMarker`/`streamExecInContainer` (`runtime/docker.ts`), `prepareCodexWorkspaceFiles` (`codex-runtime.ts`), `env`.
|
||||
|
||||
*Worker dispatch change* (`sendWorkspaceMessage`, worker.ts:1624): replace the `isCodexLoginProfile(claim) ? runCodexTurn : env.runtime==='docker' ? runOpenCodeTurn : localRun` branching with:
|
||||
```ts
|
||||
const adapter = getAdapter(workspace.runtime);
|
||||
const onEvent = (event: NormalizedAgentEvent) => handleAgentEvent({ workspace, event, assistantMessageId, assistantContent });
|
||||
const result = await adapter.runTurn(workspace, prompt, onEvent);
|
||||
if (result.sessionId) await persistRuntimeSession(workspace, result.sessionId); // generic setter, replaces setCodexSessionId call site
|
||||
if (!assistantContent.value.trim() && result.finalMessage) { assistantContent.value = truncate(workspace.redact(result.finalMessage), 40_000); await updateMessage(...); }
|
||||
if (!assistantContent.value.trim() && result.error) throw new Error(`${workspace.runtime} failed:\n${result.error}`);
|
||||
```
|
||||
Resolve `workspace.runtime` in `runClaim` (worker.ts:1348): for now set `runtime: isCodexLoginProfile(claim) ? 'codex' : 'opencode'` so behavior is byte-identical to today (job.runtime becomes authoritative in Task 8). Route `abortWorkspaceAgent` (worker.ts:1562) through `getAdapter(workspace.runtime).abort(workspace)`.
|
||||
|
||||
**Steps:**
|
||||
- [ ] Write failing `apps/agent-worker/tests/unit/codex-adapter.test.ts`: build the adapter with a fake `execStream` that calls `onStdoutLine` with fixture lines (reuse the exact JSON shapes from `agent-events.test.ts:110-172`, e.g. `{"type":"item.completed","item":{"id":"item-1","type":"agent_message","text":"done"}}` and `{"type":"turn.completed"}`) then resolves `{ exitCode: 0, output: '' }`. Assert `onEvent` received `{ kind: 'assistant_delta', content: 'done\n\n', externalMessageId: 'item-1' }` and `{ kind: 'assistant_completed' }`, and that `runTurn` returns a `TurnResult`. Add a second case: fake execStream that emits `{"type":"turn.failed","error":{"message":"boom"}}` and returns exitCode 0 → `result.error` contains `boom`. Assert `buildMarkedCommand` was used (spy the fake to capture `args.command[0] === 'setsid'`).
|
||||
- [ ] Run `bun run test:unit` → FAIL.
|
||||
- [ ] Create `runtime/codex-adapter.ts`; move Codex logic out of `worker.ts` (`prepareCodexAuth`, `runCodexTurn`, `writeJsonFile`, `readCodexTurnError`). Register it: add `registerAdapter('codex', createCodexAdapter)` in a new `runtime/register.ts` imported once from `worker.ts` top-level.
|
||||
- [ ] Rewire `sendWorkspaceMessage` and `abortWorkspaceAgent` per above; add `persistRuntimeSession` (writes `setCodexSessionId`/`opencodeSessionId`/`claudeSessionId` based on `workspace.runtime`). Keep the maintenance-decision parse, diff artifact, `recordChangedFiles` tail (worker.ts:1731-1758) unchanged.
|
||||
- [ ] Run `bun run test:unit` (all existing + new) → PASS. `bun run typecheck` → PASS.
|
||||
- [ ] Commit: `refactor(worker): move Codex turn logic into CodexAdapter behind AgentRuntime`.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: OpenCodeAdapter — run `opencode run --format json` inside the box
|
||||
|
||||
**Files:**
|
||||
- Create `apps/agent-worker/src/runtime/opencode-adapter.ts` (new).
|
||||
- Edit `apps/agent-worker/src/agent-events.ts` (add `normalizeOpenCodeRunLine` for `opencode run --format json` line output; may delegate to `normalizeOpenCodeEvent`).
|
||||
- Edit `apps/agent-worker/src/runtime/register.ts` (register `opencode`).
|
||||
- Create `apps/agent-worker/tests/unit/opencode-adapter.test.ts` (new).
|
||||
- Edit `apps/agent-worker/tests/unit/agent-events.test.ts` (add `normalizeOpenCodeRunLine` cases).
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
*Produces* (`runtime/opencode-adapter.ts`):
|
||||
```ts
|
||||
export const createOpenCodeAdapter: AdapterFactory = (deps) => {
|
||||
const execStream = deps?.execStream ?? streamExecInContainer;
|
||||
return {
|
||||
name: 'opencode',
|
||||
prepareAuth: async (ws) => { /* opencode reads ANTHROPIC_API_KEY/OPENAI_API_KEY from env; for opencode_auth_json profiles, write ~/.local/share/opencode/auth.json 0600 (reuse logic from worker.ts:472-480) */ },
|
||||
runTurn,
|
||||
abort: (ws) => killBoxProcessesByMarker({ containerName: ws.boxName, marker: ws.turnMarker ?? '' }),
|
||||
};
|
||||
};
|
||||
```
|
||||
`runTurn(ws, prompt, onEvent)`:
|
||||
1. `ws.runtime = 'opencode'`; `ws.turnMarker = 'spoon-turn-' + randomUUID()`.
|
||||
2. argv: `['opencode', 'run', '--format', 'json', '--model', opencodeModel(ws.claim), ...(ws.opencodeSessionId ? ['--session', ws.opencodeSessionId] : []), '--', prompt]`. **Verify the continuity flag** against `opencode run --help` during the smoke step (Task 11); if `--session` is unsupported, drop it and note that OpenCode turns are stateless (acceptable — Codex/Claude carry continuity).
|
||||
3. `command = buildMarkedCommand(ws.turnMarker, argv)`; `execStream({ containerName: ws.boxName, containerCwd: ws.containerRepo, command, environment: { ...providerEnvironment(ws.claim), ...secretEnv }, redact, timeoutMs, onStdoutLine: (line) => normalizeOpenCodeRunLine(line).forEach(onEvent), onStderrLine })`.
|
||||
4. Return `{ finalMessage, sessionId: capturedSessionId, error }` (session id parsed from the run output if present).
|
||||
|
||||
*Produces* (`agent-events.ts`): `export const normalizeOpenCodeRunLine = (line: string): NormalizedAgentEvent[]` — `JSON.parse` each line; if it matches the SSE-style `{ type, properties }` shape, delegate to `normalizeOpenCodeEvent(parsed)`; otherwise map `opencode run`'s result shape (capture `sessionID`, final text). Unparseable lines → `[{ kind: 'status', status: line }]`.
|
||||
|
||||
*Consumes:* `opencodeModel`/`providerEnvironment` (`runtime/provider.ts`), `buildMarkedCommand`/`killBoxProcessesByMarker`/`streamExecInContainer` (`runtime/docker.ts`).
|
||||
|
||||
**Steps:**
|
||||
- [ ] Write failing `agent-events.test.ts` case: `normalizeOpenCodeRunLine(JSON.stringify({ type: 'message.part.delta', properties: { part: { text: 'hi' }, messageID: 'm1' } }))` → `toContainEqual({ kind: 'assistant_delta', content: 'hi', externalMessageId: 'm1' })`; and a non-JSON line → `{ kind: 'status', status: '<line>' }`.
|
||||
- [ ] Write failing `opencode-adapter.test.ts`: fake `execStream` emits the delta line + a final line, returns `{ exitCode: 0, output: '' }`; assert `onEvent` got the delta, `runTurn` resolves, and the captured `args.command[0] === 'setsid'` and `args.command[3]` (script) contains `opencode run --format json`.
|
||||
- [ ] Run `bun run test:unit` → FAIL.
|
||||
- [ ] Implement `normalizeOpenCodeRunLine`, `createOpenCodeAdapter`, and register it.
|
||||
- [ ] Run `bun run test:unit` → PASS. `bun run typecheck` → PASS.
|
||||
- [ ] Commit: `feat(worker): OpenCodeAdapter runs opencode in the per-user box (no side container)`.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Delete the per-job OpenCode container path and dead docker helpers
|
||||
|
||||
**Files:**
|
||||
- Edit `apps/agent-worker/src/worker.ts` (remove `ensureOpenCodeSession`, `runOpenCodeTurn`, `workspaceContainerName`, `startWorkspaceContainer` import + usage, `stopWorkspaceContainer` calls, `containerName`/`containerId`/`opencodePassword`/`opencodeSession` workspace fields, the OpenCode-server branches of `abortWorkspaceAgent`/`replyToInteraction`/`getWorkspaceAgentStatus`; retarget health/cleanup to `spoon-box-`).
|
||||
- Delete `apps/agent-worker/src/opencode-session.ts`.
|
||||
- Edit `apps/agent-worker/src/runtime/docker.ts` (delete `runInJobContainer`, `streamInJobContainer`, `execInWorkspaceContainer`, `inspectWorkspaceContainer`, `startWorkspaceContainer`, `getPublishedPort`; keep `ensureUserContainer`, `streamExecInContainer`, `runExecInContainer`, `stopWorkspaceContainer`, `listWorkspaceContainerNames`, `jobWorkspaceVolumeSpec`, `normalizeRunResult`, `ensureJobImagePulled`).
|
||||
- Edit `apps/agent-worker/src/env.ts` (remove `containerAccess`; remove now-dead `terminalImage`/`terminalIdleMs`/`maxConcurrentJobs` only if not read anywhere else — grep first).
|
||||
- Edit `packages/backend/convex/*` — none here.
|
||||
- Edit `apps/agent-worker/tests/unit/*` as needed (no test imports the deleted symbols; if any do, update).
|
||||
|
||||
**Interfaces:** *Consumes:* now only box-scoped docker helpers. `getWorkerHealth` (worker.ts:1880) enumerates `listWorkspaceContainerNames('spoon-box-')`; `cleanupOrphanedWorkspaces` (worker.ts:1911) enumerates `spoon-box-` for the container list (do **not** touch the `homes/` workdir removal here — that is Phase 1's layout-aware rewrite; leave whatever Phase 1 produced).
|
||||
|
||||
**Steps:**
|
||||
- [ ] `grep -rn "spoon-agent-job\|startWorkspaceContainer\|ensureOpenCodeSession\|opencode-session\|containerAccess\|host_port\|runInJobContainer\|streamInJobContainer\|execInWorkspaceContainer\|inspectWorkspaceContainer" apps/agent-worker/src` → build the deletion checklist from the hits.
|
||||
- [ ] `grep -rn "env.terminalImage\|env.terminalIdleMs\|env.maxConcurrentJobs" apps/agent-worker/src` → only remove the env var if zero hits remain after Phase 1 (`maxConcurrentJobs` is surfaced in `getWorkerHealth` at worker.ts:1903 — drop that line too, or keep the field; note it in the commit).
|
||||
- [ ] Remove the symbols above. In `abortWorkspaceAgent`/`replyToInteraction`/`getWorkspaceAgentStatus`, drop the `opencodeSession` branches: `abortWorkspaceAgent` → `await getAdapter(workspace.runtime).abort(workspace)`; `replyToInteraction` → `throw new Error('Interactive replies are not supported by exec-based runtimes.')` (no runtime generates interaction requests after this refactor); `getWorkspaceAgentStatus` → return `{ runtimeMode: workspace.runtime, codexSessionId, opencodeSessionId, claudeSessionId, active }`.
|
||||
- [ ] In `openWorkspacePullRequest` (worker.ts:1852-1855) and `stopWorkspace` (worker.ts:1869-1872), delete the `workspace.opencodeSession?.close()` and `stopWorkspaceContainer(workspace.containerName)` lines (box lifecycle is now purely `releaseUserBox`).
|
||||
- [ ] Delete `apps/agent-worker/src/opencode-session.ts` and its imports (worker.ts:31-35).
|
||||
- [ ] Run `cd apps/agent-worker && bun run typecheck` → PASS (fix any dangling refs). Run `bun run test:unit` → PASS.
|
||||
- [ ] Commit: `refactor(worker): delete per-job OpenCode container path, host_port split, and dead docker helpers`.
|
||||
|
||||
---
|
||||
|
||||
## Task 6: ClaudeCodeAdapter — `claude -p --output-format stream-json` in the box
|
||||
|
||||
**Files:**
|
||||
- Create `apps/agent-worker/src/runtime/claude-adapter.ts` (new).
|
||||
- Edit `apps/agent-worker/src/agent-events.ts` (add `normalizeClaudeJsonLine`).
|
||||
- Edit `apps/agent-worker/src/runtime/register.ts` (register `claude`).
|
||||
- Create `apps/agent-worker/tests/unit/claude-adapter.test.ts` (new).
|
||||
- Edit `apps/agent-worker/tests/unit/agent-events.test.ts` (add Claude normalizer cases).
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
*Produces* (`agent-events.ts`): `export const normalizeClaudeJsonLine = (line: string): NormalizedAgentEvent[]`. Claude Code `stream-json` emits one JSON object per line:
|
||||
- `{"type":"system","subtype":"init","session_id":"abc",...}` → `{ kind: 'session', sessionId: 'abc' }`.
|
||||
- `{"type":"assistant","message":{"content":[{"type":"text","text":"..."}]}}` → `{ kind: 'assistant_delta', content: '...' }` (concatenate text blocks); `{"type":"content_block_delta",...}` if present → `assistant_delta`.
|
||||
- `{"type":"assistant","message":{"content":[{"type":"tool_use","name":"Bash","input":{...}}]}}` → `{ kind: 'tool_started', name, input: JSON.stringify(input) }`.
|
||||
- `{"type":"user","message":{"content":[{"type":"tool_result",...}]}}` → `{ kind: 'tool_completed', name: 'tool', output }`.
|
||||
- `{"type":"result","subtype":"success","result":"final text","session_id":"abc"}` → `{ kind: 'assistant_completed', content: 'final text' }` (+ a `session` event if the id is new).
|
||||
- `{"type":"result","subtype":"error_max_turns"|"error_during_execution",...}` → `{ kind: 'error', message }`.
|
||||
- Unparseable → `[{ kind: 'status', status: line }]`.
|
||||
|
||||
*Produces* (`runtime/claude-adapter.ts`):
|
||||
```ts
|
||||
export const createClaudeAdapter: AdapterFactory = (deps) => {
|
||||
const execStream = deps?.execStream ?? streamExecInContainer;
|
||||
return {
|
||||
name: 'claude',
|
||||
prepareAuth, // API-key kind: nothing on disk (ANTHROPIC_API_KEY comes from providerEnvironment). OAuth-json kind: write <homeDir>/.claude/.credentials.json 0600 from claim.aiProviderProfile.secret (reuse writeJsonFile).
|
||||
runTurn,
|
||||
abort: (ws) => killBoxProcessesByMarker({ containerName: ws.boxName, marker: ws.turnMarker ?? '' }),
|
||||
};
|
||||
};
|
||||
```
|
||||
`runTurn(ws, prompt, onEvent)`:
|
||||
1. `ws.runtime = 'claude'`; `ws.turnMarker = 'spoon-turn-' + randomUUID()`.
|
||||
2. argv: `['claude', '-p', prompt, '--output-format', 'stream-json', '--verbose', '--dangerously-skip-permissions', '--model', claudeModel(ws.claim), ...(ws.claudeSessionId ? ['--resume', ws.claudeSessionId] : [])]` (cwd = `containerRepo`). `--verbose` is required for `stream-json`. **Confirm exact flags** in the smoke step against the pinned CLI (`claude --help`).
|
||||
3. `command = buildMarkedCommand(ws.turnMarker, argv)`; `execStream({ containerName: ws.boxName, containerCwd: ws.containerRepo, command, environment: { ...claudeEnv(ws.claim, ws.containerHome), ...secretEnv }, redact, timeoutMs, onStdoutLine: (line) => normalizeClaudeJsonLine(line).forEach(onEvent), onStderrLine })`. `claudeEnv`: for API-key kind `{ ANTHROPIC_API_KEY: secret, HOME: ws.containerHome }`; for OAuth-json kind `{ HOME: ws.containerHome }` (CLI reads `~/.claude/.credentials.json`).
|
||||
4. Return `{ finalMessage, sessionId, error }` (`finalMessage` from the `result` event; `sessionId` from `system:init`/`result`).
|
||||
|
||||
*Consumes:* `normalizeClaudeJsonLine` (`agent-events.ts`), `providerEnvironment` (`runtime/provider.ts`), box helpers (`runtime/docker.ts`), `writeJsonFile` (move to `runtime/provider.ts` or a small `runtime/auth.ts` shared by Codex + Claude adapters).
|
||||
|
||||
**Steps:**
|
||||
- [ ] Write failing `agent-events.test.ts` Claude cases: `system:init` → session; `assistant` text block → `assistant_delta`; `result:success` → `assistant_completed` with content; `result:error_max_turns` → `error`.
|
||||
- [ ] Write failing `claude-adapter.test.ts`: fake `execStream` emits `system:init` + `assistant` text + `result:success` lines, returns `{ exitCode: 0, output: '' }`; assert `onEvent` received the mapped events, `runTurn` returns `{ finalMessage: 'final text', sessionId: 'abc' }`, and `args.command[0] === 'setsid'` with the script containing `claude -p`.
|
||||
- [ ] Run `bun run test:unit` → FAIL.
|
||||
- [ ] Implement `normalizeClaudeJsonLine`, `createClaudeAdapter`, register it. Move `writeJsonFile` to a shared `runtime/auth.ts`.
|
||||
- [ ] Run `bun run test:unit` → PASS. `bun run typecheck` → PASS.
|
||||
- [ ] Commit: `feat(worker): ClaudeCodeAdapter runs claude -p stream-json in the per-user box`.
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Backend schema — add `claude` runtime, Anthropic OAuth auth kind, and profile→runtimes derivation
|
||||
|
||||
**Files:**
|
||||
- Edit `packages/backend/convex/schema.ts` (runtime unions + authType union).
|
||||
- Create `packages/backend/convex/runtimeSupport.ts` (new — pure derivation helper, no `'use node'`).
|
||||
- Edit `packages/backend/convex/agentJobs.ts` and `packages/backend/convex/spoonAgentSettings.ts` and `packages/backend/convex/aiProviderProfiles.ts`/`aiProviderProfilesNode.ts` (union constants).
|
||||
- Create `packages/backend/tests/unit/runtime-support.test.ts` (new).
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
*Schema changes:*
|
||||
- `agentJobs.runtime` (schema.ts:536): `v.optional(v.union(v.literal('openai_direct'), v.literal('opencode'), v.literal('codex'), v.literal('claude')))` — keep `openai_direct` for legacy reads (deprecated, never written).
|
||||
- `spoonAgentSettings.runtime` (schema.ts:472): same widened union.
|
||||
- `aiProviderProfiles.authType` (schema.ts:389): add `v.literal('anthropic_oauth_json')`.
|
||||
|
||||
*Produces* (`runtimeSupport.ts`):
|
||||
```ts
|
||||
import type { Doc } from './_generated/dataModel';
|
||||
export type AgentRuntimeName = 'codex' | 'opencode' | 'claude';
|
||||
|
||||
// Which runtimes a provider profile can drive.
|
||||
export const runtimesForProfile = (
|
||||
profile: Pick<Doc<'aiProviderProfiles'>, 'provider' | 'authType'>,
|
||||
): AgentRuntimeName[] => {
|
||||
if (profile.provider === 'opencode_openai_login' || profile.authType === 'opencode_auth_json') {
|
||||
return ['codex']; // ChatGPT-login snapshot → Codex CLI only
|
||||
}
|
||||
if (profile.provider === 'anthropic') {
|
||||
return profile.authType === 'anthropic_oauth_json' ? ['claude'] : ['claude', 'opencode'];
|
||||
}
|
||||
if (profile.provider === 'openai') return ['opencode', 'codex'];
|
||||
// Every remaining API-key provider is OpenAI-compatible → OpenCode.
|
||||
return ['opencode'];
|
||||
};
|
||||
```
|
||||
|
||||
**Steps:**
|
||||
- [ ] Write failing `packages/backend/tests/unit/runtime-support.test.ts`: `runtimesForProfile({ provider: 'anthropic', authType: 'api_key' })` → `['claude','opencode']`; `{ provider: 'opencode_openai_login', authType: 'opencode_auth_json' }` → `['codex']`; `{ provider: 'openrouter', authType: 'api_key' }` → `['opencode']`; `{ provider: 'openai', authType: 'api_key' }` → `['opencode','codex']`.
|
||||
- [ ] Run `cd packages/backend && bun run test:unit` → FAIL.
|
||||
- [ ] Edit `schema.ts` unions; create `runtimeSupport.ts`; widen the `authType` union constant in `aiProviderProfiles.ts:24`, `aiProviderProfilesNode.ts:24`.
|
||||
- [ ] Run `cd packages/backend && bun run codegen` then `bun run typecheck` → PASS. `bun run test:unit` → PASS.
|
||||
- [ ] Commit: `feat(backend): add claude runtime, anthropic OAuth auth kind, and profile→runtime derivation`.
|
||||
|
||||
---
|
||||
|
||||
## Task 8: Backend queue-time runtime validation + claim payload carries runtime
|
||||
|
||||
**Files:**
|
||||
- Edit `packages/backend/convex/agentJobs.ts` (`insertJob` at line 379, the three `runtime` union constants at line 21, `createFromRequest`/`createForThread`/`createForThreadInternal` args, `claimNextInternal` return).
|
||||
- Edit `packages/backend/convex/agentJobsNode.ts` (map `job.runtime` into the `WorkerClaim`).
|
||||
- Edit `packages/backend/convex/threads.ts` (`createUserThread` passes `runtime`).
|
||||
- Edit `packages/backend/convex/aiProviderProfiles.ts` (`publicProfile` exposes `supportedRuntimes`).
|
||||
- Edit `apps/agent-worker/src/runtime/provider.ts` (`Claim.job.runtime` already widened in Task 1 — confirm) and `worker.ts` `runClaim` (use `claim.job.runtime` as authoritative).
|
||||
- Edit `packages/backend/tests/unit/harness.test.ts` or add `packages/backend/tests/unit/runtime-validation.test.ts` (new).
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
- `const runtime = v.union(v.literal('codex'), v.literal('opencode'), v.literal('claude'))` (agentJobs.ts:21). The three create mutations gain `runtime: v.optional(runtime)`.
|
||||
- `insertJob` resolves and validates:
|
||||
```ts
|
||||
const profile = await getJobProfile(ctx, ownerId, aiProviderProfileId);
|
||||
const supported = runtimesForProfile(profile);
|
||||
const resolvedRuntime =
|
||||
requestedRuntime ?? (settings.runtime as AgentRuntimeName | undefined) ?? supported[0];
|
||||
if (!supported.includes(resolvedRuntime)) {
|
||||
throw new ConvexError(
|
||||
`Provider "${profile.name}" cannot run the "${resolvedRuntime}" runtime. Supported: ${supported.join(', ')}.`,
|
||||
);
|
||||
}
|
||||
// insert agentJobs with runtime: resolvedRuntime
|
||||
```
|
||||
- `claimNextInternal` already returns the full `job` doc (schema.ts row includes `runtime`), so no change there; `agentJobsNode.claimNextForWorker` must forward `job.runtime` — it already returns `claimed.job` verbatim in `WorkerClaim.job`, so `job.runtime` flows through. Confirm the worker's `Claim.job.runtime` type (widened in Task 1) matches.
|
||||
- `publicProfile` (aiProviderProfiles.ts:45): add `supportedRuntimes: runtimesForProfile(profile)` so the UI can filter.
|
||||
|
||||
*Worker change:* `runClaim` (worker.ts:1301) currently throws for non-`opencode`. Replace with: resolve `const runtime = (claim.job.runtime ?? 'opencode') as AgentRuntimeName;` set `workspace.runtime = runtime`; delete the legacy `openai_direct` throw (queue-time guard now prevents bad runtimes). `prepareAuth` for the resolved adapter runs in place of the `isCodexLoginProfile` special-case (worker.ts:1375): `await getAdapter(runtime).prepareAuth(workspace)`.
|
||||
|
||||
**Steps:**
|
||||
- [ ] Write failing `packages/backend/tests/unit/runtime-validation.test.ts` (convex-test style from harness.test.ts): create a user + GitHub spoon + an `anthropic`/`api_key` profile set default; call `createForThread`/`createUserThread` equivalent with `runtime: 'codex'` → expect throw `/cannot run the "codex" runtime/`; with `runtime: 'claude'` → resolves; with no runtime and an `opencode_openai_login` profile → job.runtime === 'codex'.
|
||||
- [ ] Run `cd packages/backend && bun run test:unit` → FAIL.
|
||||
- [ ] Implement the union widening, `insertJob` validation, thread/request arg plumbing, and `publicProfile.supportedRuntimes`.
|
||||
- [ ] Update `worker.ts` `runClaim` to set `workspace.runtime` from `claim.job.runtime` and call `getAdapter(runtime).prepareAuth(workspace)`; delete the legacy throw.
|
||||
- [ ] Run `cd packages/backend && bun run codegen && bun run typecheck && bun run test:unit` → PASS. Run `cd apps/agent-worker && bun run typecheck && bun run test:unit` → PASS.
|
||||
- [ ] Commit: `feat(backend): queue-time runtime/profile validation; job.runtime authoritative in worker`.
|
||||
|
||||
---
|
||||
|
||||
## Task 9: Migrate/deprecate `openai_direct`
|
||||
|
||||
**Files:**
|
||||
- Create `packages/backend/convex/migrations.ts` (new internalMutation) OR add to an existing maintenance module — grep for an existing pattern first (`grep -rn "internalMutation" packages/backend/convex | grep -i migrat`; there is none, so create `migrations.ts`).
|
||||
- Edit `packages/backend/tests/unit/` (new test or extend).
|
||||
|
||||
**Interfaces:**
|
||||
```ts
|
||||
export const migrateOpenAiDirectRuntime = internalMutation({
|
||||
args: {},
|
||||
handler: async (ctx) => {
|
||||
let patched = 0;
|
||||
for (const job of await ctx.db.query('agentJobs').collect()) {
|
||||
if (job.runtime === 'openai_direct') { await ctx.db.patch(job._id, { runtime: 'opencode', updatedAt: Date.now() }); patched++; }
|
||||
}
|
||||
for (const s of await ctx.db.query('spoonAgentSettings').collect()) {
|
||||
if (s.runtime === 'openai_direct') { await ctx.db.patch(s._id, { runtime: 'opencode', updatedAt: Date.now() }); patched++; }
|
||||
}
|
||||
return { patched };
|
||||
},
|
||||
});
|
||||
```
|
||||
Run once post-deploy via `npx convex run migrations:migrateOpenAiDirectRuntime` (document in the smoke/deploy notes). `openai_direct` stays in the schema unions as a legacy-read literal; nothing writes it.
|
||||
|
||||
**Steps:**
|
||||
- [ ] Write failing test: insert a `spoonAgentSettings` row with `runtime: 'openai_direct'` (via `t.run`/raw ctx), run the migration, assert it becomes `'opencode'` and `patched >= 1`.
|
||||
- [ ] Run `cd packages/backend && bun run test:unit` → FAIL.
|
||||
- [ ] Implement `migrations.ts`.
|
||||
- [ ] Run `cd packages/backend && bun run codegen && bun run typecheck && bun run test:unit` → PASS.
|
||||
- [ ] Commit: `feat(backend): migration to retire openai_direct runtime`.
|
||||
|
||||
---
|
||||
|
||||
## Task 10: Next UI — runtime picker (thread create + spoon agent settings) and Anthropic OAuth profile kind
|
||||
|
||||
**Files:**
|
||||
- Edit `apps/next/src/components/threads/thread-workspace-form.tsx` (replace the disabled `Input value='OpenCode workspace'` at line 142-144 with a runtime `Select`; pass `runtime` to `createUserThread`).
|
||||
- Edit `apps/next/src/components/spoons/spoon-agent-settings-form.tsx` (replace the disabled Runtime input at line 187-190 with a `Select`; pass `runtime` to `spoonAgentSettings.update`).
|
||||
- Edit `apps/next/src/components/integrations/ai-provider-profiles-panel.tsx` (add an Anthropic OAuth JSON option to `providerOptions` / auth-type handling at line 71-97).
|
||||
- Edit `packages/backend/convex/threads.ts` `createUserThread` args + `spoonAgentSettings.ts` `update` args + `agentJobs.ts` create mutations to accept `runtime` (done in Task 8 for agentJobs; add to `createUserThread`/`update`).
|
||||
|
||||
**Interfaces:**
|
||||
- `createUserThread` gains `runtime: v.optional(v.union(v.literal('codex'), v.literal('opencode'), v.literal('claude')))`, forwarded to `createForThreadInternal` → `insertJob`.
|
||||
- `spoonAgentSettings.update` `runtime` arg union widens to `codex|opencode|claude` and `patch.runtime = args.runtime` (drop the `= 'opencode'` hardcode at spoonAgentSettings.ts:126).
|
||||
- UI: both forms read `selectedProfile.supportedRuntimes` (new field from Task 8) and render only those `SelectItem`s; default the value to the settings runtime or `supportedRuntimes[0]`; disable submit if the chosen runtime ∉ supportedRuntimes.
|
||||
- Profiles panel: add `{ value: 'anthropic', label: 'Anthropic OAuth (paste credentials)', authType: 'anthropic_oauth_json' }` alongside the existing `anthropic` API-key option; when `anthropic_oauth_json`, render a `Textarea` for the pasted `~/.claude/.credentials.json` (same UX as the Codex `opencode_auth_json` textarea).
|
||||
|
||||
**Steps:**
|
||||
- [ ] Add `runtime` to `createUserThread` and `spoonAgentSettings.update` args + handlers; `cd packages/backend && bun run codegen && bun run typecheck`.
|
||||
- [ ] Edit the three Next components: runtime `Select`s wired to `supportedRuntimes`; Anthropic OAuth option in the profiles panel.
|
||||
- [ ] Typecheck Next: `cd apps/next && bun run typecheck` (or the repo's Next typecheck script — confirm via `apps/next/package.json`). If component tests exist for these forms, extend them; otherwise cover behavior in the Task 11 manual checklist (note: no jsdom test is required by the spec for the picker, but add one if a sibling form test already exists).
|
||||
- [ ] Commit: `feat(web): per-thread/per-spoon runtime picker and Anthropic OAuth provider profile`.
|
||||
|
||||
---
|
||||
|
||||
## Task 11: Add Claude Code CLI to the job image + extend the smoke test
|
||||
|
||||
**Files:**
|
||||
- Edit `docker/agent-job.Dockerfile` (line 50 npm install line).
|
||||
- Edit `scripts/smoke-agent-container` (version checks + one turn per runtime).
|
||||
- Edit `README.md` (the "Production agent runtime images" `<details>` at line 189-260: mention Claude Code CLI + the `anthropic_oauth_json` credential snapshot pattern, mirroring the Codex `auth.json` paragraph at line 255-258).
|
||||
|
||||
**Interfaces / concrete commands:**
|
||||
- Dockerfile line 50 becomes:
|
||||
```dockerfile
|
||||
RUN npm install -g pnpm yarn bun@1.3.10 opencode-ai@1.17.9 @openai/codex@0.142.0 @anthropic-ai/claude-code@<PIN> \
|
||||
&& npm cache clean --force
|
||||
```
|
||||
Pin `<PIN>` to the latest published `@anthropic-ai/claude-code` version at implementation time (`npm view @anthropic-ai/claude-code version`); record the exact version in the commit message.
|
||||
- `scripts/smoke-agent-container`: add to the in-container check block (after `codex --version` at line 30): `claude --version`. Then add a documented manual per-runtime turn section (commented, runnable by hand) — build the image, start a throwaway box, exec one turn per runtime:
|
||||
```sh
|
||||
# Manual per-runtime smoke (requires creds in your shell env):
|
||||
# docker build -f docker/agent-job.Dockerfile -t spoon-agent-job:latest .
|
||||
# docker run -d --name spoon-box-smoke --init spoon-agent-job:latest sleep infinity
|
||||
# docker exec -e OPENAI_API_KEY -w /workspace spoon-box-smoke \
|
||||
# opencode run --format json --model openai/gpt-5.5 -- 'print hello'
|
||||
# docker exec -e ANTHROPIC_API_KEY -w /workspace spoon-box-smoke \
|
||||
# claude -p 'print hello' --output-format stream-json --verbose --dangerously-skip-permissions
|
||||
# docker exec -e CODEX_HOME=/root/.codex -w /workspace spoon-box-smoke \
|
||||
# codex exec --json --dangerously-bypass-approvals-and-sandbox 'print hello'
|
||||
# docker rm -f spoon-box-smoke
|
||||
```
|
||||
Use these commands to also **confirm the real flag names** assumed in Tasks 4/6 (`opencode run --help`, `claude --help`) and fix the adapters if they differ.
|
||||
|
||||
**Steps:**
|
||||
- [ ] `npm view @anthropic-ai/claude-code version` → choose the pin. Edit the Dockerfile.
|
||||
- [ ] Build the image and run `SPOON_AGENT_JOB_IMAGE=spoon-agent-job:latest ./scripts/smoke-agent-container` → all `--version` checks pass (`claude --version` included).
|
||||
- [ ] Run the manual per-runtime turns; reconcile adapter flags with reality (patch Task 4/6 code if needed, keeping unit tests green).
|
||||
- [ ] Update `README.md`.
|
||||
- [ ] Commit: `build(agent-job): pin Claude Code CLI; smoke one turn per runtime`.
|
||||
|
||||
---
|
||||
|
||||
## Task 12: Secrets hygiene — 0600 env files, deleted on stop, re-materialized per run
|
||||
|
||||
**Files:**
|
||||
- Edit `apps/agent-worker/src/worker.ts` (`materializeEnvFile` at line 1167; `stopWorkspace`/`openWorkspacePullRequest` teardown; call `materializeEnvFile` per run).
|
||||
- Create `apps/agent-worker/tests/unit/materialize-env.test.ts` (new).
|
||||
|
||||
**Interfaces:**
|
||||
- `materializeEnvFile` (worker.ts:1167): write with `{ mode: 0o600 }` (currently no mode). Extract the file path resolution + write into a pure-ish helper `writeMaterializedEnv(repoDir, envFilePath, secrets): Promise<string>` returning the absolute path, so it is unit-testable with `mkdtemp`.
|
||||
- Track the materialized path on the workspace (`workspace.materializedEnvPath?: string`).
|
||||
- On `stopWorkspace` (worker.ts:1866) and `openWorkspacePullRequest` (worker.ts:1851) and the `runClaim` catch (worker.ts:1410) teardown: if `workspace.materializedEnvPath` exists, `await rm(workspace.materializedEnvPath, { force: true })`.
|
||||
- Re-materialize per run: call `materializeEnvFile(workspace)` at the top of `sendWorkspaceMessage` (worker.ts:1624) when `claim.job.materializeEnvFile` is set, before the adapter turn — so the env file is fresh each turn and can be deleted between turns. Keep the existing `ensureNoEnvFilesStaged` staging guard (worker.ts:1269) unchanged.
|
||||
|
||||
**Steps:**
|
||||
- [ ] Write failing `materialize-env.test.ts`: `mkdtemp` a repo dir, call `writeMaterializedEnv(dir, '.env.local', [{ name: 'A', value: 'x' }])`, assert the file exists, `(await stat(path)).mode & 0o777 === 0o600`, and content is `A="x"\n` (matches worker.ts:1172-1174 formatting). Mirror the `codex-runtime.test.ts` `mode()` helper + `afterEach` cleanup.
|
||||
- [ ] Run `cd apps/agent-worker && bun run test:unit` → FAIL.
|
||||
- [ ] Implement `writeMaterializedEnv` (0600), wire deletion on stop/PR/failure, re-materialize per run.
|
||||
- [ ] Run `bun run test:unit` → PASS. `bun run typecheck` → PASS.
|
||||
- [ ] Commit: `fix(worker): materialize env files 0600, re-materialize per run, delete on stop`.
|
||||
|
||||
---
|
||||
|
||||
## Self-review against the Phase 2 spec
|
||||
|
||||
- **Adapter interface, all exec into the box, process-group kill** → Tasks 1–2 (`buildMarkedCommand` = `setsid` + marker; `killBoxProcessesByMarker` = in-box `pgrep`+`kill -TERM/-KILL` of the group; never the local execa client). ✅
|
||||
- **Codex refactored behind the interface** → Task 3. ✅
|
||||
- **OpenCode inside the box, per-job path deleted** → Tasks 4–5 (`opencode run --format json` in the box; `startWorkspaceContainer`/`spoon-agent-job-*`/`containerAccess host_port|network`/dead helpers all removed). ✅
|
||||
- **Claude Code adapter + new provider kind + pinned CLI + prepareAuth** → Tasks 6, 7, 10, 11 (`anthropic_oauth_json` snapshot mirrors Codex `auth.json`; API-key path uses `ANTHROPIC_API_KEY`). ✅
|
||||
- **Runtime picker + profiles declare supported runtimes + queue-time validation replacing run-time `openai_direct` discovery + migrate/deprecate** → Tasks 7–10. ✅
|
||||
- **Secrets hygiene (0600, delete on stop, re-materialize; staging guard stays)** → Task 12. ✅
|
||||
- **Testing approach** → adapter-level fixture stream tests (injectable `execStream`), normalizer fixtures matching the existing style, convex-test backend tests, extended manual smoke. ✅
|
||||
Reference in New Issue
Block a user