spoon/README.md

# Spoon

Spoon is a self-hostable fork maintenance cockpit.

Forking a project should not mean supporting it alone. Spoon tracks managed
forks, called **Spoons**, watches upstream for drift, automatically syncs clean
forks when it can, and opens durable **Threads** when upstream changes need
review, context, or code.

This repository is the Spoon application itself, not a generic starter.

## What Spoon Does

- Tracks GitHub-backed managed forks and their upstream projects.
- Shows raw and effective drift, fork-only commits, pull requests, clone URLs,
  additional remotes, sync history, and open maintenance work.
- Uses Threads as the product center for upstream reviews, merge conflicts,
  ignored commits, user-requested changes, worker logs, and draft PR handoff.
- Auto-syncs clean behind forks when there are no fork-only commits.
- Creates maintenance threads when custom fork work means upstream changes need
  a decision.
- Runs optional OpenCode-backed workspaces in isolated agent-job containers.
- Lets users configure encrypted AI provider profiles, Codex/OpenCode auth,
  per-Spoon secrets, commands, and agent settings.
- Opens draft PRs for code changes instead of auto-merging custom forks.

## Current Scope

Implemented today:

- Public Next.js landing page for Spoon's thread-first maintenance model.
- Authenticated web routes:
  - `/dashboard`
  - `/spoons`
  - `/spoons/new`
  - `/spoons/[spoonId]`
  - `/spoons/[spoonId]/agent/[jobId]`
  - `/threads`
  - `/threads/[threadId]`
  - `/settings/profile`
  - `/settings/integrations`
  - `/settings/ai-providers`
- Legacy `/updates` and `/agents` routes redirect into Threads.
- GitHub App connection, repository listing, fork creation, drift refresh,
  commit/PR cache, and safe sync foundation.
- Thread-first maintenance model with ignored upstream changes and effective
  drift.
- Optional `apps/agent-worker` service that claims queued jobs, clones the
  current GitHub fork, starts an isolated workspace, exposes file browsing and
  edits through server-side Next proxies, runs commands, and opens draft PRs.
- Browser workspace with persisted thread messages, file tree, Monaco editor
  with optional Vim mode, diff view, command panel, logs, artifacts, and draft
  PR actions.
- Encrypted per-user AI provider profiles and per-Spoon project secrets.
- Password auth and Authentik/GitHub OAuth through Convex Auth.
- Expo companion app shell with password and Authentik sign-in.
- Self-hosted local Convex using Postgres storage.

Not implemented yet:

- Automatic merge of custom/diverged forks.
- Git provider automation beyond GitHub.
- Additional remotes as push targets.
- Long-running service-stack orchestration inside agent jobs.
- Direct browser access to worker containers.
- Production mobile build/release setup.

## Architecture

- `apps/next`: Next.js 16 web app and primary product UI.
- `apps/agent-worker`: optional server-side worker for OpenCode workspaces and
  draft PR jobs.
- `apps/expo`: Expo companion app.
- `packages/backend/convex`: self-hosted Convex schema, functions, auth, and
  HTTP routes.
- `packages/ui`: shared shadcn-based UI components.
- `tools`: shared ESLint, Prettier, Tailwind, TypeScript, and Vitest config.
- `docker`: local and production Compose files.
- `scripts`: environment, database, codegen, and CI helpers.

Core domain objects:

- `spoons`: managed fork records.
- `threads`: durable maintenance and work conversations.
- `threadMessages`: persisted thread messages.
- `syncRuns`: upstream checks, sync attempts, and maintenance decisions.
- `ignoredUpstreamChanges`: intentional ignore decisions that affect effective
  drift.
- `gitConnections`: Git provider connection metadata.
- `agentJobs`: worker-executed workspace jobs and PR lifecycle.
- `agentJobEvents` and `agentJobArtifacts`: logs and structured job outputs.
- `agentWorkspaceChanges`: recorded file changes from user, agent, or command
  activity.
- `spoonSecrets`: encrypted per-Spoon environment variables.
- `spoonAgentSettings`: per-Spoon runtime, branch, command, and env-file
  settings.
- `aiProviderProfiles`: encrypted provider/auth profiles used by OpenCode.

## Local Setup

Requirements:

- Bun 1.3.10
- Node 22
- Docker or Podman
- Infisical CLI

```sh
bun install --frozen-lockfile
infisical login
infisical init
bun db:up
bun dev:next
```

Local services:

- Next.js: `http://localhost:3000`
- Convex API: `http://localhost:3210`
- Convex site HTTP routes: `http://localhost:3211`
- Convex dashboard: `http://localhost:6791`
- Convex Postgres: `localhost:5432`

Next and Expo run on the host. Local Convex runs in containers with Postgres
storage. Normal `bun db:up` never contacts staging; it starts local Postgres,
Convex, and the dashboard, generates a machine-local Convex admin key in
`.local/dev.generated.env` when needed, deploys functions/schema, and configures
local Convex Auth keys.

```sh
bun db:down        # stop; preserve local data
bun db:down:wipe   # remove local data volumes and generated admin key
```

Use staging services explicitly:

```sh
INFISICAL_ENV=staging bun dev:next
```

Run the optional local agent worker in a separate terminal:

```sh
bun dev:agent
```

The worker starts an internal HTTP API, defaulting to `http://localhost:3921`,
for server-side Next route handlers. The browser never receives the worker token
or talks to this API directly.

The Docker Compose local worker service is disabled by default behind the
`agent` profile. Build the job image before using Docker-backed jobs:

```sh
docker build -f docker/agent-job.Dockerfile -t spoon-agent-job:latest .
docker compose -f docker/compose.local.yml --profile agent up spoon-agent-worker
```

The job image includes the OpenCode CLI. Rebuild it after changes to
`docker/agent-job.Dockerfile`.

## Environment Model

Local `dev` and `staging` values come from Infisical through
`scripts/with-env`. App commands do not fall back to root `.env` files.

Generated local state belongs in:

```txt
.local/<environment>.generated.env
```

CI uses Gitea-provided secrets or `CI_ENV_FILE` and must not call Infisical.

Useful helpers:

```sh
sh scripts/with-env dev -- <command>
sh scripts/export-env dev
bun sync:convex
bun sync:convex:staging
```

### Convex Deployment Env

Convex functions and HTTP actions read environment variables from the Convex
deployment environment, not directly from the host process. OAuth providers,
GitHub App credentials, UseSend, encryption keys, worker tokens, and Convex Auth
signing keys must be synced into the selected Convex deployment.

`packages/backend` runs `scripts/sync-convex-env` before `convex dev`, so
`bun dev:next`, `bun dev:backend`, and `bun db:up` sync the relevant Infisical
values into local Convex first. Run it manually when needed:

```sh
sh scripts/sync-convex-env dev
sh scripts/sync-convex-env staging
INFISICAL_ENV=staging bun sync:convex
```

For local `dev`, `JWT_PRIVATE_KEY`, `JWKS`, `SPOON_ENCRYPTION_KEY`,
`SPOON_WORKER_TOKEN`, and related generated values are created automatically if
they are not already present. The generated Convex admin key remains
machine-local in `.local/dev.generated.env`; do not put it in Infisical.

Local OAuth callback URLs:

```txt
http://localhost:3211/api/auth/callback/authentik
http://localhost:3211/api/auth/callback/github
```

If GitHub App actions fail with `GITHUB_APP_PRIVATE_KEY is not configured`, add
the full PEM contents to Infisical as `GITHUB_APP_PRIVATE_KEY` and rerun the
sync command.

## Development

```sh
bun dev:next
bun dev:expo
bun dev:agent
```

Physical devices cannot resolve their own `localhost`; override the public
Convex URL with the development host's LAN address when testing Expo on-device.

Shared dependency versions belong in root catalogs. Edit the root catalog, run
`bun install`, then `bun lint:ws`. Do not run `bun update` inside a workspace.

## Validation

Routine checks:

```sh
bun lint:ws
bun format
bun lint
bun typecheck
bun run test
```

Full local gate without e2e:

```sh
SKIP_E2E=1 bun run ci:check
```

Local-stack smoke e2e:

```sh
bun test:e2e
```

`bun test:e2e` starts the isolated local stack when needed and stops it
afterward only when it was not already running.

Use `bun run test`, not bare `bun test`; bare `bun test` invokes Bun's built-in
test runner instead of the repo's Turbo/Vitest test script.

## Deployment

Production Compose runs the Next image, self-hosted Convex backend/dashboard,
and Postgres. The deployed Next image is expected to be named
`spoon-next:latest` in the Gitea registry.

Gitea runs the quality gate first, runs Convex codegen with deployment env,
builds the Next image from injected secrets or `CI_ENV_FILE`, then pushes SHA
and `latest` tags. CI never installs or invokes Infisical.