Add agent workflows & stuff
This commit is contained in:
Gabriel Brown
@@ -1,11 +1,77 @@
|
||||
# Spoon
|
||||
|
||||
A reusable Bun/Turborepo template with Next.js 16, Expo, self-hosted Convex,
|
||||
shared UI/config packages, Vitest, and Docker deployment.
|
||||
Spoon is a self-hostable fork maintenance dashboard.
|
||||
|
||||
The product goal is simple: make it practical to fork a project, customize it,
|
||||
and still stay close to upstream. Spoon tracks managed forks, called
|
||||
**Spoons**, and lays the foundation for upstream update checks, AI-assisted
|
||||
change review, and agent-authored merge requests.
|
||||
|
||||
This repository is the Spoon application itself, not a generic starter.
|
||||
|
||||
## Current scope
|
||||
|
||||
Implemented today:
|
||||
|
||||
- Public Spoon landing page in Next.js.
|
||||
- Authenticated web dashboard routes:
|
||||
- `/dashboard`
|
||||
- `/spoons`
|
||||
- `/spoons/new`
|
||||
- `/updates`
|
||||
- `/spoons/[spoonId]`
|
||||
- `/settings`
|
||||
- Manual and GitHub-created Spoon records stored in Convex.
|
||||
- GitHub App connection, repository listing, fork creation, drift refresh,
|
||||
commit/PR cache, and safe manual sync foundation.
|
||||
- Per-user OpenAI settings for upstream compatibility review.
|
||||
- Per-Spoon encrypted project secrets and agent runtime settings.
|
||||
- Optional `apps/agent-worker` service that can claim queued jobs, clone the
|
||||
GitHub fork, ask OpenAI for bounded file edits, run checks, push a branch, and
|
||||
open a draft PR.
|
||||
- Password auth and Authentik OAuth through Convex Auth.
|
||||
- Expo companion app shell with password and Authentik sign-in.
|
||||
- Self-hosted local Convex using Postgres storage.
|
||||
|
||||
Not implemented yet:
|
||||
|
||||
- Browser IDE/editor.
|
||||
- Automatic merge.
|
||||
- Additional Git provider automation beyond preserving provider-neutral fields.
|
||||
- Additional remotes as push targets.
|
||||
- Long-running service-stack orchestration inside agent jobs.
|
||||
- Production mobile build/release setup.
|
||||
|
||||
## Architecture
|
||||
|
||||
- `apps/next`: Next.js 16 web app and primary product UI.
|
||||
- `apps/agent-worker`: optional server-side worker for queued coding-agent jobs.
|
||||
- `apps/expo`: Expo companion app.
|
||||
- `packages/backend/convex`: self-hosted Convex schema, functions, auth, and
|
||||
HTTP routes.
|
||||
- `packages/ui`: shared shadcn-based UI components.
|
||||
- `tools`: shared ESLint, Prettier, Tailwind, TypeScript, and Vitest config.
|
||||
- `docker`: local and production Compose files.
|
||||
- `scripts`: environment, database, and CI helpers.
|
||||
|
||||
The core domain objects are:
|
||||
|
||||
- `spoons`: managed fork records.
|
||||
- `gitConnections`: future Git provider connection metadata.
|
||||
- `syncRuns`: future upstream checks, merge attempts, and AI reviews.
|
||||
- `agentRequests`: prompt-driven agent work requests.
|
||||
- `agentJobs`: worker-executed coding-agent jobs and their PR lifecycle.
|
||||
- `spoonSecrets`: encrypted per-Spoon environment variables.
|
||||
- `spoonAgentSettings`: per-Spoon agent model, branch, and command settings.
|
||||
|
||||
## Local setup
|
||||
|
||||
Requirements: Bun 1.3.10, Docker or Podman, Node 22, and the Infisical CLI.
|
||||
Requirements:
|
||||
|
||||
- Bun 1.3.10
|
||||
- Node 22
|
||||
- Docker or Podman
|
||||
- Infisical CLI
|
||||
|
||||
```sh
|
||||
bun install --frozen-lockfile
|
||||
@@ -15,72 +81,172 @@ bun db:up
|
||||
bun dev:next
|
||||
```
|
||||
|
||||
The committed `.infisical.json` links this repository to its own Infisical
|
||||
project. Local commands read `dev` by default and never fall back to `.env`
|
||||
files. Select staging with `INFISICAL_ENV=staging bun dev:next`.
|
||||
|
||||
Local services:
|
||||
|
||||
- Next.js: `http://localhost:3000`
|
||||
- Convex API: `http://localhost:3210`
|
||||
- Convex site HTTP routes: `http://localhost:3211`
|
||||
- Convex dashboard: `http://localhost:6791`
|
||||
- Convex Postgres: `localhost:5432`
|
||||
|
||||
Next and Expo run on the host. Spoon runs local Convex with Postgres storage by
|
||||
default. Local Compose creates the database named by `LOCAL_INSTANCE_NAME`
|
||||
because the Convex backend opens that database inside the Postgres cluster.
|
||||
Convex receives a database-cluster URL without a path.
|
||||
Next and Expo run on the host. Local Convex runs in containers with Postgres
|
||||
storage. Normal `bun db:up` never contacts staging; it starts local Postgres,
|
||||
Convex, and the dashboard, generates a machine-local Convex admin key in
|
||||
`.local/dev.generated.env` when needed, deploys functions/schema, and
|
||||
configures local Convex Auth keys.
|
||||
|
||||
```sh
|
||||
bun db:down # stop; preserve local data
|
||||
bun db:down:wipe # remove local data volumes and generated admin key
|
||||
```
|
||||
|
||||
Normal `bun db:up` never contacts staging. It starts local Postgres, Convex, and
|
||||
the dashboard, generates a machine-local Convex admin key in
|
||||
`.local/dev.generated.env` when needed, deploys functions/schema, and configures
|
||||
local Convex Auth keys.
|
||||
Use staging services explicitly:
|
||||
|
||||
Physical devices cannot resolve their own `localhost`; override the public
|
||||
Convex URL with the development host's LAN address when testing Expo on-device.
|
||||
```sh
|
||||
INFISICAL_ENV=staging bun dev:next
|
||||
```
|
||||
|
||||
Run the optional local agent worker in a separate terminal:
|
||||
|
||||
```sh
|
||||
bun dev:agent
|
||||
```
|
||||
|
||||
The Docker Compose local worker service is disabled by default behind the
|
||||
`agent` profile. Build the job image before using Docker-backed jobs:
|
||||
|
||||
```sh
|
||||
docker build -f docker/agent-job.Dockerfile -t spoon-agent-job:latest .
|
||||
docker compose -f docker/compose.local.yml --profile agent up spoon-agent-worker
|
||||
```
|
||||
|
||||
## Environment model
|
||||
|
||||
- Local `dev` and `staging`: Infisical.
|
||||
- Generated local state: `.local/<environment>.generated.env`.
|
||||
- CI/CD: Gitea `DOTENV_PROD`, materialized only as a temporary runner file.
|
||||
- Docker compilation: explicit Compose build args; `.env*` stays outside the
|
||||
image context.
|
||||
Local `dev` and `staging` values come from Infisical through `scripts/with-env`.
|
||||
App commands do not fall back to root `.env` files.
|
||||
|
||||
Run `sh scripts/with-env dev -- <command>` for an environment-aware command or
|
||||
`sh scripts/export-env dev` to materialize a temporary merged dotenv stream.
|
||||
Do not commit or maintain root `.env` files.
|
||||
Generated local state belongs in:
|
||||
|
||||
## Development and quality
|
||||
```txt
|
||||
.local/<environment>.generated.env
|
||||
```
|
||||
|
||||
CI uses Gitea-provided secrets or `CI_ENV_FILE` and must not call Infisical.
|
||||
|
||||
Useful helpers:
|
||||
|
||||
```sh
|
||||
sh scripts/with-env dev -- <command>
|
||||
sh scripts/export-env dev
|
||||
bun sync:convex
|
||||
```
|
||||
|
||||
### Convex deployment env
|
||||
|
||||
Convex functions and HTTP actions read environment variables from the Convex
|
||||
deployment environment, not directly from the host process. For OAuth providers,
|
||||
that means Infisical values must also be present in local Convex env.
|
||||
|
||||
`packages/backend` runs `scripts/sync-convex-env` before `convex dev`, so
|
||||
`bun dev:next`, `bun dev:backend`, and `bun db:up` sync the relevant Infisical
|
||||
values into the selected Convex deployment first. Run it manually when needed:
|
||||
|
||||
```sh
|
||||
sh scripts/sync-convex-env dev
|
||||
sh scripts/sync-convex-env staging
|
||||
INFISICAL_ENV=staging bun sync:convex
|
||||
```
|
||||
|
||||
The sync includes:
|
||||
|
||||
```txt
|
||||
AUTH_AUTHENTIK_ID
|
||||
AUTH_AUTHENTIK_SECRET
|
||||
AUTH_AUTHENTIK_ISSUER
|
||||
AUTH_GITHUB_ID
|
||||
AUTH_GITHUB_SECRET
|
||||
GITHUB_APP_ID
|
||||
GITHUB_APP_CLIENT_ID
|
||||
GITHUB_APP_CLIENT_SECRET
|
||||
GITHUB_APP_PRIVATE_KEY
|
||||
GITHUB_APP_WEBHOOK_SECRET
|
||||
GITHUB_APP_SLUG
|
||||
GITHUB_APP_INSTALLATION_ID
|
||||
GITHUB_APP_OWNER
|
||||
SPOON_ENCRYPTION_KEY
|
||||
SPOON_WORKER_TOKEN
|
||||
USESEND_API_KEY
|
||||
USESEND_URL
|
||||
USESEND_FROM_EMAIL
|
||||
JWT_PRIVATE_KEY
|
||||
JWKS
|
||||
SITE_URL
|
||||
```
|
||||
|
||||
For local `dev`, `JWT_PRIVATE_KEY`, `JWKS`, `SPOON_ENCRYPTION_KEY`, and
|
||||
`SPOON_WORKER_TOKEN` are generated automatically if they are not already present
|
||||
in Convex. The generated Convex admin key remains machine-local in
|
||||
`.local/dev.generated.env`; do not put it in Infisical.
|
||||
|
||||
The local OAuth callback URLs are:
|
||||
|
||||
```txt
|
||||
http://localhost:3211/api/auth/callback/authentik
|
||||
http://localhost:3211/api/auth/callback/github
|
||||
```
|
||||
|
||||
If GitHub App actions fail with `GITHUB_APP_PRIVATE_KEY is not configured`, add
|
||||
the full PEM contents to Infisical as `GITHUB_APP_PRIVATE_KEY` and rerun the
|
||||
sync command.
|
||||
|
||||
## Development
|
||||
|
||||
```sh
|
||||
bun dev:next
|
||||
bun dev:expo
|
||||
```
|
||||
|
||||
Physical devices cannot resolve their own `localhost`; override the public
|
||||
Convex URL with the development host's LAN address when testing Expo on-device.
|
||||
|
||||
Shared dependency versions belong in root catalogs. Edit the root catalog, run
|
||||
`bun install`, then `bun lint:ws`. Do not run `bun update` inside a workspace.
|
||||
|
||||
## Validation
|
||||
|
||||
Routine checks:
|
||||
|
||||
```sh
|
||||
bun lint:ws
|
||||
bun format
|
||||
bun lint
|
||||
bun typecheck
|
||||
bun test:unit
|
||||
bun test:integration
|
||||
bun test:component
|
||||
bun run test
|
||||
```
|
||||
|
||||
Full local gate without e2e:
|
||||
|
||||
```sh
|
||||
SKIP_E2E=1 bun run ci:check
|
||||
```
|
||||
|
||||
`bun test:e2e` starts the isolated local stack and currently performs generic
|
||||
stack smoke checks. It stops the stack afterward only when the stack was not
|
||||
already running. It skips in CI and when `SKIP_E2E=1` is set.
|
||||
Local-stack smoke e2e:
|
||||
|
||||
Shared dependency versions belong in root catalogs. Edit the root catalog, run
|
||||
`bun install`, then `bun lint:ws`; do not run `bun update` inside a workspace.
|
||||
```sh
|
||||
bun test:e2e
|
||||
```
|
||||
|
||||
`bun test:e2e` starts the isolated local stack when needed and stops it
|
||||
afterward only when it was not already running.
|
||||
|
||||
Use `bun run test`, not bare `bun test`; bare `bun test` invokes Bun's built-in
|
||||
test runner instead of the repo's Turbo/Vitest test script.
|
||||
|
||||
## Deployment
|
||||
|
||||
Production Compose retains the self-hosted Convex backend/dashboard and expects
|
||||
`POSTGRES_URL` to be a database-cluster URL without a database path. Gitea runs
|
||||
the quality gate first, builds the Next image from a temporary Gitea-secret env
|
||||
file, then pushes SHA and `latest` tags. CI never installs or invokes Infisical.
|
||||
Production Compose keeps the self-hosted Convex backend/dashboard and expects
|
||||
`POSTGRES_URL` to be a database-cluster URL without a database path.
|
||||
|
||||
Gitea runs the quality gate first, builds the Next image from a temporary
|
||||
Gitea-secret env file, then pushes SHA and `latest` tags. CI never installs or
|
||||
invokes Infisical.
|
||||
|
||||
Reference in New Issue
Block a user