Panama learns what a server is: from a root login to running containers

A machine's role is now the interview's first question and the one answer
Panama records. Servers get the same shell minus the screen: core packages,
nvm, Bun, Claude Code and Codex (desktops get Codex too), linger, rootless
ports from 80, firewalld, the nginx-bridge network, and a nightly image
updater that replaced watchtower for cause.

server/containers/ carries junior's 23 compose services -- secrets moved to
per-machine .env files that never enter this public repo, every transformed
compose proven to render byte-identical to what is live. 'panama server'
enables, disables and relinks them; nothing here restarts a running service.
'boot --server' walks a fresh VPS from its root login to a normal install.

Five new contracts pin the secrets rule, the catalog's shape, panama-server's
behavior, the role plumbing, and the dotfile classification.

Claude-Session: https://claude.ai/code/session_01NU5JGiN3JfzqrLQB6wmJ1E

.claude/docs/specs/2026-08-25-server-role.md

@@ -0,0 +1,230 @@
+# Panama server role: headless installs and versioned containers
+
+2026-08-25. Designed with Gib across two interview rounds; every decision below
+was made explicitly, none inferred.
+
+## Goal
+
+Panama becomes the one repo for every Fedora machine Gib runs. A desktop gets
+what it gets today. A server — a Hetzner VPS, a future home server — gets the
+same shell environment (bash profile, prompt, tmux-on-ssh, agents) minus
+everything that needs a screen, plus rootless podman, systemd user units, and a
+versioned catalog of compose services that `panama server enable` brings up.
+The repo stays public; secrets never enter it.
+
+## Decisions of record
+
+| Decision | Choice |
+|---|---|
+| Repo | One public monorepo. No companion repo. `.gitignore` + a secrets contract keep it safe |
+| Role | Asked first in the interview (`desktop` / `server`), persisted to `~/.local/state/panama/role`; `--server` on `boot`/`install` presets it |
+| Layout | Top-level `server/`, services under `server/containers/<Name>/` (TitleCase, matching junior) |
+| Tracked per service | `compose.yml`, `podman-<name>.service`, `.env.example`. Nothing else |
+| `~/Server` | Real directories. Only `compose.yml` is symlinked into them; units symlink into `~/.config/systemd/user/`. Live `.env` and data stay outside the repo tree entirely |
+| Data dirs | Standardized to `./data/`, applied per service at cutover (a stop/rename/start), never at import |
+| Enablement | `panama server` subcommand family, not the interview |
+| Units | Checked in per service (they legitimately differ), installed as symlinks so edits flow with `panama update` |
+| Updates | `panama update` relinks and `daemon-reload`s but never restarts a running service; it names services whose definitions changed |
+| Nightly image updates | junior's `update-containers.sh` + `podman-update.timer` adopted into the repo (it replaced watchtower for cause) |
+| Codex | Installed everywhere, same pattern as Claude Code |
+| Ports | 80/443/81 only. Anything else is opened by hand and documented in the service's folder |
+| VPS bootstrap | `boot` run as root walks user-creation/keys/sshd-hardening, each step skipped when already true, then re-runs as the user |
+| Out of scope | server.gib (docker; translate later), backup and verify-rootless (own design later), cutover of junior's live services (later, per service) |
+
+## 1. Machine role
+
+- New interview question, asked first: "What is this machine?" — `desktop` or
+  `server` via `gum choose`. `./install --server` (and `boot --server`) records
+  the answer without asking.
+- Persisted to `~/.local/state/panama/role`, plain text, one word. This is the
+  first durable answer Panama keeps, and deliberately so: the transient-answers
+  principle exists to keep *personal* data out of durable paths, and a machine
+  role is a fact about the machine. `install`, `panama update`, `panama doctor`
+  and future migrations read it. A machine with no role file is a desktop —
+  every machine that predates this work is one.
+- On role=server the interview skips: hardware probes (NVIDIA, Secure Boot,
+  firmware — and their bootstrap tools; only `gum` is bootstrapped), the
+  debloat question, and the extras checklist. It keeps: hostname, git
+  identity, gh sign-in, SSH key, user-content.
+
+## 2. Stages by role
+
+Role=desktop is exactly today's behavior. Role=server runs:
+
+| Stage | Server behavior |
+|---|---|
+| `install-packages` | Core + server lists (§3). No flatpaks, no COPRs, no desktop repos. Claude Code and Codex install for both roles |
+| `link-dotfiles` | Universal dot dirs only: `btop`, `nvim`, `ohmyposh`, `tmux`, `vim`, `panama`. Desktop-only (everything else: `hypr`, `quickshell`, `kitty`, `ghostty`, `espanso`, `gtk-*`, `uwsm`, `vicinae`, `wofi`, `xdg-desktop-portal`) is skipped via an explicit classification list in the stage — a new dot dir must be classified or the contract fails. Bash profile links as today; Firefox/wallpaper/cursor seeding skipped |
+| `link-user` | Unchanged (SSH aliases and agent rules are exactly what a server wants) |
+| `setup-identity` | Unchanged |
+| `setup-server` (new) | §4 |
+| `link-server` (new) | Creates `~/Server`, then delegates to `panama-server relink` so link repair has one owner |
+| Skipped | `link-skills` (all three shipped skills are desktop-specific today; revisit when a server skill exists), `change-settings`, `link-vicinae-scripts`, `install-hardware` |
+
+Migrations run on servers too — they are self-guarding no-ops where they do
+not apply. `panama doctor` learns the role: on a server it reports service
+health via `panama-server status` instead of probing desktop units.
+
+## 3. Package restructure
+
+`setup/packages/` gains `core-packages` (both roles) and `server-packages`;
+`initial-packages` keeps what is desktop-only. Nothing is added or dropped for
+desktops — the union of core + desktop lists equals today's set.
+
+- **core-packages**: awk, bat, btop, curl, dnf5-plugins, eza, fontconfig, fzf,
+  gh, git-all, gum, jq, neovim, oh-my-posh, openssh, openssl, podman,
+  podman-compose, python3-dnf, ripgrep, tmux, unzip, wget, zoxide.
+  (`podman` moves here from development-packages; `podman-compose` is what
+  provides `podman compose` on junior — verified, 1.6.0.)
+- **server-packages**: nvm (Node via nvm, as on desktop — agents and MCP
+  servers want it, and Codex installs through npm).
+- **initial-packages** (desktop-only remainder): cargo, rustup, flatpak, fwupd,
+  kitty, ksshaskpass, libselinux-utils, pciutils, python3-neovim,
+  wireguard-tools, vim-X11 — plus everything else currently there that the
+  core list did not claim.
+- Codex CLI: installed in `install-packages` beside the Claude Code step —
+  skip when `codex` is on PATH, `npm install -g @openai/codex` (after nvm/node
+  is available), soft-fail like Claude Code does.
+- The package-hash mechanic in `install` is untouched; new files under
+  `-maxdepth 1` join the hash automatically.
+
+## 4. `setup-server` stage
+
+Idempotent, like every stage; each item checks before acting:
+
+1. `loginctl enable-linger $USER`.
+2. `/etc/sysctl.d/99-rootless-ports.conf` → `net.ipv4.ip_unprivileged_port_start=80`,
+   applied with `sysctl --system` (junior already has exactly this file).
+3. firewalld: open 80, 443, 81 (NPM admin portal), permanent + runtime.
+   Nothing else — reverse-proxy philosophy, ports open only when unavoidable.
+4. `podman network create nginx-bridge` if absent (every compose expects it,
+   `external: true`).
+5. Nightly image updates: `update-containers.sh` is adopted into the repo as
+   `server/scripts/update-containers` (no `.sh`, per repo rule — junior's copy
+   is imported and its `ExecStart`/log paths updated), with
+   `server/systemd/podman-update.service` and `.timer` symlinked into
+   `~/.config/systemd/user/` and the timer enabled. Logs go to
+   `~/Server/logs/` as today.
+
+## 5. The `panama server` command
+
+`bin/panama-server`, surfaced as `panama server <verb>` (README's subcommand
+table and its contract update with it):
+
+- `list` — every service in `server/containers/`, with enabled/active state.
+- `enable <Name>` — create `~/Server/<Name>/`; symlink `compose.yml`; seed
+  `~/Server/<Name>/.env` from `.env.example` when the example exists and the
+  live file does not; if the seeded `.env` still contains placeholder values,
+  stop there and say what to fill in — do **not** start; otherwise symlink the
+  unit, `daemon-reload`, `enable --now`.
+- `disable <Name>` — `disable --now`, remove the unit symlink,
+  `daemon-reload`. `~/Server/<Name>` and its data are never touched.
+- `status [<Name>]` — unit state plus `podman compose ps` per service.
+- `relink` — for every service whose unit symlink exists: refresh both
+  symlinks, `daemon-reload`, and report services whose definitions changed
+  since the last run (so the operator restarts deliberately). Called by the
+  `link-server` stage and by `panama update` on role=server.
+
+Live `.env` files live in `~/Server/<Name>/`, not in the repo tree — the repo
+carries only `.env.example`. `.gitignore` still covers `.env` everywhere under
+`server/` as belt and braces.
+
+## 6. VPS bootstrap (`boot` as root)
+
+`boot` gains a root path. When `EUID == 0`, before anything else, walk these
+steps, each checked and skipped when already true:
+
+1. Ask for a username (default `gib`). Create it with `wheel` if absent; if it
+   exists, use it.
+2. Ensure the user has a password (needed for sudo) — `passwd` interactively
+   if none is set.
+3. Copy root's `authorized_keys` to the user when the user has none.
+4. Offer sshd hardening (yes/no, default yes): drop-in under
+   `/etc/ssh/sshd_config.d/` with `PermitRootLogin no`,
+   `PasswordAuthentication no`, then reload sshd. Skipped when already set.
+5. Move/clone the checkout under the user's home, chown it, and re-exec
+   `install --server` as that user.
+
+Run as a normal user, `boot` behaves exactly as today (plus passing `--server`
+through).
+
+## 7. Importing junior's services (the first population)
+
+A one-time operation, performed against junior over SSH, that **never stops,
+starts, or restarts anything**. For each dedicated compose dir:
+
+1. Copy `compose.yml` into `server/containers/<Name>/`.
+2. Take the **installed** unit (`~/.config/systemd/user/podman-<name>.service`)
+   as truth; diff against the copy in the service dir and flag any divergence.
+3. Move inline secrets out: secret-bearing values in the repo copy of
+   `compose.yml` become `${VAR}` interpolations; extracted values are written
+   to `~/Server/<Name>/.env` **on junior** (inert until cutover — the live
+   compose still carries them inline); `.env.example` with placeholder values
+   is committed. Services already using `.env`/`env_file` get an
+   `.env.example` generated from the live `.env` with values blanked.
+4. Verify without touching the stack: `podman compose config` rendered from
+   the repo copy (with the live `.env`) must match the render of the live
+   compose. Volume paths are kept **verbatim** at import (`volumes/`,
+   `volume/`, `pg_data/` and all) so this diff can be exact; the `data/`
+   standardization happens per service at cutover.
+5. Commit. The secrets contract (§8) gates every one of these commits.
+
+**Import list (23):** Adminer, Agentchat, Authentik, Beszel, CalCom,
+ConvexMonorepo, Documenso, Gitea, Infisical, ISponsorBlockTV, LashAddict, N8n,
+Nginx_Proxy_Manager, Penpot, Portainer, PostgreSQL, Rustdesk, Spoon, StPeteIT,
+Tools, Uptime, UseSend, Vaultwarden. CalCom, Documenso, Penpot and N8n are
+currently parked (dir present, unit not active) — imported anyway, noted as
+parked in the catalog.
+
+**Excluded:** Watchtower (retired — replaced by the update timer after it took
+gitea down; its dir stays on junior until cleanup), Completeuphoria,
+sierraandtyler, Bang (source checkouts deployed by CI from their own repos —
+Panama owning a second copy of their compose would drift), `ci/`, `logs/`,
+and junior's loose scripts other than `update-containers.sh` (§4).
+
+**Cutover** (per service, later, not this work): stop → rename the volume dir
+to `data/` and update the repo compose to match → replace live compose with
+the symlink, installed unit with the symlink → `daemon-reload` → start →
+verify. Ten minutes a service, done when Gib chooses, favoring quiet hours.
+
+## 8. Contracts and docs
+
+New `tests/server/`, same discipline as the rest:
+
+- `compose-secrets-contract` — fails when any tracked file under `server/`
+  contains an inline secret-looking value (the `user-content-contract`
+  pattern), when `.gitignore` stops covering `.env` under `server/`, or when
+  `git ls-files` shows a tracked `.env`.
+- `containers-shape-contract` — every `server/containers/<Name>/` carries
+  `compose.yml` + `podman-<name>.service`; every unit's `WorkingDirectory` is
+  `%h/Server/<Name>`; every `${VAR}` used in a compose file appears in that
+  service's `.env.example`.
+- `panama-server-contract` — throwaway `HOME`, stubbed `systemctl`/`podman` on
+  PATH: `enable` creates the right symlinks and seeds `.env`; placeholder
+  values block start; `disable` leaves data alone; `relink` reports changed
+  definitions.
+- `role-contract` — the interview asks role first, `--server` presets it, the
+  role file is written, stages filter correctly per role (via the stage
+  scripts' static/stubbed seams).
+- `dotfile-classification-contract` — every dir in `config/dot/` appears in
+  exactly one of link-dotfiles' universal/desktop lists.
+- README: layout section gains `server/`, the subcommand table gains
+  `panama server`, the pinned contract count moves — `readme-contract` and the
+  interview/link-dotfiles contracts that assert today's behavior update in the
+  same change.
+
+## Phasing
+
+1. **Role plumbing** — interview question, role file, `--server`, package
+   split, stage filters, `setup-server`, boot's root path.
+2. **Server skeleton** — `server/` tree, `panama-server`, `link-server`,
+   update-timer adoption, contracts, README/docs.
+3. **Import** — junior's 23 services into `server/containers/` per §7.
+4. **Later, separately** — per-service cutover on junior; backup and
+   verify-rootless design; server.gib docker→podman translation; a real
+   end-to-end test on a fresh VPS.
+
+Testing note: contracts for phases 1–2 run with stubs and throwaway HOMEs, but
+per standing practice all suite runs batch at the end of the session with
+Gib's go-ahead. Phase 3 touches junior only with read commands plus the inert
+`.env` writes described in §7.