723 lines
30 KiB
Markdown
723 lines
30 KiB
Markdown
# Verification Gate Remediation Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan.
|
|
|
|
**Goal:** Make Panama's contract gate honest, safe by default when requested, diagnostic on failure, and capable of proving the later repository-audit fixes.
|
|
|
|
**Architecture:** Replace the incomplete desktop-hijacking ledger with one explicit manifest covering every collected contract. `bin/panama` remains the public runner: it parses that manifest, applies capability policy, executes each selected contract under a bounded timeout, and preserves its diagnostics. Contracts that currently depend on the calendar, tracked files, the live settings store, or source-text inspection are changed to exercise disposable fixtures and real public commands.
|
|
|
|
**Tech Stack:** Bash 5, Python 3 standard library, Git fixture repositories, existing Panama contract executables, Quickshell test harnesses.
|
|
|
|
**Spec:** `docs/superpowers/specs/2026-08-26-repository-audit-remediation-design.md`, Package 1: Verification gate.
|
|
|
|
## Global Constraints
|
|
|
|
- Preserve the user's existing `config/bash/.bashrc` changes. Do not stage or edit that file.
|
|
- Do not run plain `panama test` during implementation. It is intentionally interactive once this plan lands and several entries manipulate the live desktop.
|
|
- Do not start or restart the production Quickshell, Hyprland, or system services.
|
|
- Use `apply_patch` for every repository edit.
|
|
- Test public behavior. Source inspection may enforce a narrow declarative invariant, but must not substitute for executing `bin/panama` or a helper.
|
|
- Keep every contract self-cleaning on success, failure, signal, and timeout.
|
|
- Commit after each task with only that task's files staged.
|
|
|
|
---
|
|
|
|
## Task 1: Replace the desktop ledger with a complete contract manifest
|
|
|
|
**Files:**
|
|
|
|
- Create: `tests/contracts.manifest`
|
|
- Create: `tests/setup/contract-manifest-contract`
|
|
- Delete: `tests/setup/desktop-hijacking-contract`
|
|
- Delete: `tests/desktop-hijacking`
|
|
- Modify: `README.md`
|
|
|
|
### Step 1: Write the failing manifest contract
|
|
|
|
Create `tests/setup/contract-manifest-contract` as an executable Bash contract. It must independently discover contracts exactly as the runner does:
|
|
|
|
```bash
|
|
while IFS= read -r path; do
|
|
[[ -x "$path" || "$path" == *_test.py ]] || continue
|
|
printf 'tests/%s\n' "${path#"$repo_dir/tests/"}"
|
|
done < <(find "$repo_dir/tests" -type f \
|
|
-not -path '*/fixtures/*' -not -path '*__pycache__*' | sort)
|
|
```
|
|
|
|
Parse non-comment manifest lines as two whitespace-separated fields:
|
|
|
|
```text
|
|
<comma-separated-capabilities> <repo-relative-contract-path>
|
|
```
|
|
|
|
Assert all of the following:
|
|
|
|
- every discovered contract appears exactly once;
|
|
- every manifest path is discovered by the runner;
|
|
- paths are sorted lexicographically;
|
|
- `hermetic` appears alone;
|
|
- non-hermetic entries use only `live-host`, `live-compositor`, `live-desktop`, `network`, and `privileged`;
|
|
- every non-hermetic entry has a directly preceding comment explaining the external effect;
|
|
- no path or capability is duplicated on one line;
|
|
- the manifest contains at least one entry for each capability except `privileged`, which is supported but not currently required by a contract.
|
|
|
|
Include a parser-fixture section in the same contract. Copy the manifest parser logic into a temporary malformed manifest for each rejection case (missing contract, stale path, duplicate path, unknown capability, `hermetic,network`, unsorted paths, and uncommented non-hermetic entry) and assert each fixture fails with a message naming the defect.
|
|
|
|
Run:
|
|
|
|
```bash
|
|
chmod +x tests/setup/contract-manifest-contract
|
|
tests/setup/contract-manifest-contract
|
|
```
|
|
|
|
Expected: FAIL because `tests/contracts.manifest` does not exist.
|
|
|
|
### Step 2: Add the complete manifest
|
|
|
|
Add one line for each contract discovered by the command in Step 1. Start every path not named below as `hermetic`. Apply this audited non-hermetic classification exactly:
|
|
|
|
| Capabilities | Contract |
|
|
|---|---|
|
|
| `live-compositor,live-desktop` | `tests/quickshell/activity-indicator-contract` |
|
|
| `live-desktop` | `tests/quickshell/activity-state-contract` |
|
|
| `live-host` | `tests/quickshell/adwaita-accent-contract` |
|
|
| `live-host` | `tests/quickshell/application-volume-contract` |
|
|
| `live-compositor,live-desktop` | `tests/quickshell/calendar-agenda-contract` |
|
|
| `live-host` | `tests/quickshell/calendar-agenda-helper-contract` |
|
|
| `live-compositor,live-desktop` | `tests/quickshell/cheatsheet-contract` |
|
|
| `live-host` | `tests/quickshell/connectivity-contract` |
|
|
| `live-compositor,live-desktop` | `tests/quickshell/control-center-contract` |
|
|
| `live-compositor,live-desktop` | `tests/quickshell/control-center-services-contract` |
|
|
| `live-desktop` | `tests/quickshell/default-apps-roles-contract` |
|
|
| `live-host` | `tests/quickshell/default-apps-family-contract` |
|
|
| `live-desktop` | `tests/quickshell/display-arrangement-contract` |
|
|
| `live-compositor,live-desktop` | `tests/quickshell/displays-contract` |
|
|
| `live-host` | `tests/quickshell/disks-contract` |
|
|
| `live-desktop` | `tests/quickshell/dock-position-contract` |
|
|
| `live-compositor` | `tests/quickshell/enum-hypr-map-contract` |
|
|
| `network` | `tests/setup/extras-contract` |
|
|
| `live-compositor,live-desktop` | `tests/quickshell/focus-session-contract` |
|
|
| `live-desktop` | `tests/quickshell/focus-session-expiry` |
|
|
| `live-desktop` | `tests/quickshell/focus-session-restart` |
|
|
| `live-host` | `tests/quickshell/firewall-contract` |
|
|
| `live-host` | `tests/quickshell/gaming-contract` |
|
|
| `live-desktop` | `tests/quickshell/health-ui-contract` |
|
|
| `live-compositor` | `tests/hypr/keybind-categories-contract` |
|
|
| `live-compositor` | `tests/quickshell/keybind-rebind-contract` |
|
|
| `live-compositor` | `tests/quickshell/keybinds-contract` |
|
|
| `network` | `tests/quickshell/home-assistant-helper-contract` |
|
|
| `live-compositor,live-desktop` | `tests/quickshell/my-home-settings-contract` |
|
|
| `live-host` | `tests/quickshell/overview-window-actions-contract` |
|
|
| `live-compositor,live-desktop` | `tests/quickshell/overview-live-actions` |
|
|
| `live-compositor,live-desktop` | `tests/quickshell/overview-search-contract` |
|
|
| `live-desktop` | `tests/quickshell/panama-action-ipc-contract` |
|
|
| `live-compositor,live-desktop` | `tests/quickshell/phone-page-contract` |
|
|
| `live-host` | `tests/quickshell/polkit-agent-contract` |
|
|
| `live-host` | `tests/quickshell/printers-contract` |
|
|
| `live-compositor,live-desktop` | `tests/quickshell/scratchpad-shelf-contract` |
|
|
| `live-compositor` | `tests/quickshell/schema-hypr-shape-contract` |
|
|
| `live-desktop` | `tests/quickshell/screen-intelligence-contract` |
|
|
| `live-compositor,live-desktop` | `tests/quickshell/settings-hyprland-write-contract` |
|
|
| `live-compositor,live-desktop` | `tests/quickshell/settings-pages-contract` |
|
|
| `live-compositor,live-desktop` | `tests/quickshell/settings-system-contract` |
|
|
| `live-compositor,live-desktop` | `tests/quickshell/settings-window-contract` |
|
|
| `live-compositor,live-desktop` | `tests/quickshell/settings-write-sweep-contract` |
|
|
| `live-host` | `tests/quickshell/sharing-contract` |
|
|
| `live-compositor,live-desktop` | `tests/quickshell/signal-glass-contract` |
|
|
| `live-host` | `tests/quickshell/snapshots-contract` |
|
|
| `live-host` | `tests/quickshell/sound-page-contract` |
|
|
| `live-host` | `tests/quickshell/ssh-keys-contract` |
|
|
| `live-desktop` | `tests/quickshell/status-events-contract` |
|
|
| `live-compositor,live-desktop` | `tests/quickshell/switcher-contract` |
|
|
| `live-host` | `tests/quickshell/updates-contract` |
|
|
| `live-host` | `tests/quickshell/user-accounts-contract` |
|
|
| `live-host` | `tests/setup/hardware-predicates-contract` |
|
|
| `live-compositor,live-desktop` | `tests/quickshell/welcome-contract` |
|
|
|
|
For each non-hermetic line, move the useful explanation from `tests/desktop-hijacking` directly above the entry. Add concise comments for the newly discovered entries describing the exact network, compositor-read, or window-mapping behavior.
|
|
|
|
Run:
|
|
|
|
```bash
|
|
tests/setup/contract-manifest-contract
|
|
```
|
|
|
|
Expected: PASS with the discovered and manifested counts equal.
|
|
|
|
### Step 3: Remove the old ledger and update the documented count
|
|
|
|
Delete `tests/desktop-hijacking` and `tests/setup/desktop-hijacking-contract`. Update the exact contract count in `README.md`; removing one executable and adding one executable keeps the current count at 185 at this point.
|
|
|
|
Run:
|
|
|
|
```bash
|
|
tests/setup/readme-contract
|
|
git diff --check
|
|
```
|
|
|
|
Expected: both PASS.
|
|
|
|
### Step 4: Commit
|
|
|
|
```bash
|
|
git add tests/contracts.manifest tests/setup/contract-manifest-contract \
|
|
tests/setup/desktop-hijacking-contract tests/desktop-hijacking README.md
|
|
git commit -m "Test: Classify every contract capability"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 2: Make `panama test` enforce the manifest and preserve diagnostics
|
|
|
|
**Files:**
|
|
|
|
- Create: `tests/setup/test-runner-contract`
|
|
- Modify: `bin/panama`
|
|
- Modify: `tests/contracts.manifest`
|
|
- Modify: `README.md`
|
|
|
|
### Step 1: Write a public runner contract
|
|
|
|
Create `tests/setup/test-runner-contract`. Build a temporary repository containing a copy of `bin/panama`, a minimal `tests/contracts.manifest`, and executable fixture contracts named `pass-contract`, `fail-contract`, `stderr-contract`, `hang-contract`, `host-contract`, `desktop-contract`, and `network-contract`. Initialize it as a Git repository so the copied CLI behaves like the real installation.
|
|
|
|
The fixtures must record executions beneath the temporary directory and produce these behaviors:
|
|
|
|
```bash
|
|
# pass-contract
|
|
printf 'pass stdout\n'
|
|
|
|
# fail-contract
|
|
printf 'failure stdout\n'
|
|
printf 'failure stderr\n' >&2
|
|
exit 7
|
|
|
|
# stderr-contract
|
|
printf 'warning on success\n' >&2
|
|
|
|
# hang-contract
|
|
trap 'printf terminated >"$PANAMA_TEST_FIXTURE/terminated"; exit 124' TERM
|
|
while :; do sleep 1; done
|
|
```
|
|
|
|
Assert the copied `panama` executable provides these public behaviors:
|
|
|
|
1. `panama test --safe` runs only `hermetic` fixtures and reports each skipped capability count, including `live-host`.
|
|
2. `panama test desktop` in a non-TTY fails without running the fixture and tells automation to pass `--allow live-desktop`.
|
|
3. `panama test --allow live-desktop desktop` runs it without a prompt.
|
|
4. Multiple `--allow` flags are repeatable and capability-specific; allowing desktop does not allow network.
|
|
5. An unknown flag, a second pattern, an unknown capability, and `--safe --allow live-desktop` are usage errors.
|
|
6. `PANAMA_TEST_TIMEOUT_SECONDS=1 panama test hang` terminates the fixture, reports a timeout, and exits nonzero.
|
|
7. A failed contract prints both captured stdout and captured stderr.
|
|
8. A successful contract with stderr prints a warning and the captured stderr.
|
|
9. Successful stdout stays hidden to keep the normal run scannable.
|
|
10. A pattern matching only skipped contracts is a distinct error from no pattern match.
|
|
11. Capture files are removed after success and failure.
|
|
12. `panama contracts <file>` labels hits with all manifest capabilities.
|
|
|
|
Run:
|
|
|
|
```bash
|
|
chmod +x tests/setup/test-runner-contract
|
|
tests/setup/test-runner-contract
|
|
```
|
|
|
|
Expected: FAIL against the current runner at the first manifest-policy assertion.
|
|
|
|
### Step 2: Replace ledger parsing with manifest parsing
|
|
|
|
In `bin/panama`, replace `DESKTOP_HIJACKING_LEDGER` and `hijacking_entries` with:
|
|
|
|
```bash
|
|
CONTRACT_MANIFEST="tests/contracts.manifest"
|
|
CONTRACT_CAPABILITIES=(hermetic live-host live-compositor live-desktop network privileged)
|
|
|
|
contract_manifest_entries() {
|
|
local line capabilities path
|
|
while IFS= read -r line || [[ -n "$line" ]]; do
|
|
line="${line%%#*}"
|
|
read -r capabilities path _ <<<"$line"
|
|
[[ -n "${capabilities:-}" && -n "${path:-}" ]] || continue
|
|
printf '%s\t%s\n' "$path" "$capabilities"
|
|
done < "$PANAMA_DIR/$CONTRACT_MANIFEST"
|
|
}
|
|
```
|
|
|
|
Do not make a missing or malformed manifest mean “everything is safe.” Before either `cmd_test` or `cmd_contracts` consumes manifest entries, the public CLI independently validates the complete actual manifest: exactly two fields, known/non-empty/non-duplicate capabilities, exclusive `hermetic`, unique sorted paths, the executable/`*_test.py` fixture-excluded discovery set, and a non-empty directly preceding comment for every non-hermetic entry. Missing discovered contracts and stale manifest paths are fatal before selection or execution. Keep `tests/setup/contract-manifest-contract` as an independent validator rather than sourcing runtime code; a later contract cannot protect earlier execution.
|
|
|
|
### Step 3: Implement argument and capability policy
|
|
|
|
Parse position-independent arguments with one optional pattern and repeatable `--allow <capability>`. Apply these rules:
|
|
|
|
- `--safe` means only `hermetic` entries;
|
|
- `--safe` and any `--allow` together are a usage error;
|
|
- `--allow hermetic` is a usage error because hermetic contracts need no grant;
|
|
- plain `panama test` selects all matching manifest entries;
|
|
- selected non-hermetic capabilities missing from `--allow` require confirmation only when stdin and stderr are TTYs;
|
|
- non-TTY execution fails before running anything and names every missing capability;
|
|
- the prompt lists selected contract count and each external capability; default is no;
|
|
- there is no `--allow-all` path.
|
|
|
|
Use associative arrays for selected grants, manifest capabilities, and skipped counts so ordering stays deterministic.
|
|
|
|
### Step 4: Bound execution and retain output
|
|
|
|
At the beginning of `cmd_test`, validate:
|
|
|
|
```bash
|
|
local timeout_seconds="${PANAMA_TEST_TIMEOUT_SECONDS:-180}"
|
|
[[ "$timeout_seconds" =~ ^[1-9][0-9]*$ ]] || {
|
|
err 'PANAMA_TEST_TIMEOUT_SECONDS must be a positive integer.'
|
|
return 2
|
|
}
|
|
```
|
|
|
|
The public CLI process must own contract supervision. Check that `mktemp -d` succeeds before constructing any capture path, then install top-level `EXIT`, `INT`, and `TERM` cleanup around the checked directory. Launch the active `timeout` asynchronously and retain its PID/process-group ownership. On INT or TERM sent to the exact CLI PID, signal the active timeout/process group, wait for it, remove capture storage, and exit 130 or 143. On normal completion, remove capture storage and clear the temporary traps without changing the aggregate test status. For each contract, run either `python3 path` or the executable through:
|
|
|
|
```bash
|
|
timeout --signal=TERM --kill-after=5 "$timeout_seconds" \
|
|
"${runner[@]}" >"$stdout_file" 2>"$stderr_file"
|
|
```
|
|
|
|
Capture the exit status explicitly instead of relying on `set -e`. Treat status 124 or 137 as timeout. On failure, print a labeled stdout section when nonempty and a labeled stderr section when nonempty. On success, suppress stdout; if stderr is nonempty, call `warn` and print it. Continue through the selected suite and return nonzero if any contract failed or timed out.
|
|
|
|
### Step 5: Update `panama contracts`
|
|
|
|
Load the same manifest mapping and render labels such as:
|
|
|
|
```text
|
|
tests/quickshell/settings-pages-contract [live-compositor,live-desktop]
|
|
tests/setup/readme-contract [hermetic]
|
|
```
|
|
|
|
The grep-based relationship search remains unchanged. Its capability label must come only from the manifest.
|
|
|
|
### Step 6: Make the tests pass and update the count
|
|
|
|
Add `hermetic tests/setup/test-runner-contract` to `tests/contracts.manifest`. Update the README count from 185 to 186.
|
|
|
|
Run:
|
|
|
|
```bash
|
|
tests/setup/contract-manifest-contract
|
|
tests/setup/test-runner-contract
|
|
tests/setup/readme-contract
|
|
./bin/panama test --safe test-runner
|
|
```
|
|
|
|
Expected: all PASS. The final command reports one passing contract and does not request permission.
|
|
|
|
### Step 7: Commit
|
|
|
|
```bash
|
|
git add bin/panama tests/contracts.manifest tests/setup/test-runner-contract README.md
|
|
git commit -m "Fix: Make contract execution safe and diagnostic"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 3: Remove the calendar-pinned agent usage failure
|
|
|
|
**Files:**
|
|
|
|
- Modify: `tests/quickshell/agent-usage-contract`
|
|
|
|
### Step 1: Prove the current fixture is date-dependent
|
|
|
|
Run:
|
|
|
|
```bash
|
|
tests/quickshell/agent-usage-contract
|
|
```
|
|
|
|
Expected before the change: FAIL on `todayTotalTokens` when the current day is not 2026-08-25.
|
|
|
|
### Step 2: Generate the session path and timestamps dynamically
|
|
|
|
Replace the fixed `2026/08/25` directory and timestamps with local-day fixture values, matching the collector's `datetime.now()` and `astimezone()` behavior:
|
|
|
|
```bash
|
|
session_day="$(date +%Y/%m/%d)"
|
|
session_date="$(date +%Y-%m-%d)"
|
|
session_offset="$(date +%:z)"
|
|
mkdir -p "$work/codex/sessions/$session_day"
|
|
|
|
jq -nc --arg timestamp "${session_date}T10:00:00${session_offset}" \
|
|
'{type:"token_count",timestamp:$timestamp,payload:{type:"token_count",info:{total_token_usage:{input_tokens:999999,output_tokens:999999},last_token_usage:{input_tokens:1200,cached_input_tokens:1000,output_tokens:300}}}}'
|
|
```
|
|
|
|
Write two records at local `10:00:00` and `10:05:00` with `session_offset`, preserving the existing expectation of 3,000 last-turn tokens. Do not add a production clock override: only the fixture was wrong.
|
|
|
|
### Step 3: Verify and commit
|
|
|
|
Run:
|
|
|
|
```bash
|
|
tests/quickshell/agent-usage-contract
|
|
./bin/panama test --safe agent-usage
|
|
```
|
|
|
|
Expected: both PASS with `todayTotalTokens == 3000`.
|
|
|
|
```bash
|
|
git add tests/quickshell/agent-usage-contract
|
|
git commit -m "Test: Make agent usage fixtures date independent"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 4: Replace regex secret scanning with semantic fixtures
|
|
|
|
**Files:**
|
|
|
|
- Create: `tests/server/scan-tracked-secrets.py`
|
|
- Create: `tests/server/fixtures/secrets/clean/compose.yml`
|
|
- Create: `tests/server/fixtures/secrets/clean/.env.example`
|
|
- Create: `tests/server/fixtures/secrets/clean/README.md`
|
|
- Create: `tests/server/fixtures/secrets/leaked/compose.yml`
|
|
- Create: `tests/server/fixtures/secrets/leaked/.env.example`
|
|
- Modify: `tests/server/compose-secrets-contract`
|
|
|
|
### Step 1: Add false-positive and true-positive fixtures
|
|
|
|
The clean fixture must include:
|
|
|
|
- Compose interpolation such as `POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}`;
|
|
- `CHANGE_ME`, empty values, and boolean switches in `.env.example`;
|
|
- prose containing `password: example`, `secret key`, and the current server README sentence that triggered the audit.
|
|
|
|
The leaked fixture must include:
|
|
|
|
- a literal YAML credential `POSTGRES_PASSWORD: fixture-should-be-rejected`;
|
|
- a literal env credential `API_TOKEN=fixture-should-be-rejected`.
|
|
- a plain-text PEM private-key header with an exact `path:line: private key` finding;
|
|
- plain-text synthetic `sk-ant-`, minimum-supported-length `ghp_`, and `xoxb-` signatures, each with an exact `path:line: provider token` finding.
|
|
|
|
Add clean plain-text near-misses for each signature. Keep semantic fixture credentials intentionally invalid and signature fixtures clearly synthetic while still matching the supported signature shapes.
|
|
|
|
### Step 2: Write the scanner with only the Python standard library
|
|
|
|
`tests/server/scan-tracked-secrets.py` takes a root directory followed by tracked paths on argv. It must:
|
|
|
|
- parse `.env` and `.env.example` lines as key/value records;
|
|
- parse YAML Compose files by indentation-aware `KEY: VALUE` / `- KEY=VALUE` environment entries and top-level secret-bearing scalar assignments;
|
|
- treat `${VAR}`, `${VAR:-default}`, empty strings, `CHANGE_ME`, and booleans as non-secret;
|
|
- ignore keys ending `_FILE`, `_PATH`, `_NAME`, or `_KEY_NAME`;
|
|
- scan all text file types only for PEM private-key headers and known provider-token signatures;
|
|
- never interpret Markdown credential-shaped prose as a YAML assignment;
|
|
- print `path:line: key` for semantic leaks and return 1 when any are found.
|
|
|
|
Keep the parser deliberately scoped to Compose and env syntax; do not add a general YAML dependency that the installer does not provide.
|
|
|
|
### Step 3: Make the contract prove both directions
|
|
|
|
Before scanning the real tracked `server/` paths, have `tests/server/compose-secrets-contract` call the scanner on the clean fixture and require success, then on each leaked fixture and require failure with the expected key named. Finally pass the actual paths from:
|
|
|
|
```bash
|
|
git -C "$repo_dir" ls-files 'server/**' 'server/*'
|
|
```
|
|
|
|
Keep the existing `.gitignore` and tracked-`.env` assertions.
|
|
|
|
Run:
|
|
|
|
```bash
|
|
tests/server/compose-secrets-contract
|
|
```
|
|
|
|
Expected: PASS; the README prose is accepted while both literal fixture secrets are rejected.
|
|
|
|
### Step 4: Commit
|
|
|
|
```bash
|
|
git add tests/server/scan-tracked-secrets.py tests/server/fixtures/secrets \
|
|
tests/server/compose-secrets-contract
|
|
git commit -m "Test: Scan compose secrets by data shape"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 5: Make the update contract execute the real CLI
|
|
|
|
**Files:**
|
|
|
|
- Modify: `tests/setup/update-command-contract`
|
|
|
|
### Step 1: Replace the source-body assertions with a failing CLI fixture
|
|
|
|
Keep the existing `install --upgrade` stage and package-hash assertions, but replace the `sed`/`grep` inspection of `cmd_update` and `cmd_sync` with disposable Git repositories:
|
|
|
|
1. create a bare `origin.git`;
|
|
2. create an `upstream` clone with a copied `bin/panama` and a stub executable `install`;
|
|
3. push the initial branch;
|
|
4. clone it to `machine`;
|
|
5. append an upstream commit;
|
|
6. run `machine/bin/panama update` with `PANAMA_UPDATE_FIXTURE_LOG` consumed by the copied stub installer.
|
|
|
|
The installer stub must record its argv and optionally fail according to `PANAMA_UPDATE_INSTALL_RC`.
|
|
|
|
Add assertions that the real CLI:
|
|
|
|
- fast-forwards a clean machine clone to the new upstream commit;
|
|
- invokes the fixture installer with `--upgrade`;
|
|
- returns the installer's nonzero status;
|
|
- leaves `panama sync` separate from the installer path;
|
|
- on a dirty-tree stash-pop conflict, leaves no conflict markers and preserves a recoverable stash.
|
|
|
|
Do not yet assert that installation happens after WIP restoration. Package 3 of the approved design changes that ordering and will extend this same executable fixture with the new failing assertion before changing production code.
|
|
|
|
### Step 2: Repair status propagation in the contract harness
|
|
|
|
Change `run_install` so it returns the installer's status rather than `cat`'s status:
|
|
|
|
```bash
|
|
local status=0
|
|
PATH="$root/shim:$PATH" PANAMA_PATH="$root" PANAMA_RAN="$root/ran" \
|
|
XDG_STATE_HOME="$root/state" bash "$root/install" "$@" \
|
|
>"$root/out" 2>&1 || status=$?
|
|
cat "$root/ran"
|
|
return "$status"
|
|
```
|
|
|
|
Replace the indented-assignment parser with a tolerant anchored expression:
|
|
|
|
```python
|
|
match = re.match(r'^\s*STAGES=\((.*)\)\s*$', line)
|
|
```
|
|
|
|
Fail explicitly when the assignment is not found. Do not run parser code in process substitution whose exit status is ignored.
|
|
|
|
### Step 3: Verify and commit
|
|
|
|
Run:
|
|
|
|
```bash
|
|
tests/setup/update-command-contract
|
|
./bin/panama test --safe update-command
|
|
```
|
|
|
|
Expected: both PASS and the fixture log proves `bin/panama update` actually ran.
|
|
|
|
```bash
|
|
git add tests/setup/update-command-contract
|
|
git commit -m "Test: Exercise updates through the real command"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 6: Stop contracts from mutating tracked files or reading live preferences
|
|
|
|
**Files:**
|
|
|
|
- Modify: `config/dot/quickshell/scripts/panama-settings-docs`
|
|
- Modify: `tests/quickshell/settings-docs-contract`
|
|
- Modify: `tests/quickshell/settings-sync-contract`
|
|
|
|
### Step 1: Add a failing output-path contract for generated settings docs
|
|
|
|
Extend `tests/quickshell/settings-docs-contract` to copy the committed document to a temporary path, append drift to the copy, and run:
|
|
|
|
```bash
|
|
"$generator" --check --output "$scratch/settings.md"
|
|
```
|
|
|
|
Capture `git diff -- docs/settings.md` before and after the contract and require byte-for-byte equality. Run the contract now.
|
|
|
|
Expected: FAIL because the generator does not accept `--output` and the current test still edits the tracked document.
|
|
|
|
### Step 2: Add a real output seam to the generator
|
|
|
|
Add:
|
|
|
|
```python
|
|
parser.add_argument("--output", type=pathlib.Path, default=OUTPUT)
|
|
```
|
|
|
|
Resolve relative explicit paths against the current working directory, use that path for write/check behavior, and keep the existing default unchanged. `--stdout` must remain mutually exclusive with writing but may coexist with no explicit output.
|
|
|
|
Update the contract so the stale-copy check never writes `docs/settings.md`. Its trap should only remove the temporary directory.
|
|
|
|
Run:
|
|
|
|
```bash
|
|
tests/quickshell/settings-docs-contract
|
|
git diff --exit-code -- docs/settings.md
|
|
```
|
|
|
|
Expected: PASS and no diff.
|
|
|
|
### Step 3: Seed settings-sync from a fixture instead of the live settings store
|
|
|
|
In `tests/quickshell/settings-sync-contract`, set `XDG_CONFIG_HOME="$work/source-config"` before the first export and create a representative `panama/settings.json` there from schema-valid values. Include at least:
|
|
|
|
- one boolean;
|
|
- one integer;
|
|
- one numeric enum;
|
|
- one string;
|
|
- one setting omitted from the import bundle to prove merge semantics.
|
|
|
|
Remove every read of `${XDG_CONFIG_HOME:-$HOME/.config}/panama/settings.json` that occurs before the temporary home is established. Keep the allow-list completeness assertion, but compare against the explicit fixture rather than the user's preferences.
|
|
|
|
Run:
|
|
|
|
```bash
|
|
tests/quickshell/settings-sync-contract
|
|
```
|
|
|
|
Expected: PASS without reading or writing the real settings store.
|
|
|
|
### Step 4: Commit
|
|
|
|
```bash
|
|
git add config/dot/quickshell/scripts/panama-settings-docs \
|
|
tests/quickshell/settings-docs-contract tests/quickshell/settings-sync-contract
|
|
git commit -m "Test: Isolate generated docs and settings fixtures"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 7: Isolate the dock geometry contract
|
|
|
|
**Files:**
|
|
|
|
- Modify: `tests/quickshell/dock-position-contract`
|
|
|
|
### Step 1: Write the safety assertions before changing the harness
|
|
|
|
At contract start, record whether `config/dot/quickshell/dock-position-probe.qml` exists and its checksum if present. At every exit, assert the source tree matches that initial state. Also fail if the contract reads `${HOME}/.config/panama/settings.json`.
|
|
|
|
Run:
|
|
|
|
```bash
|
|
tests/quickshell/dock-position-contract
|
|
```
|
|
|
|
Expected: FAIL because the contract creates its probe inside the tracked QML tree and reads live settings.
|
|
|
|
### Step 2: Copy the QML tree and seed deterministic settings
|
|
|
|
After creating `work`, copy the Quickshell tree to a fixture root:
|
|
|
|
```bash
|
|
fixture_shell="$work/quickshell"
|
|
cp -a "$shell_dir/." "$fixture_shell/"
|
|
probe="$fixture_shell/dock-position-probe.qml"
|
|
```
|
|
|
|
Write the probe only to `$probe`. Seed `$work/config/panama/settings.json` with the schema defaults needed by the dock plus the position under test; do not copy the user's settings. Run each `qs -p` invocation from `$fixture_shell` with `XDG_CONFIG_HOME="$work/config"`.
|
|
|
|
The contract still maps isolated panel surfaces into the current Wayland session. Keep its manifest capability as `live-desktop`: moving its files and settings into temporary storage removes mutation risk but does not make the visible surfaces hermetic.
|
|
|
|
### Step 3: Verify isolation under an explicit desktop grant
|
|
|
|
Run:
|
|
|
|
```bash
|
|
tests/quickshell/dock-position-contract
|
|
tests/setup/contract-manifest-contract
|
|
./bin/panama test --allow live-desktop dock-position
|
|
```
|
|
|
|
Expected: all PASS, the source-tree checksum is unchanged, and the manifest entry remains `live-desktop`.
|
|
|
|
### Step 4: Commit
|
|
|
|
```bash
|
|
git add tests/quickshell/dock-position-contract
|
|
git commit -m "Test: Isolate dock geometry fixtures"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 8: Update operator documentation and run the Package 1 gate
|
|
|
|
**Files:**
|
|
|
|
- Modify: `README.md`
|
|
- Modify: `.claude/skills/panama/SKILL.md`
|
|
- Modify: `skills/panama-desktop/SKILL.md`
|
|
- Modify: `bin/panama`
|
|
|
|
### Step 1: Update the public command documentation
|
|
|
|
Document:
|
|
|
|
```text
|
|
panama test --safe
|
|
panama test --allow live-host updates
|
|
panama test --allow live-compositor keybinds
|
|
PANAMA_TEST_TIMEOUT_SECONDS=300 panama test --safe
|
|
```
|
|
|
|
State that:
|
|
|
|
- `tests/contracts.manifest` classifies every contract;
|
|
- `--safe` means hermetic only;
|
|
- plain full runs prompt in a terminal before any non-hermetic contract;
|
|
- automation must grant each required capability with repeatable `--allow`;
|
|
- failures print captured stdout/stderr and every contract has a default 180-second outer timeout.
|
|
|
|
Remove current claims that `--safe` is defined by `tests/desktop-hijacking`. Update `bin/panama`'s top comment, usage text, `cmd_test` comments, and `cmd_contracts` comments to match behavior. Leave historical design specs unchanged.
|
|
|
|
### Step 2: Run focused verification
|
|
|
|
```bash
|
|
bash -n bin/panama \
|
|
tests/setup/contract-manifest-contract \
|
|
tests/setup/test-runner-contract \
|
|
tests/setup/update-command-contract \
|
|
tests/quickshell/agent-usage-contract \
|
|
tests/quickshell/settings-docs-contract \
|
|
tests/quickshell/settings-sync-contract \
|
|
tests/server/compose-secrets-contract
|
|
python3 -m py_compile \
|
|
config/dot/quickshell/scripts/panama-settings-docs \
|
|
tests/server/scan-tracked-secrets.py
|
|
tests/setup/contract-manifest-contract
|
|
tests/setup/test-runner-contract
|
|
tests/setup/update-command-contract
|
|
tests/quickshell/agent-usage-contract
|
|
tests/quickshell/settings-docs-contract
|
|
tests/quickshell/settings-sync-contract
|
|
tests/server/compose-secrets-contract
|
|
tests/setup/readme-contract
|
|
git diff --check
|
|
```
|
|
|
|
Expected: every command exits 0.
|
|
|
|
### Step 3: Run the complete hermetic gate
|
|
|
|
```bash
|
|
./bin/panama test --safe
|
|
```
|
|
|
|
Expected: all selected hermetic contracts pass; the summary reports skipped counts for `live-host`, `live-compositor`, `live-desktop`, and `network`; no confirmation appears; no production shell or service is restarted.
|
|
|
|
If a manifest-labeled hermetic contract reaches the network, live compositor, live settings, root prompt, or tracked-file write, stop and reclassify or isolate it before continuing. Do not grant a capability merely to make this gate green.
|
|
|
|
### Step 4: Confirm worktree scope
|
|
|
|
```bash
|
|
git status --short
|
|
git diff -- config/bash/.bashrc
|
|
```
|
|
|
|
Expected: the pre-existing `.bashrc` modification remains unstaged and unchanged from its pre-plan state; only Package 1 files are staged or committed.
|
|
|
|
### Step 5: Commit
|
|
|
|
```bash
|
|
git add README.md .claude/skills/panama/SKILL.md skills/panama-desktop/SKILL.md bin/panama
|
|
git commit -m "Docs: Explain contract capability gates"
|
|
```
|
|
|
|
### Step 6: Record the package handoff
|
|
|
|
Capture these facts in the implementation summary before beginning Package 2:
|
|
|
|
- final hermetic pass/fail/skip counts;
|
|
- every non-hermetic capability (`live-host`, `live-compositor`, `live-desktop`, `network`, and `privileged`) and its contract count;
|
|
- focused verification commands and exit codes;
|
|
- confirmation that `config/bash/.bashrc` stayed out of all commits;
|
|
- any contract retained as non-hermetic after an attempted isolation, with the exact reason.
|