Fix: Complete SSH bootstrap hardening

This commit is contained in:
Gabriel Brown
2026-08-27 05:19:58 -04:00
parent 98e29eb8f3
commit fc8f226747
9 changed files with 852 additions and 186 deletions
@@ -14,9 +14,21 @@
- Preserve the public `boot --server` command and the non-root desktop bootstrap path.
- Do not reload a real SSH service, edit `/etc/ssh`, create a real account, or run a live privileged check.
- A missing or unsafe target key keeps root/password access unchanged and continues bootstrap.
- A missing or unsafe target key, missing SSH unit, declined prompt, or unsupported
pre-existing Panama drop-in keeps root/password access unchanged and continues bootstrap.
- Candidate validation or reload failure restores the prior drop-in and stops before clone/install handoff.
- Exact safe state is target ownership plus `.ssh` mode `0700` and `authorized_keys` mode `0600`; symlinks, root UID, relative homes, blank/comment-only keys, and foreign ownership are refused.
- Exact safe state is target ownership plus `.ssh` mode `0700` and `authorized_keys`
mode `0600`; symlinks, root UID, relative homes, blank/comment-only keys,
malformed non-comment lines, and foreign ownership are refused.
- The installed policy is `00-panama.conf` with `PermitRootLogin no`,
`PasswordAuthentication no`, and `KbdInteractiveAuthentication no`.
- Before reload, `sshd -t` and root/target `sshd -T -C` checks must prove the
desired effective policy. Earlier main-config precedence therefore fails closed.
- Root-key destination creation and writing run as the target UID. Revalidation
follows, and no same-named primary group is assumed.
- Existing regular drop-ins retain complete metadata on rollback. Symlinks,
directories, FIFOs, and other non-regular objects make hardening unavailable.
- Transaction traps are armed before the first candidate or backup artifact.
- `PANAMA_BOOT_FIXTURE_ROOT` is accepted only by a real non-root process whose stubbed `id -u` reports root. Real root plus that variable must fail closed.
- The new public contract is hermetic and must run under `panama test --safe`.
- Preserve the user's unstaged `/home/gib/.local/share/Panama/config/bash/.bashrc` change outside this worktree.
@@ -75,6 +87,9 @@ Run one table row per unsafe state:
missing
empty
comment-only
malformed-key
mixed-valid-and-malformed-key
malformed-root-key
ssh-directory-symlink
authorized-keys-symlink
directory-wrong-mode
@@ -124,11 +139,22 @@ safe_authorized_keys() {
[[ -d "$ssh_dir" && ! -L "$ssh_dir" && -f "$keys" && ! -L "$keys" ]] || return 1
[[ "$(stat -Lc '%u:%a' "$ssh_dir")" == "$uid:700" ]] || return 1
[[ "$(stat -Lc '%u:%a' "$keys")" == "$uid:600" ]] || return 1
grep -qEv '^[[:space:]]*(#|$)' "$keys"
valid_authorized_keys "$keys"
}
```
Resolve the target home, `/root/.ssh/authorized_keys`, and `/etc/ssh/sshd_config.d` through `system_path`. A copyable root key must be a non-symlinked regular file owned by UID 0, mode `0600`, with at least one nonblank/non-comment line. Never overwrite an existing `authorized_keys`. When it is absent, either create a missing `.ssh` or require an existing `.ssh` to already be a real directory owned by the target UID with mode `0700`; then copy only the key and apply ownership/mode to those two paths. Never use `chown -R`. If `safe_authorized_keys` still fails, print the unavailable message and skip the prompt/transaction.
`valid_authorized_keys` skips blank/comment lines, requires at least one remaining
line, and runs `ssh-keygen -l` on every remaining line. Resolve the target home,
`/root/.ssh/authorized_keys`, and `/etc/ssh/sshd_config.d` through `system_path`.
A copyable root key must be a non-symlinked regular file owned by UID 0, mode
`0600`, whose non-comment lines all parse. Never overwrite an existing
`authorized_keys`. When it is absent, either create a missing `.ssh` or require
an existing `.ssh` to already be a real directory owned by the target UID with
mode `0700`. Run final directory creation and key writing as the target UID,
then revalidate ownership, modes, and key parsing. Do not chown the destination
or assume the user's primary group is named after the user. If
`safe_authorized_keys` still fails, print the unavailable message and skip the
prompt/transaction.
- [ ] **Step 4: Add the hermetic manifest entry and run focused checks**
@@ -181,12 +207,20 @@ Extend the command state with `SSHD_RESULTS` and `RELOAD_RESULTS`, consumed one
success-without-prior-dropin: validate=0 reload=0
success-replaces-prior-dropin: validate=0 reload=0
candidate-invalid: validate=1,0 reload=<none>
effective-root-policy-conflict: syntax=0 root-policy=conflict rollback-validate=0 reload=<none>
effective-target-policy-conflict: syntax=0 root-policy=safe target-policy=conflict rollback-validate=0 reload=<none>
candidate-reload-fails: validate=0,0 reload=1,0
rollback-validation-fails: validate=0,1 reload=1
rollback-reload-fails: validate=0,0 reload=1,1
```
Assert the desired two-line content, validation before reload, only the detected unit, byte-for-byte restoration, restored validation/reload ordering, nonzero status and no install handoff on every transactional failure, no `*.tmp`/`*.backup` residue on success, and retained backup plus recovery commands when rollback fails.
Assert the desired three-line content; syntax and root/target effective validation
before reload; only the detected unit; content and complete metadata restoration;
restored validation/reload ordering; nonzero status and no install handoff on every
transactional failure; no `*.tmp`/`*.backup` residue on success; and retained backup
or no-prior-file removal instructions when rollback fails. Add declined-hardening,
missing-unit, symlink/directory/FIFO drop-in, target-UID normalization, candidate and
backup preparation signals, and actual-root fixture-guard cases.
- [ ] **Step 2: Run the focused contract and confirm it fails on current code**
@@ -212,9 +246,9 @@ detect_ssh_unit() {
}
restore_ssh_dropin() {
if [[ -n "${ssh_backup:-}" && -e "$ssh_backup" ]]; then
if (( ssh_had_prior )); then
local restore
restore="$(mktemp --tmpdir="$sshd_dir" .90-panama.XXXXXX.restore)" || return 1
restore="$(mktemp --tmpdir="$sshd_dir" .00-panama.XXXXXX.restore)" || return 1
cp -a -- "$ssh_backup" "$restore"
mv -f -- "$restore" "$ssh_dropin"
else
@@ -223,7 +257,15 @@ restore_ssh_dropin() {
}
```
Create the candidate with `umask 077; mktemp --tmpdir="$sshd_dir" .90-panama.XXXXXX.tmp`, write exact desired content, and preserve an existing final file in a collision-safe same-directory `mktemp` name ending `.backup`, not `.conf`. Save any prior `EXIT`, `INT`, and `TERM` traps, then arm transaction traps before `mv -f` activates the candidate. The EXIT handler restores only while `ssh_transaction_active=1`; every success or handled failure path restores the prior traps before returning.
Reject an existing `00-panama.conf` unless it is a non-symlink regular file. Save
the prior `EXIT`, `INT`, and `TERM` traps and arm state-aware preparation cleanup
before creating any artifact. Create the candidate with
`umask 077; mktemp --tmpdir="$sshd_dir" .00-panama.XXXXXX.tmp`, write the exact
three-line desired content, and preserve an existing final file with `cp -a` in a
collision-safe same-directory `mktemp` name ending `.backup`, not `.conf`.
Atomically activate with `mv -f`. Preparation signals remove known artifacts without
touching the final path; activated signals restore, validate, and reload. Every
success or handled failure path restores the prior traps before returning.
After activation:
@@ -233,6 +275,11 @@ if ! sshd -t; then
sshd -t || rollback_failed=1
return 1
fi
if ! effective_ssh_policy_is_hardened "$username"; then
restore_ssh_dropin
sshd -t || rollback_failed=1
return 1
fi
if ! systemctl reload "$ssh_unit"; then
restore_ssh_dropin
sshd -t || rollback_failed=1
@@ -241,7 +288,13 @@ if ! systemctl reload "$ssh_unit"; then
fi
```
On clean success set `ssh_transaction_active=0`, clear the local traps, and remove the backup. On rollback failure keep the backup and print its absolute path plus `sshd -t` and `systemctl reload UNIT` recovery commands. Do not continue to clone/install after a hardening transaction returns nonzero.
`effective_ssh_policy_is_hardened` uses `sshd -T -C` for root and target
contexts. Root must report all three denials; target must report both authentication
denials. On clean success clear the transaction state, restore traps, and remove the
backup. On rollback failure with a prior file, keep the metadata-preserving backup
and print its absolute path plus validation/reload commands. With no prior file,
print `rm -f -- /etc/ssh/sshd_config.d/00-panama.conf`, `sshd -t`, and the detected
reload command. Do not continue to clone/install after a transactional failure.
- [ ] **Step 4: Verify failure status, cleanup, and old public behavior**
@@ -281,7 +334,11 @@ git commit -m "Fix: Roll back failed SSH hardening"
- [ ] **Step 1: Write the documentation assertions first**
Extend `tests/setup/readme-contract` to require nearby root-bootstrap prose containing all of: verified target key, `sshd -t`, atomic drop-in, reload rollback, and hardening unavailable without a key. Reject wording that says Panama merely writes the file or that reload failure is ignored.
Extend `tests/setup/readme-contract` to require nearby root-bootstrap prose containing
all of: OpenSSH-parsed target keys, `00-panama.conf`, all three denials, `sshd -t`,
root/target `sshd -T`, atomic installation, reload rollback, and hardening unavailable
without a key. Reject wording that says Panama merely writes the file or that reload
failure is ignored.
- [ ] **Step 2: Run the README contract and confirm the old prose fails**
@@ -295,7 +352,11 @@ Expected: nonzero until README describes the transactional behavior.
- [ ] **Step 3: Update the documentation without claiming a live reload**
State plainly that Panama copies or verifies the target key, offers hardening only with exact safe ownership/modes, validates the complete config, reloads the detected unit, and restores the previous drop-in on failure. State that fixture contracts test these paths and no real daemon reload runs under `panama test --safe`.
State plainly that Panama copies or verifies every target key line with OpenSSH,
offers hardening only with exact safe ownership/modes and a supported regular
drop-in, validates syntax and effective root/target policy, reloads the detected
unit, and restores the previous file with metadata on failure. State that fixture
contracts test these paths and no real daemon reload runs under `panama test --safe`.
- [ ] **Step 4: Run the plan gate**
@@ -33,8 +33,11 @@ real firewall, install a real package, start a service, or apply changes to a se
- Safe target SSH state means a non-root account, an absolute home, a real `.ssh`
directory owned by the target UID with mode `0700`, and a nonempty regular
`authorized_keys` file owned by the target UID with mode `0600`. Symlinks are
refused. Panama may create and normalize files it copied from root, but it does
not take ownership of an unsafe pre-existing target path.
refused. Every nonblank, non-comment key line must parse with OpenSSH tooling.
Panama may create and normalize files it copied from root, but final destination
creation and writing run as the target UID. It does not take ownership of an
unsafe pre-existing target path or assume that the user's primary group has the
same name as the user.
- Failed SSH reload rollback includes restored-config validation and a reload of the
restored configuration, because a command can apply state and still return
nonzero.
@@ -92,16 +95,21 @@ Before offering hardening, Panama verifies:
absolute and nonempty;
- neither the home-relative `.ssh` path nor `authorized_keys` is a symlink;
- `.ssh` and `authorized_keys` have the exact ownership and modes in Decisions;
- `authorized_keys` contains at least one nonblank, non-comment line;
- exactly one installed SSH unit is detected, preferring `sshd.service` and falling
back to `ssh.service` only when the first unit is absent.
- `authorized_keys` contains at least one nonblank, non-comment line, and OpenSSH
parses every such line as a public key;
- an installed SSH unit is detected, preferring `sshd.service` and falling back to
`ssh.service` only when the first unit is absent.
If the target has no key and root has a safe regular key file, Panama copies only that
file, creates `.ssh`, applies `0700/0600`, and changes ownership only on those two
paths. It does not recursively take ownership of an existing directory tree.
file. The target UID creates or normalizes `.ssh` at `0700` and writes
`authorized_keys` at `0600` through an already-open root-key input. Panama then
revalidates exact UID ownership, modes, and OpenSSH key parsing. It does not chown a
target-controlled path or assume a same-named primary group.
If the preconditions fail, Panama prints why hardening is unavailable, keeps
root/password authentication unchanged, and continues the clone/install handoff.
If the key preconditions fail, no SSH unit is installed, or the Panama drop-in path
already names a symlink or non-regular object, Panama prints why hardening is
unavailable, keeps root/password authentication unchanged, and continues the
clone/install handoff. Declining the prompt has the same unchanged-state outcome.
### Transaction
@@ -110,22 +118,32 @@ The desired drop-in is exactly:
```text
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
```
Panama creates the candidate with `umask 077` and `mktemp` in
`/etc/ssh/sshd_config.d`. Its temporary name does not end in `.conf`, so the normal
include glob cannot activate it early. It preserves an existing
`90-panama.conf` in the same directory, arms EXIT/INT/TERM rollback, and atomically
renames the candidate over the final path.
include glob cannot activate it early. It refuses a pre-existing Panama path unless
it is a non-symlink regular file. It preserves an existing `00-panama.conf`, including
its ownership, mode, timestamps, ACLs, and extended attributes, in the same directory.
State-aware EXIT/INT/TERM cleanup is armed before the first candidate or backup
artifact, and the candidate is atomically renamed over the final path.
It then runs `sshd -t` against the complete active configuration and reloads only the
detected unit. Success disarms rollback and removes the backup.
It then runs `sshd -t` against the complete active configuration. Before reload,
`sshd -T -C` must report `permitrootlogin no`, `passwordauthentication no`, and
`kbdinteractiveauthentication no` for the root context. The target-user context must
report both authentication directives as `no`. This fails closed when an earlier
main-config directive wins despite the precedence-safe filename. Panama reloads only
the detected unit after every check passes. Success disarms rollback and removes the
backup.
On validation failure, Panama restores or removes the new drop-in, validates the
restored configuration, and returns nonzero without reloading the rejected candidate.
On reload failure, Panama restores the previous drop-in, validates it, reloads the
restored unit, and returns nonzero. A rollback validation/reload failure preserves the
backup and prints its path plus exact recovery commands.
backup and prints its path plus exact recovery commands. When no prior file existed,
recovery instead instructs the operator to remove `00-panama.conf`, run `sshd -t`,
and reload the detected unit.
Existing drop-ins go through the same desired-content, validation, and reload path;
mere existence is not treated as proof of hardening.
@@ -143,12 +161,18 @@ and `systemctl`, and provides temporary account and filesystem state.
Required cases are:
- missing, empty, comment-only, symlinked, wrong-owner, and wrong-mode target keys;
- missing, empty, comment-only, malformed, mixed valid/malformed, symlinked,
wrong-owner, and wrong-mode target keys;
- safe root-key copy and safe existing target key;
- target-UID copy normalization with a primary group whose name differs from the user;
- declined hardening and no installed SSH unit;
- pre-existing symlink, directory, and FIFO Panama drop-ins;
- successful initial install and replacement of an existing drop-in;
- invalid candidate rollback;
- invalid syntax and conflicting effective-policy rollback;
- failed reload rollback, including restored validation and reload;
- rollback failure retaining its recovery artifact;
- rollback failure retaining its recovery artifact or printing no-prior-file removal;
- INT/TERM during candidate and backup preparation as well as after activation;
- actual-root rejection of `PANAMA_BOOT_FIXTURE_ROOT` in a user namespace;
- exact command ordering and no install handoff after a transactional failure.
## Verified bootstrap and installer inputs