test(bastion): cover the rescue boot path and record what it exposed
Some checks failed
CI/CD / lint (pull_request) Failing after 9s
CI/CD / test (pull_request) Failing after 8s
CI/CD / typecheck (pull_request) Failing after 23s
CI/CD / build (pull_request) Has been skipped
CI/CD / publish-rpm (pull_request) Has been skipped
CI/CD / publish-deb (pull_request) Has been skipped

`provision debug` is the lab's recovery tool of last resort and had no
integration coverage on any architecture. Adding it -- x86_64 on KVM so
it runs in ~15 minutes, plus the aarch64 equivalent -- showed the rescue
environment coming up correctly but nothing ever listening on port 22.

Reproduced on both architectures, so it is neither ARM-specific nor an
emulation artefact, and it is orthogonal to the multi-arch work: x86_64
is unchanged by that. Documented in ARCHITECTURE.md with the leads worth
checking, rather than left as a silent gap.

Also restructures the ARM rescue suite to seed the machine into state
instead of discovering it first. That mirrors the DGX Spark situation --
SSH-onboarded, never PXE-discovered, architecture known only from its
record -- and holds the test to one emulated boot, since each spends
~15 of its ~18 minutes fetching Anaconda's stage2 under TCG.

KEEP_VM=1 leaves the VM up on failure; half-hour emulated runs are too
expensive to pay twice just to see what happened.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nRFZXpKwUVE4SRSHw6GjF
This commit is contained in:
Michal
2026-08-11 15:25:57 +01:00
parent b75a4e0118
commit 346bd80c13
5 changed files with 326 additions and 49 deletions

View File

@@ -481,6 +481,28 @@ Hardcoded `/dev/sda` default broke NVMe-only machines. Fix: default to empty str
### Anaconda Rescue Mode Limitations
`%pre` and `%post` sections do not execute in `inst.rescue` mode. SSH in rescue mode is provided by Anaconda's `inst.sshd` kernel parameter + `sshpw` kickstart directive. Manual setup via `curl bastion:8080/debug-setup.sh | bash` for nc listener.
**Unresolved (2026-08-11): rescue SSH has never been observed working.** Adding the first
integration coverage for `provision debug` (`tests/integration/pxe-rescue.test.ts`) showed the
rescue environment coming up correctly — the bastion serves the kernel and initrd, Anaconda
boots, fetches `debug.ks`, and reaches its installer environment — but **nothing ever listens on
port 22**. Confirmed on aarch64 by probing the port for 30 minutes while the Anaconda environment
was demonstrably running (NetworkManager, polkitd, rsyslog all up), and reproduced on x86_64 with
KVM, so it is not architecture-specific and not an emulation artefact.
This is orthogonal to the multi-architecture work: the same failure occurs on x86_64, which that
work does not touch. Leads worth checking, in order:
- Does `inst.sshd` actually start `sshd` in `inst.rescue` mode, or only in install mode? The
port never opens, so this is the prime suspect — an auth problem would still show an open port.
- `sshkey` may apply only to the *installed* system, leaving the installer environment
password-only via `sshpw`. That would matter once sshd does listen: the test authenticates
key-only (`BatchMode=yes`).
- The `%anaconda`-context directives in `debug.ks` may be skipped entirely when a kickstart is
supplied alongside `inst.rescue`.
Until this is resolved, `provision debug` gets you a booted rescue environment on the console
(including on arm64), but not an SSH shell. The `debug-setup.sh` nc-listener path is the
documented workaround and is unaffected.
---
## Planned Work (Taskmaster)