feat(bastion): first-class aarch64 support in the network-PXE path #17

Open
michal wants to merge 11 commits from feat/arm64-pxe-support into main
Owner

Why

On 2026-08-11 spark-2935 (DGX Spark, aarch64) panicked with VFS: Unable to mount root fs on unknown-block(0,0). The obvious recovery tool — labctl provision debug, "PXE boot into Fedora rescue mode" — was useless: the bastion served a single x86_64 kernel to every machine regardless of architecture, so an ARM64 box got a binary its UEFI will not execute.

The boot-ISO path had already solved this. This brings the network PXE path to parity.

What changed

Architecture is resolved, never flagged. No --arch anywhere. /dispatch resolves in order: tracked machine record → iPXE's ${buildarch} reported at boot → configured default. boot.ipxe gained &arch=${buildarch} because DHCP option 93 only ever reaches dnsmasq, never the HTTP endpoint; one script then covers network PXE, UEFI HTTP boot and the boot ISO alike.

Per-arch artifacts. x86_64 keeps unsuffixed /vmlinuz + /initrd.img; aarch64 gets /vmlinuz-aarch64 + /initrd-aarch64.img, its own Fedora mirror, and console=tty0 console=ttyAMA0,115200 instead of nomodeset — which does not mean the same thing on arm64 and can leave a headless machine with no console at all.

--pxe-boot no longer hardcodes labvg. Root device and dracut args come from the machine's record, observed three ways: provision recheck collects them, provision debug --pxe-boot probes over SSH when the machine answers, and debug-setup.sh reports them from a rescue shell (mounting each candidate read-only, picking the one with /etc/fstab and /usr). When nothing is known, dispatch serves rescue rather than guessing a root= that would leave a machine unbootable.

The Sparks are guarded. Machines carry onboard: "pxe" | "ssh" and the vendor_os they must keep running. Installs are refused at both entry points, naming the machine and pointing at provision debug; rescue is never guarded — being unable to reinstall is exactly when you need a rescue shell. Modelled as a fact about the machine, not a blocklist: the refusal follows from "no image in our pipeline restores dgx-os", so adding a SparkOS image is what unblocks these boxes.

Two corrections to the brief

  1. Option 93 handling already existed in dnsmasq.conf.ts. The real bug was httpboot-arm64 matching client-arch 20, which per IANA is pc/at bios boot from http; arm64 UEFI HTTP boot is 19. This is load-bearing, not cosmetic — AAVMF prefers HTTP Boot, and the VM run shows it sending HTTPClient:Arch:00019.

  2. LoadFile2 is supported — no GRUB2 chainload needed. EFI_LOAD_FILE2_PROTOCOL_GUID (4006c0c1-fcb3-403e-996d-4a6c8724e06d) is at offset 0x3bbf0 of ipxe-bootimgs-aarch64-20240119's snponly.efi, with the Linux initrd media device path at 0x3e530. Confirmed empirically below.

Also: Ubuntu publishes no arm64 netboot artifacts, so that combination is refused rather than served an x86 kernel. (Separately and pre-existing: the Ubuntu URL the code already fetches 404s even for amd64, so that path looks dead today. Left alone as out of scope.)

VM evidence

Emulated aarch64 on an isolated libvirt network:

dnsmasq-dhcp: vendor class: PXEClient:Arch:00011:UNDI:003000   -> tag efi-arm64
dnsmasq-dhcp: sent option 67 bootfile-name  ipxe-arm64.efi
dnsmasq-dhcp: vendor class: HTTPClient:Arch:00019:UNDI:003000  -> tag httpboot-arm64

HTTP: GET /ipxe-arm64.efi
HTTP: GET /boot.ipxe
HTTP: GET /dispatch?mac=52:54:00:eb:a6:3c&arch=arm64
HTTP: GET /vmlinuz-aarch64
HTTP: GET /initrd-aarch64.img
POST /api/discover
NEW MACHINE DISCOVERED: 52:54:00:eb:a6:3c

Recorded state: {"arch": "aarch64", "onboard": "pxe", ...}.

LoadFile2 confirmed working: the kernel reached systemd inside the initramfs and went on to run Anaconda. Had iPXE dropped the initrd, that is precisely where it would have panicked with unknown-block(0,0).

x86_64 regression

The four dispatch-served scripts are byte-identical, pinned by a golden fixture dumped from the templates before any change (tests/fixtures/ipxe-x86_64-golden.json). The single intended diff is &arch= in boot.ipxe, asserted explicitly.

 Test Files  25 passed (25)
      Tests  283 passed (283)

Completions are up to date.

Known gap — rescue SSH

Adding the first-ever integration coverage for provision debug exposed that rescue SSH has never been observed working: the environment boots correctly and Anaconda comes up, but nothing listens on port 22.

Evidence, stated at its actual strength because it decides where to look next:

  • aarch64 — direct. Port 22 probed every 20s for 30 minutes while the Anaconda installer environment was demonstrably running (NetworkManager, polkitd, rsyslog on the console). Never opened.
  • x86_64 — corroborating, not conclusive. One clean KVM run (943s) where SSH never became available inside a 15-minute budget. That VM's progress into the rescue environment was not observed — vitest's final reporter discards the streamed log — so it is consistent with the aarch64 result but does not independently prove it. Re-run with KEEP_VM=1 and probe port 22 directly to settle it.

If the x86_64 result holds, this is orthogonal to the multi-arch work, since x86_64 is untouched by it. Prime suspect is that inst.sshd does not start sshd in inst.rescue mode at all — an auth failure would still leave the port open. Leads documented in ARCHITECTURE.md. The debug-setup.sh nc-listener workaround is unaffected.

Consequence: labctl provision debug spark-2935 will now boot a Spark into a rescue environment on its console — which it could not do at all before — but not hand you an SSH shell until this is resolved.

Testing

sudo ./scripts/test-provision.sh rescue        # x86_64 rescue, KVM      (~15 min)
sudo ./scripts/test-provision.sh arm-pxe       # arm64 NBP + rescue      (~25-30 min)
sudo ./scripts/test-provision.sh arm-pxe-full  # + full arm64 install    (~75-95 min)

arm-pxe-full has not been run yet. KEEP_VM=1 leaves the VM up for inspection.

Not deployed, and neither Spark has been touched.

## Why On 2026-08-11 `spark-2935` (DGX Spark, aarch64) panicked with `VFS: Unable to mount root fs on unknown-block(0,0)`. The obvious recovery tool — `labctl provision debug`, "PXE boot into Fedora rescue mode" — was useless: the bastion served a single x86_64 kernel to every machine regardless of architecture, so an ARM64 box got a binary its UEFI will not execute. The **boot-ISO** path had already solved this. This brings the **network PXE** path to parity. ## What changed **Architecture is resolved, never flagged.** No `--arch` anywhere. `/dispatch` resolves in order: tracked machine record → iPXE's `${buildarch}` reported at boot → configured default. `boot.ipxe` gained `&arch=${buildarch}` because DHCP option 93 only ever reaches dnsmasq, never the HTTP endpoint; one script then covers network PXE, UEFI HTTP boot and the boot ISO alike. **Per-arch artifacts.** x86_64 keeps unsuffixed `/vmlinuz` + `/initrd.img`; aarch64 gets `/vmlinuz-aarch64` + `/initrd-aarch64.img`, its own Fedora mirror, and `console=tty0 console=ttyAMA0,115200` instead of `nomodeset` — which does not mean the same thing on arm64 and can leave a headless machine with no console at all. **`--pxe-boot` no longer hardcodes `labvg`.** Root device and dracut args come from the machine's record, observed three ways: `provision recheck` collects them, `provision debug --pxe-boot` probes over SSH when the machine answers, and `debug-setup.sh` reports them from a rescue shell (mounting each candidate read-only, picking the one with `/etc/fstab` and `/usr`). When nothing is known, dispatch serves rescue rather than guessing a `root=` that would leave a machine unbootable. **The Sparks are guarded.** Machines carry `onboard: "pxe" | "ssh"` and the `vendor_os` they must keep running. Installs are refused at both entry points, naming the machine and pointing at `provision debug`; **rescue is never guarded** — being unable to reinstall is exactly when you need a rescue shell. Modelled as a fact about the machine, not a blocklist: the refusal follows from "no image in our pipeline restores `dgx-os`", so adding a SparkOS image is what unblocks these boxes. ## Two corrections to the brief 1. **Option 93 handling already existed** in `dnsmasq.conf.ts`. The real bug was `httpboot-arm64` matching client-arch **20**, which per IANA is *pc/at bios boot from http*; arm64 UEFI HTTP boot is **19**. This is load-bearing, not cosmetic — AAVMF prefers HTTP Boot, and the VM run shows it sending `HTTPClient:Arch:00019`. 2. **LoadFile2 is supported — no GRUB2 chainload needed.** `EFI_LOAD_FILE2_PROTOCOL_GUID` (`4006c0c1-fcb3-403e-996d-4a6c8724e06d`) is at offset `0x3bbf0` of `ipxe-bootimgs-aarch64-20240119`'s `snponly.efi`, with the Linux initrd media device path at `0x3e530`. Confirmed empirically below. Also: **Ubuntu publishes no arm64 netboot artifacts**, so that combination is refused rather than served an x86 kernel. (Separately and pre-existing: the Ubuntu URL the code already fetches 404s even for amd64, so that path looks dead today. Left alone as out of scope.) ## VM evidence Emulated aarch64 on an isolated libvirt network: ``` dnsmasq-dhcp: vendor class: PXEClient:Arch:00011:UNDI:003000 -> tag efi-arm64 dnsmasq-dhcp: sent option 67 bootfile-name ipxe-arm64.efi dnsmasq-dhcp: vendor class: HTTPClient:Arch:00019:UNDI:003000 -> tag httpboot-arm64 HTTP: GET /ipxe-arm64.efi HTTP: GET /boot.ipxe HTTP: GET /dispatch?mac=52:54:00:eb:a6:3c&arch=arm64 HTTP: GET /vmlinuz-aarch64 HTTP: GET /initrd-aarch64.img POST /api/discover NEW MACHINE DISCOVERED: 52:54:00:eb:a6:3c ``` Recorded state: `{"arch": "aarch64", "onboard": "pxe", ...}`. **LoadFile2 confirmed working**: the kernel reached systemd *inside the initramfs* and went on to run Anaconda. Had iPXE dropped the initrd, that is precisely where it would have panicked with `unknown-block(0,0)`. ## x86_64 regression The four dispatch-served scripts are **byte-identical**, pinned by a golden fixture dumped from the templates before any change (`tests/fixtures/ipxe-x86_64-golden.json`). The single intended diff is `&arch=` in `boot.ipxe`, asserted explicitly. ``` Test Files 25 passed (25) Tests 283 passed (283) Completions are up to date. ``` ## Known gap — rescue SSH Adding the first-ever integration coverage for `provision debug` exposed that **rescue SSH has never been observed working**: the environment boots correctly and Anaconda comes up, but nothing listens on port 22. Evidence, stated at its actual strength because it decides where to look next: - **aarch64 — direct.** Port 22 probed every 20s for 30 minutes while the Anaconda installer environment was demonstrably running (NetworkManager, polkitd, rsyslog on the console). Never opened. - **x86_64 — corroborating, not conclusive.** One clean KVM run (943s) where SSH never became available inside a 15-minute budget. That VM's progress into the rescue environment was *not* observed — vitest's final reporter discards the streamed log — so it is consistent with the aarch64 result but does not independently prove it. Re-run with `KEEP_VM=1` and probe port 22 directly to settle it. If the x86_64 result holds, this is orthogonal to the multi-arch work, since x86_64 is untouched by it. Prime suspect is that `inst.sshd` does not start sshd in `inst.rescue` mode at all — an auth failure would still leave the port open. Leads documented in `ARCHITECTURE.md`. The `debug-setup.sh` nc-listener workaround is unaffected. Consequence: `labctl provision debug spark-2935` will now boot a Spark into a rescue environment on its console — which it could not do at all before — but not hand you an SSH shell until this is resolved. ## Testing ``` sudo ./scripts/test-provision.sh rescue # x86_64 rescue, KVM (~15 min) sudo ./scripts/test-provision.sh arm-pxe # arm64 NBP + rescue (~25-30 min) sudo ./scripts/test-provision.sh arm-pxe-full # + full arm64 install (~75-95 min) ``` `arm-pxe-full` has **not** been run yet. `KEEP_VM=1` leaves the VM up for inspection. Not deployed, and neither Spark has been touched.
michal added 10 commits 2026-08-11 14:27:01 +00:00
The aarch64 PXE work touches every iPXE template. Capture what an x86_64
machine is served today, straight from the templates rather than by hand,
so any unintended change to that path fails a test instead of a machine.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nRFZXpKwUVE4SRSHw6GjF
The bastion served one x86_64 kernel to every machine regardless of
architecture, so an ARM64 box was handed a binary its UEFI will not
execute. This is why `labctl provision debug` could not rescue the DGX
Sparks during the 2026-08-11 kernel panic.

Architecture is resolved, never typed by an operator: the tracked machine
record first, then iPXE's ${buildarch} reported on the dispatch URL, then
the configured default. boot.ipxe gains &arch=${buildarch} so that signal
reaches the HTTP endpoint -- DHCP option 93 only ever reaches dnsmasq.
One script covers network PXE, UEFI HTTP boot and the boot ISO alike.

x86_64 keeps its unsuffixed /vmlinuz and /initrd.img so its rendered
scripts are byte-identical; aarch64 gets suffixed paths, its own Fedora
mirror, and console=ttyAMA0 instead of nomodeset, which does not mean the
same thing on arm64 and can leave a headless machine with no console.

--pxe-boot no longer hardcodes the Fedora LVM layout: root device and
dracut args come from the machine's record, and dispatch falls back to
rescue rather than guessing a root= that would leave a machine unbootable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nRFZXpKwUVE4SRSHw6GjF
Per the IANA Processor Architecture Types registry, 0x0013 (19) is
"arm uefi 64 boot from http". 20 is "pc/at bios boot from http", so an
arm64 machine using UEFI HTTP Boot never matched the tag and was never
offered the arm64 iPXE binary.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nRFZXpKwUVE4SRSHw6GjF
The DGX Sparks run DGX OS with a proprietary NVIDIA driver and firmware
stack. No image in our pipeline restores it, so an install destroys the
machine's software permanently -- and `labctl provision install` would
happily do it.

Machines carry an `onboard` classification and the `vendor_os` they must
keep running. Both install entry points (the HTTP route and the labd
command handler) refuse, naming the machine, what it runs, and pointing
at `provision debug` instead. Rescue is deliberately never guarded: being
unable to reinstall is exactly when you need a rescue shell.

Classification is a fact about the machine, not a blocklist. When a DGX
OS image joins the pipeline, teaching the installer about that vendor_os
is what unblocks these boxes. It is keyed on DMI identity, with the two
known Sparks also matched by MAC -- neither has DMI in bastion state
today, so a DMI-only rule would fail open on exactly the machines this
protects.

Also refuses an OS/architecture combination with no netboot artifacts,
which is Ubuntu on aarch64.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nRFZXpKwUVE4SRSHw6GjF
--pxe-boot boots the installed system with a kernel and initrd from the
network, so it needs a root= for that machine. It used to hardcode our
Fedora LVM layout, which is wrong for anything else -- including both
DGX Sparks.

Where the root device comes from, in order of preference:
  - already recorded on the machine (provision recheck now collects it)
  - probed over SSH when --pxe-boot is requested and the machine answers
  - reported from the rescue shell by debug-setup.sh, which mounts each
    candidate read-only and picks the one with /etc/fstab and /usr

The rescue image cannot report it unprompted -- %pre/%post do not run in
rescue mode -- so the probe lives in the script the operator curls, which
already existed for the nc listener.

/api/discover now preserves fields a report omits. The probe posts only a
root device, and blanking a machine's inventory as a side effect of that
would be silent data loss.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nRFZXpKwUVE4SRSHw6GjF
Includes the case that motivated all of this: a machine recorded as
aarch64 and queued for rescue is served an aarch64 kernel, and installs
targeting either Spark are refused while rescue still works.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nRFZXpKwUVE4SRSHw6GjF
Covers what the boot-ISO ARM test never did: DHCP option 93 handing an
arm64 client an arm64 iPXE binary, dispatch serving an aarch64 kernel,
and `provision debug` reaching a rescue shell over SSH.

Split by cost. `arm-pxe` runs NBP handoff, discovery and rescue in about
20-30 minutes -- that is the path the DGX Sparks need. The full install
is another hour on top and only runs with ARM_PXE_FULL=1; an hour-plus
test that runs by default is a test nobody runs.

The suite fails fast if the arm64 iPXE build lacks LoadFile2, because the
symptom otherwise is a 30-minute boot ending in unknown-block(0,0) --
byte-identical to the DGX Spark bug this all started with, and easy to
misdiagnose as a reproduction of it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nRFZXpKwUVE4SRSHw6GjF
The install guard catches an unsupported OS/architecture pair when the
machine's architecture is already known, but a machine queued before it
was ever discovered reaches dispatch with nothing having checked. Serving
it the x86-only Ubuntu kernel is exactly the failure this work exists to
fix, so stop with a legible reason on the console instead -- a machine
handed a kernel it cannot execute fails later and far less clearly.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nRFZXpKwUVE4SRSHw6GjF
Records the two things that are easy to get wrong and expensive to
rediscover: the DHCP option 93 value table (19 is arm64 UEFI HTTP boot,
20 is PC/AT BIOS), and that arm64 needs iPXE with LoadFile2 or the
kernel panics with unknown-block(0,0) and looks like a disk fault.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nRFZXpKwUVE4SRSHw6GjF
test(bastion): cover the rescue boot path and record what it exposed
Some checks failed
CI/CD / lint (pull_request) Failing after 9s
CI/CD / test (pull_request) Failing after 8s
CI/CD / typecheck (pull_request) Failing after 23s
CI/CD / build (pull_request) Has been skipped
CI/CD / publish-rpm (pull_request) Has been skipped
CI/CD / publish-deb (pull_request) Has been skipped
346bd80c13
`provision debug` is the lab's recovery tool of last resort and had no
integration coverage on any architecture. Adding it -- x86_64 on KVM so
it runs in ~15 minutes, plus the aarch64 equivalent -- showed the rescue
environment coming up correctly but nothing ever listening on port 22.

Reproduced on both architectures, so it is neither ARM-specific nor an
emulation artefact, and it is orthogonal to the multi-arch work: x86_64
is unchanged by that. Documented in ARCHITECTURE.md with the leads worth
checking, rather than left as a silent gap.

Also restructures the ARM rescue suite to seed the machine into state
instead of discovering it first. That mirrors the DGX Spark situation --
SSH-onboarded, never PXE-discovered, architecture known only from its
record -- and holds the test to one emulated boot, since each spends
~15 of its ~18 minutes fetching Anaconda's stage2 under TCG.

KEEP_VM=1 leaves the VM up on failure; half-hour emulated runs are too
expensive to pay twice just to see what happened.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nRFZXpKwUVE4SRSHw6GjF
michal added 1 commit 2026-08-11 14:37:01 +00:00
docs(bastion): state the rescue-SSH evidence at its actual strength
Some checks failed
CI/CD / typecheck (pull_request) Failing after 10s
CI/CD / test (pull_request) Failing after 10s
CI/CD / lint (pull_request) Failing after 23s
CI/CD / build (pull_request) Has been skipped
CI/CD / publish-rpm (pull_request) Has been skipped
CI/CD / publish-deb (pull_request) Has been skipped
820fbd4353
The previous wording claimed the failure was "reproduced on x86_64", which
overstates it. The aarch64 observation is direct -- port 22 probed for 30
minutes with the Anaconda environment demonstrably up. The x86_64 run only
shows SSH not becoming available inside a 15-minute budget; that VM was
never observed reaching the rescue environment, because vitest's reporter
discards the streamed log.

The distinction decides where the next person looks, so it should not rest
on an inference presented as an observation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nRFZXpKwUVE4SRSHw6GjF
Some checks failed
CI/CD / typecheck (pull_request) Failing after 10s
CI/CD / test (pull_request) Failing after 10s
CI/CD / lint (pull_request) Failing after 23s
CI/CD / build (pull_request) Has been skipped
CI/CD / publish-rpm (pull_request) Has been skipped
CI/CD / publish-deb (pull_request) Has been skipped
This pull request has changes conflicting with the target branch.
  • bastion/src/bastion/src/routes/api.ts
  • bastion/src/bastion/src/routes/dispatch.ts
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin feat/arm64-pxe-support:feat/arm64-pxe-support
git checkout feat/arm64-pxe-support
Sign in to join this conversation.
No Reviewers
No Label
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: michal/lab#17