Files
Michal 46b017d77e
Some checks failed
CI/CD / lint (pull_request) Failing after 13s
CI/CD / test (pull_request) Failing after 10s
CI/CD / typecheck (pull_request) Failing after 36s
CI/CD / build (pull_request) Has been skipped
CI/CD / publish-rpm (pull_request) Has been skipped
CI/CD / publish-deb (pull_request) Has been skipped
feat: install logging, error trapping, PXE/ISO integration tests
Kickstart installs on real hardware failed silently — no error reporting,
only 3 progress callbacks, zero log streaming. This overhaul makes every
install fully observable.

Kickstart improvements:
- Error trapping in %pre and %post (trap ERR sends failure details to bastion)
- 12+ granular progress stages (was 3): SSH, hostname, k3s prep, EFI boot, metadata
- Background log streamer: tails %post output and batch-sends to /api/log
- bastion_log() function for explicit log lines from kickstart scripts

Bastion API:
- POST /api/log — receives raw log lines from kickstart (single or batch)
- InstallLogBuffer — per-MAC ring buffer (2000 lines) + file persistence
- GET /api/logs/:mac — now returns log_lines + log_total alongside stages
- SSE /api/logs/:mac/follow — uses named events (event: stage vs event: log)
- Progress events forwarded to labd via bastion-progress WebSocket message
- Post-provision k3s logs routed through progressBus (was console-only)

dnsmasq fixes found during VM testing:
- HTTP Boot filename: ipxe-real.efi → ipxe.efi (leftover from old 2-stage approach)
- pxe-service directives: only in proxy mode (breaks OVMF PXE in full mode)
- PXEClient vendor class echo for UEFI firmware compatibility

Integration tests:
- PXE boot test: blank UEFI VM → dnsmasq → HTTP Boot → iPXE → bastion → install
- ISO boot test: blank VM boots from bastion-generated ISO → same flow
- Shared helpers: pxe-network (no DHCP, nftables fix), pxe-vm (UEFI + ISO boot)
- test-provision.sh: runs both PXE + ISO tests with prerequisite checks
- 250GB sparse QCOW2 disk (LVM layout needs ~204GB)

201 unit tests passing (11 new).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 22:26:33 +00:00

107 lines
2.6 KiB
TypeScript

// SSH execution helper for integration tests.
import { spawn, spawnSync } from "node:child_process";
import { log } from "./libvirt.js";
export interface SshResult {
exitCode: number;
stdout: string;
stderr: string;
}
/** Execute a command via SSH and return the result. */
export function sshExec(
ip: string,
user: string,
command: string,
options?: { keyPath?: string; timeout?: number },
): SshResult {
const args = [
"-o", "StrictHostKeyChecking=no",
"-o", "ConnectTimeout=10",
"-o", "BatchMode=yes",
...(options?.keyPath ? ["-i", options.keyPath] : []),
`${user}@${ip}`,
command,
];
const result = spawnSync("ssh", args, {
stdio: "pipe",
timeout: options?.timeout ?? 120_000,
encoding: "utf-8",
});
return {
exitCode: result.status ?? 1,
stdout: result.stdout ?? "",
stderr: result.stderr ?? "",
};
}
/** Execute a command via SSH with live streaming output. */
export async function sshExecStreaming(
ip: string,
user: string,
command: string,
onLine: (line: string) => void,
options?: { keyPath?: string; timeout?: number },
): Promise<number> {
return new Promise((resolve, reject) => {
const args = [
"-o", "StrictHostKeyChecking=no",
"-o", "ConnectTimeout=10",
"-o", "BatchMode=yes",
...(options?.keyPath ? ["-i", options.keyPath] : []),
`${user}@${ip}`,
command,
];
const proc = spawn("ssh", args, {
stdio: ["ignore", "pipe", "pipe"],
timeout: options?.timeout ?? 600_000,
});
let buffer = "";
proc.stdout.on("data", (data: Buffer) => {
buffer += data.toString();
const lines = buffer.split("\n");
buffer = lines.pop() ?? "";
for (const line of lines) {
onLine(line);
}
});
proc.stderr.on("data", (data: Buffer) => {
const text = data.toString().trim();
if (text) onLine(`[stderr] ${text}`);
});
proc.on("close", (code) => {
if (buffer.trim()) onLine(buffer.trim());
resolve(code ?? 1);
});
proc.on("error", (err) => {
reject(err);
});
});
}
/** Run a command on the VM via SSH and log it with a prefix. */
export async function sshRun(
ip: string,
user: string,
command: string,
label: string,
options?: { keyPath?: string; timeout?: number },
): Promise<number> {
log(`${label}: ${command.slice(0, 80)}${command.length > 80 ? "..." : ""}`);
const code = await sshExecStreaming(ip, user, command, (line) => {
console.log(` ${line}`);
}, options);
if (code !== 0) {
log(`${label}: FAILED (exit ${code})`);
}
return code;
}