6fe71eba63
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V6HNx6cyr4J4CiMGCy1wkF
4.9 KiB
thebeast → prod k8s worker — conversion runbook
Converting the old desktop thebeast (currently Fedora/ZFS desktop at
10.0.1.22) into a k8s-worker node of the prod cluster (k8s0) via the
PXE bastion, using the dedicated-data-disk layout (--longhorn-disk /
--rancher-disk, introduced with this runbook).
Decisions already taken (2026-10-14): Windows disk is sacrificed (unused for
years, no backup wanted) · cluster = prod k8s0 · OS = Fedora k8s-worker
(thebeast is mainly an AMD-GPU machine: 2× AMD + 1× RTX 3080 Ti, no unified
memory) · the node will carry a dedicated taint and also serve CI/CD.
VM-drilled 2026-10-14 (on thebeast itself, evidence:
drill-evidence/data-disks-2026-10-14.txt): fresh install lands the exact
layout below, and a flagless reprovision rediscovers both data disks by label
and remounts them without reformatting (UUIDs unchanged, marker files intact).
Disk map — ALWAYS by-id, never nvme0n1
Kernel NVMe names are not stable across boots on this machine (observed).
Every command below uses /dev/disk/by-id.
| target | disk | by-id |
|---|---|---|
| OS (wipes Windows) | Samsung 970 EVO Plus 1TB | nvme-Samsung_SSD_970_EVO_Plus_1TB_S4EWNJ0N138234M |
| /var/lib/longhorn | Sabrent Rocket 4.0 2TB | nvme-Sabrent_Rocket_4.0_2TB_7F600711179D89261778 |
| /var/lib/rancher | Sabrent Rocket 4.0 2TB | nvme-Sabrent_Rocket_4.0_2TB_7FD4070C1EDA92197301 |
| DISCONNECT after final shutdown | Samsung 970 EVO Plus 2TB S4J4NF0NC00809J |
sits on its own motherboard add-on card, PCIe x4, model discontinued |
NICs: enp69s0 34:97:f6:32:76:e9 (current uplink — this MAC is what the
bastion will see), enp70s0 34:97:f6:31:b3:99 (down).
0. Before the final shutdown — data
rpool is a 4-wide raidz1 already DEGRADED (member
S4J4NF0R102776W removed since 2026-09-14). There is zero redundancy
left: one more failure loses the pool, and a disk pulled out of it is NOT
importable alone. Finish the copies first, then never boot the desktop again.
Already safe (verified 2026-10-14): ~/developer (all 113 repos) + the
credential/dotfile set → bazzite (Oct 12 rsync, 1 file of drift).
Copied to bazzite and rsync-verified 2026-10-14: ~/keyssecrets,
~/dicesecrets, ~/Videos/canaries (24G), ~/Pictures, ~/Documents.
~/opt/vaultbackup and ~/migration-backup declared done/not-needed (user,
2026-10-14).
Still open before shutdown:
~/Videos/{Webcam,subscribescreen,Screencasts,music}(~370M) and~/backups/sogo-final-2026-09-13— small, not yet copied- Verify the Syncthing replicas on synology0-nas for
~/Logseq,~/DICESharedBoostNote,~/Boostnote,~/MangaReader_backup— thebeast's syncthing shows 0 connected peers, so the NAS copy is unverified - Final re-sync of
~/.claude+ atuin (activity postdates the Oct 12 copy)
1. Physical + network
- Final desktop shutdown.
- Disconnect the Samsung 2TB
S4J4NF0NC00809J(the add-on card). - Move thebeast's switch port to the
192.168.8.0/24VLAN (same as k8s0 and labmaster). The bastion's proxy-DHCP (dhcp-range=192.168.8.0,proxy) coexists with the real DHCP there — no DHCP changes needed. - Make sure network boot is enabled/first in firmware (UEFI PXE, IPv4).
2. Discovery
PXE boot. Unknown MAC → discovery image → the machine self-reports:
labctl provision list # thebeast appears under discovered (MAC 34:97:f6:32:76:e9)
3. Install
labctl provision install 34:97:f6:32:76:e9 thebeast \
--role k8s-worker \
--disk /dev/disk/by-id/nvme-Samsung_SSD_970_EVO_Plus_1TB_S4EWNJ0N138234M \
--longhorn-disk /dev/disk/by-id/nvme-Sabrent_Rocket_4.0_2TB_7F600711179D89261778 \
--rancher-disk /dev/disk/by-id/nvme-Sabrent_Rocket_4.0_2TB_7FD4070C1EDA92197301
(k8s-worker is v2-only, so no --schema-version needed.)
What the layout does: OS disk gets EFI + /boot + labvg with the v2 base LVs
only — no longhorn/rancher LVs. Each 2 TB disk gets ONE xfs partition
spanning it, labelled lab-longhorn / lab-rancher, mounted at
/var/lib/longhorn / /var/lib/rancher. The ZFS signatures on those disks
are wiped by the install — which is the point of section 0.
Reinstall semantics from then on: the labels are on-disk truth. A reprovision
rediscovers both data disks by label (no flags needed) and mounts them
--noformat; a fresh install refuses to wipe or shadow a labelled disk it was
not told about.
4. Join k8s0
As on every k8s-worker, k3s is installed with INSTALL_K3S_SKIP_START; the
join is deliberate (see /root/README on the node):
curl -sfL https://get.k3s.io | K3S_URL=https://<k8s0-server>:6443 K3S_TOKEN=<token> sh -
Add the agreed CI/CD taint/labels at join time (k3s agent flags), e.g.:
--node-taint dedicated=cicd:NoSchedule --node-label itaz.eu/cicd=true
(exact taint TBD with the Pulumi model; GPU bits — ROCm for the 2 AMD cards, nvidia for the 3080 Ti — are cluster-side follow-ups in thelab-kubernetes-pulumi, not part of the install.)