provisioning: default new nodes to EUI-64 link-locals so DHCPv6 reservations match
Some checks failed
CI/CD / lint (push) Failing after 8s
CI/CD / test (push) Failing after 9s
CI/CD / typecheck (push) Failing after 24s
CI/CD / build (push) Has been skipped
CI/CD / publish-rpm (push) Has been skipped
CI/CD / publish-deb (push) Has been skipped

Proven and rolled out 2026-09-06. The MAC-keyed DHCPv6 reservation only matches
when kea can recover the node's MAC, and for a client that sends a DUID-UUID (no
MAC) the only source is an EUI-64 link-local. NetworkManager's default,
stable-privacy (RFC 7217), hides the MAC -- so a node on the default silently
never gets its reserved address, and with a reservations-only subnet gets
nothing.

Proof: worker0/worker2 (already eui64) held ::23/::25; worker1 and spark were on
the default and did NOT bind, with kea logging ALLOC_ENGINE_V6_ALLOC_FAIL_NO_POOLS.
Setting ipv6.addr-gen-mode=eui64 flipped both to their reserved ::13/::12 within a
DHCPv6 cycle. Not architecture -- worker2 is aarch64 and always worked.

Shipped as a NetworkManager conf.d drop-in in both install paths (kickstart
%post, ubuntu-autoinstall late-commands), so a new node is correct from first
boot, before its connection is ever activated. The same drop-in was placed on
all four reachable existing nodes; aitopatom-3a1c refused key auth and still
needs it applied by hand.

Trade-off accepted: EUI-64 leaks the MAC into the address (irrelevant for infra
nodes), and this MUST stay enforced or a future node silently fails to bind --
which is exactly the trap that produced today's split. The k3s-config preflight
(914135c) is the backstop: it refuses to write a node-ip the node does not hold.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DMVzWZgiKW2wquf5z8S1yH
This commit is contained in:
Michal
2026-09-06 23:07:39 +01:00
parent 7c2cbfaf31
commit 6c94371c8e
3 changed files with 27 additions and 0 deletions

View File

@@ -372,6 +372,21 @@ fs.inotify.max_user_watches = 1048576
SYSCTL SYSCTL
sysctl --system || true sysctl --system || true
# -- IPv6 link-local address generation: EUI-64, fleet-wide --
# A cluster node takes its IPv6 from a MAC-keyed DHCPv6 reservation. kea can only
# match that reservation when it can recover the node's MAC, and for a modern
# client that sends a DUID-UUID (no MAC in it) the only place kea can find one is
# an EUI-64 link-local. NetworkManager's default is stable-privacy (RFC 7217),
# whose link-local hides the MAC -- so a node on the default silently never gets
# its reserved address, and with a reservations-only subnet it gets nothing at
# all. Proven on 2026-09-06: worker0/worker2 (eui64) bound; worker1/spark
# (default) did not, until flipped. Setting it here means a new node is correct
# from first boot, before its connection is ever activated.
cat > /etc/NetworkManager/conf.d/10-ipv6-eui64.conf << 'NMEUI64'
[connection]
ipv6.addr-gen-mode=eui64
NMEUI64
# -- Disable firewalld permanently (k3s/Cilium manage iptables directly) -- # -- Disable firewalld permanently (k3s/Cilium manage iptables directly) --
# Note: no '--now' — systemd is not running in the Anaconda chroot # Note: no '--now' — systemd is not running in the Anaconda chroot
systemctl disable firewalld || true systemctl disable firewalld || true

View File

@@ -86,6 +86,11 @@ export function renderUbuntuAutoinstall(params: UbuntuAutoinstallParams): string
`curtin in-target -- bash -c 'cat > /etc/modules-load.d/k3s.conf << EOF\nbr_netfilter\noverlay\nip_conntrack\nEOF'`, `curtin in-target -- bash -c 'cat > /etc/modules-load.d/k3s.conf << EOF\nbr_netfilter\noverlay\nip_conntrack\nEOF'`,
// Sysctl for k3s networking // Sysctl for k3s networking
`curtin in-target -- bash -c 'cat > /etc/sysctl.d/90-k3s.conf << EOF\nnet.bridge.bridge-nf-call-iptables = 1\nnet.bridge.bridge-nf-call-ip6tables = 1\nnet.ipv4.ip_forward = 1\nnet.ipv6.conf.all.forwarding = 1\nfs.inotify.max_user_instances = 524288\nfs.inotify.max_user_watches = 1048576\nEOF'`, `curtin in-target -- bash -c 'cat > /etc/sysctl.d/90-k3s.conf << EOF\nnet.bridge.bridge-nf-call-iptables = 1\nnet.bridge.bridge-nf-call-ip6tables = 1\nnet.ipv4.ip_forward = 1\nnet.ipv6.conf.all.forwarding = 1\nfs.inotify.max_user_instances = 524288\nfs.inotify.max_user_watches = 1048576\nEOF'`,
// IPv6 link-local = EUI-64, so a MAC-keyed DHCPv6 reservation can match: kea
// recovers the node's MAC from an EUI-64 link-local when the client sends a
// DUID-UUID (no MAC in it). NM's stable-privacy default hides the MAC and the
// node silently never gets its reserved address. Proven 2026-09-06.
`curtin in-target -- bash -c 'cat > /etc/NetworkManager/conf.d/10-ipv6-eui64.conf << EOF\n[connection]\nipv6.addr-gen-mode=eui64\nEOF'`,
// Disable ufw firewall // Disable ufw firewall
`curtin in-target -- systemctl disable ufw || true`, `curtin in-target -- systemctl disable ufw || true`,
// Enable chrony/ntp // Enable chrony/ntp

View File

@@ -67,6 +67,13 @@ describe("renderUbuntuAutoinstall", () => {
expect(ids).toContain("mount-longhorn"); expect(ids).toContain("mount-longhorn");
}); });
it("sets EUI-64 link-local so MAC-keyed DHCPv6 reservations can match", () => {
const doc = parseYaml(renderUbuntuAutoinstall({ ...base, role: "worker" }), "eui64");
const late = (doc.autoinstall["late-commands"] as string[]).join("\n");
expect(late).toContain("10-ipv6-eui64.conf");
expect(late).toContain("ipv6.addr-gen-mode=eui64");
});
it("requests both address families on the primary NIC", () => { it("requests both address families on the primary NIC", () => {
const doc = parseYaml(renderUbuntuAutoinstall({ ...base, role: "worker" }), "net"); const doc = parseYaml(renderUbuntuAutoinstall({ ...base, role: "worker" }), "net");
const eth = doc.autoinstall.network.ethernets.primary; const eth = doc.autoinstall.network.ethernets.primary;