provisioning: default new nodes to EUI-64 link-locals so DHCPv6 reservations match
Some checks failed
Some checks failed
Proven and rolled out 2026-09-06. The MAC-keyed DHCPv6 reservation only matches
when kea can recover the node's MAC, and for a client that sends a DUID-UUID (no
MAC) the only source is an EUI-64 link-local. NetworkManager's default,
stable-privacy (RFC 7217), hides the MAC -- so a node on the default silently
never gets its reserved address, and with a reservations-only subnet gets
nothing.
Proof: worker0/worker2 (already eui64) held ::23/::25; worker1 and spark were on
the default and did NOT bind, with kea logging ALLOC_ENGINE_V6_ALLOC_FAIL_NO_POOLS.
Setting ipv6.addr-gen-mode=eui64 flipped both to their reserved ::13/::12 within a
DHCPv6 cycle. Not architecture -- worker2 is aarch64 and always worked.
Shipped as a NetworkManager conf.d drop-in in both install paths (kickstart
%post, ubuntu-autoinstall late-commands), so a new node is correct from first
boot, before its connection is ever activated. The same drop-in was placed on
all four reachable existing nodes; aitopatom-3a1c refused key auth and still
needs it applied by hand.
Trade-off accepted: EUI-64 leaks the MAC into the address (irrelevant for infra
nodes), and this MUST stay enforced or a future node silently fails to bind --
which is exactly the trap that produced today's split. The k3s-config preflight
(914135c) is the backstop: it refuses to write a node-ip the node does not hold.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DMVzWZgiKW2wquf5z8S1yH
This commit is contained in:
@@ -372,6 +372,21 @@ fs.inotify.max_user_watches = 1048576
|
|||||||
SYSCTL
|
SYSCTL
|
||||||
sysctl --system || true
|
sysctl --system || true
|
||||||
|
|
||||||
|
# -- IPv6 link-local address generation: EUI-64, fleet-wide --
|
||||||
|
# A cluster node takes its IPv6 from a MAC-keyed DHCPv6 reservation. kea can only
|
||||||
|
# match that reservation when it can recover the node's MAC, and for a modern
|
||||||
|
# client that sends a DUID-UUID (no MAC in it) the only place kea can find one is
|
||||||
|
# an EUI-64 link-local. NetworkManager's default is stable-privacy (RFC 7217),
|
||||||
|
# whose link-local hides the MAC -- so a node on the default silently never gets
|
||||||
|
# its reserved address, and with a reservations-only subnet it gets nothing at
|
||||||
|
# all. Proven on 2026-09-06: worker0/worker2 (eui64) bound; worker1/spark
|
||||||
|
# (default) did not, until flipped. Setting it here means a new node is correct
|
||||||
|
# from first boot, before its connection is ever activated.
|
||||||
|
cat > /etc/NetworkManager/conf.d/10-ipv6-eui64.conf << 'NMEUI64'
|
||||||
|
[connection]
|
||||||
|
ipv6.addr-gen-mode=eui64
|
||||||
|
NMEUI64
|
||||||
|
|
||||||
# -- Disable firewalld permanently (k3s/Cilium manage iptables directly) --
|
# -- Disable firewalld permanently (k3s/Cilium manage iptables directly) --
|
||||||
# Note: no '--now' — systemd is not running in the Anaconda chroot
|
# Note: no '--now' — systemd is not running in the Anaconda chroot
|
||||||
systemctl disable firewalld || true
|
systemctl disable firewalld || true
|
||||||
|
|||||||
@@ -86,6 +86,11 @@ export function renderUbuntuAutoinstall(params: UbuntuAutoinstallParams): string
|
|||||||
`curtin in-target -- bash -c 'cat > /etc/modules-load.d/k3s.conf << EOF\nbr_netfilter\noverlay\nip_conntrack\nEOF'`,
|
`curtin in-target -- bash -c 'cat > /etc/modules-load.d/k3s.conf << EOF\nbr_netfilter\noverlay\nip_conntrack\nEOF'`,
|
||||||
// Sysctl for k3s networking
|
// Sysctl for k3s networking
|
||||||
`curtin in-target -- bash -c 'cat > /etc/sysctl.d/90-k3s.conf << EOF\nnet.bridge.bridge-nf-call-iptables = 1\nnet.bridge.bridge-nf-call-ip6tables = 1\nnet.ipv4.ip_forward = 1\nnet.ipv6.conf.all.forwarding = 1\nfs.inotify.max_user_instances = 524288\nfs.inotify.max_user_watches = 1048576\nEOF'`,
|
`curtin in-target -- bash -c 'cat > /etc/sysctl.d/90-k3s.conf << EOF\nnet.bridge.bridge-nf-call-iptables = 1\nnet.bridge.bridge-nf-call-ip6tables = 1\nnet.ipv4.ip_forward = 1\nnet.ipv6.conf.all.forwarding = 1\nfs.inotify.max_user_instances = 524288\nfs.inotify.max_user_watches = 1048576\nEOF'`,
|
||||||
|
// IPv6 link-local = EUI-64, so a MAC-keyed DHCPv6 reservation can match: kea
|
||||||
|
// recovers the node's MAC from an EUI-64 link-local when the client sends a
|
||||||
|
// DUID-UUID (no MAC in it). NM's stable-privacy default hides the MAC and the
|
||||||
|
// node silently never gets its reserved address. Proven 2026-09-06.
|
||||||
|
`curtin in-target -- bash -c 'cat > /etc/NetworkManager/conf.d/10-ipv6-eui64.conf << EOF\n[connection]\nipv6.addr-gen-mode=eui64\nEOF'`,
|
||||||
// Disable ufw firewall
|
// Disable ufw firewall
|
||||||
`curtin in-target -- systemctl disable ufw || true`,
|
`curtin in-target -- systemctl disable ufw || true`,
|
||||||
// Enable chrony/ntp
|
// Enable chrony/ntp
|
||||||
|
|||||||
@@ -67,6 +67,13 @@ describe("renderUbuntuAutoinstall", () => {
|
|||||||
expect(ids).toContain("mount-longhorn");
|
expect(ids).toContain("mount-longhorn");
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("sets EUI-64 link-local so MAC-keyed DHCPv6 reservations can match", () => {
|
||||||
|
const doc = parseYaml(renderUbuntuAutoinstall({ ...base, role: "worker" }), "eui64");
|
||||||
|
const late = (doc.autoinstall["late-commands"] as string[]).join("\n");
|
||||||
|
expect(late).toContain("10-ipv6-eui64.conf");
|
||||||
|
expect(late).toContain("ipv6.addr-gen-mode=eui64");
|
||||||
|
});
|
||||||
|
|
||||||
it("requests both address families on the primary NIC", () => {
|
it("requests both address families on the primary NIC", () => {
|
||||||
const doc = parseYaml(renderUbuntuAutoinstall({ ...base, role: "worker" }), "net");
|
const doc = parseYaml(renderUbuntuAutoinstall({ ...base, role: "worker" }), "net");
|
||||||
const eth = doc.autoinstall.network.ethernets.primary;
|
const eth = doc.autoinstall.network.ethernets.primary;
|
||||||
|
|||||||
Reference in New Issue
Block a user