This caused a real outage. unifi-reserve-all.py recomputed its plan at --apply
time by re-reading stat/sta, so a client that renewed between the dry run and
the apply was pinned to whatever transient address it happened to hold at that
instant. worker1-k8s0 was reviewed at 192.168.8.13 and written as
192.168.8.242. On its next reboot it could not get an address at all, taking a
k8s node down.
A plan that gets reviewed and a plan that gets applied must be the same object.
The dry run now WRITES the plan to a file and --apply READS it and applies
exactly that, reporting any client whose current address has since drifted
rather than silently preferring the new value.
1 of 51 diverged; the rest were verified against the reviewed list and were
correct. worker1 has been restored to .13 and confirmed: DHCPOFFER for its own
MAC returns 192.168.8.13, and the node is up with a full lease and working
internet.
The second half of the outage was drift between controller and device: the USG
was still running config from ~16h before these changes, so the controller
looked perfectly correct while the gateway handed out something else. Writing
the controller is only half the job, so the script now says so explicitly and
gives the force-provision and DHCP-probe commands to verify with. `nmap
--script broadcast-dhcp-discover --script-args broadcast-dhcp-discover.mac=...`
is the way to prove a specific reservation is live without disturbing the
client -- it elicits an OFFER without ever sending a REQUEST.
_unifi.py gains post() for device commands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DMVzWZgiKW2wquf5z8S1yH
kea does not inherit UniFi's lease database. At cutover it starts with an empty
view of who holds what, so it can hand an address that is currently in use to a
different device. Reservations are what carry "this device has this address"
across the switch, because they live in config rather than in lease state.
unifi-reserve-all.py creates one per active client, dry run by default. 51
written, 51/51 verified live by reading the records back; the controller now
holds 85 reservations and the generator emits all 85 with unique, valid
hostnames and no duplicate addresses. Active clients with no reservation went
from 48 to 4.
Three guards, each of which caught something real in the dry run:
- VRRP virtual addresses are excluded. UniFi reports them as ordinary client
addresses because the firewalls' bond MACs answer for them, and their
apparent IP flips between the real interface address and the VIP. Without
this, 192.168.1.254 -- the gateway VIP itself -- would have been given a
DHCP reservation.
- The firewalls' own interface MACs are excluded; those are statically
configured routers, not DHCP clients.
- Any address claimed by more than one MAC is dropped rather than guessed
at. This is how the VIPs surfaced in the first place.
Also skipped: addresses already reserved to another MAC, network gateways, and
anything on a network that does not serve DHCP (which excludes the WAN transit
VLANs automatically).
labsim-dhcp-test.sh gained a lease-database flush, and it is not tidiness. Two
findings, both of which first appeared as a PASSING test:
- Re-running against stale leases, kea gave dynamic addresses to three
devices that have reservations. The reservations were present and correct
in kea's own config throughout. Kea saw the reserved address as leased to
"another client" -- same MAC, different client-id from the earlier boot --
and allocated elsewhere. Cutover starts with an empty lease database so
this is a testing artifact, but a reservation is evidently not
unconditional once leases exist.
- Removing only dhcp4-leases.csv does nothing: kea's memfile backend keeps
lease-file-cleanup rotations (.csv.2) and restores from them on start.
The verdict logic no longer takes the first matching lease row. Doing so
reported an hours-old lease as the current answer and scored three failures as
passes, including one where the device had plainly been given a dynamic
address. A MAC with more than one lease is now an explicit failure rather than
a guess.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DMVzWZgiKW2wquf5z8S1yH
