This caused a real outage. unifi-reserve-all.py recomputed its plan at --apply
time by re-reading stat/sta, so a client that renewed between the dry run and
the apply was pinned to whatever transient address it happened to hold at that
instant. worker1-k8s0 was reviewed at 192.168.8.13 and written as
192.168.8.242. On its next reboot it could not get an address at all, taking a
k8s node down.
A plan that gets reviewed and a plan that gets applied must be the same object.
The dry run now WRITES the plan to a file and --apply READS it and applies
exactly that, reporting any client whose current address has since drifted
rather than silently preferring the new value.
1 of 51 diverged; the rest were verified against the reviewed list and were
correct. worker1 has been restored to .13 and confirmed: DHCPOFFER for its own
MAC returns 192.168.8.13, and the node is up with a full lease and working
internet.
The second half of the outage was drift between controller and device: the USG
was still running config from ~16h before these changes, so the controller
looked perfectly correct while the gateway handed out something else. Writing
the controller is only half the job, so the script now says so explicitly and
gives the force-provision and DHCP-probe commands to verify with. `nmap
--script broadcast-dhcp-discover --script-args broadcast-dhcp-discover.mac=...`
is the way to prove a specific reservation is live without disturbing the
client -- it elicits an OFFER without ever sending a REQUEST.
_unifi.py gains post() for device commands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DMVzWZgiKW2wquf5z8S1yH
Prerequisite for the cutover. eth2 on both firewalls was DHCP-served by the USG,
and both addresses (192.168.8.143/.144) sit inside the pool VyOS will serve --
with no reservation for either MAC. After the switch they would renew from kea,
get no mapping, and the management addresses could move. That is the worst
possible moment for the address you SSH to to change, because losing the USG
also means losing internet and any outside help.
On both boxes, driven over the LoT path (bond0.10) so the interface being
changed was never the one carrying the session:
- eth2 pinned static at its current address, so it no longer depends on DHCP
- `system name-server eth2` replaced with 10.0.0.194. That setting inherited
resolvers from the DHCP lease, i.e. the boxes were resolving via the USG and
would have lost DNS with it. 10.0.0.194 is reachable directly over bond0.10
and is authoritative for ad.itaz.eu, so internal names now resolve on the
firewalls -- they did not before.
- static default route via 192.168.8.1, replacing the one the lease provided.
Superseded by PPPoE in vyos mode; this keeps unifi mode as it was.
Verified after each: SSH on the pinned address, external and internal DNS, NTP
still synced, VRRP unchanged (vyos001 MASTER, vyos002 BACKUP).
migration/unifi-reserve.py adds the matching UniFi reservations so the
controller cannot lease those addresses to anything else, keeping the
management address identical in both modes. It reads the record back after
writing, because a controller accepting a PUT is not proof it stored what was
asked for, and it is idempotent.
Also noted while doing this: VyOS `commit-confirm` REBOOTS the box if not
confirmed -- "Minutes until reboot, unless 'confirm'" -- it does not roll the
config back in place. For a gateway that means a real outage window, which
changes how the switch script must use it. `config-mgmt commit_confirm -y`
executes without the interactive prompt.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DMVzWZgiKW2wquf5z8S1yH
Groundwork for replacing the USG with the VyOS pair without anything on the
network noticing. Three pieces:
migration/unifi-export.py pulls 13 endpoints off the classic controller into
timestamped JSON plus a normalised inventory: 11 networks, 31 DHCP
reservations, 4 port forwards, 2 firewall rules, 0 static routes. Two things
this turned up that a naive export would have lost:
- 23 of the 31 reservations carry no network_id at all -- UniFi simply does
not store the binding -- so they are resolved by subnet containment
instead. Without that, three quarters of the reservations have no subnet
to be placed in.
- 30 of the 31 sit INSIDE the DHCP pool, which UniFi's dhcpd tolerates and
which is flagged as a warning rather than discovered at cutover.
migration/unifi-to-vyos.py turns that inventory into VyOS `set` commands for
DHCP and DNS only -- the services the USG owns that VyOS must reproduce. Not a
general converter. --prod and --sim come from one code path so the config
proven in the sim and the config applied to the firewalls cannot drift. Prod
mode hard-fails if any reservation is missing, since a silent drop is the
failure mode that matters.
DNS is included because the USG resolves for 5 of 6 VLANs today: UniFi hands
out the gateway's own address whenever dhcpd_dns is empty, verified by
labmaster resolving against 192.168.8.1. Replacing the USG without a forwarder
would take DNS away from those VLANs entirely.
labsim/labsim-dhcp-test.sh proves it by booting throwaway VMs with real
production MACs -- the one piece of production config that transplants
verbatim. Safe because ovs-labsim has no physical NIC, so those MACs cannot
reach the real LAN.
Result on VyOS 2026.08 (kea), 4/4: printer1 got 172.31.10.46 from inside the
pool, sonoff-matter got 172.31.11.67 across the /23 boundary, Hubitat got its
out-of-pool .2, and an unreserved MAC got an unreserved address. kea honours
in-pool host reservations -- the open question blocking the cutover.
Supporting changes to labsim:
- VLAN 10 widened to /23. Every reservation is in LoT and LoT spans 10.0.0.x
and 10.0.1.x, which a /24 cannot represent.
- LoT's host leg moved to .3, because 10.0.0.2 is a real reservation
(Hubitat) that maps onto the host's own address.
- vlans.conf gained optional masklen and host_octet fields, defaulting to
24 and 2 so the other five VLANs are untouched.
- Fixed /etc/network/interfaces hardcoding 255.255.255.0. That file is what
actually takes effect on these Alpine guests -- cloud-init's
network-config is ignored -- so any non-/24 VLAN was silently wrong.
Two generator bugs found by VyOS rejecting the output: static-mapping names
are validated as hostnames, so underscores fail; and two devices named
"espressif" plus two named "thebeast" collided into single names, which would
have overwritten one reservation with another's address.
The raw export holds WiFi passphrases and the WAN PPPoE credentials and is
gitignored.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DMVzWZgiKW2wquf5z8S1yH
