migration: offline recovery card, and stop the leak test trusting a silent router

The change itself takes Management down, so the session that performs it has no
internet and no Claude. RECOVERY-CARD-vlan1-move.md is what is actually needed at
that point, on one page: the recovery path, the order, the commands, and what to
do when locked out.

The recovery path is `ssh vyos@10.0.1.252`, and the reason it is trustworthy is
that it is not routed -- the workstation is 10.0.0.210/23 and the router's LoT
leg is 10.0.1.252/23, so `ip route get` returns dev lanbr0 with no `via`. It
therefore survives Management, VRRP, the VIPs, DNS and the switch trunk all being
wrong at once. bond0.10 is untouched by the change and stays in the LAN group.

Order is switch-first, which is the opposite of what seems natural. The UniFi
controller is 192.168.1.5, on Management, reached from LoT *through vyos001*: do
the router first and you lose the controller you still need for the switch.

Two corrections to the leak test, both because it reported a router fault that
was its own:

  - it counted `pgrep -f 'tcpdump -i bond0'`, so a stray tcpdump from an earlier
    run satisfied the >=2 guard with none of this run's captures alive. A capture
    that records nothing reads as "the router sent no reply at all".
  - it believed a single silent run. Kea can be is-active and answering nothing
    for tens of seconds after a restart, so the first VLANs of a loop failed and
    the last passed. That produced two OPPOSITE and equally wrong conclusions
    about `listen-interface` before a retry showed the pattern.

On `listen-interface`: it is a real second branch in kea-dhcp4.conf.j2 that keeps
dhcp-socket-type raw, unlike listen-address which forces udp and took DHCP down
when it was tried live. But naming the sub-interfaces explicitly left Management
DHCP dead in the sim, reproducibly, against a clean control with interfaces:["*"].
Not adopted, and not needed -- the address move is the proven fix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DMVzWZgiKW2wquf5z8S1yH

labsim/labsim-vlan-leak-test.sh

@@ -49,6 +49,19 @@ die() { printf '\033[31merror:\033[0m %s\n' "$*" >&2; exit 1; }
 
 command -v sshpass >/dev/null || die "sshpass required"
 
+# A silent router is the one verdict worth double-checking before reporting.
+#
+# Kea can be `is-active` and answering nothing -- it reopens sockets on a retry
+# loop, and some configurations (`listen-interface`, notably) leave individual
+# VLANs dead while the rest work. Both look identical to a one-shot test: "no
+# reply at all". Two opposite and equally wrong conclusions about
+# `listen-interface` came out of believing a single negative run, in both
+# directions, before a retry made the real pattern obvious.
+#
+# Kea's fallback UDP socket appearing is NOT a readiness signal -- it is bound
+# well before the server actually answers. Checked, and it does not work.
+RETRIED="${RETRIED:-0}"
+
 router() {
   timeout 40 sshpass -p "$ROUTER_PW" ssh -o StrictHostKeyChecking=no \
     -o ConnectTimeout=8 "vyos@$ROUTER_IP" "$@" 2>/dev/null
@@ -71,16 +84,23 @@ PARENT="bond0"
 VIF="bond0.${VLAN}"
 
 log "router $ROUTER_IP -- capturing on $PARENT and $VIF"
+# Kill EVERY tcpdump first, not just ones matching this run's pattern, and count
+# only afterwards. Counting `pgrep -f 'tcpdump -i bond0'` while a stray tcpdump
+# from an earlier session was still running satisfied the >=2 guard with zero of
+# THIS run's captures alive -- and a capture that records nothing reports
+# "the router sent no reply at all", which reads as a DHCP outage. That sent me
+# chasing a fault in the router that was entirely in the test harness.
 started="$(router_sh <<EOF
-sudo pkill -f 'tcpdump -i bond0' >/dev/null 2>&1
+sudo pkill -x tcpdump >/dev/null 2>&1
+sleep 1
 sudo rm -f /tmp/leak-*.txt
 sudo nohup tcpdump -i $PARENT -e -nn -l 'udp port 67 or udp port 68' > /tmp/leak-parent.txt 2>/dev/null &
 sudo nohup tcpdump -i $VIF -e -nn -l 'udp port 67 or udp port 68' > /tmp/leak-vif.txt 2>/dev/null &
 sleep 3
-pgrep -c -f 'tcpdump -i bond0'
+pgrep -c -x tcpdump
 EOF
 )"
-[ "${started:-0}" -ge 2 ] || die "capture did not start on the router (got ${started:-0} of 2)"
+[ "${started:-0}" -eq 2 ] || die "capture did not start on the router (got ${started:-0}, expected exactly 2)"
 
 # -s /bin/true: ask, observe the answer, apply nothing. The client's existing
 # static address is left alone, so this is safe to run against a live sim VM.
@@ -89,7 +109,7 @@ client_out="$(client "udhcpc -n -q -f -i eth0 -s /bin/true -t 3 -T 3 2>&1")"
 [ -n "$client_out" ] || die "no response from client $CLIENT"
 
 sleep 2
-router "sudo pkill -f 'tcpdump -i bond0'" >/dev/null
+router "sudo pkill -x tcpdump" >/dev/null
 parent="$(router 'sudo cat /tmp/leak-parent.txt')"
 vif="$(router 'sudo cat /tmp/leak-vif.txt')"
 
@@ -125,7 +145,11 @@ want_prefix="172.31.${VLAN}."
 
 echo "=== verdict ==="
 if [ -z "$replies" ]; then
-  echo "INCONCLUSIVE: the router sent no reply at all -- is DHCP running?"
+  if [ "$RETRIED" -eq 0 ]; then
+    log "no reply -- retrying once in 20s before calling DHCP down"
+    sleep 20; RETRIED=1 exec "$0" --vlan "$VLAN" --client "$CLIENT" ${SAVE:+--save "$SAVE"}
+  fi
+  echo "INCONCLUSIVE: the router sent no reply at all, twice -- DHCP is down on VLAN $VLAN"
   exit 2
 fi