Files
lab/migration
Michal 85819e40c1
Some checks failed
CI/CD / lint (push) Failing after 8s
CI/CD / test (push) Failing after 9s
CI/CD / typecheck (push) Failing after 23s
CI/CD / build (push) Has been skipped
CI/CD / publish-rpm (push) Has been skipped
CI/CD / publish-deb (push) Has been skipped
wan-drill-watchdog: bound the blast radius of a failover drill
The drill takes the internet down for as long as the new master needs a WAN.
If it never gets one, whoever is running the drill is offline too -- and an
agent simply stops responding mid-incident. The abort therefore cannot depend
on anyone being present.

Armed on the router expected to take over: if it holds the VIP and has had no
WAN for HOLD consecutive seconds, it sets force-fault on itself, sheds every
VIP, and the healthy non-preempting peer takes them straight back. setsid so it
outlives the ssh session that armed it, which is the entire point.

150s by default, deliberately shorter than GRACE=300. GRACE is sized for a real
hostile-ISP takeover that is still making progress; this is sized for "the
drill failed, give the house its internet back".

Both paths proven on vyos002 in production: it stayed silent for 35s while a
BACKUP (a misfire here would itself cause an outage), and fired within 20s when
pointed at an address the box does hold with no WAN, logging
"ABORT: held the VIP with no WAN for 20s -- standing down".

The first firing test was my own bug, worth noting: sed'ing the default VIP=
line does nothing, because vrrp-wan.conf is sourced afterwards and
VRRP_WAN_VIP puts 192.168.1.1 straight back. The watchdog was watching an
address the box does not hold and correctly stayed quiet -- a test that proved
nothing while looking like it proved the feature was broken.
2026-09-06 09:34:05 +01:00
..