5ed0e4888a
Some checks failed
Final confirming run with GRACE=300 and every fix installed:
--all ALL PASS (T0 T3 T5 T8 T11 T12)
--hard ALL PASS (replace / deny / disable, each policy verified)
Second independent measurement of the hard failover, which is what makes the
figures trustworthy rather than anecdotal:
run 1 run 2
replace 26s 26s
deny 148s 141s
disable 21s 20s
`deny` sits at ~141-148s across both, so it is the AC's dead-peer behaviour
and not a one-off; GRACE=300 keeps roughly 2x margin.
Corrected an overclaim in PPPOE-HA.md while confirming it: the invariant row
read "AC never showed two simdsl sessions", and under session-control=disable
it did -- one live, one orphaned from the destroyed router. Only ever one LIVE
router dialled, which is the invariant that matters. Said so plainly rather
than leaving a table that reads better than the evidence.
72 lines
3.5 KiB
Plaintext
72 lines
3.5 KiB
Plaintext
# Settings for the vrrp-wan scripts. Installed to /config/vrrp-wan.conf.
|
|
#
|
|
# Why a file and not systemd Environment=: keepalived invokes vrrp-wan-health
|
|
# with NO environment at all, so an Environment= line in the .service would be
|
|
# read by the reconciler and ignored by the health check -- two sources of truth
|
|
# for the one value that decides who is master. It is also what stops a repeat
|
|
# of 2026-09-05, when the sim's proof was obtained from scripts hand-`sed`-ed in
|
|
# place: /config/vrrp-wan-health differed from git, and a reinstall would have
|
|
# silently reverted the tested behaviour.
|
|
|
|
# The management VIP. "Do I hold this address" IS the definition of master here
|
|
# -- ground truth, not a marker written by a script that may not have run.
|
|
VRRP_WAN_VIP=192.168.1.1
|
|
|
|
# The DHCP WAN sub-interface. Stays on the config plane because its lease is
|
|
# bound to a cloned MAC, which only VyOS config can move.
|
|
WAN_VIF=53
|
|
|
|
# Seconds a new master may go without any WAN before the health check fails it.
|
|
#
|
|
# Must exceed the ISP's stale-session hold-down, or a hard failover blows the
|
|
# window and BOTH routers end up in FAULT -- worse than the outage the check
|
|
# exists to prevent.
|
|
#
|
|
# MEASURED, labsim T4, master destroyed with `virsh destroy`, time until the
|
|
# survivor held a PPPoE session (labsim/wan-failover-evidence/T4-*). Two
|
|
# independent runs, so these are the AC's behaviour rather than one-offs:
|
|
#
|
|
# session-control=replace 26s / 26s
|
|
# session-control=deny 148s / 141s <-- worst
|
|
# session-control=disable 21s / 20s
|
|
#
|
|
# `deny` is the hostile case and the only one that matters for sizing: the AC
|
|
# refuses the survivor until its own dead-peer timer frees the dead session.
|
|
# The poller caught it happening -- the destroyed router's session stayed in the
|
|
# table while the survivor's dial attempts appeared and were rejected, twice,
|
|
# before it finally got in at 148s.
|
|
#
|
|
# 148s also lands well past the theoretical lcp-echo-interval(30) x
|
|
# failure(3) = 90s budget that 180 was originally sized against, which left only
|
|
# 32s of margin. 300 gives roughly 2x the worst observed, on IDLE 2-vCPU sim
|
|
# VMs; the VP2440s under kea, BGP and conntrack will be slower, and Vodafone's
|
|
# actual policy and timers are unknown.
|
|
#
|
|
# The cost is real and worth stating: this is also how long a master that is
|
|
# alive but genuinely cannot route keeps holding every VIP before yielding --
|
|
# the 2026-09-02 outage shape. That case is mostly covered by bond0.53, which
|
|
# satisfies the check within seconds of getting a DHCP lease; GRACE only
|
|
# dominates when PPPoE is the only path left.
|
|
#
|
|
# Do not lower this below the worst measured handover without re-running
|
|
# `labsim/labsim-pppoe-ha-test.sh --hard`. Before 2026-09-06 that matrix never
|
|
# actually set session-control and reported `deny` at 25s -- a number that did
|
|
# not exist.
|
|
GRACE=300
|
|
|
|
# may-dial is a LEASE, not a flag. vrrp-wan-reconcile renews its mtime every
|
|
# tick; vrrp-wan-guard revokes it once it goes stale. A plain flag survives the
|
|
# reconciler dying, and a router that stops reconciling while demoted would keep
|
|
# dialling for ever.
|
|
LEASE_TTL=75
|
|
|
|
# Flap damper. Two routers that both believe they hold the VIP (a VRRP
|
|
# partition) will both dial; with the AC set to `replace` each dial kills the
|
|
# other's session, the loser's pppd exits non-zero, systemd redials in 5s, and
|
|
# the pair hammers the access concentrator indefinitely. Against a real ISP that
|
|
# is how an account gets rate-limited. More than FLAP_MAX dials in FLAP_WINDOW
|
|
# puts this box in hold-off and logs loudly.
|
|
FLAP_MAX=6
|
|
FLAP_WINDOW=600
|
|
FLAP_HOLDOFF=900
|