97efb5abb2
The three T4 measurements (replace 26s, deny 148s, disable 21s), why deny is the only one that sizes GRACE, and why 148s is a floor rather than a worst case -- idle VMs, and an AC that shares an OVS bridge so `virsh destroy` lets it see the peer vanish in a way a real BRAS never would. Also writes down the broken setter that made the previous numbers fiction, so the next person to touch the matrix knows to check the policy actually landed.
3.7 KiB
WAN failover evidence
Captured by labsim/labsim-pppoe-ha-test.sh. Each directory holds the state of
both routers and the access concentrator at the end of one test.
The number that sizes GRACE
T4 destroys the master with virsh destroy — no LCP Terminate, no PADT, the
router simply ceases — and times how long until the survivor holds a PPPoE
session. Run across every policy VyOS can express, because Vodafone's is
unknown:
session-control |
takeover | |
|---|---|---|
replace |
26s | accel-ppp default; the new auth kills the old session |
deny |
148s | the AC refuses the survivor until its own dead-peer timer frees the dead session |
disable |
21s | no single-session enforcement at all |
deny is the only one that matters for sizing, and GRACE=300 in
migration/vrrp-wan.conf comes from it. Session polling during that run caught
the mechanism directly: the destroyed router's session stayed in the AC's table
while the survivor's dial attempts appeared and were rejected — twice — before
one finally took.
Treat 148s as a floor, not a worst case. These are idle 2-vCPU VMs, and the
AC shares an OVS bridge with the routers, so virsh destroy removes the port
and accel-ppp sees the peer physically vanish. A real BRAS reached over DSL
never learns our router died; it waits out its own timers, which are longer and
not ours to know.
How these numbers were nearly wrong
Until 2026-09-06 the matrix set the policy with:
isp "vbash -c 'source /opt/vyatta/etc/functions/script-template; configure; \
set service pppoe-server session-control $mode; commit; save; exit'"
That form never starts a config session. commit fails with
Invalid command: [commit] on stderr, which isp() discards — so all three
iterations ran against the accel-ppp default while printing the mode they were
supposedly testing. show configuration commands | grep session-control on the
ISP VM came back empty after a full run. The matrix reported deny at 25s; the
real figure is 148s, and GRACE was sized against the fiction.
isp_session_control() now drives it from a real script file, reads the value
back, and skips the iteration rather than measure the wrong policy. A harness
that reports coverage it does not have is worse than one that reports a failure.
The other tests
| what it proves | |
|---|---|
T0-baseline |
exactly one session, held by the VIP holder |
T3-clean-failover |
force-fault moves pppoe0 in 20–26s |
T5-tengig-down |
10 gig down → route falls to pppoe0, LAN back in 5s |
T8-lease-expiry |
the guard hangs up a stale lease within ~5s |
T11-no-peers-file |
a blessed box with no peers file does not restart-loop |
T12-holdoff-keeps-session |
a 900s flap hold-off does not tear down a live session |
T12 exists because it did. ppp_dial() checked the hold-off and returned
before renewing /run/vrrp-wan/may-dial; that lease is what vrrp-wan-guard
expires, so tripping the damper stopped the renewal and the guard hung up
pppoe0 on the master ~80s later:
DIAL FLAP: >=6 attempts in 600s -- holding off 900s
GUARD: lease stale (81s > 75s) -- hanging up pppoe0
A damper meant to suppress repeated dials was destroying an established session instead.
Reading check_invariant
The invariant that matters is at most one of our routers holds pppoe0.
The AC's session count is only a proxy for it, and only while the AC enforces
single-session — under disable it does not, so a destroyed router's session
lingers and the count reads 2 with exactly one live router dialled. That is
reported as a WARN, not a failure. It is not silenced, because an orphaned
session still occupies the single slot at a real ISP: it is exactly what made
deny take 148s.