vrrp-wan: size GRACE from the measured hostile failover, not the theory
Some checks failed
Some checks failed
With the matrix actually setting session-control, T4 timed a destroyed
master's takeover at:
replace 26s
deny 148s <-- sizing case
disable 21s
`deny` is the case GRACE exists for: the AC refuses the survivor until its own
dead-peer timer frees the dead session. The session poller caught it happening
-- the destroyed router's session stayed in the table while the survivor's
dials appeared and were rejected, twice, before one took at 148s.
148s is well past the lcp-echo-interval(30) x failure(3) = 90s budget that 180
was sized against, leaving 32s of margin. GRACE=300 is ~2x the worst observed.
Treat 148s as a floor, not a worst case: these are idle 2-vCPU VMs, and the AC
shares an OVS bridge with the routers, so `virsh destroy` removes the port and
accel-ppp sees the peer vanish. A real BRAS over DSL never learns our router
died and waits out longer timers of its own.
The cost is stated in the conf: GRACE is also how long an alive-but-unroutable
master holds every VIP before yielding. bond0.53 covers most of that -- a DHCP
lease satisfies the check in seconds -- so GRACE only dominates when PPPoE is
the last path. Kept the health check's fallback in step, since keepalived runs
it with no environment and that number decides mastership if the conf is ever
missing.
check_invariant no longer fails blind on the AC's session count. That count is
only a proxy for "two of our routers dialled", and only while the AC enforces
single-session; under `disable` it does not, so a destroyed router's session
lingers and the count reads 2 with exactly one live router dialled. The real
invariant -- at most one router holds pppoe0 -- is now the failing one, and the
stale session is reported as a WARN rather than silenced, because it still
occupies the slot at a real ISP and is precisely what made `deny` take 148s.
This commit is contained in:
@@ -95,11 +95,35 @@ status() { r "$1" 'sudo /config/vrrp-wan-reconcile --status'; }
|
||||
ppp_holders() { n=0; for h in "$R1" "$R2"; do
|
||||
[ "$(ppp_on "$h")" != 0 ] && n=$((n+1)); done; echo "$n"; }
|
||||
|
||||
warn() { printf ' \033[33mWARN\033[0m %s\n' "$*"; }
|
||||
|
||||
check_invariant() {
|
||||
local s p ok=0
|
||||
s="$(ac_sessions)"; p="$(ppp_holders)"
|
||||
[ "${s:-0}" -le 1 ] || { fail "INVARIANT: AC reports $s simdsl sessions"; ok=1; }
|
||||
|
||||
# THE invariant. Two of OUR routers dialled at once is the failure that
|
||||
# matters: one ISP account, and against a real ISP that is how you get
|
||||
# rate-limited or locked out.
|
||||
[ "${p:-0}" -le 1 ] || { fail "INVARIANT: $p routers hold pppoe0"; ok=1; }
|
||||
|
||||
# The AC's session count is a PROXY for the above, and only a valid one
|
||||
# while the AC enforces single-session. Under session-control=disable it
|
||||
# does not, so a destroyed router's session simply stays in the table and
|
||||
# the count reads 2 while exactly one live router is dialled -- which is AC
|
||||
# bookkeeping, not a double dial. Attribute it rather than failing blind:
|
||||
# only call it a violation when more than one router is ACTUALLY dialled.
|
||||
#
|
||||
# Do not silence it either. An orphaned session still occupies the single
|
||||
# slot at a real ISP, and that is exactly what made session-control=deny
|
||||
# take 148s while the survivor's dial attempts were refused.
|
||||
if [ "${s:-0}" -gt 1 ]; then
|
||||
if [ "${p:-0}" -gt 1 ]; then
|
||||
fail "INVARIANT: AC reports $s simdsl sessions AND $p routers are dialled"
|
||||
ok=1
|
||||
else
|
||||
warn "AC reports $s simdsl sessions but only ${p:-0} router is dialled -- stale session from the destroyed peer (expected where the AC does not enforce single-session; it is what a hostile AC holds against the survivor)"
|
||||
fi
|
||||
fi
|
||||
return $ok
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user