7bf3f42e19
Some checks failed
The ISP is consumer with one static IP, so "both routers hold WAN" is not
available: the 10 gig lease is anchored to a cloned MAC and the PPPoE line to a
single credential. The WAN therefore has to move with mastership.
Proven end to end in the sim: the secondary was promoted, took the WAN, got a
lease, installed a default route, and a LAN VM reached the internet through it
(3/3, 9ms). Demoting released it -- link down, no address. Rebooting the master
converged correctly too: it came back BACKUP with the WAN disabled while the
peer kept it.
The rehearsal earned its keep four times over, and none of these were visible
from reading the docs:
- `transition-script` alone is NOT safe to hang internet on. VyOS delivers it
through keepalived-fifo.py, and on one promotion that helper logged NOTHING
while Keepalived_vrrp logged all six instances entering MASTER and the
built-in notify_master for conntrack-sync ran normally. The result was a
router holding every VIP with no WAN -- the 2026-09-02 outage, recreated by
the mechanism meant to prevent it. Hence vrrp-wan-reconcile on a 30s timer:
the scripts give speed, the timer gives correctness.
- The health check must ask REALITY, not a marker. Keying "am I master" on a
/run file written by the transition script meant that when the script did
not run, the router believed it was backup, passed the check, and kept the
VIPs it could not serve. It now asks whether the VIP is actually on the box.
- The old address-based check DEADLOCKED this design: may-I-be-master required
already having WAN, and only the master gets WAN. That is why vyos002 sat in
FAULT for ever -- the safety check had silently removed the redundancy it
existed to protect.
- script-template must be the FIRST thing a script does. Sourced after an if,
an exec and a mkdir it terminated the script inside the source, rc=0, no
output: the reconciler reported success having done nothing. Hence the split
into vrrp-wan-apply, matching the shape /config/vyos-known-good already uses.
Two hazards found and handled rather than discovered in production:
- A `configure` session whose process dies leaks a unionfs mount under
/opt/vyatta/config/tmp, and one of those holds the commit lock -- after
which every commit fails, including the manual one you try to fix it with.
A 30s job that can leak one per failure wedges the box on its own, so the
reconciler reaps dead sessions before it starts. It cleared 8 on the sim.
- Any `save` while a box is master persists the enabled WAN into config.boot,
so a reboot would claim the shared MAC regardless of VRRP state. Observed:
an ordinary console-apply did exactly this. config.boot must keep `disable`
on BOTH routers; the model asserts it and vyos:verify reports it as drift.
NOT yet applied to production, and it should not be until the remaining item is
settled: a clean, deliberately-triggered failover has been seen via reboot, but
`restart vrrp` twice failed to move mastership at all, so the trigger for a
planned failover is still unproven.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DMVzWZgiKW2wquf5z8S1yH
33 lines
1.4 KiB
Plaintext
33 lines
1.4 KiB
Plaintext
#!/bin/vbash
|
|
# Enable or disable the WAN. Split out from vrrp-wan-reconcile for one reason:
|
|
# `source /opt/vyatta/etc/functions/script-template` must be the FIRST thing the
|
|
# script does. Sourced after a few statements -- an if, an exec, a mkdir -- it
|
|
# silently terminated the script; `set -x` showed execution stopping inside the
|
|
# source with no error and rc=0, so the reconciler reported success having done
|
|
# nothing. Only a single assignment may precede it (the template resets the
|
|
# positional parameters, so the mode is captured first), which is the same shape
|
|
# /config/vyos-known-good uses.
|
|
#
|
|
# vrrp-wan-apply enable take the WAN
|
|
# vrrp-wan-apply disable release it
|
|
MODE="${1:-}"
|
|
source /opt/vyatta/etc/functions/script-template
|
|
|
|
WAN_VIF=53
|
|
cfg() { /opt/vyatta/bin/vyatta-op-cmd-wrapper show configuration commands 2>/dev/null; }
|
|
wan_disabled(){ cfg | grep -q "vif ${WAN_VIF} disable"; }
|
|
ppp_disabled(){ cfg | grep -q "pppoe pppoe0 disable"; }
|
|
|
|
configure
|
|
if [ "$MODE" = enable ]; then
|
|
# Guarded: `delete` of an absent node aborts the whole batch with
|
|
# "Nothing to delete", which left the box detected-but-unfixed.
|
|
wan_disabled && delete interfaces bonding bond0 vif ${WAN_VIF} disable
|
|
ppp_disabled && delete interfaces pppoe pppoe0 disable
|
|
else
|
|
wan_disabled || set interfaces bonding bond0 vif ${WAN_VIF} disable
|
|
ppp_disabled || set interfaces pppoe pppoe0 disable
|
|
fi
|
|
commit
|
|
exit
|