michal/lab
1
0
Fork 0
Code Issues Pull Requests 3 Actions Packages Projects Releases Wiki Activity

labsim: both matrices green end to end, numbers reproduced
Some checks failed
CI/CD / typecheck (push) Failing after 9s
Details
CI/CD / test (push) Failing after 9s
Details
CI/CD / lint (push) Failing after 24s
Details
CI/CD / build (push) Has been skipped
Details
CI/CD / publish-rpm (push) Has been skipped
Details
CI/CD / publish-deb (push) Has been skipped
Details

Browse Source 
Final confirming run with GRACE=300 and every fix installed:

  --all   ALL PASS  (T0 T3 T5 T8 T11 T12)
  --hard  ALL PASS  (replace / deny / disable, each policy verified)

Second independent measurement of the hard failover, which is what makes the
figures trustworthy rather than anecdotal:

                run 1   run 2
  replace        26s     26s
  deny          148s    141s
  disable        21s     20s

`deny` sits at ~141-148s across both, so it is the AC's dead-peer behaviour
and not a one-off; GRACE=300 keeps roughly 2x margin.

Corrected an overclaim in PPPOE-HA.md while confirming it: the invariant row
read "AC never showed two simdsl sessions", and under session-control=disable
it did -- one live, one orphaned from the destroyed router. Only ever one LIVE
router dialled, which is the invariant that matters. Said so plainly rather
than leaving a table that reads better than the evidence.
This commit is contained in:
Michal
2026-09-06 00:42:01 +01:00
parent 97efb5abb2
commit 5ed0e4888a
11 changed files with 177 additions and 163 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
labsim/wan-failover-evidence/README.md
View File

@@ -10,11 +10,13 @@ router simply ceases — and times how long until the survivor holds a PPPoE
session. Run across every policy VyOS can express, because Vodafone's is
unknown:
Two independent runs, so these describe the AC's behaviour rather than one-offs:
| `session-control` | takeover | |
|---|---|---|
| `replace` | 26s | accel-ppp default; the new auth kills the old session |
| **`deny`** | **148s** | the AC refuses the survivor until its own dead-peer timer frees the dead session |
| `disable` | 21s | no single-session enforcement at all |
| `replace` | 26s / 26s | accel-ppp default; the new auth kills the old session |
| **`deny`** | **148s / 141s** | the AC refuses the survivor until its own dead-peer timer frees the dead session |
| `disable` | 21s / 20s | no single-session enforcement at all |
`deny` is the only one that matters for sizing, and `GRACE=300` in
`migration/vrrp-wan.conf` comes from it. Session polling during that run caught

42
labsim/wan-failover-evidence/T11-no-peers-file/state.txt
View File

@@ -1,26 +1,26 @@
=== 2026-09-06T00:08:14+01:00 ===
=== 2026-09-06T00:29:55+01:00 ===
--- AC sessions ---
ifname | username | ip | ip6 | ip6-dp | calling-sid | rate-limit | state | uptime | rx-bytes | tx-bytes
--------+----------+----+-----+--------+-------------------+------------+-------+----------+----------+----------
| | | | | 52:54:00:e5:95:a2 | | start | 00:00:00 | 0 B | 0 B
ifname | username | ip | ip6 | ip6-dp | calling-sid | rate-limit | state | uptime | rx-bytes | tx-bytes
--------+----------+----+-----+--------+-------------+------------+-------+--------+----------+----------
--- 172.31.1.252 ---
vip=172.31.1.1 holds_vip=yes wan_disabled=no wan_up=no ppp_up=no ppp_active=yes may_dial=yes lease_age=1 dropin=yes role=master
Sep 05 18:11:42 apitest vrrp-wan[3017050]: MASTER with bond0.53 disabled -> enabling
Sep 05 18:11:46 apitest vrrp-wan[3017685]: bond0.53 enable commit took 4s
-- Boot 7cb878bdb4c74380ae7869ffd0dad9eb --
Sep 05 23:06:18 apitest vrrp-wan[18533]: MASTER: dialling pppoe0
Sep 05 23:06:18 apitest vrrp-wan[18704]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:06:22 apitest vrrp-wan[19398]: bond0.53 enable commit took 4s
Sep 05 23:07:35 apitest vrrp-wan[22111]: GUARD: lease stale (206s > 75s; is vrrp-wan-reconcile.timer running?) -- hanging up pppoe0
Sep 05 23:07:43 apitest vrrp-wan[22667]: MASTER: dialling pppoe0
Sep 05 23:08:14 apitest vrrp-wan[24120]: MASTER: dialling pppoe0
Sep 05 23:18:31 apitest vrrp-wan[6411]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:18:36 apitest vrrp-wan[7038]: bond0.53 enable commit took 5s
-- Boot 68bb1e8d56b34d55a511a301c24270fe --
Sep 05 23:27:40 apitest vrrp-wan[12502]: MASTER: dialling pppoe0
Sep 05 23:27:40 apitest vrrp-wan[12654]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:27:45 apitest vrrp-wan[13361]: bond0.53 enable commit took 5s
Sep 05 23:29:01 apitest vrrp-wan[16369]: GUARD: lease stale (200s > 75s; is vrrp-wan-reconcile.timer running?) -- hanging up pppoe0
Sep 05 23:29:27 apitest vrrp-wan[17398]: MASTER: dialling pppoe0
Sep 05 23:29:57 apitest vrrp-wan[18610]: MASTER: dialling pppoe0
--- 172.31.1.253 ---
vip=172.31.1.1 holds_vip=no wan_disabled=yes wan_up=no ppp_up=no ppp_active=no may_dial=no lease_age=- dropin=yes role=backup
Sep 05 22:56:19 vyos vrrp-wan[196724]: DIAL FLAP: >=6 attempts in 600s -- holding off 900s
Sep 05 22:57:24 vyos vrrp-wan[199011]: MASTER with bond0.53 disabled -> enabling
Sep 05 22:57:28 vyos vrrp-wan[199711]: bond0.53 enable commit took 4s
Sep 05 22:57:40 vyos vrrp-wan[199948]: GUARD: lease stale (81s > 75s; is vrrp-wan-reconcile.timer running?) -- hanging up pppoe0
Sep 05 23:02:41 vyos vrrp-wan[206596]: MASTER: dialling pppoe0
Sep 05 23:06:17 vyos vrrp-wan[213698]: not MASTER: hanging up pppoe0
Sep 05 23:06:18 vyos vrrp-wan[213859]: not MASTER but bond0.53 enabled -> releasing
Sep 05 23:06:23 vyos vrrp-wan[214503]: bond0.53 disable commit took 5s
Sep 05 23:16:11 vyos vrrp-wan[7190]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:16:15 vyos vrrp-wan[7979]: bond0.53 enable commit took 4s
-- Boot c0e38585d7354ad6ab018800c7f3f6be --
Sep 05 23:22:54 vyos vrrp-wan[5970]: MASTER: dialling pppoe0
Sep 05 23:22:54 vyos vrrp-wan[6126]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:22:59 vyos vrrp-wan[6915]: bond0.53 enable commit took 5s
Sep 05 23:27:40 vyos vrrp-wan[16037]: not MASTER: hanging up pppoe0
Sep 05 23:27:40 vyos vrrp-wan[16198]: not MASTER but bond0.53 enabled -> releasing
Sep 05 23:27:47 vyos vrrp-wan[16840]: bond0.53 disable commit took 7s

45
labsim/wan-failover-evidence/T12-holdoff-keeps-session/state.txt
View File

@@ -1,28 +1,29 @@
=== 2026-09-06T00:10:11+01:00 ===
=== 2026-09-06T00:31:55+01:00 ===
--- AC sessions ---
ifname | username | ip | ip6 | ip6-dp | calling-sid | rate-limit | state | uptime | rx-bytes | tx-bytes
--------+----------+----------------+-----+--------+-------------------+------------+--------+----------+----------+----------
ppp0 | simdsl | 198.51.100.126 | | | 52:54:00:e5:95:a2 | | active | 00:01:57 | 514 B | 204 B
ppp0 | simdsl | 198.51.100.134 | | | 52:54:00:e5:95:a2 | | active | 00:01:58 | 670 B | 555 B
--- 172.31.1.252 ---
vip=172.31.1.1 holds_vip=yes wan_disabled=no wan_up=no ppp_up=yes ppp_active=yes may_dial=yes lease_age=23 dropin=yes role=master
pppoe0 UNKNOWN 198.51.100.126 peer 198.51.100.1/32
default nhid 78 dev pppoe0 proto static metric 20
Sep 05 18:11:42 apitest vrrp-wan[3017050]: MASTER with bond0.53 disabled -> enabling
Sep 05 18:11:46 apitest vrrp-wan[3017685]: bond0.53 enable commit took 4s
-- Boot 7cb878bdb4c74380ae7869ffd0dad9eb --
Sep 05 23:06:18 apitest vrrp-wan[18533]: MASTER: dialling pppoe0
Sep 05 23:06:18 apitest vrrp-wan[18704]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:06:22 apitest vrrp-wan[19398]: bond0.53 enable commit took 4s
Sep 05 23:07:35 apitest vrrp-wan[22111]: GUARD: lease stale (206s > 75s; is vrrp-wan-reconcile.timer running?) -- hanging up pppoe0
Sep 05 23:07:43 apitest vrrp-wan[22667]: MASTER: dialling pppoe0
Sep 05 23:08:14 apitest vrrp-wan[24120]: MASTER: dialling pppoe0
vip=172.31.1.1 holds_vip=yes wan_disabled=no wan_up=no ppp_up=yes ppp_active=yes may_dial=yes lease_age=21 dropin=yes role=master
pppoe0 UNKNOWN 198.51.100.134 peer 198.51.100.1/32
default nhid 74 dev pppoe0 proto static metric 20
Sep 05 23:18:31 apitest vrrp-wan[6411]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:18:36 apitest vrrp-wan[7038]: bond0.53 enable commit took 5s
-- Boot 68bb1e8d56b34d55a511a301c24270fe --
Sep 05 23:27:40 apitest vrrp-wan[12502]: MASTER: dialling pppoe0
Sep 05 23:27:40 apitest vrrp-wan[12654]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:27:45 apitest vrrp-wan[13361]: bond0.53 enable commit took 5s
Sep 05 23:29:01 apitest vrrp-wan[16369]: GUARD: lease stale (200s > 75s; is vrrp-wan-reconcile.timer running?) -- hanging up pppoe0
Sep 05 23:29:27 apitest vrrp-wan[17398]: MASTER: dialling pppoe0
Sep 05 23:29:57 apitest vrrp-wan[18610]: MASTER: dialling pppoe0
--- 172.31.1.253 ---
vip=172.31.1.1 holds_vip=no wan_disabled=yes wan_up=no ppp_up=no ppp_active=no may_dial=no lease_age=- dropin=yes role=backup
Sep 05 22:56:19 vyos vrrp-wan[196724]: DIAL FLAP: >=6 attempts in 600s -- holding off 900s
Sep 05 22:57:24 vyos vrrp-wan[199011]: MASTER with bond0.53 disabled -> enabling
Sep 05 22:57:28 vyos vrrp-wan[199711]: bond0.53 enable commit took 4s
Sep 05 22:57:40 vyos vrrp-wan[199948]: GUARD: lease stale (81s > 75s; is vrrp-wan-reconcile.timer running?) -- hanging up pppoe0
Sep 05 23:02:41 vyos vrrp-wan[206596]: MASTER: dialling pppoe0
Sep 05 23:06:17 vyos vrrp-wan[213698]: not MASTER: hanging up pppoe0
Sep 05 23:06:18 vyos vrrp-wan[213859]: not MASTER but bond0.53 enabled -> releasing
Sep 05 23:06:23 vyos vrrp-wan[214503]: bond0.53 disable commit took 5s
Sep 05 23:16:11 vyos vrrp-wan[7190]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:16:15 vyos vrrp-wan[7979]: bond0.53 enable commit took 4s
-- Boot c0e38585d7354ad6ab018800c7f3f6be --
Sep 05 23:22:54 vyos vrrp-wan[5970]: MASTER: dialling pppoe0
Sep 05 23:22:54 vyos vrrp-wan[6126]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:22:59 vyos vrrp-wan[6915]: bond0.53 enable commit took 5s
Sep 05 23:27:40 vyos vrrp-wan[16037]: not MASTER: hanging up pppoe0
Sep 05 23:27:40 vyos vrrp-wan[16198]: not MASTER but bond0.53 enabled -> releasing
Sep 05 23:27:47 vyos vrrp-wan[16840]: bond0.53 disable commit took 7s

44
labsim/wan-failover-evidence/T3-clean-failover/state.txt
View File

@@ -1,28 +1,30 @@
=== 2026-09-06T00:06:31+01:00 ===
=== 2026-09-06T00:27:58+01:00 ===
--- AC sessions ---
ifname | username | ip | ip6 | ip6-dp | calling-sid | rate-limit | state | uptime | rx-bytes | tx-bytes
--------+----------+----------------+-----+--------+-------------------+------------+--------+----------+----------+----------
ppp0 | simdsl | 198.51.100.124 | | | 52:54:00:e5:95:a2 | | active | 00:00:13 | 514 B | 204 B
ppp0 | simdsl | 198.51.100.132 | | | 52:54:00:e5:95:a2 | | active | 00:00:18 | 514 B | 204 B
--- 172.31.1.252 ---
vip=172.31.1.1 holds_vip=yes wan_disabled=no wan_up=yes ppp_up=yes ppp_active=yes may_dial=yes lease_age=3 dropin=yes role=master
pppoe0 UNKNOWN 198.51.100.124 peer 198.51.100.1/32
vip=172.31.1.1 holds_vip=yes wan_disabled=no wan_up=yes ppp_up=yes ppp_active=yes may_dial=yes lease_age=7 dropin=yes role=master
pppoe0 UNKNOWN 198.51.100.132 peer 198.51.100.1/32
default via 203.0.113.1 dev bond0.53 proto failover metric 1
Sep 05 18:08:16 apitest vrrp-wan[3010622]: not MASTER but bond0.53 enabled -> releasing
Sep 05 18:08:40 apitest vrrp-wan[3012085]: bond0.53 disable commit took 24s
Sep 05 18:11:42 apitest vrrp-wan[3016900]: MASTER: dialling pppoe0
Sep 05 18:11:42 apitest vrrp-wan[3017050]: MASTER with bond0.53 disabled -> enabling
Sep 05 18:11:46 apitest vrrp-wan[3017685]: bond0.53 enable commit took 4s
-- Boot 7cb878bdb4c74380ae7869ffd0dad9eb --
Sep 05 23:06:18 apitest vrrp-wan[18533]: MASTER: dialling pppoe0
Sep 05 23:06:18 apitest vrrp-wan[18704]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:06:22 apitest vrrp-wan[19398]: bond0.53 enable commit took 4s
Sep 05 23:13:04 apitest vrrp-wan[6413]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:13:08 apitest vrrp-wan[7122]: bond0.53 enable commit took 4s
-- Boot 46727d256db344d6ad4b37c142272071 --
Sep 05 23:18:31 apitest vrrp-wan[6258]: MASTER: dialling pppoe0
Sep 05 23:18:31 apitest vrrp-wan[6411]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:18:36 apitest vrrp-wan[7038]: bond0.53 enable commit took 5s
-- Boot 68bb1e8d56b34d55a511a301c24270fe --
Sep 05 23:27:40 apitest vrrp-wan[12502]: MASTER: dialling pppoe0
Sep 05 23:27:40 apitest vrrp-wan[12654]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:27:45 apitest vrrp-wan[13361]: bond0.53 enable commit took 5s
--- 172.31.1.253 ---
vip=172.31.1.1 holds_vip=no wan_disabled=yes wan_up=no ppp_up=no ppp_active=no may_dial=no lease_age=- dropin=yes role=backup
Sep 05 22:56:19 vyos vrrp-wan[196724]: DIAL FLAP: >=6 attempts in 600s -- holding off 900s
Sep 05 22:57:24 vyos vrrp-wan[199011]: MASTER with bond0.53 disabled -> enabling
Sep 05 22:57:28 vyos vrrp-wan[199711]: bond0.53 enable commit took 4s
Sep 05 22:57:40 vyos vrrp-wan[199948]: GUARD: lease stale (81s > 75s; is vrrp-wan-reconcile.timer running?) -- hanging up pppoe0
Sep 05 23:02:41 vyos vrrp-wan[206596]: MASTER: dialling pppoe0
Sep 05 23:06:17 vyos vrrp-wan[213698]: not MASTER: hanging up pppoe0
Sep 05 23:06:18 vyos vrrp-wan[213859]: not MASTER but bond0.53 enabled -> releasing
Sep 05 23:06:23 vyos vrrp-wan[214503]: bond0.53 disable commit took 5s
Sep 05 23:16:11 vyos vrrp-wan[7190]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:16:15 vyos vrrp-wan[7979]: bond0.53 enable commit took 4s
-- Boot c0e38585d7354ad6ab018800c7f3f6be --
Sep 05 23:22:54 vyos vrrp-wan[5970]: MASTER: dialling pppoe0
Sep 05 23:22:54 vyos vrrp-wan[6126]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:22:59 vyos vrrp-wan[6915]: bond0.53 enable commit took 5s
Sep 05 23:27:40 vyos vrrp-wan[16037]: not MASTER: hanging up pppoe0
Sep 05 23:27:40 vyos vrrp-wan[16198]: not MASTER but bond0.53 enabled -> releasing
Sep 05 23:27:47 vyos vrrp-wan[16840]: bond0.53 disable commit took 7s

27
labsim/wan-failover-evidence/T4-hard-failover-deny/state.txt
View File

@@ -1,20 +1,19 @@
=== 2026-09-06T00:21:00+01:00 ===
=== 2026-09-06T00:37:11+01:00 ===
--- AC sessions ---
ifname | username | ip | ip6 | ip6-dp | calling-sid | rate-limit | state | uptime | rx-bytes | tx-bytes
--------+----------+----------------+-----+--------+-------------------+------------+--------+----------+----------+----------
ppp0 | simdsl | 198.51.100.130 | | | 52:54:00:e5:95:a2 | | active | 00:00:18 | 438 B | 204 B
ppp0 | simdsl | 198.51.100.136 | | | 52:54:00:e5:95:a2 | | active | 00:00:13 | 438 B | 204 B
--- 172.31.1.252 ---
vip=172.31.1.1 holds_vip=yes wan_disabled=no wan_up=yes ppp_up=yes ppp_active=yes may_dial=yes lease_age=30 dropin=yes role=master
pppoe0 UNKNOWN 198.51.100.130 peer 198.51.100.1/32
vip=172.31.1.1 holds_vip=yes wan_disabled=no wan_up=yes ppp_up=yes ppp_active=yes may_dial=yes lease_age=26 dropin=yes role=master
pppoe0 UNKNOWN 198.51.100.136 peer 198.51.100.1/32
default via 203.0.113.1 dev bond0.53 proto failover metric 1
Sep 05 23:07:43 apitest vrrp-wan[22667]: MASTER: dialling pppoe0
Sep 05 23:08:14 apitest vrrp-wan[24120]: MASTER: dialling pppoe0
-- Boot c5f23399239b468c8c8b752a4305c515 --
Sep 05 23:13:04 apitest vrrp-wan[6261]: MASTER: dialling pppoe0
Sep 05 23:13:04 apitest vrrp-wan[6413]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:13:08 apitest vrrp-wan[7122]: bond0.53 enable commit took 4s
-- Boot 46727d256db344d6ad4b37c142272071 --
Sep 05 23:18:31 apitest vrrp-wan[6258]: MASTER: dialling pppoe0
Sep 05 23:18:31 apitest vrrp-wan[6411]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:18:36 apitest vrrp-wan[7038]: bond0.53 enable commit took 5s
Sep 05 23:27:40 apitest vrrp-wan[12654]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:27:45 apitest vrrp-wan[13361]: bond0.53 enable commit took 5s
Sep 05 23:29:01 apitest vrrp-wan[16369]: GUARD: lease stale (200s > 75s; is vrrp-wan-reconcile.timer running?) -- hanging up pppoe0
Sep 05 23:29:27 apitest vrrp-wan[17398]: MASTER: dialling pppoe0
Sep 05 23:29:57 apitest vrrp-wan[18610]: MASTER: dialling pppoe0
-- Boot 6efc3c9ba47f455e9668454ee6c2fc37 --
Sep 05 23:34:48 apitest vrrp-wan[6266]: MASTER: dialling pppoe0
Sep 05 23:34:49 apitest vrrp-wan[6422]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:34:54 apitest vrrp-wan[7130]: bond0.53 enable commit took 5s
--- 172.31.1.253 ---

31
labsim/wan-failover-evidence/T4-hard-failover-disable/state.txt
View File

@@ -1,21 +1,20 @@
=== 2026-09-06T00:23:19+01:00 ===
=== 2026-09-06T00:39:29+01:00 ===
--- AC sessions ---
ifname | username | ip | ip6 | ip6-dp | calling-sid | rate-limit | state | uptime | rx-bytes | tx-bytes
--------+----------+----------------+-----+--------+-------------------+------------+--------+----------+----------+----------
ppp0 | simdsl | 198.51.100.130 | | | 52:54:00:e5:95:a2 | | active | 00:02:37 | 438 B | 204 B
ppp1 | simdsl | 198.51.100.131 | | | 52:54:00:4e:0b:56 | | active | 00:00:27 | 1.1 KiB | 204 B
ppp0 | simdsl | 198.51.100.136 | | | 52:54:00:e5:95:a2 | | active | 00:02:31 | 438 B | 204 B
ppp1 | simdsl | 198.51.100.137 | | | 52:54:00:4e:0b:56 | | active | 00:00:26 | 1.0 KiB | 204 B
--- 172.31.1.252 ---
--- 172.31.1.253 ---
vip=172.31.1.1 holds_vip=yes wan_disabled=no wan_up=yes ppp_up=yes ppp_active=yes may_dial=yes lease_age=15 dropin=yes role=master
pppoe0 UNKNOWN 198.51.100.131 peer 198.51.100.1/32
default nhid 57 dev pppoe0 proto static metric 20
Sep 05 23:10:44 vyos vrrp-wan[221212]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:10:48 vyos vrrp-wan[221995]: bond0.53 enable commit took 4s
-- Boot 1f635fc6900c4661b3ac6d016d63d293 --
Sep 05 23:16:10 vyos vrrp-wan[7037]: MASTER: dialling pppoe0
Sep 05 23:16:11 vyos vrrp-wan[7190]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:16:15 vyos vrrp-wan[7979]: bond0.53 enable commit took 4s
-- Boot c0e38585d7354ad6ab018800c7f3f6be --
Sep 05 23:22:54 vyos vrrp-wan[5970]: MASTER: dialling pppoe0
Sep 05 23:22:54 vyos vrrp-wan[6126]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:22:59 vyos vrrp-wan[6915]: bond0.53 enable commit took 5s
vip=172.31.1.1 holds_vip=yes wan_disabled=no wan_up=yes ppp_up=yes ppp_active=yes may_dial=yes lease_age=5 dropin=yes role=master
pppoe0 UNKNOWN 198.51.100.137 peer 198.51.100.1/32
default nhid 60 dev pppoe0 proto static metric 20
Sep 05 23:27:40 vyos vrrp-wan[16198]: not MASTER but bond0.53 enabled -> releasing
Sep 05 23:27:47 vyos vrrp-wan[16840]: bond0.53 disable commit took 7s
Sep 05 23:32:26 vyos vrrp-wan[23806]: MASTER: dialling pppoe0
Sep 05 23:32:26 vyos vrrp-wan[23956]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:32:31 vyos vrrp-wan[24749]: bond0.53 enable commit took 5s
-- Boot 4b0355878444477fb92738339f85843d --
Sep 05 23:39:03 vyos vrrp-wan[5966]: MASTER: dialling pppoe0
Sep 05 23:39:04 vyos vrrp-wan[6120]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:39:09 vyos vrrp-wan[6914]: bond0.53 enable commit took 5s

27
labsim/wan-failover-evidence/T4-hard-failover-replace/state.txt
View File

@@ -1,19 +1,18 @@
=== 2026-09-06T00:16:41+01:00 ===
=== 2026-09-06T00:32:54+01:00 ===
--- AC sessions ---
ifname | username | ip | ip6 | ip6-dp | calling-sid | rate-limit | state | uptime | rx-bytes | tx-bytes
--------+----------+----------------+-----+--------+-------------------+------------+--------+----------+----------+----------
ppp0 | simdsl | 198.51.100.129 | | | 52:54:00:4e:0b:56 | | active | 00:00:31 | 438 B | 204 B
ppp0 | simdsl | 198.51.100.135 | | | 52:54:00:4e:0b:56 | | active | 00:00:30 | 438 B | 204 B
--- 172.31.1.252 ---
--- 172.31.1.253 ---
vip=172.31.1.1 holds_vip=yes wan_disabled=no wan_up=yes ppp_up=yes ppp_active=yes may_dial=yes lease_age=22 dropin=yes role=master
pppoe0 UNKNOWN 198.51.100.129 peer 198.51.100.1/32
default nhid 62 dev pppoe0 proto static metric 20
Sep 05 23:06:18 vyos vrrp-wan[213859]: not MASTER but bond0.53 enabled -> releasing
Sep 05 23:06:23 vyos vrrp-wan[214503]: bond0.53 disable commit took 5s
Sep 05 23:10:43 vyos vrrp-wan[221064]: MASTER: dialling pppoe0
Sep 05 23:10:44 vyos vrrp-wan[221212]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:10:48 vyos vrrp-wan[221995]: bond0.53 enable commit took 4s
-- Boot 1f635fc6900c4661b3ac6d016d63d293 --
Sep 05 23:16:10 vyos vrrp-wan[7037]: MASTER: dialling pppoe0
Sep 05 23:16:11 vyos vrrp-wan[7190]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:16:15 vyos vrrp-wan[7979]: bond0.53 enable commit took 4s
vip=172.31.1.1 holds_vip=yes wan_disabled=no wan_up=yes ppp_up=yes ppp_active=yes may_dial=yes lease_age=19 dropin=yes role=master
pppoe0 UNKNOWN 198.51.100.135 peer 198.51.100.1/32
default nhid 77 dev pppoe0 proto static metric 20
Sep 05 23:22:54 vyos vrrp-wan[6126]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:22:59 vyos vrrp-wan[6915]: bond0.53 enable commit took 5s
Sep 05 23:27:40 vyos vrrp-wan[16037]: not MASTER: hanging up pppoe0
Sep 05 23:27:40 vyos vrrp-wan[16198]: not MASTER but bond0.53 enabled -> releasing
Sep 05 23:27:47 vyos vrrp-wan[16840]: bond0.53 disable commit took 7s
Sep 05 23:32:26 vyos vrrp-wan[23806]: MASTER: dialling pppoe0
Sep 05 23:32:26 vyos vrrp-wan[23956]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:32:31 vyos vrrp-wan[24749]: bond0.53 enable commit took 5s

46
labsim/wan-failover-evidence/T5-tengig-down/state.txt
View File

@@ -1,28 +1,30 @@
=== 2026-09-06T00:07:02+01:00 ===
=== 2026-09-06T00:28:31+01:00 ===
--- AC sessions ---
ifname | username | ip | ip6 | ip6-dp | calling-sid | rate-limit | state | uptime | rx-bytes | tx-bytes
--------+----------+----------------+-----+--------+-------------------+------------+--------+----------+----------+----------
ppp0 | simdsl | 198.51.100.124 | | | 52:54:00:e5:95:a2 | | active | 00:00:44 | 682 B | 372 B
ppp0 | simdsl | 198.51.100.132 | | | 52:54:00:e5:95:a2 | | active | 00:00:52 | 682 B | 372 B
--- 172.31.1.252 ---
vip=172.31.1.1 holds_vip=yes wan_disabled=no wan_up=yes ppp_up=yes ppp_active=yes may_dial=yes lease_age=3 dropin=yes role=master
pppoe0 UNKNOWN 198.51.100.124 peer 198.51.100.1/32
default nhid 67 dev pppoe0 proto static metric 20
Sep 05 18:08:16 apitest vrrp-wan[3010622]: not MASTER but bond0.53 enabled -> releasing
Sep 05 18:08:40 apitest vrrp-wan[3012085]: bond0.53 disable commit took 24s
Sep 05 18:11:42 apitest vrrp-wan[3016900]: MASTER: dialling pppoe0
Sep 05 18:11:42 apitest vrrp-wan[3017050]: MASTER with bond0.53 disabled -> enabling
Sep 05 18:11:46 apitest vrrp-wan[3017685]: bond0.53 enable commit took 4s
-- Boot 7cb878bdb4c74380ae7869ffd0dad9eb --
Sep 05 23:06:18 apitest vrrp-wan[18533]: MASTER: dialling pppoe0
Sep 05 23:06:18 apitest vrrp-wan[18704]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:06:22 apitest vrrp-wan[19398]: bond0.53 enable commit took 4s
vip=172.31.1.1 holds_vip=yes wan_disabled=no wan_up=yes ppp_up=yes ppp_active=yes may_dial=yes lease_age=7 dropin=yes role=master
pppoe0 UNKNOWN 198.51.100.132 peer 198.51.100.1/32
default nhid 58 dev pppoe0 proto static metric 20
Sep 05 23:13:04 apitest vrrp-wan[6413]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:13:08 apitest vrrp-wan[7122]: bond0.53 enable commit took 4s
-- Boot 46727d256db344d6ad4b37c142272071 --
Sep 05 23:18:31 apitest vrrp-wan[6258]: MASTER: dialling pppoe0
Sep 05 23:18:31 apitest vrrp-wan[6411]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:18:36 apitest vrrp-wan[7038]: bond0.53 enable commit took 5s
-- Boot 68bb1e8d56b34d55a511a301c24270fe --
Sep 05 23:27:40 apitest vrrp-wan[12502]: MASTER: dialling pppoe0
Sep 05 23:27:40 apitest vrrp-wan[12654]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:27:45 apitest vrrp-wan[13361]: bond0.53 enable commit took 5s
--- 172.31.1.253 ---
vip=172.31.1.1 holds_vip=no wan_disabled=yes wan_up=no ppp_up=no ppp_active=no may_dial=no lease_age=- dropin=yes role=backup
Sep 05 22:56:19 vyos vrrp-wan[196724]: DIAL FLAP: >=6 attempts in 600s -- holding off 900s
Sep 05 22:57:24 vyos vrrp-wan[199011]: MASTER with bond0.53 disabled -> enabling
Sep 05 22:57:28 vyos vrrp-wan[199711]: bond0.53 enable commit took 4s
Sep 05 22:57:40 vyos vrrp-wan[199948]: GUARD: lease stale (81s > 75s; is vrrp-wan-reconcile.timer running?) -- hanging up pppoe0
Sep 05 23:02:41 vyos vrrp-wan[206596]: MASTER: dialling pppoe0
Sep 05 23:06:17 vyos vrrp-wan[213698]: not MASTER: hanging up pppoe0
Sep 05 23:06:18 vyos vrrp-wan[213859]: not MASTER but bond0.53 enabled -> releasing
Sep 05 23:06:23 vyos vrrp-wan[214503]: bond0.53 disable commit took 5s
Sep 05 23:16:11 vyos vrrp-wan[7190]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:16:15 vyos vrrp-wan[7979]: bond0.53 enable commit took 4s
-- Boot c0e38585d7354ad6ab018800c7f3f6be --
Sep 05 23:22:54 vyos vrrp-wan[5970]: MASTER: dialling pppoe0
Sep 05 23:22:54 vyos vrrp-wan[6126]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:22:59 vyos vrrp-wan[6915]: bond0.53 enable commit took 5s
Sep 05 23:27:40 vyos vrrp-wan[16037]: not MASTER: hanging up pppoe0
Sep 05 23:27:40 vyos vrrp-wan[16198]: not MASTER but bond0.53 enabled -> releasing
Sep 05 23:27:47 vyos vrrp-wan[16840]: bond0.53 disable commit took 7s

45
labsim/wan-failover-evidence/T8-lease-expiry/state.txt
View File

@@ -1,26 +1,27 @@
=== 2026-09-06T00:07:44+01:00 ===
=== 2026-09-06T00:29:16+01:00 ===
--- AC sessions ---
ifname | username | ip | ip6 | ip6-dp | calling-sid | rate-limit | state | uptime | rx-bytes | tx-bytes
--------+----------+----+-----+--------+-------------------+------------+-------+----------+----------+----------
| | | | | 52:54:00:e5:95:a2 | | start | 00:00:00 | 0 B | 0 B
ifname | username | ip | ip6 | ip6-dp | calling-sid | rate-limit | state | uptime | rx-bytes | tx-bytes
--------+----------+----+-----+--------+-------------+------------+-------+--------+----------+----------
--- 172.31.1.252 ---
vip=172.31.1.1 holds_vip=yes wan_disabled=no wan_up=no ppp_up=no ppp_active=yes may_dial=yes lease_age=2 dropin=yes role=master
Sep 05 18:11:42 apitest vrrp-wan[3016900]: MASTER: dialling pppoe0
Sep 05 18:11:42 apitest vrrp-wan[3017050]: MASTER with bond0.53 disabled -> enabling
Sep 05 18:11:46 apitest vrrp-wan[3017685]: bond0.53 enable commit took 4s
-- Boot 7cb878bdb4c74380ae7869ffd0dad9eb --
Sep 05 23:06:18 apitest vrrp-wan[18533]: MASTER: dialling pppoe0
Sep 05 23:06:18 apitest vrrp-wan[18704]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:06:22 apitest vrrp-wan[19398]: bond0.53 enable commit took 4s
Sep 05 23:07:35 apitest vrrp-wan[22111]: GUARD: lease stale (206s > 75s; is vrrp-wan-reconcile.timer running?) -- hanging up pppoe0
Sep 05 23:07:43 apitest vrrp-wan[22667]: MASTER: dialling pppoe0
vip=172.31.1.1 holds_vip=yes wan_disabled=no wan_up=no ppp_up=no ppp_active=no may_dial=no lease_age=- dropin=yes role=master
Sep 05 23:13:08 apitest vrrp-wan[7122]: bond0.53 enable commit took 4s
-- Boot 46727d256db344d6ad4b37c142272071 --
Sep 05 23:18:31 apitest vrrp-wan[6258]: MASTER: dialling pppoe0
Sep 05 23:18:31 apitest vrrp-wan[6411]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:18:36 apitest vrrp-wan[7038]: bond0.53 enable commit took 5s
-- Boot 68bb1e8d56b34d55a511a301c24270fe --
Sep 05 23:27:40 apitest vrrp-wan[12502]: MASTER: dialling pppoe0
Sep 05 23:27:40 apitest vrrp-wan[12654]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:27:45 apitest vrrp-wan[13361]: bond0.53 enable commit took 5s
Sep 05 23:29:01 apitest vrrp-wan[16369]: GUARD: lease stale (200s > 75s; is vrrp-wan-reconcile.timer running?) -- hanging up pppoe0
--- 172.31.1.253 ---
vip=172.31.1.1 holds_vip=no wan_disabled=yes wan_up=no ppp_up=no ppp_active=no may_dial=no lease_age=- dropin=yes role=backup
Sep 05 22:56:19 vyos vrrp-wan[196724]: DIAL FLAP: >=6 attempts in 600s -- holding off 900s
Sep 05 22:57:24 vyos vrrp-wan[199011]: MASTER with bond0.53 disabled -> enabling
Sep 05 22:57:28 vyos vrrp-wan[199711]: bond0.53 enable commit took 4s
Sep 05 22:57:40 vyos vrrp-wan[199948]: GUARD: lease stale (81s > 75s; is vrrp-wan-reconcile.timer running?) -- hanging up pppoe0
Sep 05 23:02:41 vyos vrrp-wan[206596]: MASTER: dialling pppoe0
Sep 05 23:06:17 vyos vrrp-wan[213698]: not MASTER: hanging up pppoe0
Sep 05 23:06:18 vyos vrrp-wan[213859]: not MASTER but bond0.53 enabled -> releasing
Sep 05 23:06:23 vyos vrrp-wan[214503]: bond0.53 disable commit took 5s
Sep 05 23:16:11 vyos vrrp-wan[7190]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:16:15 vyos vrrp-wan[7979]: bond0.53 enable commit took 4s
-- Boot c0e38585d7354ad6ab018800c7f3f6be --
Sep 05 23:22:54 vyos vrrp-wan[5970]: MASTER: dialling pppoe0
Sep 05 23:22:54 vyos vrrp-wan[6126]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:22:59 vyos vrrp-wan[6915]: bond0.53 enable commit took 5s
Sep 05 23:27:40 vyos vrrp-wan[16037]: not MASTER: hanging up pppoe0
Sep 05 23:27:40 vyos vrrp-wan[16198]: not MASTER but bond0.53 enabled -> releasing
Sep 05 23:27:47 vyos vrrp-wan[16840]: bond0.53 disable commit took 7s

16
migration/PPPOE-HA.md
View File

@@ -67,7 +67,15 @@ either "I do not hold the VIP" or "the lease is stale".
| Stale lease | guard hangs up within ~5s |
| Missing peers file | `NRestarts=0` — no loop |
| Flap holdoff vs live session | session survives a forged 900s holdoff |
| Invariant | AC never showed two `simdsl` sessions |
| Invariant | never more than one router dialled, in any run |
On the last row, precisely: the AC *did* report two `simdsl` sessions during the
`session-control=disable` run — one live, one orphaned from the router that had
just been destroyed, which that policy does not clean up. Only one live router
was ever dialled. The count is a proxy for the real invariant and only a valid
one while the AC enforces single-session, so it is reported as a `WARN` rather
than silenced: an orphaned session still occupies the single slot at a real ISP,
and that is exactly what made `deny` take 141–148s.
## Two failure modes found by running it, not by reading it
@@ -145,9 +153,9 @@ a separate change."* This is that change.
| policy | takeover |
|---|---|
| `replace` (accel-ppp default) | 26s |
| **`deny`** (hostile) | **148s** |
| `disable` | 21s |
| `replace` (accel-ppp default) | 26s / 26s |
| **`deny`** (hostile) | **148s / 141s** |
| `disable` | 21s / 20s |
`deny` is the sizing case: the AC refuses the survivor until its own
dead-peer timer frees the dead session, and the poller caught two dial

9
migration/vrrp-wan.conf
View File

@@ -23,11 +23,12 @@ WAN_VIF=53
# exists to prevent.
#
# MEASURED, labsim T4, master destroyed with `virsh destroy`, time until the
# survivor held a PPPoE session (labsim/wan-failover-evidence/T4-*):
# survivor held a PPPoE session (labsim/wan-failover-evidence/T4-*). Two
# independent runs, so these are the AC's behaviour rather than one-offs:
#
# session-control=replace 26s
# session-control=deny 148s <-- worst
# session-control=disable 21s
# session-control=replace 26s / 26s
# session-control=deny 148s / 141s <-- worst
# session-control=disable 21s / 20s
#
# `deny` is the hostile case and the only one that matters for sizing: the AC
# refuses the survivor until its own dead-peer timer frees the dead session.