michal/lab
1
0
Fork 0
Code Issues Pull Requests 3 Actions Packages Projects Releases Wiki Activity
Files
d0b733f8315b16bb3f391c5d5a90463e9526ada1
lab/labsim/ipv6-ha-evidence/V0-baseline/state.txt

44 lines
2.9 KiB
Plaintext
Raw Normal View History

IPv6 was never HA, and the WAN becoming HA is what exposed it Reviewed the parked IPv6 task against the PPPoE-HA work of 2026-09-05/06. The gate that parked it ("WI-8 before IPv6") is cleared, but the same work invalidated the assumption the IPv6 design rested on. Verified on the live routers: vyos002 has no tun0, no he-tunnel-follow, no he-secrets, no VLAN 9 prefix and no route6 ::/0 -- only the pre-staged default-deny v6 firewall, which is correctly on both. Failover is now automatic and drill-proven, so every failover takes the whole v6 estate down for as long as vyos002 holds the VIP. Four things that came out of checking rather than reading: - PPPOE-HA.md's "tun0 survived untouched and IPv6 stayed up at 15.5ms" does not follow from its own premise and is corrected in place. The endpoint address is stable, but it MOVES to vyos002, which has nothing to decapsulate protocol 41. wan-drill had no IPv6 check at all, which is why nobody caught it. - A 22-second near-miss: vif53-pin-boot-disable bounced the 10 gig, he-tunnel- follow ticked once and saw the PPPoE address, and vyos-failover restored the route 22s before the second tick would have pointed HE at an address Vodafone reissues on every dial. - VyOS does NOT leave a tunnel down when its source-address is absent (the override's stated reason for leaving IPv6 single-homed). Measured in labsim: it commits rc=0 and brings the link UP -- a blackhole that attracts the v6 default route. The runtime gate is load-bearing, like the PPPoE gate. - The RA link-mtu was pinned at 1480 while the tunnel correctly drops to 1472 on the PPPoE path. Mechanism, mirroring PPPoE HA -- identical config on both, gated at runtime, no commit in the failover path: - vrrp-wan-reconcile: a v6 kernel plane. tun0 and radvd follow the VIP; radvd is stopped BEFORE the WAN goes so its farewell RA (router-lifetime 0) still has a path out. The WAN early-exits became if-blocks so the plane runs every tick. It deliberately does NOT call he-tunnel-follow: that would halve the hysteresis the near-miss above showed we depend on. - he-tunnel-follow: a master guard reading the same vrrp-wan.conf VIP, so the backup copy cannot point HE at its own idle PPPoE line, plus a stubbable HE_UPDATE_URL. - vrrp-wan-install carries both, so --check and the upgrade runbook cover IPv6. - wan-drill measures IPv6 in both timing loops and asserts zero HE API calls across a router failover. labsim finally has an HE endpoint, closing the gap the override itself cited as why this was never rehearsed. Both ISP islands already share the libvirt network, so that becomes the backbone and HE lives behind it on one address reachable over either WAN. Proven in the sim: backup tun=DOWN radvd=inactive, master tun=UP radvd=active, hysteresis then HE call then MTU 1480->1472, and VLAN 9 hosts autoconfiguring from the RA. The end-to-end v6 datapath is NOT yet proven -- inter-island transit crosses libvirt NAT and the return path is lost. Recorded as a KNOWN SIM GAP rather than papered over. The model change is staged, not merged: another agent runs pulumi up on that repo, and the gate must exist on vyos002 before the tunnel does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DMVzWZgiKW2wquf5z8S1yH
2026-09-06 14:30:44 +01:00
=== 2026-09-06T14:25:25+01:00 ===
--- HE endpoint ---
HE address : 192.0.2.10/32
he-sim : he-sim: ipv6/ip remote 198.51.100.137 local 192.0.2.10 ttl 64 6rd-prefix 2002::/16
he-sim v6 : 2001:db8:1f1c:f6::1/64
API : running
API bound : nohost
API calls : 4
route back : 198.51.100.0/24 via 192.168.122.63 dev eth1
--- 172.31.1.252 ---
vip=172.31.1.1 holds_vip=no wan_disabled=yes wan_up=no ppp_up=no ppp_active=no may_dial=no lease_age=- dropin=yes role=backup tun=DOWN radvd=inactive
inactive
tun0@NONE DOWN 203.0.113.108 <POINTOPOINT,NOARP>
tun0: ipv6/ip remote 192.0.2.10 local 203.0.113.108 ttl 64 tos inherit 6rd-prefix 2002::/16
inactive
Sep 05 23:27:45 apitest vrrp-wan[13361]: bond0.53 enable commit took 5s
Sep 05 23:29:01 apitest vrrp-wan[16369]: GUARD: lease stale (200s > 75s; is vrrp-wan-reconcile.timer running?) -- hanging up pppoe0
Sep 05 23:29:27 apitest vrrp-wan[17398]: MASTER: dialling pppoe0
Sep 05 23:29:57 apitest vrrp-wan[18610]: MASTER: dialling pppoe0
-- Boot 6efc3c9ba47f455e9668454ee6c2fc37 --
Sep 05 23:34:48 apitest vrrp-wan[6266]: MASTER: dialling pppoe0
Sep 05 23:34:49 apitest vrrp-wan[6422]: MASTER with bond0.53 disabled -> enabling
Sep 05 23:34:54 apitest vrrp-wan[7130]: bond0.53 enable commit took 5s
-- Boot 866611afd7c542d4bf8c5117978dcd4a --
Sep 06 13:21:54 apitest vrrp-wan[544215]: not MASTER: stopping radvd (deprecates the v6 gateway)
Sep 06 13:21:54 apitest vrrp-wan[544221]: not MASTER: bringing tun0 down
Sep 06 13:22:01 apitest he-tunnel-follow[544377]: role is now backup
--- 172.31.1.253 ---
vip=172.31.1.1 holds_vip=yes wan_disabled=no wan_up=yes ppp_up=yes ppp_active=yes may_dial=yes lease_age=25 dropin=yes role=master tun=UNKNOWN radvd=active
tun0@NONE UNKNOWN 198.51.100.137 <POINTOPOINT,NOARP,UP,LOWER_UP>
tun0: ipv6/ip remote 192.0.2.10 local 198.51.100.137 ttl 64 tos inherit 6rd-prefix 2002::/16
default nhid 111 via 2001:db8:1f1c:f6::1 dev tun0 proto static metric 20 pref medium
active
Sep 06 09:29:21 vyos vrrp-wan[422441]: MASTER with bond0.53 disabled -> enabling
Sep 06 09:29:25 vyos vrrp-wan[423516]: bond0.53 enable commit took 4s
Sep 06 13:22:01 vyos he-tunnel-follow[593096]: role is now master
Sep 06 13:22:01 vyos he-tunnel-follow[593109]: change seen (203.0.113.108 -> 198.51.100.137) but waiting for stability (1/2)
Sep 06 13:23:01 vyos he-tunnel-follow[594799]: HE endpoint set to 198.51.100.137 (good 198.51.100.137)
Sep 06 13:23:01 vyos he-tunnel-follow[594805]: moved tun0 to pppoe0: src 203.0.113.108 -> 198.51.100.137, mtu 1480 -> 1472
Sep 06 13:24:02 vyos he-tunnel-follow[595700]: in sync: tun0 via pppoe0 src 198.51.100.137 mtu 1472
Sep 06 13:24:26 vyos he-tunnel-follow[595886]: in sync: tun0 via pppoe0 src 198.51.100.137 mtu 1472
Sep 06 13:24:26 vyos he-tunnel-follow[595904]: in sync: tun0 via pppoe0 src 198.51.100.137 mtu 1472
Sep 06 13:25:01 vyos he-tunnel-follow[596311]: in sync: tun0 via pppoe0 src 198.51.100.137 mtu 1472
Reference in New Issue Copy Permalink