conntrack-sync proven working on the sim pair -- bidirectional replication with
zero errors:
MASTER internal 34 external(from peer) 52 62 pkts sent / 109 recv 0 err
BACKUP internal 76 external(from peer) 36 142 pkts sent / 73 recv 0 err
Getting there required learning something that changes the production config:
**VyOS only engages conntrack when a firewall or NAT is configured.** With
neither present, both routers reported zero conntrack entries and conntrack-sync
had nothing to replicate. Adding a single state-matching forward rule turned
tracking on and replication began immediately.
That is a defect in the delta, not just a test artifact. NAT and the firewall
were both gated behind --with-wan, so the BACKUP would have had neither -- it
would not have tracked connections at all, and replicated entries are useless to
a box whose conntrack is not engaged. Exactly the failure that only shows up
during a failover, when it is too late to notice.
Fixed: a stateful forward rule (accept established/related, default-action
accept) is now emitted on BOTH boxes, outside the WAN gate. Only NAT and the
WAN-scoped rules remain master-only. Verified: vyos002 now carries stateful
tracking and conntrack-sync but zero NAT lines. Master delta re-validated
against a real VyOS config -- no errors.
Also incidentally confirmed no-preempt: router1 rebooted and came back as
BACKUP rather than seizing the VIP, which is the opposite of what the production
pair did this afternoon (still on default preempt until cutover).
Two traps recorded while doing this:
- The detached `setsid nohup` config-apply pattern can strand a VyOS config
session. An orphaned session (dirs under /opt/vyatta/config/tmp/, PID long
dead) blocked every subsequent `set` on that box with a bare "Set failed",
and the dirs are overlay mounts so they cannot simply be deleted. Rebooting
cleared it. This pattern is used to survive losing SSH mid-change, so it is
worth knowing it has a failure mode of its own.
- Only VLAN 10 passes traffic between the two sim routers; every other VLAN
fails ARP despite identical vlan_mode/tag/trunks on both OVS bonds and
distinct MACs. VRRP forms on all six groups regardless. The sync link had to
be bond0.10 as a result. OVS-specific, absent in production, but it means
the sim proves mechanism rather than topology.
Production deltas regenerated with --conntrack-link: eth3 at 10.255.255.1/30 and
.2/30 awaiting the cable, which is not yet plugged (carrier=0 on both).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DMVzWZgiKW2wquf5z8S1yH
64e748ea94