labsim: capture BGP, dual WAN and both ISP VMs as code

The sim's routing config existed only as running state on the VMs. It was
applied by hand over SSH, so rebuilding a VM lost the rehearsal and nothing
recorded why any of it was shaped the way it was. The two ISP VMs were not
referenced anywhere in the repo at all.

sim-net-config.py generates all four roles; sim-net-apply.sh applies them over
the serial console, or diffs them against the running VMs. Verified reproducing
live state exactly before committing: primary 40/40 commands, secondary 16/16,
isp-dhcp 19/19, isp-pppoe 21/21.

Carries the reasoning that was previously nowhere: RFC 8212 needing policy in
both directions or the session carries zero prefixes; probe targets that must
not double as system name-servers; default-route-distance 210 rather than
no-default-route, which blanks new_routers and hands the default route to the
backup line; and the WI-8 bootstrap bug that pinned /32s fix.

Dropped a stale `pppoe-server interface eth0` on isp-pppoe (a NIC that does not
exist there) so a green drift check stays meaningful.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DMVzWZgiKW2wquf5z8S1yH

labsim/README.md

@@ -143,6 +143,49 @@ unreserved MAC gets an unreserved address.
 - **http://localhost:9101/metrics** — `labsim_reachable{src,dst,proto}` and
   `labsim_rtt_ms{src,dst}`.
 
+## Routing: BGP, dual WAN, and the ISP VMs
+
+`sim-ha-config.py` covers the LAN side of the routers. `sim-net-config.py`
+covers everything that makes this a rehearsal for production *routing*:
+
+| role | VM | what it generates |
+|---|---|---|
+| `primary` | `labsim-vyos` | BGP + dual WAN + health-checked failover |
+| `secondary` | `labsim-vyos2` | BGP only |
+| `isp-dhcp` | `labsim-isp-dhcp` | 10gig-equivalent ISP on VLAN 53 |
+| `isp-pppoe` | `labsim-isp-pppoe` | Vodafone-equivalent PPPoE ISP on VLAN 51 |
+
+Both ISP VMs are VyOS with two NICs: one on the OVS trunk facing the sim
+router, one on libvirt's `default` network, NATing customers to the real
+internet. They use RFC 5737 documentation ranges (`203.0.113.0/24`,
+`198.51.100.0/24`) so a leaked sim route cannot blackhole anything real.
+
+```sh
+./sim-net-apply.sh check     # VM state vs what the code says — run this first
+./sim-net-apply.sh apply     # push generated config over the serial console
+```
+
+`check` is the important one. All of this previously existed only as running
+state, applied by hand over SSH; rebuilding a VM lost it, and nothing recorded
+why any of it was shaped the way it was.
+
+### Known gaps vs production
+
+- **WAN is on the primary router only.** Production has WAN on both. Two PPPoE
+  clients sharing one credential against a single access concentrator is a
+  failure mode production does not have, so the sim does not model it. VRRP and
+  conntrack failover are still exercised.
+- **ISP VM interface names are not stable across a rebuild** — `isp-dhcp` came
+  up as `eth0`/`eth1` and `isp-pppoe` as `eth2`/`eth3` from identical XML.
+  Check `show interfaces` and pass `--wan-if` / `--uplink-if` rather than
+  trusting the defaults.
+- **`eth2` on the primary router** is a libvirt-NAT uplink predating the ISP
+  VMs: a third default route with no production equivalent that masks real WAN
+  failures during a failover test. `--drop-scaffold` removes it.
+- **Committing on `isp-pppoe` drops the router's PPPoE session**, and the
+  client does not redial promptly. After any change there, check `pppoe0` on
+  the router and `sudo systemctl restart ppp@pppoe0` if it is missing.
+
 ## Notes for whoever extends this
 
 Things that cost time the first time round, all verified on this image: