lab/labsim/README.md

labsim — libvirt replica of the lab network

A throwaway copy of the production VLAN topology for testing routing, firewall rules and failover without touching the real network. Same VLAN IDs and roles as UniFi, deliberately different IP ranges so nothing can be confused for production.

Topology

Each VLAN is its own isolated libvirt network with one tiny Alpine VM on it.

VLAN	Name	Sim subnet	VM address	Mirrors production
1	management	172.31.1.0/24	172.31.1.10	192.168.1.0/24
2	k8s	172.31.2.0/24	172.31.2.10	192.168.8.0/23
3	kvm	172.31.3.0/24	172.31.3.10	192.168.3.0/24
9	private	172.31.9.0/24	172.31.9.10	10.0.9.0/23
10	lot	172.31.10.0/24	172.31.10.10	10.0.0.0/23
200	roomates	172.31.200.0/24	172.31.200.10	192.168.2.0/24

The sim subnet always encodes the VLAN id: 172.31.<vlan>.0/24.

Address plan, identical on every VLAN:

Address	Role
.1	gateway under test — a router VM you add (not created by default)
.2	host bridge — how you reach the VMs from this workstation
.10	the VLAN's micro VM
.254	reserved for a VRRP VIP, mirroring production

The host sits at .2 purely so you can SSH in. It is deliberately not the VMs' default route — that is .1 — so inter-VLAN tests fail loudly when no router is present instead of being silently served by the host's own routing table. libvirt also installs reject rules that stop these networks forwarding to each other, so traffic between VLANs only works once a router VM bridges them.

Usage

./labsim-up.sh              # bring up every VLAN (idempotent)
./labsim-up.sh 2 3          # only VLANs 2 and 3
./labsim-down.sh            # destroy VMs + networks, keep the base image
./labsim-down.sh --purge    # also delete the downloaded Alpine image

Each VM: 256 MB, 1 vCPU, a copy-on-write overlay on one shared 176 MB Alpine image (so six VMs cost a few MB of disk, not 1 GB).

Access

ssh alpine@172.31.2.10       # normal user (password: labsim)
ssh root@172.31.2.10         # privileged — this image has no sudo
curl http://172.31.2.10/     # hello-world page naming the VLAN

Console, when the network is the thing that is broken:

sudo virsh console labsim-2-k8s     # root / labsim

Watching it

./labsim-matrix.py --watch 2     # terminal grid, changed cells highlighted
./monitoring-up.sh               # topology page + Prometheus + Grafana

• http://localhost:9101/ — live mesh: a node per VLAN, the router in the middle, one line per pair coloured green/red with the ICMP RTT on it. Hover a line for per-direction detail. Refreshes every 5s. This is the one to watch while changing firewall rules.
• http://localhost:3000/d/labsim-matrix — Grafana (anonymous, no login) for history: when did a path flip, and how has latency moved.
• http://localhost:9101/metrics — labsim_reachable{src,dst,proto} and labsim_rtt_ms{src,dst}.

Notes for whoever extends this

Things that cost time the first time round, all verified on this image:

• No sudo. Alpine ships doas; cloud-init's sudo: directive is inert here. Use root@ for privileged work.
• cloud-init leaves users locked (!* in /etc/shadow) unless lock_passwd: false, and sshd then refuses key auth for that user.
• One failing runcmd aborts every command after it. Each entry is || true for that reason.
• busybox here has no httpd applet, and the VMs have no internet to apk add one — so the hello-world server is python3 -m http.server (python3 is already present because cloud-init depends on it).
• start-stop-daemon --exec /usr/bin/python3 matches cloud-init's own python3 at boot and refuses to start anything.
• busybox pgrep -f PATTERN matches its own argv, so a "skip if already running" guard always fires. Verified: guard_exit=0 with nothing listening.

Not modelled (yet)

VLANs are separate L2 segments rather than one 802.1Q trunk, so this exercises inter-VLAN routing but not a bond0.<vif> trunk config specifically. A router VM would attach one NIC per VLAN. Adding a tagged-trunk variant is the obvious next step if the bond/vif config itself needs testing.