Michal 86c2a36f00 feat(labsim): a real Kubernetes cluster for rehearsing Cilium <-> VyOS BGP ...

BGP is about to be added to a network that currently works, where a bad
advertisement blackholes the house. That needs somewhere to fail first.

Three Debian nodes (4 GB / 2 vCPU) on the OVS `vlan2` access ports running k3s
with flannel disabled, so Cilium is the CNI under test. Three rather than two
because ECMP is only meaningfully tested if a node can be drained and more than
one path survives.

VMs rather than k3d: the thing under test is eBGP between Cilium and VyOS across
the switch fabric, nodes peering with the router's bond0.2 leg, directly
connected. k3d would put the nodes on a container bridge -- a different L2 path,
proving something else. (It also wants Docker; this host has podman.) The
existing micro VMs are Alpine with 256 MB and 1 vCPU, which is not close to
enough.

Debian rather than the sim's Alpine base: glibc, a stock kernel that Cilium's
eBPF probes are tested against, and cloud-init that actually applies
network-config -- the Alpine base notably does not.

One trap worth recording. An earlier draft called `selected_vlans "$K8S_VLAN"`
before `ovs_up`, and since `ovs_up` re-defines the libvirt network from
SELECTED, that silently deleted the portgroups for every other VLAN. Running
VMs kept working -- their taps were already attached -- so nothing complained
until the ISP VMs needed vlan51 and vlan53 and could not be attached. It now
selects every VLAN.

The generated kubeconfig is gitignored: it carries cluster-admin credentials and
is one `git add -A` away from being committed. Regenerate with
`k8s-up.sh --kubeconfig`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DMVzWZgiKW2wquf5z8S1yH

2026-08-19 13:15:45 +01:00

8.4 KiB

Executable File

Raw Blame History

View Raw