feat(migration): export UniFi config and generate VyOS DHCP+DNS from it
Groundwork for replacing the USG with the VyOS pair without anything on the
network noticing. Three pieces:
migration/unifi-export.py pulls 13 endpoints off the classic controller into
timestamped JSON plus a normalised inventory: 11 networks, 31 DHCP
reservations, 4 port forwards, 2 firewall rules, 0 static routes. Two things
this turned up that a naive export would have lost:
- 23 of the 31 reservations carry no network_id at all -- UniFi simply does
not store the binding -- so they are resolved by subnet containment
instead. Without that, three quarters of the reservations have no subnet
to be placed in.
- 30 of the 31 sit INSIDE the DHCP pool, which UniFi's dhcpd tolerates and
which is flagged as a warning rather than discovered at cutover.
migration/unifi-to-vyos.py turns that inventory into VyOS `set` commands for
DHCP and DNS only -- the services the USG owns that VyOS must reproduce. Not a
general converter. --prod and --sim come from one code path so the config
proven in the sim and the config applied to the firewalls cannot drift. Prod
mode hard-fails if any reservation is missing, since a silent drop is the
failure mode that matters.
DNS is included because the USG resolves for 5 of 6 VLANs today: UniFi hands
out the gateway's own address whenever dhcpd_dns is empty, verified by
labmaster resolving against 192.168.8.1. Replacing the USG without a forwarder
would take DNS away from those VLANs entirely.
labsim/labsim-dhcp-test.sh proves it by booting throwaway VMs with real
production MACs -- the one piece of production config that transplants
verbatim. Safe because ovs-labsim has no physical NIC, so those MACs cannot
reach the real LAN.
Result on VyOS 2026.08 (kea), 4/4: printer1 got 172.31.10.46 from inside the
pool, sonoff-matter got 172.31.11.67 across the /23 boundary, Hubitat got its
out-of-pool .2, and an unreserved MAC got an unreserved address. kea honours
in-pool host reservations -- the open question blocking the cutover.
Supporting changes to labsim:
- VLAN 10 widened to /23. Every reservation is in LoT and LoT spans 10.0.0.x
and 10.0.1.x, which a /24 cannot represent.
- LoT's host leg moved to .3, because 10.0.0.2 is a real reservation
(Hubitat) that maps onto the host's own address.
- vlans.conf gained optional masklen and host_octet fields, defaulting to
24 and 2 so the other five VLANs are untouched.
- Fixed /etc/network/interfaces hardcoding 255.255.255.0. That file is what
actually takes effect on these Alpine guests -- cloud-init's
network-config is ignored -- so any non-/24 VLAN was silently wrong.
Two generator bugs found by VyOS rejecting the output: static-mapping names
are validated as hostnames, so underscores fail; and two devices named
"espressif" plus two named "thebeast" collided into single names, which would
have overwritten one reservation with another's address.
The raw export holds WiFi passphrases and the WAN PPPoE credentials and is
gitignored.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DMVzWZgiKW2wquf5z8S1yH
This commit is contained in:
@@ -15,10 +15,27 @@ Each VLAN is its own isolated libvirt network with one tiny Alpine VM on it.
|
|||||||
| 2 | k8s | 172.31.2.0/24 | 172.31.2.10 | 192.168.8.0/23 |
|
| 2 | k8s | 172.31.2.0/24 | 172.31.2.10 | 192.168.8.0/23 |
|
||||||
| 3 | kvm | 172.31.3.0/24 | 172.31.3.10 | 192.168.3.0/24 |
|
| 3 | kvm | 172.31.3.0/24 | 172.31.3.10 | 192.168.3.0/24 |
|
||||||
| 9 | private | 172.31.9.0/24 | 172.31.9.10 | 10.0.9.0/23 |
|
| 9 | private | 172.31.9.0/24 | 172.31.9.10 | 10.0.9.0/23 |
|
||||||
| 10 | lot | 172.31.10.0/24 | 172.31.10.10 | 10.0.0.0/23 |
|
| 10 | lot | **172.31.10.0/23** | 172.31.10.10 | 10.0.0.0/23 |
|
||||||
| 200 | roomates | 172.31.200.0/24 | 172.31.200.10 | 192.168.2.0/24 |
|
| 200 | roomates | 172.31.200.0/24 | 172.31.200.10 | 192.168.2.0/24 |
|
||||||
|
|
||||||
The sim subnet always encodes the VLAN id: `172.31.<vlan>.0/24`.
|
The sim subnet encodes the VLAN id: `172.31.<vlan>.0/24`, with one exception.
|
||||||
|
**VLAN 10 is a `/23`** because every UniFi DHCP reservation lives in LoT and LoT
|
||||||
|
spans `10.0.0.x` *and* `10.0.1.x`, which a `/24` cannot hold. The mapping stays
|
||||||
|
readable — `10.0.0.46 → 172.31.10.46`, `10.0.1.67 → 172.31.11.67`.
|
||||||
|
|
||||||
|
LoT's host leg is `.3`, not `.2`, because `10.0.0.2` is a real reservation
|
||||||
|
(Hubitat) that maps onto `172.31.10.2`. `.3` is unreserved and sits below the
|
||||||
|
DHCP pool, so it can never be handed out.
|
||||||
|
|
||||||
|
`vlans.conf` therefore takes two optional trailing fields:
|
||||||
|
|
||||||
|
```
|
||||||
|
vlan_id:name:sim_prefix:real_subnet[:masklen][:host_octet]
|
||||||
|
```
|
||||||
|
|
||||||
|
defaulting to `24` and `2`. k8s and Private are also `/23` in production but
|
||||||
|
hold no reservations, so they keep their `/24` and their DHCP range is clamped
|
||||||
|
— reported at generation time, never silently.
|
||||||
|
|
||||||
Address plan, identical on every VLAN:
|
Address plan, identical on every VLAN:
|
||||||
|
|
||||||
@@ -69,6 +86,32 @@ sudo virsh console labsim-2-k8s # root / labsim
|
|||||||
./monitoring-up.sh # topology page + Prometheus + Grafana
|
./monitoring-up.sh # topology page + Prometheus + Grafana
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Testing the DHCP migration
|
||||||
|
|
||||||
|
`./labsim-dhcp-test.sh` boots throwaway VMs whose MACs are **real production
|
||||||
|
MACs** and checks each gets the address UniFi reserved for it. MACs are the one
|
||||||
|
piece of production config that transplants verbatim, which is what makes this a
|
||||||
|
test rather than a rehearsal. It is safe because `ovs-labsim` has no physical
|
||||||
|
NIC — verified with `ovs-vsctl show` — so a production MAC cannot reach the real
|
||||||
|
LAN.
|
||||||
|
|
||||||
|
Apply the config first, from `../migration`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 unifi-to-vyos.py --mode sim -o /tmp/sim.conf # 6 subnets, 31 mappings
|
||||||
|
# load onto labsim-vyos, then:
|
||||||
|
./labsim-dhcp-test.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
**Result on VyOS 2026.08 (kea): all four cases pass.** The one that mattered:
|
||||||
|
30 of the 31 UniFi reservations sit *inside* the DHCP pool, and **kea honours
|
||||||
|
in-pool host reservations** — `printer1` received `172.31.10.46` from within the
|
||||||
|
`.10.11–.11.254` pool. That was the open question blocking the cutover.
|
||||||
|
|
||||||
|
Still open: whether kea will hand a *reserved* address to a *different* client
|
||||||
|
while the reserved device is offline. The negative case here only proves an
|
||||||
|
unreserved MAC gets an unreserved address.
|
||||||
|
|
||||||
- **http://localhost:9101/** — live mesh: a node per VLAN, the router in the
|
- **http://localhost:9101/** — live mesh: a node per VLAN, the router in the
|
||||||
middle, one line per pair coloured green/red with the ICMP RTT on it. Hover a
|
middle, one line per pair coloured green/red with the ICMP RTT on it. Hover a
|
||||||
line for per-direction detail. Refreshes every 5s. This is the one to watch
|
line for per-direction detail. Refreshes every 5s. This is the one to watch
|
||||||
|
|||||||
189
labsim/labsim-dhcp-test.sh
Executable file
189
labsim/labsim-dhcp-test.sh
Executable file
@@ -0,0 +1,189 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Prove that VyOS hands each device the address UniFi reserved for it.
|
||||||
|
#
|
||||||
|
# The question this answers is narrow and important: 30 of the 31 UniFi
|
||||||
|
# reservations sit INSIDE the DHCP pool (LoT's pool is 10.0.0.11-10.0.1.254 and
|
||||||
|
# only 10.0.0.2 falls outside it). UniFi's dhcpd tolerates that. VyOS uses kea,
|
||||||
|
# and whether kea honours in-pool host reservations decides whether the cutover
|
||||||
|
# silently renumbers 30 devices. That is not something to predict.
|
||||||
|
#
|
||||||
|
# Method: boot throwaway VMs whose MAC is a REAL production MAC, on the sim
|
||||||
|
# VLAN, and check the address they are given. MACs are the one piece of
|
||||||
|
# production config that transplants verbatim -- the subnet is rewritten, the
|
||||||
|
# MAC is not -- which is what makes this a real test rather than a rehearsal.
|
||||||
|
#
|
||||||
|
# Safe: the ovs-labsim bridge contains only internal ports and VM taps, with no
|
||||||
|
# physical NIC, so a production MAC here cannot reach or confuse the real LAN.
|
||||||
|
# Verified with `ovs-vsctl show` before this script was written.
|
||||||
|
#
|
||||||
|
# ./labsim-dhcp-test.sh run the standard cases
|
||||||
|
# ./labsim-dhcp-test.sh --keep leave the VMs up for inspection
|
||||||
|
# ./labsim-dhcp-test.sh --clean just remove any leftover test VMs
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||||
|
source "$SCRIPT_DIR/lib.sh"
|
||||||
|
|
||||||
|
ROUTER_IP="${ROUTER_IP:-172.31.1.1}"
|
||||||
|
ROUTER_PW="${ROUTER_PW:-vyos}"
|
||||||
|
TEST_VLAN="${TEST_VLAN:-10}"
|
||||||
|
BOOT_WAIT="${BOOT_WAIT:-150}"
|
||||||
|
TAG="labsim-dhcptest"
|
||||||
|
|
||||||
|
# mac|expected|why. "POOL" means: must get an address from the pool and must
|
||||||
|
# NOT get any reserved address -- the negative case that stops a pass from
|
||||||
|
# meaning merely "DHCP works".
|
||||||
|
CASES=(
|
||||||
|
"f8:0d:ac:90:65:c6|172.31.10.46|printer1 - reservation inside the pool"
|
||||||
|
"1c:69:20:7f:bc:77|172.31.11.67|sonoff-matter - in-pool AND across the /23 boundary"
|
||||||
|
"34:e1:d1:80:29:ce|172.31.10.2|Hubitat - the one reservation OUTSIDE the pool"
|
||||||
|
"52:54:00:ab:cd:ef|POOL|unreserved MAC - must get a pool address, not a reserved one"
|
||||||
|
)
|
||||||
|
|
||||||
|
vm_of() { echo "${TAG}-$(echo "$1" | tr -d ':')"; }
|
||||||
|
|
||||||
|
cleanup_vms() {
|
||||||
|
local n=0
|
||||||
|
while read -r vm; do
|
||||||
|
[ -z "$vm" ] && continue
|
||||||
|
virsh_q destroy "$vm" >/dev/null 2>&1
|
||||||
|
virsh_q undefine "$vm" --remove-all-storage >/dev/null 2>&1
|
||||||
|
n=$((n + 1))
|
||||||
|
done < <(virsh_q list --all --name 2>/dev/null | grep "^${TAG}-" || true)
|
||||||
|
[ "$n" -gt 0 ] && log "removed $n test VM(s)"
|
||||||
|
sudo rm -f "$IMG_DIR/${TAG}-"*.qcow2 "$IMG_DIR/${TAG}-"*-seed.iso 2>/dev/null
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# A seed that asks for DHCP instead of taking a static address. Alpine's
|
||||||
|
# cloud-init ignores network-config here (verified previously and documented in
|
||||||
|
# README), so /etc/network/interfaces is what actually takes effect.
|
||||||
|
build_dhcp_seed() {
|
||||||
|
local iso="$1" vm="$2" pubkey="$3"
|
||||||
|
local tmp; tmp="$(mktemp -d)"
|
||||||
|
cat > "$tmp/meta-data" <<EOF
|
||||||
|
instance-id: $vm
|
||||||
|
local-hostname: $vm
|
||||||
|
EOF
|
||||||
|
cat > "$tmp/user-data" <<EOF
|
||||||
|
#cloud-config
|
||||||
|
hostname: $vm
|
||||||
|
users:
|
||||||
|
- name: alpine
|
||||||
|
shell: /bin/ash
|
||||||
|
lock_passwd: false
|
||||||
|
plain_text_passwd: labsim
|
||||||
|
ssh_authorized_keys:
|
||||||
|
- $pubkey
|
||||||
|
ssh_authorized_keys:
|
||||||
|
- $pubkey
|
||||||
|
disable_root: false
|
||||||
|
chpasswd:
|
||||||
|
list: |
|
||||||
|
root:labsim
|
||||||
|
expire: false
|
||||||
|
write_files:
|
||||||
|
- path: /etc/network/interfaces
|
||||||
|
content: |
|
||||||
|
auto lo
|
||||||
|
iface lo inet loopback
|
||||||
|
auto eth0
|
||||||
|
iface eth0 inet dhcp
|
||||||
|
runcmd:
|
||||||
|
- [ sh, -c, "ifdown eth0 2>/dev/null; ifup eth0 || udhcpc -i eth0 -q || true" ]
|
||||||
|
EOF
|
||||||
|
python3 - "$tmp/user-data" <<'PY' || die "generated user-data is not valid YAML"
|
||||||
|
import sys, yaml
|
||||||
|
yaml.safe_load(open(sys.argv[1]).read().split("#cloud-config",1)[1])
|
||||||
|
PY
|
||||||
|
sudo genisoimage -quiet -output "$iso" -volid cidata -joliet -rock \
|
||||||
|
"$tmp/user-data" "$tmp/meta-data" >/dev/null 2>&1 || die "seed build failed"
|
||||||
|
rm -rf "$tmp"
|
||||||
|
}
|
||||||
|
|
||||||
|
router() {
|
||||||
|
timeout 30 sshpass -p "$ROUTER_PW" ssh -o StrictHostKeyChecking=no \
|
||||||
|
-o BatchMode=no -o ConnectTimeout=8 "vyos@$ROUTER_IP" "$@" 2>/dev/null
|
||||||
|
}
|
||||||
|
|
||||||
|
# --- argument handling ----------------------------------------------------
|
||||||
|
KEEP=0
|
||||||
|
case "${1:-}" in
|
||||||
|
--clean) cleanup_vms; exit 0 ;;
|
||||||
|
--keep) KEEP=1 ;;
|
||||||
|
"") ;;
|
||||||
|
*) die "usage: $0 [--keep|--clean]" ;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
command -v sshpass >/dev/null || die "sshpass required"
|
||||||
|
require_tools
|
||||||
|
[ -f "$BASE_IMAGE" ] || die "base image missing: $BASE_IMAGE (run labsim-up.sh first)"
|
||||||
|
|
||||||
|
log "checking the router is serving DHCP..."
|
||||||
|
subnets=$(router '/opt/vyatta/bin/vyatta-op-cmd-wrapper show configuration commands | grep -c subnet-id')
|
||||||
|
maps=$(router '/opt/vyatta/bin/vyatta-op-cmd-wrapper show configuration commands | grep -c "static-mapping .* mac"')
|
||||||
|
log " router has ${subnets:-0} subnets and ${maps:-0} static-mappings"
|
||||||
|
[ "${maps:-0}" -gt 0 ] || die "router has no static-mappings -- apply the generated config first"
|
||||||
|
|
||||||
|
cleanup_vms
|
||||||
|
SSH_PUB="$(find_ssh_pubkey)"
|
||||||
|
sudo mkdir -p "$IMG_DIR"
|
||||||
|
|
||||||
|
# --- boot one VM per case -------------------------------------------------
|
||||||
|
for c in "${CASES[@]}"; do
|
||||||
|
IFS='|' read -r mac expected why <<<"$c"
|
||||||
|
vm="$(vm_of "$mac")"
|
||||||
|
disk="$IMG_DIR/${vm}.qcow2"; seed="$IMG_DIR/${vm}-seed.iso"
|
||||||
|
log "booting $vm mac=$mac ($why)"
|
||||||
|
sudo qemu-img create -q -f qcow2 -F qcow2 -b "$BASE_IMAGE" "$disk" "$VM_DISK" >/dev/null
|
||||||
|
build_dhcp_seed "$seed" "$vm" "$SSH_PUB"
|
||||||
|
sudo virt-install --connect "$LIBVIRT_URI" --name "$vm" \
|
||||||
|
--memory "$VM_MEM" --vcpus "$VM_CPUS" \
|
||||||
|
--disk "path=$disk,format=qcow2,bus=virtio" \
|
||||||
|
--disk "path=$seed,device=cdrom,readonly=on" \
|
||||||
|
--network "network=labsim-ovs,portgroup=vlan${TEST_VLAN},model=virtio,mac=$mac" \
|
||||||
|
--os-variant alpinelinux3.18 --graphics none --noautoconsole --import >/dev/null \
|
||||||
|
|| die "virt-install failed for $vm"
|
||||||
|
done
|
||||||
|
|
||||||
|
log "waiting ${BOOT_WAIT}s for boot + DHCP..."
|
||||||
|
sleep "$BOOT_WAIT"
|
||||||
|
|
||||||
|
# --- verdict --------------------------------------------------------------
|
||||||
|
# The lease table on the router is the authority: it says what the server
|
||||||
|
# decided, independent of whether the guest brought the interface up cleanly.
|
||||||
|
leases="$(router '/opt/vyatta/bin/vyatta-op-cmd-wrapper show dhcp server leases')"
|
||||||
|
echo
|
||||||
|
echo "=== router lease table ==="
|
||||||
|
echo "$leases"
|
||||||
|
echo
|
||||||
|
|
||||||
|
reserved_ips="$(cd "$SCRIPT_DIR/../migration" && python3 unifi-to-vyos.py --mode sim 2>/dev/null \
|
||||||
|
| awk '/static-mapping .* ip-address/ {print $NF}')"
|
||||||
|
|
||||||
|
pass=0; fail=0
|
||||||
|
printf '%-19s %-16s %-16s %s\n' "MAC" "EXPECTED" "GOT" "RESULT"
|
||||||
|
for c in "${CASES[@]}"; do
|
||||||
|
IFS='|' read -r mac expected why <<<"$c"
|
||||||
|
got="$(echo "$leases" | awk -v m="$mac" 'tolower($0) ~ tolower(m) {print $1; exit}')"
|
||||||
|
got="${got:-<none>}"
|
||||||
|
if [ "$expected" = "POOL" ]; then
|
||||||
|
if [ "$got" = "<none>" ]; then
|
||||||
|
result="FAIL (no lease at all)"
|
||||||
|
elif echo "$reserved_ips" | grep -qx "$got"; then
|
||||||
|
result="FAIL (got a RESERVED address)"
|
||||||
|
else
|
||||||
|
result="pass"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
[ "$got" = "$expected" ] && result="pass" || result="FAIL"
|
||||||
|
fi
|
||||||
|
[ "$result" = "pass" ] && pass=$((pass + 1)) || fail=$((fail + 1))
|
||||||
|
printf '%-19s %-16s %-16s %s\n' "$mac" "$expected" "$got" "$result"
|
||||||
|
printf ' %s\n' "$why"
|
||||||
|
done
|
||||||
|
|
||||||
|
echo
|
||||||
|
log "$pass passed, $fail failed"
|
||||||
|
[ "$KEEP" -eq 1 ] && log "VMs left running (--keep). Remove with: $0 --clean" || cleanup_vms
|
||||||
|
[ "$fail" -eq 0 ] || exit 1
|
||||||
@@ -78,9 +78,15 @@ def load_vlans() -> list[dict]:
|
|||||||
line = line.strip()
|
line = line.strip()
|
||||||
if not line or line.startswith("#"):
|
if not line or line.startswith("#"):
|
||||||
continue
|
continue
|
||||||
vid, name, prefix, real = line.split(":", 3)
|
# masklen and host_octet are optional trailing fields; VLAN 10 sets
|
||||||
|
# both because it must be a /23 (see vlans.conf).
|
||||||
|
parts = line.split(":")
|
||||||
|
vid, name, prefix, real = parts[0], parts[1], parts[2], parts[3]
|
||||||
|
masklen = int(parts[4]) if len(parts) > 4 and parts[4] else 24
|
||||||
|
host = parts[5] if len(parts) > 5 and parts[5] else "2"
|
||||||
vlans.append({"vid": vid, "name": name, "ip": f"{prefix}.10",
|
vlans.append({"vid": vid, "name": name, "ip": f"{prefix}.10",
|
||||||
"label": f"{vid}:{name}", "real": real})
|
"label": f"{vid}:{name}", "real": real,
|
||||||
|
"masklen": masklen, "host_ip": f"{prefix}.{host}"})
|
||||||
return vlans
|
return vlans
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -25,7 +25,8 @@ ovs_up
|
|||||||
|
|
||||||
# --- VMs ------------------------------------------------------------------
|
# --- VMs ------------------------------------------------------------------
|
||||||
for entry in "${SELECTED[@]}"; do
|
for entry in "${SELECTED[@]}"; do
|
||||||
IFS=: read -r vid name prefix real <<<"$entry"
|
parse_vlan_entry "$entry"
|
||||||
|
vid="$V_VID"; name="$V_NAME"; prefix="$V_PREFIX"; real="$V_REAL"
|
||||||
vm="$(vm_name "$vid" "$name")"
|
vm="$(vm_name "$vid" "$name")"
|
||||||
ip="${prefix}.10"
|
ip="${prefix}.10"
|
||||||
|
|
||||||
@@ -48,7 +49,7 @@ for entry in "${SELECTED[@]}"; do
|
|||||||
# Copy-on-write overlay: each VM costs a few MB, not 176.
|
# Copy-on-write overlay: each VM costs a few MB, not 176.
|
||||||
sudo qemu-img create -q -f qcow2 -F qcow2 -b "$BASE_IMAGE" "$disk" "$VM_DISK" >/dev/null
|
sudo qemu-img create -q -f qcow2 -F qcow2 -b "$BASE_IMAGE" "$disk" "$VM_DISK" >/dev/null
|
||||||
|
|
||||||
build_seed "$seed" "$vm" "$vid" "$name" "$prefix" "$ip" "$real" "$SSH_PUB"
|
build_seed "$seed" "$vm" "$vid" "$name" "$prefix" "$ip" "$real" "$SSH_PUB" "$V_MASK"
|
||||||
|
|
||||||
sudo virt-install \
|
sudo virt-install \
|
||||||
--connect "$LIBVIRT_URI" \
|
--connect "$LIBVIRT_URI" \
|
||||||
|
|||||||
@@ -58,9 +58,32 @@ selected_vlans() {
|
|||||||
[ ${#SELECTED[@]} -gt 0 ] || die "no VLANs selected (checked $CONF)"
|
[ ${#SELECTED[@]} -gt 0 ] || die "no VLANs selected (checked $CONF)"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# Split one vlans.conf line, applying defaults for the two optional trailing
|
||||||
|
# fields. Sets V_VID V_NAME V_PREFIX V_REAL V_MASK V_HOST.
|
||||||
|
parse_vlan_entry() {
|
||||||
|
IFS=: read -r V_VID V_NAME V_PREFIX V_REAL V_MASK V_HOST <<<"$1"
|
||||||
|
V_MASK="${V_MASK:-24}"
|
||||||
|
V_HOST="${V_HOST:-2}"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Dotted netmask for a prefix length — cloud-init's network-config v1 wants the
|
||||||
|
# dotted form, not a /len. /24 -> 255.255.255.0, /23 -> 255.255.254.0.
|
||||||
|
netmask_for() {
|
||||||
|
local len="$1" i bits out=()
|
||||||
|
for i in 0 1 2 3; do
|
||||||
|
bits=$(( len - i * 8 ))
|
||||||
|
(( bits > 8 )) && bits=8
|
||||||
|
(( bits < 0 )) && bits=0
|
||||||
|
out+=( $(( 256 - 2 ** (8 - bits) )) )
|
||||||
|
done
|
||||||
|
local IFS=.; echo "${out[*]}"
|
||||||
|
}
|
||||||
|
|
||||||
# cloud-init NoCloud seed: static addressing + SSH key + hello-world HTTP.
|
# cloud-init NoCloud seed: static addressing + SSH key + hello-world HTTP.
|
||||||
build_seed() {
|
build_seed() {
|
||||||
local iso="$1" vm="$2" vid="$3" name="$4" prefix="$5" ip="$6" real="$7" pubkey="$8"
|
local iso="$1" vm="$2" vid="$3" name="$4" prefix="$5" ip="$6" real="$7" pubkey="$8"
|
||||||
|
local masklen="${9:-24}"
|
||||||
|
local netmask; netmask="$(netmask_for "$masklen")"
|
||||||
local tmp; tmp="$(mktemp -d)"
|
local tmp; tmp="$(mktemp -d)"
|
||||||
|
|
||||||
cat > "$tmp/meta-data" <<EOF
|
cat > "$tmp/meta-data" <<EOF
|
||||||
@@ -85,7 +108,7 @@ config:
|
|||||||
subnets:
|
subnets:
|
||||||
- type: static
|
- type: static
|
||||||
address: $ip
|
address: $ip
|
||||||
netmask: 255.255.255.0
|
netmask: $netmask
|
||||||
# Default route via the router under test. Without this the VMs can
|
# Default route via the router under test. Without this the VMs can
|
||||||
# reach their own /24 and their gateway, but nothing beyond it — which
|
# reach their own /24 and their gateway, but nothing beyond it — which
|
||||||
# looks exactly like "the router is broken" in the matrix.
|
# looks exactly like "the router is broken" in the matrix.
|
||||||
@@ -122,14 +145,14 @@ write_files:
|
|||||||
auto eth0
|
auto eth0
|
||||||
iface eth0 inet static
|
iface eth0 inet static
|
||||||
address $ip
|
address $ip
|
||||||
netmask 255.255.255.0
|
netmask $netmask
|
||||||
post-up ip route add default via ${prefix}.1 || true
|
post-up ip route add default via ${prefix}.1 || true
|
||||||
- path: /var/www/index.html
|
- path: /var/www/index.html
|
||||||
content: |
|
content: |
|
||||||
<html><body>
|
<html><body>
|
||||||
<h1>labsim vlan $vid — $name</h1>
|
<h1>labsim vlan $vid — $name</h1>
|
||||||
<p>host: $vm</p>
|
<p>host: $vm</p>
|
||||||
<p>address: $ip/24</p>
|
<p>address: $ip/$masklen</p>
|
||||||
<p>gateway under test: ${prefix}.1</p>
|
<p>gateway under test: ${prefix}.1</p>
|
||||||
<p>mirrors production: $real</p>
|
<p>mirrors production: $real</p>
|
||||||
</body></html>
|
</body></html>
|
||||||
|
|||||||
@@ -41,12 +41,15 @@ ovs_up() {
|
|||||||
# default route (.1 is), so inter-VLAN tests exercise the router, not the
|
# default route (.1 is), so inter-VLAN tests exercise the router, not the
|
||||||
# host's routing table.
|
# host's routing table.
|
||||||
for entry in "${SELECTED[@]}"; do
|
for entry in "${SELECTED[@]}"; do
|
||||||
IFS=: read -r vid _name prefix _real <<<"$entry"
|
parse_vlan_entry "$entry"
|
||||||
local port="hostv${vid}"
|
local port="hostv${V_VID}"
|
||||||
ovs --may-exist add-port "$OVS_BR" "$port" tag="$vid" \
|
ovs --may-exist add-port "$OVS_BR" "$port" tag="$V_VID" \
|
||||||
-- set interface "$port" type=internal
|
-- set interface "$port" type=internal
|
||||||
sudo ip link set "$port" up 2>/dev/null || true
|
sudo ip link set "$port" up 2>/dev/null || true
|
||||||
sudo ip addr replace "${prefix}.2/24" dev "$port"
|
# Drop any address from a previous mask/octet so a changed vlans.conf does
|
||||||
|
# not leave a stale second address on the port.
|
||||||
|
sudo ip -4 addr flush dev "$port" 2>/dev/null || true
|
||||||
|
sudo ip addr replace "${V_PREFIX}.${V_HOST}/${V_MASK}" dev "$port"
|
||||||
done
|
done
|
||||||
|
|
||||||
ovs_define_libvirt_net
|
ovs_define_libvirt_net
|
||||||
|
|||||||
@@ -12,10 +12,21 @@
|
|||||||
# .10 the micro VM for this VLAN
|
# .10 the micro VM for this VLAN
|
||||||
# .254 VRRP VIP (reserved, mirrors production)
|
# .254 VRRP VIP (reserved, mirrors production)
|
||||||
#
|
#
|
||||||
# Format: vlan_id:name:sim_subnet_prefix:real_subnet(for reference)
|
# Format: vlan_id:name:sim_subnet_prefix:real_subnet:[masklen]:[host_octet]
|
||||||
|
#
|
||||||
|
# masklen defaults to 24 and host_octet to 2. Both exist for VLAN 10, which is
|
||||||
|
# the one VLAN that has to be a /23 here: every UniFi DHCP reservation lives in
|
||||||
|
# LoT, and LoT spans 10.0.0.x AND 10.0.1.x, which a /24 cannot represent. With
|
||||||
|
# /23 the mapping stays readable — 10.0.0.46 -> 172.31.10.46 and
|
||||||
|
# 10.0.1.67 -> 172.31.11.67.
|
||||||
|
#
|
||||||
|
# LoT's host leg is .3 rather than .2 because 10.0.0.2 is a real reservation
|
||||||
|
# (Hubitat) and would map straight onto the host's own address. .3 is free in
|
||||||
|
# production and sits below the DHCP pool (which starts at .11), so it can
|
||||||
|
# never be handed out.
|
||||||
1:management:172.31.1:192.168.1.0/24
|
1:management:172.31.1:192.168.1.0/24
|
||||||
2:k8s:172.31.2:192.168.8.0/23
|
2:k8s:172.31.2:192.168.8.0/23
|
||||||
3:kvm:172.31.3:192.168.3.0/24
|
3:kvm:172.31.3:192.168.3.0/24
|
||||||
9:private:172.31.9:10.0.9.0/23
|
9:private:172.31.9:10.0.9.0/23
|
||||||
10:lot:172.31.10:10.0.0.0/23
|
10:lot:172.31.10:10.0.0.0/23:23:3
|
||||||
200:roomates:172.31.200:192.168.2.0/24
|
200:roomates:172.31.200:192.168.2.0/24
|
||||||
|
|||||||
4
migration/.gitignore
vendored
Normal file
4
migration/.gitignore
vendored
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
# Raw UniFi export: contains WiFi passphrases (wlanconf) and controller auth
|
||||||
|
# material (setting). The inventory is regenerable — never commit it.
|
||||||
|
export/
|
||||||
|
__pycache__/
|
||||||
51
migration/_unifi.py
Executable file
51
migration/_unifi.py
Executable file
@@ -0,0 +1,51 @@
|
|||||||
|
"""Shared UniFi API client for the migration tooling.
|
||||||
|
|
||||||
|
The controller is a CLASSIC self-hosted UniFi Network app (server_version
|
||||||
|
10.4.x), not UniFi OS: login is /api/login and data lives under
|
||||||
|
/api/s/<site>/... . UniFi OS would use /api/auth/login + /proxy/network/api.
|
||||||
|
Credentials come from the mcpctl server definition so they are not duplicated
|
||||||
|
here.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json, re, ssl, subprocess, urllib.request, http.cookiejar
|
||||||
|
|
||||||
|
|
||||||
|
def client():
|
||||||
|
raw = subprocess.run(["mcpctl", "describe", "server", "unifi-network"],
|
||||||
|
capture_output=True, text=True).stdout
|
||||||
|
m = re.search(r"UNIFI_TARGETS\s+(\[.*)", raw)
|
||||||
|
if not m:
|
||||||
|
raise SystemExit("could not read UNIFI_TARGETS from mcpctl")
|
||||||
|
blob = m.group(1).strip()
|
||||||
|
try:
|
||||||
|
targets = json.loads(blob)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
targets = json.loads(blob + "}" * (blob.count("{") - blob.count("}")))
|
||||||
|
t = targets[0]
|
||||||
|
base = t["base_url"].rstrip("/")
|
||||||
|
auth = t.get("auth", {})
|
||||||
|
site = t.get("default_site", "default")
|
||||||
|
|
||||||
|
ctx = ssl.create_default_context()
|
||||||
|
ctx.check_hostname = False
|
||||||
|
ctx.verify_mode = ssl.CERT_NONE
|
||||||
|
opener = urllib.request.build_opener(
|
||||||
|
urllib.request.HTTPCookieProcessor(http.cookiejar.CookieJar()),
|
||||||
|
urllib.request.HTTPSHandler(context=ctx))
|
||||||
|
req = urllib.request.Request(
|
||||||
|
f"{base}/api/login",
|
||||||
|
data=json.dumps({"username": auth.get("username"),
|
||||||
|
"password": auth.get("password")}).encode(),
|
||||||
|
headers={"Content-Type": "application/json"})
|
||||||
|
opener.open(req, timeout=20).read()
|
||||||
|
return opener, base, site
|
||||||
|
|
||||||
|
|
||||||
|
def get(opener, base, site, path):
|
||||||
|
"""GET /api/s/<site>/<path>, returning the `data` list (never raising)."""
|
||||||
|
try:
|
||||||
|
body = opener.open(f"{base}/api/s/{site}/{path}", timeout=30).read()
|
||||||
|
return json.loads(body).get("data", [])
|
||||||
|
except Exception as exc:
|
||||||
|
return {"__error__": f"{type(exc).__name__}: {exc}"}
|
||||||
335
migration/unifi-export.py
Executable file
335
migration/unifi-export.py
Executable file
@@ -0,0 +1,335 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Export everything from the UniFi controller that the VyOS cutover must preserve.
|
||||||
|
|
||||||
|
The point is that nothing quietly stops working after the switch. That means
|
||||||
|
capturing not just the networks but every DHCP reservation, every port forward
|
||||||
|
and every firewall rule — the things nobody remembers configuring until they
|
||||||
|
break.
|
||||||
|
|
||||||
|
Writes one JSON file per endpoint into ./export/ (raw, unmodified — the source
|
||||||
|
of truth) plus inventory.json, a normalised view used by the VyOS generator.
|
||||||
|
|
||||||
|
./unifi-export.py # export to ./export/
|
||||||
|
./unifi-export.py --out /tmp/x # elsewhere
|
||||||
|
./unifi-export.py --summary # print a human summary of what was found
|
||||||
|
|
||||||
|
WARNING: the raw export contains secrets (wlanconf holds WiFi passphrases,
|
||||||
|
setting holds RADIUS/auth material). ./export/ is gitignored — keep it that way.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import ipaddress
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
|
||||||
|
import _unifi
|
||||||
|
|
||||||
|
# endpoint -> why it matters for the cutover
|
||||||
|
ENDPOINTS = {
|
||||||
|
"rest/networkconf": "networks: VLANs, subnets, DHCP ranges, DNS, lease time",
|
||||||
|
"rest/user": "known clients — this is where fixed DHCP reservations live",
|
||||||
|
"stat/sta": "currently active clients and their live IPs",
|
||||||
|
"rest/firewallrule": "firewall rules",
|
||||||
|
"rest/firewallgroup": "address/port groups referenced by rules",
|
||||||
|
"rest/portforward": "port forwards (inbound NAT)",
|
||||||
|
"rest/routing": "static routes",
|
||||||
|
"rest/dhcpoption": "custom DHCP options",
|
||||||
|
"rest/wlanconf": "wireless networks (VLAN bindings)",
|
||||||
|
"stat/device": "switches/APs incl. per-port VLAN config",
|
||||||
|
"rest/setting": "controller settings (incl. USG/gateway config)",
|
||||||
|
"rest/usergroup": "bandwidth groups referenced by clients",
|
||||||
|
"rest/dynamicdns": "dynamic DNS",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def build_inventory(raw: dict) -> dict:
|
||||||
|
"""Normalise the parts a migration actually has to reproduce."""
|
||||||
|
nets_by_id = {n["_id"]: n for n in raw.get("rest/networkconf", []) if isinstance(n, dict)}
|
||||||
|
|
||||||
|
networks = []
|
||||||
|
for n in raw.get("rest/networkconf", []):
|
||||||
|
if not isinstance(n, dict):
|
||||||
|
continue
|
||||||
|
networks.append({
|
||||||
|
"id": n.get("_id"),
|
||||||
|
"name": n.get("name"),
|
||||||
|
"purpose": n.get("purpose"),
|
||||||
|
"vlan": n.get("vlan"),
|
||||||
|
"vlan_enabled": n.get("vlan_enabled"),
|
||||||
|
"subnet": n.get("ip_subnet"),
|
||||||
|
"domain_name": n.get("domain_name"),
|
||||||
|
"dhcp_enabled": n.get("dhcpd_enabled"),
|
||||||
|
"dhcp_start": n.get("dhcpd_start"),
|
||||||
|
"dhcp_stop": n.get("dhcpd_stop"),
|
||||||
|
"dhcp_lease": n.get("dhcpd_leasetime"),
|
||||||
|
"dhcp_dns": [n.get(f"dhcpd_dns_{i}") for i in (1, 2, 3, 4) if n.get(f"dhcpd_dns_{i}")],
|
||||||
|
"dhcp_gateway": n.get("dhcpd_gateway") or n.get("dhcpd_gateway_enabled"),
|
||||||
|
"dhcp_ntp": [n.get(f"dhcpd_ntp_{i}") for i in (1, 2) if n.get(f"dhcpd_ntp_{i}")],
|
||||||
|
"igmp_snooping": n.get("igmp_snooping"),
|
||||||
|
"enabled": n.get("enabled", True),
|
||||||
|
})
|
||||||
|
|
||||||
|
# ip_subnet is the GATEWAY address with a prefix ("10.0.0.1/23"), not the
|
||||||
|
# network address — so derive the real subnet before matching against it.
|
||||||
|
subnets = []
|
||||||
|
for n in networks:
|
||||||
|
if not n["subnet"]:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
iface = ipaddress.ip_interface(n["subnet"])
|
||||||
|
except ValueError:
|
||||||
|
continue
|
||||||
|
subnets.append((iface.network, n))
|
||||||
|
|
||||||
|
def resolve_net(ip: str | None, network_id: str | None) -> dict:
|
||||||
|
"""Which network does this reservation belong to?
|
||||||
|
|
||||||
|
Most reservations here (23 of 31 as of the first export) carry no
|
||||||
|
network_id at all — UniFi simply does not bind them. VyOS needs the
|
||||||
|
subnet to place a static-mapping, so fall back to containment.
|
||||||
|
"""
|
||||||
|
if network_id and network_id in nets_by_id:
|
||||||
|
nid = nets_by_id[network_id]
|
||||||
|
return {"name": nid.get("name"), "vlan": nid.get("vlan"), "by": "network_id"}
|
||||||
|
if ip:
|
||||||
|
try:
|
||||||
|
addr = ipaddress.ip_address(ip)
|
||||||
|
except ValueError:
|
||||||
|
return {"name": None, "vlan": None, "by": "unresolved"}
|
||||||
|
for net, meta in subnets:
|
||||||
|
if addr in net:
|
||||||
|
return {"name": meta["name"], "vlan": meta["vlan"], "by": "subnet"}
|
||||||
|
return {"name": None, "vlan": None, "by": "unresolved"}
|
||||||
|
|
||||||
|
# Fixed reservations: the single most important thing to carry over, and
|
||||||
|
# the easiest to lose — nobody has these written down anywhere else.
|
||||||
|
reservations = []
|
||||||
|
for u in raw.get("rest/user", []):
|
||||||
|
if not isinstance(u, dict) or not u.get("use_fixedip"):
|
||||||
|
continue
|
||||||
|
net = resolve_net(u.get("fixed_ip"), u.get("network_id"))
|
||||||
|
reservations.append({
|
||||||
|
"mac": (u.get("mac") or "").lower(),
|
||||||
|
"ip": u.get("fixed_ip"),
|
||||||
|
"name": u.get("name") or u.get("hostname") or "",
|
||||||
|
"hostname": u.get("hostname") or "",
|
||||||
|
"network_id": u.get("network_id"),
|
||||||
|
"network_name": net["name"],
|
||||||
|
"network_vlan": net["vlan"],
|
||||||
|
"resolved_by": net["by"],
|
||||||
|
"note": (u.get("note") or "").strip(),
|
||||||
|
})
|
||||||
|
reservations.sort(key=lambda r: tuple(int(p) for p in r["ip"].split(".")) if r["ip"] else (0,))
|
||||||
|
|
||||||
|
# Active leases without a reservation: these devices work today by luck of
|
||||||
|
# the lease database. After a DHCP server swap they get a NEW address.
|
||||||
|
reserved_macs = {r["mac"] for r in reservations}
|
||||||
|
dynamic = []
|
||||||
|
for c in raw.get("stat/sta", []):
|
||||||
|
if not isinstance(c, dict):
|
||||||
|
continue
|
||||||
|
mac = (c.get("mac") or "").lower()
|
||||||
|
if mac in reserved_macs or not c.get("ip"):
|
||||||
|
continue
|
||||||
|
dynamic.append({
|
||||||
|
"mac": mac,
|
||||||
|
"ip": c.get("ip"),
|
||||||
|
"name": c.get("name") or c.get("hostname") or "",
|
||||||
|
"network": c.get("network"),
|
||||||
|
})
|
||||||
|
dynamic.sort(key=lambda r: tuple(int(p) for p in r["ip"].split(".")) if r["ip"] else (0,))
|
||||||
|
|
||||||
|
port_forwards = [{
|
||||||
|
"name": p.get("name"), "enabled": p.get("enabled"),
|
||||||
|
"proto": p.get("proto"), "src": p.get("src"),
|
||||||
|
"dst_port": p.get("dst_port"), "fwd": p.get("fwd"),
|
||||||
|
"fwd_port": p.get("fwd_port"), "log": p.get("log"),
|
||||||
|
} for p in raw.get("rest/portforward", []) if isinstance(p, dict)]
|
||||||
|
|
||||||
|
firewall_rules = [{
|
||||||
|
"name": r.get("name"), "enabled": r.get("enabled"), "action": r.get("action"),
|
||||||
|
"ruleset": r.get("ruleset"), "rule_index": r.get("rule_index"),
|
||||||
|
"protocol": r.get("protocol"),
|
||||||
|
"src_address": r.get("src_address"), "dst_address": r.get("dst_address"),
|
||||||
|
"src_firewallgroup_ids": r.get("src_firewallgroup_ids"),
|
||||||
|
"dst_firewallgroup_ids": r.get("dst_firewallgroup_ids"),
|
||||||
|
"src_networkconf_id": r.get("src_networkconf_id"),
|
||||||
|
"dst_networkconf_id": r.get("dst_networkconf_id"),
|
||||||
|
} for r in raw.get("rest/firewallrule", []) if isinstance(r, dict)]
|
||||||
|
|
||||||
|
firewall_groups = [{
|
||||||
|
"id": g.get("_id"), "name": g.get("name"),
|
||||||
|
"type": g.get("group_type"), "members": g.get("group_members"),
|
||||||
|
} for g in raw.get("rest/firewallgroup", []) if isinstance(g, dict)]
|
||||||
|
|
||||||
|
static_routes = [{
|
||||||
|
"name": r.get("name"), "enabled": r.get("enabled"),
|
||||||
|
"network": r.get("static-route_network"),
|
||||||
|
"nexthop": r.get("static-route_nexthop"),
|
||||||
|
"distance": r.get("static-route_distance"),
|
||||||
|
"type": r.get("static-route_type"),
|
||||||
|
} for r in raw.get("rest/routing", []) if isinstance(r, dict)]
|
||||||
|
|
||||||
|
return {
|
||||||
|
"networks": networks,
|
||||||
|
"reservations": reservations,
|
||||||
|
"dynamic_clients": dynamic,
|
||||||
|
"port_forwards": port_forwards,
|
||||||
|
"firewall_rules": firewall_rules,
|
||||||
|
"firewall_groups": firewall_groups,
|
||||||
|
"static_routes": static_routes,
|
||||||
|
"warnings": find_warnings(networks, reservations),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def find_warnings(networks: list, reservations: list) -> list:
|
||||||
|
"""Things that are fine under UniFi but bite when rebuilt on VyOS."""
|
||||||
|
warns = []
|
||||||
|
|
||||||
|
for n in networks:
|
||||||
|
if not n["subnet"]:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
iface = ipaddress.ip_interface(n["subnet"])
|
||||||
|
except ValueError:
|
||||||
|
warns.append({"kind": "bad_subnet", "network": n["name"], "detail": n["subnet"]})
|
||||||
|
continue
|
||||||
|
# UniFi stores the gateway in ip_subnet. A gateway equal to the network
|
||||||
|
# address is legal in a /23 but plenty of tooling rejects it, so it must
|
||||||
|
# not be discovered during the cutover window.
|
||||||
|
if iface.ip == iface.network.network_address:
|
||||||
|
warns.append({
|
||||||
|
"kind": "gateway_is_network_address", "network": n["name"],
|
||||||
|
"detail": f"gateway {iface.ip} is the network address of {iface.network}",
|
||||||
|
})
|
||||||
|
|
||||||
|
# Reservations that sit inside the dynamic pool. UniFi's dhcpd tolerates
|
||||||
|
# this; whether VyOS does depends on its DHCP backend, so every one of these
|
||||||
|
# is a config that must be proven on the sim before cutover.
|
||||||
|
ranges = []
|
||||||
|
for n in networks:
|
||||||
|
if n["dhcp_enabled"] and n["dhcp_start"] and n["dhcp_stop"]:
|
||||||
|
try:
|
||||||
|
ranges.append((n["name"], ipaddress.ip_address(n["dhcp_start"]),
|
||||||
|
ipaddress.ip_address(n["dhcp_stop"])))
|
||||||
|
except ValueError:
|
||||||
|
pass
|
||||||
|
inside = []
|
||||||
|
for r in reservations:
|
||||||
|
if not r["ip"]:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
addr = ipaddress.ip_address(r["ip"])
|
||||||
|
except ValueError:
|
||||||
|
continue
|
||||||
|
for name, lo, hi in ranges:
|
||||||
|
if lo <= addr <= hi:
|
||||||
|
inside.append(f"{r['ip']} ({r['name'] or r['mac']}) in {name} pool")
|
||||||
|
break
|
||||||
|
if inside:
|
||||||
|
warns.append({"kind": "reservation_inside_dhcp_pool",
|
||||||
|
"count": len(inside), "detail": inside})
|
||||||
|
|
||||||
|
unresolved = [f"{r['ip']} {r['mac']} {r['name']}"
|
||||||
|
for r in reservations if r["resolved_by"] == "unresolved"]
|
||||||
|
if unresolved:
|
||||||
|
warns.append({"kind": "reservation_matches_no_subnet",
|
||||||
|
"count": len(unresolved), "detail": unresolved})
|
||||||
|
return warns
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
ap = argparse.ArgumentParser()
|
||||||
|
ap.add_argument("--out", default=os.path.join(os.path.dirname(os.path.abspath(__file__)), "export"))
|
||||||
|
ap.add_argument("--summary", action="store_true")
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
os.makedirs(args.out, exist_ok=True)
|
||||||
|
opener, base, site = _unifi.client()
|
||||||
|
print(f"controller {base} site {site}")
|
||||||
|
|
||||||
|
raw: dict = {}
|
||||||
|
errors = []
|
||||||
|
for path, why in ENDPOINTS.items():
|
||||||
|
data = _unifi.get(opener, base, site, path)
|
||||||
|
if isinstance(data, dict) and "__error__" in data:
|
||||||
|
errors.append((path, data["__error__"]))
|
||||||
|
print(f" {path:22} FAILED {data['__error__'][:50]}")
|
||||||
|
continue
|
||||||
|
raw[path] = data
|
||||||
|
fname = path.replace("/", "_") + ".json"
|
||||||
|
with open(os.path.join(args.out, fname), "w") as fh:
|
||||||
|
json.dump(data, fh, indent=2, sort_keys=True)
|
||||||
|
print(f" {path:22} {len(data):4d} -> {fname}")
|
||||||
|
|
||||||
|
inventory = build_inventory(raw)
|
||||||
|
with open(os.path.join(args.out, "inventory.json"), "w") as fh:
|
||||||
|
json.dump(inventory, fh, indent=2, sort_keys=True)
|
||||||
|
|
||||||
|
print(f"\nwrote {args.out}/inventory.json")
|
||||||
|
print(f" networks {len(inventory['networks'])}")
|
||||||
|
print(f" DHCP reservations {len(inventory['reservations'])}")
|
||||||
|
print(f" dynamic clients {len(inventory['dynamic_clients'])} (no reservation — see summary)")
|
||||||
|
print(f" port forwards {len(inventory['port_forwards'])}")
|
||||||
|
print(f" firewall rules {len(inventory['firewall_rules'])}")
|
||||||
|
print(f" static routes {len(inventory['static_routes'])}")
|
||||||
|
if errors:
|
||||||
|
print(f" endpoints failed {len(errors)}: {[e[0] for e in errors]}")
|
||||||
|
|
||||||
|
if args.summary:
|
||||||
|
print_summary(inventory)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def print_summary(inv: dict) -> None:
|
||||||
|
print("\n=== networks ===")
|
||||||
|
print(f"{'name':22} {'vlan':>5} {'subnet':20} {'dhcp range':32} lease")
|
||||||
|
for n in sorted(inv["networks"], key=lambda x: (x["vlan"] or 0)):
|
||||||
|
rng = f"{n['dhcp_start']} - {n['dhcp_stop']}" if n["dhcp_enabled"] else "(dhcp off)"
|
||||||
|
print(f"{(n['name'] or '')[:22]:22} {str(n['vlan'] or '-'):>5} "
|
||||||
|
f"{(n['subnet'] or '-'):20} {rng:32} {n['dhcp_lease'] or '-'}")
|
||||||
|
|
||||||
|
print(f"\n=== DHCP reservations ({len(inv['reservations'])}) ===")
|
||||||
|
for r in inv["reservations"]:
|
||||||
|
print(f" {r['ip']:16} {r['mac']:18} {(r['network_name'] or '?')[:14]:14} {r['name'][:30]}")
|
||||||
|
|
||||||
|
if inv["port_forwards"]:
|
||||||
|
print(f"\n=== port forwards ({len(inv['port_forwards'])}) ===")
|
||||||
|
for p in inv["port_forwards"]:
|
||||||
|
state = "" if p["enabled"] else " [DISABLED]"
|
||||||
|
print(f" {p['proto']:6} {str(p['src']):16}:{str(p['dst_port']):11} -> "
|
||||||
|
f"{p['fwd']}:{p['fwd_port']} {p['name']}{state}")
|
||||||
|
|
||||||
|
if inv["firewall_rules"]:
|
||||||
|
print(f"\n=== firewall rules ({len(inv['firewall_rules'])}) ===")
|
||||||
|
for r in inv["firewall_rules"]:
|
||||||
|
state = "" if r["enabled"] else " [DISABLED]"
|
||||||
|
print(f" {str(r['ruleset']):22} {str(r['action']):8} {r['name']}{state}")
|
||||||
|
|
||||||
|
if inv["warnings"]:
|
||||||
|
print(f"\n=== {len(inv['warnings'])} things to settle before cutover ===")
|
||||||
|
for w in inv["warnings"]:
|
||||||
|
n = w.get("count")
|
||||||
|
print(f" [{w['kind']}]" + (f" x{n}" if n else ""))
|
||||||
|
det = w["detail"]
|
||||||
|
for line in (det if isinstance(det, list) else [det])[:6]:
|
||||||
|
print(f" {line}")
|
||||||
|
if isinstance(det, list) and len(det) > 6:
|
||||||
|
print(f" ... and {len(det) - 6} more (see inventory.json)")
|
||||||
|
|
||||||
|
n_dyn = len(inv["dynamic_clients"])
|
||||||
|
if n_dyn:
|
||||||
|
print(f"\n=== {n_dyn} active clients WITHOUT a reservation ===")
|
||||||
|
print(" These hold their address only via the current lease database. A DHCP")
|
||||||
|
print(" server swap hands them a different one — fine for phones, not fine for")
|
||||||
|
print(" anything another host reaches by IP. Review before cutover:")
|
||||||
|
for c in inv["dynamic_clients"][:40]:
|
||||||
|
print(f" {c['ip']:16} {c['mac']:18} {(c['network'] or '')[:14]:14} {c['name'][:30]}")
|
||||||
|
if n_dyn > 40:
|
||||||
|
print(f" ... and {n_dyn - 40} more (see inventory.json)")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
253
migration/unifi-to-vyos.py
Executable file
253
migration/unifi-to-vyos.py
Executable file
@@ -0,0 +1,253 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Turn the UniFi export into the VyOS config that replaces it.
|
||||||
|
|
||||||
|
Scope is deliberately narrow: DHCP and DNS. Those are the services the USG owns
|
||||||
|
that VyOS must reproduce byte-for-byte in behaviour, because getting them wrong
|
||||||
|
means clients lose their addresses or their name resolution. Everything else
|
||||||
|
either stays on UniFi (wireless), has no VyOS equivalent (user groups), or is
|
||||||
|
hand-written because it is not in the export (WAN, NAT, VRRP).
|
||||||
|
|
||||||
|
This is not a general UniFi-to-VyOS converter and should not grow into one.
|
||||||
|
|
||||||
|
./unifi-to-vyos.py --mode prod # the cutover artifact
|
||||||
|
./unifi-to-vyos.py --mode sim # same MACs, labsim addresses
|
||||||
|
./unifi-to-vyos.py --mode prod --check # counts only, no output
|
||||||
|
|
||||||
|
Both modes come from one code path on purpose: the config proven in labsim and
|
||||||
|
the config applied to the firewalls must not be able to drift apart.
|
||||||
|
|
||||||
|
DNS note: UniFi hands out the gateway's own IP as resolver whenever a network
|
||||||
|
has no explicit dhcpd_dns -- true for 5 of the 6 VLANs, verified by labmaster
|
||||||
|
resolving against 192.168.8.1. So VyOS must run `service dns forwarding` or
|
||||||
|
those VLANs lose DNS entirely at cutover. LoT's explicit 10.0.0.194 is preserved
|
||||||
|
as-is.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import ipaddress
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
|
||||||
|
# Where the production upstream resolver lives. Verified authoritative for
|
||||||
|
# ad.itaz.eu (SOA nas001.ad.itaz.eu) and also recursive, so it can serve as the
|
||||||
|
# single forwarder target.
|
||||||
|
UPSTREAM_DNS = "10.0.0.194"
|
||||||
|
|
||||||
|
# labsim equivalents, keyed by VLAN id. Only VLAN 10 needs a /23: every one of
|
||||||
|
# the 31 reservations is in LoT, which spans 10.0.0.x and 10.0.1.x, and a /24
|
||||||
|
# cannot represent that. k8s and Private are also /23 in production but hold no
|
||||||
|
# reservations, so they keep their existing /24 and their DHCP range is clamped
|
||||||
|
# (reported at generation time -- never silently).
|
||||||
|
SIM_SUBNETS = {
|
||||||
|
1: "172.31.1.0/24",
|
||||||
|
2: "172.31.2.0/24",
|
||||||
|
3: "172.31.3.0/24",
|
||||||
|
9: "172.31.9.0/24",
|
||||||
|
10: "172.31.10.0/23",
|
||||||
|
200: "172.31.200.0/24",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def vlan_of(net: dict) -> int:
|
||||||
|
"""VLAN id, treating the untagged Management network as 1.
|
||||||
|
|
||||||
|
UniFi stores vlan=None for the native network; the VyOS side already uses
|
||||||
|
vrid 1 for it (high-availability group 'native'), so 1 is the consistent id.
|
||||||
|
"""
|
||||||
|
return int(net["vlan"]) if net.get("vlan") else 1
|
||||||
|
|
||||||
|
|
||||||
|
def sanitize(name: str, fallback: str) -> str:
|
||||||
|
"""Reduce a UniFi client name to something VyOS will accept as a node name.
|
||||||
|
|
||||||
|
VyOS validates static-mapping names as *hostnames*, so underscores are
|
||||||
|
rejected outright -- verified: `Dongle-M_C0D4` fails with "Invalid static
|
||||||
|
mapping hostname". Letters, digits and hyphens only, no leading digit or
|
||||||
|
hyphen, no trailing hyphen.
|
||||||
|
"""
|
||||||
|
cleaned = re.sub(r"[^A-Za-z0-9-]", "-", (name or "").strip())
|
||||||
|
cleaned = re.sub(r"-{2,}", "-", cleaned).strip("-")
|
||||||
|
if cleaned and cleaned[0].isdigit():
|
||||||
|
cleaned = "h" + cleaned
|
||||||
|
return cleaned or fallback
|
||||||
|
|
||||||
|
|
||||||
|
class Mapper:
|
||||||
|
"""Translates production addresses into the target mode's address space.
|
||||||
|
|
||||||
|
In prod mode this is the identity. In sim mode an address is mapped by its
|
||||||
|
offset from the network address, so the host part is preserved: 10.0.0.46
|
||||||
|
-> 172.31.10.46 and 10.0.1.67 -> 172.31.11.67. That is what makes the sim
|
||||||
|
test meaningful -- the MAC is identical and the host octet is recognisable.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, mode: str, networks: list) -> None:
|
||||||
|
self.mode = mode
|
||||||
|
self.clamped: list[str] = []
|
||||||
|
self.map: dict[int, tuple] = {}
|
||||||
|
for n in networks:
|
||||||
|
prod = ipaddress.ip_network(
|
||||||
|
ipaddress.ip_interface(n["subnet"]).network)
|
||||||
|
if mode == "sim":
|
||||||
|
sim = ipaddress.ip_network(SIM_SUBNETS[vlan_of(n)])
|
||||||
|
else:
|
||||||
|
sim = prod
|
||||||
|
self.map[vlan_of(n)] = (prod, sim)
|
||||||
|
|
||||||
|
def net(self, vlan: int) -> ipaddress.IPv4Network:
|
||||||
|
return self.map[vlan][1]
|
||||||
|
|
||||||
|
def addr(self, vlan: int, ip: str, what: str) -> str | None:
|
||||||
|
"""Map one address, or None if it does not fit the target subnet."""
|
||||||
|
prod, sim = self.map[vlan]
|
||||||
|
offset = int(ipaddress.ip_address(ip)) - int(prod.network_address)
|
||||||
|
if offset < 0 or offset >= sim.num_addresses:
|
||||||
|
self.clamped.append(f"{what}: {ip} does not fit {sim}")
|
||||||
|
return None
|
||||||
|
return str(ipaddress.ip_address(int(sim.network_address) + offset))
|
||||||
|
|
||||||
|
def gateway(self, vlan: int, net: dict) -> str:
|
||||||
|
"""The address clients are told to use as their default route.
|
||||||
|
|
||||||
|
Production: the USG's current address (VIPs move .254 -> .1 at cutover),
|
||||||
|
so no client has to change anything. Sim: the sim router at .1.
|
||||||
|
"""
|
||||||
|
if self.mode == "sim":
|
||||||
|
return str(self.net(vlan).network_address + 1)
|
||||||
|
return str(ipaddress.ip_interface(net["subnet"]).ip)
|
||||||
|
|
||||||
|
|
||||||
|
def build(inv: dict, mode: str) -> tuple[list[str], dict]:
|
||||||
|
nets = [n for n in inv["networks"] if n["dhcp_enabled"] and n["subnet"]]
|
||||||
|
nets.sort(key=vlan_of)
|
||||||
|
m = Mapper(mode, nets)
|
||||||
|
|
||||||
|
out: list[str] = []
|
||||||
|
used_tags: set[str] = set()
|
||||||
|
stats = {"subnets": 0, "mappings": 0, "dropped": []}
|
||||||
|
by_vlan: dict[int, list] = {}
|
||||||
|
for r in inv["reservations"]:
|
||||||
|
if r["network_vlan"] is None and r["network_name"] != "Management":
|
||||||
|
# resolved_by == "unresolved"; cannot place it without a subnet
|
||||||
|
stats["dropped"].append(f"{r['ip']} {r['mac']} (no network)")
|
||||||
|
continue
|
||||||
|
by_vlan.setdefault(r["network_vlan"] or 1, []).append(r)
|
||||||
|
|
||||||
|
out.append("# --- DHCP ---------------------------------------------------")
|
||||||
|
for n in nets:
|
||||||
|
vlan = vlan_of(n)
|
||||||
|
sub = m.net(vlan)
|
||||||
|
base = f"set service dhcp-server shared-network-name {sanitize(n['name'], f'vlan{vlan}')} subnet {sub}"
|
||||||
|
gw = m.gateway(vlan, n)
|
||||||
|
|
||||||
|
out.append("")
|
||||||
|
out.append(f"# {n['name']} (VLAN {vlan}) <- {n['subnet']}")
|
||||||
|
# subnet-id is required by kea and must be stable across regenerations;
|
||||||
|
# the VLAN id is already the unique per-network number in this lab.
|
||||||
|
out.append(f"{base} subnet-id {vlan}")
|
||||||
|
out.append(f"{base} option default-router {gw}")
|
||||||
|
|
||||||
|
# Preserve UniFi's explicit resolver where it set one (LoT -> the NAS);
|
||||||
|
# otherwise hand out the gateway, which is what the USG does today and
|
||||||
|
# what `service dns forwarding` below will answer on.
|
||||||
|
for ns in (n["dhcp_dns"] or [gw]):
|
||||||
|
mapped = ns if ns not in (UPSTREAM_DNS,) or mode == "prod" else gw
|
||||||
|
if mode == "sim" and ns == UPSTREAM_DNS:
|
||||||
|
mapped = gw # no NAS in the sim; the router resolves
|
||||||
|
out.append(f"{base} option name-server {mapped}")
|
||||||
|
|
||||||
|
if n["domain_name"]:
|
||||||
|
out.append(f"{base} option domain-name '{n['domain_name']}'")
|
||||||
|
if n["dhcp_lease"]:
|
||||||
|
out.append(f"{base} lease {n['dhcp_lease']}")
|
||||||
|
|
||||||
|
start = m.addr(vlan, n["dhcp_start"], f"{n['name']} range start")
|
||||||
|
stop = m.addr(vlan, n["dhcp_stop"], f"{n['name']} range stop")
|
||||||
|
if start is None:
|
||||||
|
start = str(sub.network_address + 11)
|
||||||
|
if stop is None:
|
||||||
|
# Clamp to the last usable address rather than dropping the pool.
|
||||||
|
stop = str(sub.broadcast_address - 1)
|
||||||
|
out.append(f"{base} range LAN start {start}")
|
||||||
|
out.append(f"{base} range LAN stop {stop}")
|
||||||
|
stats["subnets"] += 1
|
||||||
|
|
||||||
|
for r in sorted(by_vlan.get(vlan, []), key=lambda x: ipaddress.ip_address(x["ip"])):
|
||||||
|
ip = m.addr(vlan, r["ip"], f"reservation {r['name']}")
|
||||||
|
if ip is None:
|
||||||
|
stats["dropped"].append(f"{r['ip']} {r['mac']} ({r['name']})")
|
||||||
|
continue
|
||||||
|
tag = sanitize(r["name"] or r["hostname"], "host-" + r["mac"].replace(":", ""))
|
||||||
|
# Distinct clients can sanitize to the same name; a collision would
|
||||||
|
# silently overwrite one reservation with another's address.
|
||||||
|
if tag in used_tags:
|
||||||
|
tag = f"{tag}-{r['mac'].replace(':', '')[-4:]}"
|
||||||
|
used_tags.add(tag)
|
||||||
|
out.append(f"{base} static-mapping {tag} mac {r['mac']}")
|
||||||
|
out.append(f"{base} static-mapping {tag} ip-address {ip}")
|
||||||
|
stats["mappings"] += 1
|
||||||
|
|
||||||
|
out.append("")
|
||||||
|
out.append("# --- DNS ----------------------------------------------------")
|
||||||
|
out.append("# The USG resolves for 5 of 6 VLANs today (it hands out its own")
|
||||||
|
out.append("# address when dhcpd_dns is empty). Without this, they lose DNS.")
|
||||||
|
for n in nets:
|
||||||
|
vlan = vlan_of(n)
|
||||||
|
out.append(f"set service dns forwarding listen-address {m.gateway(vlan, n)}")
|
||||||
|
out.append(f"set service dns forwarding allow-from {m.net(vlan)}")
|
||||||
|
if mode == "prod":
|
||||||
|
out.append(f"set service dns forwarding name-server {UPSTREAM_DNS}")
|
||||||
|
else:
|
||||||
|
# The sim has no NAS; forward to whatever the sim host can reach.
|
||||||
|
out.append("set service dns forwarding name-server 1.1.1.1")
|
||||||
|
out.append("set service dns forwarding cache-size 10000")
|
||||||
|
|
||||||
|
stats["clamped"] = m.clamped
|
||||||
|
return out, stats
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
ap = argparse.ArgumentParser()
|
||||||
|
here = os.path.dirname(os.path.abspath(__file__))
|
||||||
|
ap.add_argument("--mode", choices=("prod", "sim"), required=True)
|
||||||
|
ap.add_argument("--inventory", default=os.path.join(here, "export", "inventory.json"))
|
||||||
|
ap.add_argument("-o", "--out")
|
||||||
|
ap.add_argument("--check", action="store_true", help="counts only, no config")
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
with open(args.inventory) as fh:
|
||||||
|
inv = json.load(fh)
|
||||||
|
|
||||||
|
lines, stats = build(inv, args.mode)
|
||||||
|
|
||||||
|
expected = len(inv["reservations"])
|
||||||
|
print(f"mode={args.mode} subnets={stats['subnets']} "
|
||||||
|
f"static-mappings={stats['mappings']}/{expected}", file=sys.stderr)
|
||||||
|
for c in stats["clamped"]:
|
||||||
|
print(f" clamped: {c}", file=sys.stderr)
|
||||||
|
for d in stats["dropped"]:
|
||||||
|
print(f" DROPPED: {d}", file=sys.stderr)
|
||||||
|
|
||||||
|
if stats["mappings"] != expected and args.mode == "prod":
|
||||||
|
print(f"ERROR: {expected - stats['mappings']} reservation(s) missing from "
|
||||||
|
f"prod output -- every one must survive the cutover", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
|
||||||
|
if args.check:
|
||||||
|
return 0
|
||||||
|
|
||||||
|
text = "\n".join(lines) + "\n"
|
||||||
|
if args.out:
|
||||||
|
with open(args.out, "w") as fh:
|
||||||
|
fh.write(text)
|
||||||
|
print(f"wrote {args.out}", file=sys.stderr)
|
||||||
|
else:
|
||||||
|
sys.stdout.write(text)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
Reference in New Issue
Block a user