lab/migration/MANAGEMENT-VLAN-TAGGED.md

# Moving Management onto a tagged VLAN

Rehearsed end to end in labsim on 2026-09-02. This is the fix for kea serving
addresses from the wrong VLAN's pool.

## Why

ISC Kea [#1117](https://gitlab.isc.org/isc-projects/kea/-/issues/1117): with
`dhcp-socket-type: raw`, a frame tagged for a sub-interface is **also** delivered
to the parent's `AF_PACKET` socket. If the parent serves a subnet, kea answers
from it too. Ours does — Management is the native/untagged VLAN on `bond0` while
VLANs 2/3/9/10/200 are sub-interfaces of that same bond — so one DISCOVER on
VLAN 3 produces two OFFERs and the *client* decides which to keep:

```
bond0.3 :  192.168.3.14  correct
bond0   :  192.168.1.28  UNTAGGED, Management pool, wrong
```

The fix is to leave **no subnet on the parent**: every VLAN tagged, Management
included, moved from `bond0` to `bond0.1`.

Confirmed in labsim across all six LAN VLANs: fails before, passes after.
`labsim/labsim-vlan-leak-test.sh` is the test; evidence in
`labsim/vlan-leak-evidence/`.

## What must change together

Per router:

| | from | to |
|---|---|---|
| address | `interfaces bonding bond0 address` | `interfaces bonding bond0 vif 1 address` |
| firewall | `interface-group LAN interface bond0` | `... interface bond0.1` |
| VRRP | `vrrp group native interface bond0` | `... interface bond0.1` |
| kea | — | **restart it** (see traps) |

On the switch: Native VLAN = **None** on the trunk to that firewall, with VLAN 1
added to the tagged set.

## The ordering constraint

**There is no overlap state.** An 802.1Q port always egresses its native VLAN
untagged, so while VLAN 1 is native the router can *send* tagged VLAN 1 but can
never *receive* it. Verified: a tagged VLAN 1 ARP sent from the switch arrived on
`bond0` untagged and never on `bond0.1`. Configuring "native VLAN 1 **and** VLAN 1
tagged" as a make-before-break does not work; the switch and router changes for a
given firewall are strictly simultaneous, and that router loses Management in
between.

What makes this safe anyway: **tagged and untagged Management coexist on the
same VLAN.** One VLAN is one broadcast domain no matter how each port tags it, so
the firewalls can be converted one at a time — verified with the primary untagged
and the secondary already tagged, both reachable, VIP up, VLAN 1 clients fine.

Access ports are untouched throughout. The UniFi controller at 192.168.1.5 and
your workstation are on access ports and never traverse the firewall trunks, so
you keep the controller you are making the change from. Only the router being
converted goes dark, and only until its own config lands.

## Procedure

Do the **backup** router first, then fail the VIPs over and do the other. You
need console (JetKVM) on the router being converted — its Management SSH dies the
moment the switch port changes.

For each router in turn:

1. Confirm the *other* router is MASTER and healthy:
   `show vrrp` and `sudo /config/vrrp-wan-health; echo $?` (must be 0).
2. Start the monitor from a workstation on an access port:
   `labsim/vlan1-move-monitor.sh` (edit the three addresses for production).
3. UniFi: on this firewall's trunk ports, Native VLAN → None, VLAN 1 → tagged.
   This router's Management drops now.
4. Over the console, in one commit:
   ```
   set interfaces bonding bond0 vif 1 address '192.168.1.252/24'   # .253 on vyos002
   set interfaces bonding bond0 vif 1 description 'management'
   delete interfaces bonding bond0 address
   set firewall group interface-group LAN interface 'bond0.1'
   delete firewall group interface-group LAN interface 'bond0'
   set high-availability vrrp group native interface 'bond0.1'
   commit
   save
   ```
5. `sudo systemctl restart isc-kea-dhcp4-server` — see traps.
6. Verify: Management SSH back, `show vrrp` shows `native` on `bond0.1`, and the
   leak test passes.

Then fail back if the VIPs moved (below), and repeat for the other router.

### Measured windows (labsim)

| | |
|---|---|
| this router's own Management unreachable | ~27 s (the console apply) |
| VIP `.1` unreachable, peer already converted | **0 s** |
| VIP `.1` unreachable, converting the current MASTER | ~6 s (VRRP failover) |
| VIP unreachable if you convert both routers before the switch | **5 min 30 s** |

That last row is the failure mode to avoid: with both routers untagged and the
trunks already changed, the VIP is a black hole and **the healthy BACKUP does not
take over**. Its `native` group stays BACKUP because the *other* VLANs still hear
the master, and the sync group holds them together. Redundancy does not help you
here; only ordering does.

## Traps

- **Restart kea.** VyOS does not restart it for an interface address change, so
  it keeps a raw socket bound with the old address and keeps emitting the wrong
  offers. The first post-fix test in the sim failed for this reason alone and
  looked exactly like the fix not working.
- **`interface-group LAN`.** Moving the address without moving the group means
  Management falls outside the group, and with default-deny that is every
  management session and all VLAN 1 inter-VLAN routing, gone on commit — on a
  router you reach through itself. Use `commit-confirm` if you are not on console.
- **The VIPs may move, and `no-preempt` keeps them moved.** Converting a router
  restarts keepalived and re-initialises *every* group, not just `native`. In one
  rehearsal the priority-100 secondary took all six VIPs and held them while the
  priority-200 primary sat at BACKUP; in another the restart was quick enough that
  nothing moved. It is non-deterministic — check afterwards, every time.
  Fail back with `restart vrrp` **on the router currently holding them**.
- **Duplicate delivery does not stop**, and should not be read as failure. #1117
  only promises there is no longer a subnet on the parent to match. Expect two
  identical replies per DISCOVER, both from the correct pool.
- **Both firewalls' trunks must end up the same.** If UniFi shares one port
  profile between them, changing it converts both at once and you get the 5m30s
  row above. Check before you start; use per-port overrides if it does.

## Not covered by the rehearsal

- Whether UniFi's port profile can express "no native VLAN" the way OVS can, and
  whether the two firewalls share a profile. Unverified — check on the controller.
- Why the JetKVM consoles specifically accepted the wrong OFFER when a VLAN 3
  access port should not receive an untagged VLAN 1 frame at all. Their port
  profile likely passes VLAN 1 untagged. Worth confirming, though it does not
  change the fix.