michal/lab

Files

Michal 41b5448f56 feat(pulumi-vyos): prototype VyOS subtrees as Pulumi resources with commit-confirm

Goal: change VyOS and Kubernetes in one codebase and one plan -- so a BGP change
touches both sides in a single `pulumi preview`.

First, the worry about per-command pushes turned out to be unfounded for the
community providers. Read foltik/vyos and its client library: a
`vyos_config_block_tree` flattens the whole subtree into a single payload array
and sends ONE POST to /configure, so one resource is one commit. Good.

What they do not do is send `confirm_time`. Their payload is only
op/path/value, so every change is an unprotected commit -- on a router you reach
through the router, that is the difference between a mistake and an outage. The
VyOS API itself supports commit-confirm; the providers simply do not use it.

So this is a ~180-line Pulumi dynamic provider that does. Verified end to end on
labsim: create and update each land in ~6s as one commit-confirmed transaction,
update reports [diff: ~commands], destroy removes the subtree, and an
unconfirmed commit was observed reverting the router on its own.

Three API details found the hard way, all now encoded and commented:

  - confirm_time is ONLY read when the body parses as ConfigureListModel, i.e.
    {"commands": [...], "confirm_time": N}. A bare array is accepted and
    committed with NO timer armed, and the response looks like success. This
    silently discards the entire safety net, so the resource now checks the
    response actually says "commit-confirm" and refuses to proceed otherwise.
  - There is no /confirm endpoint; confirm is an op on /configure.
  - Confirm requires a `path` field even though it ignores it -- the Union
    resolves to ConfigureModel, which mandates path. Without it: "missing 'path'
    field", and the timer keeps running.

Apply is `delete <path>` followed by the sets, in one request, so the result is
the declared state rather than a merge -- otherwise `pulumi up` accumulates
instead of converging.

Known gaps, in the README rather than hidden: no read/refresh so out-of-band
drift is not detected, and the API runs with a self-signed certificate and
verification disabled. Both need addressing before production. The cutover
itself should still use vyos-unifi-switch, which the API cannot replace.

Sim left as found: test resource destroyed, dns forwarding restored to 15 lines.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DMVzWZgiKW2wquf5z8S1yH

2026-08-17 01:09:05 +01:00

3.0 KiB

Raw Blame History

VyOS as Pulumi resources — prototype

Proves that VyOS config can be managed from the same Pulumi plan as the Kubernetes side, without giving up the safety property that matters on a gateway: a config push that breaks your access undoes itself.

Why not the community Terraform providers

They are better than expected. foltik/vyos's vyos_config_block_tree flattens an entire subtree into one payload and sends one POST to /configure — so one resource is one commit, not one commit per config line. That worry was unfounded.

What they do not do is send confirm_time. The payload is only op/path/value, so every change is an unprotected commit. On a router you reach through the router, that is the difference between a mistake and an outage.

What the VyOS API actually supports

Read from rest/models.py and rest/routers.py on the box, then verified by hand against a live router:

Batching: a list of operations in one request, applied as one commit.
commit-confirm: confirm_time on the request starts the revert timer. Response says Initialized commit-confirm; N minutes to confirm before reload.

Three details that cost time and are easy to get wrong:

confirm_time is only read when the body parses as ConfigureListModel — i.e. {"commands": [...], "confirm_time": N}. A bare array is accepted and committed happily with no timer armed. It looks identical to success. The resource therefore checks the response actually mentions commit-confirm and refuses to continue if it does not.
There is no /confirm endpoint. Confirming is an op on /configure.
Confirm still requires a path field, even though it ignores it — the Union resolves to ConfigureModel, which mandates path. Without it you get missing 'path' field and the timer keeps running.

Shape

One resource per subtree, not per line:

new VyosConfigTree("dns-forwarding", {
    host, apiKey,
    path: ["service", "dns", "forwarding"],
    commands: [["cache-size", "20000"], ["name-server", "8.8.8.8"]],
    confirmMinutes: 2,
});

Apply is delete <path> + all the sets in one request, so the result is the declared state rather than a merge with whatever was there — which is what makes pulumi up converge instead of accumulate.

Verified on labsim

pulumi up create and update both land in ~6s, each a single commit-confirmed transaction; update shows [diff: ~commands].
Auto-revert observed: an unconfirmed commit returned the router to its saved config on its own.
pulumi destroy removes the subtree.

Not done

The API is HTTP with a self-signed certificate and rejectUnauthorized: false. Bind it to the management VLAN or the peer link and install a real certificate before this goes near production.
No refresh/drift detection yet: read is not implemented, so out-of-band changes are not noticed until the next up overwrites them.
The cutover itself should still use vyos-unifi-switch. This is for day-2.

3.0 KiB Raw Blame History