feat(pulumi-vyos): prototype VyOS subtrees as Pulumi resources with commit-confirm

Goal: change VyOS and Kubernetes in one codebase and one plan -- so a BGP change
touches both sides in a single `pulumi preview`.

First, the worry about per-command pushes turned out to be unfounded for the
community providers. Read foltik/vyos and its client library: a
`vyos_config_block_tree` flattens the whole subtree into a single payload array
and sends ONE POST to /configure, so one resource is one commit. Good.

What they do not do is send `confirm_time`. Their payload is only
op/path/value, so every change is an unprotected commit -- on a router you reach
through the router, that is the difference between a mistake and an outage. The
VyOS API itself supports commit-confirm; the providers simply do not use it.

So this is a ~180-line Pulumi dynamic provider that does. Verified end to end on
labsim: create and update each land in ~6s as one commit-confirmed transaction,
update reports [diff: ~commands], destroy removes the subtree, and an
unconfirmed commit was observed reverting the router on its own.

Three API details found the hard way, all now encoded and commented:

  - confirm_time is ONLY read when the body parses as ConfigureListModel, i.e.
    {"commands": [...], "confirm_time": N}. A bare array is accepted and
    committed with NO timer armed, and the response looks like success. This
    silently discards the entire safety net, so the resource now checks the
    response actually says "commit-confirm" and refuses to proceed otherwise.
  - There is no /confirm endpoint; confirm is an op on /configure.
  - Confirm requires a `path` field even though it ignores it -- the Union
    resolves to ConfigureModel, which mandates path. Without it: "missing 'path'
    field", and the timer keeps running.

Apply is `delete <path>` followed by the sets, in one request, so the result is
the declared state rather than a merge -- otherwise `pulumi up` accumulates
instead of converging.

Known gaps, in the README rather than hidden: no read/refresh so out-of-band
drift is not detected, and the API runs with a self-signed certificate and
verification disabled. Both need addressing before production. The cutover
itself should still use vyos-unifi-switch, which the API cannot replace.

Sim left as found: test resource destroyed, dns forwarding restored to 15 lines.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DMVzWZgiKW2wquf5z8S1yH

pulumi-vyos/Pulumi.labsim.yaml

@@ -0,0 +1 @@
+encryptionsalt: v1:tMNG4q79HiI=:v1:K35iEOw3pgyukCr6:LmKO37/jghjyT2sLuoLmCQIDPOIsgA==

pulumi-vyos/Pulumi.yaml

@@ -0,0 +1,3 @@
+name: vyos-proto
+runtime: nodejs
+description: Prototype — VyOS config subtrees as Pulumi resources, with commit-confirm

pulumi-vyos/README.md

@@ -0,0 +1,72 @@
+# VyOS as Pulumi resources — prototype
+
+Proves that VyOS config can be managed from the same Pulumi plan as the
+Kubernetes side, **without** giving up the safety property that matters on a
+gateway: a config push that breaks your access undoes itself.
+
+## Why not the community Terraform providers
+
+They are better than expected. `foltik/vyos`'s `vyos_config_block_tree` flattens
+an entire subtree into one payload and sends **one POST to `/configure`** —
+so one resource is one commit, not one commit per config line. That worry was
+unfounded.
+
+What they do *not* do is send `confirm_time`. The payload is only
+`op`/`path`/`value`, so every change is an unprotected commit. On a router you
+reach *through* the router, that is the difference between a mistake and an
+outage.
+
+## What the VyOS API actually supports
+
+Read from `rest/models.py` and `rest/routers.py` on the box, then verified by
+hand against a live router:
+
+- **Batching**: a list of operations in one request, applied as one commit.
+- **commit-confirm**: `confirm_time` on the request starts the revert timer.
+  Response says `Initialized commit-confirm; N minutes to confirm before reload`.
+
+Three details that cost time and are easy to get wrong:
+
+1. **`confirm_time` is only read when the body parses as `ConfigureListModel`** —
+   i.e. `{"commands": [...], "confirm_time": N}`. A **bare array** is accepted
+   and committed happily with **no timer armed**. It looks identical to success.
+   The resource therefore checks the response actually mentions commit-confirm
+   and refuses to continue if it does not.
+2. **There is no `/confirm` endpoint.** Confirming is an op on `/configure`.
+3. **Confirm still requires a `path` field**, even though it ignores it — the
+   Union resolves to `ConfigureModel`, which mandates `path`. Without it you get
+   `missing 'path' field` and the timer keeps running.
+
+## Shape
+
+One resource per **subtree**, not per line:
+
+```ts
+new VyosConfigTree("dns-forwarding", {
+    host, apiKey,
+    path: ["service", "dns", "forwarding"],
+    commands: [["cache-size", "20000"], ["name-server", "8.8.8.8"]],
+    confirmMinutes: 2,
+});
+```
+
+Apply is `delete <path>` + all the `set`s in one request, so the result is the
+declared state rather than a merge with whatever was there — which is what makes
+`pulumi up` converge instead of accumulate.
+
+## Verified on labsim
+
+- `pulumi up` create and update both land in ~6s, each a single
+  commit-confirmed transaction; update shows `[diff: ~commands]`.
+- Auto-revert observed: an unconfirmed commit returned the router to its saved
+  config on its own.
+- `pulumi destroy` removes the subtree.
+
+## Not done
+
+- The API is HTTP with a self-signed certificate and `rejectUnauthorized: false`.
+  Bind it to the management VLAN or the peer link and install a real
+  certificate before this goes near production.
+- No `refresh`/drift detection yet: `read` is not implemented, so out-of-band
+  changes are not noticed until the next `up` overwrites them.
+- The cutover itself should still use `vyos-unifi-switch`. This is for day-2.

pulumi-vyos/index.ts

@@ -0,0 +1,27 @@
+import * as pulumi from "@pulumi/pulumi";
+import { VyosConfigTree } from "./vyosConfigTree";
+
+const cfg = new pulumi.Config();
+const host = cfg.get("host") ?? "172.31.1.252";
+const apiKey = cfg.get("apiKey") ?? "labsim-proto-key";
+
+// One subtree, one resource, one commit. This is the shape a BGP change would
+// take: edit the commands array, `pulumi up`, and it lands as a single
+// commit-confirmed transaction alongside whatever Kubernetes resources changed
+// in the same plan.
+const dnsForwarding = new VyosConfigTree("dns-forwarding", {
+    host, apiKey,
+    path: ["service", "dns", "forwarding"],
+    commands: [
+        ["cache-size", "20000"],
+        ["listen-address", "172.31.10.1"],
+        ["allow-from", "172.31.10.0/23"],
+        ["name-server", "8.8.8.8"],
+        ["name-server", "8.8.4.4"],
+        ["name-server", "1.1.1.1"],
+    ],
+    confirmMinutes: 2,
+    save: true,
+});
+
+export const managed = dnsForwarding.id;

pulumi-vyos/package.json

@@ -0,0 +1,11 @@
+{
+  "name": "vyos-proto",
+  "main": "index.ts",
+  "devDependencies": {
+    "@types/node": "^22",
+    "typescript": "^5.9.3"
+  },
+  "dependencies": {
+    "@pulumi/pulumi": "^3.140.0"
+  }
+}

pulumi-vyos/tsconfig.json

@@ -0,0 +1,2 @@
+{ "compilerOptions": { "strict": true, "target": "es2020", "module": "commonjs",
+  "moduleResolution": "node", "skipLibCheck": true, "esModuleInterop": true } }

pulumi-vyos/vyosConfigTree.ts

@@ -0,0 +1,205 @@
+import * as pulumi from "@pulumi/pulumi";
+import * as https from "https";
+
+/**
+ * A VyOS config subtree, managed as ONE Pulumi resource.
+ *
+ * Why a dynamic provider rather than the community Terraform providers: they
+ * batch correctly (one `vyos_config_block_tree` becomes a single POST to
+ * /configure, so one commit) but they never send `confirm_time`. VyOS's own API
+ * supports it -- `ConfigureListModel.confirm_time`, and "A non-zero confirm_time
+ * will start commit-confirm timer on commit" in rest/routers.py -- so a config
+ * push that breaks your access to the router can undo itself. On a gateway that
+ * is the difference between a mistake and an outage, and it is worth ~150 lines
+ * to keep.
+ *
+ * The unit of change is a SUBTREE, not a line. `protocols bgp` is one resource;
+ * so is `service dhcp-server`. That keeps one Pulumi resource == one commit,
+ * rather than turning 300 config lines into 300 commits with no ordering
+ * guarantee and no way to revert them as a unit.
+ */
+
+export interface VyosConfigTreeArgs {
+    /** Router address, e.g. "10.0.1.252". */
+    host: pulumi.Input<string>;
+    /** API key from `set service https api keys id <n> key <k>`. */
+    apiKey: pulumi.Input<string>;
+    /** Subtree root as a path array, e.g. ["protocols", "bgp"]. */
+    path: pulumi.Input<string[]>;
+    /**
+     * Desired state of the subtree: `set` command suffixes relative to `path`,
+     * each already split into path components with the value last.
+     */
+    commands: pulumi.Input<string[][]>;
+    /**
+     * Minutes before an unconfirmed commit reverts itself. 0 disables it.
+     * Non-zero is strongly preferred for anything reachable only through the
+     * router being changed.
+     */
+    confirmMinutes?: pulumi.Input<number>;
+    /** Persist to config.boot after a successful confirm. */
+    save?: pulumi.Input<boolean>;
+}
+
+interface Inputs {
+    host: string;
+    apiKey: string;
+    path: string[];
+    commands: string[][];
+    confirmMinutes: number;
+    save: boolean;
+}
+
+// Kept inside the module so the dynamic provider can serialise it.
+function apiCall(
+    host: string, apiKey: string, endpoint: string,
+    form: Record<string, string>,
+): Promise<any> {
+    const body = Object.entries(form)
+        .map(([k, v]) => `${encodeURIComponent(k)}=${encodeURIComponent(v)}`)
+        .join("&");
+    return new Promise((resolve, reject) => {
+        const req = https.request({
+            host, port: 443, path: `/${endpoint}`, method: "POST",
+            // VyOS ships a self-signed certificate by default. Verification is
+            // disabled deliberately; if this ever leaves a trusted segment,
+            // install a real certificate and turn it back on.
+            rejectUnauthorized: false,
+            headers: {
+                "Content-Type": "application/x-www-form-urlencoded",
+                "Content-Length": Buffer.byteLength(body),
+            },
+            timeout: 120_000,
+        }, (res) => {
+            let data = "";
+            res.on("data", (c) => (data += c));
+            res.on("end", () => {
+                try {
+                    const parsed = JSON.parse(data);
+                    if (parsed.success === false) {
+                        reject(new Error(`VyOS API: ${parsed.error ?? data}`));
+                    } else {
+                        resolve(parsed);
+                    }
+                } catch {
+                    reject(new Error(`VyOS API returned non-JSON: ${data.slice(0, 300)}`));
+                }
+            });
+        });
+        req.on("error", reject);
+        req.on("timeout", () => { req.destroy(); reject(new Error("VyOS API timed out")); });
+        req.write(body);
+        req.end();
+    });
+}
+
+/**
+ * Replace a subtree: delete it, then set the desired state, in ONE request so
+ * it is a single commit. Deleting first makes the result the declared state
+ * rather than a merge with whatever was there -- which is what makes `pulumi
+ * up` converge instead of accumulating.
+ */
+async function applyTree(i: Inputs, desired: string[][]): Promise<void> {
+    const commands: any[] = [{ op: "delete", path: i.path }];
+    for (const c of desired) {
+        commands.push({ op: "set", path: [...i.path, ...c] });
+    }
+
+    // The payload MUST be {commands: [...], confirm_time: N}, not a bare array.
+    // VyOS only reads confirm_time when the body parses as ConfigureListModel
+    // (`if isinstance(data, (ConfigureModel, ConfigureListModel, ...))` in
+    // rest/routers.py). A bare array is accepted and committed happily -- with
+    // NO timer armed, silently discarding the safety net. Verified both ways:
+    // bare array gives no "Initialized commit-confirm" in the response, the
+    // object form returns "Initialized commit-confirm; N minutes to confirm".
+    const payload: any = { commands };
+    if (i.confirmMinutes > 0) payload.confirm_time = i.confirmMinutes;
+
+    const res = await apiCall(i.host, i.apiKey, "configure",
+                              { key: i.apiKey, data: JSON.stringify(payload) });
+
+    if (i.confirmMinutes > 0) {
+        // Refuse to continue if the timer did not actually arm -- otherwise a
+        // silent fallback to an unprotected commit is exactly the failure this
+        // resource exists to prevent.
+        const said = String((res && res.data) || "");
+        if (!said.includes("commit-confirm")) {
+            throw new Error(
+                "commit-confirm was requested but VyOS did not arm a timer " +
+                `(response: ${said.trim() || "<empty>"}). Refusing to proceed.`);
+        }
+    }
+
+    if (i.confirmMinutes > 0) {
+        // The commit landed but is on a timer. Reaching the API again proves
+        // the box is still answering *after* the change -- the only evidence
+        // worth confirming on. If this throws we deliberately do NOT confirm,
+        // and the router reverts itself.
+        await apiCall(i.host, i.apiKey, "retrieve", {
+            key: i.apiKey,
+            data: JSON.stringify({ op: "showConfig", path: [] }),
+        });
+        // There is no /confirm endpoint -- confirm is an op on /configure, and
+        // it requires a `path` field even though it ignores it (the Union
+        // resolves to ConfigureModel, which mandates path). Without it the API
+        // answers "missing 'path' field" and the timer keeps running.
+        await apiCall(i.host, i.apiKey, "configure", {
+            key: i.apiKey,
+            data: JSON.stringify({ op: "confirm", path: [] }),
+        });
+    }
+
+    if (i.save) {
+        await apiCall(i.host, i.apiKey, "config-file", {
+            key: i.apiKey, data: JSON.stringify({ op: "save" }),
+        });
+    }
+}
+
+const provider: pulumi.dynamic.ResourceProvider = {
+    async create(inputs: Inputs) {
+        await applyTree(inputs, inputs.commands);
+        return { id: `${inputs.host}:${inputs.path.join("/")}`, outs: inputs };
+    },
+
+    async update(_id, _old: Inputs, news: Inputs) {
+        await applyTree(news, news.commands);
+        return { outs: news };
+    },
+
+    async delete(_id, props: Inputs) {
+        const form: Record<string, string> = {
+            key: props.apiKey,
+            data: JSON.stringify([{ op: "delete", path: props.path }]),
+        };
+        if (props.confirmMinutes > 0) form.confirm_time = String(props.confirmMinutes);
+        await apiCall(props.host, props.apiKey, "configure", form);
+        if (props.confirmMinutes > 0) {
+            await apiCall(props.host, props.apiKey, "configure",
+                { key: props.apiKey, data: JSON.stringify({ op: "confirm", path: [] }) });
+        }
+    },
+
+    async diff(_id, olds: Inputs, news: Inputs) {
+        const changed =
+            JSON.stringify(olds.commands) !== JSON.stringify(news.commands) ||
+            JSON.stringify(olds.path) !== JSON.stringify(news.path) ||
+            olds.host !== news.host;
+        return {
+            changes: changed,
+            // Changing which subtree or which router is a different resource.
+            replaces: olds.host !== news.host ||
+                      JSON.stringify(olds.path) !== JSON.stringify(news.path) ? ["path"] : [],
+        };
+    },
+};
+
+export class VyosConfigTree extends pulumi.dynamic.Resource {
+    constructor(name: string, args: VyosConfigTreeArgs, opts?: pulumi.CustomResourceOptions) {
+        super(provider, name, {
+            confirmMinutes: 5,
+            save: true,
+            ...args,
+        }, opts);
+    }
+}