Merge origin/main into feat/agentic-teams

124 commits of main against 33 of agentic-teams. 22 files overlapped, 7
conflicted. Three needed real reconciliation rather than a side-pick:

**The `volumes` collision.** Both branches independently added a `volumes`
field to ContainerSpec meaning different things: main's is persistent storage
keyed by a server-derived `claimName` that deliberately outlives an instance;
mine is emptyDir scratch shared between an init container and the main
container. Worse, each emitted the pod-level `volumes` list from its own
helper, so spreading both would have silently kept only the last — a server
with a PVC would have lost it the moment a session scratch volume appeared.

Resolved by renaming the unreleased side: mine is now `scratchVolumes`, main's
keeps `volumes` since it is already shipped and in use. `buildPodVolumes` now
owns the whole pod-level list and emits both kinds; `buildPodExtras` is init
containers only. fsGroup/securityContext stays tied to the PVC case, because
an emptyDir is created writable by the container's own user and chowning it
is work with no purpose. Five tests pin the coexistence so a later change
cannot drop one kind again.

**The extracted permission mapper.** I moved `mapUrlToPermission` out of
main.ts into middleware/url-permission.ts so it could be tested; main kept it
inline and kept editing it. Kept the extraction and ported main's one change
across — the `/api/v1/audit/events/purge` mapping onto the existing
`audit-purge` operation. Verified rule-by-rule that the extracted version is a
superset: all 15 of main's rules plus the 5 agentic-teams ones.

**The NUL byte in create.ts.** Both branches fixed the same defect
independently, mine as `''` and main's as `'\0'`. Took main's.

The rest were one-liners: .gitignore comment wording, and an add/add on
proxy-long-running.test.ts where main's copy carries a progressive-delivery
test mine lacked — took main's in full.

Not changed, but worth flagging: main's generateDeploymentSpec hardcodes
`automountServiceAccountToken: false` while generatePodSpec honours
`spec.automountServiceAccountToken`. That asymmetry is main's own and predates
this merge, so it is left exactly as-is rather than altered inside a merge
commit.

Verified: 3138 tests pass (237 files), build clean, prisma schema valid.
Lint 809/80 vs 794/77 pre-merge — the delta is main's own debt; every file
touched during conflict resolution was checked and introduces no new errors.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4wNHWf7xSwnZCWpJcyv9p

docs/claude-integration.md

@@ -0,0 +1,181 @@
+# mcpctl × Claude Code
+
+## What `mcpctl config claude --project X` wires up
+
+| Piece | Where | Purpose |
+|-------|-------|---------|
+| MCP entry | `~/.claude.json` (**user scope, every directory**) | one server, always named `mcpctl`, running the stdio bridge `mcpctl mcp -p X` |
+| Project marker | `./.mcpctl-project` | what `skills sync` reads to know the scope (`--skip-marker` opts out) |
+| Skills | `~/.claude/skills/` | the project's `SKILL.md` bundles |
+| SessionStart hook | `~/.claude/settings.json` | `mcpctl skills sync --quiet` on every session |
+| Status line | `~/.claude/settings.json` | the active project, bottom of the screen |
+| `/mcpctl` | `~/.claude/commands/mcpctl.md` | switch projects from inside a session |
+
+`--claude-dir` (or Claude Code's own `CLAUDE_CONFIG_DIR`) redirects everything
+under `~/.claude`.
+
+## User scope by default: wire it once, not per checkout
+
+Claude Code has two MCP scopes: **project** (`./.mcp.json`, that directory only,
+and usually committed — so writing to it dirties the repo) and **user**
+(`mcpServers` in `.claude.json`, every directory and every window).
+
+mcpctl now defaults to **user** scope. One active project everywhere is how the
+pi, prime-agent and opencode integrations already behave; per-directory wiring
+meant re-running `config claude` in every checkout you opened. Switching the
+project is then one command from anywhere.
+
+```bash
+mcpctl config claude --project homeautomation          # applies everywhere
+mcpctl config claude --project docmost --scope project # just this repo
+mcpctl config claude --project docmost -o .mcp.json    # same thing; -o implies project scope
+```
+
+User scope writes **no `.mcpctl-project` marker** — it scopes nothing to a
+directory, and a marker beside `.claude.json` would sit in `$HOME` and scope
+every repo under it. `--inspect` stays project-scope: it is a debugging server
+you turn on for one checkout.
+
+> `.claude.json` also holds onboarding state, caches and a per-project map that
+> Claude Code rewrites while running, so mcpctl merges into it and writes
+> through a temp file + rename. A truncated write there costs far more than a
+> stale MCP entry.
+>
+> Note the path asymmetry: with `CLAUDE_CONFIG_DIR` set the file is
+> `$CLAUDE_CONFIG_DIR/.claude.json`, but by default it is `$HOME/.claude.json` —
+> *beside* `~/.claude/`, not inside it.
+
+## One server named `mcpctl`, not one per project
+
+The MCP entry used to be named after the project. Because `.mcp.json` is
+*merged*, configuring a second project left the first mounted too — every
+project you had ever configured stayed connected, with duplicate tool names and
+nothing marking which was active.
+
+There is now exactly one entry, `mcpctl`, and switching rewrites what sits
+behind it:
+
+```jsonc
+{ "mcpServers": { "mcpctl": { "command": "mcpctl", "args": ["mcp", "-p", "docmost"] } } }
+```
+
+Two things follow. The tool prefix is stable across switches, so the model never
+sees a tool namespace disappear mid-conversation. And because Claude Code can
+reconnect an existing MCP server from `/mcp`, a switch lands without restarting
+the app.
+
+Entries an older CLI wrote are retired on the next run. They are recognised by
+the pairing that makes retiring them safe — *our* command, named after the very
+project it bridges to. A hand-configured server is never touched, even one
+called `docmost`, unless it also runs `mcpctl mcp -p docmost`, at which point it
+is the same entry anyway.
+
+## Switching: `/mcpctl`
+
+```
+❯ /mcpctl docmost
+● Bash(mcpctl config claude --project docmost --skip-marker)
+  ⎿  Wrote .mcp.json (1 server(s))
+● Now on docmost. Reconnect the mcpctl server from /mcp for its tools to load.
+```
+
+With no argument it lists the projects and asks.
+
+Claude Code slash commands are **prompt files, not code**, so unlike opencode's
+picker this drives the model through CLI calls — there is no keyboard picker to
+be had. `allowed-tools` is scoped to the four exact `mcpctl` invocations it
+needs, so accepting it does not hand the session a general shell.
+
+> Every `` !`…` `` block in a slash command is permission-checked against
+> `allowed-tools` *before* the model runs. Omitting one fails the whole command
+> with a permission error and no explanation. A test asserts every pre-executed
+> command in ours is covered.
+
+`--skip-marker` is deliberate: the session's directory is whatever you happened
+to open, and re-scoping it would silently change which skills sync into it.
+
+## The status line
+
+```
+  mcpctl:docmost
+  ⏵⏵ bypass permissions on · ← for agents
+```
+
+`mcpctl statusline` reads the directory from the JSON Claude Code pipes in, so it
+follows `/cwd` rather than reporting wherever the binary was launched, and prints
+**nothing** when no project is active — an empty status line beats one saying
+"none" on every unrelated repo.
+
+It then takes the project from the most deliberate source that names one:
+
+1. a canonical `mcpctl` entry in that directory's `.mcp.json` — a repo that
+   pinned itself wins, and it is the scope Claude Code itself prefers when both
+   define that server name;
+2. the user-scope entry in `.claude.json` — what `config claude --project`
+   writes, so a switch takes effect everywhere it is not overridden;
+3. a **legacy** project-named entry in `.mcp.json` (`homeautomation`,
+   `docmost`, …), left by an mcpctl older than the constant server name;
+4. a `.mcpctl-project` marker up the tree, so a checkout that is scoped but not
+   yet wired still reports.
+
+> **Legacy entries rank below user scope on purpose.** They used to outrank it,
+> which made switching look broken: a user-scope switch never rewrites a
+> checkout's `.mcp.json`, so the leftover kept naming the old project for good.
+> A pin is a decision; residue is not.
+
+A server Claude Code has switched off for that directory (`disabledMcpServers` /
+`disabledMcpjsonServers`) is skipped at every step — a disabled server is not
+mounted, so naming its project would be a lie. A `.mcp.json` server that is in
+neither list is still awaiting its approval prompt and does count, since blanking
+the status line on a fresh checkout is the more confusing failure.
+
+### When a directory contradicts a switch
+
+Claude Code merges the two scopes rather than picking one, so switching in user
+scope cannot clean up what a directory declares. `config claude` says so rather
+than reporting plain success:
+
+```
+Warning: /path/to/repo/.mcp.json still registers 'homeautomation' for this
+directory — mounted alongside 'sre', not replaced by it.
+  Re-run with --scope project to retire it, or delete the entry by hand.
+```
+
+A canonical entry pinned to another project gets the stronger wording — it
+*overrides* the switch in that directory rather than sitting beside it.
+
+### It is never installed over yours
+
+A status line is a single slot, so overwriting a custom one silently deletes
+work. When `config claude` finds a foreign one it leaves it and prints the
+snippet to add instead:
+
+```
+Left your existing status line alone (my-fancy-prompt).
+  To show the project too, append: $(mcpctl statusline)
+```
+
+> **Ownership is decided by the command string, not a marker.** Claude Code
+> rewrites `settings.json` against its own schema and **strips unknown keys from
+> `statusLine`** — a tagged entry comes back as a bare `{type, command}`.
+> (Hooks keep their marker; `statusLine` does not.) Matching on the command is
+> what stops us reporting our own line as foreign forever. A line that merely
+> *composes* ours — `my-prompt && mcpctl statusline` — is yours, and is left
+> alone.
+
+## Skills
+
+`mcpctl skills sync` installs into `~/.claude/skills/`. Claude Code is the only
+target that also gets hooks, `postInstall` and `mcpServers` auto-attach; pi,
+prime-agent and opencode share the simpler flat-tree semantics.
+
+The SessionStart hook keeps them current. It carries a `_mcpctl_managed` marker,
+and an install now also drops **untagged duplicates of that exact command** —
+rows left behind before the marker existed (or by a test suite that used to
+write into a real `~/.claude`). They are invisible in the UI and just run the
+sync twice per session. A hook you wrote is never touched, even one that also
+calls `mcpctl skills sync` with different flags.
+
+## Running Claude Code on the homelab LLM
+
+See [claude-vllm.md](claude-vllm.md).

docs/claude-vllm.md

@@ -0,0 +1,131 @@
+# `claude-vllm` — Claude Code on the homelab LLM
+
+## The short version
+
+```bash
+claude-vllm                                   # default provider + model
+claude-vllm --model deepseek-v4-max
+claude-vllm --list                            # what would it use?
+claude-vllm --print-env                       # the exports, without running claude
+claude-vllm -- -p "summarise this repo"       # anything after -- goes to claude
+```
+
+## Why no bridge is needed
+
+The homelab gateway at `llm.ad.itaz.eu` is **LiteLLM in front of vLLM**, and
+LiteLLM already serves the Anthropic Messages API:
+
+```console
+$ curl -s https://llm.ad.itaz.eu/v1/messages -X POST \
+    -H 'x-api-key: <key>' -H 'anthropic-version: 2023-06-01' \
+    -d '{"model":"deepseek-v4-fast","max_tokens":32,
+         "messages":[{"role":"user","content":"say only: ok"}]}'
+{"type":"message","role":"assistant","model":"deepseek-v4-fast",
+ "content":[{"type":"text","text":"ok"}],"stop_reason":"end_turn", …}
+```
+
+That is the exact protocol Claude Code speaks. So pointing `ANTHROPIC_BASE_URL`
+at the gateway is the whole integration — no translation layer, no proxy, no
+mcpctl in the request path.
+
+### Why not route it through mcpctl
+
+mcpctl's LLM layer is a **client**, not a server: `mcpd` exposes
+`/api/v1/llms` (management) and the adapters in
+`src/mcpd/src/services/llm/adapters/` *call out* to Anthropic/OpenAI for gating,
+prompt selection and agent chat. Nothing in mcpd or mcplocal serves
+`/v1/messages`. A `claude-mcpctl` would therefore mean adding an
+Anthropic-shaped passthrough to mcpd that re-wraps what LiteLLM already does —
+new surface, new failure mode, no new capability.
+
+It would only be worth it if you wanted mcpctl *in* the LLM path for its own
+sake: per-project gating of model calls, audit of prompts, or budget
+enforcement. Those are real features, but they are a passthrough endpoint in
+mcpd, not a wrapper script.
+
+## Its own config
+
+```bash
+claude-vllm --init                       # prompts for the key, writes 0600
+claude-vllm --init --api-key '${MY_KEY}' # keep the secret in the environment instead
+claude-vllm --init --base-url https://other-gateway/v1
+```
+
+Written to `$XDG_CONFIG_HOME/mcpctl/claude-vllm.jsonc` (`~/.config/…`), shaped
+like `opencode.jsonc` — a `provider` map with `options.baseURL` /
+`options.apiKey` and a `models` map:
+
+```jsonc
+{
+  "model": "itaz/deepseek-v4-think",
+  "provider": {
+    "itaz": {
+      "options": { "baseURL": "https://llm.ad.itaz.eu/v1", "apiKey": "${MY_LLM_KEY}" },
+      "models": { "deepseek-v4-fast": { "limit": { "context": 393216 } } }
+    }
+  }
+}
+```
+
+**Same shape as opencode's config, but a separate file.** Reading opencode's own
+config would mean a credential rotation there silently changing what Claude Code
+authenticates with, and would couple two tools' configs for no reason.
+
+### Keys are never stored in the tool
+
+`apiKey` may be a literal (in a 0600 file), `${ENV_VAR}`, or a bare env var
+NAME — so the secret can live in your environment or a password manager instead
+of on disk. `--init` reads the key from **stdin** when `--api-key` is omitted,
+keeping it out of shell history and out of the process table, where `--api-key`
+would be visible to every user via `ps`. `--list` never prints more than a
+10-character prefix.
+
+### Discovery order
+
+| Order | Source | Supplies |
+|-------|--------|----------|
+| 1 | environment (`ANTHROPIC_BASE_URL`, `ANTHROPIC_AUTH_TOKEN`, `ANTHROPIC_MODEL`) | anything already set is respected |
+| 2 | `--provider` / `--model` flags | provider, model |
+| 3 | `~/.config/mcpctl/claude-vllm.jsonc` | its own config |
+| 4 | `~/.pi/agent` | `settings.json` → default provider/model · `models.json` → base URL, context windows · `auth.json` → key |
+| 5 | `~/.prime/agent` | same shape |
+
+The pi/prime homes remain a convenience fallback so `claude-vllm` works before
+you have written a config at all. The base URL and the credential always come
+from the *same* source, so one gateway's URL is never paired with another's key.
+
+`--init` records the context window for **every** model it can see, not just the
+active one — otherwise `--model something-else` silently falls back to Claude
+Code's assumed 200k.
+
+## What it sets, and why each one
+
+| Variable | Reason |
+|----------|--------|
+| `ANTHROPIC_BASE_URL` | The gateway. A stored OpenAI-style `…/v1` base has the suffix stripped — Claude Code appends `/v1/messages` itself, and `/v1/v1/messages` 404s. |
+| `ANTHROPIC_AUTH_TOKEN` + `ANTHROPIC_API_KEY` | The gateway takes either; which one Claude Code sends has changed between releases. |
+| `ANTHROPIC_MODEL` | The model to drive the session. |
+| `ANTHROPIC_SMALL_FAST_MODEL`, `ANTHROPIC_DEFAULT_HAIKU_MODEL` | Background/summarisation calls otherwise ask for a real Haiku the gateway does not serve, and every one 404s. |
+| `CLAUDE_CODE_MAX_CONTEXT_TOKENS` | Claude Code assumes 200k for models it has no table for. `deepseek-v4-*` is 393k, so without this it auto-compacts at half capacity. Taken from `contextWindow` in the provider config. |
+| `CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS` | Beta headers the gateway does not implement make it reject otherwise-fine requests. |
+| `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC` | No telemetry about a non-Anthropic endpoint. |
+
+## Expected noise
+
+```
+⚠ claude.ai connectors are disabled because ANTHROPIC_API_KEY or another auth
+  source is set and takes precedence over your claude.ai login
+```
+
+Unavoidable and harmless: it fires for *any* non-claude.ai auth, including
+`ANTHROPIC_AUTH_TOKEN` alone (verified). It means your claude.ai org connectors
+are not loaded for this session — which is the point of running against the
+homelab.
+
+## Verified
+
+```console
+$ claude-vllm --model deepseek-v4-fast -- -p "Reply with exactly: PARITY-OK"
+claude-vllm: itaz · deepseek-v4-fast · https://llm.ad.itaz.eu
+PARITY-OK
+```

docs/opencode-extension.md

@@ -0,0 +1,210 @@
+# mcpctl × opencode — project switching from the TUI
+
+## Motivation
+
+`mcpctl config claude` and `mcpctl config prime-agent` both work the same way:
+write the project's MCP server into the host's config file, then restart the
+host so it picks the change up. That is fine when you pick a project once, and
+tiresome when you switch between `homeautomation`, `sre` and `docmost` all day.
+
+[opencode](https://opencode.ai) can do better, because it exposes two things
+the other hosts do not:
+
+- an **HTTP API for its own MCP registry** (`POST /mcp`,
+  `/mcp/{name}/disconnect`) — so a mount can be re-pointed while the app runs;
+- a **TUI plugin API** (`tui.json`) with slash commands, dialogs and UI slots —
+  so the switcher can be a first-class part of the interface rather than a
+  shell command you run in another terminal.
+
+So the opencode integration does not write an MCP entry into `opencode.json` at
+all. It ships two plugins, and switching projects takes effect on the next turn
+with no restart.
+
+## How it works
+
+```
+~/.mcpctl/opencode-state.json   (0600)
+   { project, gatewayUrl, tokens: { <project>: <mcpctl PAT> } }
+        │
+        │ read by both plugins
+        ▼
+~/.config/opencode/plugin/mcpctl.ts        server plugin  ─┐
+~/.config/opencode/mcpctl/mcpctl-tui.tsx   TUI plugin     ─┤
+                                                           │  client.mcp.add({
+                                                           │    name: "mcpctl",
+                                                           ▼    type: "remote", url, headers })
+                                              opencode's MCP registry
+                                                           │
+                                                           ▼
+                              https://<gateway>/projects/<project>/mcp
+```
+
+### The state file, not `opencode.json`
+
+1. **The restart.** A config file is read at startup. Re-pointing the mount
+   through the running server's MCP API is what makes `/mcpctl` instant. This is
+   the load-bearing reason.
+2. **The token.** Having chosen the gateway, we need
+   `Authorization: Bearer <mcpctl PAT>` somewhere. `opencode.json` is a mode-0644
+   file people paste into bug reports; `~/.mcpctl/opencode-state.json` is 0600,
+   like every other mcpctl credential.
+
+> **Reason 2 is not an argument for this design over the alternative.** A
+> `type: "local"` entry running `mcpctl mcp -p <project>` — the same stdio
+> bridge `config claude` uses — needs no bearer token at all, because the bridge
+> reads your own `~/.mcpctl/credentials`. So "no secret in a 0644 file" is not a
+> point against that approach; it is just a consequence of having picked the
+> HTTP gateway.
+>
+> The honest trade is: the gateway works against a remote mcpctl with no local
+> `mcplocal` daemon, and mounts through an API that can be re-pointed live. The
+> stdio bridge is simpler and credential-free, but requires `mcpctl` and a
+> reachable mcplocal on the same machine. Both are defensible; this one was
+> chosen for the remote case and for the live re-point, not for the token.
+
+Tokens are kept **per project**, so switching back to a project you have
+already used needs no new mint — and a failed mint for project B cannot cost
+you the credential for project A.
+
+### The server plugin (`plugin/mcpctl.ts`)
+
+Auto-discovered by opencode. It mounts the active project's gateway under the
+fixed MCP server name `mcpctl`, on the first server event and again before
+every user turn.
+
+Two details matter:
+
+- **The name is constant.** Tools keep a stable `mcpctl_*` prefix across
+  switches, and because opencode re-resolves the tool list per request, the
+  model simply sees the new project's tools on its next turn. (Contrast pi,
+  where the extension has to *tell* the model its old tool names are dead.)
+- **It does not re-register an unchanged mount.** `mcp.add` rebuilds the
+  connection, and mcplocal binds a gated project's unlocked state to that
+  connection's `mcp-session-id` — re-adding every turn would re-lock a project
+  you had just opened with `begin_session`.
+
+It also exists so that **headless** runs (`opencode run …`), which load no TUI
+plugins at all, still get the project's tools.
+
+> The mount is deliberately *not* performed during plugin setup. Setup runs
+> before opencode's server accepts connections, and `client.mcp.add` calls back
+> into that same server — awaiting it there hangs the app on a blank screen
+> before the TUI ever draws.
+
+### The TUI plugin (`mcpctl/mcpctl-tui.tsx`)
+
+Registered in `~/.config/opencode/tui.json` (opencode does not auto-discover
+TUI plugins). It adds:
+
+| Command | What it does |
+|---------|--------------|
+| `/mcpctl` (or `<leader>m`) | Filterable project picker; switches live |
+| `/mcpctl-status` | Active project, mount state, gateway URL |
+| `/mcpctl-skills` | Re-sync this project's skills |
+
+Switching is the thing you do repeatedly, so it gets a chord as well as a slash
+command; the other two stay palette-only.
+
+and a `mcpctl:<project>` indicator in the prompt footer, next to the model name
+and one line above the token counter.
+
+Switching delegates to the CLI —
+`mcpctl config opencode --project X --skip-plugin --skip-marker` — so token
+minting, state and skills stay in one place and the plugin stays a UI shell.
+`--skip-plugin` avoids rewriting the very file opencode has already loaded;
+`--skip-marker` stops a switch from silently re-scoping whichever repository
+opencode happened to be started in.
+
+The picker needs no pre-filter prompt (unlike the pi and prime-agent
+switchers): opencode's select dialog filters as you type, so the plugin only
+has to order the list — active project first, then alphabetical.
+
+A switch **disconnects before re-adding**. `mcp.add` under the same name does
+re-point the tools on its own, but leaves it to opencode whether the previous
+client is closed, and an abandoned one keeps its `mcp-session-id` alive on
+mcplocal — the very thing that holds a gated project open. Best-effort: on a
+first mount there is nothing to disconnect.
+
+The indicator is published through `api.kv`, which is a reactive store: writing
+it re-renders the slot with no signal plumbing, and it persists across sessions
+so the label is right on the first frame.
+
+#### Why the footer and not the status bar
+
+opencode exposes UI slots, not arbitrary layout. In the footer region the
+options are:
+
+| Slot | Result |
+|------|--------|
+| `session_prompt_right` / `home_prompt_right` | **used** — renders on the prompt's bottom line, right of the model name, directly above the token counter |
+| `home_footer` | sits on the counter's line, but *replaces* the cwd/version footer instead of adding to it |
+| `app_bottom` | costs a whole extra terminal row |
+
+There is no slot on the status-bar line itself. The home prompt box is narrow
+enough that the default wrap breaks `mcpctl:homeautomation` across two lines
+mid-word, so the label renders `wrapMode="none" truncate` — clipping the tail of
+a long name reads better than a two-line footer. In a session the prompt is full
+width and it always fits.
+
+### Skills
+
+opencode implements the Agent Skills standard and loads `SKILL.md` trees from
+`~/.config/opencode/skill/` (XDG-aware). `mcpctl skills sync --agent opencode`
+syncs there, with the same shared-tree semantics as `--agent pi` and
+`--agent prime-agent`: a flat tree with per-project ownership, its own state
+file (`~/.mcpctl/skills-state-opencode.json`), no SessionStart hooks, no
+`postInstall`, and untracked skill directories are never clobbered.
+
+## Usage
+
+```bash
+mcpctl config opencode --project monitoring
+```
+
+That mints (or reuses) the project token, writes the 0600 state file, installs
+both plugins, registers the TUI plugin in `tui.json`, and runs an initial
+skills sync. Start opencode and the project's tools are there.
+
+Skip individual steps:
+
+```bash
+mcpctl config opencode --project monitoring --token mcpctl_pat_xxx  # provide token, don't mint
+mcpctl config opencode --project monitoring --skip-skills           # don't sync skills
+mcpctl config opencode --project monitoring --skip-plugin           # state only, leave plugins alone
+mcpctl config opencode --project monitoring --skip-marker           # don't touch .mcpctl-project here
+mcpctl config opencode --project monitoring --dry-run               # print the plan, write nothing
+mcpctl config opencode --project monitoring --opencode-dir /path    # non-default opencode config dir
+```
+
+## Failure semantics
+
+- **No usable credential → the switch fails.** The command exits non-zero and
+  leaves the state file untouched, so the previously active project keeps
+  working rather than being replaced by a mount that 401s. `/mcpctl` reads that
+  exit code and reports the switch as failed instead of claiming success over a
+  project with no tools.
+- **Gateway unreachable → no mcpctl tools.** Never a failed startup: the server
+  plugin swallows mount errors.
+- **Corrupt `tui.json` → refuses to write.** One syntax error must not silently
+  drop every other TUI plugin you installed.
+
+## Files
+
+| Path | Purpose |
+|------|---------|
+| `src/opencode-ext/mcpctl-opencode.ts` | Server plugin source |
+| `src/opencode-ext/mcpctl-opencode-tui.tsx` | TUI plugin source |
+| `src/cli/src/config/opencode-extension.ts` | Generated embed of both (do not edit) |
+| `scripts/generate-opencode-extension.ts` | Regenerates that embed |
+| `src/cli/src/utils/opencode-settings.ts` | Install / register / state helpers |
+
+The plugin sources are **embedded** in the CLI so `mcpctl config opencode`
+works from an installed binary with no source tree. After editing either
+source, re-run:
+
+```bash
+npx tsx scripts/generate-opencode-extension.ts
+```
+
+A test fails if you forget. They are typechecked against the real
+`@opencode-ai/plugin` types via `pnpm typecheck:opencode-ext`.

docs/pi-extension.md

@@ -0,0 +1,174 @@
+# mcpctl × pi — native integration (no MCP, no Claude)
+
+## Motivation
+
+With Claude Code, `mcpctl config claude --project X` wires a project's MCP
+servers into the agent two ways:
+
+1. a `.mcp.json` entry running `mcpctl mcp -p X` (an MCP **stdio bridge**), and
+2. a `SessionStart` hook + `mcpctl skills sync` that materialises server-side
+   **skills** under `~/.claude/skills/`.
+
+Claude Code supports MCP natively, so tools travel over MCP. **pi does not
+support MCP.** But pi has two native mechanisms that cover the same ground:
+
+- **Extensions** — TypeScript modules that can register tools, commands, and
+  custom UI, and can talk HTTP directly.
+- **Skills** — pi implements the [Agent Skills standard](https://agentskills.io/specification)
+  and loads `SKILL.md` trees (the exact format mcpctl already syncs).
+
+This addon exploits both so mcpctl works with **pi only**, with **no MCP** and
+**no dependency on `~/.claude/`** (so it keeps working if you drop Claude).
+
+## How it works
+
+The MCP layer is not magic. `mcpctl mcp` is just a JSON-RPC ↔ stdio bridge that
+forwards to mcplocal's Streamable-HTTP endpoint:
+
+```
+POST {mcplocalUrl}/projects/<project>/mcp
+Accepts: application/json, text/event-stream
+Authorization: Bearer <token>
+```
+
+The protocol is JSON-RPC over HTTP:
+- `initialize` → returns an `mcp-session-id` header
+- `notifications/initialized`
+- `tools/list` → `{ tools: [{ name, description, inputSchema }] }`
+- `tools/call` → `{ name, arguments }` → `{ content, isError }`
+
+The pi extension is a **direct JSON-RPC client** to that same endpoint. It
+replaces the MCP transport with a plain function call, so pi never needs an MCP
+client library.
+
+### Gating
+
+Some projects are *gated*: until `begin_session` is called, `tools/list` returns
+only `begin_session`. After it is called the full tool set appears.
+
+**Critical subtlety:** mcplocal's gate state is bound to a `mcp-session-id`. A
+fresh session is gated again even after another session was ungated — so the
+extension keeps **one persistent `McpHttpSession` per active project** and
+routes every `tools/call` through it. After each call it re-runs `tools/list`
+and reconciles the active pi tool set, so calling `begin_session` automatically
+opens the gate and reveals the rest — no push listener required (mcplocal's HTTP
+responses are request/response, not a persistent push stream).
+
+## Deliverables
+
+| Artifact | Purpose |
+|----------|---------|
+| `src/pi-ext/mcpctl-pi.ts` | Self-contained pi extension (load via `pi -e` or settings `extensions`) |
+| `mcpctl config pi` | CLI wiring: registers the extension with pi + initial skill sync |
+| `docs/pi-extension.md` | This document |
+
+## Project switching
+
+The active project is persisted in `~/.mcpctl/pi-state.json` (settable, immune to
+your shell's cwd) and is also inferred from a `.mcpctl-project` marker walk-up.
+`/mcpctl` opens an interactive menu; **switch project** lists projects via
+`ctx.ui.select()` and reloads the tool set for the new project, namespaced so
+projects never collide.
+
+### Tool naming
+
+To allow multiple projects and avoid schema collisions, registered pi tools are
+namespaced: `mc_<project-slug>_<mcp-tool-name>`. Only the **current project's**
+tools are in the active set at any time.
+
+## Skills
+
+Skills already sync in the Agent Skills `SKILL.md` format. `mcpctl config pi`
+(when given a project) writes them under pi's own directory
+`~/.pi/agent/skills/` and registers that directory in `~/.pi/agent/settings.json`
+— no Claude involvement. Re-sync later with:
+
+```bash
+mcpctl skills sync --agent pi --project <name>
+```
+
+`--agent pi` is what keeps skills out of `~/.claude/skills`; the default target
+is still Claude Code. The `/mcpctl` menu's **sync skills** action runs exactly
+this command. Sync state is tracked separately per agent
+(`~/.mcpctl/skills-state-pi.json`), so pi and Claude never fight over the same
+bookkeeping.
+
+## Layout
+
+```
+src/pi-ext/
+  mcpctl-pi.ts          # the extension (self-contained)
+  mcp-http.ts           # vendored Streamable-HTTP JSON-RPC client (no deps)
+```
+
+The extension imports only from pi-bundled packages, so it loads standalone.
+
+### `typebox` is the only bare runtime import
+
+pi does not let an extension resolve modules the ordinary way: it hands jiti a
+hard-coded alias table built from its *own* dependencies, and that table is not
+the same across pi distributions. The newer `@earendil-works/pi-coding-agent`
+aliases both the `@earendil-works/*` and the legacy `@mariozechner/*` names;
+older `@mariozechner/pi-coding-agent` installs (0.73.x and earlier) alias only
+the `@mariozechner/*` ones. Neither resolves the other's namespace.
+
+So an import of anything outside the intersection kills the *whole* extension on
+someone else's pi — every tool, the `/mcpctl` command, the status line — with:
+
+```
+Failed to load extension ".../mcpctl-pi.ts": Cannot find module '@earendil-works/pi-ai'
+```
+
+which is exactly what `import { StringEnum } from "@earendil-works/pi-ai"` did.
+`typebox` is aliased by every published pi, so it is the only bare specifier
+allowed at runtime. Everything else must be a `node:` builtin, a relative path,
+an `import type` (erased before jiti resolves anything), or inlined — pi-ai's
+`StringEnum` is now a six-line local `stringEnum`. The
+`tests/config/pi-extension-embed.test.ts` guard fails the build on a reintroduced
+runtime import.
+
+If a user does hit this error, check `type -a pi`: two installs on `$PATH` is the
+usual cause, and the extension has to load under whichever one wins.
+
+## Typechecking
+
+The extension ships as *source*: it is embedded into the CLI
+(`src/cli/src/config/pi-extension.ts`, generated by
+`scripts/generate-pi-extension.ts`) and written verbatim into
+`~/.pi/agent/extensions/mcpctl/`. The CLI's own build never compiles it, so
+without a dedicated project nothing would check it against pi's API — which is
+how a `ctx.ui.select()` call with the wrong option shape shipped in the first
+place.
+
+`src/pi-ext/tsconfig.json` closes that gap, checking against the **real**
+published `@earendil-works/pi-coding-agent` types (a dev dependency, not a
+hand-written shim — a shim that drifts from the published API is the exact
+failure this guards against):
+
+```bash
+pnpm run typecheck        # includes typecheck:pi-ext
+```
+
+After editing `src/pi-ext/*.ts`, regenerate the embedded copy or the CLI will
+keep installing the old sources:
+
+```bash
+npx tsx scripts/generate-pi-extension.ts
+```
+
+A test (`tests/config/pi-extension-embed.test.ts`) fails if you forget.
+
+## Authentication caveat
+
+The extension sends the mcpd session token from `~/.mcpctl/credentials` as its
+bearer. A locally running `mcpctl-local` daemon does not authenticate
+`/projects/*`, so this works — the header is simply ignored.
+
+`mcplocal serve` is different: it registers a token-auth preHandler that accepts
+**only** `mcpctl_pat_` bearers (project mcptokens), and rejects a session token
+with `401 Only mcpctl_pat_ bearers are accepted on this endpoint`. Pointing the
+extension at an authenticated `mcplocal serve` therefore needs a project
+mcptoken (`mcpctl create mcptoken <token-name> --project <name>`, printed once).
+Wiring that token through to the extension is not yet implemented — the
+`config prime-agent` path does the equivalent by storing it in
+`~/.prime/agent/auth.json`.

docs/prime-agent-extension.md

@@ -0,0 +1,101 @@
+# mcpctl × prime-agent
+
+## What `mcpctl config prime-agent --project X` wires up
+
+| Piece | Where | Purpose |
+|-------|-------|---------|
+| Proxy MCP | `~/.prime/agent/settings.json` | `{ type: "http", url: "<gateway>/projects/X/mcp" }` |
+| Bearer credential | `~/.prime/agent/auth.json` | `mcp:X` → a minted mcpctl PAT |
+| Skills | `~/.prime/agent/skills/` | the project's `SKILL.md` bundles |
+| `/mcpctl` switcher | `~/.prime/agent/extensions/mcpctl-switch.ts` | switch projects, and the active-project indicator |
+
+Unlike Claude Code (a stdio bridge, no token) and pi (native tools over
+JSON-RPC), prime-agent talks to the **HTTP gateway**, so the switch is really
+three things: a credential, a settings entry, and a reload.
+
+## Credentials are the fragile part
+
+`config prime-agent` mints an `mcptoken` per project, stores it under
+`mcp:<project>`, and retires the one it replaced. Three rules make that safe:
+
+- **A key being present proves nothing.** A revoked token would short-circuit
+  provisioning and leave prime-agent unable to reach the gateway while the
+  command reported success. mcptokens are shown once, so the stored token's
+  16-char `tokenPrefix` is compared against the project's *active* tokens rather
+  than sending the secret.
+- **No credential means the switch fails.** Non-zero exit, `settings.json`
+  untouched, so the previously active project keeps working instead of being
+  replaced by a mount that 401s — and the `/mcpctl` switcher, which reads that
+  exit code, reports failure rather than success over a project with no tools.
+- **Only the token we replaced is revoked.** Sweeping every `prime-agent` token
+  for a project would kill the one another machine is using. Anything else that
+  looks orphaned is reported, not deleted.
+
+This plumbing is shared with `config opencode`, parameterised by agent rather
+than copied.
+
+## The `/mcpctl` switcher
+
+`pi.registerCommand('mcpctl', …)` opens a picker, shells out to
+`mcpctl config prime-agent --project X --skip-extension --skip-marker`, then
+calls `ctx.reload()` — which re-reads `settings.json` and `auth.json` and
+rebuilds the MCP map, so the switch lands without restarting the app.
+
+`--skip-extension` stops it rewriting the very file it is running from;
+`--skip-marker` stops it re-scoping whatever repository prime-agent was started
+in, which Claude Code's own skills sync would then pick up.
+
+Above 20 projects it asks for a filter first: prime-agent's selector is an
+arrow-key list with no search, and real installs run to hundreds of projects.
+(opencode's dialog filters as you type, so its switcher needs no such prompt.)
+
+## The active-project indicator
+
+Published with `ctx.ui.setStatus('mcpctl', …)`, which prime-agent renders in its
+tray line next to the model name.
+
+Two quirks worth knowing:
+
+- prime-agent emits `session_start` **only from `reload()`**, never at startup —
+  so the indicator is also published on `turn_start`, the earliest moment with a
+  real UI context bound.
+- `resetExtensionUI()` clears extension statuses *after* `session_start`, so the
+  indicator set there is wiped before anyone sees it. It is re-published on a
+  short retry schedule to land after that reset.
+
+> The rendering itself only exists in prime-agent from
+> `prime-agent-extension-status.patch` (upstream PR pending). On an unpatched
+> build `setStatus` silently does nothing and no indicator appears.
+
+## The extension is real source now
+
+It used to exist **only** as a string literal inside
+`src/cli/src/config/prime-agent-extension.ts` — so nothing typechecked or linted
+it, which is precisely the gap that let a wrong `ctx.ui.select()` option shape
+ship in the pi extension.
+
+It now lives at `src/prime-agent-ext/mcpctl-switch.ts`, checked against the real
+`@earendil-works/pi-coding-agent` types:
+
+```bash
+pnpm typecheck:prime-agent-ext
+npx tsx scripts/generate-prime-agent-extension.ts   # after editing it
+```
+
+A test fails if the embed goes stale. Extracting it found six lint problems in
+code that had never been linted — all null-guard and return-type issues rather
+than live bugs, but exactly the class of thing that ships silently when nothing
+is looking.
+
+## What it deliberately does *not* do
+
+The MCP entry is still **named after the project**, not the constant `mcpctl`
+that `config claude` and `config opencode` now use. prime-agent's switcher
+already unmounts the previous project, so it never accumulates entries the way
+`config claude` did — the bug the constant name fixes does not exist here.
+
+The remaining difference is tool-prefix stability: switching changes tool names,
+so the model can hold stale ones. Moving prime-agent to a constant name would
+also re-key `auth.json` from `mcp:<project>` to `mcp:mcpctl`, giving up
+per-project token caching and needing a migration. Worth doing, but as its own
+change rather than folded into a parity pass.

docs/project-summary.md

@@ -884,6 +884,39 @@ All pushed to `mysources.co.uk/michal/` registry.
 source .env && bash scripts/release.sh
 ```
 
+**The build refuses to run from a branch that is behind `main`.** Everyone
+branches off main, so a stale branch still builds and installs cleanly — it just
+ships a binary missing whatever landed on main meanwhile, and `rpm -U --force`
+overwrites the good one with it. That happened on 2026-08-10: a build from a
+stale checkout replaced `/usr/bin/mcpctl` with one that had no `statusline`
+command, months after the status line landed. `scripts/check-main-sync.sh`
+fetches `main`, compares, and fails before any work happens, listing the commits
+you are missing. It gates every path that produces something others consume:
+`build-rpm.sh`, `build-deb.sh`, `build-mcpd.sh` (each is also run standalone, so
+none can rely on another having checked) and `deploy-k8s.sh` — where a stale
+branch would pin its sha in Pulumi and make it the cluster's source of truth.
+`deploy-k8s.sh --dry-run` skips the check: it builds and cuts over nothing, and
+blocking a read-only inspection only teaches people to export the escape hatch
+permanently, disabling the gate for real deploys too.
+
+```bash
+git merge main                                   # the fix
+MCPCTL_ALLOW_BEHIND_MAIN=1 bash scripts/release.sh  # deliberate old-tree build
+MCPCTL_BASE_BRANCH=release-2.x bash scripts/build-rpm.sh  # compare to another branch
+```
+
+Offline it falls back to the last fetched `origin/main`, then to a local `main`,
+and says which it used; outside a git checkout it skips entirely.
+
+**A failing smoke run fails the release.** It used to print
+`WARNING: Smoke tests failed!` and exit 0 — which is exactly how four broken
+readiness probes shipped unnoticed (see `docs/reliability.md`): the warning
+scrolled past and the release reported success. Note what the gate does and does
+not do — smoke runs *last*, against the installed binary, so the package is
+already published and installed by the time it fails. It reports the breakage
+rather than preventing it, so investigate the fleet rather than assuming the
+artifact is bad. Override with `MCPCTL_ALLOW_SMOKE_FAILURE=1`.
+
 Installs via nfpm:
 - `/usr/bin/mcpctl` — CLI binary (bun compiled)
 - `/usr/bin/mcpctl-local` — Local proxy binary (bun compiled)

docs/reliability.md

@@ -19,15 +19,124 @@ and report the degradation — never hang and never degrade silently.**
   (<reason>)…` and sets `degraded: true` + `degradedReason` on the audit
   `gate_decision` event.
 
-Applied in: the gate's `begin_session` prompt selection
-(`proxymodel/plugins/gate.ts`, cap `MCPCTL_GATE_LLM_TIMEOUT_MS`, default 8s) and
-pagination's smart index (`llm/pagination.ts`, `MCPCTL_PAGINATION_LLM_TIMEOUT_MS`,
-default 10s). `read_prompts` is LLM-free by design.
+**One implementation:** [`util/degrade.ts`](../src/mcplocal/src/util/degrade.ts)'s
+`bounded()`. It cannot throw — the caller always gets a value or a reason — so
+the deterministic fallback is unconditional rather than something a `catch`
+block has to remember. `degradationNotice()` produces the `⚠ … unavailable
+(reason). hint` wording, and `degradationAudit()` the `degraded`/`degradedReason`
+payload, so every degradation reads the same whichever subsystem produced it.
+
+**Nothing optional is unbounded by construction.** The budget lives in
+`LLMProviderAdapter.complete()` (`proxymodel/llm-adapter.ts`), not at each call
+site, so a stage that passes no options is still bounded and a stage written
+next year inherits the guarantee. One budget spans the whole failover chain —
+a per-provider timeout would make the worst case N × timeout.
+
+| Knob | Default | Bounds |
+|---|---|---|
+| `MCPCTL_LLM_CALL_BUDGET_MS` | 20s | one `ctx.llm.complete()`, failover included |
+| `MCPCTL_LLM_PROVIDER_TIMEOUT_MS` | 10s | a single provider attempt inside that budget |
+| `MCPCTL_STAGE_LLM_BUDGET_MS` | 30s | all LLM work in one stage invocation |
+| `MCPCTL_GATE_LLM_TIMEOUT_MS` | 8s | the gate's `begin_session` prompt selection |
+| `MCPCTL_PAGINATION_LLM_TIMEOUT_MS` | 10s | pagination's smart index (`llm/pagination.ts`) |
+
+The **stage** budget exists because a per-call timeout multiplies rather than
+bounds when a stage loops: `summarize-tree` recurses to `maxDepth` (3) and loops
+per section at every level, so hundreds of sequential calls are reachable. One
+budget is shared across the whole recursion; when it is gone the stage switches
+to first-line excerpts and says so. A **warm cache is never budget-gated** — a
+cached summary costs nothing, so an exhausted budget must not degrade a result
+we already hold.
+
+`read_prompts` is LLM-free by design.
+
+### Why this is written down twice
+
+This document stated the principle while `proxymodel/stages/paginate.ts` awaited
+`ctx.llm.complete()` with no timeout at all. When a provider hung rather than
+erroring, the promise never settled, the tool call never returned, and the
+client waited out its own 1800s timeout — three such requests in production,
+misdiagnosed twice as an upstream "transport fault". The doc named the gate and
+`llm/pagination.ts` as the compliant sites, and the newer stages simply never
+joined the list. Naming **one** helper here, rather than a list of call sites,
+is what stops that drift recurring.
 
 Note: the gate's prompt-ranking uses the **heavy client provider's own model** —
 it deliberately does *not* force the project's vLLM model onto it (doing so made
 every selection fail silently when the model wasn't anthropic-servable).
 
+## Instance health: `live` is not `healthy`
+
+An MCP server answers `tools/list` from a **static, in-process table**. It costs
+a few milliseconds, needs no credentials, and reaches no upstream — so it stays
+green while the thing the server exists to talk to is unreachable. Treating that
+as a health signal is how `mcpctl get instances` showed eight healthy servers
+while the UniFi one had never once reached its controller.
+
+So the probe reports two different passes:
+
+| Status | Probe | Means |
+|---|---|---|
+| `healthy` | **readiness** — `tools/call` on `healthCheck.tool` | The upstream answered. The server can do its job. |
+| `live` | **liveness** — `tools/list` only | The process is up and speaks MCP. Its upstream is **unverified**. |
+| `degraded` | either, failing | Failing, but under `failureThreshold`. |
+| `unhealthy` | either, failing | Failed `failureThreshold` times in a row. |
+
+`live` is the default for any server with no `healthCheck.tool`. It is not a
+warning — it is an admission that nothing is watching that server's upstream.
+
+**Configure a readiness probe on every server.** Pick a read-only tool that
+genuinely round-trips to the upstream, and verify it passes before configuring
+it — a probe naming a local-only tool (`get_..._version`) or a tool the server
+doesn't expose reproduces the same false green it was meant to remove.
+
+```bash
+mcpctl create server unifi-network --health-check-tool list_sites \
+  --health-check-interval 60 --health-check-timeout 15 --force
+```
+
+or declaratively — `healthCheck` round-trips through `get -o yaml | apply -f`:
+
+```yaml
+healthCheck:
+  tool: list_sites
+  arguments: {}
+  intervalSeconds: 60
+  timeoutSeconds: 15
+  failureThreshold: 3
+```
+
+Omit `tool` to keep liveness while still tuning the timings.
+
+Latency is the tell: a probe answering in single-digit milliseconds is reading a
+local table, not crossing a network. The UniFi probe went from 3ms (`tools/list`,
+lying) to 1847ms on its first real `list_sites` — login, TLS, controller round
+trip — and ~40ms once the session was warm.
+
+### Where a failing readiness probe usually points
+
+Turning these probes on for the first time took the fleet from "8/8 healthy" to
+three genuine failures in under a minute. All three were network shape, not
+code — check these before suspecting the server:
+
+1. **Egress port.** MCP server pods default to TCP 80/443 only
+   (`servers-allow-external-egress`). Any upstream on another port — the UniFi
+   controller on `:8443` — times out on every call. Declare it in Pulumi's
+   `mcpctl.serverEgressTargets`; don't widen the blanket rule.
+2. **Ingress hairpin.** A co-located service reached over its *public* hostname
+   goes out and back through the per-host Envoy L7 policy, which doesn't
+   reliably carry the caller's identity and replies with a bare `Access denied`.
+   Grafana 403'd on every call this way while the identical token succeeded from
+   a laptop. The tell is the error *shape*: plain text, not the upstream's own
+   JSON error. Use the ClusterIP (`serverEgressTargets` with `namespace:`).
+3. **Address reachability.** A pod cannot reach a **Tailscale** `100.64.0.0/10`
+   address. Config pointing at one connect-timeouts forever. Use LAN IPs. (This
+   one turned out to be a retired service, which is its own kind of answer.)
+
+Also check the *dialect*: UniFi's `controller_type` must be `classic` for a
+self-hosted controller (login `/api/login`, no `/proxy/network` prefix).
+`unifi_os` sends every request to a path that 404s.
+
 ## LLM-*essential* operations — failover chain
 
 Chat needs *an* LLM but not a *specific* one. Instead of failing when the pinned

docs/secret-backends.md

@@ -118,8 +118,110 @@ That's the whole point of keeping plaintext around — it's the trust root:
    token itself. DB access is now equivalent to OpenBao token access (a single
    key), not equivalent to all API keys in the system.
 
-Follow-up work (not shipped yet) replaces static token auth with Kubernetes
-ServiceAccount auth so no bootstrap token is needed at all.
+#### Kubernetes ServiceAccount auth (no bootstrap token)
+
+`auth: kubernetes` removes the chicken-and-egg entirely: mcpd exchanges its
+projected ServiceAccount JWT for an OpenBao token at
+`auth/<authMount>/role/<role>`, so there is no static credential in the database
+at all. The token is cached for its lease and re-minted lazily with a 60s grace
+window.
+
+```yaml
+kind: secretbackend
+name: bao-k8s
+type: openbao
+isDefault: true
+config:
+  url: https://bao.example
+  auth: kubernetes
+  role: mcpctl
+  authMount: kubernetes-worker0   # defaults to `kubernetes`
+```
+
+Note that the daily **rotator does not apply** to these backends — there is no
+stored token to rotate. That has a consequence for monitoring, see below.
+
+## Reliability
+
+Remote backends are network dependencies on the critical path of nearly
+everything: server env resolution, LLM api keys, chat, git providers, code
+repos, webhooks. Three mechanisms keep an outage from cascading.
+
+### Request hardening
+
+Every call carries a timeout (default 5s) and retries `5xx`/`429`/network
+failures with full-jitter exponential backoff (3 attempts). A **sealed** OpenBao
+answers `503`, so this covers unseal windows and failovers.
+
+The `403` path is separate and deliberately single-shot: the driver purges its
+cached token, re-authenticates and retries **once**. That is a credential
+refresh, not a backend-unavailable condition — looping on it would hide a
+genuinely revoked grant.
+
+### Value cache with stale-while-error
+
+Resolved values are cached per backend (default TTL 5 minutes, LRU-bounded).
+Past the TTL the backend is always consulted; if it fails *as a transport
+failure*, the last known-good value is served instead of throwing.
+
+| Failure | Behaviour |
+|---|---|
+| Backend unreachable / timeout / exhausted 5xx | Serve last known-good, mark degraded, log `BACKEND_UNREACHABLE` once |
+| Secret deleted (404) | **Evict and throw.** Never served stale — that would resurrect a revoked credential |
+| 403 after a token refresh | Throw. Revoked grants must stay loud |
+| Nothing cached yet | Throw |
+
+The stale window is unbounded on purpose: a cap would mean a long outage
+eventually takes mcpd down anyway.
+
+`plaintext` backends are not cached — their `read()` is an identity function
+over the row the caller already supplied.
+
+**Cold cache is the known gap.** If mcpd restarts *while* the backend is
+unreachable, nothing has a last-known-good value and secret-bearing servers fail
+to start. That is deliberate: booting a server with an empty credential is worse
+(gitea-mcp once ran for weeks with an empty `GITEA_ACCESS_TOKEN`, answering
+`tools/list` and reporting healthy while every authenticated call failed). mcpd
+mitigates it by warming the cache at boot — one read per referenced secret — so
+an outage that starts *after* startup is fully absorbed.
+
+### Health: `live` vs `ready`
+
+```bash
+curl $MCPD/api/v1/secretbackends/<id>/health
+```
+
+```json
+{ "live": true, "liveDetail": "active",
+  "ready": false, "readyDetail": "OpenBao list: HTTP 403 permission denied",
+  "cache": { "entries": 9, "servingStale": 0 },
+  "rotation": { "rotatable": false, "lastRotationError": null } }
+```
+
+- **`live`** — unauthenticated `sys/health`. Distinguishes *down* from *sealed*
+  from *standby*.
+- **`ready`** — a real read with our credentials.
+
+The two are separate because `live && !ready` is a distinct, important state: a
+re-initialised OpenBao hands back valid-looking tokens that grant nothing.
+Collapsing them into one boolean is what let that go unnoticed for four days.
+
+`mcpctl status` renders the probe directly:
+
+```
+Secrets:    bao-k8s* ✓ reachable, default ✓ reachable
+Secrets:    bao-k8s* ⚠ degraded — serving 7 cached secret(s)
+Secrets:    bao-k8s* ✗ unreachable: sealed
+Secrets:    bao-k8s* ✗ auth failed: HTTP 403 permission denied
+Secrets:    bao-k8s* ? unknown
+```
+
+> **Historical note.** This verdict used to come solely from
+> `tokenMeta.lastRotationError`, which only the rotator writes — and the rotator
+> skips `auth: kubernetes` backends. The Secrets line was therefore *incapable*
+> of going red for a k8s-auth backend, and reported OpenBao healthy while it was
+> unreachable. `?` (probe failed) renders yellow, never green: not knowing is not
+> health.
 
 ## Migration — `mcpctl migrate secrets`
 

docs/web-search.md

@@ -0,0 +1,148 @@
+# Web search and documentation lookup
+
+Three templates. All self-hosted, none needs an API key or a vendor account, and
+all three are deployed by mcpctl like any other server.
+
+| Template | Package / image | Needs |
+|---|---|---|
+| `duckduckgo` | `duckduckgo-mcp-server` (python) | nothing |
+| `searxng` | `mcp-searxng` (node) | a SearXNG instance |
+| `docs-mcp` | `ghcr.io/arabold/docs-mcp-server` | nothing |
+
+Search and docs are different jobs, not competing options. A search engine will
+hand you a 2023 blog post with a stale method signature; a docs index cannot
+tell you why a daemon is crash-looping. Attach both to a project that does real
+engineering work.
+
+`docs-mcp` is the open-source replacement for Context7 / Ref.tools — same job,
+but the index lives on your infrastructure and can include private repos.
+
+## Start here: `duckduckgo`
+
+The only one with no infrastructure behind it. It scrapes DuckDuckGo's HTML
+endpoint directly, so there is no engine to run and no key to hold.
+
+```bash
+mcpctl create server websearch --from-template duckduckgo --env DDG_SAFE_SEARCH=OFF
+mcpctl get instances | grep websearch     # RUNNING / healthy within ~a minute
+```
+
+Tools: `search` (`query`, `max_results`, `region`) and `fetch_content` (`url`,
+`start_index`, `max_length`) for pulling a result as markdown.
+
+The tradeoff is honest: scraping has no SLA. DuckDuckGo can change its markup or
+rate-limit you, and the server caps itself at 30 searches/min. Set
+`DDG_SEARCH_BACKEND=curl` if bot checks start biting. When it becomes a problem,
+move to `searxng`.
+
+## Better results: `searxng`
+
+SearXNG aggregates ~25 engines and gives you real filtering, which is what keeps
+search results from flooding a context window:
+
+- `min_score` (0.0–1.0) — relevance floor. The most useful knob; start at `0.3`.
+- `time_range` — `day` / `week` / `month` / `year`. Essential for "did this break
+  in the last release" questions.
+- `language`, `safesearch`, `pageno`.
+
+`web_url_read` then takes `section`, `paragraphRange`, `startChar`/`maxLength`
+and `readHeadings`, so you pull one section of a long page rather than all of it.
+Bounded PDF extraction is included, which covers most vendor documentation.
+
+The cost is that SearXNG is a service you have to run. It is plain
+infrastructure, not an MCP server, so mcpctl has nothing to manage it with — it
+belongs in your cluster or in the compose stack:
+
+```bash
+cd stack
+cp .env.example .env          # set SEARXNG_SECRET
+docker compose --profile websearch up -d
+```
+
+Then point the MCP server at it:
+
+```bash
+mcpctl create secret searxng-conf --data SEARXNG_URL=http://mcpctl-searxng:8080
+mcpctl create server searxng --from-template searxng --env-from-secret searxng-conf
+```
+
+### The one SearXNG gotcha
+
+SearXNG ships with `search.formats: [html]`. Every `format=json` request against
+a stock instance returns **403**, and `mcp-searxng` comes back empty with no
+useful error. `stack/searxng/settings.yml` exists only to add `json` to that
+list — it is the one setting with no environment-variable override.
+
+This is also why pointing the template at a *public* SearXNG instance usually
+fails: nearly all of them leave the JSON API off.
+
+The compose healthcheck probes `format=json` specifically, so a misconfigured
+instance shows up as unhealthy rather than as silently empty search results.
+
+## Documentation: `docs-mcp`
+
+```bash
+mcpctl create server docs --from-template docs-mcp
+```
+
+Tools: `scrape_docs` (index a site, GitHub repo, npm or PyPI package, or local
+files), `search_docs` (query, optionally pinned to a version), and `fetch_url`.
+
+Index the things you actually run — Pulumi, the Kubernetes API, Grafana, the
+Terraform provider docs — rather than everything.
+
+### Persistence
+
+The scraped index is a SQLite file (`better-sqlite3` + `sqlite-vec`) under
+`/data`. There is **no external-database mode** — no `DATABASE_URL`, no
+pgvector — so a Postgres cluster cannot help here. The template declares a
+volume instead:
+
+```yaml
+volumes:
+  - name: data
+    mountPath: /data
+    sizeGb: 20
+```
+
+The backing PVC is named after the *server* (`mcpctl-docs-data`), not the
+instance, so editing the server or restarting the pod re-attaches to the same
+index rather than starting empty. See "Persistent volumes" in the README.
+
+The readiness probe is `list_libraries`: no arguments, read-only, and it reads
+the SQLite store, so a pass proves the volume is mounted and readable rather
+than merely that the process started. It answers "No libraries indexed yet." on
+a fresh instance instead of erroring, which is why `search_docs` (which needs a
+library that only exists after a scrape) cannot serve as the probe.
+
+### Embeddings
+
+Keyword search works out of the box. Semantic search is noticeably better and
+needs an embedding model. To keep it local, point the server at Ollama:
+
+```bash
+ollama pull nomic-embed-text
+mcpctl create server docs --from-template docs-mcp \
+  --env DOCS_MCP_EMBEDDING_MODEL=openai:nomic-embed-text \
+  --env OPENAI_API_BASE=http://ollama:11434/v1 \
+  --env OPENAI_API_KEY=ollama
+```
+
+Ollama's OpenAI-compatible endpoint is why the provider prefix is `openai:` —
+the key is a placeholder and never leaves the network.
+
+Changing the embedding model invalidates the index: embeddings from different
+models are not comparable, so everything has to be re-scraped.
+
+## Alternatives considered
+
+- **Tavily / Brave / Exa / Perplexity** — better formatted results, all require
+  an API key and send every query to a vendor.
+- **Context7** — the popular docs MCP, but a cloud index of public-docs snippets
+  only. `docs-mcp` covers the same ground locally and takes private sources.
+- **agent-search** — bundles SearXNG and a 10-strategy extraction cascade in one
+  deploy. Attractive, but at ~67 stars it is too young to build the stack on.
+  Worth revisiting.
+- **Writing our own** — the hard parts are extraction heuristics and the
+  embedding pipeline, and all three projects above already solved them under
+  MIT.