All checks were successful
CI/CD / typecheck (pull_request) Successful in 51s
CI/CD / lint (pull_request) Successful in 1m47s
CI/CD / test (pull_request) Successful in 1m3s
CI/CD / smoke (pull_request) Successful in 4m34s
CI/CD / build (pull_request) Successful in 3m50s
CI/CD / publish (pull_request) Has been skipped
Why: API keys live in Postgres as plaintext JSON. A DB read exposes every credential in the system. Before centralising more secrets (LLM keys, etc.) we want to be able to point at an external KV store and drop DB access to sensitive rows. New model: - `SecretBackend` resource (CRUD + isDefault invariant) owns how a secret is stored. `Secret` gains `backendId` FK and `externalRef`. Reads/writes dispatch through a driver. - `plaintext` driver (near-noop, uses existing Secret.data column) is seeded as the `default` row at startup. Acts as trust root / bootstrap. - `openbao` driver (also HashiCorp Vault KV v2 compatible) talks plain HTTP, no SDK dependency. Auth via static token pulled from a plaintext-backed `Secret` through the injected SecretRefResolver. Caches resolved token. - `SecretMigrateService` moves secrets one-at-a-time: read → write dest → flip row → best-effort source delete. Interrupted runs are idempotent (skips secrets already on destination). CLI surface: - `mcpctl create|get|describe|delete secretbackend` + `--default` on create. - `mcpctl migrate secrets --from X --to Y [--names a,b] [--keep-source] [--dry-run]` - `apply -f` round-trips secretbackends (yaml/json multi-doc + grouped). - RBAC: `secretbackends` resource + `run:migrate-secrets` operation. - Fish + bash completions regenerated. docs/secret-backends.md covers the OpenBao policy, chicken-and-egg auth flow, and the migration semantics. Broke the circular dep (OpenBao needs SecretService to resolve its own token, SecretService needs SecretBackendService) with a deferred-resolver bridge in mcpd startup. 11 new driver unit tests; existing env-resolver/secret-route/ backup tests updated for the new service signatures. Full suite: 1792/1792. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
168 lines
6.1 KiB
Markdown
168 lines
6.1 KiB
Markdown
# Secret backends
|
|
|
|
`mcpctl` stores the raw data for `Secret` resources in a pluggable **backend**.
|
|
The default is `plaintext` — the secret payload lives in Postgres as plain JSON
|
|
— which is fine for laptop development but a poor fit for shared clusters. For
|
|
production, point at an external KV store and delete secrets from the DB after
|
|
migration.
|
|
|
|
This guide covers the model, the shipped drivers, and how to migrate without
|
|
downtime.
|
|
|
|
## Model
|
|
|
|
- A `SecretBackend` resource is a single named driver instance (e.g. a pointer
|
|
at one OpenBao deployment).
|
|
- Every `Secret` row carries a `backendId` FK — the backend that owns its data.
|
|
- Exactly one `SecretBackend` has `isDefault: true`. New secrets created through
|
|
the API/CLI land on that backend.
|
|
- The `plaintext` backend is seeded at startup and named `default`. It cannot
|
|
be deleted — there needs to always be one row where the driver's own
|
|
credentials can bootstrap from (see below).
|
|
|
|
## CLI
|
|
|
|
```bash
|
|
mcpctl get secretbackends # list backends
|
|
mcpctl describe secretbackend <name> # inspect config (credentials masked)
|
|
mcpctl create secretbackend <name> --type plaintext [--default] [--description ...]
|
|
mcpctl create secretbackend <name> --type openbao \
|
|
--url http://bao.example:8200 \
|
|
--token-secret bao-creds/token \
|
|
[--namespace <ns>] [--mount secret] [--path-prefix mcpctl] \
|
|
[--default]
|
|
mcpctl delete secretbackend <name> # blocked if any secret still points at it
|
|
|
|
mcpctl migrate secrets --from default --to bao
|
|
mcpctl migrate secrets --from default --to bao --names a,b --keep-source
|
|
mcpctl migrate secrets --from default --to bao --dry-run
|
|
```
|
|
|
|
Anything you can do with `create secretbackend` also works via `apply -f`:
|
|
|
|
```yaml
|
|
kind: secretbackend
|
|
name: bao
|
|
type: openbao
|
|
description: "shared cluster OpenBao"
|
|
isDefault: true
|
|
config:
|
|
url: http://bao.svc.cluster.local:8200
|
|
tokenSecretRef: { name: bao-creds, key: token }
|
|
namespace: platform
|
|
```
|
|
|
|
## Drivers
|
|
|
|
### plaintext
|
|
|
|
Trivial. `Secret.data` holds the JSON, `externalRef` is empty.
|
|
|
|
- Storage: Postgres column.
|
|
- Bootstrap: seeded as `default` at startup.
|
|
- Cost: zero setup, zero encryption at rest, full access for any DB reader.
|
|
|
|
Use for development, CI, or single-tenant self-hosts where the DB itself is
|
|
treated as sensitive.
|
|
|
|
### openbao
|
|
|
|
Talks HTTP to an [OpenBao](https://openbao.org) (MPL 2.0 Vault fork) KV v2
|
|
mount. Also compatible with HashiCorp Vault KV v2 — the wire protocol is the
|
|
same.
|
|
|
|
| Config key | Required? | Description |
|
|
|------------------|-----------|-------------|
|
|
| `url` | yes | Base URL, e.g. `http://bao.svc.cluster.local:8200`. |
|
|
| `tokenSecretRef` | yes | `{ name, key }` pointing at a `Secret` on the **plaintext** backend that holds the bootstrap token. |
|
|
| `mount` | no | KV v2 mount name. Default `secret`. |
|
|
| `pathPrefix` | no | Path prefix under the mount. Default `mcpctl`. Secrets land at `<mount>/<pathPrefix>/<secretName>`. |
|
|
| `namespace` | no | `X-Vault-Namespace` header for OpenBao/Vault Enterprise namespaces. |
|
|
|
|
The driver only stores a reference in `Secret.externalRef` (`mount/path`). The
|
|
`Secret.data` column is left empty for openbao-backed rows — you can safely
|
|
drop DB-level access to secrets after migration.
|
|
|
|
#### Required OpenBao policy
|
|
|
|
Minimum token policy for a backend that lives at `secret/mcpctl/`:
|
|
|
|
```hcl
|
|
path "secret/data/mcpctl/*" {
|
|
capabilities = ["create", "read", "update"]
|
|
}
|
|
|
|
path "secret/metadata/mcpctl/*" {
|
|
capabilities = ["list", "delete"]
|
|
}
|
|
|
|
path "secret/metadata/mcpctl/" {
|
|
capabilities = ["list"]
|
|
}
|
|
```
|
|
|
|
Grant `delete` on `metadata/...` only if you need mcpctl to fully remove
|
|
secrets — OpenBao soft-deletes until the metadata is gone.
|
|
|
|
#### Chicken-and-egg: where does the OpenBao token live?
|
|
|
|
mcpd reads the OpenBao token from a `Secret` on the **plaintext** backend.
|
|
That's the whole point of keeping plaintext around — it's the trust root:
|
|
|
|
1. Operator creates a plaintext `Secret` holding the bootstrap token.
|
|
2. Operator creates the `openbao` backend, pointing at that secret via
|
|
`tokenSecretRef`.
|
|
3. Operator runs `mcpctl migrate secrets --from default --to bao` to move all
|
|
other secrets off plaintext.
|
|
4. After migration, the only sensitive row left on plaintext is the OpenBao
|
|
token itself. DB access is now equivalent to OpenBao token access (a single
|
|
key), not equivalent to all API keys in the system.
|
|
|
|
Follow-up work (not shipped yet) replaces static token auth with Kubernetes
|
|
ServiceAccount auth so no bootstrap token is needed at all.
|
|
|
|
## Migration — `mcpctl migrate secrets`
|
|
|
|
Atomicity is **per secret**, not per batch. Remote writes can't roll back, so we
|
|
don't pretend. For each secret the service:
|
|
|
|
1. Reads the plaintext from the source driver.
|
|
2. Writes it to the destination driver.
|
|
3. Updates the `Secret` row: flips `backendId`, sets new `externalRef`, clears
|
|
`data`.
|
|
4. Deletes from source (skipped with `--keep-source`).
|
|
|
|
If the command is interrupted between step 2 and 3, the destination has an
|
|
orphan entry but the source still owns the row. Re-running is idempotent — the
|
|
service skips secrets that are already on the destination and picks up the
|
|
rest.
|
|
|
|
```bash
|
|
# Dry-run first: see what would move.
|
|
mcpctl migrate secrets --from default --to bao --dry-run
|
|
|
|
# Migrate everything.
|
|
mcpctl migrate secrets --from default --to bao
|
|
|
|
# Migrate a subset only.
|
|
mcpctl migrate secrets --from default --to bao --names api-keys,oauth-client
|
|
|
|
# Leave the source copy in place (useful for A/B validation).
|
|
mcpctl migrate secrets --from default --to bao --keep-source
|
|
```
|
|
|
|
The command prints a per-secret summary (migrated / skipped / failed) and exits
|
|
non-zero if any secret failed. Ctrl-C during the run is safe — restart when you
|
|
want, no duplicate writes.
|
|
|
|
## RBAC
|
|
|
|
- `resource: secretbackends` — gated like any other resource (`view`,
|
|
`create`, `edit`, `delete`).
|
|
- `role: run, action: migrate-secrets` — required to call
|
|
`POST /api/v1/secrets/migrate`.
|
|
|
|
Describe output masks config values whose keys look like credentials
|
|
(`token`, `secret`, `password`, `key`), so `mcpctl describe secretbackend` is
|
|
safe to paste into tickets.
|