Files
mcpctl/src/mcpd/tests/secret-backends.test.ts

245 lines
11 KiB
TypeScript
Raw Normal View History

feat(mcpd): pluggable SecretBackend abstraction + OpenBao driver + migrate Why: API keys live in Postgres as plaintext JSON. A DB read exposes every credential in the system. Before centralising more secrets (LLM keys, etc.) we want to be able to point at an external KV store and drop DB access to sensitive rows. New model: - `SecretBackend` resource (CRUD + isDefault invariant) owns how a secret is stored. `Secret` gains `backendId` FK and `externalRef`. Reads/writes dispatch through a driver. - `plaintext` driver (near-noop, uses existing Secret.data column) is seeded as the `default` row at startup. Acts as trust root / bootstrap. - `openbao` driver (also HashiCorp Vault KV v2 compatible) talks plain HTTP, no SDK dependency. Auth via static token pulled from a plaintext-backed `Secret` through the injected SecretRefResolver. Caches resolved token. - `SecretMigrateService` moves secrets one-at-a-time: read → write dest → flip row → best-effort source delete. Interrupted runs are idempotent (skips secrets already on destination). CLI surface: - `mcpctl create|get|describe|delete secretbackend` + `--default` on create. - `mcpctl migrate secrets --from X --to Y [--names a,b] [--keep-source] [--dry-run]` - `apply -f` round-trips secretbackends (yaml/json multi-doc + grouped). - RBAC: `secretbackends` resource + `run:migrate-secrets` operation. - Fish + bash completions regenerated. docs/secret-backends.md covers the OpenBao policy, chicken-and-egg auth flow, and the migration semantics. Broke the circular dep (OpenBao needs SecretService to resolve its own token, SecretService needs SecretBackendService) with a deferred-resolver bridge in mcpd startup. 11 new driver unit tests; existing env-resolver/secret-route/ backup tests updated for the new service signatures. Full suite: 1792/1792. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 19:29:55 +01:00
import { describe, it, expect, vi } from 'vitest';
import { PlaintextDriver } from '../src/services/secret-backends/plaintext.js';
import { OpenBaoDriver } from '../src/services/secret-backends/openbao.js';
describe('PlaintextDriver', () => {
const driver = new PlaintextDriver({ listAllPlaintext: async () => [{ name: 'a', data: { k: 'v' } }] });
it('read returns the data passed in', async () => {
const result = await driver.read({ name: 's', externalRef: '', data: { token: 'abc' } });
expect(result).toEqual({ token: 'abc' });
});
it('write returns storedData = input, externalRef = empty', async () => {
const result = await driver.write({ name: 's', data: { k: 'v' } });
expect(result).toEqual({ externalRef: '', storedData: { k: 'v' } });
});
it('list delegates to the injected dep', async () => {
const list = await driver.list();
expect(list).toEqual([{ name: 'a', externalRef: '' }]);
});
it('delete is a no-op', async () => {
await expect(driver.delete({ name: 's', externalRef: '' })).resolves.toBeUndefined();
});
});
describe('OpenBaoDriver', () => {
function makeFetch(responses: Array<{ url: RegExp; status: number; body?: unknown }>): ReturnType<typeof vi.fn> {
return vi.fn(async (url: string | URL, _init?: RequestInit) => {
const urlStr = String(url);
const match = responses.find((r) => r.url.test(urlStr));
if (!match) throw new Error(`unexpected fetch: ${urlStr}`);
return new Response(match.body ? JSON.stringify(match.body) : '', { status: match.status });
});
}
const resolver = { resolve: vi.fn(async () => 'test-vault-token') };
it('write sends POST to .../data/<path> with {data: ...}', async () => {
const fetchFn = makeFetch([{ url: /\/v1\/secret\/data\/mcpctl\/mytoken$/, status: 200 }]);
const driver = new OpenBaoDriver(
{ url: 'http://bao.example:8200', tokenSecretRef: { name: 'bao', key: 'token' } },
{ fetch: fetchFn as unknown as typeof fetch, secretRefResolver: resolver },
);
const result = await driver.write({ name: 'mytoken', data: { api_key: 'secret-xyz' } });
expect(result.externalRef).toBe('secret/mcpctl/mytoken');
expect(result.storedData).toEqual({});
expect(fetchFn).toHaveBeenCalledTimes(1);
const [, init] = fetchFn.mock.calls[0] as [unknown, RequestInit];
expect(init.method).toBe('POST');
expect(JSON.parse(init.body as string)).toEqual({ data: { api_key: 'secret-xyz' } });
const headers = init.headers as Record<string, string>;
expect(headers['X-Vault-Token']).toBe('test-vault-token');
});
it('read returns body.data.data', async () => {
const fetchFn = makeFetch([{
url: /\/v1\/secret\/data\/mcpctl\/mytoken$/,
status: 200,
body: { data: { data: { api_key: 'secret-xyz' } } },
}]);
const driver = new OpenBaoDriver(
{ url: 'http://bao.example:8200', tokenSecretRef: { name: 'bao', key: 'token' } },
{ fetch: fetchFn as unknown as typeof fetch, secretRefResolver: resolver },
);
const result = await driver.read({ name: 'mytoken', externalRef: 'secret/mcpctl/mytoken', data: {} });
expect(result).toEqual({ api_key: 'secret-xyz' });
});
it('read throws when the path 404s', async () => {
const fetchFn = makeFetch([{ url: /\/data\//, status: 404 }]);
const driver = new OpenBaoDriver(
{ url: 'http://bao.example:8200', tokenSecretRef: { name: 'bao', key: 'token' } },
{ fetch: fetchFn as unknown as typeof fetch, secretRefResolver: resolver },
);
await expect(driver.read({ name: 'missing', externalRef: '', data: {} })).rejects.toThrow(/not found/);
});
it('delete swallows 404', async () => {
const fetchFn = makeFetch([{ url: /\/metadata\//, status: 404 }]);
const driver = new OpenBaoDriver(
{ url: 'http://bao.example:8200', tokenSecretRef: { name: 'bao', key: 'token' } },
{ fetch: fetchFn as unknown as typeof fetch, secretRefResolver: resolver },
);
await expect(driver.delete({ name: 'gone', externalRef: '' })).resolves.toBeUndefined();
});
it('list returns names from the metadata LIST call', async () => {
const fetchFn = makeFetch([{
url: /\/v1\/secret\/metadata\/mcpctl\/$/,
status: 200,
body: { data: { keys: ['token1', 'token2', 'sub-folder/'] } },
}]);
const driver = new OpenBaoDriver(
{ url: 'http://bao.example:8200', tokenSecretRef: { name: 'bao', key: 'token' } },
{ fetch: fetchFn as unknown as typeof fetch, secretRefResolver: resolver },
);
const result = await driver.list();
// Sub-folders (trailing slash) are excluded; only leaf keys are returned.
expect(result).toEqual([
{ name: 'token1', externalRef: 'secret/mcpctl/token1' },
{ name: 'token2', externalRef: 'secret/mcpctl/token2' },
]);
});
it('caches the vault token after first resolve', async () => {
const fetchFn = makeFetch([
{ url: /\/v1\/secret\/data\/mcpctl\//, status: 200, body: { data: { data: { k: 'v' } } } },
]);
const singleResolver = { resolve: vi.fn(async () => 'test-vault-token') };
const driver = new OpenBaoDriver(
{ url: 'http://bao.example:8200', tokenSecretRef: { name: 'bao', key: 'token' } },
{ fetch: fetchFn as unknown as typeof fetch, secretRefResolver: singleResolver },
);
await driver.read({ name: 'a', externalRef: '', data: {} });
await driver.read({ name: 'a', externalRef: '', data: {} });
expect(singleResolver.resolve).toHaveBeenCalledTimes(1);
});
it('propagates X-Vault-Namespace when configured', async () => {
const fetchFn = makeFetch([{ url: /\/v1\/secret\/data\/mcpctl\//, status: 200 }]);
const driver = new OpenBaoDriver(
{ url: 'http://bao.example:8200', namespace: 'myteam', tokenSecretRef: { name: 'bao', key: 'token' } },
{ fetch: fetchFn as unknown as typeof fetch, secretRefResolver: resolver },
);
await driver.write({ name: 'x', data: { k: 'v' } });
const [, init] = fetchFn.mock.calls[0] as [unknown, RequestInit];
const headers = init.headers as Record<string, string>;
expect(headers['X-Vault-Namespace']).toBe('myteam');
});
feat(openbao): kubernetes ServiceAccount auth — no static token in DB Why: requiring a static OpenBao root token to live (even once-bootstrap) on the plaintext backend is the weakest link in the chain. With the bao-side Kubernetes auth method enabled, mcpd's pod can authenticate using its own projected SA token, exchange it for a short-lived Vault client token, and keep the database free of any vault credentials at all. Driver changes (src/mcpd/src/services/secret-backends/openbao.ts): - New `OpenBaoConfig.auth = 'token' | 'kubernetes'`. Defaults to 'token' so existing rows keep working. Both shapes share url + mount + pathPrefix + namespace; auth-specific fields are mutually exclusive in the config schema. - Kubernetes auth flow: read JWT from /var/run/secrets/.../token, POST to /v1/auth/<authMount>/login {role, jwt}, cache the returned client_token for `lease_duration - 60s` (grace window), then re-login. - One-shot 403-retry: if a request comes back 403 (revoked / clock skew), purge cache and retry the original request once with a fresh login. - Reads + writes go through the same getToken() path so token-auth is unchanged for existing deployments. CLI (src/cli/src/commands/create.ts): - `mcpctl create secretbackend bao --type openbao --auth kubernetes \ --url https://bao.example:8200 --role mcpctl` - Optional `--auth-mount` (default 'kubernetes') + `--sa-token-path` (default the standard projected-token path) for non-default deployments. - Token-auth path unchanged: `--auth token --token-secret SECRET/KEY` (or omit `--auth` since 'token' is the default). Validation (factory.ts) gates on the auth strategy: each path enforces its own required fields and produces a clear error if misconfigured. Tests: 6 new k8s-auth unit cases (login wire shape, lease-based caching, custom authMount, 403-on-login, missing-role rejection, missing-tokenSecretRef rejection). Full suite 1859/1859. Completions regenerated for the new flags. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-19 23:23:05 +01:00
describe('kubernetes auth', () => {
it('exchanges the SA JWT for a vault client token via /v1/auth/kubernetes/login', async () => {
const calls: Array<{ url: string; init: RequestInit }> = [];
const fetchFn = vi.fn(async (url: string | URL, init?: RequestInit) => {
const u = String(url);
calls.push({ url: u, init: init ?? {} });
if (u.endsWith('/v1/auth/kubernetes/login')) {
return new Response(JSON.stringify({
auth: { client_token: 'vault.client.token.xyz', lease_duration: 3600 },
}), { status: 200 });
}
return new Response(JSON.stringify({}), { status: 200 });
});
const driver = new OpenBaoDriver(
{ url: 'http://bao.example:8200', auth: 'kubernetes', role: 'mcpctl' },
{
fetch: fetchFn as unknown as typeof fetch,
readServiceAccountToken: async () => 'eyJ.fake.sa.jwt',
},
);
await driver.write({ name: 'x', data: { k: 'v' } });
// Two calls: login + write
expect(calls).toHaveLength(2);
expect(calls[0]!.url).toBe('http://bao.example:8200/v1/auth/kubernetes/login');
expect(JSON.parse(calls[0]!.init.body as string)).toEqual({ role: 'mcpctl', jwt: 'eyJ.fake.sa.jwt' });
// Write uses the returned client token
const writeHeaders = calls[1]!.init.headers as Record<string, string>;
expect(writeHeaders['X-Vault-Token']).toBe('vault.client.token.xyz');
});
it('caches the vault token across requests and renews after lease expiry', async () => {
let nowMs = 1_000_000_000_000;
let loginCount = 0;
const fetchFn = vi.fn(async (url: string | URL) => {
const u = String(url);
if (u.endsWith('/v1/auth/kubernetes/login')) {
loginCount++;
// 600s lease leaves 540s of cached window after the 60s grace.
return new Response(JSON.stringify({
auth: { client_token: `tok-${String(loginCount)}`, lease_duration: 600 },
}), { status: 200 });
}
return new Response(JSON.stringify({}), { status: 200 });
});
const driver = new OpenBaoDriver(
{ url: 'http://bao.example:8200', auth: 'kubernetes', role: 'mcpctl' },
{
fetch: fetchFn as unknown as typeof fetch,
readServiceAccountToken: async () => 'jwt',
now: () => nowMs,
},
);
await driver.write({ name: 'a', data: { k: 'v' } });
await driver.write({ name: 'b', data: { k: 'v' } });
expect(loginCount).toBe(1); // both writes share the cached token
// Advance past lease - grace window → driver re-logs in
nowMs += 600_000;
await driver.write({ name: 'c', data: { k: 'v' } });
expect(loginCount).toBe(2);
});
it('honours custom authMount path', async () => {
const calls: string[] = [];
const fetchFn = vi.fn(async (url: string | URL) => {
calls.push(String(url));
if (String(url).includes('/login')) {
return new Response(JSON.stringify({ auth: { client_token: 't', lease_duration: 3600 } }), { status: 200 });
}
return new Response(JSON.stringify({}), { status: 200 });
});
const driver = new OpenBaoDriver(
{ url: 'http://bao.example:8200', auth: 'kubernetes', role: 'mcpctl', authMount: 'kubernetes/cluster-a' },
{ fetch: fetchFn as unknown as typeof fetch, readServiceAccountToken: async () => 'jwt' },
);
await driver.write({ name: 'x', data: {} });
expect(calls[0]).toBe('http://bao.example:8200/v1/auth/kubernetes/cluster-a/login');
});
it('throws on login failure with a clear error', async () => {
const fetchFn = vi.fn(async () => new Response('permission denied', { status: 403 }));
const driver = new OpenBaoDriver(
{ url: 'http://bao.example:8200', auth: 'kubernetes', role: 'mcpctl' },
{ fetch: fetchFn as unknown as typeof fetch, readServiceAccountToken: async () => 'jwt' },
);
await expect(driver.read({ name: 'x', externalRef: '', data: {} }))
.rejects.toThrow(/kubernetes login.*role=mcpctl.*HTTP 403/);
});
it('rejects construction when role is missing', () => {
expect(() => new OpenBaoDriver(
// eslint-disable-next-line @typescript-eslint/no-explicit-any
{ url: 'http://bao.example:8200', auth: 'kubernetes' } as any,
{ fetch: vi.fn() as unknown as typeof fetch, readServiceAccountToken: async () => 'jwt' },
)).toThrow(/role.*required/);
});
it('rejects token-auth construction when tokenSecretRef is missing', () => {
expect(() => new OpenBaoDriver(
// eslint-disable-next-line @typescript-eslint/no-explicit-any
{ url: 'http://bao.example:8200' } as any,
{ fetch: vi.fn() as unknown as typeof fetch, secretRefResolver: resolver },
)).toThrow(/tokenSecretRef.*required/);
});
});
feat(mcpd): pluggable SecretBackend abstraction + OpenBao driver + migrate Why: API keys live in Postgres as plaintext JSON. A DB read exposes every credential in the system. Before centralising more secrets (LLM keys, etc.) we want to be able to point at an external KV store and drop DB access to sensitive rows. New model: - `SecretBackend` resource (CRUD + isDefault invariant) owns how a secret is stored. `Secret` gains `backendId` FK and `externalRef`. Reads/writes dispatch through a driver. - `plaintext` driver (near-noop, uses existing Secret.data column) is seeded as the `default` row at startup. Acts as trust root / bootstrap. - `openbao` driver (also HashiCorp Vault KV v2 compatible) talks plain HTTP, no SDK dependency. Auth via static token pulled from a plaintext-backed `Secret` through the injected SecretRefResolver. Caches resolved token. - `SecretMigrateService` moves secrets one-at-a-time: read → write dest → flip row → best-effort source delete. Interrupted runs are idempotent (skips secrets already on destination). CLI surface: - `mcpctl create|get|describe|delete secretbackend` + `--default` on create. - `mcpctl migrate secrets --from X --to Y [--names a,b] [--keep-source] [--dry-run]` - `apply -f` round-trips secretbackends (yaml/json multi-doc + grouped). - RBAC: `secretbackends` resource + `run:migrate-secrets` operation. - Fish + bash completions regenerated. docs/secret-backends.md covers the OpenBao policy, chicken-and-egg auth flow, and the migration semantics. Broke the circular dep (OpenBao needs SecretService to resolve its own token, SecretService needs SecretBackendService) with a deferred-resolver bridge in mcpd startup. 11 new driver unit tests; existing env-resolver/secret-route/ backup tests updated for the new service signatures. Full suite: 1792/1792. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 19:29:55 +01:00
});