michal/mcpctl
1
0
Fork 0
Code Issues Pull Requests 7 Actions Packages Projects Releases 1 Wiki Activity

feat(secrets): report real backend health instead of a hard-coded tick

Browse Source 
`mcpctl status` derived its Secrets verdict entirely from
`tokenMeta.lastRotationError`. The rotator writes that field only for
`auth: 'token'` backends (SecretBackendRotator.isRotatable), so a
`kubernetes`-auth backend never wrote it and the line rendered a green
tick unconditionally — including with OpenBao sealed, unreachable, or
answering 403 to every read. The one signal we had was structurally
incapable of going red for the backend we actually run.

New `GET /api/v1/secretbackends/:id/health` reports two signals, kept
separate on purpose:

  live   — reachable at all?          (unauthenticated sys/health)
  ready  — can we read through it?    (uses our credentials)

live-but-not-ready is the exact shape of a re-initialised OpenBao handing
back valid-looking tokens that grant nothing; collapsing both into one
boolean is what hid that for four days. Needs no RBAC mapping — it falls
through to the generic `secretbackends` resource, so a GET is
`view:secretbackends`.

`mcpctl status` now renders four states — reachable / degraded (serving N
cached secrets) / unreachable / auth failed — with rotation error demoted
to a trailing clause rather than the verdict. A failed probe renders
"? unknown", never green: not knowing is not health. JSON output carries
the same probe, so scripts stop being told every k8s-auth backend is fine.

Also adds a boot-time cache warm. The stale-while-error cache can only
absorb an outage for secrets it has already seen, so a cold mcpd during a
backend outage still fails; resolving each running server's refs once at
startup closes that for the common case. Best-effort and deliberately
partial — if the backend is also down at boot this is a no-op and
instances fail loudly, which is correct. Persisting last-known-good to
Postgres or disk would just be plaintext-at-rest again.

Tests: 15 new. The five status assertions were confirmed to fail against
the old rotation-only logic before being kept.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018vybEitX4FykeMatKe5Xki
This commit is contained in:
Michal
2026-08-20 22:14:12 +01:00
parent bd3e1134c9
commit 0fbfc72d68
7 changed files with 488 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
src/cli/tests/commands/status.test.ts
View File

@@ -30,6 +30,7 @@ function baseDeps(overrides?: Partial<StatusCommandDeps>): Partial<StatusCommand
fetchServerLlms: async () => null,
probeServerLlm: async () => ({ ok: true, ms: 12, say: 'hi' }),
fetchSecretBackends: async () => null,
probeSecretBackend: async () => ({ live: true, ready: true, cache: { entries: 0, servingStale: 0 } }),
isTTY: false,
...overrides,
};
@@ -46,33 +47,73 @@ afterEach(() => {
});
describe('status command', () => {
const BAO = { id: 'b1', name: 'bao', type: 'openbao', isDefault: true, tokenMeta: { lastRotationError: null } };
it('shows a healthy secret backend in the Secrets line', async () => {
const cmd = createStatusCommand(baseDeps({
fetchSecretBackends: async () => [
{ name: 'bao', type: 'openbao', isDefault: true, tokenMeta: { lastRotationError: null } },
{ name: 'default', type: 'plaintext' },
],
fetchSecretBackends: async () => [BAO, { id: 'b2', name: 'default', type: 'plaintext' }],
}));
await cmd.parseAsync([], { from: 'user' });
const out = output.join('\n');
expect(out).toContain('Secrets:');
expect(out).toContain('bao* ✓');
expect(out).toContain('default ✓');
expect(out).toContain('bao* ✓ reachable');
expect(out).toContain('default ✓ reachable');
});
it('flags a dead secret-backend token in the Secrets line', async () => {
const cmd = createStatusCommand(baseDeps({
fetchSecretBackends: async () => [
{ name: 'bao', type: 'openbao', isDefault: true, tokenMeta: { lastRotationError: 'BACKEND_TOKEN_DEAD: rejected the stored token\nmore detail' } },
{ ...BAO, tokenMeta: { lastRotationError: 'BACKEND_TOKEN_DEAD: rejected the stored token\nmore detail' } },
],
}));
await cmd.parseAsync([], { from: 'user' });
const out = output.join('\n');
expect(out).toContain('bao* ✗');
expect(out).toContain('BACKEND_TOKEN_DEAD');
expect(out).not.toContain('more detail'); // only first line, truncated
});
it('reports an unreachable backend even when rotation never errored', async () => {
// THE bug: a kubernetes-auth backend never writes tokenMeta.lastRotationError,
// so this line used to render a green tick with OpenBao completely down.
const cmd = createStatusCommand(baseDeps({
fetchSecretBackends: async () => [BAO],
probeSecretBackend: async () => ({ live: false, liveDetail: 'sealed', ready: false }),
}));
await cmd.parseAsync([], { from: 'user' });
const out = output.join('\n');
expect(out).toContain('bao* ✗ unreachable: sealed');
expect(out).not.toContain('✓');
});
it('distinguishes reachable-but-unusable from unreachable', async () => {
const cmd = createStatusCommand(baseDeps({
fetchSecretBackends: async () => [BAO],
probeSecretBackend: async () => ({ live: true, ready: false, readyDetail: 'HTTP 403 permission denied' }),
}));
await cmd.parseAsync([], { from: 'user' });
expect(output.join('\n')).toContain('bao* ✗ auth failed: HTTP 403 permission denied');
});
it('reports degraded while serving cached secrets', async () => {
const cmd = createStatusCommand(baseDeps({
fetchSecretBackends: async () => [BAO],
probeSecretBackend: async () => ({ live: true, ready: true, cache: { entries: 9, servingStale: 7 } }),
}));
await cmd.parseAsync([], { from: 'user' });
expect(output.join('\n')).toContain('bao* ⚠ degraded — serving 7 cached secret(s)');
});
it('reports unknown — never healthy — when the probe itself fails', async () => {
const cmd = createStatusCommand(baseDeps({
fetchSecretBackends: async () => [BAO],
probeSecretBackend: async () => null,
}));
await cmd.parseAsync([], { from: 'user' });
const out = output.join('\n');
expect(out).toContain('bao* ? unknown');
expect(out).not.toContain('✓');
});
it('omits the Secrets line when mcpd returns no backends', async () => {
const cmd = createStatusCommand(baseDeps({ fetchSecretBackends: async () => null }));
await cmd.parseAsync([], { from: 'user' });