michal/mcpctl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(llm): probe upstream auth at registration time

Browse Source 
mcpd now runs a cheap auth probe whenever an Llm is created (or its
apiKeyRef/url is updated). Catches misconfigured tokens / wrong URLs at
registration with a 422 + structured error message, instead of silently
500-ing on first chat with a generic "fetch failed". Caught in the wild
today: the homelab Pulumi config exposed `MCPCTL_GATEWAY_TOKEN` (which
is mcpctl_pat_-prefixed, intended for LiteLLM→mcplocal direction) where
LiteLLM expects `LITELLM_MASTER_KEY` (sk-prefixed). The probe makes
this immediate.

Probe shape (LlmAdapter.verifyAuth):
  - OpenAI passthrough → GET <url>/v1/models. Cheap, idempotent, gated
    by the same auth as chat/completions.
  - Anthropic → POST /v1/messages with max_tokens:1, "ping". Anthropic
    has no list-models endpoint; this is the cheapest auth-exercising
    call.
  - Returns one of:
      { ok: true }
      { ok: false, reason: "auth", status, body }    — 401/403, fail hard
      { ok: false, reason: "unreachable", error }    — network, warn-only
      { ok: false, reason: "unexpected", status, body } — non-auth 4xx, warn-only

Behavior:
  - LlmService.create()/update() runs the probe after resolveApiKey.
    Throws LlmAuthVerificationError on `auth`, logs warn for
    unreachable/unexpected, swallows for offline registration.
  - Probe is skipped when there's no apiKeyRef (nothing to verify) or
    when the caller passes skipAuthCheck=true.
  - update() probes only when apiKeyRef OR url changes — pure
    description/tier updates don't trigger upstream calls.
  - Routes catch LlmAuthVerificationError and return 422 with
    `{ error, status }`. The CLI surfaces the message verbatim via
    ApiError.

Opt-out:
  - CLI: `mcpctl create llm ... --skip-auth-check` for offline
    registration before the upstream is reachable.
  - HTTP: side-channel body field `_skipAuthCheck: true` (stripped
    before validation, never persisted on the row).

Side fix in same commit (caught while testing): src/cli/src/index.ts
read `program.opts()` BEFORE `program.parse()`, so `--direct` was a
no-op for ApiClient — every command went to mcplocal regardless. Some
commands accidentally still worked because mcplocal forwards plain
`/api/v1/*` to mcpd, but flows that need direct SSE streaming (e.g.
`mcpctl chat`) couldn't reach mcpd. Fixed by peeking at process.argv
directly for the two global flags before Commander's parse runs.

Tests:
  - llm-adapters.test.ts (+8): OpenAI 200/401/403/404/network, Anthropic
    200/401/400 (typo'd model = unexpected, NOT auth — registration
    shouldn't block on bad model names that surface at chat time).
  - llm-service.test.ts (+6): create-throws-on-auth-fail (no row
    written), warn-only on unreachable/unexpected, skipAuthCheck
    bypass, no-key skip, update-only-probes-on-auth-affecting-change.

mcpd 775/775, mcplocal 715/715, cli 430/430.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Michal
2026-04-26 16:51:55 +01:00
parent 1f0be8a5c1
commit cc225eb70f
12 changed files with 495 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

99
src/mcpd/tests/llm-adapters.test.ts
View File

@@ -208,3 +208,102 @@ describe('LlmAdapterRegistry', () => {
expect(() => reg.get('bogus')).toThrow(UnsupportedProviderError);
});
});
describe('verifyAuth — registration-time probe', () => {
it('OpenAI passthrough: 200 from /v1/models → ok', async () => {
const fetchImpl = mockFetch([
{ match: /\/v1\/models$/, status: 200, body: { data: [{ id: 'gpt-4o-mini' }] } },
]);
const adapter = new OpenAiPassthroughAdapter('openai', { fetch: fetchImpl as unknown as typeof fetch });
const result = await adapter.verifyAuth(makeCtx({ url: 'http://lite:4000', apiKey: 'sk-good' }));
expect(result).toEqual({ ok: true });
expect(fetchImpl).toHaveBeenCalledWith('http://lite:4000/v1/models', expect.objectContaining({ method: 'GET' }));
const callInit = fetchImpl.mock.calls[0][1] as RequestInit;
expect((callInit.headers as Record<string, string>)['Authorization']).toBe('Bearer sk-good');
});
it('OpenAI passthrough: 401 → reason=auth (caller throws)', async () => {
const fetchImpl = mockFetch([
{ match: /\/v1\/models$/, status: 401, text: '{"error":"invalid_api_key"}' },
]);
const adapter = new OpenAiPassthroughAdapter('openai', { fetch: fetchImpl as unknown as typeof fetch });
const result = await adapter.verifyAuth(makeCtx({ url: 'http://lite:4000', apiKey: 'sk-bad' }));
expect(result.ok).toBe(false);
if (!result.ok) {
expect(result.reason).toBe('auth');
if (result.reason === 'auth') {
expect(result.status).toBe(401);
expect(result.body).toContain('invalid_api_key');
}
}
});
it('OpenAI passthrough: 403 → reason=auth', async () => {
const fetchImpl = mockFetch([
{ match: /\/v1\/models$/, status: 403, text: 'forbidden' },
]);
const adapter = new OpenAiPassthroughAdapter('openai', { fetch: fetchImpl as unknown as typeof fetch });
const result = await adapter.verifyAuth(makeCtx({ url: 'http://lite:4000', apiKey: 'k' }));
expect(result.ok).toBe(false);
if (!result.ok) expect(result.reason).toBe('auth');
});
it('OpenAI passthrough: 404 (proxy without /v1/models) → reason=unexpected (warn-only)', async () => {
const fetchImpl = mockFetch([
{ match: /\/v1\/models$/, status: 404, text: 'not found' },
]);
const adapter = new OpenAiPassthroughAdapter('openai', { fetch: fetchImpl as unknown as typeof fetch });
const result = await adapter.verifyAuth(makeCtx({ url: 'http://lite:4000', apiKey: 'k' }));
expect(result.ok).toBe(false);
if (!result.ok) expect(result.reason).toBe('unexpected');
});
it('OpenAI passthrough: network error → reason=unreachable (warn-only)', async () => {
const fetchImpl = vi.fn(async () => { throw new Error('ECONNREFUSED 127.0.0.1:9999'); });
const adapter = new OpenAiPassthroughAdapter('openai', { fetch: fetchImpl as unknown as typeof fetch });
const result = await adapter.verifyAuth(makeCtx({ url: 'http://localhost:9999', apiKey: 'k' }));
expect(result.ok).toBe(false);
if (!result.ok) {
expect(result.reason).toBe('unreachable');
if (result.reason === 'unreachable') {
expect(result.error).toContain('ECONNREFUSED');
}
}
});
it('Anthropic: 200 from /v1/messages probe → ok', async () => {
const fetchImpl = mockFetch([
{ match: /\/v1\/messages$/, status: 200, body: { id: 'msg_x', content: [{ type: 'text', text: 'pong' }] } },
]);
const adapter = new AnthropicAdapter({ fetch: fetchImpl as unknown as typeof fetch });
const result = await adapter.verifyAuth(makeCtx({ url: 'https://api.anthropic.com', apiKey: 'sk-ant-good' }));
expect(result.ok).toBe(true);
const callInit = fetchImpl.mock.calls[0][1] as RequestInit;
expect((callInit.headers as Record<string, string>)['x-api-key']).toBe('sk-ant-good');
const reqBody = JSON.parse(callInit.body as string) as { max_tokens: number };
expect(reqBody.max_tokens).toBe(1);
});
it('Anthropic: 401 → reason=auth', async () => {
const fetchImpl = mockFetch([
{ match: /\/v1\/messages$/, status: 401, text: '{"type":"authentication_error"}' },
]);
const adapter = new AnthropicAdapter({ fetch: fetchImpl as unknown as typeof fetch });
const result = await adapter.verifyAuth(makeCtx({ apiKey: 'bad' }));
expect(result.ok).toBe(false);
if (!result.ok) expect(result.reason).toBe('auth');
});
it('Anthropic: 400 (typo\'d model) → reason=unexpected, NOT auth', async () => {
// Auth was fine; the request was rejected for a different reason. We
// don't want to block registration on bad model names — that error
// surfaces at chat time when the user actually picks a model.
const fetchImpl = mockFetch([
{ match: /\/v1\/messages$/, status: 400, text: '{"error":"model not found"}' },
]);
const adapter = new AnthropicAdapter({ fetch: fetchImpl as unknown as typeof fetch });
const result = await adapter.verifyAuth(makeCtx({ apiKey: 'sk-ant-x', modelOverride: 'claude-fake' }));
expect(result.ok).toBe(false);
if (!result.ok) expect(result.reason).toBe('unexpected');
});
});