feat(gate): route prompt-selection through mcpd server Llm (credential tiering) #87

michal · 2026-07-24T11:51:54Z

michal commented

2026-07-24 11:51:54 +00:00

Per the credential-tiering principle: cloud/server keys live at mcpd/k8s; mcplocal handles only personal tokens. The gate previously used mcplocal's local heavy provider (the user's OAuth token → 404s). Now LlmPromptSelector tries the project's server Llm via mcpd infer first, falling back to the local personal-token provider.

Validated live: the sre gate failure reason moved from Anthropic-HTTP-404 to server-path timeout, proving the server path engages; POST /llms/vllm-current/infer returns valid selection JSON. Deployed as 2729580.

Known residual (ops, not code): qwen3-thinking takes ~40s (always reasons), exceeding the gate's 8s responsiveness budget → graceful degradation to priority-ordering until a FAST server Llm is served (or a real key lights up anthropic-fallback as a fast selection model). mcplocal suite green (755).

🤖 Generated with Claude Code

Per the credential-tiering principle: cloud/server keys live at mcpd/k8s; mcplocal handles only personal tokens. The gate previously used mcplocal's local heavy provider (the user's OAuth token → 404s). Now LlmPromptSelector tries the project's server Llm via mcpd infer first, falling back to the local personal-token provider. Validated live: the sre gate failure reason moved from Anthropic-HTTP-404 to server-path timeout, proving the server path engages; POST /llms/vllm-current/infer returns valid selection JSON. Deployed as 2729580. Known residual (ops, not code): qwen3-thinking takes ~40s (always reasons), exceeding the gate's 8s responsiveness budget → graceful degradation to priority-ordering until a FAST server Llm is served (or a real key lights up anthropic-fallback as a fast selection model). mcplocal suite green (755). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

michal added 1 commit 2026-07-24 11:51:55 +00:00

feat(gate): route prompt-selection through mcpd's server Llm (credential tiering)

CI/CD / typecheck (pull_request) Successful in 1m3s

Details

CI/CD / lint (pull_request) Successful in 2m6s

Details

CI/CD / test (pull_request) Successful in 1m18s

Details

CI/CD / build (pull_request) Successful in 2m10s

Details

CI/CD / smoke (pull_request) Failing after 3m9s

Details

CI/CD / publish (pull_request) Has been skipped

Details

2729580974

Cloud/server keys belong at the k8s/mcpd level; mcplocal handles only the user's
personal tokens. The gate's prompt-selection used mcplocal's LOCAL provider
registry (heavy=anthropic = the user's OAuth token, which is API-gated and 404s),
so selection always degraded.

Now LlmPromptSelector tries sources in priority order: (1) the project's server
Llm via mcpd's inference proxy (POST /api/v1/llms/:name/infer) — cloud/server
keys stay at k8s — then (2) the local personal-token provider as fallback. First
source that returns valid selection JSON wins. Extracted pickCompletionText
(content ?? reasoning_content) + extractSelection helpers.

Wiring: GatePluginConfig.llmProvider (from the project), threaded via
createDefaultPlugin; handleBeginSession builds serverInfer from ctx.postToMcpd.
Verified live: POST .../llms/vllm-current/infer returns valid selection JSON
(qwen3-thinking, 4000-token budget). + selector tests (server preferred, local
fallback on error, server-only). mcplocal suite green (755).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

michal merged commit 86942b1e91 into main

2026-07-24 11:52:04 +00:00

michal referenced this issue from a commit

2026-07-24 11:52:06 +00:00

feat(gate): route prompt-selection through mcpd server Llm (#87)

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: michal/mcpctl#87