feat(llm): resolve the gate's selection Llm by tier, not by name
Some checks failed
CI/CD / lint (pull_request) Successful in 1m19s
CI/CD / typecheck (pull_request) Successful in 2m21s
CI/CD / test (pull_request) Successful in 1m19s
CI/CD / build (pull_request) Successful in 2m17s
CI/CD / smoke (pull_request) Failing after 2m47s
CI/CD / publish (pull_request) Has been skipped

The served model changes (glm-4.6-reap -> deepseek-v4-flash, and again after
that). Pulumi already repoints mcpd's long-lived rows -- vllm-current/fast and
vllm-think/heavy -- at whatever LiteLLM serves, so no model id is written down
in mcpctl. But nothing resolved those rows: gate prompt-selection took
MCPCTL_GATE_SELECTION_LLM or the project's llmProvider, and neither is set on
any project or either deployment. Unpinned projects fell through to the local
personal-token provider, whose heavy entry names a retired claude-opus-4 -- so
every gated session has been running on priority-ordered prompts, not LLM
ranking, announcing it only in a mcplocal log line nobody reads.

Consumers now ask for a role. tier is already a field on the Llm resource and
is set by the same Pulumi resource that sets model, so the two cannot drift.
Explicit pins still win; this only changes what happens when nothing is pinned.

Eligibility excludes inactive rows (selecting one defers the failure to the
first inference call) and virtual rows -- those are backed by some user's
mcplocal over SSE, and automatic resolution must not route a project's traffic
through a laptop nobody chose. The heavy tier already holds two rows, so the
tiebreak is load-bearing now rather than future-proofing: an ordered list of
well-known names, carrying both the current names and the symmetric ones the
rows might be renamed to, so a rename stays a Pulumi-only change and neither
deploy order breaks the other. Names matching nothing are inert by design.

The rows keep the names they have. name is immutable in mcpd (agents and
projects reference it), so a rename costs a data migration plus a resource
replace, and buys nothing once nothing reads the names. vllm-fast would also
collide in meaning with LiteLLM's deepseek-v4-fast route, which is a different
thing one word apart.

Smoke covers the drift case the unit tests cannot: the resolved fast row has to
answer real inference, which is exactly what fails when a row is left pointing
at a suspended model.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B5NAoE7VJA5TWvHsVfEmUr
This commit is contained in:
Michal
2026-08-05 01:00:47 +01:00
parent cf85e36ede
commit 2da627e77f
7 changed files with 510 additions and 12 deletions

View File

@@ -516,17 +516,19 @@ description "I review security design — ask me after each major change."
That's how agents consult each other.
```bash
# 1) point at an LLM. For your in-cluster qwen3-thinking via LiteLLM:
# 1) point at an LLM. Name the ROW for the role it fills, not for the model —
# the served model changes, the row should not. See docs/llm-tiers.md.
mcpctl create secret litellm-key --data API_KEY=sk-…
mcpctl create llm qwen3-thinking \
mcpctl create llm vllm-current \
--type openai \
--model qwen3-thinking \
--model deepseek-v4-flash \
--tier fast \
--url http://litellm.nvidia-nim.svc.cluster.local:4000/v1 \
--api-key-ref litellm-key/API_KEY
# 2) create an agent, pinned to that Llm and attached to a project
mcpctl create agent reviewer \
--llm qwen3-thinking \
--llm vllm-current \
--project mcpctl-dev \
--description "I review security design — ask me after each major change." \
--system-prompt-file ./prompts/reviewer.md \
@@ -586,7 +588,8 @@ systemctl --user restart mcplocal
mcpctl get llm
# NAME KIND STATUS TYPE MODEL TIER ID
# qwen3-thinking public active openai qwen3-thinking fast ...
# vllm-current public active openai deepseek-v4-flash fast ...
# vllm-think public active openai deepseek-v4-think heavy ...
# vllm-local virtual active openai Qwen/Qwen2.5-7B-Instruct-AWQ fast ...
mcpctl chat-llm vllm-local