feat(llm): resolve the gate's selection Llm by tier, not by name
Some checks failed
CI/CD / lint (pull_request) Successful in 1m19s
CI/CD / typecheck (pull_request) Successful in 2m21s
CI/CD / test (pull_request) Successful in 1m19s
CI/CD / build (pull_request) Successful in 2m17s
CI/CD / smoke (pull_request) Failing after 2m47s
CI/CD / publish (pull_request) Has been skipped
Some checks failed
CI/CD / lint (pull_request) Successful in 1m19s
CI/CD / typecheck (pull_request) Successful in 2m21s
CI/CD / test (pull_request) Successful in 1m19s
CI/CD / build (pull_request) Successful in 2m17s
CI/CD / smoke (pull_request) Failing after 2m47s
CI/CD / publish (pull_request) Has been skipped
The served model changes (glm-4.6-reap -> deepseek-v4-flash, and again after that). Pulumi already repoints mcpd's long-lived rows -- vllm-current/fast and vllm-think/heavy -- at whatever LiteLLM serves, so no model id is written down in mcpctl. But nothing resolved those rows: gate prompt-selection took MCPCTL_GATE_SELECTION_LLM or the project's llmProvider, and neither is set on any project or either deployment. Unpinned projects fell through to the local personal-token provider, whose heavy entry names a retired claude-opus-4 -- so every gated session has been running on priority-ordered prompts, not LLM ranking, announcing it only in a mcplocal log line nobody reads. Consumers now ask for a role. tier is already a field on the Llm resource and is set by the same Pulumi resource that sets model, so the two cannot drift. Explicit pins still win; this only changes what happens when nothing is pinned. Eligibility excludes inactive rows (selecting one defers the failure to the first inference call) and virtual rows -- those are backed by some user's mcplocal over SSE, and automatic resolution must not route a project's traffic through a laptop nobody chose. The heavy tier already holds two rows, so the tiebreak is load-bearing now rather than future-proofing: an ordered list of well-known names, carrying both the current names and the symmetric ones the rows might be renamed to, so a rename stays a Pulumi-only change and neither deploy order breaks the other. Names matching nothing are inert by design. The rows keep the names they have. name is immutable in mcpd (agents and projects reference it), so a rename costs a data migration plus a resource replace, and buys nothing once nothing reads the names. vllm-fast would also collide in meaning with LiteLLM's deepseek-v4-fast route, which is a different thing one word apart. Smoke covers the drift case the unit tests cannot: the resolved fast row has to answer real inference, which is exactly what fails when a row is left pointing at a suspended model. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B5NAoE7VJA5TWvHsVfEmUr
This commit is contained in:
13
README.md
13
README.md
@@ -516,17 +516,19 @@ description "I review security design — ask me after each major change."
|
||||
That's how agents consult each other.
|
||||
|
||||
```bash
|
||||
# 1) point at an LLM. For your in-cluster qwen3-thinking via LiteLLM:
|
||||
# 1) point at an LLM. Name the ROW for the role it fills, not for the model —
|
||||
# the served model changes, the row should not. See docs/llm-tiers.md.
|
||||
mcpctl create secret litellm-key --data API_KEY=sk-…
|
||||
mcpctl create llm qwen3-thinking \
|
||||
mcpctl create llm vllm-current \
|
||||
--type openai \
|
||||
--model qwen3-thinking \
|
||||
--model deepseek-v4-flash \
|
||||
--tier fast \
|
||||
--url http://litellm.nvidia-nim.svc.cluster.local:4000/v1 \
|
||||
--api-key-ref litellm-key/API_KEY
|
||||
|
||||
# 2) create an agent, pinned to that Llm and attached to a project
|
||||
mcpctl create agent reviewer \
|
||||
--llm qwen3-thinking \
|
||||
--llm vllm-current \
|
||||
--project mcpctl-dev \
|
||||
--description "I review security design — ask me after each major change." \
|
||||
--system-prompt-file ./prompts/reviewer.md \
|
||||
@@ -586,7 +588,8 @@ systemctl --user restart mcplocal
|
||||
|
||||
mcpctl get llm
|
||||
# NAME KIND STATUS TYPE MODEL TIER ID
|
||||
# qwen3-thinking public active openai qwen3-thinking fast ...
|
||||
# vllm-current public active openai deepseek-v4-flash fast ...
|
||||
# vllm-think public active openai deepseek-v4-think heavy ...
|
||||
# vllm-local virtual active openai Qwen/Qwen2.5-7B-Instruct-AWQ fast ...
|
||||
|
||||
mcpctl chat-llm vllm-local
|
||||
|
||||
Reference in New Issue
Block a user