feat(llm): resolve the gate's selection Llm by tier, not by name #91

Open

michal wants to merge 2 commits from feat/llm-tier-resolution into main

Author	SHA1	Message	Date
Michal	de906f9c75	fix(smoke): make the /inspect auth probe independent of concurrent traffic Some checks failed CI/CD / typecheck (pull_request) Successful in 1m3s Details CI/CD / lint (pull_request) Successful in 2m5s Details CI/CD / test (pull_request) Successful in 1m17s Details CI/CD / build (pull_request) Successful in 2m12s Details CI/CD / smoke (pull_request) Failing after 2m42s Details CI/CD / publish (pull_request) Has been skipped Details The probe waited for a 3s socket-inactivity timeout and treated the rejection as proof the endpoint was reachable. That only holds when nothing else is talking to mcplocal — /inspect streams every project's MCP traffic, so any other smoke file running concurrently keeps the socket busy and the timeout never fires. Adding one more traffic-generating smoke file was enough to tip it into a 10s test timeout; it passed in isolation the whole time. Resolve on response headers instead. That removes the dependence on the rest of the suite being quiet, and it is a stronger assertion than before: the test now reads the real status code and content-type, where a timeout-means-success probe would have passed just as happily against a slow 401. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B5NAoE7VJA5TWvHsVfEmUr	2026-08-05 01:27:30 +01:00
Michal	2da627e77f	feat(llm): resolve the gate's selection Llm by tier, not by name Some checks failed CI/CD / lint (pull_request) Successful in 1m19s Details CI/CD / typecheck (pull_request) Successful in 2m21s Details CI/CD / test (pull_request) Successful in 1m19s Details CI/CD / build (pull_request) Successful in 2m17s Details CI/CD / smoke (pull_request) Failing after 2m47s Details CI/CD / publish (pull_request) Has been skipped Details The served model changes (glm-4.6-reap -> deepseek-v4-flash, and again after that). Pulumi already repoints mcpd's long-lived rows -- vllm-current/fast and vllm-think/heavy -- at whatever LiteLLM serves, so no model id is written down in mcpctl. But nothing resolved those rows: gate prompt-selection took MCPCTL_GATE_SELECTION_LLM or the project's llmProvider, and neither is set on any project or either deployment. Unpinned projects fell through to the local personal-token provider, whose heavy entry names a retired claude-opus-4 -- so every gated session has been running on priority-ordered prompts, not LLM ranking, announcing it only in a mcplocal log line nobody reads. Consumers now ask for a role. tier is already a field on the Llm resource and is set by the same Pulumi resource that sets model, so the two cannot drift. Explicit pins still win; this only changes what happens when nothing is pinned. Eligibility excludes inactive rows (selecting one defers the failure to the first inference call) and virtual rows -- those are backed by some user's mcplocal over SSE, and automatic resolution must not route a project's traffic through a laptop nobody chose. The heavy tier already holds two rows, so the tiebreak is load-bearing now rather than future-proofing: an ordered list of well-known names, carrying both the current names and the symmetric ones the rows might be renamed to, so a rename stays a Pulumi-only change and neither deploy order breaks the other. Names matching nothing are inert by design. The rows keep the names they have. name is immutable in mcpd (agents and projects reference it), so a rename costs a data migration plus a resource replace, and buys nothing once nothing reads the names. vllm-fast would also collide in meaning with LiteLLM's deepseek-v4-fast route, which is a different thing one word apart. Smoke covers the drift case the unit tests cannot: the resolved fast row has to answer real inference, which is exactly what fails when a row is left pointing at a suspended model. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B5NAoE7VJA5TWvHsVfEmUr	2026-08-05 01:00:47 +01:00

Author

SHA1

Message

Date

Michal

de906f9c75

fix(smoke): make the /inspect auth probe independent of concurrent traffic

CI/CD / typecheck (pull_request) Successful in 1m3s

Details

CI/CD / lint (pull_request) Successful in 2m5s

Details

CI/CD / test (pull_request) Successful in 1m17s

Details

CI/CD / build (pull_request) Successful in 2m12s

Details

CI/CD / smoke (pull_request) Failing after 2m42s

Details

CI/CD / publish (pull_request) Has been skipped

Details

The probe waited for a 3s socket-inactivity timeout and treated the rejection
as proof the endpoint was reachable. That only holds when nothing else is
talking to mcplocal — /inspect streams every project's MCP traffic, so any
other smoke file running concurrently keeps the socket busy and the timeout
never fires. Adding one more traffic-generating smoke file was enough to tip
it into a 10s test timeout; it passed in isolation the whole time.

Resolve on response headers instead. That removes the dependence on the rest
of the suite being quiet, and it is a stronger assertion than before: the test
now reads the real status code and content-type, where a timeout-means-success
probe would have passed just as happily against a slow 401.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B5NAoE7VJA5TWvHsVfEmUr

2026-08-05 01:27:30 +01:00

Michal

2da627e77f

feat(llm): resolve the gate's selection Llm by tier, not by name

CI/CD / lint (pull_request) Successful in 1m19s

Details

CI/CD / typecheck (pull_request) Successful in 2m21s

Details

CI/CD / test (pull_request) Successful in 1m19s

Details

CI/CD / build (pull_request) Successful in 2m17s

Details

CI/CD / smoke (pull_request) Failing after 2m47s

Details

CI/CD / publish (pull_request) Has been skipped

Details

The served model changes (glm-4.6-reap -> deepseek-v4-flash, and again after
that). Pulumi already repoints mcpd's long-lived rows -- vllm-current/fast and
vllm-think/heavy -- at whatever LiteLLM serves, so no model id is written down
in mcpctl. But nothing resolved those rows: gate prompt-selection took
MCPCTL_GATE_SELECTION_LLM or the project's llmProvider, and neither is set on
any project or either deployment. Unpinned projects fell through to the local
personal-token provider, whose heavy entry names a retired claude-opus-4 -- so
every gated session has been running on priority-ordered prompts, not LLM
ranking, announcing it only in a mcplocal log line nobody reads.

Consumers now ask for a role. tier is already a field on the Llm resource and
is set by the same Pulumi resource that sets model, so the two cannot drift.
Explicit pins still win; this only changes what happens when nothing is pinned.

Eligibility excludes inactive rows (selecting one defers the failure to the
first inference call) and virtual rows -- those are backed by some user's
mcplocal over SSE, and automatic resolution must not route a project's traffic
through a laptop nobody chose. The heavy tier already holds two rows, so the
tiebreak is load-bearing now rather than future-proofing: an ordered list of
well-known names, carrying both the current names and the symmetric ones the
rows might be renamed to, so a rename stays a Pulumi-only change and neither
deploy order breaks the other. Names matching nothing are inert by design.

The rows keep the names they have. name is immutable in mcpd (agents and
projects reference it), so a rename costs a data migration plus a resource
replace, and buys nothing once nothing reads the names. vllm-fast would also
collide in meaning with LiteLLM's deepseek-v4-fast route, which is a different
thing one word apart.

Smoke covers the drift case the unit tests cannot: the resolved fast row has to
answer real inference, which is exactly what fails when a row is left pointing
at a suspended model.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B5NAoE7VJA5TWvHsVfEmUr

2026-08-05 01:00:47 +01:00

feat(llm): resolve the gate's selection Llm by tier, not by name #91

2 Commits