michal/mcpctl
1
0
Fork 0
Code Issues Pull Requests 7 Actions Packages Projects Releases Wiki Activity

feat(llm): resolve the gate's selection Llm by tier, not by name #91

Open
michal wants to merge 2 commits from feat/llm-tier-resolution into main
pull from: feat/llm-tier-resolution
merge into: michal:main
Conversation 0 Commits 2 Files Changed 8 +531 -20
michal commented 2026-08-05 00:01:09 +00:00
Owner
Copy Link

What

mcpctl consumers now ask the Llm registry for a role (tier) instead of naming a row, so a homelab model swap needs zero mcpctl edits.

Why

Pulumi (deployments/mcpctl/llm-target.ts) already repoints vllm-current (fast) and vllm-think (heavy) at whatever LiteLLM serves — currently deepseek-v4-flash / deepseek-v4-think. But nothing in mcpctl resolved those rows. Gate prompt-selection read MCPCTL_GATE_SELECTION_LLM || project.llmProvider, and neither is set on any project or on either deployment, so unpinned projects fell through to the local personal-token provider:

[gate] LLM prompt-selection failed: Anthropic HTTP 404: model: claude-opus-4-20250514
       — falling back to priority-ordered prompts

Every gated session has been running on priority order, not LLM ranking.

How

New src/mcplocal/src/server-llm.ts. A row fills a tier when tier matches, status is active, and kind is not virtual — virtual rows are backed by some user's mcplocal over SSE, and automatic resolution must not route project traffic through a laptop nobody chose. An explicit pin may still name one.

heavy already holds two rows (vllm-think + anthropic-fallback), so the tiebreak matters today: an ordered well-known-name list, seeded with both the current names and the symmetric names the rows might be renamed to (vllm-fast, vllm-thinking). Names matching nothing are inert, so a rename stays a Pulumi-only change and neither deploy order breaks the other. Unlisted rows still resolve, sorting last then by name. MCPCTL_LLM_PREFER_FAST / _HEAVY override without a release.

Resolution never throws — mcpd unreachable is a normal degraded state, and returning null preserves the caller's existing fallback.

Gate order is now: env pin → project llmProvider → fast-tier lookup. Explicit pins still win.

Naming

Rows keep vllm-current / vllm-think. name is immutable in mcpd (agents/projects reference it), so a rename costs a data migration plus a Pulumi resource replace and buys nothing once nothing reads the names. vllm-fast would also collide in meaning with LiteLLM's deepseek-v4-fast route (pins thinking: false) while the row points at the base deepseek-v4-flash.

Tests

  • tests/server-llm.test.ts — 15 unit tests: tiebreak, rename window, unknown names, inactive/virtual exclusion, env override, unreachable mcpd.
  • tests/smoke/llm-tier.smoke.test.ts — live stack: both tiers resolve; the resolved fast row answers real inference (the drift check — a row left on a suspended model fails here); an unpinned gated project gets LLM-ranked selection with no ⚠ Smart prompt-selection unavailable banner.

770/770 mcplocal unit tests pass. Typecheck clean. Lint on the touched file goes 28 → 26 pre-existing errors.

⚠️ Smoke not yet run against live — it needs this build deployed and mcplocal restarted, which drops in-flight MCP sessions.

Docs

docs/llm-tiers.md (new), README create llm example de-staled from qwen3-thinking, CLAUDE.md cheatsheet note.

🤖 Generated with Claude Code

https://claude.ai/code/session_01B5NAoE7VJA5TWvHsVfEmUr

## What mcpctl consumers now ask the Llm registry for a **role** (`tier`) instead of naming a row, so a homelab model swap needs zero mcpctl edits. ## Why Pulumi (`deployments/mcpctl/llm-target.ts`) already repoints `vllm-current` (fast) and `vllm-think` (heavy) at whatever LiteLLM serves — currently `deepseek-v4-flash` / `deepseek-v4-think`. But nothing in mcpctl resolved those rows. Gate prompt-selection read `MCPCTL_GATE_SELECTION_LLM || project.llmProvider`, and **neither is set on any project or on either deployment**, so unpinned projects fell through to the local personal-token provider: ``` [gate] LLM prompt-selection failed: Anthropic HTTP 404: model: claude-opus-4-20250514 — falling back to priority-ordered prompts ``` Every gated session has been running on priority order, not LLM ranking. ## How New `src/mcplocal/src/server-llm.ts`. A row fills a tier when `tier` matches, `status` is `active`, and `kind` is not `virtual` — virtual rows are backed by some user's mcplocal over SSE, and automatic resolution must not route project traffic through a laptop nobody chose. An explicit pin may still name one. `heavy` already holds two rows (`vllm-think` + `anthropic-fallback`), so the tiebreak matters today: an ordered well-known-name list, seeded with both the current names and the symmetric names the rows might be renamed to (`vllm-fast`, `vllm-thinking`). Names matching nothing are inert, so a rename stays a Pulumi-only change and neither deploy order breaks the other. Unlisted rows still resolve, sorting last then by name. `MCPCTL_LLM_PREFER_FAST` / `_HEAVY` override without a release. Resolution never throws — mcpd unreachable is a normal degraded state, and returning `null` preserves the caller's existing fallback. Gate order is now: env pin → project `llmProvider` → fast-tier lookup. Explicit pins still win. ## Naming Rows keep `vllm-current` / `vllm-think`. `name` is immutable in mcpd (agents/projects reference it), so a rename costs a data migration plus a Pulumi resource replace and buys nothing once nothing reads the names. `vllm-fast` would also collide in meaning with LiteLLM's `deepseek-v4-fast` route (pins `thinking: false`) while the row points at the base `deepseek-v4-flash`. ## Tests - `tests/server-llm.test.ts` — 15 unit tests: tiebreak, rename window, unknown names, inactive/virtual exclusion, env override, unreachable mcpd. - `tests/smoke/llm-tier.smoke.test.ts` — live stack: both tiers resolve; the resolved fast row **answers real inference** (the drift check — a row left on a suspended model fails here); an unpinned gated project gets LLM-ranked selection with no `⚠ Smart prompt-selection unavailable` banner. 770/770 mcplocal unit tests pass. Typecheck clean. Lint on the touched file goes 28 → 26 pre-existing errors. ⚠️ **Smoke not yet run against live** — it needs this build deployed and mcplocal restarted, which drops in-flight MCP sessions. ## Docs `docs/llm-tiers.md` (new), README `create llm` example de-staled from `qwen3-thinking`, CLAUDE.md cheatsheet note. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01B5NAoE7VJA5TWvHsVfEmUr
michal added 1 commit 2026-08-05 00:01:09 +00:00
feat(llm): resolve the gate's selection Llm by tier, not by name
Some checks failed
CI/CD / lint (pull_request) Successful in 1m19s
Details
CI/CD / typecheck (pull_request) Successful in 2m21s
Details
CI/CD / test (pull_request) Successful in 1m19s
Details
CI/CD / build (pull_request) Successful in 2m17s
Details
CI/CD / smoke (pull_request) Failing after 2m47s
Details
CI/CD / publish (pull_request) Has been skipped
Details
2da627e77f 
The served model changes (glm-4.6-reap -> deepseek-v4-flash, and again after
that). Pulumi already repoints mcpd's long-lived rows -- vllm-current/fast and
vllm-think/heavy -- at whatever LiteLLM serves, so no model id is written down
in mcpctl. But nothing resolved those rows: gate prompt-selection took
MCPCTL_GATE_SELECTION_LLM or the project's llmProvider, and neither is set on
any project or either deployment. Unpinned projects fell through to the local
personal-token provider, whose heavy entry names a retired claude-opus-4 -- so
every gated session has been running on priority-ordered prompts, not LLM
ranking, announcing it only in a mcplocal log line nobody reads.

Consumers now ask for a role. tier is already a field on the Llm resource and
is set by the same Pulumi resource that sets model, so the two cannot drift.
Explicit pins still win; this only changes what happens when nothing is pinned.

Eligibility excludes inactive rows (selecting one defers the failure to the
first inference call) and virtual rows -- those are backed by some user's
mcplocal over SSE, and automatic resolution must not route a project's traffic
through a laptop nobody chose. The heavy tier already holds two rows, so the
tiebreak is load-bearing now rather than future-proofing: an ordered list of
well-known names, carrying both the current names and the symmetric ones the
rows might be renamed to, so a rename stays a Pulumi-only change and neither
deploy order breaks the other. Names matching nothing are inert by design.

The rows keep the names they have. name is immutable in mcpd (agents and
projects reference it), so a rename costs a data migration plus a resource
replace, and buys nothing once nothing reads the names. vllm-fast would also
collide in meaning with LiteLLM's deepseek-v4-fast route, which is a different
thing one word apart.

Smoke covers the drift case the unit tests cannot: the resolved fast row has to
answer real inference, which is exactly what fails when a row is left pointing
at a suspended model.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B5NAoE7VJA5TWvHsVfEmUr
michal added 1 commit 2026-08-05 00:27:32 +00:00
fix(smoke): make the /inspect auth probe independent of concurrent traffic
Some checks failed
CI/CD / typecheck (pull_request) Successful in 1m3s
Details
CI/CD / lint (pull_request) Successful in 2m5s
Details
CI/CD / test (pull_request) Successful in 1m17s
Details
CI/CD / build (pull_request) Successful in 2m12s
Details
CI/CD / smoke (pull_request) Failing after 2m42s
Details
CI/CD / publish (pull_request) Has been skipped
Details
de906f9c75 
The probe waited for a 3s socket-inactivity timeout and treated the rejection
as proof the endpoint was reachable. That only holds when nothing else is
talking to mcplocal — /inspect streams every project's MCP traffic, so any
other smoke file running concurrently keeps the socket busy and the timeout
never fires. Adding one more traffic-generating smoke file was enough to tip
it into a 10s test timeout; it passed in isolation the whole time.

Resolve on response headers instead. That removes the dependence on the rest
of the suite being quiet, and it is a stronger assertion than before: the test
now reads the real status code and content-type, where a timeout-means-success
probe would have passed just as happily against a slow 401.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B5NAoE7VJA5TWvHsVfEmUr
Some checks failed
CI/CD / typecheck (pull_request) Successful in 1m3s
Details
CI/CD / lint (pull_request) Successful in 2m5s
Details
CI/CD / test (pull_request) Successful in 1m17s
Details
CI/CD / build (pull_request) Successful in 2m12s
Details
CI/CD / smoke (pull_request) Failing after 2m42s
Details
CI/CD / publish (pull_request) Has been skipped
Details
This pull request has changes conflicting with the target branch.
  • src/mcplocal/tests/smoke/security.test.ts
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin feat/llm-tier-resolution:feat/llm-tier-resolution
git checkout feat/llm-tier-resolution
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
michal
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: michal/mcpctl#91
Delete Branch "feat/llm-tier-resolution"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?