michal/mcpctl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(gate): MCPCTL_GATE_SELECTION_LLM — pin a dedicated fast selection Llm
Some checks failed
CI/CD / lint (pull_request) Successful in 1m2s
Details
CI/CD / typecheck (pull_request) Successful in 2m7s
Details
CI/CD / test (pull_request) Successful in 1m19s
Details
CI/CD / build (pull_request) Successful in 2m11s
Details
CI/CD / smoke (pull_request) Failing after 2m47s
Details
CI/CD / publish (pull_request) Has been skipped
Details

Browse Source 
Lets ops route gate prompt-selection at a fast (no-think) server Llm independent
of the project's chat llmProvider — so selection stays ~1-2s while chat keeps its
thinking model. Overrides the project llmProvider for the gate's server-selection
path; unset → prior behavior (use the project's llmProvider). Pairs with the
litellm qwen3-fast no-think alias (kubernetes-deployment) + a `vllm-fast` mcpd Llm.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Michal
2026-07-24 14:28:44 +01:00
parent 27480181bf
commit 5225b54901
1 changed files with 8 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
src/mcplocal/src/http/project-mcp-endpoint.ts
View File

@@ -147,10 +147,14 @@ export function registerProjectMcpEndpoint(app: FastifyInstance, mcpdClient: Mcp
providerRegistry: effectiveRegistry,
};
if (resolvedModel) pluginConfig.modelOverride = resolvedModel;
// Route gate prompt-selection through this project's server Llm (mcpd
// inference proxy) so cloud/server keys stay at the k8s level; the local
// personal-token provider is the fallback. See credential-tiering.
if (mcpdConfig.llmProvider) pluginConfig.llmProvider = mcpdConfig.llmProvider;
// Route gate prompt-selection through a server Llm (mcpd inference proxy)
// so cloud/server keys stay at the k8s level; the local personal-token
// provider is the fallback. See credential-tiering. A dedicated fast
// (no-think) selection Llm can be pinned globally via
// MCPCTL_GATE_SELECTION_LLM — it overrides the project's chat llmProvider so
// selection stays fast while chat keeps its (thinking) model.
const gateSelectionLlm = process.env['MCPCTL_GATE_SELECTION_LLM'] || mcpdConfig.llmProvider;
if (gateSelectionLlm) pluginConfig.llmProvider = gateSelectionLlm;
const basePlugin = createDefaultPlugin(pluginConfig);
// Optional favourite-index presentation: curated favourite/<tool> + full
// all/<server>/<tool> + a "prefer favourite/" instruction. Composed AFTER