feat(gate): route prompt-selection through mcpd's server Llm (credential tiering)
Some checks failed
CI/CD / typecheck (pull_request) Successful in 1m3s
CI/CD / lint (pull_request) Successful in 2m6s
CI/CD / test (pull_request) Successful in 1m18s
CI/CD / build (pull_request) Successful in 2m10s
CI/CD / smoke (pull_request) Failing after 3m9s
CI/CD / publish (pull_request) Has been skipped
Some checks failed
CI/CD / typecheck (pull_request) Successful in 1m3s
CI/CD / lint (pull_request) Successful in 2m6s
CI/CD / test (pull_request) Successful in 1m18s
CI/CD / build (pull_request) Successful in 2m10s
CI/CD / smoke (pull_request) Failing after 3m9s
CI/CD / publish (pull_request) Has been skipped
Cloud/server keys belong at the k8s/mcpd level; mcplocal handles only the user's personal tokens. The gate's prompt-selection used mcplocal's LOCAL provider registry (heavy=anthropic = the user's OAuth token, which is API-gated and 404s), so selection always degraded. Now LlmPromptSelector tries sources in priority order: (1) the project's server Llm via mcpd's inference proxy (POST /api/v1/llms/:name/infer) — cloud/server keys stay at k8s — then (2) the local personal-token provider as fallback. First source that returns valid selection JSON wins. Extracted pickCompletionText (content ?? reasoning_content) + extractSelection helpers. Wiring: GatePluginConfig.llmProvider (from the project), threaded via createDefaultPlugin; handleBeginSession builds serverInfer from ctx.postToMcpd. Verified live: POST .../llms/vllm-current/infer returns valid selection JSON (qwen3-thinking, 4000-token budget). + selector tests (server preferred, local fallback on error, server-only). mcplocal suite green (755). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -109,12 +109,49 @@ describe('LlmPromptSelector', () => {
|
||||
const selector = new LlmPromptSelector(registry);
|
||||
const ac = new AbortController();
|
||||
|
||||
await selector.selectPrompts(['test'], sampleIndex, undefined, ac.signal);
|
||||
await selector.selectPrompts(['test'], sampleIndex, { signal: ac.signal });
|
||||
|
||||
const call = (provider.complete as ReturnType<typeof vi.fn>).mock.calls[0]![0] as CompletionOptions;
|
||||
expect(call.signal).toBe(ac.signal);
|
||||
});
|
||||
|
||||
it('prefers the mcpd server Llm (serverInfer) over the local provider', async () => {
|
||||
const provider = makeMockProvider('{ "selectedNames": ["mqtt-config"], "reasoning": "local" }');
|
||||
const registry = makeRegistry(provider);
|
||||
const selector = new LlmPromptSelector(registry);
|
||||
const serverInfer = vi.fn().mockResolvedValue('{ "selectedNames": ["zigbee-pairing"], "reasoning": "server" }');
|
||||
|
||||
const result = await selector.selectPrompts(['x'], sampleIndex, { serverInfer });
|
||||
|
||||
expect(serverInfer).toHaveBeenCalledOnce();
|
||||
expect(provider.complete).not.toHaveBeenCalled(); // server succeeded → no local call
|
||||
expect(result.selectedNames).toContain('zigbee-pairing');
|
||||
expect(result.reasoning).toBe('server');
|
||||
});
|
||||
|
||||
it('falls back to the local provider when serverInfer throws', async () => {
|
||||
const provider = makeMockProvider('{ "selectedNames": ["mqtt-config"], "reasoning": "local" }');
|
||||
const registry = makeRegistry(provider);
|
||||
const selector = new LlmPromptSelector(registry);
|
||||
const serverInfer = vi.fn().mockRejectedValue(new Error('mcpd HTTP 503'));
|
||||
|
||||
const result = await selector.selectPrompts(['x'], sampleIndex, { serverInfer });
|
||||
|
||||
expect(serverInfer).toHaveBeenCalledOnce();
|
||||
expect(provider.complete).toHaveBeenCalledOnce();
|
||||
expect(result.selectedNames).toContain('mqtt-config');
|
||||
expect(result.reasoning).toBe('local');
|
||||
});
|
||||
|
||||
it('works server-only (null registry) when a serverInfer is supplied', async () => {
|
||||
const selector = new LlmPromptSelector(null);
|
||||
const serverInfer = vi.fn().mockResolvedValue('{ "selectedNames": ["mqtt-config"], "reasoning": "s" }');
|
||||
|
||||
const result = await selector.selectPrompts(['x'], sampleIndex, { serverInfer });
|
||||
|
||||
expect(result.selectedNames).toContain('mqtt-config');
|
||||
});
|
||||
|
||||
it('throws when no heavy provider is available', async () => {
|
||||
const registry = new ProviderRegistry(); // Empty registry
|
||||
const selector = new LlmPromptSelector(registry);
|
||||
|
||||
Reference in New Issue
Block a user