feat(cli+mcplocal): mcpctl provider <name> {up,down,status} for managed LLMs
Some checks failed
CI/CD / typecheck (pull_request) Successful in 57s
CI/CD / test (pull_request) Successful in 1m23s
CI/CD / lint (pull_request) Successful in 3m1s
CI/CD / smoke (pull_request) Failing after 1m47s
CI/CD / build (pull_request) Successful in 5m58s
CI/CD / publish (pull_request) Has been skipped

Adds lifecycle control for managed local LLM providers (vllm-managed)
without the nuclear option of restarting mcplocal. Practical use:

  mcpctl provider vllm-local down    # release GPU memory now
  mcpctl provider vllm-local up      # warm up before the next chat
  mcpctl provider vllm-local status  # see state, pid, uptime

mcplocal exposes three new endpoints:

  GET  /llm/providers/:name/status   → returns lifecycle state for
                                       managed providers, { managed: false }
                                       for unmanaged (anthropic, openai, …)
  POST /llm/providers/:name/start    → calls warmup() (202 + initial state)
  POST /llm/providers/:name/stop     → calls dispose() (200 + post-stop state)

Stop and start return 400 for non-managed providers — stopping an API-key
provider is meaningless. The CLI surfaces the error verbatim.

Restarting mcplocal would also free the GPU but drops the SSE connection
to mcpd and forces every virtual Llm to re-publish; this is the targeted,
non-disruptive escape hatch.

The completions test gained a `topLevelMarkers` filter so a sub-command
named `status` (under `provider`) doesn't trip the existing "non-project
commands must guard with __mcpctl_has_project" rule.

Tests: cli 437/437, mcplocal 731/731.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Michal
2026-04-29 15:58:46 +01:00
parent 3071bcee8e
commit 356cbe87b5
6 changed files with 234 additions and 3 deletions

View File

@@ -220,6 +220,64 @@ export async function createHttpServer(
});
});
// Per-provider status (managed providers expose lifecycle state). Used by
// `mcpctl provider <name> status` to read vllm-managed's state without
// burning a token like /llm/health does.
app.get<{ Params: { name: string } }>('/llm/providers/:name/status', async (request, reply) => {
const registry = deps.providerRegistry;
const provider = registry?.get(request.params.name) ?? null;
if (provider === null) {
reply.code(404).send({ error: `Provider '${request.params.name}' not found` });
return;
}
if (!('getStatus' in provider) || typeof (provider as ManagedVllmProvider).getStatus !== 'function') {
// Non-managed providers (anthropic, openai, gemini-cli) have no
// lifecycle — they're always "ready" as long as the API key works.
reply.code(200).send({ name: provider.name, managed: false });
return;
}
const status = (provider as ManagedVllmProvider).getStatus();
reply.code(200).send({ name: provider.name, managed: true, ...status });
});
// Stop a managed provider (free GPU memory). No-op on non-managed
// providers. Returns 200 with the resulting status either way.
app.post<{ Params: { name: string } }>('/llm/providers/:name/stop', async (request, reply) => {
const registry = deps.providerRegistry;
const provider = registry?.get(request.params.name) ?? null;
if (provider === null) {
reply.code(404).send({ error: `Provider '${request.params.name}' not found` });
return;
}
if (!('dispose' in provider) || typeof (provider as ManagedVllmProvider).dispose !== 'function') {
reply.code(400).send({ error: `Provider '${request.params.name}' is not managed (nothing to stop)` });
return;
}
(provider as ManagedVllmProvider).dispose();
const status = (provider as ManagedVllmProvider).getStatus();
reply.code(200).send({ name: provider.name, managed: true, ...status });
});
// Start (warm up) a managed provider so the first chat doesn't pay
// the model-load latency.
app.post<{ Params: { name: string } }>('/llm/providers/:name/start', async (request, reply) => {
const registry = deps.providerRegistry;
const provider = registry?.get(request.params.name) ?? null;
if (provider === null) {
reply.code(404).send({ error: `Provider '${request.params.name}' not found` });
return;
}
if (!('warmup' in provider) || typeof (provider as ManagedVllmProvider).warmup !== 'function') {
reply.code(400).send({ error: `Provider '${request.params.name}' is not managed (nothing to start)` });
return;
}
(provider as ManagedVllmProvider).warmup();
// warmup() is fire-and-forget — return current state immediately so
// the CLI can show 'starting' and the user knows it's been kicked.
const status = (provider as ManagedVllmProvider).getStatus();
reply.code(202).send({ name: provider.name, managed: true, ...status });
});
// ProxyModel discovery endpoints
registerProxymodelEndpoint(app);