feat(mcpd+cli+mcplocal): /llms/<name>/members + POOL column + --pool-name (v4 Stage 2)

Surfaces the v4 pool model end-to-end:

- mcpd: GET /api/v1/llms/:name/members returns the effective pool the
  named anchor belongs to, plus aggregate stats (size, activeCount,
  explicit vs implicit pool key). RBAC inherits from `view:llms` —
  same as the single-Llm route. Members are full LlmView shapes so
  callers don't need a second roundtrip to render the pool block.

- mcpd: VirtualLlmService.register accepts an optional `poolName` on
  RegisterProviderInput; the route's `coerceProviderInput` validates
  the same character set as CreateLlmSchema.poolName. Backwards
  compatible — older mcplocals that don't send the field continue to
  publish solo Llms.

- CLI `get llm` table: new POOL column right after NAME. Solo rows
  show "-" so the "no pool / pool of 1" case is unambiguous (per
  user direction "make sure we see it, prominently visible and
  impossible to mistake").

- CLI `describe llm`: fetches /members and renders a Pool block at
  the top of the detail view when the row is in an explicit pool OR
  when its implicit pool has size > 1. Each member line shows
  kind/status; the anchor row gets "← this row". Block is suppressed
  for solo rows so describe stays compact in the common case.

- CLI `create llm --pool-name <name>` flag and apply schema both
  accept the new field. Yaml round-trip preserves it: get -o yaml
  emits `poolName: <name>`, apply -f re-imports it without diff.
  Verified end-to-end against the live mcpd.

- mcplocal: LlmProviderFileEntry gains optional `poolName`; main.ts
  and registrar.ts thread it through into the register payload. Use
  case for distributed inference: each user's mcplocal picks a
  unique `name` (e.g. `vllm-<host>-qwen3`) but a shared `poolName`
  (e.g. `user-vllm-qwen3-thinking`); agents see one logical pool
  that auto-grows as workers come online.

- Shell completions: regenerated from source via the existing
  scripts/generate-completions.ts. `--pool-name` now suggests in
  fish + bash for `mcpctl create llm`.

Tests: +3 new mcpd route tests for /members (explicit pool, solo
pool of 1, missing-anchor 404). All suites green:
  mcpd 868/868 (was 865, +3),
  mcplocal 723/723,
  cli 437/437.

Stage 3 (next): live smoke against 2 publishers sharing a pool name +
docs.
This commit is contained in:
Michal
2026-04-27 23:18:53 +01:00
parent 7949e1393d
commit e21f96080d
14 changed files with 213 additions and 6 deletions

View File

@@ -93,6 +93,18 @@ export interface LlmProviderFileEntry {
* - `command`: spawn a shell command (e.g. `systemctl --user start vllm`)
*/
wake?: WakeRecipe;
/**
* v4: opt this provider into a load-balanced pool. When set, the
* published Llm row carries `poolName` and stacks with any other Llms
* (public OR virtual) sharing the same value. Agents pinned to any
* pool member dispatch across all healthy members at chat time.
*
* Convention for distributed-compute setups: each user's mcplocal
* picks a unique `name` (e.g. `vllm-<hostname>-qwen3`) but a shared
* `poolName` (e.g. `user-vllm-qwen3-thinking`). Result: agents see one
* logical pool that auto-grows as more workers come online.
*/
poolName?: string;
}
export type WakeRecipe =

View File

@@ -218,6 +218,7 @@ async function maybeStartVirtualLlmRegistrar(
};
if (entry.tier !== undefined) item.tier = entry.tier;
if (entry.wake !== undefined) item.wake = entry.wake;
if (entry.poolName !== undefined) item.poolName = entry.poolName;
published.push(item);
}
// v3: forward locally-declared agents alongside the providers. We

View File

@@ -54,6 +54,12 @@ export interface RegistrarPublishedProvider {
* the registrar runs this recipe and waits for the backend to come up.
*/
wake?: WakeRecipe;
/**
* v4: optional pool key. When set, the published Llm row carries
* `poolName` and stacks with any other Llms sharing the same value.
* Agents pinned to any pool member dispatch across all healthy members.
*/
poolName?: string;
}
/**
@@ -185,6 +191,7 @@ export class VirtualLlmRegistrar {
model: p.model,
...(p.tier !== undefined ? { tier: p.tier } : {}),
...(p.description !== undefined ? { description: p.description } : {}),
...(p.poolName !== undefined ? { poolName: p.poolName } : {}),
initialStatus,
};
}));