feat(mcpd+cli+mcplocal): /llms/<name>/members + POOL column + --pool-name (v4 Stage 2)
Surfaces the v4 pool model end-to-end: - mcpd: GET /api/v1/llms/:name/members returns the effective pool the named anchor belongs to, plus aggregate stats (size, activeCount, explicit vs implicit pool key). RBAC inherits from `view:llms` — same as the single-Llm route. Members are full LlmView shapes so callers don't need a second roundtrip to render the pool block. - mcpd: VirtualLlmService.register accepts an optional `poolName` on RegisterProviderInput; the route's `coerceProviderInput` validates the same character set as CreateLlmSchema.poolName. Backwards compatible — older mcplocals that don't send the field continue to publish solo Llms. - CLI `get llm` table: new POOL column right after NAME. Solo rows show "-" so the "no pool / pool of 1" case is unambiguous (per user direction "make sure we see it, prominently visible and impossible to mistake"). - CLI `describe llm`: fetches /members and renders a Pool block at the top of the detail view when the row is in an explicit pool OR when its implicit pool has size > 1. Each member line shows kind/status; the anchor row gets "← this row". Block is suppressed for solo rows so describe stays compact in the common case. - CLI `create llm --pool-name <name>` flag and apply schema both accept the new field. Yaml round-trip preserves it: get -o yaml emits `poolName: <name>`, apply -f re-imports it without diff. Verified end-to-end against the live mcpd. - mcplocal: LlmProviderFileEntry gains optional `poolName`; main.ts and registrar.ts thread it through into the register payload. Use case for distributed inference: each user's mcplocal picks a unique `name` (e.g. `vllm-<host>-qwen3`) but a shared `poolName` (e.g. `user-vllm-qwen3-thinking`); agents see one logical pool that auto-grows as workers come online. - Shell completions: regenerated from source via the existing scripts/generate-completions.ts. `--pool-name` now suggests in fish + bash for `mcpctl create llm`. Tests: +3 new mcpd route tests for /members (explicit pool, solo pool of 1, missing-anchor 404). All suites green: mcpd 868/868 (was 865, +3), mcplocal 723/723, cli 437/437. Stage 3 (next): live smoke against 2 publishers sharing a pool name + docs.
This commit is contained in:
@@ -93,6 +93,18 @@ export interface LlmProviderFileEntry {
|
||||
* - `command`: spawn a shell command (e.g. `systemctl --user start vllm`)
|
||||
*/
|
||||
wake?: WakeRecipe;
|
||||
/**
|
||||
* v4: opt this provider into a load-balanced pool. When set, the
|
||||
* published Llm row carries `poolName` and stacks with any other Llms
|
||||
* (public OR virtual) sharing the same value. Agents pinned to any
|
||||
* pool member dispatch across all healthy members at chat time.
|
||||
*
|
||||
* Convention for distributed-compute setups: each user's mcplocal
|
||||
* picks a unique `name` (e.g. `vllm-<hostname>-qwen3`) but a shared
|
||||
* `poolName` (e.g. `user-vllm-qwen3-thinking`). Result: agents see one
|
||||
* logical pool that auto-grows as more workers come online.
|
||||
*/
|
||||
poolName?: string;
|
||||
}
|
||||
|
||||
export type WakeRecipe =
|
||||
|
||||
@@ -218,6 +218,7 @@ async function maybeStartVirtualLlmRegistrar(
|
||||
};
|
||||
if (entry.tier !== undefined) item.tier = entry.tier;
|
||||
if (entry.wake !== undefined) item.wake = entry.wake;
|
||||
if (entry.poolName !== undefined) item.poolName = entry.poolName;
|
||||
published.push(item);
|
||||
}
|
||||
// v3: forward locally-declared agents alongside the providers. We
|
||||
|
||||
@@ -54,6 +54,12 @@ export interface RegistrarPublishedProvider {
|
||||
* the registrar runs this recipe and waits for the backend to come up.
|
||||
*/
|
||||
wake?: WakeRecipe;
|
||||
/**
|
||||
* v4: optional pool key. When set, the published Llm row carries
|
||||
* `poolName` and stacks with any other Llms sharing the same value.
|
||||
* Agents pinned to any pool member dispatch across all healthy members.
|
||||
*/
|
||||
poolName?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -185,6 +191,7 @@ export class VirtualLlmRegistrar {
|
||||
model: p.model,
|
||||
...(p.tier !== undefined ? { tier: p.tier } : {}),
|
||||
...(p.description !== undefined ? { description: p.description } : {}),
|
||||
...(p.poolName !== undefined ? { poolName: p.poolName } : {}),
|
||||
initialStatus,
|
||||
};
|
||||
}));
|
||||
|
||||
Reference in New Issue
Block a user