docs+smoke(v7): visibility section in virtual-llms.md + register/list smoke
Some checks failed
CI/CD / typecheck (pull_request) Successful in 55s
CI/CD / test (pull_request) Successful in 1m11s
CI/CD / lint (pull_request) Successful in 2m49s
CI/CD / smoke (pull_request) Failing after 1m42s
CI/CD / build (pull_request) Successful in 5m37s
CI/CD / publish (pull_request) Has been skipped

Wraps up v7 Stage 3:

- docs/virtual-llms.md gains a "Visibility scope (v7)" section that
  explains public-vs-private semantics, who skips the filter (owner +
  `*` admin), how to grant single-row exceptions via name-scoped RBAC,
  per-row override syntax in mcplocal config, the `--visibility` flag
  on `mcpctl create llm`/`create agent`, and YAML round-trip behavior.
- New smoke (virtual-llm-visibility.smoke.test.ts) publishes one
  public + one private virtual Llm via the registrar against the
  live mcpd and asserts the GET /llms response carries visibility
  + a non-empty ownerId for both, and that GET /llms/<name> returns
  the private row to its owner without 404. Cross-user filtering is
  covered by mcpd's visibility-filter unit tests; smoke proves the
  fields make the round-trip end-to-end.

Will pass once mcpd is rebuilt + deployed via fulldeploy.sh on this
branch (current main is v6, doesn't yet serialize visibility).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Michal
2026-04-29 01:08:03 +01:00
parent 2c98a21323
commit 2b2444a2c5
2 changed files with 300 additions and 2 deletions

View File

@@ -431,10 +431,100 @@ mid-task reverts the row to pending instead of failing the caller.
See [inference-tasks.md](./inference-tasks.md) for the full data
model, async API, lifecycle, RBAC, and CLI surface.
## Visibility scope (v7)
Virtual Llms and Agents now carry an explicit **visibility** field that
decides who can see the row in listings.
| Visibility | Meaning |
|-------------|----------------------------------------------------------------------------------|
| `public` | Visible to anyone with `view:llms` / `view:agents`. Default for hand-created Llms. |
| `private` | Only the **owner** plus principals with a name-scoped grant can see it. Default for virtual Llms and Agents on first publish. |
The owner is whichever user authenticated the publishing
`POST /api/v1/llms/_provider-register` (or `mcpctl create llm`). For
mcplocal that's whichever `~/.mcpctl/credentials` token is on disk.
Legacy rows from before v7 default to `visibility=public, ownerId=NULL`,
so the upgrade is a no-op for everything that already exists.
### Who skips the filter?
Two principals see every row regardless of visibility:
1. The **row owner** (`ownerId === request.userId`).
2. Anyone with a **cross-resource admin** grant — RBAC binding
`{ resource: '*' }`. Operationally this is the SRE / cluster admin.
A plain `view:llms` resource grant is *not* the same as admin: it's a
RBAC wildcard for name-scoping (you can name any Llm), but the
visibility filter still applies on top. This is the v7 split that
prevents a user with `view:llms` from enumerating every developer's
private virtual Llm.
### Granting a single-row exception
When alice wants bob to see her private virtual Llm `alice-vllm-local`
without making it public, she binds:
```sh
mcpctl create rbac bob view:llms --name alice-vllm-local
```
Same shape as any other name-scoped binding. Removing the binding
flips bob back to "row not found".
### Publishing as private from mcplocal
mcplocal defaults to `private` for every published provider and agent.
Override per-row in `~/.mcpctl/config.json`:
```jsonc
{
"llm": {
"providers": [
{ "name": "vllm-local", "type": "vllm", "model": "...", "publish": true,
"visibility": "private" }, // default; explicit for clarity
{ "name": "shared-qwen", "type": "vllm", "model": "...", "publish": true,
"visibility": "public" } // every team member can chat with it
]
},
"agents": [
{ "name": "local-coder", "llm": "vllm-local",
"visibility": "private" } // private agents pinned to private Llms
]
}
```
On a sticky reconnect (`providerSessionId` matches an existing row)
the visibility is **only** updated when the publisher explicitly sends
it — leaving the field off keeps whatever the row already has,
including any field admin set out-of-band.
### Hand-created Llms
`mcpctl create llm` defaults to `public` (matches pre-v7 behavior).
Pass `--visibility private` to opt in:
```sh
mcpctl create llm my-key --type openai --model gpt-4o \
--api-key-ref my-secret/key --visibility private
```
The same `--visibility` flag is on `mcpctl create agent`.
### CLI surface
`mcpctl get llm` and `mcpctl get agent` show a `VISIBILITY` column.
YAML round-trips cleanly: `mcpctl get llm X -o yaml | mcpctl apply -f -`
preserves visibility, and `ownerId` is stripped from the apply doc
because it's server-side state (the apply re-stamps the ownerId of the
authenticated caller, not the original creator).
## Roadmap (later stages)
(LB pool by name landed in v4; durable task queue landed in v5.)
- **v6** — multi-instance mcpd via pg `LISTEN/NOTIFY` (replaces the
(LB pool by name landed in v4; durable task queue landed in v5;
visibility scope landed in v7.)
- **v8** — multi-instance mcpd via pg `LISTEN/NOTIFY` (replaces the
per-instance EventEmitter wakeup), per-session worker capacity,
remote cancel protocol over the SSE channel.

View File

@@ -0,0 +1,208 @@
/**
* Smoke: v7 visibility round-trip.
*
* Publishes two virtual Llms via the registrar — one explicitly public,
* one explicitly private — and verifies the GET /api/v1/llms response
* carries the visibility + ownerId fields end-to-end. The cross-user
* filter (private rows hidden from non-owner non-admin) is fully
* covered by mcpd's visibility-filter unit tests; smoke only proves
* the new fields make the round-trip from registrar → mcpd → list
* payload without dropping or being defaulted away.
*/
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import http from 'node:http';
import https from 'node:https';
import { mkdtempSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import {
VirtualLlmRegistrar,
type RegistrarPublishedProvider,
} from '../../src/providers/registrar.js';
import type { LlmProvider, CompletionResult } from '../../src/providers/types.js';
const MCPD_URL = process.env.MCPD_URL ?? 'https://mcpctl.ad.itaz.eu';
const SUFFIX = Date.now().toString(36);
const PUBLIC_NAME = `smoke-vis-public-${SUFFIX}`;
const PRIVATE_NAME = `smoke-vis-private-${SUFFIX}`;
function makeFakeProvider(name: string): LlmProvider {
return {
name,
async complete(): Promise<CompletionResult> {
return {
content: 'ok',
toolCalls: [],
usage: { promptTokens: 1, completionTokens: 1, totalTokens: 2 },
finishReason: 'stop',
};
},
async listModels() { return []; },
async isAvailable() { return true; },
};
}
function healthz(url: string, timeoutMs = 5000): Promise<boolean> {
return new Promise((resolve) => {
const parsed = new URL(`${url.replace(/\/$/, '')}/healthz`);
const driver = parsed.protocol === 'https:' ? https : http;
const req = driver.get(
{
hostname: parsed.hostname,
port: parsed.port || (parsed.protocol === 'https:' ? 443 : 80),
path: parsed.pathname,
timeout: timeoutMs,
},
(res) => { resolve((res.statusCode ?? 500) < 500); res.resume(); },
);
req.on('error', () => resolve(false));
req.on('timeout', () => { req.destroy(); resolve(false); });
});
}
function readToken(): string | null {
try {
const home = process.env.HOME ?? '';
const path = `${home}/.mcpctl/credentials`;
// eslint-disable-next-line @typescript-eslint/no-require-imports
const fs = require('node:fs') as typeof import('node:fs');
if (!fs.existsSync(path)) return null;
const raw = fs.readFileSync(path, 'utf-8');
const parsed = JSON.parse(raw) as { token?: string };
return parsed.token ?? null;
} catch {
return null;
}
}
interface HttpResponse { status: number; body: string }
function httpRequest(method: string, urlStr: string): Promise<HttpResponse> {
return new Promise((resolve, reject) => {
const tokenRaw = readToken();
const parsed = new URL(urlStr);
const driver = parsed.protocol === 'https:' ? https : http;
const headers: Record<string, string> = {
Accept: 'application/json',
...(tokenRaw !== null ? { Authorization: `Bearer ${tokenRaw}` } : {}),
};
const req = driver.request({
hostname: parsed.hostname,
port: parsed.port || (parsed.protocol === 'https:' ? 443 : 80),
path: parsed.pathname + parsed.search,
method,
headers,
timeout: 30_000,
}, (res) => {
const chunks: Buffer[] = [];
res.on('data', (c: Buffer) => chunks.push(c));
res.on('end', () => {
resolve({ status: res.statusCode ?? 0, body: Buffer.concat(chunks).toString('utf-8') });
});
});
req.on('error', reject);
req.on('timeout', () => { req.destroy(); reject(new Error(`httpRequest timeout: ${method} ${urlStr}`)); });
req.end();
});
}
let mcpdUp = false;
let registrar: VirtualLlmRegistrar | null = null;
let tempDir: string;
interface LlmListRow {
id: string;
name: string;
visibility?: 'public' | 'private';
ownerId?: string | null;
}
describe('virtual-LLM smoke — visibility (v7)', () => {
beforeAll(async () => {
mcpdUp = await healthz(MCPD_URL);
if (!mcpdUp) {
// eslint-disable-next-line no-console
console.warn(`\n ○ visibility smoke: skipped — ${MCPD_URL}/healthz unreachable.\n`);
return;
}
if (readToken() === null) {
mcpdUp = false;
// eslint-disable-next-line no-console
console.warn('\n ○ visibility smoke: skipped — no ~/.mcpctl/credentials.\n');
return;
}
tempDir = mkdtempSync(join(tmpdir(), 'mcpctl-vis-smoke-'));
}, 20_000);
afterAll(async () => {
if (registrar !== null) registrar.stop();
if (tempDir !== undefined) rmSync(tempDir, { recursive: true, force: true });
if (mcpdUp) {
const list = await httpRequest('GET', `${MCPD_URL}/api/v1/llms`);
if (list.status === 200) {
const rows = JSON.parse(list.body) as LlmListRow[];
for (const target of [PUBLIC_NAME, PRIVATE_NAME]) {
const row = rows.find((r) => r.name === target);
if (row !== undefined) {
await httpRequest('DELETE', `${MCPD_URL}/api/v1/llms/${row.id}`);
}
}
}
}
});
it('publishes one public + one private virtual Llm and the list payload reflects both', async () => {
if (!mcpdUp) return;
const token = readToken();
if (token === null) return;
const published: RegistrarPublishedProvider[] = [
{ provider: makeFakeProvider(PUBLIC_NAME), type: 'openai', model: 'fake-vis', tier: 'fast', visibility: 'public' },
{ provider: makeFakeProvider(PRIVATE_NAME), type: 'openai', model: 'fake-vis', tier: 'fast', visibility: 'private' },
];
registrar = new VirtualLlmRegistrar({
mcpdUrl: MCPD_URL,
token,
publishedProviders: published,
sessionFilePath: join(tempDir, 'session'),
log: { info: () => {}, warn: () => {}, error: () => {} },
heartbeatIntervalMs: 60_000,
});
await registrar.start();
expect(registrar.getSessionId()).not.toBeNull();
await new Promise((r) => setTimeout(r, 400));
const res = await httpRequest('GET', `${MCPD_URL}/api/v1/llms`);
expect(res.status).toBe(200);
const rows = JSON.parse(res.body) as LlmListRow[];
const pub = rows.find((r) => r.name === PUBLIC_NAME);
expect(pub, `${PUBLIC_NAME} must be visible to its owner`).toBeDefined();
expect(pub!.visibility).toBe('public');
// ownerId is the auth principal that ran register; non-empty proves
// mcpd actually stamped it on the row (otherwise the v7 register
// path would have left it NULL = legacy public).
expect(typeof pub!.ownerId).toBe('string');
expect((pub!.ownerId ?? '').length).toBeGreaterThan(0);
const priv = rows.find((r) => r.name === PRIVATE_NAME);
expect(priv, `${PRIVATE_NAME} must be visible to its owner (visibility filter is owner-bypass)`).toBeDefined();
expect(priv!.visibility).toBe('private');
expect(typeof priv!.ownerId).toBe('string');
expect((priv!.ownerId ?? '').length).toBeGreaterThan(0);
// Same publisher, same session — both rows must share the same owner.
expect(priv!.ownerId).toBe(pub!.ownerId);
}, 30_000);
it('GET /api/v1/llms/<name> returns the row to its owner without 404', async () => {
if (!mcpdUp) return;
// Owner is calling — visibility filter must let the row through. A
// 404 here would indicate the service-layer filter is wrongly hiding
// it from the very user who created it.
const res = await httpRequest('GET', `${MCPD_URL}/api/v1/llms/${PRIVATE_NAME}`);
expect(res.status).toBe(200);
const row = JSON.parse(res.body) as LlmListRow;
expect(row.name).toBe(PRIVATE_NAME);
expect(row.visibility).toBe('private');
}, 30_000);
});