docs+smoke(v7): visibility section in virtual-llms.md + register/list smoke
Some checks failed
CI/CD / typecheck (pull_request) Successful in 55s
CI/CD / test (pull_request) Successful in 1m11s
CI/CD / lint (pull_request) Successful in 2m49s
CI/CD / smoke (pull_request) Failing after 1m42s
CI/CD / build (pull_request) Successful in 5m37s
CI/CD / publish (pull_request) Has been skipped
Some checks failed
CI/CD / typecheck (pull_request) Successful in 55s
CI/CD / test (pull_request) Successful in 1m11s
CI/CD / lint (pull_request) Successful in 2m49s
CI/CD / smoke (pull_request) Failing after 1m42s
CI/CD / build (pull_request) Successful in 5m37s
CI/CD / publish (pull_request) Has been skipped
Wraps up v7 Stage 3: - docs/virtual-llms.md gains a "Visibility scope (v7)" section that explains public-vs-private semantics, who skips the filter (owner + `*` admin), how to grant single-row exceptions via name-scoped RBAC, per-row override syntax in mcplocal config, the `--visibility` flag on `mcpctl create llm`/`create agent`, and YAML round-trip behavior. - New smoke (virtual-llm-visibility.smoke.test.ts) publishes one public + one private virtual Llm via the registrar against the live mcpd and asserts the GET /llms response carries visibility + a non-empty ownerId for both, and that GET /llms/<name> returns the private row to its owner without 404. Cross-user filtering is covered by mcpd's visibility-filter unit tests; smoke proves the fields make the round-trip end-to-end. Will pass once mcpd is rebuilt + deployed via fulldeploy.sh on this branch (current main is v6, doesn't yet serialize visibility). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -431,10 +431,100 @@ mid-task reverts the row to pending instead of failing the caller.
|
|||||||
See [inference-tasks.md](./inference-tasks.md) for the full data
|
See [inference-tasks.md](./inference-tasks.md) for the full data
|
||||||
model, async API, lifecycle, RBAC, and CLI surface.
|
model, async API, lifecycle, RBAC, and CLI surface.
|
||||||
|
|
||||||
|
## Visibility scope (v7)
|
||||||
|
|
||||||
|
Virtual Llms and Agents now carry an explicit **visibility** field that
|
||||||
|
decides who can see the row in listings.
|
||||||
|
|
||||||
|
| Visibility | Meaning |
|
||||||
|
|-------------|----------------------------------------------------------------------------------|
|
||||||
|
| `public` | Visible to anyone with `view:llms` / `view:agents`. Default for hand-created Llms. |
|
||||||
|
| `private` | Only the **owner** plus principals with a name-scoped grant can see it. Default for virtual Llms and Agents on first publish. |
|
||||||
|
|
||||||
|
The owner is whichever user authenticated the publishing
|
||||||
|
`POST /api/v1/llms/_provider-register` (or `mcpctl create llm`). For
|
||||||
|
mcplocal that's whichever `~/.mcpctl/credentials` token is on disk.
|
||||||
|
Legacy rows from before v7 default to `visibility=public, ownerId=NULL`,
|
||||||
|
so the upgrade is a no-op for everything that already exists.
|
||||||
|
|
||||||
|
### Who skips the filter?
|
||||||
|
|
||||||
|
Two principals see every row regardless of visibility:
|
||||||
|
|
||||||
|
1. The **row owner** (`ownerId === request.userId`).
|
||||||
|
2. Anyone with a **cross-resource admin** grant — RBAC binding
|
||||||
|
`{ resource: '*' }`. Operationally this is the SRE / cluster admin.
|
||||||
|
|
||||||
|
A plain `view:llms` resource grant is *not* the same as admin: it's a
|
||||||
|
RBAC wildcard for name-scoping (you can name any Llm), but the
|
||||||
|
visibility filter still applies on top. This is the v7 split that
|
||||||
|
prevents a user with `view:llms` from enumerating every developer's
|
||||||
|
private virtual Llm.
|
||||||
|
|
||||||
|
### Granting a single-row exception
|
||||||
|
|
||||||
|
When alice wants bob to see her private virtual Llm `alice-vllm-local`
|
||||||
|
without making it public, she binds:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
mcpctl create rbac bob view:llms --name alice-vllm-local
|
||||||
|
```
|
||||||
|
|
||||||
|
Same shape as any other name-scoped binding. Removing the binding
|
||||||
|
flips bob back to "row not found".
|
||||||
|
|
||||||
|
### Publishing as private from mcplocal
|
||||||
|
|
||||||
|
mcplocal defaults to `private` for every published provider and agent.
|
||||||
|
Override per-row in `~/.mcpctl/config.json`:
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
{
|
||||||
|
"llm": {
|
||||||
|
"providers": [
|
||||||
|
{ "name": "vllm-local", "type": "vllm", "model": "...", "publish": true,
|
||||||
|
"visibility": "private" }, // default; explicit for clarity
|
||||||
|
{ "name": "shared-qwen", "type": "vllm", "model": "...", "publish": true,
|
||||||
|
"visibility": "public" } // every team member can chat with it
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"agents": [
|
||||||
|
{ "name": "local-coder", "llm": "vllm-local",
|
||||||
|
"visibility": "private" } // private agents pinned to private Llms
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
On a sticky reconnect (`providerSessionId` matches an existing row)
|
||||||
|
the visibility is **only** updated when the publisher explicitly sends
|
||||||
|
it — leaving the field off keeps whatever the row already has,
|
||||||
|
including any field admin set out-of-band.
|
||||||
|
|
||||||
|
### Hand-created Llms
|
||||||
|
|
||||||
|
`mcpctl create llm` defaults to `public` (matches pre-v7 behavior).
|
||||||
|
Pass `--visibility private` to opt in:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
mcpctl create llm my-key --type openai --model gpt-4o \
|
||||||
|
--api-key-ref my-secret/key --visibility private
|
||||||
|
```
|
||||||
|
|
||||||
|
The same `--visibility` flag is on `mcpctl create agent`.
|
||||||
|
|
||||||
|
### CLI surface
|
||||||
|
|
||||||
|
`mcpctl get llm` and `mcpctl get agent` show a `VISIBILITY` column.
|
||||||
|
YAML round-trips cleanly: `mcpctl get llm X -o yaml | mcpctl apply -f -`
|
||||||
|
preserves visibility, and `ownerId` is stripped from the apply doc
|
||||||
|
because it's server-side state (the apply re-stamps the ownerId of the
|
||||||
|
authenticated caller, not the original creator).
|
||||||
|
|
||||||
## Roadmap (later stages)
|
## Roadmap (later stages)
|
||||||
|
|
||||||
(LB pool by name landed in v4; durable task queue landed in v5.)
|
(LB pool by name landed in v4; durable task queue landed in v5;
|
||||||
- **v6** — multi-instance mcpd via pg `LISTEN/NOTIFY` (replaces the
|
visibility scope landed in v7.)
|
||||||
|
- **v8** — multi-instance mcpd via pg `LISTEN/NOTIFY` (replaces the
|
||||||
per-instance EventEmitter wakeup), per-session worker capacity,
|
per-instance EventEmitter wakeup), per-session worker capacity,
|
||||||
remote cancel protocol over the SSE channel.
|
remote cancel protocol over the SSE channel.
|
||||||
|
|
||||||
|
|||||||
208
src/mcplocal/tests/smoke/virtual-llm-visibility.smoke.test.ts
Normal file
208
src/mcplocal/tests/smoke/virtual-llm-visibility.smoke.test.ts
Normal file
@@ -0,0 +1,208 @@
|
|||||||
|
/**
|
||||||
|
* Smoke: v7 visibility round-trip.
|
||||||
|
*
|
||||||
|
* Publishes two virtual Llms via the registrar — one explicitly public,
|
||||||
|
* one explicitly private — and verifies the GET /api/v1/llms response
|
||||||
|
* carries the visibility + ownerId fields end-to-end. The cross-user
|
||||||
|
* filter (private rows hidden from non-owner non-admin) is fully
|
||||||
|
* covered by mcpd's visibility-filter unit tests; smoke only proves
|
||||||
|
* the new fields make the round-trip from registrar → mcpd → list
|
||||||
|
* payload without dropping or being defaulted away.
|
||||||
|
*/
|
||||||
|
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||||
|
import http from 'node:http';
|
||||||
|
import https from 'node:https';
|
||||||
|
import { mkdtempSync, rmSync } from 'node:fs';
|
||||||
|
import { tmpdir } from 'node:os';
|
||||||
|
import { join } from 'node:path';
|
||||||
|
import {
|
||||||
|
VirtualLlmRegistrar,
|
||||||
|
type RegistrarPublishedProvider,
|
||||||
|
} from '../../src/providers/registrar.js';
|
||||||
|
import type { LlmProvider, CompletionResult } from '../../src/providers/types.js';
|
||||||
|
|
||||||
|
const MCPD_URL = process.env.MCPD_URL ?? 'https://mcpctl.ad.itaz.eu';
|
||||||
|
const SUFFIX = Date.now().toString(36);
|
||||||
|
const PUBLIC_NAME = `smoke-vis-public-${SUFFIX}`;
|
||||||
|
const PRIVATE_NAME = `smoke-vis-private-${SUFFIX}`;
|
||||||
|
|
||||||
|
function makeFakeProvider(name: string): LlmProvider {
|
||||||
|
return {
|
||||||
|
name,
|
||||||
|
async complete(): Promise<CompletionResult> {
|
||||||
|
return {
|
||||||
|
content: 'ok',
|
||||||
|
toolCalls: [],
|
||||||
|
usage: { promptTokens: 1, completionTokens: 1, totalTokens: 2 },
|
||||||
|
finishReason: 'stop',
|
||||||
|
};
|
||||||
|
},
|
||||||
|
async listModels() { return []; },
|
||||||
|
async isAvailable() { return true; },
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function healthz(url: string, timeoutMs = 5000): Promise<boolean> {
|
||||||
|
return new Promise((resolve) => {
|
||||||
|
const parsed = new URL(`${url.replace(/\/$/, '')}/healthz`);
|
||||||
|
const driver = parsed.protocol === 'https:' ? https : http;
|
||||||
|
const req = driver.get(
|
||||||
|
{
|
||||||
|
hostname: parsed.hostname,
|
||||||
|
port: parsed.port || (parsed.protocol === 'https:' ? 443 : 80),
|
||||||
|
path: parsed.pathname,
|
||||||
|
timeout: timeoutMs,
|
||||||
|
},
|
||||||
|
(res) => { resolve((res.statusCode ?? 500) < 500); res.resume(); },
|
||||||
|
);
|
||||||
|
req.on('error', () => resolve(false));
|
||||||
|
req.on('timeout', () => { req.destroy(); resolve(false); });
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function readToken(): string | null {
|
||||||
|
try {
|
||||||
|
const home = process.env.HOME ?? '';
|
||||||
|
const path = `${home}/.mcpctl/credentials`;
|
||||||
|
// eslint-disable-next-line @typescript-eslint/no-require-imports
|
||||||
|
const fs = require('node:fs') as typeof import('node:fs');
|
||||||
|
if (!fs.existsSync(path)) return null;
|
||||||
|
const raw = fs.readFileSync(path, 'utf-8');
|
||||||
|
const parsed = JSON.parse(raw) as { token?: string };
|
||||||
|
return parsed.token ?? null;
|
||||||
|
} catch {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
interface HttpResponse { status: number; body: string }
|
||||||
|
|
||||||
|
function httpRequest(method: string, urlStr: string): Promise<HttpResponse> {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const tokenRaw = readToken();
|
||||||
|
const parsed = new URL(urlStr);
|
||||||
|
const driver = parsed.protocol === 'https:' ? https : http;
|
||||||
|
const headers: Record<string, string> = {
|
||||||
|
Accept: 'application/json',
|
||||||
|
...(tokenRaw !== null ? { Authorization: `Bearer ${tokenRaw}` } : {}),
|
||||||
|
};
|
||||||
|
const req = driver.request({
|
||||||
|
hostname: parsed.hostname,
|
||||||
|
port: parsed.port || (parsed.protocol === 'https:' ? 443 : 80),
|
||||||
|
path: parsed.pathname + parsed.search,
|
||||||
|
method,
|
||||||
|
headers,
|
||||||
|
timeout: 30_000,
|
||||||
|
}, (res) => {
|
||||||
|
const chunks: Buffer[] = [];
|
||||||
|
res.on('data', (c: Buffer) => chunks.push(c));
|
||||||
|
res.on('end', () => {
|
||||||
|
resolve({ status: res.statusCode ?? 0, body: Buffer.concat(chunks).toString('utf-8') });
|
||||||
|
});
|
||||||
|
});
|
||||||
|
req.on('error', reject);
|
||||||
|
req.on('timeout', () => { req.destroy(); reject(new Error(`httpRequest timeout: ${method} ${urlStr}`)); });
|
||||||
|
req.end();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
let mcpdUp = false;
|
||||||
|
let registrar: VirtualLlmRegistrar | null = null;
|
||||||
|
let tempDir: string;
|
||||||
|
|
||||||
|
interface LlmListRow {
|
||||||
|
id: string;
|
||||||
|
name: string;
|
||||||
|
visibility?: 'public' | 'private';
|
||||||
|
ownerId?: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('virtual-LLM smoke — visibility (v7)', () => {
|
||||||
|
beforeAll(async () => {
|
||||||
|
mcpdUp = await healthz(MCPD_URL);
|
||||||
|
if (!mcpdUp) {
|
||||||
|
// eslint-disable-next-line no-console
|
||||||
|
console.warn(`\n ○ visibility smoke: skipped — ${MCPD_URL}/healthz unreachable.\n`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (readToken() === null) {
|
||||||
|
mcpdUp = false;
|
||||||
|
// eslint-disable-next-line no-console
|
||||||
|
console.warn('\n ○ visibility smoke: skipped — no ~/.mcpctl/credentials.\n');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
tempDir = mkdtempSync(join(tmpdir(), 'mcpctl-vis-smoke-'));
|
||||||
|
}, 20_000);
|
||||||
|
|
||||||
|
afterAll(async () => {
|
||||||
|
if (registrar !== null) registrar.stop();
|
||||||
|
if (tempDir !== undefined) rmSync(tempDir, { recursive: true, force: true });
|
||||||
|
if (mcpdUp) {
|
||||||
|
const list = await httpRequest('GET', `${MCPD_URL}/api/v1/llms`);
|
||||||
|
if (list.status === 200) {
|
||||||
|
const rows = JSON.parse(list.body) as LlmListRow[];
|
||||||
|
for (const target of [PUBLIC_NAME, PRIVATE_NAME]) {
|
||||||
|
const row = rows.find((r) => r.name === target);
|
||||||
|
if (row !== undefined) {
|
||||||
|
await httpRequest('DELETE', `${MCPD_URL}/api/v1/llms/${row.id}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('publishes one public + one private virtual Llm and the list payload reflects both', async () => {
|
||||||
|
if (!mcpdUp) return;
|
||||||
|
const token = readToken();
|
||||||
|
if (token === null) return;
|
||||||
|
const published: RegistrarPublishedProvider[] = [
|
||||||
|
{ provider: makeFakeProvider(PUBLIC_NAME), type: 'openai', model: 'fake-vis', tier: 'fast', visibility: 'public' },
|
||||||
|
{ provider: makeFakeProvider(PRIVATE_NAME), type: 'openai', model: 'fake-vis', tier: 'fast', visibility: 'private' },
|
||||||
|
];
|
||||||
|
registrar = new VirtualLlmRegistrar({
|
||||||
|
mcpdUrl: MCPD_URL,
|
||||||
|
token,
|
||||||
|
publishedProviders: published,
|
||||||
|
sessionFilePath: join(tempDir, 'session'),
|
||||||
|
log: { info: () => {}, warn: () => {}, error: () => {} },
|
||||||
|
heartbeatIntervalMs: 60_000,
|
||||||
|
});
|
||||||
|
await registrar.start();
|
||||||
|
expect(registrar.getSessionId()).not.toBeNull();
|
||||||
|
await new Promise((r) => setTimeout(r, 400));
|
||||||
|
|
||||||
|
const res = await httpRequest('GET', `${MCPD_URL}/api/v1/llms`);
|
||||||
|
expect(res.status).toBe(200);
|
||||||
|
const rows = JSON.parse(res.body) as LlmListRow[];
|
||||||
|
|
||||||
|
const pub = rows.find((r) => r.name === PUBLIC_NAME);
|
||||||
|
expect(pub, `${PUBLIC_NAME} must be visible to its owner`).toBeDefined();
|
||||||
|
expect(pub!.visibility).toBe('public');
|
||||||
|
// ownerId is the auth principal that ran register; non-empty proves
|
||||||
|
// mcpd actually stamped it on the row (otherwise the v7 register
|
||||||
|
// path would have left it NULL = legacy public).
|
||||||
|
expect(typeof pub!.ownerId).toBe('string');
|
||||||
|
expect((pub!.ownerId ?? '').length).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
const priv = rows.find((r) => r.name === PRIVATE_NAME);
|
||||||
|
expect(priv, `${PRIVATE_NAME} must be visible to its owner (visibility filter is owner-bypass)`).toBeDefined();
|
||||||
|
expect(priv!.visibility).toBe('private');
|
||||||
|
expect(typeof priv!.ownerId).toBe('string');
|
||||||
|
expect((priv!.ownerId ?? '').length).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
// Same publisher, same session — both rows must share the same owner.
|
||||||
|
expect(priv!.ownerId).toBe(pub!.ownerId);
|
||||||
|
}, 30_000);
|
||||||
|
|
||||||
|
it('GET /api/v1/llms/<name> returns the row to its owner without 404', async () => {
|
||||||
|
if (!mcpdUp) return;
|
||||||
|
// Owner is calling — visibility filter must let the row through. A
|
||||||
|
// 404 here would indicate the service-layer filter is wrongly hiding
|
||||||
|
// it from the very user who created it.
|
||||||
|
const res = await httpRequest('GET', `${MCPD_URL}/api/v1/llms/${PRIVATE_NAME}`);
|
||||||
|
expect(res.status).toBe(200);
|
||||||
|
const row = JSON.parse(res.body) as LlmListRow;
|
||||||
|
expect(row.name).toBe(PRIVATE_NAME);
|
||||||
|
expect(row.visibility).toBe('private');
|
||||||
|
}, 30_000);
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user