71 Commits

Author	SHA1	Message	Date
michal	7fbb827aa5	Merge pull request 'fix(mcplocal): serve OpenAI-safe tool names on the wire' (#124) from fix/wire-safe-tool-names into main	2026-08-25 19:23:19 +00:00
Michal	eb0e97e76e	test(mcplocal): end-to-end wire-name coverage on the project endpoint ... Drives /projects/:name/mcp over the real Streamable HTTP transport with a fake websearch upstream: tools/list must serve `websearch_fetch_content` (every name matching the OpenAI function-name charset), calling that wire name must reach the upstream as bare `fetch_content`, and a legacy client echoing the canonical `websearch/fetch_content` must still route. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JaFvfHrQyUKCGv6o3N2Wir	2026-08-25 20:23:09 +01:00
Michal	065ce02a60	fix(mcplocal): serve OpenAI-safe tool names on the wire ... The proxy namespaces tools as `server/tool` (and favourite-index presents `favourite/<tool>` / `all/<server>/<tool>`). A `/` is not a valid character in an OpenAI-style function name, so hosts that forward MCP tool names verbatim as LLM function names depend on the model faithfully echoing an illegal name. LibreChat did exactly that: deepseek-v4-flash intermittently dropped the `websearch/` prefix, LibreChat's registry lookup failed, and it reported "This tool's MCP server is temporarily unavailable" while nothing was down — the calls never reached mcplocal at all (confirmed against the AuditEvent table, 2026-08-25). Claude Code and the pi extension only dodge this because they sanitize names client-side. Fix at the HTTP boundary only: a WireNameCodec rewrites tools/list responses to wire-safe names (`/` -> `_`, exact-match reverse map, deterministic suffix on collision) and maps tools/call names back before routing. Wired into both /mcp and /projects/:name/mcp. Everything inside the proxy — routing maps, plugins, favourites config, audit events — keeps canonical names, and unknown inbound names (legacy clients echoing slash names, virtual tools) pass through unchanged, so existing clients keep working. Codecs are keyed per project and outlive the router cache TTL so a client can call a tool it listed minutes earlier; after a restart the client's initialize-time tools/list repopulates the map. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JaFvfHrQyUKCGv6o3N2Wir	2026-08-25 20:17:33 +01:00
michal	eb3e558a44	Merge pull request 'feat(secrets): survive OpenBao outages and report real backend health' (#115) from feat/openbao-resilience into main	2026-08-20 21:37:57 +00:00
Michal	fe9987cefd	test(secrets): smoke-cover backend health honesty + cache correctness ... Two of these four assertions currently FAIL against the live cluster, which is the point: mcpd there still renders `Secrets: bao-k8s* ✓` from the rotation field, and its JSON status carries no live/ready pair. Per the project rule the fix is to deploy, not to relax the assertion. The status check specifically rejects a bare "name ✓" with no qualifier — that string is the old rendering and proves the probe was never consulted. Deliberately does not take the real OpenBao down. Simulating an outage against shared infrastructure to satisfy a test would be worse than the bug it covers; the outage paths are unit-tested with an injected clock. Docs: adds a Reliability section (request hardening, the stale-while-error table, the cold-cache gap and why persisting values is not the answer, live-vs-ready) and replaces the stale "Kubernetes ServiceAccount auth is not shipped yet" note — it shipped in 5152066, and it is what the live backend has used since June. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018vybEitX4FykeMatKe5Xki	2026-08-20 22:16:04 +01:00
Michal	03350856ea	fix(mcplocal): make the paginated-result contract usable by any MCP client ... UniFi tools were unusable from non-Claude agents. A tool result over 2000 chars is replaced with a table of contents and re-read by calling the tool again with _resultId/_section, but those params were never declared on the tool's inputSchema — and unifi-network and my-grafana ship `additionalProperties: false`, so for a client that validates arguments the drill-down call was illegal and the data unreachable. The stub's own wording made it worse: "Use section parameter" names a param that does not exist; sending it fell through to the upstream, re-paginated, and minted a fresh _resultId. An unbounded loop. UniFi took the blame because it is the one server whose results always trip the threshold: get_devices was 11,718 chars, get_clients 136,696 across 19 pages with a 92-char page-1. Grafana and Gitea return compact results. - content-pipeline gains onToolsList, declaring _resultId/_section on every tool it can paginate (gate tools excluded — they are intercepted before the pipeline runs). additionalProperties stays false: a property listed in `properties` is already legal under it, so declaring is enough and the upstream keeps its typo protection. - createDefaultPlugin wired only the gate's onToolsList, so the pipeline's would have been dropped on the floor. Both now chain, gate first. - _resultId without _section re-shows the table of contents instead of forwarding an unknown argument to a strict upstream. - The stub names the tool, the live _resultId and a real section id. - Nested MCP envelopes are collapsed. A server fronting another MCP server returns the inner result wrapped in its own content/structuredContent pair, so the payload arrives two or three times over. A layer is peeled only when it carries nothing the inner value lacks, which keeps it lossless. Live against the sre project: get_devices 11,718 -> 5,210 chars and no longer paginates at all; get_clients 136,696 -> 62,358 across 8 pages instead of 19. - deploy/mcplocal.service shipped MCPLOCAL_MCPD_URL=http://10.0.0.194:3100, which is dead — every working machine had a hand-written drop-in. Points at the k8s ingress now. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JNXFvxanvM6uiFcb4Mp3xU	2026-08-14 23:27:36 +01:00
Michal	bbd2195c64	test(mcplocal): prove chat SSE streams live through the proxy, not buffered ... `mcpctl chat reviewer` showed nothing until the turn finished, then dumped the whole answer at once. mcpd streams token deltas and the CLI renders them incrementally — the sole buffering point was mcplocal's catch-all /api/v1/* proxy reading the whole SSE body via res.text() before replying. The previous commit (cherry-picked from feat/agentic-teams) pipes the body through instead; this one adds the cover that was missing: - proxy-long-running.test.ts: a progressive-delivery test in which the stand-in mcpd withholds its final frame until the client has observed the first delta through the proxy. A buffering proxy cannot satisfy that ordering — verified: the test fails in 3s (no hang) against a res.text() proxy and passes against the piped one. The existing SSE test used inject(), which collects the whole body and so passes either way. - agent-chat.smoke.test.ts: a live smoke that posts through mcplocal on localhost:3200 (the path `mcpctl chat` actually takes — every other chat smoke uses --direct and bypasses the proxy entirely) and asserts delta frames arrive spread across the generation window, not in one burst at stream end. Uses its own agent: the shared smoke agent pins replies to a single token, too short to tell live streaming from a buffer dump. Also settles the strict-boolean-expressions lint on the auth-header check the streaming split touched. Local: workspace 2546 passed, proxy suite 10/10, smoke file loads + self-skips. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016dNpnBqyyz9GxfznVcX2sP	2026-08-10 22:27:21 +01:00
Michal	5a8185d7c9	fix(mcplocal): stop the 30s proxy timeout killing agent turns ... `mcpctl chat <agent>` failed with HTTP 503 {"error":"service_unavailable","message":"Cannot reach mcpd daemon. Is it running?"} while mcpd was answering /healthz in 32ms. The message was wrong in a way that cost real debugging time: mcplocal was reaching mcpd fine and giving up after 30s. journalctl shows the signature plainly — statusCode 503 with responseTime 30003.87 on POST /api/v1/agents/reviewer/chat. This blocks the agentic-teams epic outright. An agent turn is a multi-turn tool-use loop that runs for minutes by design, so a 30s ceiling on the chat path is not a safety net, it is a guaranteed failure for every non-trivial turn. Three defects, all in the same path: 1. One blanket budget for every forwarded route. DEFAULT_TIMEOUT_MS = 30_000 is right for CRUD and wrong for chat. Chat, project chat, llm infer and inference-task streams now get LONG_RUNNING_TIMEOUT_MS (600_000, override with MCPLOCAL_LONG_TIMEOUT_MS) — matching STREAM_TIMEOUT_MS, which the CLI already allowed. mcplocal in the middle was the binding constraint. 2. Timeouts were reported as connection failures. Split UpstreamTimeoutError out of ConnectionError and map it to 504 with an accurate message that says the daemon IS reachable. ConnectionError still means unreachable and still returns 503. Verified nothing else branches on ConnectionError. 3. SSE was buffered. `forward()` reads the whole body through res.text(), so even turns that finished in time arrived as one blob and the CLI's live token output never appeared. Streaming routes now use forwardStream() and pipe the body straight through, preserving content-type and x-accel-buffering (dropping the latter lets intermediaries re-buffer and reintroduces the stall). Also closes the escape that produced the sibling `500 code:23` failure: the body read in forward() was outside the try, so when mcpd had already written SSE headers the raw DOMException reached Fastify unhandled. Tests: 9 new proxy tests. The two that matter — "does not abort an agent chat that outlives the CRUD budget" and "streams SSE through instead of buffering" — were confirmed to FAIL against the pre-fix behaviour and pass after. Three existing mcpd-client tests asserted the old taxonomy and were updated to assert the new one deliberately. Local: build clean, workspace 2375 passed, lint unchanged at 869. NOT YET LIVE: mcplocal runs from the installed RPM, so this needs a package rebuild + `systemctl --user restart mcplocal` to take effect. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N4wNHWf7xSwnZCWpJcyv9p	2026-08-10 22:19:56 +01:00
Michal	822c1bb047	build: fail the release when smoke tests fail, and fix the SSE test that hung ... release.sh printed `WARNING: Smoke tests failed!` and exited 0. That is how four broken readiness probes shipped unnoticed on 2026-08-10 — the warning scrolled past in the build log and the release reported success. It now exits 1, with `MCPCTL_ALLOW_SMOKE_FAILURE=1` as the escape hatch. The message is explicit that smoke runs LAST, against the installed binary: the package is already published and installed, so the failure reports fleet breakage rather than preventing a bad artifact. Turning the gate on required fixing a latent hang first, or every release would have blocked on it. `security.test.ts > /inspect SSE endpoint …` waited for a response body that by design never ends, so it could only settle via the socket's *inactivity* timeout — and /inspect relays every project's MCP traffic, so during a full smoke run it is never idle. Run alone it passed and looked flaky; run with the suite it failed every time. httpRequest gains `headersOnly`, which resolves on the response headers and hangs up. The assertion only ever needed the status line. Verified: full smoke suite 158/158 (was 157/158 with this test timing out); the gate block lifted verbatim from release.sh exits 1 with a stubbed failing smoke run, and exits 0 reaching subsequent code under MCPCTL_ALLOW_SMOKE_FAILURE=1. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019wUmrfkVQR6CKcYKxENq7k	2026-08-10 17:15:40 +01:00
Michal	b8cedd6262	test(smoke): give the smoke-aws-docs fixture a readiness probe ... health-readiness.smoke.test.ts asserts no RUNNING server lacks a healthCheck.tool. The smoke suite's own shared fixture had none, so the suite reported its own scaffolding as a fleet regression — and because the fixture is in the PROTECTED set of clean-smoke-resources.ts, it is long-lived and failed that assertion on every run. Mirrors the production aws-docs probe (same package): search_documentation with a phrase, 300s interval since the call leaves the cluster. Verified live: with this applied, "every RUNNING server has a readiness probe configured" passes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019wUmrfkVQR6CKcYKxENq7k	2026-08-10 16:59:41 +01:00
Michal	b2547429ca	fix(health): a passing tools/list is live, not healthy ... `mcpctl get instances` showed all eight servers healthy while the UniFi one had never once reached its controller. The default probe is `tools/list`, which MCP servers answer from a static in-process table — no credentials, no upstream, ~3ms. It cannot fail for any reason the user cares about, so it was reporting `healthy` for every process that managed to start. Split the two passes: healthy — readiness: `tools/call` on `healthCheck.tool`. The upstream answered, so the server can actually do its job. live — liveness: `tools/list` only. Process up, upstream unverified. `live` is now the default for any server without a `healthCheck.tool`. It is not a warning; it is an admission that nothing is watching that server. Probe events name which probe ran and which tool ("Readiness check (list_sites) passed"), so the events log distinguishes the two after the fact. Also: - `healthCheck.tool` is optional now, so the timings can be tuned without inventing a readiness probe. - `create server --health-check-tool/-args/-interval/-timeout/ -failure-threshold`, per the rule that everything applyable is a create flag. Merges over a `--from-template` healthCheck rather than replacing it. - `describe instance` explains a `live` verdict instead of leaving it cryptic. - create.ts held a raw NUL byte in a string literal, which made grep treat the whole file as binary and silently skip it. Escaped as `\0`. Verified against the live fleet: with readiness probes configured, my-grafana went unhealthy (Grafana API 403) and my-node-red degraded (connect timeout to a Tailscale address) — both had read healthy for months. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0114dg56YmVacyqhp5fitcTb	2026-08-09 21:23:33 +01:00
Michal	a90dd091f0	fix(smoke): actually clean up smoke-test resources ... 341 of 356 projects on the shared mcpd were smoke-test leftovers, plus 11 never-expiring `smoke-*` mcptokens sitting on the real `mcpctl-development` and `sre` projects. Enough to make the project picker unusable. Two causes, both silent: - `delete project X --force` — `--force` is valid on `create project` but NOT on `delete`, so every cleanup call exited non-zero and deleted nothing. project-llm-ref's afterAll looked correct and had never worked, which is why there were 84 each of smoke-proj-{ok,orphan,none}-*. - mcptoken.smoke had no afterAll at all; its cleanup was an `it()` at the end of the file, so it was skipped whenever an earlier assertion failed. Cleanup now lives in `afterAll` (runs on failure too) and is no longer gated on the health probe: the project is created through mcpd, which can be up when the gateway probe is not, and deleting a project that was never created is a no-op. Adds `pnpm smoke:clean` for the case afterAll cannot cover — a killed process (CI timeout, Ctrl-C, OOM). Dry-run by default, `--yes` to apply. Only touches `smoke-*` names and protects the shared `smoke-data` / `smoke-aws-docs` fixture, which five suites depend on and which must survive a concurrent run. Verified end to end: created a throwaway smoke project, confirmed the dry run lists without deleting, then `--yes` removed it and left all 15 real projects untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017BMXdb2qZbPSh8Q7XpTyjB	2026-08-08 18:49:42 +01:00
Michal	2729580974	feat(gate): route prompt-selection through mcpd's server Llm (credential tiering) ... Cloud/server keys belong at the k8s/mcpd level; mcplocal handles only the user's personal tokens. The gate's prompt-selection used mcplocal's LOCAL provider registry (heavy=anthropic = the user's OAuth token, which is API-gated and 404s), so selection always degraded. Now LlmPromptSelector tries sources in priority order: (1) the project's server Llm via mcpd's inference proxy (POST /api/v1/llms/:name/infer) — cloud/server keys stay at k8s — then (2) the local personal-token provider as fallback. First source that returns valid selection JSON wins. Extracted pickCompletionText (content ?? reasoning_content) + extractSelection helpers. Wiring: GatePluginConfig.llmProvider (from the project), threaded via createDefaultPlugin; handleBeginSession builds serverInfer from ctx.postToMcpd. Verified live: POST .../llms/vllm-current/infer returns valid selection JSON (qwen3-thinking, 4000-token budget). + selector tests (server preferred, local fallback on error, server-only). mcplocal suite green (755). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-24 12:43:48 +01:00
Michal	366b505c00	fix(providers): anthropic surfaces HTTP errors too (gate parity with openai) ... The local mcplocal gate resolves heavy=[anthropic]; its OAuth-gated key 404s the configured opus-4 model, and anthropic.ts (like openai.ts before this series) JSON.parsed the error body as a completion → empty content → the gate's misleading "did not contain valid selection JSON". Reject on status >= 400 with "Anthropic HTTP <status>: <body>" so the true reason (model-not-found / auth) surfaces as the degradedReason. + parity test. Note: this makes the error honest; it does not make the local gate SUCCEED — that needs a working heavy provider (the anthropic OAuth token is fundamentally gated). The qwen3-thinking gate path (k8s mcplocal) is addressed by the budget+reasoning fix in the previous commit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 22:23:37 +01:00
Michal	c9364a2f2a	fix(favourite-index): route favourite/ calls even before tools/list (lazy resolver) ... The presented→canonical resolver was only built during tools/list, so a favourite/<tool> call arriving before any list this session fell through to the namespaced router and 404'd (all/ was fine — it prefix-strips). Real MCP clients list first, but the smoke test's initialize→call flow exposed it. Extract buildPresentation() and lazily rebuild the resolver in onToolCallBefore when a favourite//all/ name isn't resolved yet. + unit test (favourite/ call with no prior list) and realistic listTools() in the smoke routing case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 22:19:05 +01:00
Michal	9dbd2f175d	fix(gate): stop masking backend errors + rescue thinking-model prompt selection ... Two independent, live-reproduced causes of "Smart prompt-selection unavailable (LLM response did not contain valid selection JSON)" in the gate: - Cause B (openai.ts request): the response handler JSON.parsed the body with no res.statusCode check, so a LiteLLM/vLLM HTTP 500 ("Connection error") parsed as an empty completion and surfaced as generic "bad JSON", masking backend-down. Now reject on status >= 400 with "OpenAI HTTP <status>: <body>" so the selector reports the true degradedReason. - Cause A (parseResponse + selector budget): a reasoning model (qwen3-thinking) spends its whole 1024-token budget in reasoning_content and emits content=null, so the {selectedNames} JSON never appears. parseResponse now falls back to reasoning_content (incl. provider_specific_fields) when content is empty, and the selector budget is raised 1024 -> 4000 (env MCPCTL_GATE_SELECT_MAX_TOKENS) so thinking + the JSON answer both fit. Note: mcpd's chat path got a reasoning_content fallback in PR #67; this is the separate mcplocal gate-selection provider, which never did. Tests: openai-provider (500 rejects, reasoning_content capture/preference). mcplocal suite green (749). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 22:10:08 +01:00
Michal	f614e9bb98	feat(proxy): favourite-index tool presentation (favourite/ + all/ + prefer instruction) ... Measured winner from the DGX-Spark bake-off (toolsim.py, 145-tool catalog): a curated favourite/<tool> shortlist + the full all/<server>/<tool> catalog + a load-bearing "prefer favourite/ first" instruction nearly halved wander (37→20) and 2.5x'd first-pick (2→5/8) vs a flat catalog. The instruction is load-bearing; enriching descriptions did not help. - New mcplocal plugin `favourite-index.ts`: composes AFTER gate (no-ops while gated), reshapes the ungated upstream catalog into favourite/ + all/, injects the instruction (onInitialize), and rewrites presented names back to canonical server/tool in onToolCallBefore so normal routing + content-pipeline still run. Gate/agent virtual tools pass through untouched; favourites are upstream-only. - compose.ts: onInitialize now concatenates plugin instructions (was first-non-null) so favindex can contribute its banner alongside the gate's. - Per-project config `Project.favouriteIndex` {enabled, tools[], maxFavourites}; surfaced to the proxy via discovery; wired at project-mcp-endpoint when enabled. - Usage derivation: mcpd tool-usage ranking over tool_call_trace events (normalizing presented names → canonical), GET /api/v1/audit/tool-usage, and `mcpctl favourites suggest|list`. - CLI: `create project` gains --favourite/--favourite-index/--max-favourites; favouriteIndex round-trips through get -o yaml | apply -f. Completions regenerated. - Tests: plugin unit (presentation, rewrite routing, gated no-op, collisions), compose merge, canonicalizeToolName, buildFavouriteIndex, + a live smoke test. - Docs: docs/tool-presentation.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 01:20:30 +01:00
Michal	485b8f613f	feat(reliability): bound + fail over + surface LLM-optional ops ... mcpctl hung on prompt-reading whenever the LLM misbehaved (thinking model = minutes; drifted model = silent failure). Root cause: the gate's begin_session prompt-selection called the LLM with no timeout, its fallback only fired on error and was silent, and it forced the project's vLLM model onto the anthropic heavy provider (so selection failed silently every time). - New withTimeout(run, ms, label): Promise.race + AbortSignal (fetch-based providers cancel). CompletionOptions.signal threaded into anthropic/openai. - Gate begin_session: LLM selection is time-bounded (MCPCTL_GATE_LLM_TIMEOUT_MS, 8s); on timeout/error it falls back to deterministic tag matching, logs [gate] loudly, prepends a ⚠ degraded note to the response, and sets degraded/degradedReason on the audit gate_decision. - Gate selector no longer forces the project vLLM model — uses the heavy provider's own model (fixes the always-silent-fail bug). - Pagination smart-index is time-bounded too (falls back to byte-range pages). - Chat (LLM-essential) surfaces the upstream status+body (names model+reason) instead of "Adapter returned no choice". - docs/reliability.md documents the principle. Tests: with-timeout unit tests; gate degradation test (error → visible ⚠ + deterministic prompts, no hang). mcplocal 737 + mcpd 945 green; tsc clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-19 16:28:34 +01:00
Michal	4c7e648771	fix(smoke): read CLI credentials from ~/.mcpctl (active path) ... passwd smoke read ~/.config/mcpctl/credentials and silently skipped; the CLI stores creds at ~/.mcpctl/credentials. Now verified live against prod (4/4). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 22:41:30 +01:00
Michal	2cceeb7093	feat(passwd): mcpctl passwd + RBAC-gated password change ... Restores the lost `mcpctl passwd` command and builds the backend it needs. Backend (mcpd): - POST /api/v1/users/me/password — self-service change, requires current password. Gated by a new `set-own-password` operation. - PUT /api/v1/users/:id/password — admin reset of another user, gated by edit:users (admins have edit:*). Added users name-resolver for CUID→email. - UserService.setPassword/verifyPassword; UserRepository.update accepts passwordHash + findByIdWithHash. RBAC, no exceptions: self password change is a default, admin-revocable permission. Every new user gets a `self-<id>` RbacDefinition granting `set-own-password`, seeded on create + bootstrap, gated by the `allowSelfPasswordChange` system setting (stored in the mcpctl-system-settings secret, default ON; admins disable globally or revoke per-user). CLI: src/cli/src/commands/passwd.ts (self vs admin paths) + completions. Tests: users-password route tests (8), auth-bootstrap grant assertion, passwd live smoke test. Full suite 2214 passing; zero new lint errors. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 21:55:56 +01:00
Michal	2b2444a2c5	docs+smoke(v7): visibility section in virtual-llms.md + register/list smoke ... Wraps up v7 Stage 3: - docs/virtual-llms.md gains a "Visibility scope (v7)" section that explains public-vs-private semantics, who skips the filter (owner + `*` admin), how to grant single-row exceptions via name-scoped RBAC, per-row override syntax in mcplocal config, the `--visibility` flag on `mcpctl create llm`/`create agent`, and YAML round-trip behavior. - New smoke (virtual-llm-visibility.smoke.test.ts) publishes one public + one private virtual Llm via the registrar against the live mcpd and asserts the GET /llms response carries visibility + a non-empty ownerId for both, and that GET /llms/<name> returns the private row to its owner without 404. Cross-user filtering is covered by mcpd's visibility-filter unit tests; smoke proves the fields make the round-trip end-to-end. Will pass once mcpd is rebuilt + deployed via fulldeploy.sh on this branch (current main is v6, doesn't yet serialize visibility). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-29 01:08:03 +01:00
michal	3071bcee8e	feat: v6 polish — per-publisher namespacing + auto-create project (#71)	2026-04-28 23:33:39 +00:00
Michal	c346b93789	feat(mcplocal): per-publisher namespacing for virtual Llms/Agents (v6 Stage 1) ... Two mcplocals sharing the same config template (`vllm-local-qwen3`) no longer collide on mcpd's cluster-wide unique-name constraint. Each publisher can append a suffix derived from hostname (or any other stable per-host identifier) so the wire-side names become distinct (`vllm-local-qwen3-alice`, `vllm-local-qwen3-bob`). Pair with an explicit `poolName` (v4) and the rows still appear as one logical pool — agents pinned to any member load-balance across both. Config (`~/.mcpctl/config.json`): { "publisher": { "suffix": "auto" } // → os.hostname() sanitized // or { "suffix": "alice" } for explicit override } Or via env: `MCPCTL_PUBLISHER_SUFFIX=alice` (operations override). Resolution order: env var → config.publisher.suffix → empty (legacy behavior, no mangling). Sanitization lowercases, replaces non-`[a-z0-9-]` runs with `-`, strips leading/trailing dashes — the result must satisfy mcpd's name validation, otherwise the register POST would 422. Wire shape: RegistrarPublishedProvider gets an optional `publishName` field. When set, the wire payload's `name` is `publishName` (suffixed); when not, today's `provider.name`. Inbound infer/wake task lookups match `publishName ?? provider.name` so the local registry stays addressable by its original name — SSE frames carrying the suffixed wire name still find their provider. Agents are forwarded with their own suffixed name AND a `llmName` rewritten through the same per-local→wire map so the agent rows pin to the suffixed Llm wire name (otherwise registerVirtualAgents would 404). Tests: 8 new tests covering applyPublisherSuffix (empty, normal, length limit, exact-100) and loadPublisherSuffix (env override, absent, sanitization, dash stripping). Existing registrar tests untouched — no suffix means no behavior change.	2026-04-28 15:54:06 +01:00
Michal	7320b50dac	feat(cli+docs+smoke): inference-task CLI + GC ticker + smoke + docs (v5 Stage 4) ... CLI surface for the durable queue: - `mcpctl get tasks` — table view (ID, STATUS, POOL, LLM, MODEL, STREAM, AGE, WORKER). Aliases `task`, `tasks`, `inference-task`, `inference-tasks` all normalize to the canonical plural so URL construction works uniformly. RESOURCE_ALIASES + completions generator updated. - `mcpctl chat-llm <name> --async -m <msg>` — enqueue and exit. stdout is just the task id (pipeable into `xargs mcpctl get task`); stderr carries human-readable status. REPL mode is rejected for --async (fire-and-forget doesn't make sense without -m). GC ticker in mcpd: 5-min interval. Pending tasks past 1 h queue timeout flip to error with a clear message; terminal tasks past 7 d retention get deleted. Both queries are index-backed. Crash fix uncovered by the smoke: when the async route doesn't await ref.done, a later cancel/error rejected the in-flight Promise as unhandled and crashed mcpd. The route now attaches a no-op `.catch` so the legacy `done` semantic still works for sync callers (chat, direct infer) without taking out the process for async ones. The EnqueueInferOptions also gained an explicit `ownerId` field so the async API can stamp the authenticated user on the row instead of inheriting 'system' from the constructor's resolveOwner — without this, every GET/DELETE from the original caller would 404 due to foreign-owner mismatch. Smoke (tests/smoke/inference-task.smoke.test.ts): 1. POST /inference-tasks while no worker bound → row=pending. 2. Bring a registrar online → bindSession drain claims and dispatches → worker complete()s → row=completed → GET returns the assistant body. 3. Stop worker, enqueue, DELETE → row=cancelled, persisted. docs/inference-tasks.md (new): full data model, lifecycle diagram, async API reference, CLI examples, RBAC table, GC defaults, and the v5 limitations / v6 roadmap. Cross-linked from virtual-llms.md and agents.md. Tests + smoke: mcpd 893/893, mcplocal 723/723, cli 437/437, full smoke 146/146 (was 144, +2 new task smoke). Live mcpd verified via manual curl: enqueue → cancel → re-fetch — no crash, owner scoping returns 404 on foreign ids, GC ticker logs at info when it sweeps. v5 complete: durable queue (Stage 1) + VirtualLlmService rewire (Stage 2) + async API & RBAC (Stage 3) + CLI/GC/smoke/docs (Stage 4).	2026-04-28 15:25:09 +01:00
Michal	137711fdf6	feat(docs+smoke): LB pool live smoke + virtual-llms.md pool semantics (v4 Stage 3) ... Smoke (tests/smoke/llm-pool.smoke.test.ts): two in-process registrars publish virtual Llms with distinct names but a shared poolName, then: 1. /api/v1/llms/<name>/members surfaces both with the correct effective pool key, size, activeCount, and per-member kind/status. 2. Chat through an agent pinned to one pool member dispatches across the pool — verified by running 12 calls and asserting at least one response from each backend (the random-shuffle selection would have to hit only-A or only-B in 12 fair coin flips, ~1/2048). 3. Failover: stop one publisher, the surviving member still serves chat. /members shows the stopped row as inactive immediately (unbindSession runs synchronously on SSE close). docs/virtual-llms.md gets a full "LB pools (v4)" section with the two-field schema model, dispatcher selection + failover semantics, public + virtual declaration examples, list/describe rendering, the "pin to specific instance" escape hatch, and an API surface entry for /members. docs/agents.md cross-link extended. Tests: full smoke 144/144 (was 141, +3 for the new pool smoke). Stages 1-3 ship the complete v4 — public and virtual Llms can both join pools, agents transparently load-balance across them, yaml round-trip preserves poolName, and the existing single-Llm world keeps working byte-identically when poolName is null.	2026-04-27 23:22:15 +01:00
Michal	1998b733b2	feat(cli+docs): mcpctl get agent KIND/STATUS columns + virtual-agent smoke + docs (v3 Stage 4) ... CLI: `mcpctl get agent` table view gains KIND and STATUS columns mirroring the `get llm` shape from v1. Public agents render as `public/active` (the AgentRow defaults) and virtual ones surface their true lifecycle state, so `mcpctl get agent` becomes a single-pane view for both manually-created and mcplocal-published personas. Smoke: tests/smoke/virtual-agent.smoke.test.ts mirrors virtual-llm's in-process registrar pattern — publishes a fake provider + agent in one round-trip, confirms mcpd surfaces the agent kind=virtual / status=active under /api/v1/agents, then disconnects and verifies the paired Llm-and-Agent both flip to inactive (deletion is GC-driven, not disconnect-driven, so the rows must still exist post-stop). Heartbeat- stale and 4 h sweep paths are covered by the unit suite to keep smoke duration in check. Docs: docs/virtual-llms.md gets a "Virtual agents (v3)" section with a config sample, lifecycle notes, listing example, and the cluster-wide name-uniqueness caveat. The API surface block now mentions the new `agents[]` field on _provider-register, the join-by-session heartbeat behavior, and the `GET /api/v1/agents` lifecycle fields. docs/agents.md gains a one-paragraph note pointing to the v3 publishing path. Tests: full smoke suite 141/141 (was 139, +2 new), unit suites unchanged (mcpd 860/860, mcplocal 723/723).	2026-04-27 18:47:03 +01:00
Michal	610808b9e7	fix(chat): real fixes for thinking-model + URL conventions, not test tweaks ... Five real bugs surfaced by the agent-chat smoke against live qwen3-thinking. None of these are fixed by changing the test — the test was right to fail. 1. openai-passthrough adapter doubled `/v1` in the request URL. The adapter hard-codes `/v1/chat/completions` after the configured base, but every OpenAI-compat provider documents its base URL with a trailing `/v1` (api.openai.com/v1, llm.example.com/v1, …). Users pasting that conventional shape produced `https://x/v1/v1/chat/completions` → 404. endpointUrl now strips a trailing `/v1` so both forms canonicalize. `/v1beta` (Anthropic-style) is preserved. 2. Non-streaming chat returned an empty assistant when thinking models (qwen3-thinking, deepseek-reasoner, OpenAI o1) emitted only `reasoning_content` with `content: null`. extractChoice now also pulls reasoning (every spelling the streaming parser already knows about), and a new pickAssistantText helper falls back to it when content is empty. A `[response truncated by max_tokens]` marker is appended when finish_reason is `length`, so users see the cut-off instead of guessing why the answer is short. Symmetric streaming fix: the chatStream loop accumulates reasoning and yields ONE synthesized `text` frame at the end when content stayed empty, keeping the CLI's stdout (which only prints `text` deltas) in sync with the persisted thread message. 3. `mcpctl get agent X -o yaml` emitted `kind: public` (the v3 lifecycle field) instead of `kind: agent` (apply envelope), so round-tripping through `apply -f` failed. Same fix shape as the v1 Llm strip in toApplyDocs — drop kind/status/lastHeartbeatAt/ inactiveSince/providerSessionId for the agents resource too. 4. Non-streaming `mcpctl chat` printed `thread:<cuid>` (no space) on stderr; streaming printed `(thread: <cuid>)` (with space). Tests and any other regex watching for one form missed the other. Standardize on `thread: <cuid>` (single space) in both paths. 5. agent-chat.smoke's `run()` used `execSync`, which discards stderr on success — making any `expect(stderr).toMatch(...)` assertion structurally impossible to satisfy in the happy path. Switch to `spawnSync` so stderr is actually captured. Includes a small shell-style argv splitter so the existing call sites with quoted multi-word values (`--system-prompt "..."`) keep working. Tests: +6 new mcpd unit tests (4 chat-service for the reasoning fallback / truncation marker / content-preference / streaming synth; 2 llm-adapters for the URL strip + /v1beta preservation). Full mcpd + mcplocal + smoke green: 860/860 + 723/723 + 139/139.	2026-04-27 18:39:01 +01:00
Michal	e0cfe0ba4d	feat: virtual-LLM v2 smoke + docs (v2 Stage 3) ... Closes v2 (wake-on-demand). Same shape as v1's stage 6: smoke exercises the live-cluster path, docs lose the "v2 reserved" caveat and gain a full wake-recipe section. Smoke (virtual-llm.smoke.test.ts): - New "wake-on-demand" describe block runs alongside the v1 tests. - Spins a tiny in-process HTTP "wake controller"; the published provider's isAvailable() returns false until the wake POST flips the bool. Asserts: 1. Provider publishes as kind=virtual / status=hibernating. 2. First inference triggers the wake recipe, the recipe POSTs to the controller, the provider becomes available, mcpd relays the inference, and the row settles to active. - Cleans up the row + wake server in afterAll. Docs (docs/virtual-llms.md): - Lifecycle table updates the `hibernating` description from "reserved for v2" to the actual v2 semantics. - New "Wake-on-demand (v2)" section: configuration shapes for both recipe types (http + command), the wake-then-infer flow diagram, concurrent-infer dedup, failure semantics. - Roadmap drops v2; v3-v5 still listed. Workspace: 2050/2050 (smoke runs separately; the new SSE-based wake test runs only against a live cluster, not under \`pnpm test:run\`). v2 closes. v3 = virtual agents, v4 = LB pool by model, v5 = queue. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-27 15:20:18 +01:00
Michal	af0fabd84f	feat(mcplocal+mcpd): wake-recipe config + wake-task execution (v2 Stage 1) ... First half of v2 — mcplocal can now declare a hibernating backend and respond to a `wake` task by running a configured recipe. v2 Stage 2 will wire mcpd to dispatch the wake task before relaying inference. Config (LlmProviderFileEntry): - New \`wake\` block on a published provider: wake: type: http # or: command url: ... # http only method: POST # http only, default POST headers: {...} # http only body: ... # http only command: ... # command only args: [...] # command only maxWaitSeconds: 60 # how long to poll isAvailable() after wake fires Registrar (mcplocal): - At publish time, providers with a wake recipe whose isAvailable() returns false report initialStatus=hibernating to mcpd. Without a wake recipe (legacy v1) or when already up, status stays active. - handleWakeTask: runs the recipe (HTTP request OR child-process spawn), then polls isAvailable() up to maxWaitSeconds, sending a heartbeat each loop so mcpd's GC sweep doesn't time us out mid-boot. Reports { ok, ms } on success or { error } on timeout/recipe failure via the existing _provider-task/:id/result. - Replaces the v1 stub that rejected wake tasks with "not implemented". mcpd VirtualLlmService: - RegisterProviderInput gains optional initialStatus ('active' | 'hibernating'). The register/upsert path uses it for both new and reconnecting rows. Defaults to 'active' so v1 publishers still work unchanged. - Provider-register route's coercer accepts the new field. Tests: 3 new in registrar.test.ts cover initialStatus selection (hibernating when wake configured + unavailable, active otherwise, active when no wake even if unavailable). 8/8 registrar tests, 833/833 mcpd unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-27 15:15:46 +01:00
Michal	866f6abc88	feat: virtual-LLM smoke test + docs (v1 Stage 6) ... Final stage of v1. Smoke (mcplocal/tests/smoke/virtual-llm.smoke.test.ts): - Spins an in-process LlmProvider that returns canned content. - Runs the registrar against the live mcpd in fulldeploy. - Asserts: row appears with kind=virtual / status=active, infer through /api/v1/llms/<name>/infer comes back through the SSE relay with the provider's content + finish_reason, and a 503 appears immediately after registrar.stop() (publisher offline). - Times out / cleanup paths idempotent so re-runs against the same cluster don't litter rows. The 90-s heartbeat-stale flip and 4-h GC are unit-tested — too slow for smoke. Docs: - New docs/virtual-llms.md: when to use this vs creating a regular Llm row, how to opt-in via publish: true, the lifecycle table, the inference-relay sequence, the v1 streaming caveat, the v2-v5 roadmap, and the full /api/v1/llms/_provider-* surface. - agents.md cross-links virtual-llms.md alongside personalities/chat. - README's Agents section gains a "Virtual LLMs" subsection. Workspace suite: 2043/2043 (smoke files run separately). v1 closes. Stage roadmap (each its own future PR): v2 wake-on-demand · v3 virtual agents · v4 LB pool · v5 task queue Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-27 14:28:43 +01:00
Michal	97174f450f	feat(mcplocal): virtual-LLM registrar (v1 Stage 4) ... The mcplocal counterpart to mcpd's VirtualLlmService. After this stage, flipping \`publish: true\` on a provider in ~/.mcpctl/config.json makes the provider show up in mcpctl get llm with kind=virtual the next time mcplocal restarts; running an inference against it relays through this client back to the local LlmProvider. Config: - LlmProviderFileEntry gains optional \`publish: boolean\` (default false, so existing setups don't change). Registrar (new file: providers/registrar.ts): - start(): if any provider is opted-in, POSTs to /api/v1/llms/_provider-register with the publishable set, persists the returned providerSessionId to ~/.mcpctl/provider-session for sticky reconnects, then opens the SSE control channel and starts a 30-s heartbeat ticker. - SSE listener parses event/data lines from text/event-stream frames. task frames trigger handleInferTask: convert OpenAI body to CompletionOptions, call provider.complete(), POST the result back as either { status, body } (non-streaming) or two chunk POSTs (streaming: one delta + a [DONE] marker). - Disconnect → exponential backoff reconnect from 5 s up to 60 s. On successful reconnect the persisted sessionId revives the same Llm rows in mcpd (mcpd flips them back to active on heartbeat). - stop() destroys the SSE socket and clears the timer; cleanly handed off from main.ts's existing shutdown handler. Wired into mcplocal main.ts via maybeStartVirtualLlmRegistrar: - Filters opted-in providers, looks up their LlmProvider instances in the registry. - Reads ~/.mcpctl/credentials for mcpdUrl + bearer; absence is a best-effort skip (logs a warning, returns null) — never a boot blocker. v1 caveat documented in the file header: LlmProvider returns a finalized CompletionResult, not a token stream, so streaming requests get a single delta chunk + [DONE]. Real per-token streaming is a v2 concern. Tests: 5 new in tests/registrar.test.ts using a tiny in-process HTTP server. Cover: no-op when nothing opted-in, register POST + sticky sessionId persistence, sticky reconnect from disk, heartbeat ticker fires at the configured interval, register HTTP error surfaces. Workspace suite: 2043/2043 across 152 files (was 2006/149, +5 new tests + the new file gets discovered). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-27 14:20:54 +01:00
Michal	4cbf58d212	feat(mcpd+deploy): serve web UI at /ui + smoke tests + docs (Stage 6) ... The closing stage. mcpd now hosts the Stage 5 SPA, the Docker image bundles the build artifact, a smoke test exercises the personality HTTP surface end-to-end, and the user-facing docs spell out the mental model. mcpd: - Add @fastify/static dep. - New routes/web-ui.ts: registers /ui/* against a static bundle. Looks for the bundle at $MCPD_WEB_ROOT, then /usr/share/mcpd/web (the Docker image path), then a dev-tree fallback. Logs and skips cleanly if missing — API-only deploys keep working. - SPA fallback: any /ui/<path> that doesn't match a file falls through to index.html so direct hits to react-router URLs work. - /ui/* falls through to `kind: skip` in mapUrlToPermission, so the static assets are served unauthenticated. Each API call from the SPA still carries the bearer token. Deploy: - Dockerfile.mcpd builds the @mcpctl/web bundle in the same builder stage and copies dist/ to /usr/share/mcpd/web in the runtime image. Smoke (personality.smoke.test.ts): - Live mcpd flow: create secret/llm/agent/personality, attach an agent-direct prompt, verify the binding listing, reject double- attach (409) + foreign-agent prompt (400), set defaultPersonality by name, detach + delete cleanup. Docs: - New docs/personalities.md: VLAN-on-ethernet model, system-block ordering table, three prompt scopes, CLI walkthrough, web UI walkthrough, full API surface, RBAC notes. - agents.md and chat.md cross-link. - README's Agents section gains a Personalities subsection. Test count after Stage 6: mcpd: 801/801 cli: 430/430 web: 7/7 db: 58/62 (4 pre-existing) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-26 19:48:43 +01:00
Michal	e51b92473f	fix(smoke,rotator,auth): repair smoke env + close failure modes that ... caused 27 post-deploy smoke failures This commit lands the durable side of the post-deploy investigation: genuine bugs that let the upstream OpenBao re-init silently break every secret write for 4 days, plus test-code bugs that masked the same breakage in the smoke output. mcpd — fail loud on dead OpenBao tokens ======================================= secret-backend-rotator.service.ts When `mintRoleToken` or `lookupSelf` returns 403/401, classify it as BACKEND_TOKEN_DEAD (likely cause: upstream OpenBao re-init invalidated every pre-existing token), wrap the thrown error with explicit remediation (mint via root + `mcpctl create secret <name> --data <key>=<token> --force`), persist the same message to tokenMeta.lastRotationError, and emit a structured `level:fatal` console.error so it shows up in `kubectl logs deploy/mcpd` with grep- friendly `kind:BACKEND_TOKEN_DEAD`. Adds a `healthCheck(backendId)` method that runs lookup-self without minting — so the boot-time loop can detect the dead-token state immediately, not 24 hours later. secret-backend-rotator-loop.ts Boot-time health check: in `start()`, for every rotatable backend, call `rotator.healthCheck(b.id)` and on failure log a structured fatal entry. This converts the prior silent failure mode (24h wait until scheduled rotation reveals the dead token, with secret writes failing under it the entire time) into "mcpd boots, immediately sees the dead token, alerts loudly". Existing isOverdue path is unchanged. mcpd — Prisma userId crash on /me ================================= routes/auth.ts GET /api/v1/auth/me used `request.userId!` which lied: an authenticated McpToken bearer satisfies the auth middleware but has no associated User row, so userId stayed undefined and `findUnique({ id: undefined })` threw PrismaClientValidationError. Now returns 401 with a clear "service-account/token-bound principal cannot be queried via /me" message instead of bubbling a 500. mcplocal — token revocation propagation ======================================= http/token-auth.ts Lowered default introspection positiveTtl from 30s → 5s. mcpd's introspection endpoint is a single DB lookup; the cache only protects against burst restart storms, not steady-state load. The 30s window let revoked tokens keep working for the full window after revocation (caught by mcptoken.smoke's negative-cache assertion). Aligns with the existing 5s negativeTtl and the test's `wait 7s after revoke` expectation. smoke tests — read URL the same way the CLI does ================================================ mcp-client.ts Adds `loadMcpdAuth()`: URL from `~/.mcpctl/config.json`, token from `~/.mcpctl/credentials`. Critically, the URL does NOT come from credentials. credentials.mcpdUrl carries a stale field for legacy reasons and goes out of sync (left over from old `mcpctl login --mcpd-url localhost:3xxx` invocations) — tests reading it ended up hitting whatever URL the user last logged into rather than the URL the CLI is actually using right now. audit/security/system-prompts smoke now use loadMcpdAuth(), eliminating ~10 cascade failures. Also: switch httpRequest to https.request when scheme is https (matching audit/security/system-prompts/mcp-client/agent helpers). Bumps default callTool timeout from 30s → 60s; many tools that fetch external resources routinely run 10-30s. agent.smoke.test.ts - readToken read from `credentials.json`; the file is `credentials` (no extension). Caused 401 on POST /threads. - `mcpctl get <resource> <name> -o json` returns an array, not a bare object. Round-trip yaml test now indexes [0] before reading description. secretbackend.smoke.test.ts Two genuine assertion-drift fixes (env was right, test was stale): - "lists at least one secretbackend": stop hard-coding the default backend type as 'plaintext'; the invariant is "exactly one default exists". The seeded plaintext is the bootstrap default but operators routinely promote a remote backend (openbao etc.) once it's healthy. - "refuses to delete the seeded default": widen the regex from /default|in use|cannot delete/ to also accept "referenced" — the exact wording has shifted to "is still referenced by N secret(s); migrate them first". audit.test.ts / system-prompts.test.ts / security.test.ts Switch http.request → https.request when URL is https (each had its own copy of the helper). Drop the now-orphan loadMcpdCredentials in favour of loadMcpdAuth from mcp-client.ts. Tests ===== mcpd 759/759, mcplocal 715/715 unit suites still green. Smoke (live): Run 1 (pre-commit, post bao-token rotation): 27 → 12 failures. Run 2 (after fixes-batch, pre-redeploy): 12 → 2 failures. The remaining 2 (mcptoken cache TTL, proxy-pipeline timeout) are what the durable code changes here address; verify after the next redeploy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-25 18:35:13 +01:00
Michal	8b56f09f25	feat(agents): smoke tests + README + docs (Stage 6, final) ... Closes the agents feature. Smoke tests (run via `pnpm test:smoke` against a live mcpd at $MCPD_URL, default https://mcpctl.ad.itaz.eu): * tests/smoke/agent.smoke.test.ts — full CRUD round-trip: create secret + Llm + agent with sampling defaults; `get agents` surfaces it; `get agent foo -o yaml | apply -f` round-trips identically; create + list a thread via the HTTP API; agent delete leaves Llm + secret intact (Restrict + SetNull as designed). Self- skips with a warning when /healthz is unreachable. * tests/smoke/agent-chat.smoke.test.ts — gated on MCPCTL_SMOKE_LLM_URL + MCPCTL_SMOKE_LLM_KEY. Provisions secret + Llm + agent against a real upstream, runs `mcpctl chat -m … --no- stream` (asserts a reply lands), then runs the streaming default (asserts text on stdout + `(thread: …)` on stderr). The fast path for verifying the in-cluster qwen3-thinking deployment: MCPCTL_SMOKE_LLM_URL=http://litellm.nvidia-nim.svc.cluster.local:4000/v1 \ MCPCTL_SMOKE_LLM_MODEL=qwen3-thinking \ MCPCTL_SMOKE_LLM_KEY=$(pulumi config get --stack homelab \ secrets:litellmMcpctlGatewayToken) \ pnpm test:smoke Docs: * README.md — new "Agents" section under Resources with the qwen3-thinking quickstart and links to docs/agents.md and docs/chat.md. Adds llm + agent rows to the resources table. * docs/agents.md (new) — full reference: data model, chat-parameter table, HTTP API, RBAC mapping, tool-use loop semantics, yaml round-trip shorthand, the kubernetes-deployment wiring recipe, and a troubleshooting section (namespace collision, llm-in-use, pending-row recovery, Anthropic-tool limitation). * docs/chat.md (new) — user-facing `mcpctl chat` walkthrough: modes, per-call flags, slash-commands, threads, and a troubleshooting section. * CLAUDE.md — adds a "Resource types" cheatsheet with one-line pointers to each, including the new `agent` row that links to the docs. All suites still green: mcpd 759/759, mcplocal 715/715, cli 430/430. Smoke tests typecheck and self-skip when no live mcpd is reachable. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-25 17:08:37 +01:00
Michal	285be11dd5	feat(agents): mcplocal agents plugin + composePlugins helper (Stage 4) ... When a Claude (or any other MCP client) connects to a project's mcplocal endpoint, every Agent attached to that project now appears in the session's tools/list as a virtual MCP server named `agent-<agentName>` with one tool `chat`. Calling that tool POSTs to the Stage 3 chat endpoint and returns the assistant's reply as MCP content. The tool's description is the agent's own description, so connecting clients see prose like "I review security design — ask me after each major change." This is what makes one agent reachable from another's MCP session. Plumbing: * src/mcplocal/src/proxymodel/plugins/agents.ts (new) — the plugin. onSessionCreate fetches /api/v1/projects/:p/agents via mcpd, then registers a VirtualServer per agent. The chat tool's inputSchema mirrors the LiteLLM-style override surface (temperature, top_p, top_k, max_tokens, stop, seed, tools_allowlist, extra) plus threadId for follow-ups. Namespace collision with an existing upstream MCP server named `agent-<x>` is detected and skipped with a `ctx.log.warn` line — better to surface the conflict than to silently shadow real tool entries in the virtualTools map. * src/mcplocal/src/proxymodel/plugins/compose.ts (new) — generic N-plugin composition helper. Lifecycle hooks fan out in order; transform hooks (onToolsList, onResourcesList, onPromptsList, onToolCallAfter) pipeline; intercept hooks (onToolCallBefore, onResourceRead, onPromptGet, onInitialize) short-circuit on the first non-null. Generalizes what createDefaultPlugin does for two fixed parents. * src/mcplocal/src/http/project-mcp-endpoint.ts — every project session now uses composePlugins([defaultPlugin, agentsPlugin]) so agents show up no matter which proxymodel the project is on. * Plugin context: added getFromMcpd(path) alongside postToMcpd. The existing postToMcpd was hard-coded to POST; the agents plugin needs GET to discover. Wired through plugin.ts → plugin-context.ts → router.ts. Tests: plugin-agents.test.ts (8) — registers per agent, falls back to a generic description, skips on namespace collision, no-ops with zero agents, logs and continues on mcpd error, chat handler POSTs correct body and returns content array, isError surfacing on mcpd error, onSessionDestroy unregisters everything. plugin-compose.test.ts (6) — single-plugin pass-through, empty rejection, lifecycle ordering, intercept short-circuit, list pipeline, no-op composition stays minimal. mcplocal suite: 715/715. mcpd suite still 759/759. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-25 16:51:44 +01:00
Michal	d5236171cc	fix(smoke): use json output for llm apiKeyRef assertion ... The table KEY column truncates at ~34 chars so `secret://<name>/<key>` wasn't appearing verbatim in stdout — the assertion was correct but brittle against presentation choices. Switched to `-o json` where the ref round-trips as a structured object, which is what actually matters. Caught by the live-cluster smoke run right after Phase 0-4 rolled out. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-19 22:55:39 +01:00
Michal	58788bc120	test(smoke): end-to-end coverage for SecretBackend, Llm, infer proxy, project-llm-ref ... Covers the Phase 0-4 CLI contract against live mcpd. Matches the existing mcptoken.smoke pattern: skip gracefully on unreachable /healthz, cleanup fixtures in afterAll, use --direct to bypass mcplocal for admin operations. - secretbackend.smoke.test.ts · seeded plaintext default exists + isDefault · create/describe/delete round-trip · refuses to delete the default backend (409 shape) · get -o yaml output starts with `kind: secretbackend` (apply-compatible) - llm.smoke.test.ts · create secret + llm with --api-key-ref, verify describe hides the raw value but surfaces secret://name/key · yaml round-trip: get -o yaml > file → amend → apply -f → describe shows change · deleting the llm leaves the underlying Secret intact (onDelete: SetNull) - llm-infer.smoke.test.ts · 404 for unknown name, 400 for missing messages · 5xx when upstream url is unreachable (proxy returns a structured error) · opt-in happy-path gated on LLM_INFER_SMOKE_REAL=1 + LLM_INFER_SMOKE_LLM=<name> so CI doesn't need a real provider key - project-llm-ref.smoke.test.ts · describe project with --llm <registered> — no warning · describe project with --llm <nonexistent> — shows "warning: …registry default" · describe project with --llm none — explicit disable, no warning These require PRs #51-55 to be merged and fulldeploy.sh run before they'll find the new endpoints on live mcpd. Until then they skip or fail with "Not Found". Unit tests for the same code paths (1853 total) continue to pass against mocks. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-19 22:09:41 +01:00
Michal	de854b1944	feat(project): Project.llmProvider semantically names an Llm resource ... Why: Phases 0-3 built the server-managed Llm registry; this phase pivots the existing Project.llmProvider column from "local provider hint" to "named Llm reference" so operators can pick a centralised Llm per project. No schema change — the column stays a free-form string for backward compat. - `mcpctl create project --llm <name>` (+ `--llm-model <override>`) sets llmProvider/llmModel to a centralised Llm reference, or 'none' to disable. - `mcpctl describe project` fetches the Llm catalogue alongside prompts and flags values that don't resolve with a visible warning. 'none' is treated as an explicit disable, not an orphan. - `apply -f` doc comments updated; --llm-provider still accepted but now documented as naming an Llm resource. - New `resolveProjectLlmReference(mcpdClient, name)` helper in mcplocal's discovery: returns `registered`/`disabled`/`unregistered`/`unreachable`. The HTTP-mode proxy-model pipeline will consume this when it pivots to mcpd's /api/v1/llms/:name/infer proxy. - project-mcp-endpoint.ts cache-namespace path gets a comment explaining the new resolution order — behavior unchanged, just clarified. Tests: 6 resolver unit tests + 3 new describe-warning cases. Full suite 1853/1853 (+9 from Phase 3's 1844). TypeScript clean; completions regenerated for the new create-project flags. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-19 18:28:46 +01:00
Michal	4d8ee23d0e	feat(mcplocal): RBAC-bounded vllm-managed failover + name-based llm lookup ... Why: when mcpd's inference proxy is unreachable, clients with a local vllm-managed provider should be able to substitute — but only if they still have view permission on the centralized Llm. Otherwise revoking an Llm wouldn't actually stop a misbehaving client. Infrastructure (the agent + mcplocal HTTP-mode wire-up will land separately when those clients pivot to mcpd's proxy): - LlmProviderFileEntry gains optional `failoverFor: <central llm name>`. The entry is otherwise the same local provider it always was; the new field just declares which central Llm it can substitute for. - ProviderRegistry tracks a failover map (registerFailover / getFailoverFor / listFailovers). Unregister removes any failover entry pointing at the removed provider so we don't end up with dangling references. - New FailoverRouter wraps a primary inference call. On primary failure: if a local provider is registered for the Llm, HEAD-probe `mcpd /api/v1/llms/ :name` with the caller's bearer to verify view permission, then either invoke the local provider (allowed) or re-throw the primary error (403, 401, network unreachable, anything else — all fail-closed). - Server: GET /api/v1/llms/:idOrName accepts both CUID and human name. Lets FailoverRouter probe by name without a separate id-resolution call. HEAD derives automatically from GET in Fastify, which runs the same RBAC hook and drops the body — exactly what the probe needs. Tests: 11 failover unit tests (registry map, decision flow, fail-closed for forbidden + unreachable, checkAuth status mapping) + 4 new route tests (name lookup, HEAD existing/missing). Full suite 1844/1844 (+14 from Phase 2's 1830). TypeScript clean across mcpd + mcplocal. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-19 13:05:43 +01:00
Michal	6946250090	Revert "feat(mcplocal): per-McpToken gate-ungate cache so service tokens survive proxies" ... This reverts commit 39df459bb1.	2026-04-18 18:16:18 +01:00
Michal	39df459bb1	feat(mcplocal): per-McpToken gate-ungate cache so service tokens survive proxies ... Fixes the LiteLLM loop: LiteLLM's /mcp/ proxy doesn't propagate the mcp-session-id header, so every tool call from qwen3 landed on a fresh upstream session, which always started gated, so the only visible tool was begin_session — forever. The session-id gate works fine for Claude Code (stdio, long-lived), but breaks through session-stripping proxies. Identity that DOES survive: the McpToken (always in the Authorization header). So now the gate keys its ungate state on both: - sessionId → per-session (unchanged; Claude Code path) - tokenSha → per-token (NEW; service-token path) Flow for an McpToken caller: 1. first begin_session succeeds → session ungated + tokenSha cached 2. next request lands on a new mcp-session-id (proxy stripped it) 3. SessionGate.createSession sees tokenSha, finds active token entry, starts the new session ungated with the prior tags + retrievedPrompts 4. tools/list on the fresh session returns the full upstream set — no more begin_session loop Plumbing: - AuditCollector.getSessionMcpTokenSha(sessionId) exposes the already- tracked principal. - PluginSessionContext gets getMcpTokenSha() so plugins can read the token identity without knowing about the collector. - SessionGate gains (tokenSha?: string) on createSession/ungate, plus isTokenUngated and revokeToken. TTL defaults to 1hr; tunable via MCPLOCAL_TOKEN_UNGATE_TTL_MS env var. - Gate plugin passes ctx.getMcpTokenSha() at every ungate call site (begin_session, gated-intercept, intercept-fallback). Tests: 7 new cases in session-gate.test.ts covering cross-session persistence, token isolation, STDIO-path unchanged, TTL expiry, revokeToken, and the empty-string edge case. 21/21 pass; 690/690 in mcplocal overall. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-18 17:34:28 +01:00
Michal	75fe0533c1	fix(mcplocal): propagate caller's bearer to prompt-index and LLM-config calls ... The proxy-path fix (5d10728) covered upstream tools/call routing via McpdUpstream, but getOrCreateRouter in project-mcp-endpoint.ts had TWO more mcpd-bound call sites that silently fell back to the pod's empty default token: 1. fetchProjectLlmConfig(mcpdClient, projectName) 2. router.setPromptConfig(mcpdClient.withHeaders({...})) → which is what gate.ts begin_session uses via ctx.fetchPromptIndex() to hit /api/v1/projects/:name/prompts/visible Symptom: in the k8s mcplocal pod, LiteLLM would initialize + tools/list fine (showing begin_session), but tools/call begin_session returned `{isError: true, content: "McpError: Authentication failed: invalid or expired token"}`. Reproduced against the live cluster by driving LiteLLM's /mcp/ endpoint with qwen3-thinking's exact payload. Fix: build `requestClient = mcpdClient.withToken(authToken)` once at the top of getOrCreateRouter and thread it through fetchProjectLlmConfig and setPromptConfig. withHeaders still adds X-Service-Account for mcpd-side audit tagging, but the bearer now carries the caller's McpToken identity (resolves as McpToken:<sha> on mcpd). Verified: unit tests pass (mock needed withToken/withTimeout stubs). Next step: rebuild image + roll pod + retest LiteLLM→mcp flow. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-18 04:44:27 +01:00
Michal	5d1072889f	fix(mcplocal): thread client bearer into per-upstream McpdClient ... Symptom: HTTP-mode mcplocal accepted the incoming mcpctl_pat_ bearer, but every /api/v1/mcp/proxy call to mcpd for upstream discovery came back with "Authentication failed: invalid or expired token" — because those proxy calls were using the pod's DEFAULT McpdClient token, which in a container with no ~/.mcpctl/credentials is the empty string. The discovery GET was correct (explicit authOverride in forward()), but syncUpstreams() then created McpdUpstream instances bound to the original mcpdClient — so every tools/list to each upstream went out with `Authorization: Bearer ` (empty) and mcpd's auth hook rejected it. Fix: add McpdClient.withToken(token) and have refreshProjectUpstreams swap to `mcpdClient.withToken(authToken)` before handing the client to syncUpstreams. This keeps the "pod has no identity" design: the token used for downstream /api/v1/mcp/proxy calls is the caller's McpToken, same as the one used for the initial discovery GET and for introspect. Tested: project-discovery.test.ts + mcpd-upstream.test.ts pass. Next: rebuild + roll the mcplocal image and retry LiteLLM probe. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-18 03:06:55 +01:00
Michal	3061a5f6ae	test+feat: token-auth unit coverage + env-tunable introspection TTLs ... Verifies the HTTP-mode revocation lag ≤ 5s two ways: 1. Unit (tests/http/token-auth.test.ts, 8 cases): Fastify preHandler with injected fetch stub exercises the positive/negative cache directly — first call returns ok:true, we flip the stub to revoked:true, wait past the short positive TTL, next call gets 401 with "revoked". Plus: non-Bearer 401, non-mcpctl_pat_ 401, wrong- project 403, mcpd-unreachable 401, happy-path caching (1 fetch for N requests within TTL), ok:false from mcpd 401. 2. End-to-end (smoke, run manually): added MCPLOCAL_TOKEN_POSITIVE_TTL_MS and MCPLOCAL_TOKEN_NEGATIVE_TTL_MS env vars to serve.ts so the smoke can shrink the 30s positive default for testing. Confirmed: with positive TTL = 2s, the mcptoken.smoke.test.ts revocation case passes against a local serve.js pointed at prod mcpd. Operators get the same knobs in production — default behavior unchanged (30s positive, 5s negative). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-17 23:25:06 +01:00
Michal	913678e400	fix(smoke): mcptoken — runtime gatewayUp gate + scope revocation case to HTTP-mode ... Two bugs found while trying to point MCPGW_URL=http://localhost:3200 (the systemd mcplocal) so we could get real smoke coverage before the Pulumi stack for mcp.ad.itaz.eu lands: 1. describe.skipIf(!gatewayUp) was evaluated at parse time, before beforeAll ran, so gatewayUp was always false and the whole suite skipped. Switched to the vllm-managed.test.ts pattern: runtime `if (!gatewayUp) return` at the start of each it(). 2. The revocation 401 assertion only makes sense against the containerized serve.ts entry, which has a 5s negative introspection cache. Against systemd mcplocal the whole project router is cached for minutes, so a deleted token with a warm session still succeeds. Added IS_HTTP_MODE detection (hostname not localhost/127/0.0.0.0, or MCPGW_IS_HTTP_MODE=true) and skip the assertion otherwise — still revoking the token so cleanup runs identically. Run against systemd mcplocal locally: MCPGW_URL=http://localhost:3200 pnpm --filter @mcpctl/mcplocal \\ exec vitest run --config vitest.smoke.config.ts mcptoken → 6/6 pass (revocation case explicitly deferred). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-17 23:20:36 +01:00
Michal	2127b41d9f	feat: HTTP-mode mcplocal container + mcpctl test mcp + token-auth preHandler ... Delivers the final piece of the mcptoken stack: a containerized, network-accessible mcplocal that serves Streamable-HTTP MCP to off-host clients (the vLLM use case), authenticated by project-scoped McpTokens. New binary (same package, new entry): - src/mcplocal/src/serve.ts — HTTP-only entry. Reads MCPLOCAL_MCPD_URL, MCPLOCAL_MCPD_TOKEN, MCPLOCAL_HTTP_HOST/PORT, MCPLOCAL_CACHE_DIR from env. No StdioProxyServer, no --upstream. - src/mcplocal/src/http/token-auth.ts — Fastify preHandler that validates mcpctl_pat_ bearers via mcpd's /api/v1/mcptokens/introspect. 30s positive / 5s negative TTL. Rejects wrong-project with 403. Shared HTTP MCP client: - src/shared/src/mcp-http/ — reusable McpHttpSession with initialize, listTools, callTool, close. Handles http+https, SSE, id correlation, distinct McpProtocolError / McpTransportError. Plus mcpHealthCheck and deriveBaseUrl helpers. New CLI verb `mcpctl test mcp <url>`: - Flags: --token (also $MCPCTL_TOKEN), --tool, --args (JSON), --expect-tools, --timeout, -o text|json, --no-health. - Exit codes: 0 PASS, 1 TRANSPORT/AUTH FAIL, 2 CONTRACT FAIL. Container + deploy: - deploy/Dockerfile.mcplocal (Node 20 alpine, multi-stage, pnpm workspace, CMD node src/mcplocal/dist/serve.js, VOLUME /var/lib/mcplocal/cache, HEALTHCHECK on :3200/healthz). - scripts/build-mcplocal.sh mirrors build-mcpd.sh. - fulldeploy.sh is now a 4-step pipeline that also builds + rolls out mcplocal (gated on `kubectl get deployment/mcplocal` so the script stays green before the Pulumi stack lands). Audit + cache: - project-mcp-endpoint.ts passes MCPLOCAL_CACHE_DIR into FileCache at both construction sites and, when request.mcpToken is present, calls collector.setSessionMcpToken(id, ...) so audit events carry the tokenName/tokenSha. Tests: - 9 unit cases on `mcpctl test mcp` (happy path, health miss, expect-tools hit/miss, transport throw, tool isError, json report, $MCPCTL_TOKEN env fallback, invalid --args). - Smoke test src/mcplocal/tests/smoke/mcptoken.smoke.test.ts — gated on healthz($MCPGW_URL), skipped cleanly when unreachable. Covers happy path, wrong-project 403, --expect-tools contract failure, and revocation 401 within the negative-cache window. 1773/1773 workspace tests pass. Pulumi resources (Deployment, Service, Ingress, PVC, Secret, NetworkPolicy) still need to land in ../kubernetes-deployment before the smoke gate flips on. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-17 01:21:42 +01:00
Michal	3149ea3ae7	fix: MCP proxy resilience — discovery cache, default liveness probes ... Adds a per-server tools/list cache in McpRouter (positive + negative TTL) so a slow or dead upstream only stalls the first discovery call, not every subsequent client request. Invalidated on upstream add/remove. Health probes now apply a default liveness spec (tools/list via the real production path) to any RUNNING instance without an explicit healthCheck, so synthetic and real failures converge on the same signal. Includes supporting updates in mcpd-client, discovery, upstream/mcpd, seeder, and fulldeploy/release scripts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-17 00:48:57 +01:00
Michal	857f8c72ae	fix: MCP proxy resilience — timeouts, parallel discovery, error propagation ... - McpdClient: add 30s AbortSignal timeout to all fetch calls (was infinite) - CLI bridge: return JSON-RPC error on stdout when HTTP fails (was silent) - Router: parallel tool/resource discovery via Promise.allSettled (was sequential — one slow server blocked all) - vllm-managed: 60s error cooldown prevents retry-on-every-call when vLLM is broken - Tests: McpdClient timeout suite (9), parallel discovery, vllm cooldown, bridge error response Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-10 18:28:03 +01:00
Michal	383be66286	feat: add backup + server type smoke tests ... New smoke test file: backup-and-servers.test.ts - Backup completeness: prompts, templates, runtime, command, containerPort, replicas - SSE server proxy (my-home-assistant): 84 tools - Docker-image STDIO proxy (docmost): 11 tools - Package STDIO proxy (aws-docs): 4 tools - Instance status accuracy: RUNNING instances must respond to proxy These tests would have caught every migration bug: - Missing runtime (python servers on node runner) - Missing command (HA SSE in STDIO mode) - Missing containerPort (SSE on wrong port) - Backup data loss (prompts, templates, server fields) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-10 00:05:54 +01:00
Michal	225e0dddfc	fix: rate limiting breaking smoke tests and backup routes 404 when disabled ... - Exempt /healthz and /health from rate limiter - Increase rate limit from 500 to 2000 req/min - Register backup routes even when disabled (status shows disabled) - Guard restore endpoints with 503 when backup not configured - Add retry with backoff on 429 in audit smoke tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-08 13:32:17 +00:00