diff --git a/README.md b/README.md index 7b8db4b..fe68342 100644 --- a/README.md +++ b/README.md @@ -880,17 +880,21 @@ Notes: ### Web search and docs lookup -The `duckduckgo`, `searxng` and `docs-mcp` templates give an agent web search and -version-pinned library documentation. All are self-hosted and none needs an API key. -`duckduckgo` needs no backing service at all: +The `searxng`, `firecrawl`, `duckduckgo` and `docs-mcp` templates give an agent web +search, page reading and version-pinned library documentation. All can run +self-hosted without an API key. For agents, pair `searxng` (search) with `firecrawl` +(page reading); `duckduckgo` needs no backing service but scrapes from your IP and +gets CAPTCHA-blocked under agent traffic: ```bash -mcpctl create server websearch --from-template duckduckgo +mcpctl create server searxng --from-template searxng --env SEARXNG_URL=http://searxng:8080 +mcpctl create server firecrawl --from-template firecrawl --env FIRECRAWL_API_URL=http://firecrawl-api:3002 ``` See [docs/web-search.md](docs/web-search.md) for the SearXNG engine setup (including -the `json` format setting, without which every search silently returns nothing) and -the `docs-mcp` index-persistence caveat. +the `json` format setting, without which every search silently returns nothing), why +none of the search templates probes with a search, and the `docs-mcp` +index-persistence caveat. ## Gated Sessions diff --git a/completions/mcpctl.bash b/completions/mcpctl.bash index 50da079..e4debf6 100644 --- a/completions/mcpctl.bash +++ b/completions/mcpctl.bash @@ -254,7 +254,7 @@ _mcpctl() { COMPREPLY=($(compgen -W "--project --url --branch --path --secret --key -h --help" -- "$cur")) ;; server) - COMPREPLY=($(compgen -W "-d --description --package-name --runtime --docker-image --transport --repository-url --external-url --command --container-port --replicas --env --volume --health-check-tool --health-check-args --health-check-interval --health-check-timeout --tool-call-timeout --health-check-failure-threshold --secret-delivery --entrypoint --from-template --env-from-secret --force -h --help" -- "$cur")) + COMPREPLY=($(compgen -W "-d --description --package-name --runtime --docker-image --transport --repository-url --external-url --command --container-port --replicas --env --volume --health-check-tool --health-check-args --health-check-interval --health-check-timeout --tool-call-timeout --memory-limit-mb --health-check-failure-threshold --secret-delivery --entrypoint --from-template --env-from-secret --force -h --help" -- "$cur")) ;; secret) COMPREPLY=($(compgen -W "--data --force -h --help" -- "$cur")) diff --git a/completions/mcpctl.fish b/completions/mcpctl.fish index 777ce22..acbc043 100644 --- a/completions/mcpctl.fish +++ b/completions/mcpctl.fish @@ -525,6 +525,7 @@ complete -c mcpctl -n "__mcpctl_subcmd_active create server" -l health-check-arg complete -c mcpctl -n "__mcpctl_subcmd_active create server" -l health-check-interval -d 'Readiness probe interval in seconds (default 60)' -x complete -c mcpctl -n "__mcpctl_subcmd_active create server" -l health-check-timeout -d 'Readiness probe timeout in seconds (default 10)' -x complete -c mcpctl -n "__mcpctl_subcmd_active create server" -l tool-call-timeout -d 'Per-server tool-call deadline in seconds (default: mcplocal\'s global deadline)' -x +complete -c mcpctl -n "__mcpctl_subcmd_active create server" -l memory-limit-mb -d 'Per-server memory ceiling in MiB (default: 512). Raise it for a server that drives a browser or holds a large index — at the default it is OOMKilled with no error, just a restart' -x complete -c mcpctl -n "__mcpctl_subcmd_active create server" -l health-check-failure-threshold -d 'Consecutive failures before the instance is marked unhealthy (default 3)' -x complete -c mcpctl -n "__mcpctl_subcmd_active create server" -l secret-delivery -d 'How secret env reaches the container: env (default, value written into the pod spec) or injector (pod fetches its own secrets from OpenBao under a scoped identity)' -x complete -c mcpctl -n "__mcpctl_subcmd_active create server" -l entrypoint -d 'Comma-separated argv to exec under the injector wrapper. Required for --secret-delivery injector on a dockerImage server, whose ENTRYPOINT mcpd cannot introspect' -x diff --git a/docs/reliability.md b/docs/reliability.md index 0eae89f..2611cf9 100644 --- a/docs/reliability.md +++ b/docs/reliability.md @@ -137,6 +137,42 @@ Also check the *dialect*: UniFi's `controller_type` must be `classic` for a self-hosted controller (login `/api/login`, no `/proxy/network` prefix). `unifi_os` sends every request to a path that 404s. +### A server that restarts instead of erroring is out of memory + +Server pods get **512 MiB** by default (`DEFAULT_MEMORY_LIMIT`). That is ample +for a server that proxies an API and nowhere near enough for one that drives a +browser or holds an index in memory. + +An OOMKill is the quietest failure in the fleet, because **nothing reports an +error**. The kernel kills the container, Kubernetes restarts it, the readiness +probe passes again, and `mcpctl get instances` reads `healthy`. Whatever the +server was doing is simply gone — for `docs`, six scrape jobs whose queue lived +in memory, leaving an index with 17 pages in it and no failed job to look at. + +The tells, in order of how fast they answer the question: + +```bash +kubectl -n mcpctl-servers get pods | grep # RESTARTS climbing +kubectl -n mcpctl-servers get pod -o jsonpath='{.status.containerStatuses[0].lastState.terminated.reason}' +``` + +`OOMKilled` there is conclusive. `mcpctl logs` will not show it: the process +never got to say anything. + +Raise the ceiling per server rather than for the fleet — most servers do not +need it, and a bigger default wastes real memory on every node: + +```bash +mcpctl create server docs --memory-limit-mb 2048 --force +``` + +Declared in MiB, stored on the server, converted to bytes in the container +spec. Null keeps the 512 MiB default, so existing servers are unchanged. Sizing +rule of thumb: measure idle first (`/sys/fs/cgroup/memory.current` inside the +pod), then leave headroom for the peak — `docs` idles at ~228 MiB and crosses +512 MiB within seconds of a scrape, because each page render is a Chromium +process. + ## LLM-*essential* operations — failover chain Chat needs *an* LLM but not a *specific* one. Instead of failing when the pinned diff --git a/docs/ungated-global.md b/docs/ungated-global.md new file mode 100644 index 0000000..b77c0d7 --- /dev/null +++ b/docs/ungated-global.md @@ -0,0 +1,56 @@ +# ungated-global: servers under every project, before the gate + +`ungated-global` is a system project with one special property: **every server +attached to it is exposed under every project's MCP endpoint, in front of the +gate** — its tools are listed and callable before `begin_session`, and the +project's `prompt` is prepended to every project's instructions. + +Fresh mcpd instances seed it **once** with the default `calcmcp` server +(scientific calculator, patched fork of +[thinkitpossible/CalcMCP](https://github.com/thinkitpossible/CalcMCP) at +) and a prompt telling models to never +do math themselves and always use calcmcp. + +## Semantics + +- **Seed-once, not resurrect**: the first successful mcpd startup writes + `ungatedGlobalSeeded: true` into the `mcpctl-system-settings` secret. After + that, nothing is ever re-created — deletions are durable: + - `mcpctl delete server calcmcp -p ungated-global` detaches the default + server (delete the server resource itself to also stop its instance); + - deleting the whole `ungated-global` project turns the feature off. + To re-create the default by hand there is a `calcmcp` template: + `mcpctl create server calcmcp --from-template calcmcp`, then attach it to + a (re-created) `ungated-global` project. +- **Merging** happens in mcpd: + - `GET /api/v1/projects/:id/servers` appends ungated-global's servers, + flagged `ungatedGlobal: true`. A server attached to the project directly + wins (no duplicate, no flag). + - `GET /api/v1/projects/:id/instructions` prepends ungated-global's + `prompt` and appends its servers to the server list. +- **Gate exemption** happens in mcplocal: discovery passes the flagged names + into the gate plugin (`ungatedServers`); while a session is gated, + `tools/list` returns `begin_session` **plus** the exempt servers' tools, and + calls to them pass through without triggering the gate intercept or + ungating the session. Initialize instructions list them under + "Available immediately (no begin_session required)". + +## Editing the prompt + +The pre-gate directive lives on the project itself: + +```bash +mcpctl patch project ungated-global --prompt "…" +``` + +Caveat: clients that ignore MCP initialize instructions (e.g. LibreChat uses +config-literal serverInstructions) never see project prompts — for those, the +carrier is the tool/server descriptions themselves, which is why calcmcp's +server description also says "use it for every mathematical calculation". + +## Security note + +Anything attached to ungated-global is reachable from **every** project by +**every** token that can reach any project endpoint. Only attach servers that +are safe in every context: no secrets, no egress, no side effects. calcmcp is +pure compute (no env, no network). diff --git a/docs/web-search.md b/docs/web-search.md index ff39cf6..438ff47 100644 --- a/docs/web-search.md +++ b/docs/web-search.md @@ -1,14 +1,19 @@ # Web search and documentation lookup -Three templates. All self-hosted, none needs an API key or a vendor account, and -all three are deployed by mcpctl like any other server. +Four templates. All can run self-hosted without an API key or a vendor account, +and all are deployed by mcpctl like any other server. | Template | Package / image | Needs | |---|---|---| -| `duckduckgo` | `duckduckgo-mcp-server` (python) | nothing | | `searxng` | `mcp-searxng` (node) | a SearXNG instance | +| `firecrawl` | `firecrawl-mcp` (node) | a Firecrawl instance | +| `duckduckgo` | `duckduckgo-mcp-server` (python) | nothing | | `docs-mcp` | `ghcr.io/arabold/docs-mcp-server` | nothing | +For agent use, pair `searxng` (finding pages) with `firecrawl` (reading them). +`duckduckgo` does both with no infrastructure, but it scrapes from your IP and +does not hold up under agent traffic — see below. + Search and docs are different jobs, not competing options. A search engine will hand you a 2023 blog post with a stale method signature; a docs index cannot tell you why a daemon is crash-looping. Attach both to a project that does real @@ -17,23 +22,31 @@ engineering work. `docs-mcp` is the open-source replacement for Context7 / Ref.tools — same job, but the index lives on your infrastructure and can include private repos. -## Start here: `duckduckgo` +## No infrastructure: `duckduckgo` -The only one with no infrastructure behind it. It scrapes DuckDuckGo's HTML -endpoint directly, so there is no engine to run and no key to hold. +The only one with nothing behind it. It scrapes DuckDuckGo's HTML endpoint +directly, so there is no engine to run and no key to hold. ```bash mcpctl create server websearch --from-template duckduckgo --env DDG_SAFE_SEARCH=OFF -mcpctl get instances | grep websearch # RUNNING / healthy within ~a minute +mcpctl get instances | grep websearch # RUNNING / healthy within a few minutes ``` Tools: `search` (`query`, `max_results`, `region`) and `fetch_content` (`url`, `start_index`, `max_length`) for pulling a result as markdown. -The tradeoff is honest: scraping has no SLA. DuckDuckGo can change its markup or -rate-limit you, and the server caps itself at 30 searches/min. Set -`DDG_SEARCH_BACKEND=curl` if bot checks start biting. When it becomes a problem, -move to `searxng`. +Fine for a person searching occasionally; not for agents. Scraping has no SLA, +and agents search in bursts — one question becomes several parallel queries. +From a single home or office IP, DuckDuckGo starts answering CAPTCHA and stays +that way, and every further request from that IP deepens the block, including +for a SearXNG instance sharing the address. `DDG_SEARCH_BACKEND=curl` survives +some bot checks but does not change that. Use `searxng` for agent search. + +The readiness probe is `fetch_content` on `https://example.com` every 300s, not +`search`. A search probe is a DuckDuckGo scrape every interval — 1,440 a day at +the 60s default — from the same IP your agents depend on. The trade: `healthy` +proves the process and its outbound fetch work, not that DuckDuckGo is still +answering you. ## Better results: `searxng` @@ -79,6 +92,43 @@ fails: nearly all of them leave the JSON API off. The compose healthcheck probes `format=json` specifically, so a misconfigured instance shows up as unhealthy rather than as silently empty search results. +### Probe and engines + +The template's readiness probe is `searxng_instance_info`, which reads the +instance's `/config`: a pass proves SearXNG answers without sending a query to +any engine. Do not probe with `searxng_web_search` — every interval becomes a +real search fanned out to every engine, which bills API-key engines and feeds +the IP blocks on scraping ones. + +Those blocks are the long-term problem. Scraping engines (duckduckgo, google, +startpage, qwant) CAPTCHA a single IP under agent traffic. SearXNG also ships +API-key engines — `braveapi`, `kagi` — that are not subject to IP reputation; +enable at least one if agents are the main caller, and cap its spend in the +vendor's dashboard. + +## Reading pages: `firecrawl` + +`firecrawl` is the reader to pair with `searxng`: give it a URL, get the page's +main content as markdown. Firecrawl renders pages in a real browser, so +JavaScript-heavy pages work where a plain fetch comes back empty. + +```bash +mcpctl create server firecrawl --from-template firecrawl \ + --env FIRECRAWL_API_URL=http://firecrawl-api:3002 +``` + +Point it at a self-hosted Firecrawl (no key needed) or leave `FIRECRAWL_API_URL` +unset and set `FIRECRAWL_API_KEY` for Firecrawl's cloud. The tool agents want is +`firecrawl_scrape` with `formats: ["markdown"]` and `onlyMainContent: true`. + +- **Egress port.** Self-hosted Firecrawl listens on `:3002`, outside the default + 80/443 server egress, so declare it or every call times out. +- **Search stays off** unless the Firecrawl instance has `SEARXNG_ENDPOINT` + set, so `firecrawl_search` cannot quietly become a second scraper. Use the + `searxng` template for search. +- **Probe:** `firecrawl_scrape` of `https://example.com` every 300s. It is a + real outbound fetch, hence the longer interval. + ## Documentation: `docs-mcp` ```bash diff --git a/src/cli/src/commands/apply.ts b/src/cli/src/commands/apply.ts index 8899719..05cd440 100644 --- a/src/cli/src/commands/apply.ts +++ b/src/cli/src/commands/apply.ts @@ -49,6 +49,7 @@ const ServerSpecSchema = z.object({ // nullable: `get server -o yaml` emits null for servers with no override, // and that YAML must apply back unchanged. toolCallTimeoutSeconds: z.number().int().min(1).max(3600).nullable().optional(), + memoryLimitMb: z.number().int().min(64).max(16384).nullable().optional(), }); const SecretSpecSchema = z.object({ diff --git a/src/cli/src/commands/create.ts b/src/cli/src/commands/create.ts index 65aa65e..6dfbd19 100644 --- a/src/cli/src/commands/create.ts +++ b/src/cli/src/commands/create.ts @@ -375,6 +375,7 @@ export function createCreateCommand(deps: CreateCommandDeps): Command { .option('--health-check-interval ', 'Readiness probe interval in seconds (default 60)') .option('--health-check-timeout ', 'Readiness probe timeout in seconds (default 10)') .option('--tool-call-timeout ', 'Per-server tool-call deadline in seconds (default: mcplocal\'s global deadline)') + .option('--memory-limit-mb ', 'Per-server memory ceiling in MiB (default: 512). Raise it for a server that drives a browser or holds a large index — at the default it is OOMKilled with no error, just a restart') .option('--health-check-failure-threshold ', 'Consecutive failures before the instance is marked unhealthy (default 3)') .option('--secret-delivery ', 'How secret env reaches the container: env (default, value written into the pod spec) or injector (pod fetches its own secrets from OpenBao under a scoped identity)') .option('--entrypoint ', 'Comma-separated argv to exec under the injector wrapper. Required for --secret-delivery injector on a dockerImage server, whose ENTRYPOINT mcpd cannot introspect') @@ -458,6 +459,9 @@ export function createCreateCommand(deps: CreateCommandDeps): Command { if (opts.toolCallTimeout !== undefined) { body.toolCallTimeoutSeconds = parsePositiveInt('--tool-call-timeout', opts.toolCallTimeout as string); } + if (opts.memoryLimitMb !== undefined) { + body.memoryLimitMb = parsePositiveInt('--memory-limit-mb', opts.memoryLimitMb as string); + } if (opts.packageName) body.packageName = opts.packageName; if (opts.runtime) body.runtime = opts.runtime; if (opts.dockerImage) body.dockerImage = opts.dockerImage; diff --git a/src/db/prisma/migrations/20260917000000_add_server_memory_limit/migration.sql b/src/db/prisma/migrations/20260917000000_add_server_memory_limit/migration.sql new file mode 100644 index 0000000..ad07a51 --- /dev/null +++ b/src/db/prisma/migrations/20260917000000_add_server_memory_limit/migration.sql @@ -0,0 +1,9 @@ +-- Per-server memory ceiling, in MiB. +-- +-- NULL keeps today's behaviour exactly: the server gets the orchestrator's +-- DEFAULT_MEMORY_LIMIT (512 MiB). That default is fine for a server that +-- proxies an API, and fatal for one that drives a browser — docs-mcp-server +-- scrapes with headless Chromium, sits at ~228 MiB idle, and crosses 512 MiB +-- within seconds of the first scrape. The pod is OOMKilled and restarts, so +-- the only trace is a restart count and an empty job queue. +ALTER TABLE "McpServer" ADD COLUMN "memoryLimitMb" INTEGER; diff --git a/src/db/prisma/schema.prisma b/src/db/prisma/schema.prisma index 4ec2538..0da281f 100644 --- a/src/db/prisma/schema.prisma +++ b/src/db/prisma/schema.prisma @@ -98,6 +98,13 @@ model McpServer { /// genuinely slow tools; the deadline exists so a wedged call answers instead /// of hanging, not to cut off honest work. toolCallTimeoutSeconds Int? + + /// Per-server memory ceiling in MiB. Null = the orchestrator's 512 MiB + /// default. Raise it for a server that legitimately needs more than that: + /// docs-mcp-server scrapes with headless Chromium and is OOMKilled at the + /// default before it indexes a single page, with nothing in the logs to say + /// so — the pod simply restarts and the job queue is gone. + memoryLimitMb Int? version Int @default(1) createdAt DateTime @default(now()) updatedAt DateTime @updatedAt diff --git a/src/mcpd/src/bootstrap/ungated-global.ts b/src/mcpd/src/bootstrap/ungated-global.ts new file mode 100644 index 0000000..1787447 --- /dev/null +++ b/src/mcpd/src/bootstrap/ungated-global.ts @@ -0,0 +1,137 @@ +/** + * Bootstrap the `ungated-global` system project. + * + * Servers attached to this project are merged into EVERY project's MCP + * endpoint, in front of the gate: their tools are listed and callable + * before begin_session, and the project's prompt is prepended to every + * project's instructions. See docs/ungated-global.md. + * + * Seeding is ONCE-EVER, not idempotent-per-startup like the other + * bootstraps: the whole point of `mcpctl delete server calcmcp -p + * ungated-global` is that the default stays gone. A marker key in the + * `mcpctl-system-settings` secret records that seeding already happened, + * so later startups leave admin deletions alone. + */ + +import type { PrismaClient } from '@prisma/client'; +import { SYSTEM_SETTINGS_SECRET } from './system-settings.js'; +import { DEFAULT_PLAINTEXT_BACKEND_NAME } from './secret-backends.js'; + +/** Well-known project whose servers are exposed under every project, pre-gate. */ +export const UNGATED_GLOBAL_PROJECT_NAME = 'ungated-global'; + +/** Settings key: set after the one-time seed so deletions stay deleted. */ +export const SETTING_UNGATED_GLOBAL_SEEDED = 'ungatedGlobalSeeded'; + +/** Well-known email for the system user (shared with bootstrapSystemProject). */ +const SYSTEM_USER_EMAIL = 'system@mcpctl.local'; + +const DEFAULT_SERVER = { + name: 'calcmcp', + description: + 'Scientific calculator (SymPy/NumPy/SciPy): symbolic algebra, calculus, ' + + 'equation & ODE solving, linear algebra, statistics, FFT. ' + + 'Use it for EVERY mathematical calculation instead of computing yourself.', + dockerImage: 'mysources.co.uk/michal/calcmcp:1.1.0', + transport: 'STDIO' as const, + repositoryUrl: 'https://mysources.co.uk/michal/CalcMCP', + healthCheck: { + tool: 'symbolic_tool', + arguments: { operation: 'simplify', expression: '1+1' }, + intervalSeconds: 300, + timeoutSeconds: 10, + }, +}; + +const DEFAULT_PROMPT = `Mathematical reliability rule: never perform mathematical calculations yourself, not even simple-looking arithmetic. Always use the calcmcp tools (symbolic_tool, numpy_tool, scipy_tool) for every calculation — arithmetic, algebra, calculus, equations, matrices, statistics. They are available immediately, no begin_session needed. Better safe than sorry: if an answer involves numbers, compute it with calcmcp.`; + +async function isSeeded(prisma: PrismaClient): Promise { + const secret = await prisma.secret.findUnique({ where: { name: SYSTEM_SETTINGS_SECRET } }); + if (!secret) return false; + const data = (secret.data ?? {}) as Record; + return data[SETTING_UNGATED_GLOBAL_SEEDED] === true; +} + +async function markSeeded(prisma: PrismaClient): Promise { + const existing = await prisma.secret.findUnique({ where: { name: SYSTEM_SETTINGS_SECRET } }); + const data = { ...((existing?.data ?? {}) as Record), [SETTING_UNGATED_GLOBAL_SEEDED]: true }; + const keyNames = Object.keys(data).sort(); + if (existing) { + await prisma.secret.update({ + where: { name: SYSTEM_SETTINGS_SECRET }, + data: { data, keyNames, version: existing.version + 1 }, + }); + } else { + // The Secret.backendId FK is enforced at insert time, so a fresh row + // must point at a real backend. bootstrapUngatedGlobal runs after + // bootstrapSecretBackends, so the default plaintext backend exists; + // fall back to any backend if the name ever changes. + const backend = + (await prisma.secretBackend.findUnique({ where: { name: DEFAULT_PLAINTEXT_BACKEND_NAME } })) + ?? (await prisma.secretBackend.findFirst({ where: { isDefault: true } })) + ?? (await prisma.secretBackend.findFirst()); + if (!backend) { + // No backend at all — skip marking rather than crash startup. The seed + // above still ran; worst case it re-runs (idempotent upserts) next boot. + console.warn('ungated-global: no SecretBackend found, seed marker not written'); + return; + } + await prisma.secret.create({ + data: { name: SYSTEM_SETTINGS_SECRET, data, keyNames, backendId: backend.id }, + }); + } +} + +/** + * Seed the ungated-global project with its default calcmcp server, once ever. + * Safe to call on every startup; after the first successful seed (or after an + * admin deletes any of it) nothing is re-created. + */ +export async function bootstrapUngatedGlobal(prisma: PrismaClient): Promise { + if (await isSeeded(prisma)) return; + + const systemUser = await prisma.user.upsert({ + where: { email: SYSTEM_USER_EMAIL }, + create: { + email: SYSTEM_USER_EMAIL, + name: 'System', + passwordHash: '!locked', + role: 'USER', + }, + update: {}, + }); + + const project = await prisma.project.upsert({ + where: { name: UNGATED_GLOBAL_PROJECT_NAME }, + create: { + name: UNGATED_GLOBAL_PROJECT_NAME, + description: 'System project: servers here are exposed under every project, before the gate', + prompt: DEFAULT_PROMPT, + gated: false, + ownerId: systemUser.id, + }, + update: {}, + }); + + const server = await prisma.mcpServer.upsert({ + where: { name: DEFAULT_SERVER.name }, + create: { + name: DEFAULT_SERVER.name, + description: DEFAULT_SERVER.description, + dockerImage: DEFAULT_SERVER.dockerImage, + transport: DEFAULT_SERVER.transport, + repositoryUrl: DEFAULT_SERVER.repositoryUrl, + healthCheck: DEFAULT_SERVER.healthCheck, + }, + update: {}, + }); + + await prisma.projectServer.upsert({ + where: { projectId_serverId: { projectId: project.id, serverId: server.id } }, + create: { projectId: project.id, serverId: server.id }, + update: {}, + }); + + await markSeeded(prisma); + console.log(`Seeded ${UNGATED_GLOBAL_PROJECT_NAME} project with default server '${DEFAULT_SERVER.name}'`); +} diff --git a/src/mcpd/src/main.ts b/src/mcpd/src/main.ts index a4603d4..cb976da 100644 --- a/src/mcpd/src/main.ts +++ b/src/mcpd/src/main.ts @@ -65,6 +65,7 @@ import { registerPersonalityRoutes } from './routes/personalities.js'; import { registerWebUi } from './routes/web-ui.js'; import { bootstrapSystemProject } from './bootstrap/system-project.js'; import { mapUrlToPermission } from './middleware/url-permission.js'; +import { bootstrapUngatedGlobal } from './bootstrap/ungated-global.js'; import { registerLogRoutes } from './routes/logs.js'; import { errorLogBuffer } from './services/error-log-buffer.js'; import { bootstrapSystemSkills } from './bootstrap/system-skills.js'; @@ -286,6 +287,11 @@ async function main(): Promise { // pre-existing `Secret` rows are pointed at it. Idempotent per run. await bootstrapSecretBackends(prisma); + // Seed-once ungated-global project (default calcmcp server, deletable). + // MUST run after bootstrapSecretBackends: its seed marker lives in the + // mcpctl-system-settings Secret, whose creation needs a valid backend FK. + await bootstrapUngatedGlobal(prisma); + // CUID detection for RBAC name resolution const CUID_RE = /^c[^\s-]{8,}$/i; const nameResolvers: Record }> = { diff --git a/src/mcpd/src/repositories/mcp-server.repository.ts b/src/mcpd/src/repositories/mcp-server.repository.ts index b5edd92..bb48743 100644 --- a/src/mcpd/src/repositories/mcp-server.repository.ts +++ b/src/mcpd/src/repositories/mcp-server.repository.ts @@ -37,6 +37,7 @@ export class McpServerRepository implements IMcpServerRepository { secretDelivery: data.secretDelivery, entrypoint: (data.entrypoint ?? Prisma.DbNull) as Prisma.InputJsonValue, toolCallTimeoutSeconds: data.toolCallTimeoutSeconds ?? null, + memoryLimitMb: data.memoryLimitMb ?? null, }, }); } @@ -59,6 +60,7 @@ export class McpServerRepository implements IMcpServerRepository { if (data.secretDelivery !== undefined) updateData['secretDelivery'] = data.secretDelivery; if (data.entrypoint !== undefined) updateData['entrypoint'] = (data.entrypoint ?? Prisma.JsonNull) as Prisma.InputJsonValue; if (data.toolCallTimeoutSeconds !== undefined) updateData['toolCallTimeoutSeconds'] = data.toolCallTimeoutSeconds; + if (data.memoryLimitMb !== undefined) updateData['memoryLimitMb'] = data.memoryLimitMb; return this.prisma.mcpServer.update({ where: { id }, data: updateData }); } diff --git a/src/mcpd/src/routes/projects.ts b/src/mcpd/src/routes/projects.ts index c49ed37..a4aac10 100644 --- a/src/mcpd/src/routes/projects.ts +++ b/src/mcpd/src/routes/projects.ts @@ -49,21 +49,40 @@ export function registerProjectRoutes(app: FastifyInstance, service: ProjectServ reply.code(204); }); - // List servers in a project (for mcplocal discovery) + // List servers in a project (for mcplocal discovery). Servers of the + // ungated-global system project ride along for every project, flagged + // `ungatedGlobal: true` so mcplocal exempts them from the gate. A server + // attached to the project directly wins over its global copy. app.get<{ Params: { id: string } }>('/api/v1/projects/:id/servers', async (request) => { const project = await service.resolveAndGet(request.params.id); - return project.servers.map((ps) => ps.server); + const own = project.servers.map((ps) => ps.server); + const ownNames = new Set(own.map((s) => (s as Record).name as string)); + const globals = (await service.getGlobalServers(project.name)) + .filter((g) => !ownNames.has(g.name as string)); + return [...own, ...globals]; }); - // Get project instructions for LLM (prompt + server list) + // Get project instructions for LLM (prompt + server list). The + // ungated-global project's prompt is prepended so its directive (e.g. + // "always use calcmcp for math") reaches every project's model pre-gate. app.get<{ Params: { id: string } }>('/api/v1/projects/:id/instructions', async (request) => { const project = await service.resolveAndGet(request.params.id); + const globalPrompt = await service.getGlobalPrompt(project.name); + const ownNames = new Set(project.servers.map((ps) => (ps.server as Record).name as string)); + const globals = (await service.getGlobalServers(project.name)) + .filter((g) => !ownNames.has(g.name as string)); return { - prompt: project.prompt, - servers: project.servers.map((ps) => ({ - name: (ps.server as Record).name as string, - description: (ps.server as Record).description as string, - })), + prompt: [globalPrompt, project.prompt].filter(Boolean).join('\n\n'), + servers: [ + ...project.servers.map((ps) => ({ + name: (ps.server as Record).name as string, + description: (ps.server as Record).description as string, + })), + ...globals.map((g) => ({ + name: g.name as string, + description: g.description as string, + })), + ], }; }); } diff --git a/src/mcpd/src/services/chat.service.ts b/src/mcpd/src/services/chat.service.ts index 3251c4f..2c3eac5 100644 --- a/src/mcpd/src/services/chat.service.ts +++ b/src/mcpd/src/services/chat.service.ts @@ -22,6 +22,7 @@ * adapter default. `extra` is forwarded as-is for provider-specific knobs. */ import type { ChatMessage } from '@prisma/client'; +import { ambientContextLine } from '@mcpctl/shared'; import type { AgentService } from './agent.service.js'; import type { LlmService } from './llm.service.js'; import { effectivePoolName } from './llm.service.js'; @@ -772,6 +773,9 @@ export class ChatService { ...sortedProjectPrompts.map((p) => p.content), ...personalityPromptContents, mergedParams.systemAppend ?? '', + // Rebuilt every turn, so the agent's clock stays current. Reads the + // same MCPLOCAL_AMBIENT_TZ env as mcplocal (see shared/ambient-time). + ambientContextLine(), ] .filter((s) => s.length > 0) .join('\n\n'); @@ -916,6 +920,8 @@ export class ChatService { baseSystem, ...projectPrompts.map((p) => p.content), mergedParams.systemAppend ?? '', + // Rebuilt every turn — the model's clock (same env as mcplocal). + ambientContextLine(), ] .filter((s) => s.length > 0) .join('\n\n'); diff --git a/src/mcpd/src/services/instance.service.ts b/src/mcpd/src/services/instance.service.ts index 4e68821..d81e657 100644 --- a/src/mcpd/src/services/instance.service.ts +++ b/src/mcpd/src/services/instance.service.ts @@ -481,6 +481,16 @@ export class InstanceService { spec.containerPort = server.containerPort ?? 3000; } + // Per-server memory ceiling. Null leaves DEFAULT_MEMORY_LIMIT (512 MiB) + // in place, which is right for an API proxy and fatal for a server that + // drives a browser: docs-mcp-server was OOMKilled seconds into every + // scrape, and an OOMKill leaves no error — the pod restarts, the + // in-memory job queue is gone, and the instance reports healthy again. + const memoryLimitMb = server.memoryLimitMb as number | null; + if (memoryLimitMb !== null && memoryLimitMb !== undefined) { + spec.memoryLimit = memoryLimitMb * 1024 * 1024; + } + // Volumes are keyed on the server, not this instance: `spec.name` carries // the instance id and a claim named after it would be torn down on the // next server edit — exactly when the data has to survive. diff --git a/src/mcpd/src/services/project.service.ts b/src/mcpd/src/services/project.service.ts index d1d982c..97ae671 100644 --- a/src/mcpd/src/services/project.service.ts +++ b/src/mcpd/src/services/project.service.ts @@ -2,6 +2,7 @@ import type { IProjectRepository, ProjectWithRelations } from '../repositories/p import type { IMcpServerRepository } from '../repositories/interfaces.js'; import { CreateProjectSchema, UpdateProjectSchema } from '../validation/project.schema.js'; import { NotFoundError, ConflictError } from './mcp-server.service.js'; +import { UNGATED_GLOBAL_PROJECT_NAME } from '../bootstrap/ungated-global.js'; import type { McpConfig } from './mcp-config-generator.js'; export class ProjectService { @@ -22,6 +23,39 @@ export class ProjectService { return project; } + /** + * Servers of the `ungated-global` system project, for merging into another + * project's server list / instructions. Returns [] when asked for the + * global project itself (no self-merge), when the project was deleted, or + * on any lookup failure — global servers are an additive convenience and + * must never break a project endpoint. + */ + async getGlobalServers(forProjectName: string): Promise>> { + if (forProjectName === UNGATED_GLOBAL_PROJECT_NAME) return []; + try { + const global = await this.projectRepo.findByName(UNGATED_GLOBAL_PROJECT_NAME); + if (global === null) return []; + return global.servers.map((ps) => ({ + ...(ps.server as unknown as Record), + ungatedGlobal: true, + })); + } catch { + return []; + } + } + + /** Prompt of the `ungated-global` project (prepended to every project's instructions). */ + async getGlobalPrompt(forProjectName: string): Promise { + if (forProjectName === UNGATED_GLOBAL_PROJECT_NAME) return null; + try { + const global = await this.projectRepo.findByName(UNGATED_GLOBAL_PROJECT_NAME); + const prompt = global?.prompt?.trim(); + return prompt ? prompt : null; + } catch { + return null; + } + } + /** Resolve by ID or name. */ async resolveAndGet(idOrName: string): Promise { // Try by ID first diff --git a/src/mcpd/src/validation/mcp-server.schema.ts b/src/mcpd/src/validation/mcp-server.schema.ts index 7eaaed0..6ef0e36 100644 --- a/src/mcpd/src/validation/mcp-server.schema.ts +++ b/src/mcpd/src/validation/mcp-server.schema.ts @@ -54,6 +54,10 @@ export const CreateMcpServerSchema = z.object({ // the deadline exists so a wedged call answers instead of hanging, and a // value beyond that is indistinguishable from no deadline at all. toolCallTimeoutSeconds: z.number().int().min(1).max(3600).optional(), + // Per-server memory ceiling in MiB. Floor 64 because nothing useful runs + // below it; ceiling 16384 so a typo cannot ask for a pod no node can + // schedule, which fails as a silently Pending instance rather than an error. + memoryLimitMb: z.number().int().min(64).max(16384).optional(), }).refine( (s) => s.volumes.length === 0 || s.replicas <= 1, { @@ -91,6 +95,7 @@ export const UpdateMcpServerSchema = z.object({ secretDelivery: SecretDeliverySchema.optional(), entrypoint: z.array(z.string()).nullable().optional(), toolCallTimeoutSeconds: z.number().int().min(1).max(3600).nullable().optional(), + memoryLimitMb: z.number().int().min(64).max(16384).nullable().optional(), }); export type CreateMcpServerInput = z.infer; diff --git a/src/mcpd/tests/bootstrap-ungated-global.test.ts b/src/mcpd/tests/bootstrap-ungated-global.test.ts new file mode 100644 index 0000000..7bb7099 --- /dev/null +++ b/src/mcpd/tests/bootstrap-ungated-global.test.ts @@ -0,0 +1,117 @@ +import { describe, it, expect, vi, beforeEach } from 'vitest'; +import { + bootstrapUngatedGlobal, + UNGATED_GLOBAL_PROJECT_NAME, + SETTING_UNGATED_GLOBAL_SEEDED, +} from '../src/bootstrap/ungated-global.js'; +import { SYSTEM_SETTINGS_SECRET } from '../src/bootstrap/system-settings.js'; +import type { PrismaClient } from '@prisma/client'; + +function mockPrisma(opts: { settingsData?: Record | null } = {}): PrismaClient { + const secretRow = opts.settingsData + ? { id: 'secret-1', name: SYSTEM_SETTINGS_SECRET, data: opts.settingsData, version: 3 } + : null; + + return { + secret: { + findUnique: vi.fn(async () => secretRow), + update: vi.fn(async () => ({})), + create: vi.fn(async () => ({})), + }, + secretBackend: { + findUnique: vi.fn(async () => ({ id: 'backend-default', name: 'default', isDefault: true })), + findFirst: vi.fn(async () => ({ id: 'backend-default', name: 'default', isDefault: true })), + }, + user: { + upsert: vi.fn(async () => ({ id: 'sys-user-id', email: 'system@mcpctl.local' })), + }, + project: { + upsert: vi.fn(async (args: { where: { name: string } }) => ({ id: 'proj-ug', name: args.where.name })), + }, + mcpServer: { + upsert: vi.fn(async (args: { where: { name: string } }) => ({ id: 'srv-calc', name: args.where.name })), + }, + projectServer: { + upsert: vi.fn(async () => ({})), + }, + } as unknown as PrismaClient; +} + +describe('bootstrapUngatedGlobal', () => { + let prisma: PrismaClient; + + beforeEach(() => { + prisma = mockPrisma(); + }); + + it('seeds the ungated-global project with the calcmcp server and attaches it', async () => { + await bootstrapUngatedGlobal(prisma); + + expect(prisma.project.upsert).toHaveBeenCalledWith( + expect.objectContaining({ + where: { name: UNGATED_GLOBAL_PROJECT_NAME }, + create: expect.objectContaining({ gated: false, prompt: expect.stringContaining('calcmcp') }), + }), + ); + expect(prisma.mcpServer.upsert).toHaveBeenCalledWith( + expect.objectContaining({ + where: { name: 'calcmcp' }, + create: expect.objectContaining({ + dockerImage: expect.stringContaining('calcmcp'), + transport: 'STDIO', + }), + }), + ); + expect(prisma.projectServer.upsert).toHaveBeenCalledWith( + expect.objectContaining({ + where: { projectId_serverId: { projectId: 'proj-ug', serverId: 'srv-calc' } }, + }), + ); + }); + + it('marks the seed as done in the system settings secret', async () => { + await bootstrapUngatedGlobal(prisma); + + // Settings secret didn't exist → created with the marker key and a valid backend FK + expect(prisma.secret.create).toHaveBeenCalledWith( + expect.objectContaining({ + data: expect.objectContaining({ + name: SYSTEM_SETTINGS_SECRET, + backendId: 'backend-default', + data: expect.objectContaining({ [SETTING_UNGATED_GLOBAL_SEEDED]: true }), + }), + }), + ); + }); + + it('does NOT re-seed when the marker is set (deletions stay deleted)', async () => { + prisma = mockPrisma({ settingsData: { [SETTING_UNGATED_GLOBAL_SEEDED]: true } }); + + await bootstrapUngatedGlobal(prisma); + + expect(prisma.project.upsert).not.toHaveBeenCalled(); + expect(prisma.mcpServer.upsert).not.toHaveBeenCalled(); + expect(prisma.projectServer.upsert).not.toHaveBeenCalled(); + expect(prisma.secret.update).not.toHaveBeenCalled(); + expect(prisma.secret.create).not.toHaveBeenCalled(); + }); + + it('preserves existing settings keys when writing the marker', async () => { + prisma = mockPrisma({ settingsData: { allowSelfPasswordChange: false } }); + + await bootstrapUngatedGlobal(prisma); + + expect(prisma.secret.update).toHaveBeenCalledWith( + expect.objectContaining({ + where: { name: SYSTEM_SETTINGS_SECRET }, + data: expect.objectContaining({ + data: { + allowSelfPasswordChange: false, + [SETTING_UNGATED_GLOBAL_SEEDED]: true, + }, + version: 4, + }), + }), + ); + }); +}); diff --git a/src/mcpd/tests/chat-service-project.test.ts b/src/mcpd/tests/chat-service-project.test.ts index 6bb21ec..db9be9e 100644 --- a/src/mcpd/tests/chat-service-project.test.ts +++ b/src/mcpd/tests/chat-service-project.test.ts @@ -155,10 +155,11 @@ describe('ChatService — project chat', () => { expect(t.projectId).toBe('proj-1'); expect(t.agentId).toBeNull(); expect(t.ownerId).toBe('owner-1'); - // system message = project.prompt then HIGH then LOW + // system message = project.prompt then HIGH then LOW, then the ambient + // context line (the model's clock, appended every turn). const sys = (adapter.infer as ReturnType).mock.calls[0][0].body.messages[0]; expect(sys.role).toBe('system'); - expect(sys.content).toBe('Project system prompt.\n\nHIGH\n\nLOW'); + expect(sys.content).toMatch(/^Project system prompt\.\n\nHIGH\n\nLOW\n\n\[context\] now: /); }); it('routes tool calls to the project dispatcher with projectId', async () => { diff --git a/src/mcpd/tests/chat-service.test.ts b/src/mcpd/tests/chat-service.test.ts index 9d02bf4..6de6109 100644 --- a/src/mcpd/tests/chat-service.test.ts +++ b/src/mcpd/tests/chat-service.test.ts @@ -397,6 +397,8 @@ describe('ChatService', () => { expect(text.indexOf('HIGH prompt')).toBeLessThan(text.indexOf('LOW prompt')); // Agent's own system prompt leads. expect(text.indexOf('You are a helpful agent.')).toBeLessThan(text.indexOf('HIGH prompt')); + // The ambient context line (the model's clock) closes the block. + expect(text).toMatch(/\n\n\[context\] now: .+$/); }); it('refuses tool calls when the agent has no project attached', async () => { diff --git a/src/mcpd/tests/instance-service.test.ts b/src/mcpd/tests/instance-service.test.ts index 2e4d89e..5d860e0 100644 --- a/src/mcpd/tests/instance-service.test.ts +++ b/src/mcpd/tests/instance-service.test.ts @@ -70,7 +70,7 @@ function mockOrchestrator(): McpOrchestrator { }; } -function makeServer(overrides: Partial<{ id: string; name: string; replicas: number; dockerImage: string | null; externalUrl: string | null; transport: string; command: unknown; containerPort: number | null; volumes: unknown }> = {}) { +function makeServer(overrides: Partial<{ id: string; name: string; replicas: number; dockerImage: string | null; externalUrl: string | null; transport: string; command: unknown; containerPort: number | null; volumes: unknown; memoryLimitMb: number | null }> = {}) { return { id: overrides.id ?? 'srv-1', name: overrides.name ?? 'slack', @@ -83,6 +83,7 @@ function makeServer(overrides: Partial<{ id: string; name: string; replicas: num externalUrl: overrides.externalUrl ?? null, command: overrides.command ?? null, containerPort: overrides.containerPort ?? null, + memoryLimitMb: overrides.memoryLimitMb ?? null, replicas: overrides.replicas ?? 1, env: [], version: 1, @@ -191,6 +192,37 @@ describe('InstanceService', () => { }); }); + describe('memory limit', () => { + // A server that drives a browser (docs-mcp-server scrapes with headless + // Chromium) is OOMKilled seconds into its first job at the orchestrator's + // 512 MiB default, and an OOMKill says nothing: the pod restarts, the + // in-memory job queue is gone, and the instance reads healthy again. + it('converts the declared MiB ceiling into the spec, in bytes', async () => { + vi.mocked(serverRepo.findById).mockResolvedValue(makeServer({ + name: 'docs', + memoryLimitMb: 2048, + })); + vi.mocked(instanceRepo.findAll).mockResolvedValue([]); + + await service.reconcile('srv-1'); + + const spec = vi.mocked(orchestrator.createContainer).mock.calls[0]![0]; + expect(spec.memoryLimit).toBe(2048 * 1024 * 1024); + }); + + it('leaves the spec free of a limit when the server declares none', async () => { + // Absent, not zero: the orchestrator falls back to DEFAULT_MEMORY_LIMIT, + // so every existing server keeps exactly the limit it has today. + vi.mocked(serverRepo.findById).mockResolvedValue(makeServer({})); + vi.mocked(instanceRepo.findAll).mockResolvedValue([]); + + await service.reconcile('srv-1'); + + const spec = vi.mocked(orchestrator.createContainer).mock.calls[0]![0]; + expect(spec.memoryLimit).toBeUndefined(); + }); + }); + describe('reconcile', () => { it('starts instances when below desired replicas', async () => { vi.mocked(serverRepo.findById).mockResolvedValue(makeServer({ replicas: 2 })); diff --git a/src/mcpd/tests/mcp-server-repository-fields.test.ts b/src/mcpd/tests/mcp-server-repository-fields.test.ts index 843abaf..bd5f53a 100644 --- a/src/mcpd/tests/mcp-server-repository-fields.test.ts +++ b/src/mcpd/tests/mcp-server-repository-fields.test.ts @@ -45,4 +45,31 @@ describe('McpServerRepository field mapping', () => { } as never); expect(create.mock.calls[0]?.[0].data).toMatchObject({ secretDelivery: 'injector' }); }); + + it('persists memoryLimitMb on update', async () => { + const { spy, update } = prismaSpy(); + await new McpServerRepository(spy).update('id1', { memoryLimitMb: 2048 }); + expect(update.mock.calls[0]?.[0].data).toMatchObject({ memoryLimitMb: 2048 }); + }); + + it('persists memoryLimitMb on create, and nulls it when absent', async () => { + const { spy, create } = prismaSpy(); + await new McpServerRepository(spy).create({ + name: 'x', description: '', transport: 'STDIO', replicas: 1, env: [], volumes: [], + memoryLimitMb: 2048, + } as never); + expect(create.mock.calls[0]?.[0].data).toMatchObject({ memoryLimitMb: 2048 }); + + const second = prismaSpy(); + await new McpServerRepository(second.spy).create({ + name: 'y', description: '', transport: 'STDIO', replicas: 1, env: [], volumes: [], + } as never); + expect(second.create.mock.calls[0]?.[0].data).toMatchObject({ memoryLimitMb: null }); + }); + + it('leaves memoryLimitMb untouched on update when not supplied', async () => { + const { spy, update } = prismaSpy(); + await new McpServerRepository(spy).update('id1', { description: 'x' }); + expect(update.mock.calls[0]?.[0].data ?? {}).not.toHaveProperty('memoryLimitMb'); + }); }); diff --git a/src/mcpd/tests/project-routes.test.ts b/src/mcpd/tests/project-routes.test.ts index c21757e..e3c119d 100644 --- a/src/mcpd/tests/project-routes.test.ts +++ b/src/mcpd/tests/project-routes.test.ts @@ -345,4 +345,78 @@ describe('Project Routes', () => { expect(body.serverOverrides).toEqual({ ha: { proxyModel: 'ha-special' } }); }); }); + + describe('ungated-global merge', () => { + const mkPs = (id: string, name: string, description = '') => + ({ id: `ps-${id}`, projectId: 'x', serverId: id, createdAt: new Date(), server: { id, name, description } }) as never; + + function repoWithGlobal(ownServers: never[] = []) { + const repo = mockProjectRepo(); + vi.mocked(repo.findById).mockImplementation(async (id: string) => + id === 'p1' ? makeProject({ id: 'p1', name: 'my-proj', prompt: 'own prompt', servers: ownServers }) : null); + vi.mocked(repo.findByName).mockImplementation(async (name: string) => { + if (name === 'ungated-global') { + return makeProject({ + id: 'p-ug', + name: 'ungated-global', + prompt: 'Always use calcmcp for math.', + gated: false, + servers: [mkPs('srv-calc', 'calcmcp', 'scientific calculator')], + }); + } + return null; + }); + return repo; + } + + it('GET /servers appends global servers flagged ungatedGlobal', async () => { + await createApp(repoWithGlobal([mkPs('srv-ha', 'ha')])); + const res = await app.inject({ method: 'GET', url: '/api/v1/projects/p1/servers' }); + expect(res.statusCode).toBe(200); + const body = res.json>(); + expect(body.map((s) => s.name)).toEqual(['ha', 'calcmcp']); + expect(body[0]!.ungatedGlobal).toBeUndefined(); + expect(body[1]!.ungatedGlobal).toBe(true); + }); + + it('GET /servers does not duplicate a server also attached directly', async () => { + await createApp(repoWithGlobal([mkPs('srv-calc', 'calcmcp')])); + const res = await app.inject({ method: 'GET', url: '/api/v1/projects/p1/servers' }); + const body = res.json>(); + expect(body).toHaveLength(1); + expect(body[0]!.name).toBe('calcmcp'); + expect(body[0]!.ungatedGlobal).toBeUndefined(); + }); + + it('GET /servers on ungated-global itself does not self-merge', async () => { + const repo = repoWithGlobal(); + vi.mocked(repo.findById).mockImplementation(async (id: string) => + id === 'p-ug' + ? makeProject({ id: 'p-ug', name: 'ungated-global', servers: [mkPs('srv-calc', 'calcmcp')] }) + : null); + await createApp(repo); + const res = await app.inject({ method: 'GET', url: '/api/v1/projects/p-ug/servers' }); + const body = res.json>(); + expect(body).toHaveLength(1); + }); + + it('GET /instructions prepends the global prompt and lists global servers', async () => { + await createApp(repoWithGlobal([mkPs('srv-ha', 'ha', 'home assistant')])); + const res = await app.inject({ method: 'GET', url: '/api/v1/projects/p1/instructions' }); + expect(res.statusCode).toBe(200); + const body = res.json<{ prompt: string; servers: Array<{ name: string }> }>(); + expect(body.prompt).toBe('Always use calcmcp for math.\n\nown prompt'); + expect(body.servers.map((s) => s.name)).toEqual(['ha', 'calcmcp']); + }); + + it('GET /servers is unchanged when ungated-global does not exist', async () => { + const repo = mockProjectRepo(); + vi.mocked(repo.findById).mockImplementation(async (id: string) => + id === 'p1' ? makeProject({ id: 'p1', name: 'my-proj', servers: [mkPs('srv-ha', 'ha')] }) : null); + await createApp(repo); + const res = await app.inject({ method: 'GET', url: '/api/v1/projects/p1/servers' }); + const body = res.json>(); + expect(body.map((s) => s.name)).toEqual(['ha']); + }); + }); }); diff --git a/src/mcplocal/src/discovery.ts b/src/mcplocal/src/discovery.ts index 1d52a1e..951a127 100644 --- a/src/mcplocal/src/discovery.ts +++ b/src/mcplocal/src/discovery.ts @@ -11,6 +11,12 @@ interface McpdServer { status?: string; /** Per-server tool-call deadline override, in seconds. Null = use the global. */ toolCallTimeoutSeconds?: number | null; + /** + * True when the server comes from the `ungated-global` system project + * rather than this project's own attachments. mcplocal exempts such + * servers from the gate: their tools are listed and callable pre-begin_session. + */ + ungatedGlobal?: boolean; } /** @@ -28,12 +34,19 @@ export async function refreshUpstreams(router: McpRouter, mcpdClient: McpdClient * * @param authToken - Optional bearer token forwarded to mcpd for RBAC checks. */ +export interface ProjectUpstreams { + /** Names of all registered upstream servers. */ + names: string[]; + /** Subset that came from the ungated-global project (gate-exempt). */ + ungatedServers: string[]; +} + export async function refreshProjectUpstreams( router: McpRouter, mcpdClient: McpdClient, projectName: string, authToken?: string, -): Promise { +): Promise { const path = `/api/v1/projects/${encodeURIComponent(projectName)}/servers`; let servers: McpdServer[]; @@ -54,7 +67,11 @@ export async function refreshProjectUpstreams( // Bind a per-request client with the caller's bearer so each McpdUpstream // forwards the same identity that passed project discovery. const upstreamClient = authToken ? mcpdClient.withToken(authToken) : mcpdClient; - return syncUpstreams(router, upstreamClient, servers); + const names = await syncUpstreams(router, upstreamClient, servers); + return { + names, + ungatedServers: servers.filter((s) => s.ungatedGlobal === true).map((s) => s.name), + }; } /** @@ -167,12 +184,16 @@ function syncUpstreams(router: McpRouter, mcpdClient: McpdClient, servers: McpdS } } - // Add/update upstreams for each server + // Add/update upstreams for each server. Always rebuild, even when an + // upstream with this name already exists: McpdUpstream binds the caller's + // bearer token (and the server id) at construction, and a cached project + // router outlives both. Reusing the old object meant a token rotation left + // every pre-rotation server silently 401-ing on tools/list — attached, + // healthy, zero tools, no error — while servers attached after the rotation + // worked, because only THEY got an upstream carrying the fresh token. for (const server of servers) { - if (!currentNames.has(server.name)) { - const upstream = new McpdUpstream(server.id, server.name, toolClient, server.description, discoveryClient); - router.addUpstream(upstream); - } + const upstream = new McpdUpstream(server.id, server.name, toolClient, server.description, discoveryClient); + router.addUpstream(upstream); // Applied on every sync, not just on first registration: raising a // server's timeout should take effect at the next refresh rather than // requiring the upstream to be dropped and rebuilt. diff --git a/src/mcplocal/src/http/project-mcp-endpoint.ts b/src/mcplocal/src/http/project-mcp-endpoint.ts index 9a1340e..68fb73d 100644 --- a/src/mcplocal/src/http/project-mcp-endpoint.ts +++ b/src/mcplocal/src/http/project-mcp-endpoint.ts @@ -22,6 +22,8 @@ import type { TrafficCapture } from './traffic.js'; import { LLMProviderAdapter } from '../proxymodel/llm-adapter.js'; import { FileCache } from '../proxymodel/file-cache.js'; import { createDefaultPlugin } from '../proxymodel/plugins/default.js'; +import { createAmbientContextPlugin } from '../proxymodel/plugins/ambient-context.js'; +import { ambientContextLine } from '@mcpctl/shared'; import { createAgentsPlugin } from '../proxymodel/plugins/agents.js'; import { createFavouriteIndexPlugin } from '../proxymodel/plugins/favourite-index.js'; import { composePlugins } from '../proxymodel/plugins/compose.js'; @@ -92,7 +94,7 @@ export function registerProjectMcpEndpoint(app: FastifyInstance, mcpdClient: Mcp // Create new router or refresh existing one const router = existing?.router ?? new McpRouter(); - await refreshProjectUpstreams(router, mcpdClient, projectName, authToken); + const { ungatedServers } = await refreshProjectUpstreams(router, mcpdClient, projectName, authToken); // Resolve project LLM model: local override → mcpd recommendation → global default const localOverride = loadProjectLlmOverride(projectName); @@ -163,6 +165,9 @@ export function registerProjectMcpEndpoint(app: FastifyInstance, mcpdClient: Mcp providerRegistry: effectiveRegistry, }; if (resolvedModel) pluginConfig.modelOverride = resolvedModel; + // Servers from the ungated-global project bypass the gate: listed and + // callable before begin_session, in every project. + if (ungatedServers.length > 0) pluginConfig.ungatedServers = ungatedServers; // Route gate prompt-selection through a server Llm (mcpd inference proxy) // so cloud/server keys stay at the k8s level; the local personal-token // provider is the fallback. See credential-tiering. A dedicated fast @@ -190,6 +195,11 @@ export function registerProjectMcpEndpoint(app: FastifyInstance, mcpdClient: Mcp // project show up as virtual MCP servers in tools/list, regardless of // which proxymodel the project is using. chain.push(createAgentsPlugin()); + // Ambient context LAST: its onToolsList must see the gate's finished + // catalog (append get_current_time only when ungated), and its + // date/project instruction line belongs at the end of the merged + // instructions. Always on for every project. + chain.push(createAmbientContextPlugin()); router.setPlugin(composePlugins(chain)); // Fetch project instructions and set on router. Must ride the CALLER's @@ -247,6 +257,24 @@ export function registerProjectMcpEndpoint(app: FastifyInstance, mcpdClient: Mcp return response; } + /** + * Append the ambient "now" line to every tools/call result as a SEPARATE + * trailing content item — never mutate existing items, so clients that + * JSON.parse a single text item keep working. Endpoint-level on purpose: + * this is the only chokepoint that also covers virtual tools (which bypass + * plugin onToolCallAfter), gate intercepts, drill-down pages, and + * failureResponse results. Initialize instructions carry the same line but + * freeze at session start; this trailer is what keeps a days-old chat + * session's model on today's date. + */ + function appendAmbientTrailer(response: unknown, method?: string): unknown { + if (method !== 'tools/call') return response; + const r = response as { result?: { content?: unknown } }; + if (!r.result || !Array.isArray(r.result.content)) return response; + r.result.content.push({ type: 'text', text: ambientContextLine() }); + return response; + } + /** * Turn a deadline or an unexpected throw into something the client can read. * @@ -523,7 +551,10 @@ export function registerProjectMcpEndpoint(app: FastifyInstance, mcpdClient: Mcp try { await transport.send( - maybeAnnotateRecreation(response, sid, method) as unknown as JSONRPCMessage, + appendAmbientTrailer( + maybeAnnotateRecreation(response, sid, method), + method, + ) as unknown as JSONRPCMessage, ); } catch (sendErr) { console.error( diff --git a/src/mcplocal/src/proxymodel/plugin-context.ts b/src/mcplocal/src/proxymodel/plugin-context.ts index fbbc338..168eae4 100644 --- a/src/mcplocal/src/proxymodel/plugin-context.ts +++ b/src/mcplocal/src/proxymodel/plugin-context.ts @@ -19,6 +19,7 @@ export interface PluginContextDeps { cache: CacheProvider; log: StageLogger; discoverTools: () => Promise; + getToolDiscoveryFailures?: () => Array<{ server: string; message: string }>; routeToUpstream: (request: JsonRpcRequest) => Promise; fetchPromptIndex: () => Promise; getSystemPrompt: (name: string, fallback: string) => Promise; @@ -96,6 +97,10 @@ export class PluginContextImpl implements PluginSessionContext { return this.deps.discoverTools(); } + getToolDiscoveryFailures(): Array<{ server: string; message: string }> { + return this.deps.getToolDiscoveryFailures?.() ?? []; + } + routeToUpstream(request: JsonRpcRequest): Promise { return this.deps.routeToUpstream(request); } diff --git a/src/mcplocal/src/proxymodel/plugin.ts b/src/mcplocal/src/proxymodel/plugin.ts index 3fee6f0..4a7a122 100644 --- a/src/mcplocal/src/proxymodel/plugin.ts +++ b/src/mcplocal/src/proxymodel/plugin.ts @@ -36,6 +36,8 @@ export interface PluginSessionContext { // Upstream access discoverTools(): Promise; + /** Servers that contributed no tools on the last discoverTools run, with the error. */ + getToolDiscoveryFailures(): Array<{ server: string; message: string }>; routeToUpstream(request: JsonRpcRequest): Promise; // Prompt access diff --git a/src/mcplocal/src/proxymodel/plugins/ambient-context.ts b/src/mcplocal/src/proxymodel/plugins/ambient-context.ts new file mode 100644 index 0000000..caf1b72 --- /dev/null +++ b/src/mcplocal/src/proxymodel/plugins/ambient-context.ts @@ -0,0 +1,56 @@ +/** + * Ambient Context plugin — gives every project's model a clock. + * + * Always on, composed LAST in the project plugin chain so that: + * - onToolsList sees the gate's (and favourite-index's) finished list and + * only appends get_current_time to UNGATED catalogs; + * - its onInitialize line lands at the end of the merged instructions. + * + * The per-result "[context] now: …" trailer deliberately does NOT live here: + * virtual tools bypass plugin onToolCallAfter entirely (router dispatches + * them first), so the only chokepoint covering every tools/call result is + * endpoint-level — see appendAmbientTrailer in http/project-mcp-endpoint.ts. + */ +import { ambientContextLine, formatCurrentTimeResult } from '@mcpctl/shared'; +import type { ProxyModelPlugin } from '../plugin.js'; +import type { ToolDefinition } from '../types.js'; + +export const GET_CURRENT_TIME_TOOL_NAME = 'get_current_time'; + +function getCurrentTimeTool(): ToolDefinition { + return { + name: GET_CURRENT_TIME_TOOL_NAME, + description: + 'Current date and time, in UTC and the configured local timezone. ' + + 'Call this whenever a task depends on today\'s date or the current time.', + inputSchema: { type: 'object', properties: {} }, + }; +} + +export function createAmbientContextPlugin(): ProxyModelPlugin { + return { + name: 'ambient-context', + + async onSessionCreate(ctx) { + // Registered unconditionally: virtual dispatch precedes the gate + // intercept, so the tool is callable even while gated (same as + // read_prompts today). Visibility is controlled in onToolsList. + ctx.registerTool(getCurrentTimeTool(), async () => ({ + content: [{ type: 'text', text: formatCurrentTimeResult() }], + })); + }, + + async onInitialize(_request, ctx) { + return { instructions: `${ambientContextLine()} — project: ${ctx.projectName}` }; + }, + + async onToolsList(tools) { + // A gated catalog is [begin_session, ...gate-exempt ungated-global + // tools] (gate.ts onToolsList); begin_session never appears in an + // ungated list, so its presence remains the gated marker. + if (tools.some((t) => t.name === 'begin_session')) return tools; + if (tools.some((t) => t.name === GET_CURRENT_TIME_TOOL_NAME)) return tools; + return [...tools, getCurrentTimeTool()]; + }, + }; +} diff --git a/src/mcplocal/src/proxymodel/plugins/gate.ts b/src/mcplocal/src/proxymodel/plugins/gate.ts index 1b240b8..a0e8e26 100644 --- a/src/mcplocal/src/proxymodel/plugins/gate.ts +++ b/src/mcplocal/src/proxymodel/plugins/gate.ts @@ -7,6 +7,7 @@ * * This plugin replaces the hardcoded gate logic in router.ts. */ +import { ambientContextLine } from '@mcpctl/shared'; import type { JsonRpcRequest, JsonRpcResponse } from '../../types.js'; import type { ToolDefinition } from '../types.js'; import type { ProxyModelPlugin, PluginSessionContext } from '../plugin.js'; @@ -54,6 +55,12 @@ export interface GatePluginConfig { * personal-token provider registry. See credential-tiering principle. */ llmProvider?: string; + /** + * Server names (canonical, as in `server/tool`) exempt from the gate — + * typically servers of the ungated-global project. Their tools are listed + * and callable while the session is still gated. + */ + ungatedServers?: string[]; } const MAX_RESPONSE_CHARS = 24_000; @@ -72,6 +79,13 @@ export function createGatePlugin(config: GatePluginConfig = {}): ProxyModelPlugi // Per-session state tracking (plugin-scoped, not global SessionGate) const sessionGate = new SessionGate(); + // Gate-exempt servers (ungated-global project): tools pass the gate untouched. + const ungatedSet = new Set(config.ungatedServers ?? []); + const isGateExempt = (toolName: string): boolean => { + const slash = toolName.indexOf('/'); + return slash > 0 && ungatedSet.has(toolName.slice(0, slash)); + }; + return { name: 'gate', description: 'Gated session flow: begin_session → prompt selection → ungate.', @@ -121,12 +135,21 @@ export function createGatePlugin(config: GatePluginConfig = {}): ProxyModelPlugi // canonical `server/tool` would name functions the model cannot call. try { const tools = await ctx.discoverTools(); - if (tools.length > 0) { - parts.push('\nAvailable MCP server tools (accessible after begin_session):'); - for (const t of tools) { + const exempt = tools.filter((t) => isGateExempt(t.name)); + const gatedTools = tools.filter((t) => !isGateExempt(t.name)); + if (exempt.length > 0) { + parts.push('\nAvailable immediately (no begin_session required):'); + for (const t of exempt) { parts.push(` ${sanitizeWireName(t.name)}`); } } + if (gatedTools.length > 0) { + parts.push('\nAvailable MCP server tools (accessible after begin_session):'); + for (const t of gatedTools) { + parts.push(` ${sanitizeWireName(t.name)}`); + } + } + parts.push(...formatDiscoveryFailures(ctx.getToolDiscoveryFailures())); } catch { // Tool discovery is optional } @@ -158,9 +181,9 @@ export function createGatePlugin(config: GatePluginConfig = {}): ProxyModelPlugi }, async onToolsList(tools, ctx) { - // When gated, only show begin_session + // When gated, show begin_session plus any gate-exempt (ungated-global) tools if (sessionGate.isGated(ctx.sessionId)) { - return [getBeginSessionTool(llmSelector)]; + return [getBeginSessionTool(llmSelector), ...tools.filter((t) => isGateExempt(t.name))]; } // When ungated, show upstream tools + read_prompts + propose_prompt (no begin_session) @@ -168,8 +191,9 @@ export function createGatePlugin(config: GatePluginConfig = {}): ProxyModelPlugi }, async onToolCallBefore(toolName, args, request, ctx) { - // If gated and trying to call a real tool, auto-ungate via keyword extraction - if (sessionGate.isGated(ctx.sessionId)) { + // If gated and trying to call a real tool, auto-ungate via keyword extraction. + // Gate-exempt (ungated-global) tools pass straight through without ungating. + if (sessionGate.isGated(ctx.sessionId) && !isGateExempt(toolName)) { return handleGatedIntercept(request, ctx, toolName, args, sessionGate, tagMatcher); } return null; @@ -177,6 +201,22 @@ export function createGatePlugin(config: GatePluginConfig = {}): ProxyModelPlugi }; } +/** + * A partial catalog must never look like a healthy one. When attached servers + * contributed no tools, say so in-band — the client has no other channel to + * learn that six servers silently vanished from its tool list. + */ +function formatDiscoveryFailures(failures: Array<{ server: string; message: string }>): string[] { + if (failures.length === 0) return []; + const parts: string[] = []; + parts.push(`\n⚠ WARNING: ${failures.length} attached server(s) contributed NO tools to this catalog:`); + for (const f of failures) { + parts.push(` - ${f.server}: ${f.message}`); + } + parts.push('The tool list above is INCOMPLETE. Report this to the user instead of treating a missing tool as nonexistent.'); + return parts; +} + // ── begin_session tool definition ── function getBeginSessionTool(llmSelector: LlmPromptSelector | null): ToolDefinition { @@ -396,6 +436,10 @@ async function handleBeginSession( // Build response const responseParts: string[] = []; + // The model starting this session has no clock; the briefing anchors it. + // (The endpoint's per-result trailer keeps later turns current.) + responseParts.push(`${ambientContextLine()}\n`); + if (degradedReason !== null) { responseParts.push( `⚠ Smart prompt-selection unavailable (${degradedReason}). ` @@ -442,6 +486,7 @@ async function handleBeginSession( responseParts.push(` ${sanitizeWireName(t.name)}`); } } + responseParts.push(...formatDiscoveryFailures(ctx.getToolDiscoveryFailures())); } catch { // Tool discovery is optional } diff --git a/src/mcplocal/src/router.ts b/src/mcplocal/src/router.ts index 7aa8c76..4871939 100644 --- a/src/mcplocal/src/router.ts +++ b/src/mcplocal/src/router.ts @@ -75,6 +75,8 @@ export class McpRouter { // `tools/list` from re-hitting slow/dead upstreams and absorbs negative results so one // dead server only stalls the first POST, not every subsequent one. private listCache = new Map(); + /** Per-server failures from the most recent discoverTools run. */ + private toolDiscoveryFailures: Array<{ server: string; message: string }> = []; private readonly LIST_CACHE_POSITIVE_TTL_MS = 30_000; private readonly LIST_CACHE_NEGATIVE_TTL_MS = 30_000; @@ -176,6 +178,7 @@ export class McpRouter { error: (msg: string) => console.error(`[plugin:${sessionId.slice(0, 8)}] ${msg}`), }, discoverTools: () => this.discoverTools(), + getToolDiscoveryFailures: () => this.getToolDiscoveryFailures(), routeToUpstream: (request) => this.routeNamespacedCall(request, 'name', this.toolToServer), fetchPromptIndex: () => this.fetchPromptIndex(), getSystemPrompt: (name, fallback) => this.getSystemPrompt(name, fallback), @@ -329,7 +332,7 @@ export class McpRouter { const started = Date.now(); let cachedCount = 0; let freshCount = 0; - const failed: string[] = []; + const failed: Array<{ server: string; message: string }> = []; // Discover tools from all servers in parallel so one slow server doesn't block the rest const entries = [...this.upstreams.entries()]; @@ -369,7 +372,7 @@ export class McpRouter { const cached = result.value.cached; if (cached.kind === 'err') { cachedCount++; - failed.push(serverName); + failed.push({ server: serverName, message: cached.message }); continue; } response = { jsonrpc: '2.0', id: `cached-${serverName}`, result: cached.result }; @@ -381,7 +384,7 @@ export class McpRouter { const message = (response.error as { message?: string }).message ?? 'unknown error'; this.setListCacheEntry(serverName, 'tools/list', { kind: 'err', message, fetchedAt: Date.now() }); console.warn(`[discoverTools] ${serverName}: ${message}`); - failed.push(serverName); + failed.push({ server: serverName, message }); continue; } if (response.result !== undefined) { @@ -410,16 +413,27 @@ export class McpRouter { } } + this.toolDiscoveryFailures = failed; + if (entries.length > 0) { const elapsed = Date.now() - started; const project = this.projectName ? ` project=${this.projectName}` : ''; - const failedStr = failed.length > 0 ? ` failed=[${failed.join(',')}]` : ''; + const failedStr = failed.length > 0 ? ` failed=[${failed.map((f) => f.server).join(',')}]` : ''; console.info(`[discoverTools]${project} fresh=${freshCount} cached=${cachedCount}${failedStr} elapsed=${elapsed}ms`); } return allTools; } + /** + * Servers that contributed no tools on the most recent `discoverTools` run, + * with the error that silenced them. A partial catalog must never look like + * a healthy one: the gate reads this to warn the client in-band. + */ + getToolDiscoveryFailures(): Array<{ server: string; message: string }> { + return this.toolDiscoveryFailures; + } + /** * Discover resources from all upstreams by calling resources/list on each. * Shares the per-server list cache with `discoverTools`. diff --git a/src/mcplocal/tests/ambient-time.test.ts b/src/mcplocal/tests/ambient-time.test.ts new file mode 100644 index 0000000..637146b --- /dev/null +++ b/src/mcplocal/tests/ambient-time.test.ts @@ -0,0 +1,75 @@ +import { describe, it, expect, afterEach } from 'vitest'; +import { + AMBIENT_TZ_ENV, + ambientTz, + formatAmbientNow, + ambientContextLine, + formatCurrentTimeResult, +} from '@mcpctl/shared'; + +const SUMMER = new Date('2026-07-01T09:30:00Z'); // London = BST, +01:00 +const WINTER = new Date('2026-01-15T09:30:00Z'); // London = GMT, +00:00 + +afterEach(() => { + delete process.env[AMBIENT_TZ_ENV]; +}); + +describe('ambientTz', () => { + it('returns undefined when unset', () => { + delete process.env[AMBIENT_TZ_ENV]; + expect(ambientTz()).toBeUndefined(); + }); + + it('returns a valid IANA name', () => { + process.env[AMBIENT_TZ_ENV] = 'Europe/London'; + expect(ambientTz()).toBe('Europe/London'); + }); + + it('falls back to undefined on an invalid name without throwing', () => { + process.env[AMBIENT_TZ_ENV] = 'Not/AZone'; + expect(ambientTz()).toBeUndefined(); + }); +}); + +describe('formatAmbientNow', () => { + it('renders UTC-only when no tz', () => { + expect(formatAmbientNow(SUMMER, undefined)).toBe('Wed 2026-07-01 09:30 UTC'); + }); + + it('renders local + UTC in summer (BST +01:00)', () => { + expect(formatAmbientNow(SUMMER, 'Europe/London')) + .toBe('Wed 2026-07-01 10:30 +01:00 Europe/London (09:30 UTC)'); + }); + + it('renders local + UTC in winter (GMT +00:00)', () => { + expect(formatAmbientNow(WINTER, 'Europe/London')) + .toBe('Thu 2026-01-15 09:30 +00:00 Europe/London (09:30 UTC)'); + }); + + it('stays short (~15 tokens)', () => { + expect(formatAmbientNow(SUMMER, 'Europe/London').length).toBeLessThan(70); + }); +}); + +describe('ambientContextLine', () => { + it('prefixes with [context] now:', () => { + delete process.env[AMBIENT_TZ_ENV]; + expect(ambientContextLine(SUMMER)).toBe('[context] now: Wed 2026-07-01 09:30 UTC'); + }); +}); + +describe('formatCurrentTimeResult', () => { + it('includes UTC ISO and epoch, no local line when tz unset', () => { + delete process.env[AMBIENT_TZ_ENV]; + const text = formatCurrentTimeResult(SUMMER); + expect(text).toContain('UTC: 2026-07-01T09:30:00.000Z'); + expect(text).toContain(`Unix epoch seconds: ${Math.floor(SUMMER.getTime() / 1000)}`); + expect(text).not.toContain('Europe/London'); + }); + + it('includes the local rendering when tz set', () => { + process.env[AMBIENT_TZ_ENV] = 'Europe/London'; + const text = formatCurrentTimeResult(SUMMER); + expect(text).toContain('Europe/London: Wed 2026-07-01 10:30 +01:00 Europe/London (09:30 UTC)'); + }); +}); diff --git a/src/mcplocal/tests/discovery.test.ts b/src/mcplocal/tests/discovery.test.ts index 1c010aa..8785d87 100644 --- a/src/mcplocal/tests/discovery.test.ts +++ b/src/mcplocal/tests/discovery.test.ts @@ -1,5 +1,5 @@ import { describe, it, expect, vi } from 'vitest'; -import { refreshUpstreams } from '../src/discovery.js'; +import { refreshUpstreams, refreshProjectUpstreams } from '../src/discovery.js'; import { McpRouter } from '../src/router.js'; function mockMcpdClient(servers: Array<{ id: string; name: string; transport: string }>) { @@ -69,3 +69,61 @@ describe('refreshUpstreams', () => { expect(router.getUpstreamNames()).toHaveLength(0); }); }); + +describe('refreshProjectUpstreams', () => { + /** + * Regression: a cached project router outlives the caller's bearer token. + * Upstreams used to be created only for NEW server names, so after a token + * rotation every pre-existing upstream kept posting the old token — six + * healthy servers 401'd on tools/list forever while newly attached ones + * worked. Every refresh must rebind existing upstreams to the fresh token. + */ + it('rebinds existing upstreams to the current auth token on every refresh', async () => { + const router = new McpRouter(); + const servers = [{ id: 'srv-1', name: 'gitea', transport: 'stdio' }]; + + // Base client: withToken(t) mints a child client whose posts we can count. + const children = new Map>(); + const base = mockMcpdClient(servers); + (base as any).forward = vi.fn(async () => ({ status: 200, body: servers })); + (base as any).withToken = vi.fn((t: string) => { + if (!children.has(t)) { + const child = mockMcpdClient(servers); + child.post = vi.fn(async () => ({ result: { tools: [{ name: 'merge_pull_request' }] } })); + children.set(t, child); + } + return children.get(t); + }); + + await refreshProjectUpstreams(router, base as any, 'sre', 'old-token'); + await refreshProjectUpstreams(router, base as any, 'sre', 'new-token'); + + const tools = await router.discoverTools(); + expect(tools.map((t) => t.name)).toEqual(['gitea/merge_pull_request']); + // The proxy call must ride the NEW token's client, not the stale one. + expect(children.get('new-token')!.post).toHaveBeenCalledWith('/api/v1/mcp/proxy', expect.anything()); + expect(children.get('old-token')!.post).not.toHaveBeenCalled(); + }); + + it('updates the server id when a server was recreated under the same name', async () => { + const router = new McpRouter(); + const base = mockMcpdClient([]); + const posts: unknown[] = []; + const child = mockMcpdClient([]); + child.post = vi.fn(async (_path: string, body: unknown) => { + posts.push(body); + return { result: { tools: [] } }; + }); + (base as any).withToken = vi.fn(() => child); + (base as any).forward = vi.fn() + .mockResolvedValueOnce({ status: 200, body: [{ id: 'srv-old', name: 'gitea', transport: 'stdio' }] }) + .mockResolvedValueOnce({ status: 200, body: [{ id: 'srv-new', name: 'gitea', transport: 'stdio' }] }); + + await refreshProjectUpstreams(router, base as any, 'sre', 't'); + await refreshProjectUpstreams(router, base as any, 'sre', 't'); + + await router.discoverTools(); + expect(posts).toHaveLength(1); + expect((posts[0] as { serverId: string }).serverId).toBe('srv-new'); + }); +}); diff --git a/src/mcplocal/tests/plugin-ambient-context.test.ts b/src/mcplocal/tests/plugin-ambient-context.test.ts new file mode 100644 index 0000000..516810b --- /dev/null +++ b/src/mcplocal/tests/plugin-ambient-context.test.ts @@ -0,0 +1,117 @@ +/** + * ambient-context plugin tests — get_current_time visibility (gated vs + * ungated, no double-append), callable result, and the initialize + * instruction line. Driven through a real McpRouter with the gate composed + * first, mirroring project-mcp-endpoint wiring (ambient-context LAST). + */ +import { describe, it, expect, vi } from 'vitest'; +import { McpRouter } from '../src/router.js'; +import type { UpstreamConnection, JsonRpcRequest, JsonRpcResponse } from '../src/types.js'; +import type { McpdClient } from '../src/http/mcpd-client.js'; +import { createGatePlugin } from '../src/proxymodel/plugins/gate.js'; +import { composePlugins } from '../src/proxymodel/plugins/compose.js'; +import { + createAmbientContextPlugin, + GET_CURRENT_TIME_TOOL_NAME, +} from '../src/proxymodel/plugins/ambient-context.js'; +import type { LLMProviderAdapter } from '../src/proxymodel/llm-adapter.js'; +import { MemoryCache } from '../src/proxymodel/cache.js'; + +function mockUpstream(name: string, tools: Array<{ name: string }>): UpstreamConnection { + return { + name, + isAlive: vi.fn(() => true), + close: vi.fn(async () => {}), + onNotification: vi.fn(), + send: vi.fn(async (req: JsonRpcRequest): Promise => { + if (req.method === 'tools/list') return { jsonrpc: '2.0', id: req.id, result: { tools } }; + if (req.method === 'resources/list') return { jsonrpc: '2.0', id: req.id, result: { resources: [] } }; + if (req.method === 'prompts/list') return { jsonrpc: '2.0', id: req.id, result: { prompts: [] } }; + return { jsonrpc: '2.0', id: req.id, error: { code: -32601, message: 'Not found' } }; + }), + } as UpstreamConnection; +} + +function mockMcpdClient(): McpdClient { + return { + get: vi.fn(async () => []), + post: vi.fn(async () => ({})), + put: vi.fn(async () => ({})), + delete: vi.fn(async () => {}), + forward: vi.fn(async () => ({ status: 200, body: {} })), + withHeaders: vi.fn(function (this: McpdClient) { return this; }), + } as unknown as McpdClient; +} + +function setup(opts: { gated?: boolean } = {}) { + const router = new McpRouter(); + router.setPromptConfig(mockMcpdClient(), 'test-project'); + router.setPlugin(composePlugins([ + createGatePlugin({ gated: opts.gated ?? false, providerRegistry: null }), + createAmbientContextPlugin(), + ])); + router.setProxyModel('default', { complete: async () => '', available: () => false } as unknown as LLMProviderAdapter, new MemoryCache()); + router.addUpstream(mockUpstream('k8s', [{ name: 'get_pods' }])); + return router; +} + +async function listTools(router: McpRouter, sessionId = 's1'): Promise { + const res = await router.route({ jsonrpc: '2.0', id: 2, method: 'tools/list' }, { sessionId }); + return (res.result as { tools: Array<{ name: string }> }).tools.map((t) => t.name); +} + +describe('ambient-context plugin', () => { + it('appends get_current_time to an ungated catalog, once', async () => { + const router = setup(); + await router.route({ jsonrpc: '2.0', id: 1, method: 'initialize' }, { sessionId: 's1' }); + + const names = await listTools(router); + expect(names).toContain('k8s/get_pods'); + expect(names.filter((n) => n === GET_CURRENT_TIME_TOOL_NAME)).toHaveLength(1); + + // Second list: still exactly one. + const again = await listTools(router); + expect(again.filter((n) => n === GET_CURRENT_TIME_TOOL_NAME)).toHaveLength(1); + }); + + it('does not leak get_current_time into a gated catalog', async () => { + const router = setup({ gated: true }); + await router.route({ jsonrpc: '2.0', id: 1, method: 'initialize' }, { sessionId: 's1' }); + + const names = await listTools(router); + expect(names).toEqual(['begin_session']); + }); + + it('lists get_current_time after begin_session ungates the session', async () => { + const router = setup({ gated: true }); + await router.route({ jsonrpc: '2.0', id: 1, method: 'initialize' }, { sessionId: 's1' }); + await router.route( + { jsonrpc: '2.0', id: 2, method: 'tools/call', params: { name: 'begin_session', arguments: { description: 'testing ambient context' } } }, + { sessionId: 's1' }, + ); + + const names = await listTools(router); + expect(names).toContain(GET_CURRENT_TIME_TOOL_NAME); + }); + + it('get_current_time returns the UTC time', async () => { + const router = setup(); + await router.route({ jsonrpc: '2.0', id: 1, method: 'initialize' }, { sessionId: 's1' }); + + const res = await router.route( + { jsonrpc: '2.0', id: 3, method: 'tools/call', params: { name: GET_CURRENT_TIME_TOOL_NAME, arguments: {} } }, + { sessionId: 's1' }, + ); + const text = (res.result as { content: Array<{ text: string }> }).content[0]!.text; + expect(text).toMatch(/^UTC: \d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}/m); + expect(text).toContain('Unix epoch seconds:'); + }); + + it('adds the context line and project name to initialize instructions', async () => { + const router = setup(); + const res = await router.route({ jsonrpc: '2.0', id: 1, method: 'initialize' }, { sessionId: 's1' }); + const instructions = (res.result as { instructions?: string }).instructions ?? ''; + expect(instructions).toContain('[context] now:'); + expect(instructions).toContain('project: test-project'); + }); +}); diff --git a/src/mcplocal/tests/plugin-gate.test.ts b/src/mcplocal/tests/plugin-gate.test.ts index f33c195..1ee3116 100644 --- a/src/mcplocal/tests/plugin-gate.test.ts +++ b/src/mcplocal/tests/plugin-gate.test.ts @@ -74,6 +74,8 @@ function setupPluginRouter(opts: { /** When set, the heavy provider's complete() rejects — exercises the loud fallback. */ llmError?: boolean; byteBudget?: number; + /** Gate-exempt server names (ungated-global project). */ + ungatedServers?: string[]; } = {}): { router: McpRouter; mcpdClient: McpdClient } { const router = new McpRouter(); const prompts = opts.prompts ?? samplePrompts; @@ -105,6 +107,7 @@ function setupPluginRouter(opts: { gated: opts.gated !== false, providerRegistry, byteBudget: opts.byteBudget, + ...(opts.ungatedServers ? { ungatedServers: opts.ungatedServers } : {}), }); router.setPlugin(gatePlugin); @@ -160,6 +163,63 @@ describe('Gate Plugin via setPlugin()', () => { }); }); + describe('gate-exempt (ungated-global) servers', () => { + it('lists exempt server tools alongside begin_session while gated', async () => { + const { router } = setupPluginRouter({ ungatedServers: ['calcmcp'] }); + router.addUpstream(mockUpstream('calcmcp', { tools: [{ name: 'symbolic_tool' }] })); + router.addUpstream(mockUpstream('ha', { tools: [{ name: 'get_entities' }] })); + await router.route({ jsonrpc: '2.0', id: 1, method: 'initialize' }, { sessionId: 's1' }); + + const res = await router.route( + { jsonrpc: '2.0', id: 2, method: 'tools/list' }, + { sessionId: 's1' }, + ); + const names = (res.result as { tools: Array<{ name: string }> }).tools.map((t) => t.name); + expect(names).toContain('begin_session'); + expect(names).toContain('calcmcp/symbolic_tool'); + expect(names).not.toContain('ha/get_entities'); + }); + + it('calls an exempt tool while gated without intercept and stays gated', async () => { + const { router } = setupPluginRouter({ ungatedServers: ['calcmcp'] }); + router.addUpstream(mockUpstream('calcmcp', { tools: [{ name: 'symbolic_tool' }] })); + await router.route({ jsonrpc: '2.0', id: 1, method: 'initialize' }, { sessionId: 's1' }); + + const callRes = await router.route( + { jsonrpc: '2.0', id: 2, method: 'tools/call', params: { name: 'calcmcp/symbolic_tool', arguments: { operation: 'simplify', expression: '1+1' } } }, + { sessionId: 's1' }, + ); + const content = (callRes.result as { content: Array<{ text: string }> }).content; + // Real upstream result, not a gate intercept briefing (the router + // strips the server prefix when forwarding to the upstream) + expect(content[0]!.text).toBe('Called symbolic_tool'); + + // Session must STILL be gated: non-exempt catalog stays hidden + const toolsRes = await router.route( + { jsonrpc: '2.0', id: 3, method: 'tools/list' }, + { sessionId: 's1' }, + ); + const names = (toolsRes.result as { tools: Array<{ name: string }> }).tools.map((t) => t.name); + expect(names).toContain('begin_session'); + }); + + it('lists exempt tools under "Available immediately" in initialize instructions', async () => { + const { router } = setupPluginRouter({ ungatedServers: ['calcmcp'] }); + router.addUpstream(mockUpstream('calcmcp', { tools: [{ name: 'symbolic_tool' }] })); + router.addUpstream(mockUpstream('ha', { tools: [{ name: 'get_entities' }] })); + + const res = await router.route( + { jsonrpc: '2.0', id: 1, method: 'initialize' }, + { sessionId: 's1' }, + ); + const instructions = (res.result as { instructions?: string }).instructions ?? ''; + expect(instructions).toContain('Available immediately (no begin_session required):'); + expect(instructions).toContain('calcmcp_symbolic_tool'); + expect(instructions).toContain('Available MCP server tools (accessible after begin_session):'); + expect(instructions).toContain('ha_get_entities'); + }); + }); + describe('tools/list gating', () => { it('shows only begin_session when session is gated', async () => { const { router } = setupPluginRouter(); @@ -199,6 +259,19 @@ describe('Gate Plugin via setPlugin()', () => { }); describe('begin_session', () => { + it('opens the briefing with the ambient context line (the model has no clock)', async () => { + const { router } = setupPluginRouter(); + await router.route({ jsonrpc: '2.0', id: 1, method: 'initialize' }, { sessionId: 's1' }); + + const res = await router.route( + { jsonrpc: '2.0', id: 2, method: 'tools/call', params: { name: 'begin_session', arguments: { tags: ['zigbee'] } } }, + { sessionId: 's1' }, + ); + + const text = ((res.result as { content: Array<{ text: string }> }).content[0]!.text); + expect(text).toMatch(/^\[context\] now: /); + }); + it('falls back with a visible warning when the LLM selector errors (no silent degrade)', async () => { const { router } = setupPluginRouter({ withLlm: true, llmError: true }); await router.route({ jsonrpc: '2.0', id: 1, method: 'initialize' }, { sessionId: 's1' }); diff --git a/src/mcplocal/tests/project-discovery.test.ts b/src/mcplocal/tests/project-discovery.test.ts index 0348d4a..7c1d670 100644 --- a/src/mcplocal/tests/project-discovery.test.ts +++ b/src/mcplocal/tests/project-discovery.test.ts @@ -27,9 +27,22 @@ describe('refreshProjectUpstreams', () => { ]); const registered = await refreshProjectUpstreams(router, client as any, 'smart-home'); - expect(registered).toEqual(['grafana', 'ha']); + expect(registered.names).toEqual(['grafana', 'ha']); + expect(registered.ungatedServers).toEqual([]); + }); + + it('reports servers flagged ungatedGlobal as gate-exempt', async () => { + const router = new McpRouter(); + const client = mockMcpdClient([ + { id: 'srv-1', name: 'grafana', transport: 'stdio' }, + { id: 'srv-2', name: 'calcmcp', transport: 'stdio', ungatedGlobal: true }, + ]); + + const registered = await refreshProjectUpstreams(router, client as any, 'smart-home'); + expect(registered.names).toEqual(['grafana', 'calcmcp']); + expect(registered.ungatedServers).toEqual(['calcmcp']); expect(router.getUpstreamNames()).toContain('grafana'); - expect(router.getUpstreamNames()).toContain('ha'); + expect(router.getUpstreamNames()).toContain('calcmcp'); expect(client.get).toHaveBeenCalledWith('/api/v1/projects/smart-home/servers'); }); @@ -85,7 +98,7 @@ describe('refreshProjectUpstreams', () => { const client = mockMcpdClient([]); const registered = await refreshProjectUpstreams(router, client as any, 'empty-project'); - expect(registered).toEqual([]); + expect(registered.names).toEqual([]); expect(router.getUpstreamNames()).toHaveLength(0); }); }); diff --git a/src/mcplocal/tests/project-mcp-endpoint-ambient.test.ts b/src/mcplocal/tests/project-mcp-endpoint-ambient.test.ts new file mode 100644 index 0000000..fd98143 --- /dev/null +++ b/src/mcplocal/tests/project-mcp-endpoint-ambient.test.ts @@ -0,0 +1,138 @@ +import { describe, it, expect, vi } from 'vitest'; +import Fastify from 'fastify'; +import { registerProjectMcpEndpoint } from '../src/http/project-mcp-endpoint.js'; +import type { McpRouter } from '../src/router.js'; +import type { JsonRpcRequest, JsonRpcResponse } from '../src/types.js'; + +/** + * Ambient-context E2E on /projects/:name/mcp: every tools/call result — + * upstream AND virtual — carries a trailing `[context] now:` content item, + * initialize instructions carry the same line, tools/list stays clean, and + * get_current_time is served on an ungated project. + */ + +vi.mock('../src/discovery.js', () => ({ + refreshProjectUpstreams: vi.fn(async (router: McpRouter) => { + router.addUpstream({ + name: 'websearch', + send: async (req: JsonRpcRequest): Promise => { + if (req.method === 'tools/list') { + return { + jsonrpc: '2.0', + id: req.id, + result: { tools: [{ name: 'fetch_content', description: 'fetch a page', inputSchema: { type: 'object' } }] }, + }; + } + if (req.method === 'tools/call') { + return { jsonrpc: '2.0', id: req.id, result: { content: [{ type: 'text', text: 'page body' }] } }; + } + return { jsonrpc: '2.0', id: req.id, result: {} }; + }, + close: async () => {}, + isAlive: () => true, + }); + return { names: ['websearch'], ungatedServers: [] }; + }), + // Ungated (content-pipeline style), like chat-web and personal-michal. + fetchProjectLlmConfig: vi.fn(async () => ({ gated: false, llmProvider: 'none' })), +})); + +vi.mock('../src/http/config.js', async () => { + const actual = await vi.importActual('../src/http/config.js'); + return { ...actual, loadProjectLlmOverride: vi.fn(() => undefined) }; +}); + +function mockMcpdClient() { + const client: Record = { + baseUrl: 'http://test:3100', + token: 'test-token', + get: vi.fn(async () => []), + post: vi.fn(async () => ({})), + put: vi.fn(), + delete: vi.fn(), + forward: vi.fn(async () => ({ status: 200, body: [] })), + withHeaders: vi.fn(), + withToken: vi.fn(), + withTimeout: vi.fn(), + }; + (client.withHeaders as ReturnType).mockReturnValue(client); + (client.withToken as ReturnType).mockReturnValue(client); + (client.withTimeout as ReturnType).mockReturnValue(client); + return client; +} + +function parseSse(body: string): JsonRpcResponse { + const dataLine = body.split('\n').find((l) => l.startsWith('data: ')); + if (!dataLine) throw new Error(`no SSE data line in: ${body}`); + return JSON.parse(dataLine.slice('data: '.length)) as JsonRpcResponse; +} + +const CONTEXT_PREFIX = '[context] now:'; + +describe('registerProjectMcpEndpoint ambient context', () => { + it('stamps instructions, tool results (upstream and virtual), but not tools/list', async () => { + const app = Fastify(); + registerProjectMcpEndpoint(app, mockMcpdClient() as never); + await app.ready(); + try { + const headers = { + 'content-type': 'application/json', + accept: 'application/json, text/event-stream', + }; + + const init = await app.inject({ + method: 'POST', + url: '/projects/chat-web/mcp', + headers, + payload: { jsonrpc: '2.0', id: 1, method: 'initialize', params: { protocolVersion: '2024-11-05', capabilities: {}, clientInfo: { name: 't', version: '0' } } }, + }); + expect(init.statusCode).toBe(200); + const initResponse = parseSse(init.body); + const instructions = (initResponse.result as { instructions?: string }).instructions ?? ''; + expect(instructions).toContain(CONTEXT_PREFIX); + expect(instructions).toContain('project: chat-web'); + // The initialize RESPONSE itself is not trailed (no result.content). + expect(JSON.stringify((initResponse.result as { content?: unknown }).content ?? null)).not.toContain(CONTEXT_PREFIX); + + const sessionId = init.headers['mcp-session-id'] as string; + const sessionHeaders = { ...headers, 'mcp-session-id': sessionId }; + + const list = await app.inject({ + method: 'POST', + url: '/projects/chat-web/mcp', + headers: sessionHeaders, + payload: { jsonrpc: '2.0', id: 2, method: 'tools/list' }, + }); + const listResponse = parseSse(list.body); + const names = (listResponse.result as { tools: Array<{ name: string }> }).tools.map((t) => t.name); + expect(names).toContain('get_current_time'); + // tools/list is never trailed. + expect(JSON.stringify(listResponse.result)).not.toContain(CONTEXT_PREFIX); + + // Upstream tool call: original content intact, trailer appended LAST. + const call = await app.inject({ + method: 'POST', + url: '/projects/chat-web/mcp', + headers: sessionHeaders, + payload: { jsonrpc: '2.0', id: 3, method: 'tools/call', params: { name: 'websearch_fetch_content', arguments: { url: 'https://x' } } }, + }); + const callContent = (parseSse(call.body).result as { content: Array<{ type: string; text: string }> }).content; + expect(callContent.length).toBeGreaterThanOrEqual(2); + expect(callContent[0]?.text).toContain('page body'); + expect(callContent[callContent.length - 1]?.text).toMatch(/^\[context\] now: /); + + // Virtual tool call (bypasses plugin onToolCallAfter): still trailed. + const timeCall = await app.inject({ + method: 'POST', + url: '/projects/chat-web/mcp', + headers: sessionHeaders, + payload: { jsonrpc: '2.0', id: 4, method: 'tools/call', params: { name: 'get_current_time', arguments: {} } }, + }); + const timeContent = (parseSse(timeCall.body).result as { content: Array<{ text: string }> }).content; + expect(timeContent[0]?.text).toContain('UTC:'); + expect(timeContent[timeContent.length - 1]?.text).toMatch(/^\[context\] now: /); + } finally { + await app.close(); + } + }); +}); diff --git a/src/mcplocal/tests/project-mcp-endpoint-wire-names.test.ts b/src/mcplocal/tests/project-mcp-endpoint-wire-names.test.ts index f1f2b3c..c063bec 100644 --- a/src/mcplocal/tests/project-mcp-endpoint-wire-names.test.ts +++ b/src/mcplocal/tests/project-mcp-endpoint-wire-names.test.ts @@ -40,7 +40,7 @@ vi.mock('../src/discovery.js', () => ({ close: async () => {}, isAlive: () => true, }); - return ['websearch']; + return { names: ['websearch'], ungatedServers: [] }; }), // gated: false → no gate plugin, the full catalog is served at initialize // (the chat-web configuration). diff --git a/src/mcplocal/tests/project-mcp-endpoint.test.ts b/src/mcplocal/tests/project-mcp-endpoint.test.ts index d5540b4..aa3feed 100644 --- a/src/mcplocal/tests/project-mcp-endpoint.test.ts +++ b/src/mcplocal/tests/project-mcp-endpoint.test.ts @@ -5,7 +5,7 @@ import { registerProjectMcpEndpoint } from '../src/http/project-mcp-endpoint.js' // Mock discovery module — we don't want real HTTP calls vi.mock('../src/discovery.js', () => ({ - refreshProjectUpstreams: vi.fn(async () => ['mock-server']), + refreshProjectUpstreams: vi.fn(async () => ({ names: ['mock-server'], ungatedServers: [] })), fetchProjectLlmConfig: vi.fn(async () => ({})), })); diff --git a/src/mcplocal/tests/toolcall-deadline-e2e.test.ts b/src/mcplocal/tests/toolcall-deadline-e2e.test.ts index f69bc2c..e8a3d2d 100644 --- a/src/mcplocal/tests/toolcall-deadline-e2e.test.ts +++ b/src/mcplocal/tests/toolcall-deadline-e2e.test.ts @@ -30,7 +30,7 @@ vi.mock('../src/discovery.js', () => ({ close: async () => { /* noop */ }, isAlive: () => true, }); - return ['blackhole']; + return { names: ['blackhole'], ungatedServers: [] }; }), fetchProjectLlmConfig: vi.fn(async () => ({ gated: false, llmProvider: 'none' })), })); diff --git a/src/shared/src/utils/ambient-time.ts b/src/shared/src/utils/ambient-time.ts new file mode 100644 index 0000000..149d4ba --- /dev/null +++ b/src/shared/src/utils/ambient-time.ts @@ -0,0 +1,93 @@ +/** + * Ambient time rendering — gives models a clock. + * + * Models consuming projects have no idea what day it is, so mcplocal stamps a + * short context line into initialize instructions, the begin_session + * briefing, and every tools/call result, and mcpd appends it to agent system + * blocks. One env var, `MCPLOCAL_AMBIENT_TZ` (an IANA name like + * "Europe/London", read by BOTH services despite the mcplocal-flavoured + * name), adds a local rendering next to UTC; unset or invalid means UTC only. + * + * Minute precision on purpose: the line stays ~15 tokens and does not churn + * within a minute. + */ + +export const AMBIENT_TZ_ENV = 'MCPLOCAL_AMBIENT_TZ'; + +let warnedInvalidTz: string | undefined; + +/** + * The configured IANA timezone, or undefined for UTC-only rendering. + * Read at call time (not module load) so tests and config changes apply + * without a restart. An invalid name warns once and behaves as unset. + */ +export function ambientTz(env: NodeJS.ProcessEnv = process.env): string | undefined { + const tz = env[AMBIENT_TZ_ENV]?.trim(); + if (!tz) return undefined; + try { + new Intl.DateTimeFormat('en-GB', { timeZone: tz }); + return tz; + } catch { + if (warnedInvalidTz !== tz) { + warnedInvalidTz = tz; + console.warn(`[ambient-time] invalid ${AMBIENT_TZ_ENV} '${tz}', falling back to UTC`); + } + return undefined; + } +} + +const pad = (n: number): string => String(n).padStart(2, '0'); + +function utcParts(now: Date): { date: string; time: string; weekday: string } { + const weekday = ['Sun', 'Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat'][now.getUTCDay()] as string; + return { + weekday, + date: `${now.getUTCFullYear()}-${pad(now.getUTCMonth() + 1)}-${pad(now.getUTCDate())}`, + time: `${pad(now.getUTCHours())}:${pad(now.getUTCMinutes())}`, + }; +} + +/** + * One line of "now", ~15 tokens: + * tz set: `Thu 2026-10-02 10:30 +01:00 Europe/London (09:30 UTC)` + * tz unset: `Thu 2026-10-02 09:30 UTC` + */ +export function formatAmbientNow(now: Date = new Date(), tz: string | undefined = ambientTz()): string { + const utc = utcParts(now); + if (!tz) return `${utc.weekday} ${utc.date} ${utc.time} UTC`; + + const parts = new Intl.DateTimeFormat('en-GB', { + timeZone: tz, + weekday: 'short', + year: 'numeric', + month: '2-digit', + day: '2-digit', + hour: '2-digit', + minute: '2-digit', + hour12: false, + timeZoneName: 'longOffset', + }).formatToParts(now); + const get = (type: string) => parts.find((p) => p.type === type)?.value ?? ''; + // "GMT+01:00" → "+01:00"; plain "GMT" (winter London) → "+00:00". + const rawOffset = get('timeZoneName'); + const offset = rawOffset === 'GMT' ? '+00:00' : rawOffset.replace(/^(GMT|UTC)/, ''); + // Intl can render "24:00" for midnight with hour12:false; normalize. + const hour = get('hour') === '24' ? '00' : get('hour'); + return `${get('weekday')} ${get('year')}-${get('month')}-${get('day')} ${hour}:${get('minute')} ${offset} ${tz} (${utc.time} UTC)`; +} + +/** The ambient context line stamped into instructions and tool results. */ +export function ambientContextLine(now: Date = new Date()): string { + return `[context] now: ${formatAmbientNow(now)}`; +} + +/** Multi-line body for the get_current_time virtual tool. */ +export function formatCurrentTimeResult(now: Date = new Date()): string { + const tz = ambientTz(); + const lines = [ + `UTC: ${now.toISOString()}`, + ...(tz ? [`${tz}: ${formatAmbientNow(now, tz)}`] : []), + `Unix epoch seconds: ${Math.floor(now.getTime() / 1000)}`, + ]; + return lines.join('\n'); +} diff --git a/src/shared/src/utils/index.ts b/src/shared/src/utils/index.ts index 39156df..0aa3dd9 100644 --- a/src/shared/src/utils/index.ts +++ b/src/shared/src/utils/index.ts @@ -1,2 +1,2 @@ // Shared utility functions -// Will be expanded as tasks are implemented +export * from './ambient-time.js'; diff --git a/templates/calcmcp.yaml b/templates/calcmcp.yaml new file mode 100644 index 0000000..db4a648 --- /dev/null +++ b/templates/calcmcp.yaml @@ -0,0 +1,20 @@ +name: calcmcp +version: "1.1.0" +description: Scientific calculator (SymPy/NumPy/SciPy) — symbolic algebra, calculus, equation & ODE solving, linear algebra, statistics, FFT +dockerImage: "mysources.co.uk/michal/calcmcp:1.1.0" +transport: STDIO +repositoryUrl: https://mysources.co.uk/michal/CalcMCP +# Patched fork of github.com/thinkitpossible/CalcMCP: proper MCP inputSchema, +# English tool docs, eval() removed, complex eigenvalues serialize, symbolic +# dsolve added. Pure compute — needs no env, no secrets, no egress. +# Seeded by default into the ungated-global project (see docs/ungated-global.md); +# this template exists so a deleted default can be re-created by hand: +# mcpctl create server calcmcp --from-template calcmcp +healthCheck: + tool: symbolic_tool + arguments: + operation: simplify + expression: "1+1" + intervalSeconds: 300 + timeoutSeconds: 10 +env: [] diff --git a/templates/dav.yaml b/templates/dav.yaml new file mode 100644 index 0000000..6eaacfe --- /dev/null +++ b/templates/dav.yaml @@ -0,0 +1,26 @@ +name: dav +version: "1.0.0" +description: CalDAV/CardDAV/VTODO (Stalwart or any RFC 4791/6352 server) — calendars, events, tasks, contacts +# NOT packageName: dav-mcp declares tsdav/tsdav-utils as GitHub-URL deps, so +# `npx -y dav-mcp` dies with `spawn git ENOENT` on the git-less node-runner, +# and the upstream ghcr image starts the HTTP server as PID 1. This image +# bakes the install and execs the stdio entrypoint (NODE_OPTIONS baked in). +dockerImage: "mysources.co.uk/michal/dav-mcp:4.0.1-stdio1" +transport: STDIO +repositoryUrl: https://mysources.co.uk/michal/dav-mcp-image +healthCheck: + # A real CalDAV PROPFIND: a pass proves server URL and credentials. + tool: list_calendars + arguments: {} + intervalSeconds: 60 + timeoutSeconds: 10 +env: + - name: CALDAV_SERVER_URL + description: DAV base URL (e.g. http://stalwart.stalwart.svc.cluster.local:8080) + required: true + - name: CALDAV_USERNAME + description: DAV username + required: true + - name: CALDAV_PASSWORD + description: DAV password (use an app password) + required: true diff --git a/templates/duckduckgo.yaml b/templates/duckduckgo.yaml index c21f44c..e45e69c 100644 --- a/templates/duckduckgo.yaml +++ b/templates/duckduckgo.yaml @@ -5,11 +5,17 @@ packageName: "duckduckgo-mcp-server" runtime: python transport: STDIO repositoryUrl: https://github.com/nickclyde/duckduckgo-mcp-server +# Readiness fetches a tiny static page instead of searching. A `search` probe is +# a DuckDuckGo scrape every interval (1,440/day at 60s) from the same IP agents +# search from, which is how that IP ends up answering CAPTCHA. This proves the +# process and its outbound fetch path work, not that DuckDuckGo is answering. healthCheck: - tool: search + tool: fetch_content arguments: - query: "mcp" - max_results: 1 + url: "https://example.com" + max_length: 500 + intervalSeconds: 300 + timeoutSeconds: 20 env: - name: DDG_SAFE_SEARCH description: Result filtering — STRICT, MODERATE or OFF diff --git a/templates/firecrawl.yaml b/templates/firecrawl.yaml new file mode 100644 index 0000000..dff30d6 --- /dev/null +++ b/templates/firecrawl.yaml @@ -0,0 +1,28 @@ +name: firecrawl +version: "1.0.0" +description: Firecrawl MCP server for reading web pages as main-content markdown, JS-rendered pages included +packageName: "firecrawl-mcp" +runtime: node +transport: STDIO +repositoryUrl: https://github.com/firecrawl/firecrawl-mcp-server +# Readiness scrapes a tiny static page: a pass proves the Firecrawl API answers +# and can fetch the web, without a search. Firecrawl's own search needs a +# SearXNG endpoint on self-hosted instances; use the searxng template for +# search. 300s because each probe is a real outbound fetch. +healthCheck: + tool: firecrawl_scrape + arguments: + url: "https://example.com" + formats: ["markdown"] + onlyMainContent: true + intervalSeconds: 300 + timeoutSeconds: 60 +env: + - name: FIRECRAWL_API_URL + description: >- + Self-hosted Firecrawl API base URL (e.g. http://firecrawl-api:3002). Leave + unset only when using Firecrawl's cloud with FIRECRAWL_API_KEY. + required: false + - name: FIRECRAWL_API_KEY + description: API key — required for Firecrawl's cloud, optional for a self-hosted instance + required: false diff --git a/templates/jmap.yaml b/templates/jmap.yaml new file mode 100644 index 0000000..8b2bab1 --- /dev/null +++ b/templates/jmap.yaml @@ -0,0 +1,45 @@ +name: jmap +version: "1.0.0" +description: JMAP email (Stalwart) — search/read/organize + drafts; send and delete gated off by env +dockerImage: "mysources.co.uk/michal/jmap-mcp:0.6.5-drafts3" +transport: STDIO +repositoryUrl: https://mysources.co.uk/michal/jmap-mcp +# No command: the image's entrypoint IS the MCP server (deno run), mcpd +# attaches to PID 1 stdin/stdout. Fork of wyattjoh/jmap-mcp adding +# create_draft/create_draft_reply, JMAP_DISABLE_* gating, and Basic auth +# (jmap-jam upstream only speaks Bearer). +healthCheck: + # A real Mailbox/get: a pass proves session URL, auth, and account + # resolution, not just process liveness. + tool: get_mailboxes + arguments: {} + intervalSeconds: 60 + timeoutSeconds: 10 +env: + - name: JMAP_SESSION_URL + description: JMAP session URL (e.g. http://stalwart.stalwart.svc.cluster.local:8080/.well-known/jmap) + required: true + - name: JMAP_BEARER_TOKEN + description: Bearer token. Provide either this OR JMAP_USERNAME + JMAP_PASSWORD. + required: false + - name: JMAP_USERNAME + description: Username for HTTP Basic auth (e.g. with a Stalwart app password) + required: false + - name: JMAP_PASSWORD + description: Password/app password for HTTP Basic auth + required: false + - name: JMAP_ACCOUNT_ID + description: Account ID (auto-detected when empty) + required: false + - name: JMAP_DISABLE_SUBMISSION + description: "true = do not register send_email/reply_to_email (drafts only)" + required: false + defaultValue: "true" + - name: JMAP_DISABLE_DELETE + description: "true = do not register delete_emails (move to Trash still works)" + required: false + defaultValue: "true" + - name: JMAP_REBASE_SESSION_URLS + description: "true = rewrite the session's apiUrl/downloadUrl/uploadUrl/eventSourceUrl onto the session URL's origin. Required over a ClusterIP: Stalwart advertises its publicUrl and the hairpinned call 403s at Envoy." + required: false + defaultValue: "true" diff --git a/templates/karakeep.yaml b/templates/karakeep.yaml new file mode 100644 index 0000000..1026000 --- /dev/null +++ b/templates/karakeep.yaml @@ -0,0 +1,24 @@ +name: karakeep +version: "1.0.0" +description: Karakeep bookmarks — search, read, create, lists, tags, highlights +packageName: "@karakeep/mcp" +runtime: node +transport: STDIO +repositoryUrl: https://github.com/karakeep-app/karakeep +healthCheck: + # A real API call (lists all lists): a pass proves address and API key. + tool: get-lists + arguments: {} + intervalSeconds: 60 + timeoutSeconds: 10 +env: + - name: KARAKEEP_API_ADDR + description: Karakeep base URL (e.g. http://karakeep-web.karakeep.svc.cluster.local:3000) + required: true + - name: KARAKEEP_API_KEY + description: API key created in the Karakeep UI (Settings → API Keys) + required: true + - name: NODE_OPTIONS + description: V8 heap cap; Node sizes its heap off host RAM, not the cgroup limit + required: false + defaultValue: "--max-old-space-size=384" diff --git a/templates/searxng.yaml b/templates/searxng.yaml index cb16d87..5ecef93 100644 --- a/templates/searxng.yaml +++ b/templates/searxng.yaml @@ -4,10 +4,12 @@ description: SearXNG MCP server for private web search and URL-to-markdown readi packageName: "mcp-searxng" transport: STDIO repositoryUrl: https://github.com/ihor-sokoliuk/mcp-searxng +# Readiness reads SearXNG's /config, which proves the instance answers without +# fanning out to its engines. A searxng_web_search probe is a real search every +# interval (1,440/day at 60s): it burns API-key engines (braveapi, kagi) and +# feeds the IP blocks scraping engines put on the instance. healthCheck: - tool: searxng_web_search - arguments: - query: "mcp" + tool: searxng_instance_info env: - name: SEARXNG_URL description: >- diff --git a/templates/ttrss.yaml b/templates/ttrss.yaml new file mode 100644 index 0000000..9e7a684 --- /dev/null +++ b/templates/ttrss.yaml @@ -0,0 +1,26 @@ +name: ttrss +version: "1.0.0" +description: Tiny Tiny RSS — feeds, headlines, article content, read/star state, subscribe +dockerImage: "mysources.co.uk/michal/ttrss-mcp:1.0.1" +transport: STDIO +repositoryUrl: https://mysources.co.uk/michal/ttrss-mcp +# No command: the image's entrypoint IS the MCP server. Custom server; the +# public tt-rss MCPs only take a pre-made session id and die when it expires. +# This one logs in with user/password and re-logs-in on NOT_LOGGED_IN. +healthCheck: + # A real getCategories call: a pass proves URL, login, and that the + # user's "Enable API access" preference is on. + tool: list_categories + arguments: {} + intervalSeconds: 60 + timeoutSeconds: 10 +env: + - name: TTRSS_URL + description: Full API endpoint (e.g. http://ttrss.ttrss.svc.cluster.local/api/) + required: true + - name: TTRSS_USER + description: tt-rss user with "Enable API access" set in preferences + required: true + - name: TTRSS_PASSWORD + description: That user's password + required: true