Commit Graph

18 Commits

Author SHA1 Message Date
Michal
b0233918ff feat(servers): persistent volumes + self-hosted web search and docs templates
Instances are immutable and get recreated on any server edit, so anything an
MCP server wrote to its container filesystem was lost at exactly that point.
That ruled out every stateful MCP server, docs-mcp among them: its index is a
SQLite file (better-sqlite3 + sqlite-vec) and it has no external-database mode,
so no amount of Postgres helps.

A server or template can now declare volumes. The backing store is keyed on the
server, not the instance — `mcpctl-<server>-<name>` — which is the whole point:
an instance-scoped claim would be destroyed precisely when the data needs to
survive. On Kubernetes that is a PVC ensured in the servers namespace before the
pod is created and never deleted with it; on Docker, a named volume (named, not
anonymous, so `removeContainer`'s `v: true` leaves it alone).

Claims are ReadWriteOnce, so volumes and replicas > 1 are mutually exclusive;
validation rejects that pair instead of leaving the extra replicas
unschedulable. storageClassName is omitted rather than sent empty when no class
is configured — to Kubernetes those mean different things.

Also fixes a pre-existing bug in the same path: seedTemplates dropped `runtime`,
so every PyPI-backed template seeded from YAML silently defaulted to node and
would run `npx` against a package that only exists on PyPI. `unifi-network`
declares `runtime: python` and had been seeding with runtime unset.

Templates added, all self-hosted and none needing an API key:
  - duckduckgo — no backing service at all
  - searxng    — needs a SearXNG engine (compose profile in stack/)
  - docs-mcp   — open-source Context7/Ref alternative, uses the new volume

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017BMXdb2qZbPSh8Q7XpTyjB
2026-08-09 18:17:32 +01:00
Michal
f90e23d000 Merge remote-tracking branch 'origin/main' into feat/pi-extension
# Conflicts:
#	README.md
#	src/cli/src/commands/config.ts
#	src/cli/src/commands/skills.ts
2026-08-08 16:35:51 +01:00
Michal
28f1a411fd feat(pi): add native pi integration — extension, config pi, skills --agent pi
- src/pi-ext/: self-contained pi extension (mcpctl-pi.ts + vendored
  mcp-http client) that talks JSON-RPC directly to mcplocal and registers
  project MCP tools as native pi tools. No MCP client, no ~/.claude.
- Persistent per-project session so gated projects ungate on begin_session.
- /mcpctl command: status, switch project (GUI), refresh tools, sync skills.
- mcpctl config pi: installs extension, wires pi settings, persists active
  project, syncs skills into ~/.pi/agent/skills.
- skills sync: add --agent pi (target install root).
- docs + tests.
2026-08-08 16:23:02 +01:00
Michal
170dc06496 fix(cli): close third review — token collision, migration, ownership
Some checks failed
CI/CD / typecheck (pull_request) Successful in 1m5s
CI/CD / lint (pull_request) Successful in 2m13s
CI/CD / test (pull_request) Successful in 1m20s
CI/CD / build (pull_request) Successful in 2m9s
CI/CD / smoke (pull_request) Failing after 2m44s
CI/CD / publish (pull_request) Has been skipped
Round 2 fixed the first review but introduced regressions of its own, all
of which only bite against state written by the previously installed build.

`config prime-agent`:
- Mint each credential under a unique `prime-agent-<stamp>` name again.
  `McpToken` is unique on (name, projectId) and revoke is a soft delete, so
  round 2's fixed `prime-agent` name could only ever be minted once per
  project — and the revoke-first ordering destroyed the working credential
  before discovering the mint would fail.
- Provision the credential BEFORE touching settings.json. Registering the
  new project unmounts the previously active one, so a failed mint must not
  be able to leave prime-agent with no working project at all. The command
  now aborts with settings.json untouched.
- Retire only the token this auth.json actually held, once its replacement
  is stored. Sweeping every `prime-agent*` token for the project would
  revoke the credential another install (or a custom --output run) is
  using; anything else that looks orphaned is reported, not deleted.
- Validate a pre-existing credential instead of trusting its presence: a
  revoked or expired token used to short-circuit provisioning and leave
  prime-agent broken while the command reported success. Matched by
  tokenPrefix against the project's active tokens, so the secret is never
  sent. Fails open when the API can't be consulted.
- Actually write auth.json 0600. `writeFile`'s mode is ignored for an
  existing file and prime-agent creates auth.json itself at 0644, so chmod
  after writing.
- Recognise the untagged mcpServers entries older CLIs wrote (canonical
  proxy URL + an `mcp:<name>` mcpctl PAT in auth.json) so a switch unmounts
  them instead of leaving two gateways live. Hand-configured servers have
  no such credential and are still preserved. Same rule in the `/mcpctl`
  switcher's active-project lookup.
- Add `--skip-marker`, and pass it from the `/mcpctl` switcher: the
  extension runs from whatever directory prime-agent was started in, and
  was silently re-scoping that repo's `.mcpctl-project`.

`skills sync --agent prime-agent`:
- Record ownership from the skill's own scope, not the syncing project's.
  Globals were being pinned to whichever project happened to sync them,
  after which every other project refused to update them forever.
- Never adopt legacy, ownership-less state into the current scope. Round 2
  did, which deleted the other project's skills on the first sync after
  upgrading. Such entries are attributed to the project that last wrote the
  state file, and left alone when that isn't the project syncing now.
- Close the overwrite-guard bypass: a sync with no project, or a global
  landing on a project-owned name, could still clobber and re-own a
  tracked skill.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017BMXdb2qZbPSh8Q7XpTyjB
2026-08-08 12:01:52 +01:00
Michal
eb1642ab1a fix(cli): harden config prime-agent sync + install /mcpctl switcher extension
Some checks failed
CI/CD / lint (pull_request) Successful in 1m4s
CI/CD / typecheck (pull_request) Successful in 1m6s
CI/CD / test (pull_request) Successful in 3m9s
CI/CD / build (pull_request) Successful in 2m16s
CI/CD / smoke (pull_request) Failing after 3m27s
CI/CD / publish (pull_request) Has been skipped
Addresses a review of the `config prime-agent` feature and adds the in-app
project switcher.

Safety/correctness fixes (prime-agent's shared, hand-editable ~/.prime/agent
tree must never suffer silent data loss):
- config/prime-agent.ts: loadPrimeAgentSettings now fails loudly on corrupt
  JSON instead of swallowing it and rewriting the file (which destroyed every
  non-mcpServers setting). A project's mcpServers entry is merged (keeping
  user-added fields) rather than replaced wholesale. Added writePrimeAgentAuth
  / hasPrimeAgentAuth helpers for auth provisioning.
- skills sync: unified the near-verbatim prime-agent copy into runSkillsSync
  via a `target: 'claude' | 'prime-agent'` option (prime-agent-skills.ts is now
  a thin wrapper). Under the prime-agent target it: preserves untracked
  pre-existing skill dirs on first sync (no more rm -rf of hand-authored `sre`),
  records per-project ownership so configuring a second project never deletes
  the first project's skills, skips Claude-only hooks/postInstall, and keeps
  the mcpServers auto-attach step.
- config.ts: `config prime-agent` now (a) provisions the bearer credential in
  auth.json (--token, existing entry, or auto-mint via POST /api/v1/mcptokens),
  (b) writes the .mcpctl-project marker only when none exists up-tree and never
  from $HOME, and (c) propagates the skills sync exit code so auth failures are
  reported instead of swallowing them.
- skills.ts: `--agent` is validated; an unknown value errors instead of
  silently running the Claude sync.

New feature: `config prime-agent` installs a `/mcpctl` project-switcher
extension into ~/.prime/agent/extensions/ (skip with --skip-extension). It lists
mcpctl projects via `mcpctl get projects -o json`, lets you pick one from the
prime-agent TUI, applies the switch through the CLI, and reloads the session.

Regenerated shell completions. Tests: 538 pass (new coverage for settings
corruption, entry merge, auth provisioning, extension install/skip, marker
$HOME handling, untracked/cross-project skill preservation, --agent validation).
2026-08-08 10:22:46 +01:00
Michal
582f6f185b feat(cli): add mcpctl config prime-agent — proxy MCP + skills sync for prime-agent
Some checks failed
CI/CD / lint (pull_request) Successful in 1m6s
CI/CD / typecheck (pull_request) Successful in 2m10s
CI/CD / test (pull_request) Successful in 1m19s
CI/CD / smoke (pull_request) Failing after 1m53s
CI/CD / build (pull_request) Successful in 4m14s
CI/CD / publish (pull_request) Has been skipped
Mirror `mcpctl config claude` for prime-agent (which talks to the same mcpctl
proxy MCP gateway over HTTP instead of stdio):

`mcpctl config prime-agent --project X`:
- registers the proxy MCP gateway in ~/.prime/agent/settings.json as
  mcpServers.X = { type: "http", url: <gateway>/projects/X/mcp }, merging with
  any existing servers (e.g. the bundled `sre` project) and preserving all other
  settings
- writes a .mcpctl-project marker so later syncs resolve the project
- syncs the project's skills into ~/.prime/agent/skills/<name>/ as markdown
  skills (prime-agent auto-discovers them at session start)

New `mcpctl skills sync --agent prime-agent` target re-syncs the tree later.

- src/cli/src/config/prime-agent.ts: settings.json read/merge/write helpers
- src/cli/src/utils/prime-agent-skills.ts: prime-agent sync (reuses
  installSkillAtomic + skills-state; skips Claude-only hooks/postInstall)
- completes config.ts/skills.ts wiring; regenerated shell completions
- tests: commands/prime-agent.test.ts + utils/prime-agent-skills.test.ts
2026-08-08 09:34:15 +01:00
Michal
866f6abc88 feat: virtual-LLM smoke test + docs (v1 Stage 6)
Some checks failed
CI/CD / typecheck (pull_request) Successful in 53s
CI/CD / test (pull_request) Successful in 1m8s
CI/CD / lint (pull_request) Successful in 2m6s
CI/CD / smoke (pull_request) Failing after 1m39s
CI/CD / build (pull_request) Successful in 2m11s
CI/CD / publish (pull_request) Has been skipped
Final stage of v1.

Smoke (mcplocal/tests/smoke/virtual-llm.smoke.test.ts):
- Spins an in-process LlmProvider that returns canned content.
- Runs the registrar against the live mcpd in fulldeploy.
- Asserts: row appears with kind=virtual / status=active, infer
  through /api/v1/llms/<name>/infer comes back through the SSE
  relay with the provider's content + finish_reason, and a 503
  appears immediately after registrar.stop() (publisher offline).
- Times out / cleanup paths idempotent so re-runs against the same
  cluster don't litter rows. The 90-s heartbeat-stale flip and 4-h
  GC are unit-tested — too slow for smoke.

Docs:
- New docs/virtual-llms.md: when to use this vs creating a regular
  Llm row, how to opt-in via publish: true, the lifecycle table,
  the inference-relay sequence, the v1 streaming caveat, the v2-v5
  roadmap, and the full /api/v1/llms/_provider-* surface.
- agents.md cross-links virtual-llms.md alongside personalities/chat.
- README's Agents section gains a "Virtual LLMs" subsection.

Workspace suite: 2043/2043 (smoke files run separately). v1 closes.

Stage roadmap (each its own future PR):
  v2 wake-on-demand · v3 virtual agents · v4 LB pool · v5 task queue

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 14:28:43 +01:00
Michal
4cbf58d212 feat(mcpd+deploy): serve web UI at /ui + smoke tests + docs (Stage 6)
Some checks failed
CI/CD / lint (pull_request) Successful in 54s
CI/CD / test (pull_request) Failing after 1m8s
CI/CD / typecheck (pull_request) Successful in 2m35s
CI/CD / smoke (pull_request) Has been skipped
CI/CD / build (pull_request) Has been skipped
CI/CD / publish (pull_request) Has been skipped
The closing stage. mcpd now hosts the Stage 5 SPA, the Docker image
bundles the build artifact, a smoke test exercises the personality
HTTP surface end-to-end, and the user-facing docs spell out the
mental model.

mcpd:
- Add @fastify/static dep.
- New routes/web-ui.ts: registers /ui/* against a static bundle. Looks
  for the bundle at $MCPD_WEB_ROOT, then /usr/share/mcpd/web (the
  Docker image path), then a dev-tree fallback. Logs and skips
  cleanly if missing — API-only deploys keep working.
- SPA fallback: any /ui/<path> that doesn't match a file falls through
  to index.html so direct hits to react-router URLs work.
- /ui/* falls through to `kind: skip` in mapUrlToPermission, so the
  static assets are served unauthenticated. Each API call from the
  SPA still carries the bearer token.

Deploy:
- Dockerfile.mcpd builds the @mcpctl/web bundle in the same builder
  stage and copies dist/ to /usr/share/mcpd/web in the runtime image.

Smoke (personality.smoke.test.ts):
- Live mcpd flow: create secret/llm/agent/personality, attach an
  agent-direct prompt, verify the binding listing, reject double-
  attach (409) + foreign-agent prompt (400), set defaultPersonality
  by name, detach + delete cleanup.

Docs:
- New docs/personalities.md: VLAN-on-ethernet model, system-block
  ordering table, three prompt scopes, CLI walkthrough, web UI
  walkthrough, full API surface, RBAC notes.
- agents.md and chat.md cross-link.
- README's Agents section gains a Personalities subsection.

Test count after Stage 6:
  mcpd:   801/801      cli:  430/430
  web:    7/7          db:   58/62 (4 pre-existing)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 19:48:43 +01:00
Michal
8b56f09f25 feat(agents): smoke tests + README + docs (Stage 6, final)
Closes the agents feature.

Smoke tests (run via `pnpm test:smoke` against a live mcpd at
$MCPD_URL, default https://mcpctl.ad.itaz.eu):

* tests/smoke/agent.smoke.test.ts — full CRUD round-trip:
  create secret + Llm + agent with sampling defaults; `get agents`
  surfaces it; `get agent foo -o yaml | apply -f` round-trips
  identically; create + list a thread via the HTTP API; agent delete
  leaves Llm + secret intact (Restrict + SetNull as designed). Self-
  skips with a warning when /healthz is unreachable.

* tests/smoke/agent-chat.smoke.test.ts — gated on
  MCPCTL_SMOKE_LLM_URL + MCPCTL_SMOKE_LLM_KEY. Provisions secret +
  Llm + agent against a real upstream, runs `mcpctl chat -m … --no-
  stream` (asserts a reply lands), then runs the streaming default
  (asserts text on stdout + `(thread: …)` on stderr). The fast path
  for verifying the in-cluster qwen3-thinking deployment:

      MCPCTL_SMOKE_LLM_URL=http://litellm.nvidia-nim.svc.cluster.local:4000/v1 \
      MCPCTL_SMOKE_LLM_MODEL=qwen3-thinking \
      MCPCTL_SMOKE_LLM_KEY=$(pulumi config get --stack homelab \
        secrets:litellmMcpctlGatewayToken) \
        pnpm test:smoke

Docs:

* README.md — new "Agents" section under Resources with the
  qwen3-thinking quickstart and links to docs/agents.md and
  docs/chat.md. Adds llm + agent rows to the resources table.

* docs/agents.md (new) — full reference: data model, chat-parameter
  table, HTTP API, RBAC mapping, tool-use loop semantics, yaml
  round-trip shorthand, the kubernetes-deployment wiring recipe,
  and a troubleshooting section (namespace collision, llm-in-use,
  pending-row recovery, Anthropic-tool limitation).

* docs/chat.md (new) — user-facing `mcpctl chat` walkthrough:
  modes, per-call flags, slash-commands, threads, and a
  troubleshooting section.

* CLAUDE.md — adds a "Resource types" cheatsheet with one-line
  pointers to each, including the new `agent` row that links to
  the docs.

All suites still green: mcpd 759/759, mcplocal 715/715, cli 430/430.
Smoke tests typecheck and self-skip when no live mcpd is reachable.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 17:08:37 +01:00
Michal
c0f63e20e9 docs: fix RPM install to use public URL with manual repo file
Some checks failed
CI / lint (push) Waiting to run
CI / build (push) Blocked by required conditions
CI / package (push) Blocked by required conditions
CI / typecheck (push) Failing after 23s
CI / test (push) Failing after 23s
Gitea's auto-generated .repo file contains internal IPs. Use a manual
repo file with the public mysources.co.uk baseurl instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 17:36:26 +00:00
Michal
0ffbcfad79 docs: fix install URLs in README to use real public registry
Some checks are pending
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / test (push) Waiting to run
CI / build (push) Blocked by required conditions
CI / package (push) Blocked by required conditions
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 16:10:38 +00:00
Michal
25903a6d20 docs: clarify plugin inheritance in README
Some checks are pending
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / test (push) Waiting to run
CI / build (push) Blocked by required conditions
CI / package (push) Blocked by required conditions
Rewrite the Plugin System section to make the extends/inheritance
mechanism clear — show that default extends gate + content-pipeline,
explain hook inheritance and conflict resolution rules.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 15:05:12 +00:00
Michal
13e256aa0c docs: fix README quick start to use templates and git backup
Some checks are pending
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / test (push) Waiting to run
CI / build (push) Blocked by required conditions
CI / package (push) Blocked by required conditions
- Section 4 now uses --from-template instead of manual --docker-image
- Declarative YAML example uses fromTemplate + envFrom secretRef
- Backup section updated to git-based commands (was old JSON bundle)
- Consistent server naming (my-grafana from template, not bare grafana)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 14:58:12 +00:00
Michal
9fc31e5945 docs: ProxyModel authoring guide in README, mark cache tasks done
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 23:37:07 +00:00
Michal
d9d0a7a374 docs: update README for plugin system, add proxyModel tests
- Rewrite README Content Pipeline section as Plugin System section
  documenting built-in plugins (default, gate, content-pipeline),
  plugin hooks, and the relationship between gating and proxyModel
- Update all README examples to use --proxy-model instead of --gated
- Add unit tests: proxyModel normalization in JSON/YAML output (4 tests),
  Plugin Config section in describe output (2 tests)
- Add smoke tests: yaml/json output shows resolved proxyModel without
  gated field, round-trip compatibility (4 tests)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 01:24:47 +00:00
Michal
03827f11e4 feat: eager vLLM warmup and smart page titles in paginate stage
- Add warmup() to LlmProvider interface for eager subprocess startup
- ManagedVllmProvider.warmup() starts vLLM in background on project load
- ProviderRegistry.warmupAll() triggers all managed providers
- NamedProvider proxies warmup() to inner provider
- paginate stage generates LLM-powered descriptive page titles when
  available, cached by content hash, falls back to generic "Page N"
- project-mcp-endpoint calls warmupAll() on router creation so vLLM
  is loading while the session initializes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:07:39 +00:00
Michal
0427d7dc1a fix: correct architecture diagram in README
Some checks failed
CI / lint (push) Has been cancelled
CI / typecheck (push) Has been cancelled
CI / test (push) Has been cancelled
CI / build (push) Has been cancelled
CI / package (push) Has been cancelled
MCP server containers are managed by and proxied through mcpd,
not directly accessible. Updated diagram to show containers
nested inside mcpd boundary with explanation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 17:12:29 +00:00
Michal
69867bd47a feat: mcpctl v0.0.1 — first public release
Some checks are pending
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / test (push) Waiting to run
CI / build (push) Blocked by required conditions
CI / package (push) Blocked by required conditions
Comprehensive MCP server management with kubectl-style CLI.

Key features in this release:
- Declarative YAML apply/get round-trip with project cloning support
- Gated sessions with prompt intelligence for Claude
- Interactive MCP console with traffic inspector
- Persistent STDIO connections for containerized servers
- RBAC with name-scoped bindings
- Shell completions (fish + bash) auto-generated
- Rate-limit retry with exponential backoff in apply
- Project-scoped prompt management
- Credential scrubbing from git history

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 17:05:05 +00:00