Commit Graph

39 Commits

Author SHA1 Message Date
Michal
25da5a3a22 feat(errors): mcpd error-log ring buffer + 'mcpctl errors'
Some checks failed
CI/CD / lint (pull_request) Successful in 1m1s
CI/CD / typecheck (pull_request) Successful in 1m2s
CI/CD / test (pull_request) Successful in 1m21s
CI/CD / smoke (pull_request) Failing after 1m50s
CI/CD / build (pull_request) Successful in 5m12s
CI/CD / publish (pull_request) Has been skipped
Operators can now see recent mcpd error/fatal logs without kubectl:
- mcpd tees level>=error pino records into an in-memory ring buffer
  (src/mcpd/src/services/error-log-buffer.ts; wired via pino.multistream in
  server.ts so stdout logging is unchanged). Captures structured errors incl.
  fatal kinds like BACKEND_TOKEN_DEAD.
- GET /api/v1/logs/errors?limit=N (RBAC: 'logs' operation).
- CLI: 'mcpctl errors [-n N]' renders TIME/LEVEL/DETAIL, most-recent-first.

Buffer unit tests (6); full suite 2223 passing. Needs a deploy to go live.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 23:25:55 +01:00
Michal
2cceeb7093 feat(passwd): mcpctl passwd + RBAC-gated password change
Restores the lost `mcpctl passwd` command and builds the backend it needs.

Backend (mcpd):
- POST /api/v1/users/me/password — self-service change, requires current
  password. Gated by a new `set-own-password` operation.
- PUT /api/v1/users/:id/password — admin reset of another user, gated by
  edit:users (admins have edit:*). Added users name-resolver for CUID→email.
- UserService.setPassword/verifyPassword; UserRepository.update accepts
  passwordHash + findByIdWithHash.

RBAC, no exceptions: self password change is a default, admin-revocable
permission. Every new user gets a `self-<id>` RbacDefinition granting
`set-own-password`, seeded on create + bootstrap, gated by the
`allowSelfPasswordChange` system setting (stored in the mcpctl-system-settings
secret, default ON; admins disable globally or revoke per-user).

CLI: src/cli/src/commands/passwd.ts (self vs admin paths) + completions.

Tests: users-password route tests (8), auth-bootstrap grant assertion,
passwd live smoke test. Full suite 2214 passing; zero new lint errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 21:55:56 +01:00
Michal
58e8e956ce feat(cli+mcpd): mcpctl skills sync + config claude extension
Phase 5 of the Skills + Revisions + Proposals work. Skills are now
materialised onto disk under ~/.claude/skills/<name>/, with
hash-pinned diff against mcpd, atomic per-skill install, and
preservation of locally-modified files. `mcpctl config claude --project X`
now wires the full pickup chain: writes .mcpctl-project marker, runs
the initial sync, installs the SessionStart hook so subsequent Claude
invocations stay in sync transparently.

## Sync algorithm

1. Resolve project: `--project` flag overrides; else walk up from cwd
   looking for `.mcpctl-project`; else fall back to globals-only.
2. GET /api/v1/projects/:name/skills/visible (or
   /api/v1/skills?scope=global without a project). Server returns
   id + name + semver + scope + contentHash + metadata — no body, no
   files. The contentHash is sha256 of the canonicalised body, computed
   server-side; any reordering of keys produces the same hash, so it's
   a stable diff key.
3. Load ~/.mcpctl/skills-state.json (lives outside ~/.claude/skills/
   on purpose — Claude Code reads that tree and we don't want to
   pollute it with our bookkeeping).
4. Diff:
     - server skill not in state → INSTALL
     - server skill, state contentHash matches → SKIP (cheap path)
     - server skill, state contentHash differs → UPDATE (fetch full body)
     - state skill not in server → orphan, REMOVE (preserve if locally
       modified, unless --force)
5. Atomic per-skill install: write to <targetDir>.mcpctl-staging-<pid>/,
   rename existing tree to .mcpctl-trash-<pid>, swap staging in,
   rmtree the trash. A concurrent reader (Claude Code starting up)
   never sees a partial tree.
6. State file updated with new versions, per-file SHA-256, install
   path. saveState is atomic (temp + rename).

## Failure semantics

- `--quiet` mode (used by SessionStart hook): exit 0 on network /
  timeout / mcpd error. Fail-open is non-negotiable here — we never
  want a hung mcpd to block Claude Code starting up.
- Auth failure: exit 1, clear "run mcpctl login" message.
- Disk error during state save: exit 2.
- Per-skill errors are collected in the result and reported as a
  count; one bad skill doesn't stop the others.

Network fetches run with concurrency 5. The server-side
`/visible` endpoint is metadata-only so the cheap path (everything
unchanged) needs exactly one HTTP roundtrip total.

## Files added

### CLI utilities (src/cli/src/utils/)
- skills-state.ts — load/save state, per-file sha256, edit detection.
- project-marker.ts — walk-up to find `.mcpctl-project`, bounded by
  user home so we never search above $HOME.
- sessionhook.ts — install/remove a SessionStart hook entry tagged
  with `_mcpctl_managed: true`. Idempotent. Defensive against
  missing/empty/JSONC settings.json.
- skills-disk.ts — atomic install via staging-dir rename swap,
  symmetric atomic delete via trash-dir rename. Path-escape attempts
  in files{} are rejected.

### CLI command (src/cli/src/commands/)
- skills.ts — `mcpctl skills sync` Commander wrapper + the
  `runSkillsSync(opts, deps)` library function (also called from
  `mcpctl config claude --project`). Supports `--dry-run`, `--force`,
  `--quiet`, `--keep-orphans`. `--skip-postinstall` is reserved
  (postInstall execution lands in a follow-up PR, not this one).

### Wiring
- index.ts: registers `mcpctl skills` after `mcpctl review`.
- config.ts: `mcpctl config claude --project X` now writes the
  `.mcpctl-project` marker, runs `runSkillsSync` in-process, and calls
  `installManagedSessionHook('mcpctl skills sync --quiet')`. New flag
  `--skip-skills` opts out (used by tests; useful for CI).

## Server-side change

- src/mcpd/src/services/skill.service.ts: getVisibleSkills now
  computes contentHash on the fly from the canonical body shape the
  client will reconstruct. Cheap (sha256 of ~few KB per skill); no
  schema migration needed since hash is derived not stored.

## Tests

Four new utility test files (31 tests) under src/cli/tests/utils/:
- sessionhook.test.ts — creation, idempotency, command updates,
  preservation of user hooks, removal, empty/JSONC tolerance.
- skills-disk.test.ts — atomic write, replacement without leftovers,
  path-escape rejection, atomic delete, listing ignores
  staging/trash artifacts.
- skills-state.test.ts — sha256 determinism, state round-trip,
  schema-version drift handling, edit detection.
- project-marker.test.ts — cwd hit, walk-up, $HOME boundary, empty
  marker, write+read round-trip.

The existing `mcpctl config claude` test (claude.test.ts) was updated
to pass `--skip-skills` so it stays focused on .mcp.json generation;
the new sync flow is covered by the utility tests.

Full suite: 162 test files / 2157 tests green (up from 158 / 2127).

## Deferred to a follow-up

- `metadata.hooks` materialisation into `~/.claude/settings.json` —
  the data path exists, sync receives it; PR-7 or a focused follow-up
  will write the `_mcpctl_managed: true` entries for declarative
  hooks.
- `metadata.mcpServers` auto-attach via mcpd API — likewise.
- `metadata.postInstall` script execution — the most substantive
  deferred piece. Current sync logs a TODO and skips. The corporate
  trust model (publisher-side rigor, not client-side defence) means
  this is straightforward to add once we wire the curated env +
  timeout + audit emission. Orthogonal to file sync, easier to ship
  separately.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 16:26:35 +01:00
Michal
db57bb5856 feat(mcpd+mcplocal+cli): propose-learnings system skill, propose_skill MCP tool, mcpctl review
Phase 4 of the Skills + Revisions + Proposals work. Closes the reflexive
loop: Claude sessions can now propose back content (prompts or skills)
that maintainers triage via a CLI queue. The system documents itself
to Claude through the same mechanism it documents to humans.

## What's added

### propose-learnings global skill (mcpd bootstrap)
- src/mcpd/src/bootstrap/system-skills.ts — idempotent upsert, mirrors
  system-project.ts. Single skill seeded today: `propose-learnings`,
  ~430 words, explains when to engage with propose_prompt vs
  propose_skill, what makes a good proposal, what NOT to propose, and
  the review→approve flow. Priority 9, global scope.
- main.ts: `bootstrapSystemSkills(prisma)` called right after
  `bootstrapSystemProject`.

### gate-encouragement-propose system prompt
- system-project.ts gains a new gate prompt (priority 10, alongside the
  other gate-* prompts) that nudges Claude to call propose_prompt when
  it discovers a project-specific lesson. Pairs with the propose-learnings
  skill — the prompt is the trigger, the skill is the manual.

### propose_skill MCP tool (mcplocal)
- proxymodel/plugins/gate.ts: new virtual tool registered alongside
  propose_prompt. Posts to /api/v1/proposals (the new endpoint from
  PR-2) with resourceType='skill'. Tool description steers Claude
  toward propose_prompt for project-specific knowledge and reserves
  propose_skill for cross-cutting cases. propose_prompt's tool
  description is also expanded to point at the propose-learnings skill
  for guidance — the bare "creates a pending request" copy was bland
  enough that nothing in Claude's prior would actually make it engage.

### mcpctl review CLI
- New top-level command in src/cli/src/commands/review.ts.
  Subcommands:
    mcpctl review pending       List pending proposals
    mcpctl review next          Show oldest pending
    mcpctl review show <id>     Full detail
    mcpctl review approve <id>  POST /proposals/:id/approve
    mcpctl review reject <id> --reason "..."
    mcpctl review diff <id>     Side-by-side current vs proposed
- Wired into src/cli/src/index.ts. Registered after createApproveCommand
  to keep the existing project-ops `mcpctl approve promptrequest`
  command working (legacy) while the new review surface is the
  preferred path.

## Tests touched

- bootstrap-system-project.test.ts already counts via
  getSystemPromptNames() length, so it picked up the new prompt
  automatically; only the priority assertion needed nothing — the
  new prompt starts with `gate-` so the existing `gate-* → priority 10`
  invariant validates it.
- system-prompt-validation.test.ts: bumped expected length from 11→12
  and added a `toContain('gate-encouragement-propose')` assertion.

Full suite: 158 test files / 2127 tests green.

## What's NOT in this PR

- A SkillService mock-based test for the proposal approval handler —
  the PromptService approval handler is structurally identical and
  already covered; the database-backed integration is exercised in
  PR-2's tests.
- Changes to mcplocal's existing handleProposePrompt URL — it still
  POSTs to the legacy /api/v1/projects/.../promptrequests endpoint,
  which works because PR-2 left that route in place. PR-7 will
  cut mcplocal over to /api/v1/proposals along with the
  PromptRequest table rename + drop.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 13:13:33 +01:00
Michal
356cbe87b5 feat(cli+mcplocal): mcpctl provider <name> {up,down,status} for managed LLMs
Some checks failed
CI/CD / typecheck (pull_request) Successful in 57s
CI/CD / test (pull_request) Successful in 1m23s
CI/CD / lint (pull_request) Successful in 3m1s
CI/CD / smoke (pull_request) Failing after 1m47s
CI/CD / build (pull_request) Successful in 5m58s
CI/CD / publish (pull_request) Has been skipped
Adds lifecycle control for managed local LLM providers (vllm-managed)
without the nuclear option of restarting mcplocal. Practical use:

  mcpctl provider vllm-local down    # release GPU memory now
  mcpctl provider vllm-local up      # warm up before the next chat
  mcpctl provider vllm-local status  # see state, pid, uptime

mcplocal exposes three new endpoints:

  GET  /llm/providers/:name/status   → returns lifecycle state for
                                       managed providers, { managed: false }
                                       for unmanaged (anthropic, openai, …)
  POST /llm/providers/:name/start    → calls warmup() (202 + initial state)
  POST /llm/providers/:name/stop     → calls dispose() (200 + post-stop state)

Stop and start return 400 for non-managed providers — stopping an API-key
provider is meaningless. The CLI surfaces the error verbatim.

Restarting mcplocal would also free the GPU but drops the SSE connection
to mcpd and forces every virtual Llm to re-publish; this is the targeted,
non-disruptive escape hatch.

The completions test gained a `topLevelMarkers` filter so a sub-command
named `status` (under `provider`) doesn't trip the existing "non-project
commands must guard with __mcpctl_has_project" rule.

Tests: cli 437/437, mcplocal 731/731.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 15:58:46 +01:00
Michal
7e6b0cab44 feat(cli): mcpctl chat-llm + KIND/STATUS columns (v1 Stage 5)
Closes the loop on user-facing surface:

  $ mcpctl get llm
  NAME             KIND     STATUS    TYPE     MODEL                       TIER  KEY  ID
  qwen3-thinking   public   active    openai   qwen3-thinking              fast  ...  ...
  vllm-local       virtual  active    openai   Qwen/Qwen2.5-7B-Instruct    fast  -    ...

  $ mcpctl chat-llm vllm-local
  ────────────────────────────────────────
  LLM: vllm-local  openai → Qwen/Qwen2.5-7B-Instruct-AWQ
  Kind: virtual    Status: active
  ────────────────────────────────────────
  > hello?
  Hi! …

New: chat-llm command (commands/chat-llm.ts)
- Stateless chat with any mcpd-registered LLM. No threads, no tools,
  no project prompts. POSTs to /api/v1/llms/<name>/infer; mcpd's
  kind=virtual branch handles relay-through-mcplocal transparently,
  so the same CLI command works for both public and virtual LLMs.
- Reuses installStatusBar / formatStats / recordDelta / styleStats /
  PhaseStats from chat.ts (now exported) so the bottom-row tokens-per-
  second ticker behaves identically to mcpctl chat.
- Flags: --message (one-shot), --system, --temperature, --max-tokens,
  --no-stream. Streaming uses OpenAI chat.completion.chunk SSE.
- REPL mode keeps a per-session history array so multi-turn flows
  feel natural; each turn is an independent inference call.

Updated: get.ts
- LlmRow gains optional kind/status fields.
- llmColumns layout: NAME, KIND, STATUS, TYPE, MODEL, TIER, KEY, ID.
  Defaults gracefully when older mcpd responses don't return them.

Updated: chat.ts
- Re-exports the helpers chat-llm.ts needs (PhaseStats, newPhase,
  recordDelta, formatStats, styleStats, styleThinking, STDERR_IS_TTY,
  StatusBar, installStatusBar). No behavior change.

Completions: chat-llm picks up the standard option enumeration
automatically; bash gets a special-case for first-arg LLM-name
completion via _mcpctl_resource_names "llms".

CLI suite: 437/437 (was 430, +7 from auto-discovered test cases in
the regenerated completions golden). Workspace: 2043/2043 across
152 files.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 14:25:38 +01:00
Michal
cc225eb70f feat(llm): probe upstream auth at registration time
mcpd now runs a cheap auth probe whenever an Llm is created (or its
apiKeyRef/url is updated). Catches misconfigured tokens / wrong URLs at
registration with a 422 + structured error message, instead of silently
500-ing on first chat with a generic "fetch failed". Caught in the wild
today: the homelab Pulumi config exposed `MCPCTL_GATEWAY_TOKEN` (which
is mcpctl_pat_-prefixed, intended for LiteLLM→mcplocal direction) where
LiteLLM expects `LITELLM_MASTER_KEY` (sk-prefixed). The probe makes
this immediate.

Probe shape (LlmAdapter.verifyAuth):
  - OpenAI passthrough → GET <url>/v1/models. Cheap, idempotent, gated
    by the same auth as chat/completions.
  - Anthropic → POST /v1/messages with max_tokens:1, "ping". Anthropic
    has no list-models endpoint; this is the cheapest auth-exercising
    call.
  - Returns one of:
      { ok: true }
      { ok: false, reason: "auth", status, body }    — 401/403, fail hard
      { ok: false, reason: "unreachable", error }    — network, warn-only
      { ok: false, reason: "unexpected", status, body } — non-auth 4xx, warn-only

Behavior:
  - LlmService.create()/update() runs the probe after resolveApiKey.
    Throws LlmAuthVerificationError on `auth`, logs warn for
    unreachable/unexpected, swallows for offline registration.
  - Probe is skipped when there's no apiKeyRef (nothing to verify) or
    when the caller passes skipAuthCheck=true.
  - update() probes only when apiKeyRef OR url changes — pure
    description/tier updates don't trigger upstream calls.
  - Routes catch LlmAuthVerificationError and return 422 with
    `{ error, status }`. The CLI surfaces the message verbatim via
    ApiError.

Opt-out:
  - CLI: `mcpctl create llm ... --skip-auth-check` for offline
    registration before the upstream is reachable.
  - HTTP: side-channel body field `_skipAuthCheck: true` (stripped
    before validation, never persisted on the row).

Side fix in same commit (caught while testing): src/cli/src/index.ts
read `program.opts()` BEFORE `program.parse()`, so `--direct` was a
no-op for ApiClient — every command went to mcplocal regardless. Some
commands accidentally still worked because mcplocal forwards plain
`/api/v1/*` to mcpd, but flows that need direct SSE streaming (e.g.
`mcpctl chat`) couldn't reach mcpd. Fixed by peeking at process.argv
directly for the two global flags before Commander's parse runs.

Tests:
  - llm-adapters.test.ts (+8): OpenAI 200/401/403/404/network, Anthropic
    200/401/400 (typo'd model = unexpected, NOT auth — registration
    shouldn't block on bad model names that surface at chat time).
  - llm-service.test.ts (+6): create-throws-on-auth-fail (no row
    written), warn-only on unreachable/unexpected, skipAuthCheck
    bypass, no-key skip, update-only-probes-on-auth-affecting-change.

mcpd 775/775, mcplocal 715/715, cli 430/430.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 16:51:55 +01:00
Michal
727e7d628c feat(agents): mcpctl chat REPL + agent CRUD + completions (Stage 5)
This is the moment the user can actually talk to an agent end-to-end:

  mcpctl create llm qwen3-thinking --type openai --model qwen3-thinking \
    --url http://litellm.nvidia-nim.svc.cluster.local:4000/v1 \
    --api-key-ref litellm-key/API_KEY
  mcpctl create agent reviewer --llm qwen3-thinking --project mcpctl-dev \
    --description "I review security design — ask me after each major change."
  mcpctl chat reviewer

Pieces:

* src/cli/src/commands/chat.ts (new) — REPL + one-shot. Streams the SSE
  endpoint and prints text deltas to stdout as they arrive; tool_call /
  tool_result events go to stderr in dim-style brackets so the chat
  output stays clean. LiteLLM-style flags (--temperature / --top-p /
  --top-k / --max-tokens / --seed / --stop / --allow-tool / --extra)
  layer over agent.defaultParams. In-REPL slash-commands: /set KEY VAL,
  /system <text>, /tools (list project's MCP servers), /clear (new
  thread), /save (PATCH agent.defaultParams = current overrides),
  /quit.

* src/cli/src/commands/create.ts — `create agent` mirroring the llm
  pattern. Every yaml-applyable field has a corresponding flag (memory
  rule); --default-temperature / --default-top-p / --default-top-k /
  --default-max-tokens / --default-seed / --default-stop /
  --default-extra / --default-params-file all populate agent.defaultParams.

* src/cli/src/commands/apply.ts — AgentSpecSchema accepts both `llm:
  qwen3-thinking` shorthand and `llm: { name: ... }` long form; runs
  after llms in the apply order so apiKey/llm references resolve. Round-
  trips with `get agent foo -o yaml | apply -f -` (memory rule).

* src/cli/src/commands/get.ts — agentColumns (NAME, LLM, PROJECT,
  DESCRIPTION, ID); RESOURCE_KIND mapping for yaml export.

* src/cli/src/commands/shared.ts — `agent`/`agents`/`thread`/`threads`
  added to RESOURCE_ALIASES.

* src/cli/src/index.ts — wires createChatCommand into the program; passes
  the resolved baseUrl + token so chat can stream SSE without going
  through ApiClient (which only does buffered request/response).

* completions/mcpctl.{fish,bash} regenerated. scripts/generate-completions.ts
  knows about agents (canonical + aliases) and emits a special-case
  `chat)` block that completes the first arg with `mcpctl get agents`
  names. tests/completions.test.ts: +9 new assertions covering agents in
  the resource list, chat in the commands list, --llm flag for create
  agent, agent-name completion for chat, etc.

CLI suite: 430/430 (was 421). Completions --check is clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 17:02:38 +01:00
Michal
dd4246878d feat(openbao): wizard-provisioning + daily token rotation
Some checks failed
CI/CD / typecheck (pull_request) Successful in 55s
CI/CD / test (pull_request) Successful in 1m4s
CI/CD / lint (pull_request) Successful in 2m2s
CI/CD / smoke (pull_request) Failing after 1m36s
CI/CD / build (pull_request) Successful in 4m13s
CI/CD / publish (pull_request) Has been skipped
One-command setup replaces the 6-step manual flow — `mcpctl create
secretbackend bao --type openbao --wizard` takes the OpenBao admin token
once, provisions a narrow policy + token role, mints the first periodic
token, stores it on mcpd, verifies end-to-end, and prints the migration
command. The admin token is NEVER persisted.

The stored credential auto-rotates daily: mcpd mints a successor via the
token role (self-rotation capability is part of the policy it was issued
with), verifies the successor, writes it over the backing Secret, then
revokes the predecessor by accessor. TTL 720h means a week of rotation
failures still leaves 20+ days of runway.

Shared:
- New `@mcpctl/shared/vault` — pure HTTP wrappers (verifyHealth,
  ensureKvV2, writePolicy, ensureTokenRole, mintRoleToken, revokeAccessor,
  lookupSelf, testWriteReadDelete) and policy HCL builder.

mcpd:
- `tokenMeta Json @default("{}")` on SecretBackend. Self-healing schema
  migration — empty default lets `prisma db push` add the column cleanly.
- SecretBackendRotator.rotateOne: mint → verify → persist → revoke-old →
  update tokenMeta. Failures surface via `lastRotationError` on the row;
  the old token keeps working.
- SecretBackendRotatorLoop: on startup rotates overdue backends, schedules
  per-backend timers with ±10min jitter. Stops cleanly on shutdown.
- New `POST /api/v1/secretbackends/:id/rotate` (operation
  `rotate-secretbackend` — added to bootstrap-admin's auto-migrated ops
  alongside migrate-secrets, which was previously missing too).

CLI:
- `--wizard` on `create secretbackend` delegates to the interactive flow.
  All prompts can be pre-answered via flags (--url, --admin-token,
  --mount, --path-prefix, --policy-name, --token-role,
  --no-promote-default) for CI.
- `mcpctl rotate secretbackend <name>` — convenience verb; hits the new
  rotate endpoint.
- `describe secretbackend` renders a Token health section (healthy /
  STALE / WARNING / ERROR) with generated/renewal/expiry timestamps and
  last rotation error. Only shown when tokenMeta.rotatable is true — the
  existing k8s-auth + static-token backends don't surface it.

Tests: 15 vault-client unit tests (shared), 8 rotator unit tests (mcpd),
3 wizard flow tests (cli, including a regression test that the admin
token never appears in stdout). Full suite 1885/1885 (+32). Completions
regenerated for the new flags.

Out of scope (explicit): kubernetes-auth wizard, Vault Enterprise
namespaces in the wizard path, rotation for non-wizard static-token
backends. See plan file for details.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 17:20:37 +01:00
Michal
029c3d5f34 feat(mcpd): pluggable SecretBackend abstraction + OpenBao driver + migrate
All checks were successful
CI/CD / typecheck (pull_request) Successful in 51s
CI/CD / lint (pull_request) Successful in 1m47s
CI/CD / test (pull_request) Successful in 1m3s
CI/CD / smoke (pull_request) Successful in 4m34s
CI/CD / build (pull_request) Successful in 3m50s
CI/CD / publish (pull_request) Has been skipped
Why: API keys live in Postgres as plaintext JSON. A DB read exposes every
credential in the system. Before centralising more secrets (LLM keys, etc.)
we want to be able to point at an external KV store and drop DB access to
sensitive rows.

New model:
- `SecretBackend` resource (CRUD + isDefault invariant) owns how a secret is
  stored. `Secret` gains `backendId` FK and `externalRef`. Reads/writes
  dispatch through a driver.
- `plaintext` driver (near-noop, uses existing Secret.data column) is seeded
  as the `default` row at startup. Acts as trust root / bootstrap.
- `openbao` driver (also HashiCorp Vault KV v2 compatible) talks plain HTTP,
  no SDK dependency. Auth via static token pulled from a plaintext-backed
  `Secret` through the injected SecretRefResolver. Caches resolved token.
- `SecretMigrateService` moves secrets one-at-a-time: read → write dest →
  flip row → best-effort source delete. Interrupted runs are idempotent
  (skips secrets already on destination).

CLI surface:
- `mcpctl create|get|describe|delete secretbackend` + `--default` on create.
- `mcpctl migrate secrets --from X --to Y [--names a,b] [--keep-source] [--dry-run]`
- `apply -f` round-trips secretbackends (yaml/json multi-doc + grouped).
- RBAC: `secretbackends` resource + `run:migrate-secrets` operation.
- Fish + bash completions regenerated.

docs/secret-backends.md covers the OpenBao policy, chicken-and-egg auth flow,
and the migration semantics.

Broke the circular dep (OpenBao needs SecretService to resolve its own token,
SecretService needs SecretBackendService) with a deferred-resolver bridge in
mcpd startup. 11 new driver unit tests; existing env-resolver/secret-route/
backup tests updated for the new service signatures. Full suite: 1792/1792.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 19:29:55 +01:00
Michal
2127b41d9f feat: HTTP-mode mcplocal container + mcpctl test mcp + token-auth preHandler
Delivers the final piece of the mcptoken stack: a containerized,
network-accessible mcplocal that serves Streamable-HTTP MCP to off-host
clients (the vLLM use case), authenticated by project-scoped McpTokens.

New binary (same package, new entry):
  - src/mcplocal/src/serve.ts — HTTP-only entry. Reads MCPLOCAL_MCPD_URL,
    MCPLOCAL_MCPD_TOKEN, MCPLOCAL_HTTP_HOST/PORT, MCPLOCAL_CACHE_DIR from
    env. No StdioProxyServer, no --upstream.
  - src/mcplocal/src/http/token-auth.ts — Fastify preHandler that
    validates mcpctl_pat_ bearers via mcpd's /api/v1/mcptokens/introspect.
    30s positive / 5s negative TTL. Rejects wrong-project with 403.

Shared HTTP MCP client:
  - src/shared/src/mcp-http/ — reusable McpHttpSession with initialize,
    listTools, callTool, close. Handles http+https, SSE, id correlation,
    distinct McpProtocolError / McpTransportError. Plus mcpHealthCheck
    and deriveBaseUrl helpers.

New CLI verb `mcpctl test mcp <url>`:
  - Flags: --token (also $MCPCTL_TOKEN), --tool, --args (JSON),
    --expect-tools, --timeout, -o text|json, --no-health.
  - Exit codes: 0 PASS, 1 TRANSPORT/AUTH FAIL, 2 CONTRACT FAIL.

Container + deploy:
  - deploy/Dockerfile.mcplocal (Node 20 alpine, multi-stage, pnpm
    workspace, CMD node src/mcplocal/dist/serve.js, VOLUME
    /var/lib/mcplocal/cache, HEALTHCHECK on :3200/healthz).
  - scripts/build-mcplocal.sh mirrors build-mcpd.sh.
  - fulldeploy.sh is now a 4-step pipeline that also builds + rolls out
    mcplocal (gated on `kubectl get deployment/mcplocal` so the script
    stays green before the Pulumi stack lands).

Audit + cache:
  - project-mcp-endpoint.ts passes MCPLOCAL_CACHE_DIR into FileCache at
    both construction sites and, when request.mcpToken is present, calls
    collector.setSessionMcpToken(id, ...) so audit events carry the
    tokenName/tokenSha.

Tests:
  - 9 unit cases on `mcpctl test mcp` (happy path, health miss,
    expect-tools hit/miss, transport throw, tool isError, json report,
    $MCPCTL_TOKEN env fallback, invalid --args).
  - Smoke test src/mcplocal/tests/smoke/mcptoken.smoke.test.ts —
    gated on healthz($MCPGW_URL), skipped cleanly when unreachable.
    Covers happy path, wrong-project 403, --expect-tools contract
    failure, and revocation 401 within the negative-cache window.

1773/1773 workspace tests pass. Pulumi resources (Deployment, Service,
Ingress, PVC, Secret, NetworkPolicy) still need to land in
../kubernetes-deployment before the smoke gate flips on.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 01:21:42 +01:00
Michal
a151b2e756 feat: mcpctl mcptoken verbs + mcpd auth dispatch + audit plumbing
Adds the end-to-end CLI surface for McpTokens and the mcpd auth dispatch
that recognizes them.

mcpd auth middleware:
  - Dispatch on the `mcpctl_pat_` bearer prefix. McpToken bearers resolve
    through a new `findMcpToken(hash)` dep, populating `request.mcpToken`
    and `request.userId = ownerId`. Everything else follows the existing
    session path.
  - Returns 401 for revoked / expired / unknown tokens.
  - Global RBAC hook now threads `mcpTokenSha` into `canAccess` /
    `canRunOperation` / `getAllowedScope`, and enforces a hard
    project-scope check: a McpToken principal can only hit
    `/api/v1/projects/<its-project>/...`.

CLI verbs:
  - `mcpctl create mcptoken <name> -p <proj> [--rbac empty|clone]
    [--bind role:view,resource:servers] [--ttl 30d|never|ISO]
    [--description ...] [--force]` — returns the raw token once.
  - `mcpctl get mcptokens [-p <proj>]` — table with
    NAME/PROJECT/PREFIX/CREATED/LAST USED/EXPIRES/STATUS.
  - `mcpctl get mcptoken <name> -p <proj>` and
    `mcpctl describe mcptoken <name> -p <proj>` — describe surfaces the
    auto-created RBAC bindings.
  - `mcpctl delete mcptoken <name> -p <proj>`.
  - `apply -f` support with `kind: mcptoken`. Tokens are immutable, so
    apply creates if missing and skips if the name is already active.

Audit plumbing:
  - `AuditEvent` / collector now carry optional `tokenName` / `tokenSha`.
    `setSessionMcpToken` sits alongside `setSessionUserName`; both feed a
    per-session principal map used at emit time.
  - `AuditEventService` query accepts `tokenName` / `tokenSha` filters.
  - Console `AuditEvent` type carries the new fields so a follow-up can
    add a TOKEN column.

Completions regenerated. 1764/1764 tests pass workspace-wide.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 01:12:43 +01:00
Michal
98f3a3eda0 refactor: consolidate restore under backup command
mcpctl backup restore list/diff/to instead of separate mcpctl restore.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 01:17:03 +00:00
Michal
a2728f280a feat: file cache, pause queue, hot-reload, and cache CLI commands
- Persistent file cache in ~/.mcpctl/cache/proxymodel/ with LRU eviction
- Pause queue for temporarily holding MCP traffic
- Hot-reload watcher for custom stages and proxymodel definitions
- CLI: mcpctl cache list/clear/stats commands
- HTTP endpoints for cache and pause management

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 23:36:55 +00:00
Michal
03827f11e4 feat: eager vLLM warmup and smart page titles in paginate stage
- Add warmup() to LlmProvider interface for eager subprocess startup
- ManagedVllmProvider.warmup() starts vLLM in background on project load
- ProviderRegistry.warmupAll() triggers all managed providers
- NamedProvider proxies warmup() to inner provider
- paginate stage generates LLM-powered descriptive page titles when
  available, cached by content hash, falls back to generic "Page N"
- project-mcp-endpoint calls warmupAll() on router creation so vLLM
  is loading while the session initializes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:07:39 +00:00
Michal
69867bd47a feat: mcpctl v0.0.1 — first public release
Some checks are pending
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / test (push) Waiting to run
CI / build (push) Blocked by required conditions
CI / package (push) Blocked by required conditions
Comprehensive MCP server management with kubectl-style CLI.

Key features in this release:
- Declarative YAML apply/get round-trip with project cloning support
- Gated sessions with prompt intelligence for Claude
- Interactive MCP console with traffic inspector
- Persistent STDIO connections for containerized servers
- RBAC with name-scoped bindings
- Shell completions (fish + bash) auto-generated
- Rate-limit retry with exponential backoff in apply
- Project-scoped prompt management
- Credential scrubbing from git history

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 17:05:05 +00:00
Michal
a59d2237b9 feat: interactive MCP console (mcpctl console <project>)
Ink-based TUI that shows exactly what an LLM sees through MCP.
Browse tools/resources/prompts, execute them, and see raw JSON-RPC
traffic in a protocol log. Supports gated session flow with
begin_session, raw JSON-RPC input, and session reconnect.

- McpSession class wrapping HTTP transport with typed methods
- 12 React/Ink components (header, protocol-log, menu, tool/resource/prompt views, etc.)
- 21 unit tests for McpSession against a mock MCP server
- Fish + Bash completions with project name argument
- bun compile with --external react-devtools-core

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 23:56:23 +00:00
Michal
ecc9c48597 feat: gated project experience & prompt intelligence
Implements the full gated session flow and prompt intelligence system:

- Prisma schema: add gated, priority, summary, chapters, linkTarget fields
- Session gate: state machine (gated → begin_session → ungated) with LLM-powered
  tool selection based on prompt index
- Tag matcher: intelligent prompt-to-tool matching with project/server/action tags
- LLM selector: tiered provider selection (fast for gating, heavy for complex tasks)
- Link resolver: cross-project MCP resource references (project/server:uri format)
- Prompt summary service: LLM-generated summaries and chapter extraction
- System project bootstrap: ensures default project exists on startup
- Structural link health checks: enrichWithLinkStatus on prompt GET endpoints
- CLI: create prompt --priority/--link, create project --gated/--no-gated,
  describe project shows prompts section, get prompts shows PRI/LINK/STATUS
- Apply/edit: priority, linkTarget, gated fields supported
- Shell completions: fish updated with new flags
- 1,253 tests passing across all packages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 23:22:42 +00:00
Michal
b025ade2b0 feat: add prompt resources, fix MCP proxy transport, enrich tool descriptions
- Fix MCP proxy to support SSE and STDIO transports (not just HTTP POST)
- Enrich tool descriptions with server context for LLM clarity
- Add Prompt and PromptRequest resources with two-resource RBAC model
- Add propose_prompt MCP tool for LLM to create pending prompt requests
- Add prompt resources visible in MCP resources/list (approved + session's pending)
- Add project-level prompt/instructions in MCP initialize response
- Add ServiceAccount subject type for RBAC (SA identity from X-Service-Account header)
- Add CLI commands: create prompt, get prompts/promptrequests, approve promptrequest
- Add prompts to apply config schema
- 956 tests passing across all packages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 14:53:00 +00:00
Michal
b241b3d91c feat: add mcpctl mcp STDIO bridge, rework config claude
- New `mcpctl mcp -p PROJECT` command: STDIO-to-StreamableHTTP bridge
  that reads JSON-RPC from stdin and forwards to mcplocal project endpoint
- Rework `config claude` to write mcpctl mcp entry instead of fetching
  server configs from API (no secrets in .mcp.json)
- Keep `config claude-generate` as backward-compat alias
- Fix discovery.ts auth token not being forwarded to mcpd (RBAC bypass)
- Update fish/bash completions for new commands
- 10 new MCP bridge tests, updated claude tests, fixed project-discovery test

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 00:52:05 +00:00
Michal
846fbf8ae9 feat: context-aware completions with dynamic resource names
- Hide attach-server/detach-server from --help (only relevant with --project)
- --project shows only project-scoped commands in tab completion
- Tab after resource type fetches live resource names from API
- --project value auto-completes from existing project names
- Stop offering resource types after one is already selected

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 19:08:29 +00:00
Michal
88b9158197 feat: --project flag scopes get servers/instances to project
mcpctl --project NAME get servers — shows only servers attached to the project
mcpctl --project NAME get instances — shows only instances of project servers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 19:03:07 +00:00
Michal
329315ec71 feat: remove ProjectMember, add expose RBAC role, attach/detach-server commands
- Remove ProjectMember model entirely (RBAC manages project access)
- Add 'expose' RBAC role for /mcp-config endpoint access (edit implies expose)
- Rename CLI flags: --llm-provider → --proxy-mode-llm-provider, --llm-model → --proxy-mode-llm-model
- Add attach-server / detach-server CLI commands (mcpctl --project NAME attach-server SERVER)
- Add POST/DELETE /api/v1/projects/:id/servers endpoints for server attach/detach
- Remove members from backup/restore, apply, get, describe
- Prisma migration to drop ProjectMember table

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 17:50:01 +00:00
Michal
c5147e8270 feat: granular RBAC with resource/operation bindings, users, groups
- Replace admin role with granular roles: view, create, delete, edit, run
- Two binding types: resource bindings (role+resource+optional name) and
  operation bindings (role:run + action like backup, logs, impersonate)
- Name-scoped resource bindings for per-instance access control
- Remove role from project members (all permissions via RBAC)
- Add users, groups, RBAC CRUD endpoints and CLI commands
- describe user/group shows all RBAC access (direct + inherited)
- create rbac supports --subject, --binding, --operation flags
- Backup/restore handles users, groups, RBAC definitions
- mcplocal project-based MCP endpoint discovery
- Full test coverage for all new functionality

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 11:05:19 +00:00
Michal
e16b3a3003 fix: proper error handling and --force flag for create commands
- Add global error handler: clean messages instead of stack traces
- Add --force flag to create server/secret/project: updates on 409 conflict
- Strip null values and template-only fields from --from-template payload
- Add tests: 409 handling, --force update, null-stripping from templates

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 23:06:33 +00:00
Michal
73fb70dce4 feat: add MCP server templates and deployment infrastructure
Introduce a Helm-chart-like template system for MCP servers. Templates are
YAML files in templates/ that get seeded into the DB on startup. Users can
browse them with `mcpctl get templates`, inspect with `mcpctl describe
template`, and instantiate with `mcpctl create server --from-template=`.

Also adds Portainer deployment scripts, mcplocal systemd service,
Streamable HTTP MCP endpoint, and RPM packaging for mcpctl-local.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 22:24:35 +00:00
Michal
6d9a9f572c feat: replace profiles with kubernetes-style secrets
Replace the confused Profile abstraction with a dedicated Secret resource
following Kubernetes conventions. Servers now have env entries with inline
values or secretRef references. Env vars are resolved and passed to
containers at startup (fixes existing gap).

- Add Secret CRUD (model, repo, service, routes, CLI commands)
- Server env: {name, value} or {name, valueFrom: {secretRef: {name, key}}}
- Add env-resolver utility shared by instance startup and config generation
- Remove all profile-related code (models, services, routes, CLI, tests)
- Update backup/restore for secrets instead of profiles
- describe secret masks values by default, --show-values to reveal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 18:40:58 +00:00
Michal
ede9e10990 fix: enable positional options so -o works on subcommands
Remove global -o/--output from parent program and enable
enablePositionalOptions() so -o yaml/json is parsed by subcommands.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 16:43:35 +00:00
Michal
68d0013bfe fix: resolve resource names in get/describe (not just IDs)
fetchResource and fetchSingleResource now use resolveNameOrId so
`mcpctl get server ha-mcp` works by name, not just by ID.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 16:39:21 +00:00
Michal
e3aba76cc8 feat: add create/edit commands, apply-compatible output, better describe
- `create server/profile/project` with all CLI flags (kubectl parity)
- `edit server/profile/project` opens $EDITOR for in-flight editing
- `get -o yaml/json` now outputs apply-compatible format (strips internal fields, wraps in resource key)
- `describe` shows visually clean sectioned output with aligned columns
- Extract shared utilities (resolveResource, resolveNameOrId, stripInternalFields)
- Instances are immutable (no create/edit, like pods)
- Full test coverage for create, edit, and updated describe/get

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 14:33:25 +00:00
Michal
bd09ae9687 feat: kubectl-style CLI + Deployment/Pod model for servers/instances
Server = Deployment (defines what to run + desired replicas)
Instance = Pod (ephemeral, auto-created by reconciliation)

Backend:
- Add replicas field to McpServer schema
- Add reconcile() to InstanceService (scales instances to match replicas)
- Remove manual start/stop/restart - instances are auto-managed
- Cascade: deleting server stops all containers then cascades DB
- Server create/update auto-triggers reconciliation

CLI:
- Add top-level delete command (servers, instances, profiles, projects)
- Add top-level logs command
- Remove instance compound command (use get/delete/logs instead)
- Clean up project command (list/show/delete → top-level get/describe/delete)
- Enhance describe for instances with container inspect info
- Add replicas to apply command's ServerSpec

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 13:30:46 +00:00
Michal
b8c5cf718a feat: implement v2 3-tier architecture (mcpctl → mcplocal → mcpd)
Some checks failed
CI / lint (pull_request) Has been cancelled
CI / typecheck (pull_request) Has been cancelled
CI / test (pull_request) Has been cancelled
CI / build (pull_request) Has been cancelled
CI / package (pull_request) Has been cancelled
- Rename local-proxy to mcplocal with HTTP server, LLM pipeline, mcpd discovery
- Add LLM pre-processing: token estimation, filter cache, metrics, Gemini CLI + DeepSeek providers
- Add mcpd auth (login/logout) and MCP proxy endpoints
- Update CLI: dual URLs (mcplocalUrl/mcpdUrl), auth commands, --direct flag
- Add tiered health monitoring, shell completions, e2e integration tests
- 57 test files, 597 tests passing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 11:42:06 +00:00
Michal
6da4ae495c feat: add backup and restore with encrypted secrets
Some checks are pending
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / test (push) Waiting to run
CI / build (push) Blocked by required conditions
BackupService exports servers/profiles/projects to JSON bundle.
RestoreService imports with skip/overwrite/fail conflict strategies.
AES-256-GCM encryption for sensitive env vars via scrypt-derived keys.
REST endpoints and CLI commands for backup/restore operations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 05:40:46 +00:00
Michal
a520b9ff47 feat: add claude and project CLI commands
Some checks are pending
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / test (push) Waiting to run
CI / build (push) Blocked by required conditions
Claude command manages .mcp.json files (generate from project, show, add,
remove entries). Project command provides CRUD for projects with profile
assignment management.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 05:16:57 +00:00
Michal
9c08faa8d2 feat: add apply command for declarative config and interactive setup wizard
Some checks are pending
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / test (push) Waiting to run
CI / build (push) Blocked by required conditions
Apply reads YAML/JSON config files to sync servers, profiles, and projects
to the daemon with create-or-update semantics. Setup provides an interactive
wizard for configuring MCP servers with environment variables.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 05:14:43 +00:00
Michal
4d796e2aa7 feat: add instance lifecycle management with restart, inspect, and CLI commands
Adds restart/inspect methods to InstanceService, state validation for stop,
REST endpoints for restart and inspect, and full CLI command suite for
instance list/start/stop/restart/remove/logs/inspect.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 05:11:48 +00:00
Michal
1b8b886995 feat: add get and describe commands with API client
kubectl-style get (table/json/yaml) and describe commands for servers,
profiles, projects, instances. ApiClient for daemon communication.
118 CLI tests passing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 04:55:45 +00:00
Michal
247b4967e5 feat: build CLI core framework with Commander.js
Add CLI entry point with Commander.js, config management (~/.mcpctl/config.json
with Zod validation), output formatters (table/json/yaml), config and status
commands with dependency injection for testing. Fix sanitizeString regex ordering.
67 tests passing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 04:35:00 +00:00
Michal
d0aa0c5d63 first commit 2026-02-21 03:10:39 +00:00