Commit Graph

23 Commits

Author SHA1 Message Date
Michal
6ff90a8228 feat(mcpd): Llm resource — CRUD + CLI + apply
Why: every client that wants an LLM (the agent, HTTP-mode mcplocal, Claude
Code's STDIO mcplocal) today has to know the provider URL + key, and each
user's ~/.mcpctl/config.json carries them. Centralising the catalogue on the
server is the prerequisite for Phase 2 (mcpd proxies inference so credentials
never leave the cluster).

This phase adds the `Llm` resource and its CRUD surface — no proxy yet, no
client pivot yet. Just enough to register what you have.

Schema:
- New `Llm` model: name/type/model/url/tier/description + {apiKeySecretId,
  apiKeySecretKey} FK pair. Reverse `llms` relation on Secret.
- Provider types: anthropic | openai | deepseek | vllm | ollama | gemini-cli.
- Tiers: fast | heavy.

mcpd:
- LlmRepository + LlmService + Zod validation schema + /api/v1/llms routes.
- API surface exposes `apiKeyRef: {name, key}` — the service translates to/
  from the FK pair so clients never deal in cuids.
- `resolveApiKey(llmName)` reads through SecretService (which itself dispatches
  to the right SecretBackend). That's the hook Phase 2's inference proxy uses.
- RBAC: added `'llms'` to RBAC_RESOURCES + resource alias. Standard
  view/create/edit/delete semantics.
- Wired into main.ts (repo, service, routes).

CLI:
- `mcpctl create llm <name> --type X --model Y --tier fast|heavy --api-key-ref SECRET/KEY [--url ...] [--extra k=v ...]`
- `mcpctl get|describe|delete llm` — standard resource verbs.
- `mcpctl apply -f` with `kind: llm` (single- or multi-doc yaml/json).
  Applied after secrets, before servers — apiKeyRef resolves an existing Secret.
- Shell completions regenerated.

Tests: 11 service unit tests + 9 route tests (happy path, 404s, 409, validation).
Full suite 1812/1812 (+20 from the 1792 Phase 0 baseline). TypeScript clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 21:28:43 +01:00
Michal
029c3d5f34 feat(mcpd): pluggable SecretBackend abstraction + OpenBao driver + migrate
All checks were successful
CI/CD / typecheck (pull_request) Successful in 51s
CI/CD / lint (pull_request) Successful in 1m47s
CI/CD / test (pull_request) Successful in 1m3s
CI/CD / smoke (pull_request) Successful in 4m34s
CI/CD / build (pull_request) Successful in 3m50s
CI/CD / publish (pull_request) Has been skipped
Why: API keys live in Postgres as plaintext JSON. A DB read exposes every
credential in the system. Before centralising more secrets (LLM keys, etc.)
we want to be able to point at an external KV store and drop DB access to
sensitive rows.

New model:
- `SecretBackend` resource (CRUD + isDefault invariant) owns how a secret is
  stored. `Secret` gains `backendId` FK and `externalRef`. Reads/writes
  dispatch through a driver.
- `plaintext` driver (near-noop, uses existing Secret.data column) is seeded
  as the `default` row at startup. Acts as trust root / bootstrap.
- `openbao` driver (also HashiCorp Vault KV v2 compatible) talks plain HTTP,
  no SDK dependency. Auth via static token pulled from a plaintext-backed
  `Secret` through the injected SecretRefResolver. Caches resolved token.
- `SecretMigrateService` moves secrets one-at-a-time: read → write dest →
  flip row → best-effort source delete. Interrupted runs are idempotent
  (skips secrets already on destination).

CLI surface:
- `mcpctl create|get|describe|delete secretbackend` + `--default` on create.
- `mcpctl migrate secrets --from X --to Y [--names a,b] [--keep-source] [--dry-run]`
- `apply -f` round-trips secretbackends (yaml/json multi-doc + grouped).
- RBAC: `secretbackends` resource + `run:migrate-secrets` operation.
- Fish + bash completions regenerated.

docs/secret-backends.md covers the OpenBao policy, chicken-and-egg auth flow,
and the migration semantics.

Broke the circular dep (OpenBao needs SecretService to resolve its own token,
SecretService needs SecretBackendService) with a deferred-resolver bridge in
mcpd startup. 11 new driver unit tests; existing env-resolver/secret-route/
backup tests updated for the new service signatures. Full suite: 1792/1792.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 19:29:55 +01:00
Michal
2ddb493bb0 feat(mcpd): McpToken schema + CRUD routes + introspection
Adds a new McpToken Prisma model (project-scoped, SHA-256 hashed at rest,
optional expiry, revocable) plus backing repository, service, and REST
routes. Tokens are a first-class RBAC subject: new 'McpToken' kind is
added to the subject enum and the service auto-creates an RbacDefinition
with subject McpToken:<sha> when bindings are provided.

Creator-permission ceiling: the service rejects any requested binding
the creator cannot already satisfy themselves (re-uses
rbacService.canAccess / canRunOperation). rbacMode=clone snapshots the
creator's full permissions into the token.

Routes:
  POST   /api/v1/mcptokens              create (returns raw token once)
  GET    /api/v1/mcptokens              list (filter by project)
  GET    /api/v1/mcptokens/:id          describe (no secret in response)
  POST   /api/v1/mcptokens/:id/revoke   soft-delete + remove RbacDef
  DELETE /api/v1/mcptokens/:id          hard-delete
  GET    /api/v1/mcptokens/introspect   validate raw bearer (used by mcplocal)

Extends AuditEvent with optional tokenName/tokenSha fields (indexed) so
token-driven activity can be filtered later. Adds token helpers in
@mcpctl/shared: TOKEN_PREFIX='mcpctl_pat_', generateToken, hashToken,
isMcpToken, timingSafeEqualHex.

Follow-up PRs add the auth-hook dispatch on the prefix, the CLI verbs,
and the HTTP-mode mcplocal that calls /introspect.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 01:00:04 +01:00
Michal
3149ea3ae7 fix: MCP proxy resilience — discovery cache, default liveness probes
Some checks failed
CI/CD / lint (push) Successful in 52s
CI/CD / typecheck (push) Successful in 1m51s
CI/CD / test (push) Successful in 1m1s
CI/CD / smoke (push) Failing after 3m21s
CI/CD / build (push) Successful in 4m9s
CI/CD / publish (push) Has been skipped
Adds a per-server tools/list cache in McpRouter (positive + negative TTL)
so a slow or dead upstream only stalls the first discovery call, not every
subsequent client request. Invalidated on upstream add/remove.

Health probes now apply a default liveness spec (tools/list via the real
production path) to any RUNNING instance without an explicit healthCheck,
so synthetic and real failures converge on the same signal.

Includes supporting updates in mcpd-client, discovery, upstream/mcpd,
seeder, and fulldeploy/release scripts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 00:48:57 +01:00
Michal
7818cb2194 feat: Git-based backup system replacing JSON bundle backup/restore
DB is source of truth with git as downstream replica. SSH key generated
on first start, all resource mutations committed as apply-compatible YAML.
Supports manual commit import, conflict resolution (DB wins), disaster
recovery (empty DB restores from git), and timeline branches on restore.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 01:14:28 +00:00
Michal
0995851810 feat: remove proxyMode — all traffic goes through mcplocal proxy
proxyMode "direct" was a security hole (leaked secrets as plaintext env
vars in .mcp.json) and bypassed all mcplocal features (gating, audit,
RBAC, content pipeline, namespacing). Removed from schema, API, CLI,
and all tests. Old configs with proxyMode are accepted but silently
stripped via Zod .transform() for backward compatibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 23:36:36 +00:00
Michal
cfe0d99c8f fix: exclude db tests from workspace root and fix TS build errors
- Exclude src/db/tests from workspace vitest config (needs test DB)
- Make global-setup.ts gracefully skip when test DB unavailable
- Fix exactOptionalPropertyTypes issues in proxymodel-endpoint.ts
- Use proper ProxyModelPlugin type for getPluginHooks function

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 00:39:25 +00:00
Michal
86c5a61eaa feat: add userName tracking to audit events
- Add userName column to AuditEvent schema with index and migration
- Add GET /api/v1/auth/me endpoint returning current user identity
- AuditCollector auto-fills userName from session→user map, resolved
  lazily via /auth/me on first session creation
- Support userName and date range (from/to) filtering on audit events
  and sessions endpoints
- Audit console sidebar groups sessions by project → user
- Add date filter presets (d key: all/today/1h/24h/7d) to console
- Add scrolling and page up/down to sidebar navigation
- Tests: auth-me (4), audit-username collector (4), route filters (2),
  smoke tests (2)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 00:18:58 +00:00
Michal
03827f11e4 feat: eager vLLM warmup and smart page titles in paginate stage
- Add warmup() to LlmProvider interface for eager subprocess startup
- ManagedVllmProvider.warmup() starts vLLM in background on project load
- ProviderRegistry.warmupAll() triggers all managed providers
- NamedProvider proxies warmup() to inner provider
- paginate stage generates LLM-powered descriptive page titles when
  available, cached by content hash, falls back to generic "Page N"
- project-mcp-endpoint calls warmupAll() on router creation so vLLM
  is loading while the session initializes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:07:39 +00:00
Michal
69867bd47a feat: mcpctl v0.0.1 — first public release
Some checks are pending
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / test (push) Waiting to run
CI / build (push) Blocked by required conditions
CI / package (push) Blocked by required conditions
Comprehensive MCP server management with kubectl-style CLI.

Key features in this release:
- Declarative YAML apply/get round-trip with project cloning support
- Gated sessions with prompt intelligence for Claude
- Interactive MCP console with traffic inspector
- Persistent STDIO connections for containerized servers
- RBAC with name-scoped bindings
- Shell completions (fish + bash) auto-generated
- Rate-limit retry with exponential backoff in apply
- Project-scoped prompt management
- Credential scrubbing from git history

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 17:05:05 +00:00
Michal
ecc9c48597 feat: gated project experience & prompt intelligence
Implements the full gated session flow and prompt intelligence system:

- Prisma schema: add gated, priority, summary, chapters, linkTarget fields
- Session gate: state machine (gated → begin_session → ungated) with LLM-powered
  tool selection based on prompt index
- Tag matcher: intelligent prompt-to-tool matching with project/server/action tags
- LLM selector: tiered provider selection (fast for gating, heavy for complex tasks)
- Link resolver: cross-project MCP resource references (project/server:uri format)
- Prompt summary service: LLM-generated summaries and chapter extraction
- System project bootstrap: ensures default project exists on startup
- Structural link health checks: enrichWithLinkStatus on prompt GET endpoints
- CLI: create prompt --priority/--link, create project --gated/--no-gated,
  describe project shows prompts section, get prompts shows PRI/LINK/STATUS
- Apply/edit: priority, linkTarget, gated fields supported
- Shell completions: fish updated with new flags
- 1,253 tests passing across all packages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 23:22:42 +00:00
Michal
b025ade2b0 feat: add prompt resources, fix MCP proxy transport, enrich tool descriptions
- Fix MCP proxy to support SSE and STDIO transports (not just HTTP POST)
- Enrich tool descriptions with server context for LLM clarity
- Add Prompt and PromptRequest resources with two-resource RBAC model
- Add propose_prompt MCP tool for LLM to create pending prompt requests
- Add prompt resources visible in MCP resources/list (approved + session's pending)
- Add project-level prompt/instructions in MCP initialize response
- Add ServiceAccount subject type for RBAC (SA identity from X-Service-Account header)
- Add CLI commands: create prompt, get prompts/promptrequests, approve promptrequest
- Add prompts to apply config schema
- 956 tests passing across all packages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 14:53:00 +00:00
Michal
329315ec71 feat: remove ProjectMember, add expose RBAC role, attach/detach-server commands
- Remove ProjectMember model entirely (RBAC manages project access)
- Add 'expose' RBAC role for /mcp-config endpoint access (edit implies expose)
- Rename CLI flags: --llm-provider → --proxy-mode-llm-provider, --llm-model → --proxy-mode-llm-model
- Add attach-server / detach-server CLI commands (mcpctl --project NAME attach-server SERVER)
- Add POST/DELETE /api/v1/projects/:id/servers endpoints for server attach/detach
- Remove members from backup/restore, apply, get, describe
- Prisma migration to drop ProjectMember table

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 17:50:01 +00:00
Michal
c5147e8270 feat: granular RBAC with resource/operation bindings, users, groups
- Replace admin role with granular roles: view, create, delete, edit, run
- Two binding types: resource bindings (role+resource+optional name) and
  operation bindings (role:run + action like backup, logs, impersonate)
- Name-scoped resource bindings for per-instance access control
- Remove role from project members (all permissions via RBAC)
- Add users, groups, RBAC CRUD endpoints and CLI commands
- describe user/group shows all RBAC access (direct + inherited)
- create rbac supports --subject, --binding, --operation flags
- Backup/restore handles users, groups, RBAC definitions
- mcplocal project-based MCP endpoint discovery
- Full test coverage for all new functionality

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 11:05:19 +00:00
Michal
90f3beee50 fix: add missing passwordHash to DB test user factory
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 01:02:41 +00:00
Michal
ae695d2141 feat: add MCP healthcheck probes and new templates (grafana, home-assistant, node-red)
- Add healthCheck spec to templates and servers (tool, arguments, interval, timeout, failureThreshold)
- Add healthStatus, lastHealthCheck, events fields to instances
- Create grafana, home-assistant, node-red templates with healthcheck probes
- Add healthcheck probes to existing templates (github, slack, postgres, jira)
- Show HEALTH column in `get instances` and Events section in `describe instance`
- Display healthCheck details in `describe server` and `describe template`
- Schema + storage + display only; actual probe runner is future work

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 22:48:59 +00:00
Michal
73fb70dce4 feat: add MCP server templates and deployment infrastructure
Introduce a Helm-chart-like template system for MCP servers. Templates are
YAML files in templates/ that get seeded into the DB on startup. Users can
browse them with `mcpctl get templates`, inspect with `mcpctl describe
template`, and instantiate with `mcpctl create server --from-template=`.

Also adds Portainer deployment scripts, mcplocal systemd service,
Streamable HTTP MCP endpoint, and RPM packaging for mcpctl-local.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 22:24:35 +00:00
Michal
6d9a9f572c feat: replace profiles with kubernetes-style secrets
Replace the confused Profile abstraction with a dedicated Secret resource
following Kubernetes conventions. Servers now have env entries with inline
values or secretRef references. Env vars are resolved and passed to
containers at startup (fixes existing gap).

- Add Secret CRUD (model, repo, service, routes, CLI commands)
- Server env: {name, value} or {name, valueFrom: {secretRef: {name, key}}}
- Add env-resolver utility shared by instance startup and config generation
- Remove all profile-related code (models, services, routes, CLI, tests)
- Update backup/restore for secrets instead of profiles
- describe secret masks values by default, --show-values to reveal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 18:40:58 +00:00
Michal
bd09ae9687 feat: kubectl-style CLI + Deployment/Pod model for servers/instances
Server = Deployment (defines what to run + desired replicas)
Instance = Pod (ephemeral, auto-created by reconciliation)

Backend:
- Add replicas field to McpServer schema
- Add reconcile() to InstanceService (scales instances to match replicas)
- Remove manual start/stop/restart - instances are auto-managed
- Cascade: deleting server stops all containers then cascades DB
- Server create/update auto-triggers reconciliation

CLI:
- Add top-level delete command (servers, instances, profiles, projects)
- Add top-level logs command
- Remove instance compound command (use get/delete/logs instead)
- Clean up project command (list/show/delete → top-level get/describe/delete)
- Enhance describe for instances with container inspect info
- Add replicas to apply command's ServerSpec

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 13:30:46 +00:00
Michal
5d13a0c562 feat: add external MCP server support with streamable-http proxy
Support non-containerized MCP servers via externalUrl field and add
streamable-http session management for HA MCP proof of concept.

- Add externalUrl, command, containerPort fields to McpServer schema
- Skip Docker orchestration for external servers (virtual instances)
- Implement streamable-http proxy with Mcp-Session-Id session management
- Parse SSE-framed responses from streamable-http endpoints
- Add command passthrough to Docker container creation
- Create HA MCP example manifest (examples/ha-mcp.yaml)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 12:21:25 +00:00
Michal
b8c5cf718a feat: implement v2 3-tier architecture (mcpctl → mcplocal → mcpd)
Some checks failed
CI / lint (pull_request) Has been cancelled
CI / typecheck (pull_request) Has been cancelled
CI / test (pull_request) Has been cancelled
CI / build (pull_request) Has been cancelled
CI / package (pull_request) Has been cancelled
- Rename local-proxy to mcplocal with HTTP server, LLM pipeline, mcpd discovery
- Add LLM pre-processing: token estimation, filter cache, metrics, Gemini CLI + DeepSeek providers
- Add mcpd auth (login/logout) and MCP proxy endpoints
- Update CLI: dual URLs (mcplocalUrl/mcpdUrl), auth commands, --direct flag
- Add tiered health monitoring, shell completions, e2e integration tests
- 57 test files, 597 tests passing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 11:42:06 +00:00
Michal
dc45f5981b feat: implement database schema with Prisma ORM
Add PostgreSQL schema with 8 models (User, Session, McpServer, McpProfile,
Project, ProjectMcpProfile, McpInstance, AuditLog), comprehensive model
tests (31 passing), seed data for default MCP servers, and package exports.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 04:34:05 +00:00
Michal
d0aa0c5d63 first commit 2026-02-21 03:10:39 +00:00