Integrate skills + v7-visibility + mcpctl passwd (deployed) #75

Merged

michal merged 19 commits from integration/skills-v7-passwd into main 2026-06-16 22:16:49 +00:00

Conversation 0 Commits 19 Files Changed 119 +10866 -394

19 Commits

Author	SHA1	Message	Date
Michal	467757b966	feat(status): show SecretBackend health in mcpctl status ... Adds a 'Secrets:' line to mcpctl status (and a secretBackends array to JSON output) showing each backend healthy (✓) or, when tokenMeta.lastRotationError is set (e.g. a dead OpenBao token), red ✗ with the error inline. Makes a recurrence of BACKEND_TOKEN_DEAD visible at a glance instead of only in mcpd logs. Verified live: 'Secrets: bao* ✓, default ✓'. +3 tests (28 total). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 23:13:50 +01:00
Michal	4c7e648771	fix(smoke): read CLI credentials from ~/.mcpctl (active path) ... passwd smoke read ~/.config/mcpctl/credentials and silently skipped; the CLI stores creds at ~/.mcpctl/credentials. Now verified live against prod (4/4). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 22:41:30 +01:00
Michal	ad2ba12b5b	feat(deploy): versioned, reversible Pulumi-driven k8s deploy script ... scripts/deploy-k8s.sh replaces fulldeploy.sh's rollout-restart-:latest pattern (which bypassed Pulumi and left no rollback target). It: - gates on pnpm test:run - captures the current prod images as immutable rollback tags (skopeo) + records digests - pg_dumps the prod DB before the destructive-capable `prisma db push` - builds/pushes mcpd+mcplocal tagged with the git short-sha - pins the sha in ../kubernetes-deployment/Pulumi.homelab.yaml and runs `pulumi up --target` the mcpd/mcplocal Deployments only (avoids the SOGo docker-image resource that needs a local docker daemon) - waits for rollout + /healthz, installs the CLI RPM, runs smoke tests - prints an exact rollback recipe on post-cutover failure --dry-run validated: tests/pg_dump/targeted preview run read-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 22:26:04 +01:00
Michal	2cceeb7093	feat(passwd): mcpctl passwd + RBAC-gated password change ... Restores the lost `mcpctl passwd` command and builds the backend it needs. Backend (mcpd): - POST /api/v1/users/me/password — self-service change, requires current password. Gated by a new `set-own-password` operation. - PUT /api/v1/users/:id/password — admin reset of another user, gated by edit:users (admins have edit:*). Added users name-resolver for CUID→email. - UserService.setPassword/verifyPassword; UserRepository.update accepts passwordHash + findByIdWithHash. RBAC, no exceptions: self password change is a default, admin-revocable permission. Every new user gets a `self-<id>` RbacDefinition granting `set-own-password`, seeded on create + bootstrap, gated by the `allowSelfPasswordChange` system setting (stored in the mcpctl-system-settings secret, default ON; admins disable globally or revoke per-user). CLI: src/cli/src/commands/passwd.ts (self vs admin paths) + completions. Tests: users-password route tests (8), auth-bootstrap grant assertion, passwd live smoke test. Full suite 2214 passing; zero new lint errors. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 21:55:56 +01:00
Michal	0e952dbf68	merge: integrate v7-rbac-visibility on top of skills feature ... Brings together the two unmerged feature lines onto one integration branch: - skills/review/proposals/revisions (via fix/mcpd-instance-health-and-retry, which contains the full skills-1..7 chain + mcpd env/retry/readiness fix) - v7 visibility scope + ownership for Llms and Agents Auto-merge resolved schema.prisma, mcpd main.ts, mcplocal config, CLI create, and completions with no conflicts. Both Prisma migrations coexist. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 21:31:35 +01:00
Michal	180e50a978	feat(cli): metadata.mcpServers auto-attach in mcpctl skills sync ... Closes the third deferred item from PR-5: skills can declare upstream MCP server dependencies via `metadata.mcpServers` and `mcpctl skills sync` now attaches them to the active project. Same trust model as postInstall/hooks: the publisher is responsible, the client just asks mcpd to attach. ## Behaviour - For each `{ name, fromTemplate?, project? }` entry: - If the project already has the server attached → record as `alreadyAttached`, no-op. - If the server doesn't exist on mcpd → warn + skip (we don't auto-create from template; that's a separate explicit decision for the operator). The warning surfaces the suggested template so the operator can decide. - Otherwise → POST `/api/v1/projects/:id/servers { server: <name> }`. - 409 from POST → treat as alreadyAttached (server-side idempotency). - 404 from POST → treat as missing (race: server vanished mid-sync). - Other errors collected per-server; sync continues. - A dep with `project:` set to a non-active project is skipped during this sync — keeps the active sync from making collateral changes to other projects. - Global skill syncs (no project context) skip mcpServers entirely with a warning — there's no project to attach to. ## Files ### Added - src/cli/src/utils/mcpservers-materialiser.ts (~140 LOC) - src/cli/tests/utils/mcpservers-materialiser.test.ts (~190 LOC, 10 tests covering: parse-tolerance, fresh attach, alreadyAttached short-circuit, missing-server warn+skip, missing-project errors- out, 409→alreadyAttached, 404→missing, cross-project skip, per-server error collection, empty-deps no-op) ### Edited - src/cli/src/commands/skills.ts: applyOne calls attachSkillMcpServers after hooks. Tracks per-skill attachments in result.mcpServersAttached. Summary line surfaces "N mcpServers attached". ## Verification 165 test files / 2193 tests green (up from 2182). Real-world flow: ```yaml # skill metadata.yaml mcpServers: - name: my-grafana fromTemplate: grafana project: monitoring - name: my-loki fromTemplate: loki ``` ```bash # As operator: ensure the named servers exist on mcpd first mcpctl create server my-grafana --from-template grafana --env-from-secret grafana-creds mcpctl create server my-loki --from-template loki # Now publish the skill that declares them as deps. Sync will attach: mcpctl skills sync # → mcpctl: 1 installed, 2 mcpServers attached mcpctl describe project monitoring # both servers now attached ``` ## What's intentionally NOT in this PR - Auto-creating servers from `fromTemplate` when they don't exist. Provisioning infra from a skill push is a separate decision needing explicit operator policy. v1 warns + skips; the warning includes the suggested template name so the operator can act manually. - Detaching a server when a skill drops it from mcpServers. Detach is destructive (project loses access) and we can't tell whether the detach is intentional vs. accidental drop. PR-7 can revisit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-07 19:12:20 +01:00
Michal	7ebc8b22d1	feat(cli): metadata.hooks materialisation in mcpctl skills sync ... Closes the second-biggest deferred item from PR-5: skills can declare PreToolUse / PostToolUse / SessionStart / Stop / SubagentStop / Notification hooks in metadata.hooks, and `mcpctl skills sync` now registers them in ~/.claude/settings.json with proper source-tagging. ## Why this needs source-tagging mcpctl already managed ONE entry in settings.json (the SessionStart hook for `mcpctl skills sync --quiet`, set up by PR-5 via `installManagedSessionHook`). That entry uses `_mcpctl_managed: true` to identify itself for idempotent updates. Skill-declared hooks need finer scoping: skill A and skill B could both register PreToolUse hooks, and removing skill A must not touch skill B's. So entries written by `applyManagedHooks` carry both: _mcpctl_managed: true ← same flag the SessionStart hook uses _mcpctl_source: "<skill-name>" ← differentiates per-skill Removing a skill (orphan cleanup) drops every entry tagged with that skill's name. User-added entries (no marker) are preserved verbatim. The earlier session-hook installer keeps working: its `_mcpctl_managed: true` lacks the source tag, so the hooks-materialiser ignores it on per-skill operations. ## Behaviour - Skill installs/updates: applyManagedHooks(skillName, metadata.hooks) is called after files are atomically materialised. The function reads settings.json, drops this skill's previous entries from every declared event (and from any event it previously had entries in but no longer declares — so a skill can shrink scope), then re-inserts the new tagged set. - Skill orphan removal: removeManagedHooks(skillName) drops every entry owned by the skill. Other skills + user hooks unaffected. - Failure handling: hooks errors are logged via the warn() callback but do not fail the sync. Same shape as postInstall — scoped, not fatal. ## Files ### Added - src/cli/src/utils/hooks-materialiser.ts (~140 LOC) - src/cli/tests/utils/hooks-materialiser.test.ts (~165 LOC, 11 tests covering: write-from-scratch, multi-source coexistence, user-hook preservation, update replaces (not duplicates), shrink drops events, remove targets only the named source, multiple events independent, idempotent, empty/JSONC settings.json tolerance) ### Edited - src/cli/src/commands/skills.ts: applyOne calls applyManagedHooks when metadata.hooks is set; calls removeManagedHooks(skillName) when a previously-installed skill no longer declares hooks; orphan removal also drops the skill's hooks. New SyncResult field `hooksApplied`. Earlier `meta` declaration deduped (it was redeclared in the postInstall block). ## Verification 164 test files / 2182 tests green (up from 2171). End-to-end on a real machine after this PR ships: ```yaml # skill metadata.yaml hooks: PreToolUse: - type: command command: "audit-pretool.sh" SessionStart: - type: command command: "claude-greeting.sh" ``` ``` mcpctl skills sync # → mcpctl: 1 installed, 1 hooks applied jq '.hooks' ~/.claude/settings.json # → entries tagged with _mcpctl_managed + _mcpctl_source ``` Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-07 19:08:43 +01:00
Michal	d60ad52018	feat(cli): postInstall executor for mcpctl skills sync ... Closes the biggest deferred item from PR-5. metadata.postInstall scripts now actually run when their hash changes, with audit emission back to mcpd. Trust model unchanged from the corporate-appliance design: mcpd is the source of truth, content is reviewed at publish time, the client just runs. No sandbox, no signing, no consent prompts. The controls that matter are already on the publishing side (RBAC, audit, reviewer queue). What we DO provide is ops hygiene: - Hard timeout (default 60 s; per-skill override via metadata.postInstallTimeoutSec). SIGTERM at the deadline, SIGKILL after a 2 s grace. - Hash pinning. The script's sha256 is recorded in ~/.mcpctl/skills-state.json. Re-syncs of an unchanged script are a cheap no-op. A re-published "same version, fixed script" still triggers re-execution because its hash changed. - Curated env. PATH/HOME/USER/SHELL inherited; everything else dropped. MCPCTL_SKILL_NAME / _VERSION / _DIR / _PROJECT injected so scripts have stable context. - Per-skill install log under ~/.mcpctl/skills/<name>/install.log. Bounded at 5 × 256 KB; old entries truncated from the front. - Audit event back to mcpd (POST /api/v1/audit-events, eventKind=skill_postinstall) on every run, including hostname so admins can see fleet rollout state. Best-effort — failures are warned but never block the sync. - Path-escape rejection. metadata.postInstall must resolve inside the skill bundle; relative paths that try to climb out are refused. - Auto-chmod 0755 on the script before spawn. Some upstreams ship 0644 + a shebang and expect a shell to handle it; we always spawn the path directly so we need +x. Failure semantics: on timeout or non-zero exit, the recorded postInstallHash is NOT updated. Next sync retries. Other skills in the same sync run continue regardless — postInstall errors are scoped, not fatal. ## Files ### Added - src/cli/src/utils/postinstall.ts (~200 LOC) - src/cli/tests/utils/postinstall.test.ts (~190 LOC, 10 tests covering success, env vars, chmod, non-zero exit, timeout via exec sleep, path-escape, missing script, log file shape + append-across-runs) ### Edited - src/cli/src/commands/skills.ts: applyOne now invokes runPostInstall + emitPostInstallAudit when metadata.postInstall is set and the script hash differs from prior state. New SyncResult fields: postInstallsRan, postInstallsSkipped. Summary line surfaces "N postInstall ran". --skip-postinstall flag now actually does what it says. ## Verification 163 test files / 2171 tests green (up from 2161). End-to-end on a real machine (after this PR ships and a skill with metadata.postInstall is published): ``` mcpctl skills sync # → mcpctl: 1 installed, 1 postInstall ran cat ~/.mcpctl/skills/<name>/install.log # see stdout/stderr mcpctl skills sync # idempotent — skipped ``` If the same skill is republished with a fixed script: ``` mcpctl skills sync # → mcpctl: 1 updated, 1 postInstall ran (hash changed → rerun) ``` If the script fails (exit 7): ``` mcpctl skills sync # → mcpctl: 1 updated, 1 errors mcpctl skills sync # state.postInstallHash NOT updated → retries ``` Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-07 19:05:42 +01:00
Michal	e6cd73543a	fix(mcpd): fail-loud on env resolution + retry/backoff + readiness via proxy ... Three connected issues with how instances came up + got reported as healthy when their secret backend was unreachable. The motivating case: gitea-mcp-server starts when mcpd can't read the gitea-creds secret from OpenBao, runs with an empty GITEA_ACCESS_TOKEN, replies fine to tools/list (so liveness passes), but every authed call fails with "token is required" — and `mcpctl get instances` cheerfully reports the instance as healthy. ## What changed ### 1. Env resolution failures are now fatal for the start attempt `src/mcpd/src/services/instance.service.ts` The previous behaviour swallowed `resolveServerEnv` failures and let the container start anyway with whatever env survived ("non-fatal — container may still work if env vars are optional"). That's the bug: the gitea container started with no token, ran for weeks, and was reported healthy. The catch now calls `markInstanceError(instance, "secret resolution failed: <reason>")` and returns. Optional/missing env vars should be modelled as `value: ""` entries on the server, not as silent secret-resolution failures. ### 2. ERROR instances retry with backoff, not blind churn Adds Kubernetes-style escalation: 30 s × 5 attempts, then 5 min pauses thereafter. Retry state lives on `McpInstance.metadata` (no schema migration) — `attemptCount`, `lastAttemptAt`, `nextRetryAt`, `error`. The reconciler no longer tears down ERROR instances and creates fresh replacements (which would reset attemptCount and effectively loop at 30 s forever). Instead: - ERROR rows whose `nextRetryAt` is in the future are LEFT ALONE and counted against the replica budget — preventing tight create- fail-create churn while a previous attempt is in its backoff window. - ERROR rows whose `nextRetryAt` has elapsed are retried IN-PLACE via a new `retryInstance` method, which preserves attemptCount on the same row so the schedule actually escalates. The work has been factored into `startOne` (creates + initial attempt) + `attemptStart` (env + container) + `retryInstance` (re-attempt the same row) + `markInstanceError` (write retry metadata). ### 3. STDIO readiness probe goes through mcpProxyService `src/mcpd/src/services/health-probe.service.ts` The legacy `probeStdio` (a `docker exec node -e '... spawn(packageName) ...'` invocation) only worked for packageName-based servers. Image- based STDIO servers like gitea-mcp-server fell through with "No packageName or command for STDIO server" and were reported unhealthy for the WRONG reason — they have no packageName because they are an image, not because anything's wrong. New `probeReadinessViaProxy`: sends `tools/call` through the live running container via `mcpProxyService.execute`. Same code path as production traffic, so probe failures match real failures. Picks up: - JSON-RPC errors (e.g. "token is required" when env is empty). - Tool-level errors expressed as `result.isError: true`. - Connection failures wrapped as exceptions. - Hard timeouts via the deadline race. After this PR, configuring `gitea` with `healthCheck: { tool: get_me, intervalSeconds: 60 }` makes `mcpctl get instances` report it as `unhealthy` whenever the auth token is missing or wrong — which is honest. The dead `probeStdio` (~120 LOC) is removed; HTTP/SSE bespoke probe paths are kept for now (they work and the diff stays minimal). ## Tests `src/mcpd/tests/instance-service.test.ts`: - Replaces "cleans up ERROR instances and creates replacements" with "retries ERROR instances in-place when their backoff has elapsed". - Adds "leaves ERROR instances alone while their nextRetryAt is in the future" and "escalates the backoff: attemptCount + nextRetryAt persist on retry failures". `src/mcpd/tests/services/health-probe.test.ts`: - Swaps STDIO probe mocks from `orchestrator.execInContainer` → `mcpProxyService.execute`. - Adds "marks unhealthy when proxy returns a JSON-RPC error (e.g. broken-secret auth failure)" — explicitly the gitea case. - Adds "marks unhealthy when proxy returns a tool-level error in result.isError" — covers servers that report tool failures as isError instead of as JSON-RPC errors. - Renames "handles exec timeout" → "handles probe timeout" and exercises the deadline race rather than an exec rejection. Full suite: 162 test files / 2161 tests green (+4 new). ## Manual verification step (post-deploy) ```bash mcpctl edit server gitea # → add healthCheck: # tool: get_me # intervalSeconds: 60 # timeoutSeconds: 10 # failureThreshold: 3 ``` If OpenBao is still down: gitea instance enters ERROR with attemptCount + nextRetryAt visible in `mcpctl describe instance`. Otherwise: gitea env resolves at next start, probe passes, instance is honestly healthy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-07 18:55:23 +01:00
Michal	56735a5290	docs: skills + revisions + proposals reference, plus cheatsheet update ... Phase 7 of the Skills + Revisions + Proposals work — documentation pass for the surface added in PR-1 through PR-6. Reference material only; no code changes. ## What's added - `docs/skills.md` — skill model, scoping rules, CLI surface, the `mcpctl config claude --project` setup flow, metadata schema (with the deferred-execution note for hooks/mcpServers/postInstall), the on-disk state file shape, atomic install mechanics, failure semantics, and what's deferred. - `docs/revisions.md` — ResourceRevision model, semver auto-bump rules, contentHash diff key (cross-resource sync), CLI for history / diff / restore, RBAC, audit emission, storage growth note. - `docs/proposals.md` — ResourceProposal model, the reviewer flow (CLI + web UI), atomic-approval mechanics, the propose_prompt / propose_skill MCP tools, the propose-learnings global skill that steers Claude toward engaging with them, and the deferred legacy PromptRequest cutover. ## What's edited - Top-level `CLAUDE.md` — resource cheatsheet adds `skill`, `proposal`, `revision` with cross-references to the new docs. The legacy `promptrequest` entry stays (still on the legacy code path) but notes that new work should use `proposal`. ## What's NOT in this PR - The PromptRequest → ResourceProposal cutover migration. Both run side-by-side today; the focused cutover PR will rename + backfill + drop. Keeping that out of PR-7 means review can stay on docs. - Bundle-backup / `mcpctl apply -f` skill support (deferred from PR-3). - `metadata.hooks` / `metadata.mcpServers` / `metadata.postInstall` execution (deferred from PR-5). - Existing-page UI migration to Tailwind (deferred from PR-6 — old inline-styled pages coexist fine inside the new Layout). These are tracked as future PRs; each is its own focused change. ## Verification `pnpm test:run` whole monorepo: 162 test files / 2157 tests green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-07 17:58:04 +01:00
Michal	e8c3803fac	feat(web): bold redesign — Tailwind v4 + shadcn-style primitives + Skills/Proposals/Revisions UI ... Phase 6 of the Skills + Revisions + Proposals work. The web UI gets a new design language and first-class affordances for everything the backend now supports. ## Visual direction - Tailwind v4 with custom @theme block (oklch tokens). Dark-mode-only (internal tool — light mode doubles QA surface). - Inter for UI, JetBrains Mono for code/IDs (loaded via Google Fonts; trivial to swap for self-hosted geist later — the fallback stack reads identically). - Sidebar layout (always-visible at desktop widths) replacing the previous top-bar nav. Pending-proposals badge polls every 30 s so reviewers see a queue building without refreshing. - Lucide icons throughout. - Spacing and radii on Tailwind defaults. Existing inline-styled pages (Projects, Agents, AgentDetail, ProjectPrompts, PersonalityDetail, Login) continue to work unchanged inside the new Layout — Tailwind doesn't conflict with their inline styles. A follow-up can migrate them incrementally. ## What's added ### Build infra (src/web/) - package.json: tailwindcss@^4 + @tailwindcss/vite, lucide-react, class-variance-authority, clsx, tailwind-merge, diff, geist (held for future self-hosting). - vite.config.ts: registers the @tailwindcss/vite plugin. - src/index.css: Tailwind import + @theme tokens + @layer base. - src/main.tsx: imports index.css. - src/lib/utils.ts: shadcn-style cn() helper. ### shadcn-style primitives (src/components/ui/) Hand-written rather than generated via `npx shadcn` so the repo doesn't depend on a CLI tool that needs interactive runtime: - button.tsx — variants: primary / secondary / ghost / danger / link; sizes: sm / md / lg / icon. - card.tsx — Card + Header/Title/Description/Content/Footer subparts. - badge.tsx — variants: default / info / success / warning / danger / outline. - input.tsx — Input + Textarea + Label. - tabs.tsx — no-dep accessible Tabs (no Radix needed for our use). - separator.tsx — h/v separator with role=separator. ### Diff component (src/components/Diff.tsx) Wraps the `diff` package (already added in PR-2) for inline unified- diff display with color-coded add/remove rows. Used by both the proposal review page and the skill revision-history tab. ### New pages (src/pages/) - Dashboard.tsx — at-a-glance home. Counts for skills, prompts, projects, agents, proposals; pending-proposals call-out card if any. - Skills.tsx — list view, separated into Global vs Project/Agent- scoped sections. - SkillDetail.tsx — name + semver + description; tabs for SKILL.md / Files / Metadata / History. History tab shows revisions with click-to-diff against the live body. - Proposals.tsx — queue with Pending/Approved/Rejected tabs. Pending count is highlighted in amber. - ProposalDetail.tsx — full body, diff against current resource (or "would create new" if it doesn't exist), approve button + reject- with-required-note flow. ### usePolling hook (src/hooks/) Tiny polling-with-cancellation hook used by Layout and Proposals. ### Layout rewrite (src/components/Layout.tsx) Sidebar with nav items: Dashboard, Projects, Agents, Skills, Proposals. Lucide icons. Active-route highlighting via NavLink. Pending-proposals warning badge on the Proposals item. ### Routes (src/App.tsx) New routes: /dashboard, /skills, /skills/:name, /proposals, /proposals/:id. Default redirects to /dashboard. ### API types (src/api.ts) Type defs for Skill, VisibleSkill, Proposal, Revision (with the shapes the new pages consume). ## Tests Existing 7 web tests still pass (Login + api). New page-level tests deferred — the new pages are mostly compositions of primitives and fetch hooks that round-trip to the backend; the backend tests already cover what they call. PR-7 polish can add render-and-click tests if coverage drift surfaces. ## Verification - `pnpm --filter @mcpctl/web build` clean, no warnings. - `pnpm test:run` whole monorepo: 162 test files / 2157 tests green. - Visual smoke deferred — needs a running mcpd to populate the fixtures. Manual smoke tested locally is the next step. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-07 17:54:55 +01:00
Michal	58e8e956ce	feat(cli+mcpd): mcpctl skills sync + config claude extension ... Phase 5 of the Skills + Revisions + Proposals work. Skills are now materialised onto disk under ~/.claude/skills/<name>/, with hash-pinned diff against mcpd, atomic per-skill install, and preservation of locally-modified files. `mcpctl config claude --project X` now wires the full pickup chain: writes .mcpctl-project marker, runs the initial sync, installs the SessionStart hook so subsequent Claude invocations stay in sync transparently. ## Sync algorithm 1. Resolve project: `--project` flag overrides; else walk up from cwd looking for `.mcpctl-project`; else fall back to globals-only. 2. GET /api/v1/projects/:name/skills/visible (or /api/v1/skills?scope=global without a project). Server returns id + name + semver + scope + contentHash + metadata — no body, no files. The contentHash is sha256 of the canonicalised body, computed server-side; any reordering of keys produces the same hash, so it's a stable diff key. 3. Load ~/.mcpctl/skills-state.json (lives outside ~/.claude/skills/ on purpose — Claude Code reads that tree and we don't want to pollute it with our bookkeeping). 4. Diff: - server skill not in state → INSTALL - server skill, state contentHash matches → SKIP (cheap path) - server skill, state contentHash differs → UPDATE (fetch full body) - state skill not in server → orphan, REMOVE (preserve if locally modified, unless --force) 5. Atomic per-skill install: write to <targetDir>.mcpctl-staging-<pid>/, rename existing tree to .mcpctl-trash-<pid>, swap staging in, rmtree the trash. A concurrent reader (Claude Code starting up) never sees a partial tree. 6. State file updated with new versions, per-file SHA-256, install path. saveState is atomic (temp + rename). ## Failure semantics - `--quiet` mode (used by SessionStart hook): exit 0 on network / timeout / mcpd error. Fail-open is non-negotiable here — we never want a hung mcpd to block Claude Code starting up. - Auth failure: exit 1, clear "run mcpctl login" message. - Disk error during state save: exit 2. - Per-skill errors are collected in the result and reported as a count; one bad skill doesn't stop the others. Network fetches run with concurrency 5. The server-side `/visible` endpoint is metadata-only so the cheap path (everything unchanged) needs exactly one HTTP roundtrip total. ## Files added ### CLI utilities (src/cli/src/utils/) - skills-state.ts — load/save state, per-file sha256, edit detection. - project-marker.ts — walk-up to find `.mcpctl-project`, bounded by user home so we never search above $HOME. - sessionhook.ts — install/remove a SessionStart hook entry tagged with `_mcpctl_managed: true`. Idempotent. Defensive against missing/empty/JSONC settings.json. - skills-disk.ts — atomic install via staging-dir rename swap, symmetric atomic delete via trash-dir rename. Path-escape attempts in files{} are rejected. ### CLI command (src/cli/src/commands/) - skills.ts — `mcpctl skills sync` Commander wrapper + the `runSkillsSync(opts, deps)` library function (also called from `mcpctl config claude --project`). Supports `--dry-run`, `--force`, `--quiet`, `--keep-orphans`. `--skip-postinstall` is reserved (postInstall execution lands in a follow-up PR, not this one). ### Wiring - index.ts: registers `mcpctl skills` after `mcpctl review`. - config.ts: `mcpctl config claude --project X` now writes the `.mcpctl-project` marker, runs `runSkillsSync` in-process, and calls `installManagedSessionHook('mcpctl skills sync --quiet')`. New flag `--skip-skills` opts out (used by tests; useful for CI). ## Server-side change - src/mcpd/src/services/skill.service.ts: getVisibleSkills now computes contentHash on the fly from the canonical body shape the client will reconstruct. Cheap (sha256 of ~few KB per skill); no schema migration needed since hash is derived not stored. ## Tests Four new utility test files (31 tests) under src/cli/tests/utils/: - sessionhook.test.ts — creation, idempotency, command updates, preservation of user hooks, removal, empty/JSONC tolerance. - skills-disk.test.ts — atomic write, replacement without leftovers, path-escape rejection, atomic delete, listing ignores staging/trash artifacts. - skills-state.test.ts — sha256 determinism, state round-trip, schema-version drift handling, edit detection. - project-marker.test.ts — cwd hit, walk-up, $HOME boundary, empty marker, write+read round-trip. The existing `mcpctl config claude` test (claude.test.ts) was updated to pass `--skip-skills` so it stays focused on .mcp.json generation; the new sync flow is covered by the utility tests. Full suite: 162 test files / 2157 tests green (up from 158 / 2127). ## Deferred to a follow-up - `metadata.hooks` materialisation into `~/.claude/settings.json` — the data path exists, sync receives it; PR-7 or a focused follow-up will write the `_mcpctl_managed: true` entries for declarative hooks. - `metadata.mcpServers` auto-attach via mcpd API — likewise. - `metadata.postInstall` script execution — the most substantive deferred piece. Current sync logs a TODO and skips. The corporate trust model (publisher-side rigor, not client-side defence) means this is straightforward to add once we wire the curated env + timeout + audit emission. Orthogonal to file sync, easier to ship separately. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-07 16:26:35 +01:00
Michal	db57bb5856	feat(mcpd+mcplocal+cli): propose-learnings system skill, propose_skill MCP tool, mcpctl review ... Phase 4 of the Skills + Revisions + Proposals work. Closes the reflexive loop: Claude sessions can now propose back content (prompts or skills) that maintainers triage via a CLI queue. The system documents itself to Claude through the same mechanism it documents to humans. ## What's added ### propose-learnings global skill (mcpd bootstrap) - src/mcpd/src/bootstrap/system-skills.ts — idempotent upsert, mirrors system-project.ts. Single skill seeded today: `propose-learnings`, ~430 words, explains when to engage with propose_prompt vs propose_skill, what makes a good proposal, what NOT to propose, and the review→approve flow. Priority 9, global scope. - main.ts: `bootstrapSystemSkills(prisma)` called right after `bootstrapSystemProject`. ### gate-encouragement-propose system prompt - system-project.ts gains a new gate prompt (priority 10, alongside the other gate-* prompts) that nudges Claude to call propose_prompt when it discovers a project-specific lesson. Pairs with the propose-learnings skill — the prompt is the trigger, the skill is the manual. ### propose_skill MCP tool (mcplocal) - proxymodel/plugins/gate.ts: new virtual tool registered alongside propose_prompt. Posts to /api/v1/proposals (the new endpoint from PR-2) with resourceType='skill'. Tool description steers Claude toward propose_prompt for project-specific knowledge and reserves propose_skill for cross-cutting cases. propose_prompt's tool description is also expanded to point at the propose-learnings skill for guidance — the bare "creates a pending request" copy was bland enough that nothing in Claude's prior would actually make it engage. ### mcpctl review CLI - New top-level command in src/cli/src/commands/review.ts. Subcommands: mcpctl review pending List pending proposals mcpctl review next Show oldest pending mcpctl review show <id> Full detail mcpctl review approve <id> POST /proposals/:id/approve mcpctl review reject <id> --reason "..." mcpctl review diff <id> Side-by-side current vs proposed - Wired into src/cli/src/index.ts. Registered after createApproveCommand to keep the existing project-ops `mcpctl approve promptrequest` command working (legacy) while the new review surface is the preferred path. ## Tests touched - bootstrap-system-project.test.ts already counts via getSystemPromptNames() length, so it picked up the new prompt automatically; only the priority assertion needed nothing — the new prompt starts with `gate-` so the existing `gate-* → priority 10` invariant validates it. - system-prompt-validation.test.ts: bumped expected length from 11→12 and added a `toContain('gate-encouragement-propose')` assertion. Full suite: 158 test files / 2127 tests green. ## What's NOT in this PR - A SkillService mock-based test for the proposal approval handler — the PromptService approval handler is structurally identical and already covered; the database-backed integration is exercised in PR-2's tests. - Changes to mcplocal's existing handleProposePrompt URL — it still POSTs to the legacy /api/v1/projects/.../promptrequests endpoint, which works because PR-2 left that route in place. PR-7 will cut mcplocal over to /api/v1/proposals along with the PromptRequest table rename + drop. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-07 13:13:33 +01:00
Michal	20a541a5d6	feat(mcpd): Skill resource end-to-end (CRUD + backup + revision integration) ... Phase 3 of the Skills + Revisions + Proposals work. Skills get the same inline-content + revision-history shape as prompts, with the addition of `files` (multi-file bundles, materialised by `mcpctl skills sync` in PR-5) and a typed `metadata` Json (hooks, mcpServers, postInstall, …). ## What's added ### Validation (src/mcpd/src/validation/skill.schema.ts) Typed metadata schema with a closed list of recognised hook events (PreToolUse, PostToolUse, SessionStart, Stop, SubagentStop, Notification), typed `mcpServers` dependency declarations (name + fromTemplate + optional project), and `postInstall` / `preUninstall` paths into the bundle's `files{}`. `.passthrough()` so unknown fields survive — forward-compat for follow-on additions. ### Repository (src/mcpd/src/repositories/skill.repository.ts) Mirrors PromptRepository exactly. Same `?? ''` workaround for nullable-FK compound-key lookups. ### Service (src/mcpd/src/services/skill.service.ts) Mirrors PromptService for create / update / delete / restore / upsert, including: - Auto-bump patch on content/files/metadata change. - Revision recording (best-effort — failures don't block the save). - 'skill' approval handler registered with ResourceProposalService so proposalService.approve dispatches to skills the same way it dispatches to prompts. - `getVisibleSkills(projectId)` returns id + name + semver + scope + metadata for `mcpctl skills sync` (PR-5) to diff against on-disk state. ### Routes (src/mcpd/src/routes/skills.ts) - GET /api/v1/skills (filters: ?project= ?projectId= ?agent= ?scope=global) - GET /api/v1/skills/:id - POST /api/v1/skills - PUT /api/v1/skills/:id - DELETE /api/v1/skills/:id - GET /api/v1/projects/:name/skills - GET /api/v1/projects/:name/skills/visible — sync diffing - GET /api/v1/agents/:name/skills - POST /api/v1/skills/:id/restore-revision { revisionId, note? } ### main.ts SkillRepository + SkillService instantiated; revision/proposal services wired in. `skills` segment added to the RBAC permission map (uses the existing `prompts` permission for now — same trust shape) and to `kindFromSegment` so the git-backup hook captures skill mutations. ### Backup integration - yaml-serializer.ts: `BackupKind` adds 'skill'; APPLY_ORDER bumps to 9 with skill last (it depends on projects/agents). `parseResourcePath` recognises the `skills/` directory. - git-backup.service.ts: `serializeResource` adds the `case 'skill'` branch alongside prompts. The git-sync loop now round-trips skills on every change. - (Bundle backup-service.ts is NOT updated in this PR — deferred to PR-7 alongside the cutover. The git-based backup IS wired, which is the primary persistence path.) ### CLI - `mcpctl create skill <name>` with --content / --content-file, --description, --priority, --semver, --metadata-file (YAML/JSON), --files-dir (walks a directory tree into `files{}`, UTF-8 only; null bytes rejected). - shared.ts adds `skill` / `skills` / `sk` aliases. ### apply.ts Not updated — `mcpctl apply -f skill.yaml` is deferred to PR-7. The existing CRUD endpoints + `mcpctl create skill` cover the bootstrap need; bulk-apply will arrive with the `propose-learnings` seed and docs. ## Tests 158 test files / 2127 tests green across the workspace. The DB-level schema tests for Skill landed in PR-1; the new service-level integration is exercised through main.ts wiring + the existing prompt revision tests (skill follows the same code path through proposal service approval). A `describe('Skill service mocks')` test file deliberately not added — the PromptService mock-based tests already cover the revision/approval handler shape, and the skill handler is structurally identical (same upsert + record-revision + link-currentRevisionId pattern). PR-7 will add an integration test that walks the full propose → review → approve flow for both resource types. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-07 00:48:40 +01:00
Michal	1ec286bb14	feat(mcpd): ResourceRevision + ResourceProposal services + Prompt revision integration ... Phase 2 of the Skills + Revisions + Proposals work. Stands up the generic revision/proposal layer and wires Prompt into it. Skills will plug into the same infrastructure in PR-3 with no further service changes required. This PR is intentionally additive: PromptRequest table and routes are unchanged. The /api/v1/proposals API runs side-by-side with the legacy /api/v1/promptrequests API. The PromptRequest cutover (rename + backfill + mcplocal rewire) is deferred to a later PR so this one stays reviewable. ## What's added ### Repositories (src/mcpd/src/repositories/) - resource-revision.repository.ts — append-only revision log keyed by (resourceType, resourceId). Soft FK; no relations declared. Supports history listing, semver lookup, and contentHash cross-resource search. - resource-proposal.repository.ts — generic propose queue. Status lifecycle pending → approved | rejected. Mirrors Prompt's `?? ''` workaround for nullable-FK compound lookups. ### Services (src/mcpd/src/services/) - resource-revision.service.ts — record() inserts a revision with a stable sha256 contentHash computed from canonicalised JSON (key-sorted at every level so reordered objects produce the same hash). Caller passes a pre-computed semver; service does NOT decide bump policy. - resource-proposal.service.ts — propose / approve / reject / list, with a per-resourceType handler registry. PromptService registers the 'prompt' handler at construction; the SkillService will register 'skill' in PR-3. approve() runs in a Prisma $transaction so the resource update + revision insert + proposal status flip are atomic. ### Pure utility (src/mcpd/src/utils/semver.ts) - bumpSemver(current, kind) for major / minor / patch - compareSemver(a, b) — numeric, not lex (10 > 9) - isValidSemver(s) - Invalid input falls back to '0.1.0' rather than throwing — keeps the audit-write path from blowing up the prompt update if a row's semver ever drifts out of MAJOR.MINOR.PATCH shape. ### Routes (src/mcpd/src/routes/) - revisions.ts — GET /api/v1/revisions?resourceType=&resourceId=, GET /api/v1/revisions/:id, GET /api/v1/revisions/:id/diff?against=<id|live> (unified-format diff via the `diff` package), and POST /api/v1/prompts/:id/restore-revision { revisionId, note? }. - proposals.ts — GET / POST /api/v1/proposals, GET /api/v1/proposals/:id, PUT for body updates, POST .../approve and POST .../reject, plus DELETE. ## What's changed - PromptService.create / update now record a ResourceRevision when the revision service is wired. Update auto-bumps patch on content change; authors can override via `--bump major|minor|patch` or `--semver X.Y.Z` on the CLI (forwarded into the PUT body). Best-effort: revision write failures are swallowed so the prompt save still succeeds (revision is audit, not source of truth). - PromptService.setProposalService registers a 'prompt' approval handler with the proposal service. Approval runs in a Prisma transaction: upsert prompt → record revision → update currentRevisionId → flip proposal status. semver bumps to 0.1.0 on first approval, patch thereafter. - New CLI flags on `mcpctl edit prompt`: --bump, --semver, --note. They're prompt-only (validated client-side); other resources reject them. - Aliases in shared.ts: `proposal`/`prop` → proposals, `revision`/`rev` → revisions. - diff dependency added to mcpd. ## Tests - src/mcpd/tests/utils/semver.test.ts — covers bump/compare/validate including numeric (not lex) semver compare and invalid-input fallback. - prompt-service.test.ts updated: makePrompt fixture now sets semver + agentId + currentRevisionId; updatePrompt assertion expects the auto-bumped patch in the same update call. - prompt-routes.test.ts updated symmetrically. ## RBAC `proposals` and `revisions` URL segments map to the existing `prompts` permission for now. PR-7 may split if a "reviewer" role becomes useful. ## Verification Full suite: 158 test files / 2127 tests green. `pnpm build` clean across all 6 workspace packages. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-07 00:38:35 +01:00
Michal	fbe68fa693	feat(db): schema for ResourceRevision, ResourceProposal, Skill ... Phase 1 of the Skills + Revisions + Proposals work. Purely additive — no existing rows are touched, no tables renamed, no columns dropped. New tables: - ResourceRevision — append-only audit + diff log keyed by (resourceType, resourceId). Both Prompt and Skill produce revisions on every change. Soft FK so revisions outlive the resources they describe. Indexed for history viewer (latest-first), semver lookup, and cross-resource sync diff via contentHash. - ResourceProposal — generic propose/approve/reject queue. Drop-in replacement for the prompt-only PromptRequest. Created empty here; PR-2 will rename PromptRequest → _PromptRequest_legacy and backfill. - Skill — new resource type that mirrors Prompt for everything CRUD- shaped. Adds `files` Json (multi-file bundles, materialised onto disk by `mcpctl skills sync` in PR-5) and `metadata` Json (typed app-layer in PR-3: hooks, mcpServers, postInstall, …). New columns on Prompt: - semver (semver string, default '0.1.0') — auto-bumped patch on save by PromptService.update once PR-2 wires it. Distinct from `version`, which stays as the optimistic-concurrency counter. - currentRevisionId — soft pointer to the latest ResourceRevision row. DB tests cover scope rules (project XOR agent XOR neither), name uniqueness across both compound keys, cascade-on-delete, soft-FK survival of deletion, and JSON column persistence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-07 00:18:21 +01:00
Michal	2b2444a2c5	docs+smoke(v7): visibility section in virtual-llms.md + register/list smoke ... Wraps up v7 Stage 3: - docs/virtual-llms.md gains a "Visibility scope (v7)" section that explains public-vs-private semantics, who skips the filter (owner + `*` admin), how to grant single-row exceptions via name-scoped RBAC, per-row override syntax in mcplocal config, the `--visibility` flag on `mcpctl create llm`/`create agent`, and YAML round-trip behavior. - New smoke (virtual-llm-visibility.smoke.test.ts) publishes one public + one private virtual Llm via the registrar against the live mcpd and asserts the GET /llms response carries visibility + a non-empty ownerId for both, and that GET /llms/<name> returns the private row to its owner without 404. Cross-user filtering is covered by mcpd's visibility-filter unit tests; smoke proves the fields make the round-trip end-to-end. Will pass once mcpd is rebuilt + deployed via fulldeploy.sh on this branch (current main is v6, doesn't yet serialize visibility). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-29 01:08:03 +01:00
Michal	2c98a21323	feat(mcpd+cli+mcplocal): wire visibility filter through routes, CLI, registrar (v7 Stage 2) ... Stage 1 added the schema + service predicate. This stage threads the filter through every surface that lists or fetches Llms/Agents: - mcpd routes: viewerFromRequest helper builds a Viewer from the request's RBAC scope. List endpoints rely on the existing preSerialization hook (now two-phase: name-scope first, visibility second). get-by-id/get-by-name routes pass the viewer to the service which 404s on hidden rows. - RBAC: AllowedScope gains `isAdmin` to distinguish a `*` cross-resource grant (admins skip visibility) from a plain `view:llms` grant (wildcard for RBAC, but visibility still applies). FastifyRequest augmentation updated. - VirtualLlmService.register accepts ownerId and stamps it on freshly created virtual rows; defaults visibility to 'private' on first create, leaves existing rows untouched on sticky reconnect. - AgentService.registerVirtualAgents mirrors the same defaults. - mcplocal: LlmProviderFileEntry / AgentFileEntry / RegistrarPublishedX carry visibility through to the register payload (default 'private'). - CLI: VISIBILITY column on `mcpctl get llm` and `mcpctl get agent`, `--visibility` flag on `mcpctl create llm` / `create agent`. YAML round-trip works because visibility passes through stripInternalFields unchanged (ownerId is already stripped). Completions regenerated. Tests: mcpd 908/908, mcplocal 731/731, cli 437/437. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-29 01:03:58 +01:00
Michal	21f8bede2e	feat(mcpd+db): visibility scope + ownership for Llms and Agents (v7 Stage 1) ... Adds the schema + service-layer machinery for per-user RBAC scoping of virtual Llms and Agents. Without this, anyone with `view:llms` sees every other user's published model — fine for a single-user homelab, wrong for org use where workstation-published models or paid keys aren't meant to be broadcast. Schema: - Llm: new `ownerId String?` + `visibility String @default("public")`. NULL ownerId on legacy rows is treated as public for back-compat. - Agent: `visibility String @default("public")` (Agent already has `ownerId`, required). - Composite index `(visibility, ownerId)` on both tables for the list-filter hot path. - Migration backfills both columns to 'public' so pre-v7 setups behave identically post-deploy. Service layer: - New `Viewer` / `AgentViewer` shape: `{ userId, wildcard, allowedNames }`. The route layer computes this from `request.userId` + `RbacService.getAllowedScope` and passes it down. NULL viewer = skip the filter (internal callers — cron sweeps, audit, tests). - `isLlmVisibleTo` / `isAgentVisibleTo` pure predicates encode the decision tree: visibility=public → visible (RBAC layer above already passed) viewer=null OR wildcard → visible ownerId === viewer.userId → visible row.name in viewer.allowedNames → visible else → hidden - LlmService.list/getById/getByName + AgentService equivalents accept an optional Viewer arg and apply the predicate. Get-style methods 404 (not 403) on hidden rows so name enumeration via differential status is impossible. Repositories: CreateInput/UpdateInput types gained `ownerId`/ `visibility` (Llm) and `visibility` (Agent). Update is in place; ownerId is set-once at create time. Tests: - 13 unit tests on the predicate covering every branch (null viewer, public, wildcard, owner, name-scoped grant, foreign private, legacy null-ownerId). - mcpd 908/908 (was 893; +15 across the merge windows + this PR). Stage 2 (next): route plumbing — every list/get endpoint needs to build the Viewer from the request and pass it through. mcplocal virtuals default to visibility=private on register. CLI adds a VISIBILITY column and a --visibility flag. yaml round-trip preserves the field.	2026-04-29 00:46:06 +01:00