mcpctl/docs/mcptoken-implementation.md

mcptoken + HTTP-mode mcplocal — implementation log

Companion to the approved plan at /home/michal/.claude/plans/lets-discuss-something-i-bright-lovelace.md. This file is updated as each milestone lands, so you can review what was actually done vs. what was planned.

Context (why)

You're running your own vLLM inference outside Claude Code and want it to consume mcpctl over MCP with the same UX Claude gets: project-scoped server discovery, proxy models, the pipeline cache. Today mcplocal is systemd-only and serves STDIO — unreachable from off-host and unauthenticated. This work adds:

1. A containerized, network-accessible mcplocal serving Streamable HTTP.
2. A new McpToken resource (CLI: mcpctl get/create/delete mcptoken) — project-scoped bearer tokens with the same RBAC stack as users. Hashed at rest; raw value shown once.
3. Tokens as a first-class RBAC subject kind (McpToken:<sha>), with a creator-permission ceiling so non-admins cannot mint escalated tokens.
4. k8s deploy (Service mcp, Ingress mcp.ad.itaz.eu, PVC-backed FileCache).
5. A CLI breaking change: mcpctl create rbac --binding edit:servers → --roleBindings role:edit,resource:servers. You explicitly asked for this; only one command uses it.
6. A product-grade mcpctl test mcp <url> verb for validating any Streamable-HTTP MCP endpoint, reused by smoke tests.

Branch

All work lives on feat/mcptoken (off main at 3149ea3).

Pre-work committed to main (outside this branch)

Before starting the feature, we flushed your in-flight changes to main so they wouldn't travel with the branch:

• 3149ea3 fix: MCP proxy resilience — discovery cache, default liveness probes — per-server tools/list cache in McpRouter with positive+negative TTL so dead upstreams only stall the first call; default liveness probe (tools/list through the real production path) applied to any RUNNING instance without an explicit healthCheck. Already pushed to origin.

Status legend

• ✅ done
• 🚧 in progress
• ⬜ not started

PR 1 — Schema + token helpers + mcpd CRUD routes ✅

#	Step	Status
1	McpToken Prisma model + Project/User reverse relations; AuditEvent.tokenName / tokenSha + index	✅
2	src/shared/src/tokens/index.ts — generateToken, hashToken, isMcpToken, timingSafeEqualHex, TOKEN_PREFIX	✅
3	src/mcpd/src/repositories/mcp-token.repository.ts + new interfaces in repositories/interfaces.ts	✅
4	src/mcpd/src/services/mcp-token.service.ts — creator-ceiling via rbacService.canAccess/canRunOperation, raw token returned only once, auto-creates an RbacDefinition with subject McpToken:<sha> when bindings are non-empty	✅
5	src/mcpd/src/routes/mcp-tokens.ts — POST / GET / GET:id / DELETE:id + POST:id/revoke + GET /introspect	✅
6	Wired into main.ts — repo/service constructed, routes registered, mcptokens added to URL→permission map + name resolver; /mcptokens/introspect added to auth-skip list so mcplocal can call it with a raw McpToken bearer	✅
7	RBAC extensions: new subject kind McpToken in rbac-definition.schema.ts; mcptokens added to RBAC_RESOURCES and RESOURCE_ALIASES; rbac.service.ts threads optional mcpTokenSha through canAccess, canRunOperation, getAllowedScope, getPermissions; resolver matches {kind:'McpToken', name: sha}	✅
8	Unit tests — tests/mcp-token-service.test.ts covering: empty/clone modes, ceiling rejection, RbacDefinition auto-create with correct McpToken:<sha> subject, duplicate-name conflict, introspect valid/revoked/expired/unknown, revoke deletes the RbacDefinition. 11/11 green. Full mcpd suite still 648/648.	✅

What this PR does NOT do yet (coming in PR 3)

• The mcpd auth middleware does not yet dispatch on the token prefix. A raw mcpctl_pat_… bearer sent to any /api/v1/* endpoint (other than /introspect) is still rejected as an invalid session. That's intentional — PR 3 extends middleware/auth.ts to recognize both session bearers and McpToken bearers.
• No CLI yet. Tokens can be created only via POST /api/v1/mcptokens for now.

PR 2 — RBAC CLI migration ✅

Migrated mcpctl create rbac from positional flag syntax to the key=value form you asked for.

Before:

mcpctl create rbac developers \
  --subject User:alice@test.com \
  --binding edit:servers \
  --binding view:servers:my-ha \
  --operation logs

After:

mcpctl create rbac developers \
  --subject User:alice@test.com \
  --roleBindings role:edit,resource:servers \
  --roleBindings role:view,resource:servers,name:my-ha \
  --roleBindings action:logs

#	Step	Status
1	New shared parser at src/cli/src/commands/rbac-bindings.ts exporting parseRoleBinding(entry)	✅
2	src/cli/src/commands/create.ts — old --binding/--operation flags replaced with one repeatable --roleBindings <kv>. Uses the new parser.	✅
3	Tests in src/cli/tests/commands/create.test.ts rewritten to the new form (8 RBAC tests updated)	✅
4	New dedicated unit test src/cli/tests/commands/rbac-bindings.test.ts — 9 cases covering unscoped / name-scoped / action / trim / empty-value / unknown-key / action-conflict / missing-role rejections	✅
5	Shell completions regenerated via pnpm completions:generate — both completions/mcpctl.{bash,fish} now offer --roleBindings, no longer --binding/--operation	✅
6	Nothing in docs/ or README.md referenced the old flags	✅

Full CLI suite still 406/406 green. On-disk YAML shape (roleBindings: [...]) is unchanged, so backups and existing apply -f files keep working.

The extracted parseRoleBinding helper is what PR 3's mcpctl create mcptoken --bind <kv> flag will reuse.

PR 3 — CLI mcptoken verbs + mcpd auth dispatch + audit ✅

#	Step	Status
1	src/mcpd/src/middleware/auth.ts — dispatch on the bearer prefix. mcpctl_pat_… → new findMcpToken(hash) dep → populates request.mcpToken + request.userId = ownerId. Other bearers → existing findSession path. Returns 401 for revoked, expired, or unknown tokens. Fastify module augmentation adds request.mcpToken?: McpTokenPrincipal.	✅
2	src/mcpd/src/main.ts — wires findMcpToken: mcpTokenRepo.findByHash. Threads mcpTokenSha into canAccess / canRunOperation / getAllowedScope. Adds a second project-scope check: McpToken principals can only reach resources inside their bound project (additional guard on top of the route handler checks).	✅
3	New auth tests (tests/auth.test.ts) — 3 McpToken dispatch cases: happy path sets userId + mcpToken, revoked → 401, no findMcpToken wired → 401. Session path unchanged.	✅
4	mcpctl create mcptoken <name> -p <proj> [--rbac empty|clone] [--bind …] [--ttl …] — new subcommand. Reuses parseRoleBinding from PR 2. parseTtl helper accepts 30d/12h/never/ISO8601. --force revokes the existing active token and creates a new one. Raw token is printed once with a "copy now" banner.	✅
5	mcpctl get mcptokens + mcpctl get mcptoken <name> -p <proj> + mcpctl describe mcptoken <name> -p <proj> + mcpctl delete mcptoken <name> -p <proj>. Names are project-scoped, so all verbs require -p unless a CUID is passed. Table columns: NAME / PROJECT / PREFIX / CREATED / LAST USED / EXPIRES / STATUS. Describe surfaces the auto-created RbacDefinition's bindings (matched by mcptoken-<id> name convention).	✅
6	mcpctl apply -f — added McpTokenSpecSchema, mcpton: 'mcptokens' in KIND_TO_RESOURCE, and an applier that creates if missing or logs "already active — skipped" (tokens are immutable). Raw token printed on create.	✅
7	Resource aliases — mcptoken/mcptokens/token/tokens all resolve to mcptokens. stripInternalFields scrubs the secret and derived fields and promotes projectName → project for YAML round-trip.	✅
8	Audit pipeline — src/mcplocal/src/audit/types.ts gains tokenName?/tokenSha?; collector gets setSessionMcpToken(sessionId, {tokenName, tokenSha}) alongside setSessionUserName, both merged into a per-session principal map. src/mcpd/src/services/audit-event.service.ts accepts tokenName and tokenSha query params (repo already extended in PR 1). console/audit-types.ts carries the new optional fields so the TUI can surface them in a follow-up.	✅
9	Shell completions regenerated — mcpctl create mcptoken flags (--project, --rbac, --bind, --ttl, --description, --force) and the new resource alias land in both bash and fish completions. completions.test.ts freshness check passes.	✅

What this PR does NOT do yet (coming in PR 4)

• No HTTP-mode mcplocal binary yet. Tokens can be used to hit mcpd directly via /api/v1/… with Authorization: Bearer mcpctl_pat_…, but the containerized /projects/<p>/mcp endpoint and its token-auth preHandler don't exist yet.
• The audit-console TUI still shows only userName columns; adding a TOKEN column is a UI polish follow-up.

Test stats

• 1764/1764 tests pass workspace-wide (up from ~1750 before PR 3).
• Build clean across all 5 packages.
• Completions freshness check green.

PR 4 — HTTP-mode mcplocal + container + mcpctl test mcp + smoke

(blocked)

Design decisions recap (so you don't have to re-read the plan)

Decision	Choice
Transport	Streamable HTTP only
Binary shape	Same @mcpctl/mcplocal package, two entry files (main.ts STDIO, serve.ts HTTP)
Container runtime	Node (not bun-compiled) — mirrors mcpd
Cache	PVC at /var/lib/mcplocal/cache
Hostname	k8s Service mcp, Ingress mcp.ad.itaz.eu
Token format	mcpctl_pat_<32-byte base62>, stored as SHA-256, shown-once at create
Resource	McpToken, CLI noun mcptoken, one-project-per-token, FK cascade
Subject kind	New McpToken:<sha>
TTL	No default. Optional --ttl 30d / never / ISO date
Default bindings	--rbac=empty (default), --rbac=clone, --bind <kv> — creator ceiling enforced server-side
Binding CLI	--roleBindings role:view,resource:servers[,name:foo] or --roleBindings action:logs
Project enforcement	Endpoint visibility only (no strict create-time check) — same mechanism Claude uses