Files
mcpctl/src/db/prisma/schema.prisma

890 lines
32 KiB
Plaintext
Raw Normal View History

generator client {
provider = "prisma-client-js"
}
datasource db {
provider = "postgresql"
url = env("DATABASE_URL")
}
// ── Users ──
model User {
id String @id @default(cuid())
email String @unique
name String?
passwordHash String
role Role @default(USER)
provider String?
externalId String?
version Int @default(1)
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
sessions Session[]
auditLogs AuditLog[]
ownedProjects Project[]
groupMemberships GroupMember[]
mcpTokens McpToken[]
ownedAgents Agent[]
chatThreads ChatThread[]
@@index([email])
}
enum Role {
USER
ADMIN
}
// ── Sessions ──
model Session {
id String @id @default(cuid())
token String @unique
userId String
expiresAt DateTime
createdAt DateTime @default(now())
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
@@index([token])
@@index([userId])
@@index([expiresAt])
}
// ── MCP Servers ──
model McpServer {
id String @id @default(cuid())
name String @unique
description String @default("")
packageName String?
runtime String?
dockerImage String?
transport Transport @default(STDIO)
repositoryUrl String?
externalUrl String?
command Json?
containerPort Int?
replicas Int @default(1)
env Json @default("[]")
healthCheck Json?
version Int @default(1)
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
templateName String?
templateVersion String?
instances McpInstance[]
projects ProjectServer[]
@@index([name])
}
enum Transport {
STDIO
SSE
STREAMABLE_HTTP
}
// ── MCP Templates ──
model McpTemplate {
id String @id @default(cuid())
name String @unique
version String @default("1.0.0")
description String @default("")
packageName String?
runtime String?
dockerImage String?
transport Transport @default(STDIO)
repositoryUrl String?
externalUrl String?
command Json?
containerPort Int?
replicas Int @default(1)
env Json @default("[]")
healthCheck Json?
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
@@index([name])
}
feat(mcpd): pluggable SecretBackend abstraction + OpenBao driver + migrate Why: API keys live in Postgres as plaintext JSON. A DB read exposes every credential in the system. Before centralising more secrets (LLM keys, etc.) we want to be able to point at an external KV store and drop DB access to sensitive rows. New model: - `SecretBackend` resource (CRUD + isDefault invariant) owns how a secret is stored. `Secret` gains `backendId` FK and `externalRef`. Reads/writes dispatch through a driver. - `plaintext` driver (near-noop, uses existing Secret.data column) is seeded as the `default` row at startup. Acts as trust root / bootstrap. - `openbao` driver (also HashiCorp Vault KV v2 compatible) talks plain HTTP, no SDK dependency. Auth via static token pulled from a plaintext-backed `Secret` through the injected SecretRefResolver. Caches resolved token. - `SecretMigrateService` moves secrets one-at-a-time: read → write dest → flip row → best-effort source delete. Interrupted runs are idempotent (skips secrets already on destination). CLI surface: - `mcpctl create|get|describe|delete secretbackend` + `--default` on create. - `mcpctl migrate secrets --from X --to Y [--names a,b] [--keep-source] [--dry-run]` - `apply -f` round-trips secretbackends (yaml/json multi-doc + grouped). - RBAC: `secretbackends` resource + `run:migrate-secrets` operation. - Fish + bash completions regenerated. docs/secret-backends.md covers the OpenBao policy, chicken-and-egg auth flow, and the migration semantics. Broke the circular dep (OpenBao needs SecretService to resolve its own token, SecretService needs SecretBackendService) with a deferred-resolver bridge in mcpd startup. 11 new driver unit tests; existing env-resolver/secret-route/ backup tests updated for the new service signatures. Full suite: 1792/1792. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 19:29:55 +01:00
// ── Secret Backends ──
//
// Pluggable storage for Secret.data. Default is `plaintext` (data stored in
// Secret.data JSON). Other drivers (e.g. `openbao`) store only a reference in
// Secret.externalRef and fetch actual values from the external system at read
// time. A `plaintext` row is seeded on first startup so the system always has
// a viable backend; additional backends are user-managed via
// `mcpctl create secretbackend`.
model SecretBackend {
id String @id @default(cuid())
name String @unique
type String // plaintext | openbao | (future: vault, aws-sm, ...)
config Json @default("{}") // type-specific: url, mount, namespace, tokenSecretRef
feat(openbao): wizard-provisioning + daily token rotation One-command setup replaces the 6-step manual flow — `mcpctl create secretbackend bao --type openbao --wizard` takes the OpenBao admin token once, provisions a narrow policy + token role, mints the first periodic token, stores it on mcpd, verifies end-to-end, and prints the migration command. The admin token is NEVER persisted. The stored credential auto-rotates daily: mcpd mints a successor via the token role (self-rotation capability is part of the policy it was issued with), verifies the successor, writes it over the backing Secret, then revokes the predecessor by accessor. TTL 720h means a week of rotation failures still leaves 20+ days of runway. Shared: - New `@mcpctl/shared/vault` — pure HTTP wrappers (verifyHealth, ensureKvV2, writePolicy, ensureTokenRole, mintRoleToken, revokeAccessor, lookupSelf, testWriteReadDelete) and policy HCL builder. mcpd: - `tokenMeta Json @default("{}")` on SecretBackend. Self-healing schema migration — empty default lets `prisma db push` add the column cleanly. - SecretBackendRotator.rotateOne: mint → verify → persist → revoke-old → update tokenMeta. Failures surface via `lastRotationError` on the row; the old token keeps working. - SecretBackendRotatorLoop: on startup rotates overdue backends, schedules per-backend timers with ±10min jitter. Stops cleanly on shutdown. - New `POST /api/v1/secretbackends/:id/rotate` (operation `rotate-secretbackend` — added to bootstrap-admin's auto-migrated ops alongside migrate-secrets, which was previously missing too). CLI: - `--wizard` on `create secretbackend` delegates to the interactive flow. All prompts can be pre-answered via flags (--url, --admin-token, --mount, --path-prefix, --policy-name, --token-role, --no-promote-default) for CI. - `mcpctl rotate secretbackend <name>` — convenience verb; hits the new rotate endpoint. - `describe secretbackend` renders a Token health section (healthy / STALE / WARNING / ERROR) with generated/renewal/expiry timestamps and last rotation error. Only shown when tokenMeta.rotatable is true — the existing k8s-auth + static-token backends don't surface it. Tests: 15 vault-client unit tests (shared), 8 rotator unit tests (mcpd), 3 wizard flow tests (cli, including a regression test that the admin token never appears in stdout). Full suite 1885/1885 (+32). Completions regenerated for the new flags. Out of scope (explicit): kubernetes-auth wizard, Vault Enterprise namespaces in the wizard path, rotation for non-wizard static-token backends. See plan file for details. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 17:20:37 +01:00
// Runtime metadata for auto-rotating backend credentials (openbao token
// auth). Fields: generatedAt, nextRenewalAt, validUntil, lastRotationAt,
// lastRotationError, rotatable (true only for wizard-provisioned tokens).
// Empty object for backends that don't use rotation (plaintext, kubernetes
// auth, or static tokens). Managed entirely by the rotator service.
tokenMeta Json @default("{}")
isDefault Boolean @default(false) // exactly one row has isDefault=true
feat(mcpd): pluggable SecretBackend abstraction + OpenBao driver + migrate Why: API keys live in Postgres as plaintext JSON. A DB read exposes every credential in the system. Before centralising more secrets (LLM keys, etc.) we want to be able to point at an external KV store and drop DB access to sensitive rows. New model: - `SecretBackend` resource (CRUD + isDefault invariant) owns how a secret is stored. `Secret` gains `backendId` FK and `externalRef`. Reads/writes dispatch through a driver. - `plaintext` driver (near-noop, uses existing Secret.data column) is seeded as the `default` row at startup. Acts as trust root / bootstrap. - `openbao` driver (also HashiCorp Vault KV v2 compatible) talks plain HTTP, no SDK dependency. Auth via static token pulled from a plaintext-backed `Secret` through the injected SecretRefResolver. Caches resolved token. - `SecretMigrateService` moves secrets one-at-a-time: read → write dest → flip row → best-effort source delete. Interrupted runs are idempotent (skips secrets already on destination). CLI surface: - `mcpctl create|get|describe|delete secretbackend` + `--default` on create. - `mcpctl migrate secrets --from X --to Y [--names a,b] [--keep-source] [--dry-run]` - `apply -f` round-trips secretbackends (yaml/json multi-doc + grouped). - RBAC: `secretbackends` resource + `run:migrate-secrets` operation. - Fish + bash completions regenerated. docs/secret-backends.md covers the OpenBao policy, chicken-and-egg auth flow, and the migration semantics. Broke the circular dep (OpenBao needs SecretService to resolve its own token, SecretService needs SecretBackendService) with a deferred-resolver bridge in mcpd startup. 11 new driver unit tests; existing env-resolver/secret-route/ backup tests updated for the new service signatures. Full suite: 1792/1792. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 19:29:55 +01:00
description String @default("")
version Int @default(1)
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
secrets Secret[]
@@index([name])
@@index([isDefault])
}
// ── Secrets ──
model Secret {
feat(mcpd): pluggable SecretBackend abstraction + OpenBao driver + migrate Why: API keys live in Postgres as plaintext JSON. A DB read exposes every credential in the system. Before centralising more secrets (LLM keys, etc.) we want to be able to point at an external KV store and drop DB access to sensitive rows. New model: - `SecretBackend` resource (CRUD + isDefault invariant) owns how a secret is stored. `Secret` gains `backendId` FK and `externalRef`. Reads/writes dispatch through a driver. - `plaintext` driver (near-noop, uses existing Secret.data column) is seeded as the `default` row at startup. Acts as trust root / bootstrap. - `openbao` driver (also HashiCorp Vault KV v2 compatible) talks plain HTTP, no SDK dependency. Auth via static token pulled from a plaintext-backed `Secret` through the injected SecretRefResolver. Caches resolved token. - `SecretMigrateService` moves secrets one-at-a-time: read → write dest → flip row → best-effort source delete. Interrupted runs are idempotent (skips secrets already on destination). CLI surface: - `mcpctl create|get|describe|delete secretbackend` + `--default` on create. - `mcpctl migrate secrets --from X --to Y [--names a,b] [--keep-source] [--dry-run]` - `apply -f` round-trips secretbackends (yaml/json multi-doc + grouped). - RBAC: `secretbackends` resource + `run:migrate-secrets` operation. - Fish + bash completions regenerated. docs/secret-backends.md covers the OpenBao policy, chicken-and-egg auth flow, and the migration semantics. Broke the circular dep (OpenBao needs SecretService to resolve its own token, SecretService needs SecretBackendService) with a deferred-resolver bridge in mcpd startup. 11 new driver unit tests; existing env-resolver/secret-route/ backup tests updated for the new service signatures. Full suite: 1792/1792. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 19:29:55 +01:00
id String @id @default(cuid())
name String @unique
// FK to SecretBackend. Default empty string lets `prisma db push` add the
// column to pre-existing rows without a data-loss reset; `bootstrapSecretBackends`
// then points any empty-string values at the seeded `default` plaintext backend
// on next mcpd startup. New rows written by SecretService always carry a
// valid FK immediately.
backendId String @default("")
data Json @default("{}") // populated by plaintext backend only
externalRef String @default("") // populated by non-plaintext backends (e.g. "mount/path#v3")
feat(secrets): track key names so list/describe work for backend-stored secrets Post-migration, every Secret on a non-plaintext backend had an empty `data` column (values live in the backend; only externalRef on the row). The CLI's \`get secrets\` showed \`KEYS: -\` and \`describe secret\` showed \`(empty)\` for all 9 migrated secrets — useless without --show-values. Fix: dedicated \`keyNames Json\` column on Secret that stores the sorted key list independently from the values. Populated on every write path, lazily backfilled on first read for pre-existing rows that pre-date the column. Schema default \`[]\` keeps prisma db push self-healing on rolling upgrades. - src/db/prisma/schema.prisma: add Secret.keyNames Json @default("[]") - src/mcpd/src/repositories/secret.repository.ts: pipe keyNames through create + update - src/mcpd/src/services/secret.service.ts: - create/update populate keyNames = sorted Object.keys(data) - getById lazy-backfills empty keyNames (cheap: derives from data for plaintext, single backend read for openbao) - src/mcpd/src/services/secret-migrate.service.ts: migrate writes keyNames alongside the new backendId so freshly-migrated rows are populated without a follow-up read - src/cli/src/commands/get.ts: KEYS column reads keyNames first, falls back to Object.keys(data) for older rows - src/cli/src/commands/describe.ts: shows the Data section keys whenever keyNames OR data has entries (so backend-stored secrets render their key list); --show-values still resolves through the backend After deploy, the 9 already-migrated secrets backfill their keyNames on the next describe-by-id, with no operator action needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 00:57:06 +01:00
// Sorted list of the secret's data keys WITHOUT their values. Populated on
// every create/update/migrate so list views and describe-without-reveal can
// show "this secret has GRAFANA_URL + GRAFANA_TOKEN" without fetching the
// backing data. For pre-existing rows the field is empty until the next
// write or a lazy resolve in getById fills it in.
keyNames Json @default("[]")
feat(mcpd): pluggable SecretBackend abstraction + OpenBao driver + migrate Why: API keys live in Postgres as plaintext JSON. A DB read exposes every credential in the system. Before centralising more secrets (LLM keys, etc.) we want to be able to point at an external KV store and drop DB access to sensitive rows. New model: - `SecretBackend` resource (CRUD + isDefault invariant) owns how a secret is stored. `Secret` gains `backendId` FK and `externalRef`. Reads/writes dispatch through a driver. - `plaintext` driver (near-noop, uses existing Secret.data column) is seeded as the `default` row at startup. Acts as trust root / bootstrap. - `openbao` driver (also HashiCorp Vault KV v2 compatible) talks plain HTTP, no SDK dependency. Auth via static token pulled from a plaintext-backed `Secret` through the injected SecretRefResolver. Caches resolved token. - `SecretMigrateService` moves secrets one-at-a-time: read → write dest → flip row → best-effort source delete. Interrupted runs are idempotent (skips secrets already on destination). CLI surface: - `mcpctl create|get|describe|delete secretbackend` + `--default` on create. - `mcpctl migrate secrets --from X --to Y [--names a,b] [--keep-source] [--dry-run]` - `apply -f` round-trips secretbackends (yaml/json multi-doc + grouped). - RBAC: `secretbackends` resource + `run:migrate-secrets` operation. - Fish + bash completions regenerated. docs/secret-backends.md covers the OpenBao policy, chicken-and-egg auth flow, and the migration semantics. Broke the circular dep (OpenBao needs SecretService to resolve its own token, SecretService needs SecretBackendService) with a deferred-resolver bridge in mcpd startup. 11 new driver unit tests; existing env-resolver/secret-route/ backup tests updated for the new service signatures. Full suite: 1792/1792. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 19:29:55 +01:00
version Int @default(1)
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
backend SecretBackend @relation(fields: [backendId], references: [id])
feat(mcpd): Llm resource — CRUD + CLI + apply Why: every client that wants an LLM (the agent, HTTP-mode mcplocal, Claude Code's STDIO mcplocal) today has to know the provider URL + key, and each user's ~/.mcpctl/config.json carries them. Centralising the catalogue on the server is the prerequisite for Phase 2 (mcpd proxies inference so credentials never leave the cluster). This phase adds the `Llm` resource and its CRUD surface — no proxy yet, no client pivot yet. Just enough to register what you have. Schema: - New `Llm` model: name/type/model/url/tier/description + {apiKeySecretId, apiKeySecretKey} FK pair. Reverse `llms` relation on Secret. - Provider types: anthropic | openai | deepseek | vllm | ollama | gemini-cli. - Tiers: fast | heavy. mcpd: - LlmRepository + LlmService + Zod validation schema + /api/v1/llms routes. - API surface exposes `apiKeyRef: {name, key}` — the service translates to/ from the FK pair so clients never deal in cuids. - `resolveApiKey(llmName)` reads through SecretService (which itself dispatches to the right SecretBackend). That's the hook Phase 2's inference proxy uses. - RBAC: added `'llms'` to RBAC_RESOURCES + resource alias. Standard view/create/edit/delete semantics. - Wired into main.ts (repo, service, routes). CLI: - `mcpctl create llm <name> --type X --model Y --tier fast|heavy --api-key-ref SECRET/KEY [--url ...] [--extra k=v ...]` - `mcpctl get|describe|delete llm` — standard resource verbs. - `mcpctl apply -f` with `kind: llm` (single- or multi-doc yaml/json). Applied after secrets, before servers — apiKeyRef resolves an existing Secret. - Shell completions regenerated. Tests: 11 service unit tests + 9 route tests (happy path, 404s, 409, validation). Full suite 1812/1812 (+20 from the 1792 Phase 0 baseline). TypeScript clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 21:28:43 +01:00
llms Llm[]
@@index([name])
feat(mcpd): pluggable SecretBackend abstraction + OpenBao driver + migrate Why: API keys live in Postgres as plaintext JSON. A DB read exposes every credential in the system. Before centralising more secrets (LLM keys, etc.) we want to be able to point at an external KV store and drop DB access to sensitive rows. New model: - `SecretBackend` resource (CRUD + isDefault invariant) owns how a secret is stored. `Secret` gains `backendId` FK and `externalRef`. Reads/writes dispatch through a driver. - `plaintext` driver (near-noop, uses existing Secret.data column) is seeded as the `default` row at startup. Acts as trust root / bootstrap. - `openbao` driver (also HashiCorp Vault KV v2 compatible) talks plain HTTP, no SDK dependency. Auth via static token pulled from a plaintext-backed `Secret` through the injected SecretRefResolver. Caches resolved token. - `SecretMigrateService` moves secrets one-at-a-time: read → write dest → flip row → best-effort source delete. Interrupted runs are idempotent (skips secrets already on destination). CLI surface: - `mcpctl create|get|describe|delete secretbackend` + `--default` on create. - `mcpctl migrate secrets --from X --to Y [--names a,b] [--keep-source] [--dry-run]` - `apply -f` round-trips secretbackends (yaml/json multi-doc + grouped). - RBAC: `secretbackends` resource + `run:migrate-secrets` operation. - Fish + bash completions regenerated. docs/secret-backends.md covers the OpenBao policy, chicken-and-egg auth flow, and the migration semantics. Broke the circular dep (OpenBao needs SecretService to resolve its own token, SecretService needs SecretBackendService) with a deferred-resolver bridge in mcpd startup. 11 new driver unit tests; existing env-resolver/secret-route/ backup tests updated for the new service signatures. Full suite: 1792/1792. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 19:29:55 +01:00
@@index([backendId])
}
feat(mcpd): Llm resource — CRUD + CLI + apply Why: every client that wants an LLM (the agent, HTTP-mode mcplocal, Claude Code's STDIO mcplocal) today has to know the provider URL + key, and each user's ~/.mcpctl/config.json carries them. Centralising the catalogue on the server is the prerequisite for Phase 2 (mcpd proxies inference so credentials never leave the cluster). This phase adds the `Llm` resource and its CRUD surface — no proxy yet, no client pivot yet. Just enough to register what you have. Schema: - New `Llm` model: name/type/model/url/tier/description + {apiKeySecretId, apiKeySecretKey} FK pair. Reverse `llms` relation on Secret. - Provider types: anthropic | openai | deepseek | vllm | ollama | gemini-cli. - Tiers: fast | heavy. mcpd: - LlmRepository + LlmService + Zod validation schema + /api/v1/llms routes. - API surface exposes `apiKeyRef: {name, key}` — the service translates to/ from the FK pair so clients never deal in cuids. - `resolveApiKey(llmName)` reads through SecretService (which itself dispatches to the right SecretBackend). That's the hook Phase 2's inference proxy uses. - RBAC: added `'llms'` to RBAC_RESOURCES + resource alias. Standard view/create/edit/delete semantics. - Wired into main.ts (repo, service, routes). CLI: - `mcpctl create llm <name> --type X --model Y --tier fast|heavy --api-key-ref SECRET/KEY [--url ...] [--extra k=v ...]` - `mcpctl get|describe|delete llm` — standard resource verbs. - `mcpctl apply -f` with `kind: llm` (single- or multi-doc yaml/json). Applied after secrets, before servers — apiKeyRef resolves an existing Secret. - Shell completions regenerated. Tests: 11 service unit tests + 9 route tests (happy path, 404s, 409, validation). Full suite 1812/1812 (+20 from the 1792 Phase 0 baseline). TypeScript clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 21:28:43 +01:00
// ── LLMs ──
//
// Server-managed LLM providers. Clients (agent, HTTP-mode mcplocal) send
// OpenAI-format requests to `mcpd /api/v1/llms/:name/infer` — mcpd attaches the
// provider API key server-side so credentials never leave the cluster.
// Credentials are stored by reference: `apiKeySecret` points at a Secret, and
// `apiKeySecretKey` names the key within that secret's data.
//
// `kind=virtual` rows are *registered by an mcplocal client* (rather than a
// human via `mcpctl create llm`). Their inference is relayed back through
// the SSE control channel to the publishing mcplocal session. The lifecycle
// fields (lastHeartbeatAt, status, inactiveSince) belong to virtual rows;
// public rows ignore them.
enum LlmKind {
public // upstream-URL row, mcpd calls directly
virtual // mcplocal-registered, inference relayed via SSE control channel
}
enum LlmStatus {
active // healthy, accepting requests
inactive // publisher went away; row pending 4-h GC
hibernating // publisher present but backend asleep — wakes on demand (v2)
}
feat(mcpd): Llm resource — CRUD + CLI + apply Why: every client that wants an LLM (the agent, HTTP-mode mcplocal, Claude Code's STDIO mcplocal) today has to know the provider URL + key, and each user's ~/.mcpctl/config.json carries them. Centralising the catalogue on the server is the prerequisite for Phase 2 (mcpd proxies inference so credentials never leave the cluster). This phase adds the `Llm` resource and its CRUD surface — no proxy yet, no client pivot yet. Just enough to register what you have. Schema: - New `Llm` model: name/type/model/url/tier/description + {apiKeySecretId, apiKeySecretKey} FK pair. Reverse `llms` relation on Secret. - Provider types: anthropic | openai | deepseek | vllm | ollama | gemini-cli. - Tiers: fast | heavy. mcpd: - LlmRepository + LlmService + Zod validation schema + /api/v1/llms routes. - API surface exposes `apiKeyRef: {name, key}` — the service translates to/ from the FK pair so clients never deal in cuids. - `resolveApiKey(llmName)` reads through SecretService (which itself dispatches to the right SecretBackend). That's the hook Phase 2's inference proxy uses. - RBAC: added `'llms'` to RBAC_RESOURCES + resource alias. Standard view/create/edit/delete semantics. - Wired into main.ts (repo, service, routes). CLI: - `mcpctl create llm <name> --type X --model Y --tier fast|heavy --api-key-ref SECRET/KEY [--url ...] [--extra k=v ...]` - `mcpctl get|describe|delete llm` — standard resource verbs. - `mcpctl apply -f` with `kind: llm` (single- or multi-doc yaml/json). Applied after secrets, before servers — apiKeyRef resolves an existing Secret. - Shell completions regenerated. Tests: 11 service unit tests + 9 route tests (happy path, 404s, 409, validation). Full suite 1812/1812 (+20 from the 1792 Phase 0 baseline). TypeScript clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 21:28:43 +01:00
model Llm {
id String @id @default(cuid())
name String @unique
type String // anthropic | openai | deepseek | vllm | ollama | gemini-cli
model String // e.g. claude-3-5-sonnet-20241022
url String @default("") // endpoint (empty for provider default)
tier String @default("fast") // fast | heavy
description String @default("")
apiKeySecretId String? // FK to Secret
apiKeySecretKey String? // key inside the Secret's data
extraConfig Json @default("{}") // per-type extras
feat(mcpd+db): Llm.poolName + chat dispatcher pool failover (v4 Stage 1) Adds LB-pool-by-shared-name without introducing a new resource. The existing `Llm.name` stays globally unique; a new optional `poolName` column declares membership in a pool. Multiple Llms sharing a non-null `poolName` stack into one load-balanced pool that the chat dispatcher expands at request time. Effective pool key = `poolName ?? name`. Solo rows (poolName=null) are addressable as a "pool of 1" via their own name, so existing single-Llm agents and YAMLs keep working unchanged. A solo row whose name happens to match an explicit poolName joins the same pool — by design — so an operator can transparently promote an existing Llm to pool seed. Dispatcher (chat.service): prepareContext now resolves a randomly- shuffled list of viable pool candidates (status != inactive) once per turn. runOneInference and streamInference iterate the list on transport-level failure (network, virtual publisher disconnect) until one succeeds or the list is exhausted. Streaming failover only covers "failed before first chunk" — once we've yielded text, we're committed to that backend. Auth/4xx errors surfaced as result.status are NOT retried; siblings with the same key/model would fail identically. When the agent's pinned Llm is itself inactive but a sibling pool member is up, dispatch transparently uses the sibling — that's the whole point. When every member is inactive, prepareContext throws a clear "No active Llm in pool '<key>' (pinned: <name>)" error rather than letting the dispatcher's "exhausted" branch surface it. Tests: - 5 new chat-service tests for pool dispatch / failover / pinned-down / all-inactive (chat-service.test.ts). - 7 new db schema tests for the column, the unique-name invariant, the fallback-to-name semantics, and the solo-name-joins-explicit-pool edge case (llm-pool-schema.test.ts). - mcpd 865/865 (was 860; +5), db pool-schema 7/7, no regressions. Stage 2 (next): HTTP route /api/v1/llms/<name>/members + aggregate pool stats on the existing single-Llm route, CLI POOL column + describe block + --pool-name flag, yaml round-trip.
2026-04-27 22:02:41 +01:00
// ── LB pool by name (v4) ──
// When set, this Llm is part of a load-balanced pool. Multiple Llms
// sharing the same `poolName` stack into a pool that the chat dispatcher
// expands at request time and picks an active member from. When NULL,
// the effective pool key is the row's own `name` (i.e. "pool of 1",
// identical to pre-v4 behavior). Agents pin to a single Llm by id; the
// dispatcher transparently widens to the pool when this field is set.
poolName String?
// ── Virtual-provider lifecycle (NULL/default for kind=public) ──
kind LlmKind @default(public)
providerSessionId String? // mcplocal session that owns this row when virtual
lastHeartbeatAt DateTime? // bumped on every publisher heartbeat
status LlmStatus @default(active)
inactiveSince DateTime? // when status flipped from active; used for 4-h GC
feat(mcpd+db): visibility scope + ownership for Llms and Agents (v7 Stage 1) Adds the schema + service-layer machinery for per-user RBAC scoping of virtual Llms and Agents. Without this, anyone with `view:llms` sees every other user's published model — fine for a single-user homelab, wrong for org use where workstation-published models or paid keys aren't meant to be broadcast. Schema: - Llm: new `ownerId String?` + `visibility String @default("public")`. NULL ownerId on legacy rows is treated as public for back-compat. - Agent: `visibility String @default("public")` (Agent already has `ownerId`, required). - Composite index `(visibility, ownerId)` on both tables for the list-filter hot path. - Migration backfills both columns to 'public' so pre-v7 setups behave identically post-deploy. Service layer: - New `Viewer` / `AgentViewer` shape: `{ userId, wildcard, allowedNames }`. The route layer computes this from `request.userId` + `RbacService.getAllowedScope` and passes it down. NULL viewer = skip the filter (internal callers — cron sweeps, audit, tests). - `isLlmVisibleTo` / `isAgentVisibleTo` pure predicates encode the decision tree: visibility=public → visible (RBAC layer above already passed) viewer=null OR wildcard → visible ownerId === viewer.userId → visible row.name in viewer.allowedNames → visible else → hidden - LlmService.list/getById/getByName + AgentService equivalents accept an optional Viewer arg and apply the predicate. Get-style methods 404 (not 403) on hidden rows so name enumeration via differential status is impossible. Repositories: CreateInput/UpdateInput types gained `ownerId`/ `visibility` (Llm) and `visibility` (Agent). Update is in place; ownerId is set-once at create time. Tests: - 13 unit tests on the predicate covering every branch (null viewer, public, wildcard, owner, name-scoped grant, foreign private, legacy null-ownerId). - mcpd 908/908 (was 893; +15 across the merge windows + this PR). Stage 2 (next): route plumbing — every list/get endpoint needs to build the Viewer from the request and pass it through. mcplocal virtuals default to visibility=private on register. CLI adds a VISIBILITY column and a --visibility flag. yaml round-trip preserves the field.
2026-04-29 00:46:06 +01:00
// ── Per-user RBAC scoping (v7) ──
// `ownerId` records who created/published the row. NULL on legacy rows
// (those created before the v7 migration) — those continue to behave
// as `visibility=public` for back-compat. New rows always carry an
// ownerId set by the service layer (`User.id` of the authenticated
// caller, or the publishing mcplocal user for virtuals).
//
// `visibility` controls who can see / use the row:
// - 'public' : anyone with the resource grant (`view:llms`,
// `run:llms:<name>`, etc.) sees it. Legacy default
// mirrors today's behavior — explicit
// `mcpctl create llm` calls keep this default.
// - 'private' : only the owner sees it by default; other users need
// an explicit name-scoped RBAC binding
// (`view:llms:<name>`, `run:llms:<name>`). The list
// endpoint hides foreign-private rows from
// unauthorized callers; get/describe returns 404 to
// prevent name enumeration.
//
// mcplocal-published virtual Llms default to 'private' on register —
// a workstation-published model isn't typically meant for the whole
// org until the publisher explicitly shares it. See docs/virtual-llms.md.
ownerId String?
visibility String @default("public")
version Int @default(1)
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
feat(mcpd): Llm resource — CRUD + CLI + apply Why: every client that wants an LLM (the agent, HTTP-mode mcplocal, Claude Code's STDIO mcplocal) today has to know the provider URL + key, and each user's ~/.mcpctl/config.json carries them. Centralising the catalogue on the server is the prerequisite for Phase 2 (mcpd proxies inference so credentials never leave the cluster). This phase adds the `Llm` resource and its CRUD surface — no proxy yet, no client pivot yet. Just enough to register what you have. Schema: - New `Llm` model: name/type/model/url/tier/description + {apiKeySecretId, apiKeySecretKey} FK pair. Reverse `llms` relation on Secret. - Provider types: anthropic | openai | deepseek | vllm | ollama | gemini-cli. - Tiers: fast | heavy. mcpd: - LlmRepository + LlmService + Zod validation schema + /api/v1/llms routes. - API surface exposes `apiKeyRef: {name, key}` — the service translates to/ from the FK pair so clients never deal in cuids. - `resolveApiKey(llmName)` reads through SecretService (which itself dispatches to the right SecretBackend). That's the hook Phase 2's inference proxy uses. - RBAC: added `'llms'` to RBAC_RESOURCES + resource alias. Standard view/create/edit/delete semantics. - Wired into main.ts (repo, service, routes). CLI: - `mcpctl create llm <name> --type X --model Y --tier fast|heavy --api-key-ref SECRET/KEY [--url ...] [--extra k=v ...]` - `mcpctl get|describe|delete llm` — standard resource verbs. - `mcpctl apply -f` with `kind: llm` (single- or multi-doc yaml/json). Applied after secrets, before servers — apiKeyRef resolves an existing Secret. - Shell completions regenerated. Tests: 11 service unit tests + 9 route tests (happy path, 404s, 409, validation). Full suite 1812/1812 (+20 from the 1792 Phase 0 baseline). TypeScript clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 21:28:43 +01:00
apiKeySecret Secret? @relation(fields: [apiKeySecretId], references: [id], onDelete: SetNull)
agents Agent[]
feat(mcpd): Llm resource — CRUD + CLI + apply Why: every client that wants an LLM (the agent, HTTP-mode mcplocal, Claude Code's STDIO mcplocal) today has to know the provider URL + key, and each user's ~/.mcpctl/config.json carries them. Centralising the catalogue on the server is the prerequisite for Phase 2 (mcpd proxies inference so credentials never leave the cluster). This phase adds the `Llm` resource and its CRUD surface — no proxy yet, no client pivot yet. Just enough to register what you have. Schema: - New `Llm` model: name/type/model/url/tier/description + {apiKeySecretId, apiKeySecretKey} FK pair. Reverse `llms` relation on Secret. - Provider types: anthropic | openai | deepseek | vllm | ollama | gemini-cli. - Tiers: fast | heavy. mcpd: - LlmRepository + LlmService + Zod validation schema + /api/v1/llms routes. - API surface exposes `apiKeyRef: {name, key}` — the service translates to/ from the FK pair so clients never deal in cuids. - `resolveApiKey(llmName)` reads through SecretService (which itself dispatches to the right SecretBackend). That's the hook Phase 2's inference proxy uses. - RBAC: added `'llms'` to RBAC_RESOURCES + resource alias. Standard view/create/edit/delete semantics. - Wired into main.ts (repo, service, routes). CLI: - `mcpctl create llm <name> --type X --model Y --tier fast|heavy --api-key-ref SECRET/KEY [--url ...] [--extra k=v ...]` - `mcpctl get|describe|delete llm` — standard resource verbs. - `mcpctl apply -f` with `kind: llm` (single- or multi-doc yaml/json). Applied after secrets, before servers — apiKeyRef resolves an existing Secret. - Shell completions regenerated. Tests: 11 service unit tests + 9 route tests (happy path, 404s, 409, validation). Full suite 1812/1812 (+20 from the 1792 Phase 0 baseline). TypeScript clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 21:28:43 +01:00
@@index([name])
@@index([tier])
@@index([apiKeySecretId])
@@index([kind, status])
@@index([providerSessionId])
feat(mcpd+db): Llm.poolName + chat dispatcher pool failover (v4 Stage 1) Adds LB-pool-by-shared-name without introducing a new resource. The existing `Llm.name` stays globally unique; a new optional `poolName` column declares membership in a pool. Multiple Llms sharing a non-null `poolName` stack into one load-balanced pool that the chat dispatcher expands at request time. Effective pool key = `poolName ?? name`. Solo rows (poolName=null) are addressable as a "pool of 1" via their own name, so existing single-Llm agents and YAMLs keep working unchanged. A solo row whose name happens to match an explicit poolName joins the same pool — by design — so an operator can transparently promote an existing Llm to pool seed. Dispatcher (chat.service): prepareContext now resolves a randomly- shuffled list of viable pool candidates (status != inactive) once per turn. runOneInference and streamInference iterate the list on transport-level failure (network, virtual publisher disconnect) until one succeeds or the list is exhausted. Streaming failover only covers "failed before first chunk" — once we've yielded text, we're committed to that backend. Auth/4xx errors surfaced as result.status are NOT retried; siblings with the same key/model would fail identically. When the agent's pinned Llm is itself inactive but a sibling pool member is up, dispatch transparently uses the sibling — that's the whole point. When every member is inactive, prepareContext throws a clear "No active Llm in pool '<key>' (pinned: <name>)" error rather than letting the dispatcher's "exhausted" branch surface it. Tests: - 5 new chat-service tests for pool dispatch / failover / pinned-down / all-inactive (chat-service.test.ts). - 7 new db schema tests for the column, the unique-name invariant, the fallback-to-name semantics, and the solo-name-joins-explicit-pool edge case (llm-pool-schema.test.ts). - mcpd 865/865 (was 860; +5), db pool-schema 7/7, no regressions. Stage 2 (next): HTTP route /api/v1/llms/<name>/members + aggregate pool stats on the existing single-Llm route, CLI POOL column + describe block + --pool-name flag, yaml round-trip.
2026-04-27 22:02:41 +01:00
@@index([poolName])
feat(mcpd+db): visibility scope + ownership for Llms and Agents (v7 Stage 1) Adds the schema + service-layer machinery for per-user RBAC scoping of virtual Llms and Agents. Without this, anyone with `view:llms` sees every other user's published model — fine for a single-user homelab, wrong for org use where workstation-published models or paid keys aren't meant to be broadcast. Schema: - Llm: new `ownerId String?` + `visibility String @default("public")`. NULL ownerId on legacy rows is treated as public for back-compat. - Agent: `visibility String @default("public")` (Agent already has `ownerId`, required). - Composite index `(visibility, ownerId)` on both tables for the list-filter hot path. - Migration backfills both columns to 'public' so pre-v7 setups behave identically post-deploy. Service layer: - New `Viewer` / `AgentViewer` shape: `{ userId, wildcard, allowedNames }`. The route layer computes this from `request.userId` + `RbacService.getAllowedScope` and passes it down. NULL viewer = skip the filter (internal callers — cron sweeps, audit, tests). - `isLlmVisibleTo` / `isAgentVisibleTo` pure predicates encode the decision tree: visibility=public → visible (RBAC layer above already passed) viewer=null OR wildcard → visible ownerId === viewer.userId → visible row.name in viewer.allowedNames → visible else → hidden - LlmService.list/getById/getByName + AgentService equivalents accept an optional Viewer arg and apply the predicate. Get-style methods 404 (not 403) on hidden rows so name enumeration via differential status is impossible. Repositories: CreateInput/UpdateInput types gained `ownerId`/ `visibility` (Llm) and `visibility` (Agent). Update is in place; ownerId is set-once at create time. Tests: - 13 unit tests on the predicate covering every branch (null viewer, public, wildcard, owner, name-scoped grant, foreign private, legacy null-ownerId). - mcpd 908/908 (was 893; +15 across the merge windows + this PR). Stage 2 (next): route plumbing — every list/get endpoint needs to build the Viewer from the request and pass it through. mcplocal virtuals default to visibility=private on register. CLI adds a VISIBILITY column and a --visibility flag. yaml round-trip preserves the field.
2026-04-29 00:46:06 +01:00
// List filter on the hot path: "rows visible to caller X" decomposes
// into `visibility='public' OR ownerId=X` + an RBAC join. Composite
// index keeps the predicate fast even on a large table.
@@index([visibility, ownerId])
feat(mcpd): Llm resource — CRUD + CLI + apply Why: every client that wants an LLM (the agent, HTTP-mode mcplocal, Claude Code's STDIO mcplocal) today has to know the provider URL + key, and each user's ~/.mcpctl/config.json carries them. Centralising the catalogue on the server is the prerequisite for Phase 2 (mcpd proxies inference so credentials never leave the cluster). This phase adds the `Llm` resource and its CRUD surface — no proxy yet, no client pivot yet. Just enough to register what you have. Schema: - New `Llm` model: name/type/model/url/tier/description + {apiKeySecretId, apiKeySecretKey} FK pair. Reverse `llms` relation on Secret. - Provider types: anthropic | openai | deepseek | vllm | ollama | gemini-cli. - Tiers: fast | heavy. mcpd: - LlmRepository + LlmService + Zod validation schema + /api/v1/llms routes. - API surface exposes `apiKeyRef: {name, key}` — the service translates to/ from the FK pair so clients never deal in cuids. - `resolveApiKey(llmName)` reads through SecretService (which itself dispatches to the right SecretBackend). That's the hook Phase 2's inference proxy uses. - RBAC: added `'llms'` to RBAC_RESOURCES + resource alias. Standard view/create/edit/delete semantics. - Wired into main.ts (repo, service, routes). CLI: - `mcpctl create llm <name> --type X --model Y --tier fast|heavy --api-key-ref SECRET/KEY [--url ...] [--extra k=v ...]` - `mcpctl get|describe|delete llm` — standard resource verbs. - `mcpctl apply -f` with `kind: llm` (single- or multi-doc yaml/json). Applied after secrets, before servers — apiKeyRef resolves an existing Secret. - Shell completions regenerated. Tests: 11 service unit tests + 9 route tests (happy path, 404s, 409, validation). Full suite 1812/1812 (+20 from the 1792 Phase 0 baseline). TypeScript clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 21:28:43 +01:00
}
// ── Groups ──
model Group {
id String @id @default(cuid())
name String @unique
description String @default("")
version Int @default(1)
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
members GroupMember[]
@@index([name])
}
model GroupMember {
id String @id @default(cuid())
groupId String
userId String
createdAt DateTime @default(now())
group Group @relation(fields: [groupId], references: [id], onDelete: Cascade)
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
@@unique([groupId, userId])
@@index([groupId])
@@index([userId])
}
// ── RBAC Definitions ──
model RbacDefinition {
id String @id @default(cuid())
name String @unique
subjects Json @default("[]")
roleBindings Json @default("[]")
version Int @default(1)
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
@@index([name])
}
// ── Projects ──
model Project {
id String @id @default(cuid())
name String @unique
description String @default("")
prompt String @default("")
proxyModel String @default("")
gated Boolean @default(true)
llmProvider String?
llmModel String?
serverOverrides Json?
ownerId String
version Int @default(1)
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
feat(db): schema for ResourceRevision, ResourceProposal, Skill Phase 1 of the Skills + Revisions + Proposals work. Purely additive — no existing rows are touched, no tables renamed, no columns dropped. New tables: - ResourceRevision — append-only audit + diff log keyed by (resourceType, resourceId). Both Prompt and Skill produce revisions on every change. Soft FK so revisions outlive the resources they describe. Indexed for history viewer (latest-first), semver lookup, and cross-resource sync diff via contentHash. - ResourceProposal — generic propose/approve/reject queue. Drop-in replacement for the prompt-only PromptRequest. Created empty here; PR-2 will rename PromptRequest → _PromptRequest_legacy and backfill. - Skill — new resource type that mirrors Prompt for everything CRUD- shaped. Adds `files` Json (multi-file bundles, materialised onto disk by `mcpctl skills sync` in PR-5) and `metadata` Json (typed app-layer in PR-3: hooks, mcpServers, postInstall, …). New columns on Prompt: - semver (semver string, default '0.1.0') — auto-bumped patch on save by PromptService.update once PR-2 wires it. Distinct from `version`, which stays as the optimistic-concurrency counter. - currentRevisionId — soft pointer to the latest ResourceRevision row. DB tests cover scope rules (project XOR agent XOR neither), name uniqueness across both compound keys, cascade-on-delete, soft-FK survival of deletion, and JSON column persistence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 00:18:21 +01:00
owner User @relation(fields: [ownerId], references: [id], onDelete: Cascade)
servers ProjectServer[]
prompts Prompt[]
promptRequests PromptRequest[]
feat(db): schema for ResourceRevision, ResourceProposal, Skill Phase 1 of the Skills + Revisions + Proposals work. Purely additive — no existing rows are touched, no tables renamed, no columns dropped. New tables: - ResourceRevision — append-only audit + diff log keyed by (resourceType, resourceId). Both Prompt and Skill produce revisions on every change. Soft FK so revisions outlive the resources they describe. Indexed for history viewer (latest-first), semver lookup, and cross-resource sync diff via contentHash. - ResourceProposal — generic propose/approve/reject queue. Drop-in replacement for the prompt-only PromptRequest. Created empty here; PR-2 will rename PromptRequest → _PromptRequest_legacy and backfill. - Skill — new resource type that mirrors Prompt for everything CRUD- shaped. Adds `files` Json (multi-file bundles, materialised onto disk by `mcpctl skills sync` in PR-5) and `metadata` Json (typed app-layer in PR-3: hooks, mcpServers, postInstall, …). New columns on Prompt: - semver (semver string, default '0.1.0') — auto-bumped patch on save by PromptService.update once PR-2 wires it. Distinct from `version`, which stays as the optimistic-concurrency counter. - currentRevisionId — soft pointer to the latest ResourceRevision row. DB tests cover scope rules (project XOR agent XOR neither), name uniqueness across both compound keys, cascade-on-delete, soft-FK survival of deletion, and JSON column persistence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 00:18:21 +01:00
proposals ResourceProposal[]
skills Skill[]
mcpTokens McpToken[]
agents Agent[]
feat(chat): project-scoped chat — `mcpctl chat --project <name>` Chat directly with a Project (no Agent needed): its Prompts become the system context, its MCP-server tools are callable, its llmProvider/llmModel drive the LLM, and (opt-in) the model can read secret values. History is saved inside the project, attributed per user, resumable, and deletable (RBAC-permitting) — "use it like Claude, scoped to the project". Backend (reuses the agent-chat orchestrator): - ChatThread is now agent-XOR-project (schema + migration + CHECK constraint); new listThreadsByProject / deleteThread on the repo. - ChatService: prepareProjectContext (project prompt + Prompts by priority, llm from llmProvider with llmModel override, project tools), shared runChatLoop/runChatStreamLoop, project thread CRUD with owner enforcement (404-not-403 on foreign threads), admin-override delete. - Gated get_secret virtual tool: offered only with --allow-secrets AND the caller's view:secrets; resolves via SecretService, never routes to a server. - routes/project-chat.ts (chat SSE+non-stream, threads create/list/delete); RBAC run:projects:<name>. CLI: - `mcpctl chat --project <name>` (+ --allow-secrets), one-shot/REPL/resume. - REPL /threads, /resume <id>, /delete <id>; project-aware header + /tools. - `mcpctl get threads --project <name>`, `mcpctl delete thread <id> --project`. - completions regenerated (--project completes project names). Tests: 8 new project-chat unit tests; full mcpd (945) + CLI (508) green; schema validated against Postgres. Docs: docs/chat.md "Project chat" section. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 10:34:21 +01:00
chatThreads ChatThread[]
@@index([name])
@@index([ownerId])
}
model ProjectServer {
id String @id @default(cuid())
projectId String
serverId String
createdAt DateTime @default(now())
project Project @relation(fields: [projectId], references: [id], onDelete: Cascade)
server McpServer @relation(fields: [serverId], references: [id], onDelete: Cascade)
@@unique([projectId, serverId])
}
// ── MCP Tokens (bearer credentials for HTTP-mode mcplocal) ──
//
// Raw value format: `mcpctl_pat_<32 base62 chars>`. The raw value is shown
// exactly once at create time; only the SHA-256 hash is persisted. Tokens are
// scoped to exactly one project — they're only valid at
// `/projects/<that-project>/mcp`. Creator's RBAC is the ceiling; the service
// rejects bindings that exceed what the creator themselves can do.
model McpToken {
id String @id @default(cuid())
name String
projectId String
tokenHash String @unique
tokenPrefix String
ownerId String
description String @default("")
createdAt DateTime @default(now())
expiresAt DateTime?
lastUsedAt DateTime?
revokedAt DateTime?
project Project @relation(fields: [projectId], references: [id], onDelete: Cascade)
owner User @relation(fields: [ownerId], references: [id], onDelete: Cascade)
@@unique([name, projectId])
@@index([tokenHash])
@@index([projectId])
@@index([ownerId])
}
// ── MCP Instances (running containers) ──
model McpInstance {
id String @id @default(cuid())
serverId String
containerId String?
status InstanceStatus @default(STOPPED)
port Int?
metadata Json @default("{}")
healthStatus String?
lastHealthCheck DateTime?
events Json @default("[]")
version Int @default(1)
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
server McpServer @relation(fields: [serverId], references: [id], onDelete: Cascade)
@@index([serverId])
@@index([status])
}
enum InstanceStatus {
STARTING
RUNNING
STOPPING
STOPPED
ERROR
}
// ── Prompts (approved content resources) ──
model Prompt {
feat(db): schema for ResourceRevision, ResourceProposal, Skill Phase 1 of the Skills + Revisions + Proposals work. Purely additive — no existing rows are touched, no tables renamed, no columns dropped. New tables: - ResourceRevision — append-only audit + diff log keyed by (resourceType, resourceId). Both Prompt and Skill produce revisions on every change. Soft FK so revisions outlive the resources they describe. Indexed for history viewer (latest-first), semver lookup, and cross-resource sync diff via contentHash. - ResourceProposal — generic propose/approve/reject queue. Drop-in replacement for the prompt-only PromptRequest. Created empty here; PR-2 will rename PromptRequest → _PromptRequest_legacy and backfill. - Skill — new resource type that mirrors Prompt for everything CRUD- shaped. Adds `files` Json (multi-file bundles, materialised onto disk by `mcpctl skills sync` in PR-5) and `metadata` Json (typed app-layer in PR-3: hooks, mcpServers, postInstall, …). New columns on Prompt: - semver (semver string, default '0.1.0') — auto-bumped patch on save by PromptService.update once PR-2 wires it. Distinct from `version`, which stays as the optimistic-concurrency counter. - currentRevisionId — soft pointer to the latest ResourceRevision row. DB tests cover scope rules (project XOR agent XOR neither), name uniqueness across both compound keys, cascade-on-delete, soft-FK survival of deletion, and JSON column persistence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 00:18:21 +01:00
id String @id @default(cuid())
name String
content String @db.Text
projectId String?
agentId String?
priority Int @default(5)
summary String? @db.Text
chapters Json?
linkTarget String?
// Semantic version of the current content. Auto-bumped patch on every save
// by PromptService.update; author can pass --bump major|minor|patch or
// --semver X.Y.Z to override. NOT the same as `version` below — that one
// is the optimistic-concurrency counter and stays Int.
semver String @default("0.1.0")
// Soft pointer to the latest ResourceRevision row for this prompt. NULL
// before the first revision is recorded. Set in the same transaction as
// create/update by PromptService.
currentRevisionId String?
version Int @default(1)
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
project Project? @relation(fields: [projectId], references: [id], onDelete: Cascade)
agent Agent? @relation(fields: [agentId], references: [id], onDelete: Cascade)
personalities PersonalityPrompt[]
@@unique([name, projectId])
@@unique([name, agentId])
@@index([projectId])
@@index([agentId])
}
feat(db): schema for ResourceRevision, ResourceProposal, Skill Phase 1 of the Skills + Revisions + Proposals work. Purely additive — no existing rows are touched, no tables renamed, no columns dropped. New tables: - ResourceRevision — append-only audit + diff log keyed by (resourceType, resourceId). Both Prompt and Skill produce revisions on every change. Soft FK so revisions outlive the resources they describe. Indexed for history viewer (latest-first), semver lookup, and cross-resource sync diff via contentHash. - ResourceProposal — generic propose/approve/reject queue. Drop-in replacement for the prompt-only PromptRequest. Created empty here; PR-2 will rename PromptRequest → _PromptRequest_legacy and backfill. - Skill — new resource type that mirrors Prompt for everything CRUD- shaped. Adds `files` Json (multi-file bundles, materialised onto disk by `mcpctl skills sync` in PR-5) and `metadata` Json (typed app-layer in PR-3: hooks, mcpServers, postInstall, …). New columns on Prompt: - semver (semver string, default '0.1.0') — auto-bumped patch on save by PromptService.update once PR-2 wires it. Distinct from `version`, which stays as the optimistic-concurrency counter. - currentRevisionId — soft pointer to the latest ResourceRevision row. DB tests cover scope rules (project XOR agent XOR neither), name uniqueness across both compound keys, cascade-on-delete, soft-FK survival of deletion, and JSON column persistence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 00:18:21 +01:00
// ── Skills (Claude Code skill bundles, synced to ~/.claude/skills/<name>/) ──
//
// Skills are the on-disk counterpart to Prompts. mcpd is the source of truth;
// mcpctl skills sync materialises them onto disk under
// ~/.claude/skills/<name>/SKILL.md (+ optional aux files) where Claude Code
// reads them natively. Same scoping rules as Prompt (project XOR agent XOR
// neither = global). Multi-file bundles live in `files` (path → content).
// Typed skill metadata (hooks, mcpServers, postInstall, …) is validated in
// the app layer and stored opaquely here in `metadata`.
model Skill {
id String @id @default(cuid())
name String
description String @default("")
// Body of the SKILL.md file delivered to Claude Code.
content String @db.Text
// Auxiliary files in the skill bundle. Map of relative path → file content
// (UTF-8 text only in v1; binaries deferred). Materialised onto disk at sync
// time alongside SKILL.md.
files Json @default("{}")
// Typed-but-stored-as-Json: { hooks?, mcpServers?, postInstall?,
// preUninstall?, postInstallTimeoutSec? }. Validated at the route layer
// (see src/mcpd/src/validation/skill.schema.ts in PR-3).
metadata Json @default("{}")
projectId String?
agentId String?
priority Int @default(5)
summary String? @db.Text
chapters Json?
// Semantic version of the current content. Auto-bumped patch on every save
// by SkillService.update; author can override via --bump or --semver.
semver String @default("0.1.0")
// Soft pointer to the latest ResourceRevision row for this skill. NULL
// before the first revision is recorded.
currentRevisionId String?
// Optimistic-concurrency counter. NOT semver — that's `semver` above.
version Int @default(1)
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
project Project? @relation(fields: [projectId], references: [id], onDelete: Cascade)
agent Agent? @relation(fields: [agentId], references: [id], onDelete: Cascade)
@@unique([name, projectId])
@@unique([name, agentId])
@@index([projectId])
@@index([agentId])
@@index([name])
}
// ── Prompt Requests (pending proposals from LLM sessions) ──
model PromptRequest {
id String @id @default(cuid())
name String
content String @db.Text
projectId String?
priority Int @default(5)
createdBySession String?
createdByUserId String?
createdAt DateTime @default(now())
project Project? @relation(fields: [projectId], references: [id], onDelete: Cascade)
@@unique([name, projectId])
@@index([projectId])
@@index([createdBySession])
}
feat(db): schema for ResourceRevision, ResourceProposal, Skill Phase 1 of the Skills + Revisions + Proposals work. Purely additive — no existing rows are touched, no tables renamed, no columns dropped. New tables: - ResourceRevision — append-only audit + diff log keyed by (resourceType, resourceId). Both Prompt and Skill produce revisions on every change. Soft FK so revisions outlive the resources they describe. Indexed for history viewer (latest-first), semver lookup, and cross-resource sync diff via contentHash. - ResourceProposal — generic propose/approve/reject queue. Drop-in replacement for the prompt-only PromptRequest. Created empty here; PR-2 will rename PromptRequest → _PromptRequest_legacy and backfill. - Skill — new resource type that mirrors Prompt for everything CRUD- shaped. Adds `files` Json (multi-file bundles, materialised onto disk by `mcpctl skills sync` in PR-5) and `metadata` Json (typed app-layer in PR-3: hooks, mcpServers, postInstall, …). New columns on Prompt: - semver (semver string, default '0.1.0') — auto-bumped patch on save by PromptService.update once PR-2 wires it. Distinct from `version`, which stays as the optimistic-concurrency counter. - currentRevisionId — soft pointer to the latest ResourceRevision row. DB tests cover scope rules (project XOR agent XOR neither), name uniqueness across both compound keys, cascade-on-delete, soft-FK survival of deletion, and JSON column persistence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 00:18:21 +01:00
// ── ResourceRevision (append-only audit + diff log) ──
//
// Both Prompt and Skill rows produce revisions on every change. Hot reads
// (gate plugin, mcpctl skills sync, prompt index for LLM selection) stay on
// the resource row's inline content; revisions are only consulted by
// history/diff/restore endpoints. Approving a ResourceProposal atomically
// inserts the resource + a revision in the same transaction.
//
// `resourceId` is a soft FK with NO referential constraint — revisions
// outlive the resources they describe so audit history isn't lost when
// a resource is deleted. Validated app-layer.
model ResourceRevision {
id String @id @default(cuid())
// Discriminator: 'prompt' | 'skill'. TEXT, not enum, to make adding a
// third resource type later a non-migration change.
resourceType String
resourceId String
semver String @default("0.1.0")
// sha256 of the canonicalised body — stable diff key. Two revisions with
// the same hash are byte-identical (skills sync uses this to skip work
// even when semver hasn't bumped).
contentHash String
// Snapshot of the resource at this revision: { content, metadata?, ... }
body Json
authorUserId String?
authorSessionId String?
note String @default("")
createdAt DateTime @default(now())
// History viewer: latest-first within a resource.
@@index([resourceType, resourceId, createdAt(sort: Desc)])
// Direct lookup by semver.
@@index([resourceType, resourceId, semver])
// Sync diff cross-resource lookup ("do I already have a revision with
// this contentHash anywhere?") — useful for detecting renames + dedup.
@@index([contentHash])
@@index([authorUserId])
}
// ── ResourceProposal (generic propose/approve/reject queue) ──
//
// A pending change to a Prompt or Skill, submitted by an LLM session via
// the propose_prompt / propose_skill MCP tools (see mcplocal gate plugin)
// or by a human via the web UI / CLI. Reviewers drain the queue via
// `mcpctl review next`. Approving creates the underlying resource (if new)
// and writes a ResourceRevision; the proposal status flips to 'approved'
// and `approvedRevisionId` points at the resulting revision row.
//
// Replaces the prompt-only PromptRequest table; the cutover happens in PR-2
// (rename + backfill + service rewire).
model ResourceProposal {
id String @id @default(cuid())
resourceType String // 'prompt' | 'skill'
name String
// Proposed body — { content, metadata? } shaped per resourceType.
body Json
projectId String?
agentId String?
createdBySession String?
createdByUserId String?
status String @default("pending") // pending | approved | rejected
reviewerNote String @default("")
// Set when status='approved': the ResourceRevision the approval created.
approvedRevisionId String?
// Optimistic-concurrency counter for concurrent reviewer actions.
version Int @default(1)
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
project Project? @relation(fields: [projectId], references: [id], onDelete: Cascade)
agent Agent? @relation(fields: [agentId], references: [id], onDelete: Cascade)
// Mirrors Prompt's two-unique pattern: a (type, name) pair can have one
// proposal per project XOR one per agent. NULL semantics inherit Postgres
// defaults (NULL distinct from NULL); the app layer reuses the `?? ''`
// workaround for direct compound-key lookups, same as Prompt.
@@unique([resourceType, name, projectId])
@@unique([resourceType, name, agentId])
@@index([resourceType, status])
@@index([projectId])
@@index([createdBySession])
}
// ── Audit Events (pipeline/gate/tool trace from mcplocal) ──
model AuditEvent {
id String @id @default(cuid())
timestamp DateTime
sessionId String
projectName String
eventKind String
source String
verified Boolean @default(false)
serverName String?
correlationId String?
parentEventId String?
userName String?
tokenName String?
tokenSha String?
payload Json
createdAt DateTime @default(now())
@@index([sessionId])
@@index([projectName])
@@index([correlationId])
@@index([timestamp])
@@index([eventKind])
@@index([userName])
@@index([tokenSha])
}
// ── Backup Pending Queue ──
model BackupPending {
id String @id @default(cuid())
resourceKind String
resourceName String
action String // 'create' | 'update' | 'delete'
userName String
yamlContent String? @db.Text
createdAt DateTime @default(now())
@@index([createdAt])
}
// ── Agents (LLM personas pinned to a specific Llm) ──
//
// Agents combine a system prompt, a pinned LLM, and (optionally) a project to
// inherit Prompts from. Each Agent is also exposed by mcplocal as a virtual
// MCP server (`agent-<name>/chat`), so other clients can consult it as a tool.
// Per-call LiteLLM-style overrides stack on top of `defaultParams`.
model Agent {
feat(db+mcpd): Agent lifecycle + chat.service kind=virtual branch (v3 Stage 1) Two pieces of v3 plumbing — schema + the latent v1 chat.service bug. Schema (db): - Agent gains kind/providerSessionId/lastHeartbeatAt/status/inactiveSince mirroring Llm's v1 lifecycle. Reuses LlmKind / LlmStatus enums; no new types. Existing rows backfill kind=public/status=active so v1 CRUD is unaffected. - @@index([kind, status]) for the GC sweep, @@index([providerSessionId]) for disconnect-cascade lookups. - 4 new prisma-level tests cover defaults, persisting virtual fields, the (kind, status) GC index, and providerSessionId lookups. Total agent-schema tests: 20/20. chat.service (mcpd) — fixes the v1 latent bug: - LlmView's kind is now plumbed through prepareContext as ctx.llmKind. - Two new private helpers, runOneInference / streamInference, branch on ctx.llmKind: 'public' goes through the existing adapter registry, 'virtual' relays through VirtualLlmService.enqueueInferTask (mirrors the route-handler branch from v1 Stage 3). - Streaming bridges VirtualLlmService's onChunk callback API to an async iterator via a small queue + wake pattern. - ChatService gains an optional virtualLlms constructor parameter; main.ts wires it in. Older test wirings without it raise a clear "virtualLlms dispatcher not wired" error when the row is virtual, rather than silently falling through to the public path against an empty URL. This unblocks any Agent (public OR future v3-virtual) pinned to a kind=virtual Llm. Pre-this-stage, those agents 502'd against the empty url field. Tests: 4 new chat-service-virtual-llm.test.ts cover the relay path non-streaming, streaming, missing-dispatcher error, and rejection surfacing. mcpd suite: 841/841 (was 833, +8 across stages 1+v3-Stage-1). Workspace: 2054/2054 across 153 files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 16:55:02 +01:00
id String @id @default(cuid())
name String @unique
description String @default("") // shown in MCP tools/list
systemPrompt String @default("") @db.Text // agent persona
llmId String
projectId String?
defaultPersonalityId String? // applied at chat time when no --personality flag
proxyModelName String? // optional informational override
feat(db+mcpd): Agent lifecycle + chat.service kind=virtual branch (v3 Stage 1) Two pieces of v3 plumbing — schema + the latent v1 chat.service bug. Schema (db): - Agent gains kind/providerSessionId/lastHeartbeatAt/status/inactiveSince mirroring Llm's v1 lifecycle. Reuses LlmKind / LlmStatus enums; no new types. Existing rows backfill kind=public/status=active so v1 CRUD is unaffected. - @@index([kind, status]) for the GC sweep, @@index([providerSessionId]) for disconnect-cascade lookups. - 4 new prisma-level tests cover defaults, persisting virtual fields, the (kind, status) GC index, and providerSessionId lookups. Total agent-schema tests: 20/20. chat.service (mcpd) — fixes the v1 latent bug: - LlmView's kind is now plumbed through prepareContext as ctx.llmKind. - Two new private helpers, runOneInference / streamInference, branch on ctx.llmKind: 'public' goes through the existing adapter registry, 'virtual' relays through VirtualLlmService.enqueueInferTask (mirrors the route-handler branch from v1 Stage 3). - Streaming bridges VirtualLlmService's onChunk callback API to an async iterator via a small queue + wake pattern. - ChatService gains an optional virtualLlms constructor parameter; main.ts wires it in. Older test wirings without it raise a clear "virtualLlms dispatcher not wired" error when the row is virtual, rather than silently falling through to the public path against an empty URL. This unblocks any Agent (public OR future v3-virtual) pinned to a kind=virtual Llm. Pre-this-stage, those agents 502'd against the empty url field. Tests: 4 new chat-service-virtual-llm.test.ts cover the relay path non-streaming, streaming, missing-dispatcher error, and rejection surfacing. mcpd suite: 841/841 (was 833, +8 across stages 1+v3-Stage-1). Workspace: 2054/2054 across 153 files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 16:55:02 +01:00
defaultParams Json @default("{}") // LiteLLM-style: temperature, top_p, top_k, max_tokens, stop, ...
extras Json @default("{}") // future LoRA / tool-allowlist
// ── Virtual-agent lifecycle (NULL/default for kind=public, mirrors Llm) ──
kind LlmKind @default(public)
providerSessionId String? // mcplocal session that owns this row when virtual
lastHeartbeatAt DateTime?
status LlmStatus @default(active)
inactiveSince DateTime?
ownerId String
feat(mcpd+db): visibility scope + ownership for Llms and Agents (v7 Stage 1) Adds the schema + service-layer machinery for per-user RBAC scoping of virtual Llms and Agents. Without this, anyone with `view:llms` sees every other user's published model — fine for a single-user homelab, wrong for org use where workstation-published models or paid keys aren't meant to be broadcast. Schema: - Llm: new `ownerId String?` + `visibility String @default("public")`. NULL ownerId on legacy rows is treated as public for back-compat. - Agent: `visibility String @default("public")` (Agent already has `ownerId`, required). - Composite index `(visibility, ownerId)` on both tables for the list-filter hot path. - Migration backfills both columns to 'public' so pre-v7 setups behave identically post-deploy. Service layer: - New `Viewer` / `AgentViewer` shape: `{ userId, wildcard, allowedNames }`. The route layer computes this from `request.userId` + `RbacService.getAllowedScope` and passes it down. NULL viewer = skip the filter (internal callers — cron sweeps, audit, tests). - `isLlmVisibleTo` / `isAgentVisibleTo` pure predicates encode the decision tree: visibility=public → visible (RBAC layer above already passed) viewer=null OR wildcard → visible ownerId === viewer.userId → visible row.name in viewer.allowedNames → visible else → hidden - LlmService.list/getById/getByName + AgentService equivalents accept an optional Viewer arg and apply the predicate. Get-style methods 404 (not 403) on hidden rows so name enumeration via differential status is impossible. Repositories: CreateInput/UpdateInput types gained `ownerId`/ `visibility` (Llm) and `visibility` (Agent). Update is in place; ownerId is set-once at create time. Tests: - 13 unit tests on the predicate covering every branch (null viewer, public, wildcard, owner, name-scoped grant, foreign private, legacy null-ownerId). - mcpd 908/908 (was 893; +15 across the merge windows + this PR). Stage 2 (next): route plumbing — every list/get endpoint needs to build the Viewer from the request and pass it through. mcplocal virtuals default to visibility=private on register. CLI adds a VISIBILITY column and a --visibility flag. yaml round-trip preserves the field.
2026-04-29 00:46:06 +01:00
// v7: per-user RBAC scoping. Mirrors `Llm.visibility` semantics —
// 'public' (default, today's behavior) lets anyone with the resource
// grant see the agent; 'private' restricts to owner + explicit
// name-scoped RBAC bindings. mcplocal-published virtual agents
// default to 'private' on register so a workstation-published persona
// isn't broadcast to the whole org until shared explicitly. Existing
// rows backfill to 'public' so pre-v7 setups keep working unchanged.
visibility String @default("public")
feat(db+mcpd): Agent lifecycle + chat.service kind=virtual branch (v3 Stage 1) Two pieces of v3 plumbing — schema + the latent v1 chat.service bug. Schema (db): - Agent gains kind/providerSessionId/lastHeartbeatAt/status/inactiveSince mirroring Llm's v1 lifecycle. Reuses LlmKind / LlmStatus enums; no new types. Existing rows backfill kind=public/status=active so v1 CRUD is unaffected. - @@index([kind, status]) for the GC sweep, @@index([providerSessionId]) for disconnect-cascade lookups. - 4 new prisma-level tests cover defaults, persisting virtual fields, the (kind, status) GC index, and providerSessionId lookups. Total agent-schema tests: 20/20. chat.service (mcpd) — fixes the v1 latent bug: - LlmView's kind is now plumbed through prepareContext as ctx.llmKind. - Two new private helpers, runOneInference / streamInference, branch on ctx.llmKind: 'public' goes through the existing adapter registry, 'virtual' relays through VirtualLlmService.enqueueInferTask (mirrors the route-handler branch from v1 Stage 3). - Streaming bridges VirtualLlmService's onChunk callback API to an async iterator via a small queue + wake pattern. - ChatService gains an optional virtualLlms constructor parameter; main.ts wires it in. Older test wirings without it raise a clear "virtualLlms dispatcher not wired" error when the row is virtual, rather than silently falling through to the public path against an empty URL. This unblocks any Agent (public OR future v3-virtual) pinned to a kind=virtual Llm. Pre-this-stage, those agents 502'd against the empty url field. Tests: 4 new chat-service-virtual-llm.test.ts cover the relay path non-streaming, streaming, missing-dispatcher error, and rejection surfacing. mcpd suite: 841/841 (was 833, +8 across stages 1+v3-Stage-1). Workspace: 2054/2054 across 153 files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 16:55:02 +01:00
version Int @default(1)
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
feat(db): schema for ResourceRevision, ResourceProposal, Skill Phase 1 of the Skills + Revisions + Proposals work. Purely additive — no existing rows are touched, no tables renamed, no columns dropped. New tables: - ResourceRevision — append-only audit + diff log keyed by (resourceType, resourceId). Both Prompt and Skill produce revisions on every change. Soft FK so revisions outlive the resources they describe. Indexed for history viewer (latest-first), semver lookup, and cross-resource sync diff via contentHash. - ResourceProposal — generic propose/approve/reject queue. Drop-in replacement for the prompt-only PromptRequest. Created empty here; PR-2 will rename PromptRequest → _PromptRequest_legacy and backfill. - Skill — new resource type that mirrors Prompt for everything CRUD- shaped. Adds `files` Json (multi-file bundles, materialised onto disk by `mcpctl skills sync` in PR-5) and `metadata` Json (typed app-layer in PR-3: hooks, mcpServers, postInstall, …). New columns on Prompt: - semver (semver string, default '0.1.0') — auto-bumped patch on save by PromptService.update once PR-2 wires it. Distinct from `version`, which stays as the optimistic-concurrency counter. - currentRevisionId — soft pointer to the latest ResourceRevision row. DB tests cover scope rules (project XOR agent XOR neither), name uniqueness across both compound keys, cascade-on-delete, soft-FK survival of deletion, and JSON column persistence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 00:18:21 +01:00
llm Llm @relation(fields: [llmId], references: [id], onDelete: Restrict)
project Project? @relation(fields: [projectId], references: [id], onDelete: SetNull)
owner User @relation(fields: [ownerId], references: [id], onDelete: Cascade)
threads ChatThread[]
prompts Prompt[]
feat(db): schema for ResourceRevision, ResourceProposal, Skill Phase 1 of the Skills + Revisions + Proposals work. Purely additive — no existing rows are touched, no tables renamed, no columns dropped. New tables: - ResourceRevision — append-only audit + diff log keyed by (resourceType, resourceId). Both Prompt and Skill produce revisions on every change. Soft FK so revisions outlive the resources they describe. Indexed for history viewer (latest-first), semver lookup, and cross-resource sync diff via contentHash. - ResourceProposal — generic propose/approve/reject queue. Drop-in replacement for the prompt-only PromptRequest. Created empty here; PR-2 will rename PromptRequest → _PromptRequest_legacy and backfill. - Skill — new resource type that mirrors Prompt for everything CRUD- shaped. Adds `files` Json (multi-file bundles, materialised onto disk by `mcpctl skills sync` in PR-5) and `metadata` Json (typed app-layer in PR-3: hooks, mcpServers, postInstall, …). New columns on Prompt: - semver (semver string, default '0.1.0') — auto-bumped patch on save by PromptService.update once PR-2 wires it. Distinct from `version`, which stays as the optimistic-concurrency counter. - currentRevisionId — soft pointer to the latest ResourceRevision row. DB tests cover scope rules (project XOR agent XOR neither), name uniqueness across both compound keys, cascade-on-delete, soft-FK survival of deletion, and JSON column persistence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 00:18:21 +01:00
proposals ResourceProposal[]
skills Skill[]
personalities Personality[] @relation("AgentPersonalities")
defaultPersonality Personality? @relation("AgentDefaultPersonality", fields: [defaultPersonalityId], references: [id], onDelete: SetNull)
@@index([name])
@@index([llmId])
@@index([projectId])
@@index([ownerId])
@@index([defaultPersonalityId])
feat(db+mcpd): Agent lifecycle + chat.service kind=virtual branch (v3 Stage 1) Two pieces of v3 plumbing — schema + the latent v1 chat.service bug. Schema (db): - Agent gains kind/providerSessionId/lastHeartbeatAt/status/inactiveSince mirroring Llm's v1 lifecycle. Reuses LlmKind / LlmStatus enums; no new types. Existing rows backfill kind=public/status=active so v1 CRUD is unaffected. - @@index([kind, status]) for the GC sweep, @@index([providerSessionId]) for disconnect-cascade lookups. - 4 new prisma-level tests cover defaults, persisting virtual fields, the (kind, status) GC index, and providerSessionId lookups. Total agent-schema tests: 20/20. chat.service (mcpd) — fixes the v1 latent bug: - LlmView's kind is now plumbed through prepareContext as ctx.llmKind. - Two new private helpers, runOneInference / streamInference, branch on ctx.llmKind: 'public' goes through the existing adapter registry, 'virtual' relays through VirtualLlmService.enqueueInferTask (mirrors the route-handler branch from v1 Stage 3). - Streaming bridges VirtualLlmService's onChunk callback API to an async iterator via a small queue + wake pattern. - ChatService gains an optional virtualLlms constructor parameter; main.ts wires it in. Older test wirings without it raise a clear "virtualLlms dispatcher not wired" error when the row is virtual, rather than silently falling through to the public path against an empty URL. This unblocks any Agent (public OR future v3-virtual) pinned to a kind=virtual Llm. Pre-this-stage, those agents 502'd against the empty url field. Tests: 4 new chat-service-virtual-llm.test.ts cover the relay path non-streaming, streaming, missing-dispatcher error, and rejection surfacing. mcpd suite: 841/841 (was 833, +8 across stages 1+v3-Stage-1). Workspace: 2054/2054 across 153 files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 16:55:02 +01:00
@@index([kind, status])
@@index([providerSessionId])
feat(mcpd+db): visibility scope + ownership for Llms and Agents (v7 Stage 1) Adds the schema + service-layer machinery for per-user RBAC scoping of virtual Llms and Agents. Without this, anyone with `view:llms` sees every other user's published model — fine for a single-user homelab, wrong for org use where workstation-published models or paid keys aren't meant to be broadcast. Schema: - Llm: new `ownerId String?` + `visibility String @default("public")`. NULL ownerId on legacy rows is treated as public for back-compat. - Agent: `visibility String @default("public")` (Agent already has `ownerId`, required). - Composite index `(visibility, ownerId)` on both tables for the list-filter hot path. - Migration backfills both columns to 'public' so pre-v7 setups behave identically post-deploy. Service layer: - New `Viewer` / `AgentViewer` shape: `{ userId, wildcard, allowedNames }`. The route layer computes this from `request.userId` + `RbacService.getAllowedScope` and passes it down. NULL viewer = skip the filter (internal callers — cron sweeps, audit, tests). - `isLlmVisibleTo` / `isAgentVisibleTo` pure predicates encode the decision tree: visibility=public → visible (RBAC layer above already passed) viewer=null OR wildcard → visible ownerId === viewer.userId → visible row.name in viewer.allowedNames → visible else → hidden - LlmService.list/getById/getByName + AgentService equivalents accept an optional Viewer arg and apply the predicate. Get-style methods 404 (not 403) on hidden rows so name enumeration via differential status is impossible. Repositories: CreateInput/UpdateInput types gained `ownerId`/ `visibility` (Llm) and `visibility` (Agent). Update is in place; ownerId is set-once at create time. Tests: - 13 unit tests on the predicate covering every branch (null viewer, public, wildcard, owner, name-scoped grant, foreign private, legacy null-ownerId). - mcpd 908/908 (was 893; +15 across the merge windows + this PR). Stage 2 (next): route plumbing — every list/get endpoint needs to build the Viewer from the request and pass it through. mcplocal virtuals default to visibility=private on register. CLI adds a VISIBILITY column and a --visibility flag. yaml round-trip preserves the field.
2026-04-29 00:46:06 +01:00
@@index([visibility, ownerId])
}
// ── Personalities (named overlay bundles of prompts on top of an Agent) ──
//
// VLAN-on-ethernet semantics: an Agent works without a Personality (today's
// flow). Selecting a Personality adds its bound Prompts to the system block
// after the Agent's own systemPrompt, agent-direct prompts, and project
// prompts — additive, never replacing.
model Personality {
id String @id @default(cuid())
name String
description String @default("")
agentId String
priority Int @default(5)
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
agent Agent @relation("AgentPersonalities", fields: [agentId], references: [id], onDelete: Cascade)
prompts PersonalityPrompt[]
defaultForAgent Agent[] @relation("AgentDefaultPersonality")
@@unique([name, agentId])
@@index([agentId])
}
// ── Personality ↔ Prompt join table ──
//
// A Prompt can be bound to many Personalities; a Personality can bind many
// Prompts. The `priority` here overrides the Prompt's own priority *within
// this Personality's overlay slice* (so the same prompt can be high-priority
// in one personality and low in another).
model PersonalityPrompt {
id String @id @default(cuid())
personalityId String
promptId String
priority Int @default(5)
createdAt DateTime @default(now())
personality Personality @relation(fields: [personalityId], references: [id], onDelete: Cascade)
prompt Prompt @relation(fields: [promptId], references: [id], onDelete: Cascade)
@@unique([personalityId, promptId])
@@index([personalityId])
@@index([promptId])
}
// ── Chat Threads (persisted conversation per Agent) ──
model ChatThread {
id String @id @default(cuid())
feat(chat): project-scoped chat — `mcpctl chat --project <name>` Chat directly with a Project (no Agent needed): its Prompts become the system context, its MCP-server tools are callable, its llmProvider/llmModel drive the LLM, and (opt-in) the model can read secret values. History is saved inside the project, attributed per user, resumable, and deletable (RBAC-permitting) — "use it like Claude, scoped to the project". Backend (reuses the agent-chat orchestrator): - ChatThread is now agent-XOR-project (schema + migration + CHECK constraint); new listThreadsByProject / deleteThread on the repo. - ChatService: prepareProjectContext (project prompt + Prompts by priority, llm from llmProvider with llmModel override, project tools), shared runChatLoop/runChatStreamLoop, project thread CRUD with owner enforcement (404-not-403 on foreign threads), admin-override delete. - Gated get_secret virtual tool: offered only with --allow-secrets AND the caller's view:secrets; resolves via SecretService, never routes to a server. - routes/project-chat.ts (chat SSE+non-stream, threads create/list/delete); RBAC run:projects:<name>. CLI: - `mcpctl chat --project <name>` (+ --allow-secrets), one-shot/REPL/resume. - REPL /threads, /resume <id>, /delete <id>; project-aware header + /tools. - `mcpctl get threads --project <name>`, `mcpctl delete thread <id> --project`. - completions regenerated (--project completes project names). Tests: 8 new project-chat unit tests; full mcpd (945) + CLI (508) green; schema validated against Postgres. Docs: docs/chat.md "Project chat" section. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 10:34:21 +01:00
// A thread is scoped to an Agent XOR a Project (enforced by a CHECK
// constraint in the migration + the service layer). Project-scoped threads
// back `mcpctl chat --project <name>`.
agentId String?
projectId String?
ownerId String
title String @default("")
lastTurnAt DateTime @default(now())
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
feat(chat): project-scoped chat — `mcpctl chat --project <name>` Chat directly with a Project (no Agent needed): its Prompts become the system context, its MCP-server tools are callable, its llmProvider/llmModel drive the LLM, and (opt-in) the model can read secret values. History is saved inside the project, attributed per user, resumable, and deletable (RBAC-permitting) — "use it like Claude, scoped to the project". Backend (reuses the agent-chat orchestrator): - ChatThread is now agent-XOR-project (schema + migration + CHECK constraint); new listThreadsByProject / deleteThread on the repo. - ChatService: prepareProjectContext (project prompt + Prompts by priority, llm from llmProvider with llmModel override, project tools), shared runChatLoop/runChatStreamLoop, project thread CRUD with owner enforcement (404-not-403 on foreign threads), admin-override delete. - Gated get_secret virtual tool: offered only with --allow-secrets AND the caller's view:secrets; resolves via SecretService, never routes to a server. - routes/project-chat.ts (chat SSE+non-stream, threads create/list/delete); RBAC run:projects:<name>. CLI: - `mcpctl chat --project <name>` (+ --allow-secrets), one-shot/REPL/resume. - REPL /threads, /resume <id>, /delete <id>; project-aware header + /tools. - `mcpctl get threads --project <name>`, `mcpctl delete thread <id> --project`. - completions regenerated (--project completes project names). Tests: 8 new project-chat unit tests; full mcpd (945) + CLI (508) green; schema validated against Postgres. Docs: docs/chat.md "Project chat" section. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 10:34:21 +01:00
agent Agent? @relation(fields: [agentId], references: [id], onDelete: Cascade)
project Project? @relation(fields: [projectId], references: [id], onDelete: Cascade)
owner User @relation(fields: [ownerId], references: [id], onDelete: Cascade)
messages ChatMessage[]
@@index([agentId, lastTurnAt(sort: Desc)])
feat(chat): project-scoped chat — `mcpctl chat --project <name>` Chat directly with a Project (no Agent needed): its Prompts become the system context, its MCP-server tools are callable, its llmProvider/llmModel drive the LLM, and (opt-in) the model can read secret values. History is saved inside the project, attributed per user, resumable, and deletable (RBAC-permitting) — "use it like Claude, scoped to the project". Backend (reuses the agent-chat orchestrator): - ChatThread is now agent-XOR-project (schema + migration + CHECK constraint); new listThreadsByProject / deleteThread on the repo. - ChatService: prepareProjectContext (project prompt + Prompts by priority, llm from llmProvider with llmModel override, project tools), shared runChatLoop/runChatStreamLoop, project thread CRUD with owner enforcement (404-not-403 on foreign threads), admin-override delete. - Gated get_secret virtual tool: offered only with --allow-secrets AND the caller's view:secrets; resolves via SecretService, never routes to a server. - routes/project-chat.ts (chat SSE+non-stream, threads create/list/delete); RBAC run:projects:<name>. CLI: - `mcpctl chat --project <name>` (+ --allow-secrets), one-shot/REPL/resume. - REPL /threads, /resume <id>, /delete <id>; project-aware header + /tools. - `mcpctl get threads --project <name>`, `mcpctl delete thread <id> --project`. - completions regenerated (--project completes project names). Tests: 8 new project-chat unit tests; full mcpd (945) + CLI (508) green; schema validated against Postgres. Docs: docs/chat.md "Project chat" section. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 10:34:21 +01:00
@@index([projectId, lastTurnAt(sort: Desc)])
@@index([ownerId])
}
// ── Chat Messages ──
//
// `turnIndex` is monotonic per thread; the @@unique enforces ordering even
// under racing appends (callers retry on collision). `status` stays `pending`
// until the orchestrator confirms the turn completed successfully.
model ChatMessage {
id String @id @default(cuid())
threadId String
turnIndex Int
role String // 'system' | 'user' | 'assistant' | 'tool'
content String @db.Text
toolCalls Json? // assistant turn: [{id,name,arguments}]
toolCallId String? // tool turn: which call this answers
status String @default("complete") // 'pending' | 'complete' | 'error'
createdAt DateTime @default(now())
thread ChatThread @relation(fields: [threadId], references: [id], onDelete: Cascade)
@@unique([threadId, turnIndex])
@@index([threadId, createdAt])
}
feat(mcpd+db): durable InferenceTask queue + state machine (v5 Stage 1) The persistence + signaling layer for v5. No integration with the existing in-flight inference path yet — that's Stage 2. This commit just lands the durable queue underneath, with a state machine that mcpd's HTTP handlers, the worker result-POST route, and the GC sweep will all build on. Schema (src/db/prisma/schema.prisma + migration): - New `InferenceTask` model + `InferenceTaskStatus` enum (pending|claimed|running|completed|error|cancelled). - Routing fields stored at enqueue time so a later rename of `Llm.poolName` doesn't reroute already-queued work: `poolName` (effective pool key), `llmName` (pinned target), `model`, `tier`. - Worker tracking: `claimedBy` (providerSessionId) + `claimedAt`, cleared on revert. - Bodies as `Json`: requestBody (always set), responseBody (set at completion). Streaming chunks are NOT persisted — too expensive at delta granularity. The final assembled body lands once per task. - Lifecycle timestamps: createdAt, claimedAt, streamStartedAt, completedAt. Plus ownerId (RBAC + audit) and agentId (null for direct chat-llm calls). - Indexes for the hot paths: (status, poolName) for the dispatcher's drain query, claimedBy for the disconnect revert, completedAt for the GC retention sweep, owner/agent for the async API listing. Repository (src/mcpd/src/repositories/inference-task.repository.ts): - CRUD + state transitions as conditional CAS via `updateMany`. Two workers racing to claim the same row both run the UPDATE; whichever the DB serializes first sees affected=1 and gets the row, the loser sees 0 and falls through to the next candidate. No application- level locking required. - findPendingForPools(poolNames[]) for the worker drain on bind. - findHeldBy(claimedBy) for the unbindSession revert. - findStalePending + findExpiredTerminal for the GC sweep. Service (src/mcpd/src/services/inference-task.service.ts): - Owns the in-process EventEmitter that wakes blocked HTTP handlers when a worker POSTs results. The DB row is the source of truth for *state*; the EventEmitter just signals "go re-read row X" so we don't have to poll. Single-instance assumption for v5; pg LISTEN/NOTIFY is the v6 swap when scaling horizontally — no schema change needed, just replace the emitter wakeup. - waitFor(taskId, timeoutMs) returns { done, chunks }: the terminal promise + an async iterator of streaming deltas. Throws on cancel (clear message) or error (worker's errorMessage propagates) or timeout. Polls the row once at subscribe time so an already- terminal task resolves immediately without waiting for an event that's never coming. - gcSweep flips stale pending rows to error (with a clear message about the timeout) and deletes terminal rows past retention. Defaults: 1h pending timeout, 7d terminal retention; both configurable. Tests: - 6 db-level schema tests (defaults, json roundtrip, drain query shape, claimedBy filter, GC predicate, agentId nullable). - 13 service tests covering enqueue, the CAS race on tryClaim, complete/fail/cancel, idempotent terminal transitions, revertHeldBy on disconnect, and the full waitFor signal lifecycle (immediate resolve, wake on event, chunk streaming, cancel/error/timeout paths). Plus a gcSweep test with a fixed clock. mcpd 881/881 (was 868; +13). db pool-schema 14/14, +6 new inference-task-schema. Pre-existing failures in models.test.ts (Secret FK fixture issue, also fails on main HEAD) are unrelated. Stage 2 (next): VirtualLlmService rewires through this — remove the in-memory pendingTasks map; enqueue creates a row, dispatch picks an active session, the result-route updates the row + emits the wakeup. Worker disconnect reverts; worker bind drains.
2026-04-28 02:14:45 +01:00
// ── Inference Tasks (v5) ──
//
// Every inference call (sync infer, agent chat, async POST /inference-tasks)
// creates a row here. The DB is the source of truth; mcpd's previous
// in-memory `pendingTasks` map is gone — the result-handler updates the row
// and an in-process EventEmitter wakes any blocked HTTP handlers (single-
// instance for now; multi-instance scaling is a v6 concern that will swap
// the emitter for pg LISTEN/NOTIFY without changing the data model).
//
// Routing: `poolName` is the effective pool key at enqueue time
// (`Llm.poolName ?? Llm.name`). Workers (mcplocal sessions) drain pending
// rows whose `poolName` matches the pool keys they own when they bind their
// SSE channel — that's how queued tasks survive worker offline windows.
enum InferenceTaskStatus {
feat(db): schema for ResourceRevision, ResourceProposal, Skill Phase 1 of the Skills + Revisions + Proposals work. Purely additive — no existing rows are touched, no tables renamed, no columns dropped. New tables: - ResourceRevision — append-only audit + diff log keyed by (resourceType, resourceId). Both Prompt and Skill produce revisions on every change. Soft FK so revisions outlive the resources they describe. Indexed for history viewer (latest-first), semver lookup, and cross-resource sync diff via contentHash. - ResourceProposal — generic propose/approve/reject queue. Drop-in replacement for the prompt-only PromptRequest. Created empty here; PR-2 will rename PromptRequest → _PromptRequest_legacy and backfill. - Skill — new resource type that mirrors Prompt for everything CRUD- shaped. Adds `files` Json (multi-file bundles, materialised onto disk by `mcpctl skills sync` in PR-5) and `metadata` Json (typed app-layer in PR-3: hooks, mcpServers, postInstall, …). New columns on Prompt: - semver (semver string, default '0.1.0') — auto-bumped patch on save by PromptService.update once PR-2 wires it. Distinct from `version`, which stays as the optimistic-concurrency counter. - currentRevisionId — soft pointer to the latest ResourceRevision row. DB tests cover scope rules (project XOR agent XOR neither), name uniqueness across both compound keys, cascade-on-delete, soft-FK survival of deletion, and JSON column persistence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 00:18:21 +01:00
pending // in queue, no worker has it yet (or claim was reverted)
claimed // a worker has it (SSE frame sent), no chunks back yet
running // worker started streaming chunks back (streaming tasks only)
completed // worker POSTed the final result
error // permanent failure (auth, bad request, queue timeout)
cancelled // caller said never mind via DELETE
feat(mcpd+db): durable InferenceTask queue + state machine (v5 Stage 1) The persistence + signaling layer for v5. No integration with the existing in-flight inference path yet — that's Stage 2. This commit just lands the durable queue underneath, with a state machine that mcpd's HTTP handlers, the worker result-POST route, and the GC sweep will all build on. Schema (src/db/prisma/schema.prisma + migration): - New `InferenceTask` model + `InferenceTaskStatus` enum (pending|claimed|running|completed|error|cancelled). - Routing fields stored at enqueue time so a later rename of `Llm.poolName` doesn't reroute already-queued work: `poolName` (effective pool key), `llmName` (pinned target), `model`, `tier`. - Worker tracking: `claimedBy` (providerSessionId) + `claimedAt`, cleared on revert. - Bodies as `Json`: requestBody (always set), responseBody (set at completion). Streaming chunks are NOT persisted — too expensive at delta granularity. The final assembled body lands once per task. - Lifecycle timestamps: createdAt, claimedAt, streamStartedAt, completedAt. Plus ownerId (RBAC + audit) and agentId (null for direct chat-llm calls). - Indexes for the hot paths: (status, poolName) for the dispatcher's drain query, claimedBy for the disconnect revert, completedAt for the GC retention sweep, owner/agent for the async API listing. Repository (src/mcpd/src/repositories/inference-task.repository.ts): - CRUD + state transitions as conditional CAS via `updateMany`. Two workers racing to claim the same row both run the UPDATE; whichever the DB serializes first sees affected=1 and gets the row, the loser sees 0 and falls through to the next candidate. No application- level locking required. - findPendingForPools(poolNames[]) for the worker drain on bind. - findHeldBy(claimedBy) for the unbindSession revert. - findStalePending + findExpiredTerminal for the GC sweep. Service (src/mcpd/src/services/inference-task.service.ts): - Owns the in-process EventEmitter that wakes blocked HTTP handlers when a worker POSTs results. The DB row is the source of truth for *state*; the EventEmitter just signals "go re-read row X" so we don't have to poll. Single-instance assumption for v5; pg LISTEN/NOTIFY is the v6 swap when scaling horizontally — no schema change needed, just replace the emitter wakeup. - waitFor(taskId, timeoutMs) returns { done, chunks }: the terminal promise + an async iterator of streaming deltas. Throws on cancel (clear message) or error (worker's errorMessage propagates) or timeout. Polls the row once at subscribe time so an already- terminal task resolves immediately without waiting for an event that's never coming. - gcSweep flips stale pending rows to error (with a clear message about the timeout) and deletes terminal rows past retention. Defaults: 1h pending timeout, 7d terminal retention; both configurable. Tests: - 6 db-level schema tests (defaults, json roundtrip, drain query shape, claimedBy filter, GC predicate, agentId nullable). - 13 service tests covering enqueue, the CAS race on tryClaim, complete/fail/cancel, idempotent terminal transitions, revertHeldBy on disconnect, and the full waitFor signal lifecycle (immediate resolve, wake on event, chunk streaming, cancel/error/timeout paths). Plus a gcSweep test with a fixed clock. mcpd 881/881 (was 868; +13). db pool-schema 14/14, +6 new inference-task-schema. Pre-existing failures in models.test.ts (Secret FK fixture issue, also fails on main HEAD) are unrelated. Stage 2 (next): VirtualLlmService rewires through this — remove the in-memory pendingTasks map; enqueue creates a row, dispatch picks an active session, the result-route updates the row + emits the wakeup. Worker disconnect reverts; worker bind drains.
2026-04-28 02:14:45 +01:00
}
model InferenceTask {
feat(db): schema for ResourceRevision, ResourceProposal, Skill Phase 1 of the Skills + Revisions + Proposals work. Purely additive — no existing rows are touched, no tables renamed, no columns dropped. New tables: - ResourceRevision — append-only audit + diff log keyed by (resourceType, resourceId). Both Prompt and Skill produce revisions on every change. Soft FK so revisions outlive the resources they describe. Indexed for history viewer (latest-first), semver lookup, and cross-resource sync diff via contentHash. - ResourceProposal — generic propose/approve/reject queue. Drop-in replacement for the prompt-only PromptRequest. Created empty here; PR-2 will rename PromptRequest → _PromptRequest_legacy and backfill. - Skill — new resource type that mirrors Prompt for everything CRUD- shaped. Adds `files` Json (multi-file bundles, materialised onto disk by `mcpctl skills sync` in PR-5) and `metadata` Json (typed app-layer in PR-3: hooks, mcpServers, postInstall, …). New columns on Prompt: - semver (semver string, default '0.1.0') — auto-bumped patch on save by PromptService.update once PR-2 wires it. Distinct from `version`, which stays as the optimistic-concurrency counter. - currentRevisionId — soft pointer to the latest ResourceRevision row. DB tests cover scope rules (project XOR agent XOR neither), name uniqueness across both compound keys, cascade-on-delete, soft-FK survival of deletion, and JSON column persistence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 00:18:21 +01:00
id String @id @default(cuid())
status InferenceTaskStatus @default(pending)
feat(mcpd+db): durable InferenceTask queue + state machine (v5 Stage 1) The persistence + signaling layer for v5. No integration with the existing in-flight inference path yet — that's Stage 2. This commit just lands the durable queue underneath, with a state machine that mcpd's HTTP handlers, the worker result-POST route, and the GC sweep will all build on. Schema (src/db/prisma/schema.prisma + migration): - New `InferenceTask` model + `InferenceTaskStatus` enum (pending|claimed|running|completed|error|cancelled). - Routing fields stored at enqueue time so a later rename of `Llm.poolName` doesn't reroute already-queued work: `poolName` (effective pool key), `llmName` (pinned target), `model`, `tier`. - Worker tracking: `claimedBy` (providerSessionId) + `claimedAt`, cleared on revert. - Bodies as `Json`: requestBody (always set), responseBody (set at completion). Streaming chunks are NOT persisted — too expensive at delta granularity. The final assembled body lands once per task. - Lifecycle timestamps: createdAt, claimedAt, streamStartedAt, completedAt. Plus ownerId (RBAC + audit) and agentId (null for direct chat-llm calls). - Indexes for the hot paths: (status, poolName) for the dispatcher's drain query, claimedBy for the disconnect revert, completedAt for the GC retention sweep, owner/agent for the async API listing. Repository (src/mcpd/src/repositories/inference-task.repository.ts): - CRUD + state transitions as conditional CAS via `updateMany`. Two workers racing to claim the same row both run the UPDATE; whichever the DB serializes first sees affected=1 and gets the row, the loser sees 0 and falls through to the next candidate. No application- level locking required. - findPendingForPools(poolNames[]) for the worker drain on bind. - findHeldBy(claimedBy) for the unbindSession revert. - findStalePending + findExpiredTerminal for the GC sweep. Service (src/mcpd/src/services/inference-task.service.ts): - Owns the in-process EventEmitter that wakes blocked HTTP handlers when a worker POSTs results. The DB row is the source of truth for *state*; the EventEmitter just signals "go re-read row X" so we don't have to poll. Single-instance assumption for v5; pg LISTEN/NOTIFY is the v6 swap when scaling horizontally — no schema change needed, just replace the emitter wakeup. - waitFor(taskId, timeoutMs) returns { done, chunks }: the terminal promise + an async iterator of streaming deltas. Throws on cancel (clear message) or error (worker's errorMessage propagates) or timeout. Polls the row once at subscribe time so an already- terminal task resolves immediately without waiting for an event that's never coming. - gcSweep flips stale pending rows to error (with a clear message about the timeout) and deletes terminal rows past retention. Defaults: 1h pending timeout, 7d terminal retention; both configurable. Tests: - 6 db-level schema tests (defaults, json roundtrip, drain query shape, claimedBy filter, GC predicate, agentId nullable). - 13 service tests covering enqueue, the CAS race on tryClaim, complete/fail/cancel, idempotent terminal transitions, revertHeldBy on disconnect, and the full waitFor signal lifecycle (immediate resolve, wake on event, chunk streaming, cancel/error/timeout paths). Plus a gcSweep test with a fixed clock. mcpd 881/881 (was 868; +13). db pool-schema 14/14, +6 new inference-task-schema. Pre-existing failures in models.test.ts (Secret FK fixture issue, also fails on main HEAD) are unrelated. Stage 2 (next): VirtualLlmService rewires through this — remove the in-memory pendingTasks map; enqueue creates a row, dispatch picks an active session, the result-route updates the row + emits the wakeup. Worker disconnect reverts; worker bind drains.
2026-04-28 02:14:45 +01:00
// Routing — pool key drives worker matching at claim time. Stored at
// enqueue time so a later rename of Llm.poolName doesn't reroute
// already-queued work.
feat(db): schema for ResourceRevision, ResourceProposal, Skill Phase 1 of the Skills + Revisions + Proposals work. Purely additive — no existing rows are touched, no tables renamed, no columns dropped. New tables: - ResourceRevision — append-only audit + diff log keyed by (resourceType, resourceId). Both Prompt and Skill produce revisions on every change. Soft FK so revisions outlive the resources they describe. Indexed for history viewer (latest-first), semver lookup, and cross-resource sync diff via contentHash. - ResourceProposal — generic propose/approve/reject queue. Drop-in replacement for the prompt-only PromptRequest. Created empty here; PR-2 will rename PromptRequest → _PromptRequest_legacy and backfill. - Skill — new resource type that mirrors Prompt for everything CRUD- shaped. Adds `files` Json (multi-file bundles, materialised onto disk by `mcpctl skills sync` in PR-5) and `metadata` Json (typed app-layer in PR-3: hooks, mcpServers, postInstall, …). New columns on Prompt: - semver (semver string, default '0.1.0') — auto-bumped patch on save by PromptService.update once PR-2 wires it. Distinct from `version`, which stays as the optimistic-concurrency counter. - currentRevisionId — soft pointer to the latest ResourceRevision row. DB tests cover scope rules (project XOR agent XOR neither), name uniqueness across both compound keys, cascade-on-delete, soft-FK survival of deletion, and JSON column persistence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 00:18:21 +01:00
poolName String
llmName String // pinned target Llm name (for audit + agent backref)
model String
tier String?
feat(mcpd+db): durable InferenceTask queue + state machine (v5 Stage 1) The persistence + signaling layer for v5. No integration with the existing in-flight inference path yet — that's Stage 2. This commit just lands the durable queue underneath, with a state machine that mcpd's HTTP handlers, the worker result-POST route, and the GC sweep will all build on. Schema (src/db/prisma/schema.prisma + migration): - New `InferenceTask` model + `InferenceTaskStatus` enum (pending|claimed|running|completed|error|cancelled). - Routing fields stored at enqueue time so a later rename of `Llm.poolName` doesn't reroute already-queued work: `poolName` (effective pool key), `llmName` (pinned target), `model`, `tier`. - Worker tracking: `claimedBy` (providerSessionId) + `claimedAt`, cleared on revert. - Bodies as `Json`: requestBody (always set), responseBody (set at completion). Streaming chunks are NOT persisted — too expensive at delta granularity. The final assembled body lands once per task. - Lifecycle timestamps: createdAt, claimedAt, streamStartedAt, completedAt. Plus ownerId (RBAC + audit) and agentId (null for direct chat-llm calls). - Indexes for the hot paths: (status, poolName) for the dispatcher's drain query, claimedBy for the disconnect revert, completedAt for the GC retention sweep, owner/agent for the async API listing. Repository (src/mcpd/src/repositories/inference-task.repository.ts): - CRUD + state transitions as conditional CAS via `updateMany`. Two workers racing to claim the same row both run the UPDATE; whichever the DB serializes first sees affected=1 and gets the row, the loser sees 0 and falls through to the next candidate. No application- level locking required. - findPendingForPools(poolNames[]) for the worker drain on bind. - findHeldBy(claimedBy) for the unbindSession revert. - findStalePending + findExpiredTerminal for the GC sweep. Service (src/mcpd/src/services/inference-task.service.ts): - Owns the in-process EventEmitter that wakes blocked HTTP handlers when a worker POSTs results. The DB row is the source of truth for *state*; the EventEmitter just signals "go re-read row X" so we don't have to poll. Single-instance assumption for v5; pg LISTEN/NOTIFY is the v6 swap when scaling horizontally — no schema change needed, just replace the emitter wakeup. - waitFor(taskId, timeoutMs) returns { done, chunks }: the terminal promise + an async iterator of streaming deltas. Throws on cancel (clear message) or error (worker's errorMessage propagates) or timeout. Polls the row once at subscribe time so an already- terminal task resolves immediately without waiting for an event that's never coming. - gcSweep flips stale pending rows to error (with a clear message about the timeout) and deletes terminal rows past retention. Defaults: 1h pending timeout, 7d terminal retention; both configurable. Tests: - 6 db-level schema tests (defaults, json roundtrip, drain query shape, claimedBy filter, GC predicate, agentId nullable). - 13 service tests covering enqueue, the CAS race on tryClaim, complete/fail/cancel, idempotent terminal transitions, revertHeldBy on disconnect, and the full waitFor signal lifecycle (immediate resolve, wake on event, chunk streaming, cancel/error/timeout paths). Plus a gcSweep test with a fixed clock. mcpd 881/881 (was 868; +13). db pool-schema 14/14, +6 new inference-task-schema. Pre-existing failures in models.test.ts (Secret FK fixture issue, also fails on main HEAD) are unrelated. Stage 2 (next): VirtualLlmService rewires through this — remove the in-memory pendingTasks map; enqueue creates a row, dispatch picks an active session, the result-route updates the row + emits the wakeup. Worker disconnect reverts; worker bind drains.
2026-04-28 02:14:45 +01:00
// Worker tracking. NULL while pending; set on claim; cleared on
// unbindSession-driven revert (worker disconnect mid-task).
feat(db): schema for ResourceRevision, ResourceProposal, Skill Phase 1 of the Skills + Revisions + Proposals work. Purely additive — no existing rows are touched, no tables renamed, no columns dropped. New tables: - ResourceRevision — append-only audit + diff log keyed by (resourceType, resourceId). Both Prompt and Skill produce revisions on every change. Soft FK so revisions outlive the resources they describe. Indexed for history viewer (latest-first), semver lookup, and cross-resource sync diff via contentHash. - ResourceProposal — generic propose/approve/reject queue. Drop-in replacement for the prompt-only PromptRequest. Created empty here; PR-2 will rename PromptRequest → _PromptRequest_legacy and backfill. - Skill — new resource type that mirrors Prompt for everything CRUD- shaped. Adds `files` Json (multi-file bundles, materialised onto disk by `mcpctl skills sync` in PR-5) and `metadata` Json (typed app-layer in PR-3: hooks, mcpServers, postInstall, …). New columns on Prompt: - semver (semver string, default '0.1.0') — auto-bumped patch on save by PromptService.update once PR-2 wires it. Distinct from `version`, which stays as the optimistic-concurrency counter. - currentRevisionId — soft pointer to the latest ResourceRevision row. DB tests cover scope rules (project XOR agent XOR neither), name uniqueness across both compound keys, cascade-on-delete, soft-FK survival of deletion, and JSON column persistence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 00:18:21 +01:00
claimedBy String?
feat(mcpd+db): durable InferenceTask queue + state machine (v5 Stage 1) The persistence + signaling layer for v5. No integration with the existing in-flight inference path yet — that's Stage 2. This commit just lands the durable queue underneath, with a state machine that mcpd's HTTP handlers, the worker result-POST route, and the GC sweep will all build on. Schema (src/db/prisma/schema.prisma + migration): - New `InferenceTask` model + `InferenceTaskStatus` enum (pending|claimed|running|completed|error|cancelled). - Routing fields stored at enqueue time so a later rename of `Llm.poolName` doesn't reroute already-queued work: `poolName` (effective pool key), `llmName` (pinned target), `model`, `tier`. - Worker tracking: `claimedBy` (providerSessionId) + `claimedAt`, cleared on revert. - Bodies as `Json`: requestBody (always set), responseBody (set at completion). Streaming chunks are NOT persisted — too expensive at delta granularity. The final assembled body lands once per task. - Lifecycle timestamps: createdAt, claimedAt, streamStartedAt, completedAt. Plus ownerId (RBAC + audit) and agentId (null for direct chat-llm calls). - Indexes for the hot paths: (status, poolName) for the dispatcher's drain query, claimedBy for the disconnect revert, completedAt for the GC retention sweep, owner/agent for the async API listing. Repository (src/mcpd/src/repositories/inference-task.repository.ts): - CRUD + state transitions as conditional CAS via `updateMany`. Two workers racing to claim the same row both run the UPDATE; whichever the DB serializes first sees affected=1 and gets the row, the loser sees 0 and falls through to the next candidate. No application- level locking required. - findPendingForPools(poolNames[]) for the worker drain on bind. - findHeldBy(claimedBy) for the unbindSession revert. - findStalePending + findExpiredTerminal for the GC sweep. Service (src/mcpd/src/services/inference-task.service.ts): - Owns the in-process EventEmitter that wakes blocked HTTP handlers when a worker POSTs results. The DB row is the source of truth for *state*; the EventEmitter just signals "go re-read row X" so we don't have to poll. Single-instance assumption for v5; pg LISTEN/NOTIFY is the v6 swap when scaling horizontally — no schema change needed, just replace the emitter wakeup. - waitFor(taskId, timeoutMs) returns { done, chunks }: the terminal promise + an async iterator of streaming deltas. Throws on cancel (clear message) or error (worker's errorMessage propagates) or timeout. Polls the row once at subscribe time so an already- terminal task resolves immediately without waiting for an event that's never coming. - gcSweep flips stale pending rows to error (with a clear message about the timeout) and deletes terminal rows past retention. Defaults: 1h pending timeout, 7d terminal retention; both configurable. Tests: - 6 db-level schema tests (defaults, json roundtrip, drain query shape, claimedBy filter, GC predicate, agentId nullable). - 13 service tests covering enqueue, the CAS race on tryClaim, complete/fail/cancel, idempotent terminal transitions, revertHeldBy on disconnect, and the full waitFor signal lifecycle (immediate resolve, wake on event, chunk streaming, cancel/error/timeout paths). Plus a gcSweep test with a fixed clock. mcpd 881/881 (was 868; +13). db pool-schema 14/14, +6 new inference-task-schema. Pre-existing failures in models.test.ts (Secret FK fixture issue, also fails on main HEAD) are unrelated. Stage 2 (next): VirtualLlmService rewires through this — remove the in-memory pendingTasks map; enqueue creates a row, dispatch picks an active session, the result-route updates the row + emits the wakeup. Worker disconnect reverts; worker bind drains.
2026-04-28 02:14:45 +01:00
// Body + result. Both are Json so streaming chunks can be reconstructed
// (see TaskService.complete) and async pollers get a structured payload.
// requestBody is required (the OpenAI chat-completion request body the
// worker should run); responseBody is null until status=completed.
feat(db): schema for ResourceRevision, ResourceProposal, Skill Phase 1 of the Skills + Revisions + Proposals work. Purely additive — no existing rows are touched, no tables renamed, no columns dropped. New tables: - ResourceRevision — append-only audit + diff log keyed by (resourceType, resourceId). Both Prompt and Skill produce revisions on every change. Soft FK so revisions outlive the resources they describe. Indexed for history viewer (latest-first), semver lookup, and cross-resource sync diff via contentHash. - ResourceProposal — generic propose/approve/reject queue. Drop-in replacement for the prompt-only PromptRequest. Created empty here; PR-2 will rename PromptRequest → _PromptRequest_legacy and backfill. - Skill — new resource type that mirrors Prompt for everything CRUD- shaped. Adds `files` Json (multi-file bundles, materialised onto disk by `mcpctl skills sync` in PR-5) and `metadata` Json (typed app-layer in PR-3: hooks, mcpServers, postInstall, …). New columns on Prompt: - semver (semver string, default '0.1.0') — auto-bumped patch on save by PromptService.update once PR-2 wires it. Distinct from `version`, which stays as the optimistic-concurrency counter. - currentRevisionId — soft pointer to the latest ResourceRevision row. DB tests cover scope rules (project XOR agent XOR neither), name uniqueness across both compound keys, cascade-on-delete, soft-FK survival of deletion, and JSON column persistence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 00:18:21 +01:00
requestBody Json
responseBody Json?
errorMessage String?
feat(mcpd+db): durable InferenceTask queue + state machine (v5 Stage 1) The persistence + signaling layer for v5. No integration with the existing in-flight inference path yet — that's Stage 2. This commit just lands the durable queue underneath, with a state machine that mcpd's HTTP handlers, the worker result-POST route, and the GC sweep will all build on. Schema (src/db/prisma/schema.prisma + migration): - New `InferenceTask` model + `InferenceTaskStatus` enum (pending|claimed|running|completed|error|cancelled). - Routing fields stored at enqueue time so a later rename of `Llm.poolName` doesn't reroute already-queued work: `poolName` (effective pool key), `llmName` (pinned target), `model`, `tier`. - Worker tracking: `claimedBy` (providerSessionId) + `claimedAt`, cleared on revert. - Bodies as `Json`: requestBody (always set), responseBody (set at completion). Streaming chunks are NOT persisted — too expensive at delta granularity. The final assembled body lands once per task. - Lifecycle timestamps: createdAt, claimedAt, streamStartedAt, completedAt. Plus ownerId (RBAC + audit) and agentId (null for direct chat-llm calls). - Indexes for the hot paths: (status, poolName) for the dispatcher's drain query, claimedBy for the disconnect revert, completedAt for the GC retention sweep, owner/agent for the async API listing. Repository (src/mcpd/src/repositories/inference-task.repository.ts): - CRUD + state transitions as conditional CAS via `updateMany`. Two workers racing to claim the same row both run the UPDATE; whichever the DB serializes first sees affected=1 and gets the row, the loser sees 0 and falls through to the next candidate. No application- level locking required. - findPendingForPools(poolNames[]) for the worker drain on bind. - findHeldBy(claimedBy) for the unbindSession revert. - findStalePending + findExpiredTerminal for the GC sweep. Service (src/mcpd/src/services/inference-task.service.ts): - Owns the in-process EventEmitter that wakes blocked HTTP handlers when a worker POSTs results. The DB row is the source of truth for *state*; the EventEmitter just signals "go re-read row X" so we don't have to poll. Single-instance assumption for v5; pg LISTEN/NOTIFY is the v6 swap when scaling horizontally — no schema change needed, just replace the emitter wakeup. - waitFor(taskId, timeoutMs) returns { done, chunks }: the terminal promise + an async iterator of streaming deltas. Throws on cancel (clear message) or error (worker's errorMessage propagates) or timeout. Polls the row once at subscribe time so an already- terminal task resolves immediately without waiting for an event that's never coming. - gcSweep flips stale pending rows to error (with a clear message about the timeout) and deletes terminal rows past retention. Defaults: 1h pending timeout, 7d terminal retention; both configurable. Tests: - 6 db-level schema tests (defaults, json roundtrip, drain query shape, claimedBy filter, GC predicate, agentId nullable). - 13 service tests covering enqueue, the CAS race on tryClaim, complete/fail/cancel, idempotent terminal transitions, revertHeldBy on disconnect, and the full waitFor signal lifecycle (immediate resolve, wake on event, chunk streaming, cancel/error/timeout paths). Plus a gcSweep test with a fixed clock. mcpd 881/881 (was 868; +13). db pool-schema 14/14, +6 new inference-task-schema. Pre-existing failures in models.test.ts (Secret FK fixture issue, also fails on main HEAD) are unrelated. Stage 2 (next): VirtualLlmService rewires through this — remove the in-memory pendingTasks map; enqueue creates a row, dispatch picks an active session, the result-route updates the row + emits the wakeup. Worker disconnect reverts; worker bind drains.
2026-04-28 02:14:45 +01:00
/**
* Whether the original request asked for streaming. Drives the chunk-vs-
* final-body protocol on the result POST and tells async API callers
* whether `/stream` will yield chunks or just a single completion event.
*/
feat(db): schema for ResourceRevision, ResourceProposal, Skill Phase 1 of the Skills + Revisions + Proposals work. Purely additive — no existing rows are touched, no tables renamed, no columns dropped. New tables: - ResourceRevision — append-only audit + diff log keyed by (resourceType, resourceId). Both Prompt and Skill produce revisions on every change. Soft FK so revisions outlive the resources they describe. Indexed for history viewer (latest-first), semver lookup, and cross-resource sync diff via contentHash. - ResourceProposal — generic propose/approve/reject queue. Drop-in replacement for the prompt-only PromptRequest. Created empty here; PR-2 will rename PromptRequest → _PromptRequest_legacy and backfill. - Skill — new resource type that mirrors Prompt for everything CRUD- shaped. Adds `files` Json (multi-file bundles, materialised onto disk by `mcpctl skills sync` in PR-5) and `metadata` Json (typed app-layer in PR-3: hooks, mcpServers, postInstall, …). New columns on Prompt: - semver (semver string, default '0.1.0') — auto-bumped patch on save by PromptService.update once PR-2 wires it. Distinct from `version`, which stays as the optimistic-concurrency counter. - currentRevisionId — soft pointer to the latest ResourceRevision row. DB tests cover scope rules (project XOR agent XOR neither), name uniqueness across both compound keys, cascade-on-delete, soft-FK survival of deletion, and JSON column persistence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 00:18:21 +01:00
streaming Boolean @default(false)
feat(mcpd+db): durable InferenceTask queue + state machine (v5 Stage 1) The persistence + signaling layer for v5. No integration with the existing in-flight inference path yet — that's Stage 2. This commit just lands the durable queue underneath, with a state machine that mcpd's HTTP handlers, the worker result-POST route, and the GC sweep will all build on. Schema (src/db/prisma/schema.prisma + migration): - New `InferenceTask` model + `InferenceTaskStatus` enum (pending|claimed|running|completed|error|cancelled). - Routing fields stored at enqueue time so a later rename of `Llm.poolName` doesn't reroute already-queued work: `poolName` (effective pool key), `llmName` (pinned target), `model`, `tier`. - Worker tracking: `claimedBy` (providerSessionId) + `claimedAt`, cleared on revert. - Bodies as `Json`: requestBody (always set), responseBody (set at completion). Streaming chunks are NOT persisted — too expensive at delta granularity. The final assembled body lands once per task. - Lifecycle timestamps: createdAt, claimedAt, streamStartedAt, completedAt. Plus ownerId (RBAC + audit) and agentId (null for direct chat-llm calls). - Indexes for the hot paths: (status, poolName) for the dispatcher's drain query, claimedBy for the disconnect revert, completedAt for the GC retention sweep, owner/agent for the async API listing. Repository (src/mcpd/src/repositories/inference-task.repository.ts): - CRUD + state transitions as conditional CAS via `updateMany`. Two workers racing to claim the same row both run the UPDATE; whichever the DB serializes first sees affected=1 and gets the row, the loser sees 0 and falls through to the next candidate. No application- level locking required. - findPendingForPools(poolNames[]) for the worker drain on bind. - findHeldBy(claimedBy) for the unbindSession revert. - findStalePending + findExpiredTerminal for the GC sweep. Service (src/mcpd/src/services/inference-task.service.ts): - Owns the in-process EventEmitter that wakes blocked HTTP handlers when a worker POSTs results. The DB row is the source of truth for *state*; the EventEmitter just signals "go re-read row X" so we don't have to poll. Single-instance assumption for v5; pg LISTEN/NOTIFY is the v6 swap when scaling horizontally — no schema change needed, just replace the emitter wakeup. - waitFor(taskId, timeoutMs) returns { done, chunks }: the terminal promise + an async iterator of streaming deltas. Throws on cancel (clear message) or error (worker's errorMessage propagates) or timeout. Polls the row once at subscribe time so an already- terminal task resolves immediately without waiting for an event that's never coming. - gcSweep flips stale pending rows to error (with a clear message about the timeout) and deletes terminal rows past retention. Defaults: 1h pending timeout, 7d terminal retention; both configurable. Tests: - 6 db-level schema tests (defaults, json roundtrip, drain query shape, claimedBy filter, GC predicate, agentId nullable). - 13 service tests covering enqueue, the CAS race on tryClaim, complete/fail/cancel, idempotent terminal transitions, revertHeldBy on disconnect, and the full waitFor signal lifecycle (immediate resolve, wake on event, chunk streaming, cancel/error/timeout paths). Plus a gcSweep test with a fixed clock. mcpd 881/881 (was 868; +13). db pool-schema 14/14, +6 new inference-task-schema. Pre-existing failures in models.test.ts (Secret FK fixture issue, also fails on main HEAD) are unrelated. Stage 2 (next): VirtualLlmService rewires through this — remove the in-memory pendingTasks map; enqueue creates a row, dispatch picks an active session, the result-route updates the row + emits the wakeup. Worker disconnect reverts; worker bind drains.
2026-04-28 02:14:45 +01:00
// Timestamps for observability + GC:
// pending → claimed: claimedAt set
// claimed → running: streamStartedAt set (first chunk received)
// running/claimed → completed/error/cancelled: completedAt set
feat(db): schema for ResourceRevision, ResourceProposal, Skill Phase 1 of the Skills + Revisions + Proposals work. Purely additive — no existing rows are touched, no tables renamed, no columns dropped. New tables: - ResourceRevision — append-only audit + diff log keyed by (resourceType, resourceId). Both Prompt and Skill produce revisions on every change. Soft FK so revisions outlive the resources they describe. Indexed for history viewer (latest-first), semver lookup, and cross-resource sync diff via contentHash. - ResourceProposal — generic propose/approve/reject queue. Drop-in replacement for the prompt-only PromptRequest. Created empty here; PR-2 will rename PromptRequest → _PromptRequest_legacy and backfill. - Skill — new resource type that mirrors Prompt for everything CRUD- shaped. Adds `files` Json (multi-file bundles, materialised onto disk by `mcpctl skills sync` in PR-5) and `metadata` Json (typed app-layer in PR-3: hooks, mcpServers, postInstall, …). New columns on Prompt: - semver (semver string, default '0.1.0') — auto-bumped patch on save by PromptService.update once PR-2 wires it. Distinct from `version`, which stays as the optimistic-concurrency counter. - currentRevisionId — soft pointer to the latest ResourceRevision row. DB tests cover scope rules (project XOR agent XOR neither), name uniqueness across both compound keys, cascade-on-delete, soft-FK survival of deletion, and JSON column persistence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 00:18:21 +01:00
createdAt DateTime @default(now())
claimedAt DateTime?
streamStartedAt DateTime?
completedAt DateTime?
feat(mcpd+db): durable InferenceTask queue + state machine (v5 Stage 1) The persistence + signaling layer for v5. No integration with the existing in-flight inference path yet — that's Stage 2. This commit just lands the durable queue underneath, with a state machine that mcpd's HTTP handlers, the worker result-POST route, and the GC sweep will all build on. Schema (src/db/prisma/schema.prisma + migration): - New `InferenceTask` model + `InferenceTaskStatus` enum (pending|claimed|running|completed|error|cancelled). - Routing fields stored at enqueue time so a later rename of `Llm.poolName` doesn't reroute already-queued work: `poolName` (effective pool key), `llmName` (pinned target), `model`, `tier`. - Worker tracking: `claimedBy` (providerSessionId) + `claimedAt`, cleared on revert. - Bodies as `Json`: requestBody (always set), responseBody (set at completion). Streaming chunks are NOT persisted — too expensive at delta granularity. The final assembled body lands once per task. - Lifecycle timestamps: createdAt, claimedAt, streamStartedAt, completedAt. Plus ownerId (RBAC + audit) and agentId (null for direct chat-llm calls). - Indexes for the hot paths: (status, poolName) for the dispatcher's drain query, claimedBy for the disconnect revert, completedAt for the GC retention sweep, owner/agent for the async API listing. Repository (src/mcpd/src/repositories/inference-task.repository.ts): - CRUD + state transitions as conditional CAS via `updateMany`. Two workers racing to claim the same row both run the UPDATE; whichever the DB serializes first sees affected=1 and gets the row, the loser sees 0 and falls through to the next candidate. No application- level locking required. - findPendingForPools(poolNames[]) for the worker drain on bind. - findHeldBy(claimedBy) for the unbindSession revert. - findStalePending + findExpiredTerminal for the GC sweep. Service (src/mcpd/src/services/inference-task.service.ts): - Owns the in-process EventEmitter that wakes blocked HTTP handlers when a worker POSTs results. The DB row is the source of truth for *state*; the EventEmitter just signals "go re-read row X" so we don't have to poll. Single-instance assumption for v5; pg LISTEN/NOTIFY is the v6 swap when scaling horizontally — no schema change needed, just replace the emitter wakeup. - waitFor(taskId, timeoutMs) returns { done, chunks }: the terminal promise + an async iterator of streaming deltas. Throws on cancel (clear message) or error (worker's errorMessage propagates) or timeout. Polls the row once at subscribe time so an already- terminal task resolves immediately without waiting for an event that's never coming. - gcSweep flips stale pending rows to error (with a clear message about the timeout) and deletes terminal rows past retention. Defaults: 1h pending timeout, 7d terminal retention; both configurable. Tests: - 6 db-level schema tests (defaults, json roundtrip, drain query shape, claimedBy filter, GC predicate, agentId nullable). - 13 service tests covering enqueue, the CAS race on tryClaim, complete/fail/cancel, idempotent terminal transitions, revertHeldBy on disconnect, and the full waitFor signal lifecycle (immediate resolve, wake on event, chunk streaming, cancel/error/timeout paths). Plus a gcSweep test with a fixed clock. mcpd 881/881 (was 868; +13). db pool-schema 14/14, +6 new inference-task-schema. Pre-existing failures in models.test.ts (Secret FK fixture issue, also fails on main HEAD) are unrelated. Stage 2 (next): VirtualLlmService rewires through this — remove the in-memory pendingTasks map; enqueue creates a row, dispatch picks an active session, the result-route updates the row + emits the wakeup. Worker disconnect reverts; worker bind drains.
2026-04-28 02:14:45 +01:00
// Caller tracking — RBAC + observability. ownerId references User.id;
// agentId is set when the task came in via /agents/<name>/chat (null
// for direct /llms/<name>/infer or async POST /inference-tasks calls
// that don't pin an agent).
feat(db): schema for ResourceRevision, ResourceProposal, Skill Phase 1 of the Skills + Revisions + Proposals work. Purely additive — no existing rows are touched, no tables renamed, no columns dropped. New tables: - ResourceRevision — append-only audit + diff log keyed by (resourceType, resourceId). Both Prompt and Skill produce revisions on every change. Soft FK so revisions outlive the resources they describe. Indexed for history viewer (latest-first), semver lookup, and cross-resource sync diff via contentHash. - ResourceProposal — generic propose/approve/reject queue. Drop-in replacement for the prompt-only PromptRequest. Created empty here; PR-2 will rename PromptRequest → _PromptRequest_legacy and backfill. - Skill — new resource type that mirrors Prompt for everything CRUD- shaped. Adds `files` Json (multi-file bundles, materialised onto disk by `mcpctl skills sync` in PR-5) and `metadata` Json (typed app-layer in PR-3: hooks, mcpServers, postInstall, …). New columns on Prompt: - semver (semver string, default '0.1.0') — auto-bumped patch on save by PromptService.update once PR-2 wires it. Distinct from `version`, which stays as the optimistic-concurrency counter. - currentRevisionId — soft pointer to the latest ResourceRevision row. DB tests cover scope rules (project XOR agent XOR neither), name uniqueness across both compound keys, cascade-on-delete, soft-FK survival of deletion, and JSON column persistence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 00:18:21 +01:00
ownerId String
agentId String?
feat(mcpd+db): durable InferenceTask queue + state machine (v5 Stage 1) The persistence + signaling layer for v5. No integration with the existing in-flight inference path yet — that's Stage 2. This commit just lands the durable queue underneath, with a state machine that mcpd's HTTP handlers, the worker result-POST route, and the GC sweep will all build on. Schema (src/db/prisma/schema.prisma + migration): - New `InferenceTask` model + `InferenceTaskStatus` enum (pending|claimed|running|completed|error|cancelled). - Routing fields stored at enqueue time so a later rename of `Llm.poolName` doesn't reroute already-queued work: `poolName` (effective pool key), `llmName` (pinned target), `model`, `tier`. - Worker tracking: `claimedBy` (providerSessionId) + `claimedAt`, cleared on revert. - Bodies as `Json`: requestBody (always set), responseBody (set at completion). Streaming chunks are NOT persisted — too expensive at delta granularity. The final assembled body lands once per task. - Lifecycle timestamps: createdAt, claimedAt, streamStartedAt, completedAt. Plus ownerId (RBAC + audit) and agentId (null for direct chat-llm calls). - Indexes for the hot paths: (status, poolName) for the dispatcher's drain query, claimedBy for the disconnect revert, completedAt for the GC retention sweep, owner/agent for the async API listing. Repository (src/mcpd/src/repositories/inference-task.repository.ts): - CRUD + state transitions as conditional CAS via `updateMany`. Two workers racing to claim the same row both run the UPDATE; whichever the DB serializes first sees affected=1 and gets the row, the loser sees 0 and falls through to the next candidate. No application- level locking required. - findPendingForPools(poolNames[]) for the worker drain on bind. - findHeldBy(claimedBy) for the unbindSession revert. - findStalePending + findExpiredTerminal for the GC sweep. Service (src/mcpd/src/services/inference-task.service.ts): - Owns the in-process EventEmitter that wakes blocked HTTP handlers when a worker POSTs results. The DB row is the source of truth for *state*; the EventEmitter just signals "go re-read row X" so we don't have to poll. Single-instance assumption for v5; pg LISTEN/NOTIFY is the v6 swap when scaling horizontally — no schema change needed, just replace the emitter wakeup. - waitFor(taskId, timeoutMs) returns { done, chunks }: the terminal promise + an async iterator of streaming deltas. Throws on cancel (clear message) or error (worker's errorMessage propagates) or timeout. Polls the row once at subscribe time so an already- terminal task resolves immediately without waiting for an event that's never coming. - gcSweep flips stale pending rows to error (with a clear message about the timeout) and deletes terminal rows past retention. Defaults: 1h pending timeout, 7d terminal retention; both configurable. Tests: - 6 db-level schema tests (defaults, json roundtrip, drain query shape, claimedBy filter, GC predicate, agentId nullable). - 13 service tests covering enqueue, the CAS race on tryClaim, complete/fail/cancel, idempotent terminal transitions, revertHeldBy on disconnect, and the full waitFor signal lifecycle (immediate resolve, wake on event, chunk streaming, cancel/error/timeout paths). Plus a gcSweep test with a fixed clock. mcpd 881/881 (was 868; +13). db pool-schema 14/14, +6 new inference-task-schema. Pre-existing failures in models.test.ts (Secret FK fixture issue, also fails on main HEAD) are unrelated. Stage 2 (next): VirtualLlmService rewires through this — remove the in-memory pendingTasks map; enqueue creates a row, dispatch picks an active session, the result-route updates the row + emits the wakeup. Worker disconnect reverts; worker bind drains.
2026-04-28 02:14:45 +01:00
@@index([status, poolName])
@@index([claimedBy])
@@index([ownerId])
@@index([agentId])
// GC sweep predicate: completedAt < 7d ago. Index speeds up the daily
// cleanup without scanning the whole table.
@@index([completedAt])
}
// ── Audit Logs ──
model AuditLog {
id String @id @default(cuid())
userId String
action String
resource String
resourceId String?
details Json @default("{}")
createdAt DateTime @default(now())
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
@@index([userId])
@@index([action])
@@index([resource])
@@index([createdAt])
}