Commit Graph

5 Commits

Author SHA1 Message Date
Michal
2da627e77f feat(llm): resolve the gate's selection Llm by tier, not by name
Some checks failed
CI/CD / lint (pull_request) Successful in 1m19s
CI/CD / typecheck (pull_request) Successful in 2m21s
CI/CD / test (pull_request) Successful in 1m19s
CI/CD / build (pull_request) Successful in 2m17s
CI/CD / smoke (pull_request) Failing after 2m47s
CI/CD / publish (pull_request) Has been skipped
The served model changes (glm-4.6-reap -> deepseek-v4-flash, and again after
that). Pulumi already repoints mcpd's long-lived rows -- vllm-current/fast and
vllm-think/heavy -- at whatever LiteLLM serves, so no model id is written down
in mcpctl. But nothing resolved those rows: gate prompt-selection took
MCPCTL_GATE_SELECTION_LLM or the project's llmProvider, and neither is set on
any project or either deployment. Unpinned projects fell through to the local
personal-token provider, whose heavy entry names a retired claude-opus-4 -- so
every gated session has been running on priority-ordered prompts, not LLM
ranking, announcing it only in a mcplocal log line nobody reads.

Consumers now ask for a role. tier is already a field on the Llm resource and
is set by the same Pulumi resource that sets model, so the two cannot drift.
Explicit pins still win; this only changes what happens when nothing is pinned.

Eligibility excludes inactive rows (selecting one defers the failure to the
first inference call) and virtual rows -- those are backed by some user's
mcplocal over SSE, and automatic resolution must not route a project's traffic
through a laptop nobody chose. The heavy tier already holds two rows, so the
tiebreak is load-bearing now rather than future-proofing: an ordered list of
well-known names, carrying both the current names and the symmetric ones the
rows might be renamed to, so a rename stays a Pulumi-only change and neither
deploy order breaks the other. Names matching nothing are inert by design.

The rows keep the names they have. name is immutable in mcpd (agents and
projects reference it), so a rename costs a data migration plus a resource
replace, and buys nothing once nothing reads the names. vllm-fast would also
collide in meaning with LiteLLM's deepseek-v4-fast route, which is a different
thing one word apart.

Smoke covers the drift case the unit tests cannot: the resolved fast row has to
answer real inference, which is exactly what fails when a row is left pointing
at a suspended model.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B5NAoE7VJA5TWvHsVfEmUr
2026-08-05 01:00:47 +01:00
Michal
56735a5290 docs: skills + revisions + proposals reference, plus cheatsheet update
Phase 7 of the Skills + Revisions + Proposals work — documentation
pass for the surface added in PR-1 through PR-6. Reference material
only; no code changes.

## What's added

- `docs/skills.md` — skill model, scoping rules, CLI surface, the
  `mcpctl config claude --project` setup flow, metadata schema (with
  the deferred-execution note for hooks/mcpServers/postInstall), the
  on-disk state file shape, atomic install mechanics, failure
  semantics, and what's deferred.
- `docs/revisions.md` — ResourceRevision model, semver auto-bump
  rules, contentHash diff key (cross-resource sync), CLI for history
  / diff / restore, RBAC, audit emission, storage growth note.
- `docs/proposals.md` — ResourceProposal model, the reviewer flow
  (CLI + web UI), atomic-approval mechanics, the propose_prompt /
  propose_skill MCP tools, the propose-learnings global skill that
  steers Claude toward engaging with them, and the deferred legacy
  PromptRequest cutover.

## What's edited

- Top-level `CLAUDE.md` — resource cheatsheet adds `skill`, `proposal`,
  `revision` with cross-references to the new docs. The legacy
  `promptrequest` entry stays (still on the legacy code path) but
  notes that new work should use `proposal`.

## What's NOT in this PR

- The PromptRequest → ResourceProposal cutover migration. Both run
  side-by-side today; the focused cutover PR will rename + backfill +
  drop. Keeping that out of PR-7 means review can stay on docs.
- Bundle-backup / `mcpctl apply -f` skill support (deferred from PR-3).
- `metadata.hooks` / `metadata.mcpServers` / `metadata.postInstall`
  execution (deferred from PR-5).
- Existing-page UI migration to Tailwind (deferred from PR-6 — old
  inline-styled pages coexist fine inside the new Layout).

These are tracked as future PRs; each is its own focused change.

## Verification

`pnpm test:run` whole monorepo: 162 test files / 2157 tests green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 17:58:04 +01:00
Michal
8b56f09f25 feat(agents): smoke tests + README + docs (Stage 6, final)
Closes the agents feature.

Smoke tests (run via `pnpm test:smoke` against a live mcpd at
$MCPD_URL, default https://mcpctl.ad.itaz.eu):

* tests/smoke/agent.smoke.test.ts — full CRUD round-trip:
  create secret + Llm + agent with sampling defaults; `get agents`
  surfaces it; `get agent foo -o yaml | apply -f` round-trips
  identically; create + list a thread via the HTTP API; agent delete
  leaves Llm + secret intact (Restrict + SetNull as designed). Self-
  skips with a warning when /healthz is unreachable.

* tests/smoke/agent-chat.smoke.test.ts — gated on
  MCPCTL_SMOKE_LLM_URL + MCPCTL_SMOKE_LLM_KEY. Provisions secret +
  Llm + agent against a real upstream, runs `mcpctl chat -m … --no-
  stream` (asserts a reply lands), then runs the streaming default
  (asserts text on stdout + `(thread: …)` on stderr). The fast path
  for verifying the in-cluster qwen3-thinking deployment:

      MCPCTL_SMOKE_LLM_URL=http://litellm.nvidia-nim.svc.cluster.local:4000/v1 \
      MCPCTL_SMOKE_LLM_MODEL=qwen3-thinking \
      MCPCTL_SMOKE_LLM_KEY=$(pulumi config get --stack homelab \
        secrets:litellmMcpctlGatewayToken) \
        pnpm test:smoke

Docs:

* README.md — new "Agents" section under Resources with the
  qwen3-thinking quickstart and links to docs/agents.md and
  docs/chat.md. Adds llm + agent rows to the resources table.

* docs/agents.md (new) — full reference: data model, chat-parameter
  table, HTTP API, RBAC mapping, tool-use loop semantics, yaml
  round-trip shorthand, the kubernetes-deployment wiring recipe,
  and a troubleshooting section (namespace collision, llm-in-use,
  pending-row recovery, Anthropic-tool limitation).

* docs/chat.md (new) — user-facing `mcpctl chat` walkthrough:
  modes, per-call flags, slash-commands, threads, and a
  troubleshooting section.

* CLAUDE.md — adds a "Resource types" cheatsheet with one-line
  pointers to each, including the new `agent` row that links to
  the docs.

All suites still green: mcpd 759/759, mcplocal 715/715, cli 430/430.
Smoke tests typecheck and self-skip when no live mcpd is reachable.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 17:08:37 +01:00
Michal
f409952b0c chore: add gstack skill routing rules to CLAUDE.md
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 01:33:56 +01:00
Michal
d0aa0c5d63 first commit 2026-02-21 03:10:39 +00:00