Record the empirical finding: a Claude subscription OAuth token is gated by
Anthropic (404/429 as a raw API) and can't serve as the server-side fallback,
even though mcpd's adapter now sends it via Bearer. The anthropic-fallback is
wired-but-inert until mcpd Secret anthropic-key holds a real console API key.
Also note the chain is now Pulumi-owned so pulumi up won't wipe it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Chat is LLM-essential but not model-specific — instead of failing when the
pinned model is down/drifted, it now fails over across an ordered chain and
reports which model actually answered.
- Ordered fallback: an Llm declares `extraConfig.fallbacks: string[]`; the
dispatcher builds primary-pool → fallback-pool(s) candidates and tries them
in order (resolveCandidatesWithFallbacks).
- Fail over on real failures, not just transport: runOneInference now advances
on a non-2xx status (e.g. a 400 from a drifted model) or an empty/invalid
completion, not only thrown transport errors. Streaming fails over
pre-first-chunk (already threw on 4xx).
- Transparency: ChatResult + the SSE `final` frame carry {llm, model,
failedOver}; the CLI prints `model: <llm> (<model>)` each turn and
`⚠ failed over → answered by …` when a fallback was used.
- Exhaustion names the last model + upstream body (not "no choice").
Tests: 3 failover unit tests (primary 400 → fallback answers + model reported;
primary answers → failedOver=false; all fail → clear aggregated error).
mcpd 948 + CLI 508 green; tsc + lint clean. docs/reliability.md updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mcpctl hung on prompt-reading whenever the LLM misbehaved (thinking model =
minutes; drifted model = silent failure). Root cause: the gate's begin_session
prompt-selection called the LLM with no timeout, its fallback only fired on
error and was silent, and it forced the project's vLLM model onto the anthropic
heavy provider (so selection failed silently every time).
- New withTimeout(run, ms, label): Promise.race + AbortSignal (fetch-based
providers cancel). CompletionOptions.signal threaded into anthropic/openai.
- Gate begin_session: LLM selection is time-bounded (MCPCTL_GATE_LLM_TIMEOUT_MS,
8s); on timeout/error it falls back to deterministic tag matching, logs
[gate] loudly, prepends a ⚠ degraded note to the response, and sets
degraded/degradedReason on the audit gate_decision.
- Gate selector no longer forces the project vLLM model — uses the heavy
provider's own model (fixes the always-silent-fail bug).
- Pagination smart-index is time-bounded too (falls back to byte-range pages).
- Chat (LLM-essential) surfaces the upstream status+body (names model+reason)
instead of "Adapter returned no choice".
- docs/reliability.md documents the principle.
Tests: with-timeout unit tests; gate degradation test (error → visible ⚠ +
deterministic prompts, no hang). mcplocal 737 + mcpd 945 green; tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
