feat(reliability): bound + fail over + surface LLM-optional ops #82

michal · 2026-07-19T15:28:48Z

michal commented

2026-07-19 15:28:48 +00:00

Makes mcpctl resilient to a bad/slow/wrong LLM — the gate stays responsive and honest even when the homelab model is thinking-only, drifted, or down.

Root cause (live-debugged)

begin_session's prompt-selection called the LLM with no timeout (a thinking model blocks it for minutes), its fallback only fired on outright error and was silent, and it forced the project's vLLM model onto the anthropic heavy provider → selection failed silently every time. read_prompts is already LLM-free.

Changes

withTimeout(run, ms, label) — Promise.race + AbortSignal (fetch-based providers cancel via new CompletionOptions.signal, threaded into anthropic/openai).
Gate begin_session: time-bounded selection (MCPCTL_GATE_LLM_TIMEOUT_MS, 8s); on timeout/error → deterministic tag matching, loud [gate] log, a ⚠ Smart prompt-selection unavailable (<reason>) note in the response, and degraded/degradedReason on the audit gate_decision.
Selector no longer forces the project vLLM model (uses the heavy provider's own) — fixes the always-silent-fail bug.
Pagination smart-index time-bounded too (falls back to byte-range pages).
Chat (LLM-essential) surfaces the upstream status+body (names the model + reason) instead of "Adapter returned no choice".
docs/reliability.md documents the principle: LLM-optional ops must be time-bounded, fall back deterministically, and report degradation.

Tests

with-timeout unit tests (hang → TimeoutError + signal aborted); gate degradation test (LLM error → visible ⚠ + deterministic prompts, no hang). mcplocal 737 + mcpd 945 green; tsc clean. No new eslint errors.

Deploy: mcpd + mcplocal image rebuild (no schema change) + RPM.

🤖 Generated with Claude Code

Makes mcpctl resilient to a bad/slow/wrong LLM — the gate stays responsive and honest even when the homelab model is thinking-only, drifted, or down. ## Root cause (live-debugged) `begin_session`'s prompt-selection called the LLM with **no timeout** (a thinking model blocks it for minutes), its fallback only fired on outright error and was **silent**, and it **forced the project's vLLM model** onto the anthropic heavy provider → selection failed silently every time. `read_prompts` is already LLM-free. ## Changes - **`withTimeout(run, ms, label)`** — Promise.race + AbortSignal (fetch-based providers cancel via new `CompletionOptions.signal`, threaded into anthropic/openai). - **Gate `begin_session`**: time-bounded selection (`MCPCTL_GATE_LLM_TIMEOUT_MS`, 8s); on timeout/error → deterministic tag matching, loud `[gate]` log, a `⚠ Smart prompt-selection unavailable (<reason>)` note in the response, and `degraded`/`degradedReason` on the audit `gate_decision`. - **Selector** no longer forces the project vLLM model (uses the heavy provider's own) — fixes the always-silent-fail bug. - **Pagination** smart-index time-bounded too (falls back to byte-range pages). - **Chat** (LLM-essential) surfaces the upstream status+body (names the model + reason) instead of "Adapter returned no choice". - **docs/reliability.md** documents the principle: *LLM-optional ops must be time-bounded, fall back deterministically, and report degradation.* ## Tests with-timeout unit tests (hang → TimeoutError + signal aborted); gate degradation test (LLM error → visible ⚠ + deterministic prompts, no hang). **mcplocal 737 + mcpd 945 green; tsc clean.** No new eslint errors. Deploy: mcpd + mcplocal image rebuild (no schema change) + RPM. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

michal added 1 commit 2026-07-19 15:28:48 +00:00

feat(reliability): bound + fail over + surface LLM-optional ops

CI/CD / typecheck (pull_request) Successful in 1m6s

Details

CI/CD / lint (pull_request) Successful in 2m11s

Details

CI/CD / test (pull_request) Successful in 1m21s

Details

CI/CD / smoke (pull_request) Failing after 2m43s

Details

CI/CD / build (pull_request) Failing after 3h11m43s

Details

CI/CD / publish (pull_request) Has been cancelled

Details

485b8f613f

mcpctl hung on prompt-reading whenever the LLM misbehaved (thinking model =
minutes; drifted model = silent failure). Root cause: the gate's begin_session
prompt-selection called the LLM with no timeout, its fallback only fired on
error and was silent, and it forced the project's vLLM model onto the anthropic
heavy provider (so selection failed silently every time).

- New withTimeout(run, ms, label): Promise.race + AbortSignal (fetch-based
  providers cancel). CompletionOptions.signal threaded into anthropic/openai.
- Gate begin_session: LLM selection is time-bounded (MCPCTL_GATE_LLM_TIMEOUT_MS,
  8s); on timeout/error it falls back to deterministic tag matching, logs
  [gate] loudly, prepends a ⚠ degraded note to the response, and sets
  degraded/degradedReason on the audit gate_decision.
- Gate selector no longer forces the project vLLM model — uses the heavy
  provider's own model (fixes the always-silent-fail bug).
- Pagination smart-index is time-bounded too (falls back to byte-range pages).
- Chat (LLM-essential) surfaces the upstream status+body (names model+reason)
  instead of "Adapter returned no choice".
- docs/reliability.md documents the principle.

Tests: with-timeout unit tests; gate degradation test (error → visible ⚠ +
deterministic prompts, no hang). mcplocal 737 + mcpd 945 green; tsc clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

michal merged commit 3465596635 into main

2026-07-19 15:29:00 +00:00

michal deleted branch feat/llm-resilience

2026-07-19 15:29:00 +00:00

michal referenced this issue from a commit

2026-07-19 15:29:01 +00:00

Merge pull request 'feat(reliability): bound + fail over + surface LLM-optional ops' (#82) from feat/llm-resilience into main

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: michal/mcpctl#82