build: fail the release when smoke tests fail, and fix the SSE test that hung
Some checks failed
CI/CD / lint (pull_request) Successful in 1m11s
CI/CD / test (pull_request) Successful in 1m24s
CI/CD / typecheck (pull_request) Successful in 3m6s
CI/CD / smoke (pull_request) Failing after 1m57s
CI/CD / build (pull_request) Successful in 4m40s
CI/CD / publish (pull_request) Has been skipped

release.sh printed `WARNING: Smoke tests failed!` and exited 0. That is how four
broken readiness probes shipped unnoticed on 2026-08-10 — the warning scrolled
past in the build log and the release reported success.

It now exits 1, with `MCPCTL_ALLOW_SMOKE_FAILURE=1` as the escape hatch. The
message is explicit that smoke runs LAST, against the installed binary: the
package is already published and installed, so the failure reports fleet
breakage rather than preventing a bad artifact.

Turning the gate on required fixing a latent hang first, or every release would
have blocked on it. `security.test.ts > /inspect SSE endpoint …` waited for a
response body that by design never ends, so it could only settle via the
socket's *inactivity* timeout — and /inspect relays every project's MCP traffic,
so during a full smoke run it is never idle. Run alone it passed and looked
flaky; run with the suite it failed every time.

httpRequest gains `headersOnly`, which resolves on the response headers and
hangs up. The assertion only ever needed the status line.

Verified: full smoke suite 158/158 (was 157/158 with this test timing out); the
gate block lifted verbatim from release.sh exits 1 with a stubbed failing smoke
run, and exits 0 reaching subsequent code under MCPCTL_ALLOW_SMOKE_FAILURE=1.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019wUmrfkVQR6CKcYKxENq7k
This commit is contained in:
Michal
2026-08-10 17:15:40 +01:00
parent cd94e855aa
commit 822c1bb047
3 changed files with 52 additions and 8 deletions

View File

@@ -908,6 +908,15 @@ MCPCTL_BASE_BRANCH=release-2.x bash scripts/build-rpm.sh # compare to another b
Offline it falls back to the last fetched `origin/main`, then to a local `main`,
and says which it used; outside a git checkout it skips entirely.
**A failing smoke run fails the release.** It used to print
`WARNING: Smoke tests failed!` and exit 0 — which is exactly how four broken
readiness probes shipped unnoticed (see `docs/reliability.md`): the warning
scrolled past and the release reported success. Note what the gate does and does
not do — smoke runs *last*, against the installed binary, so the package is
already published and installed by the time it fails. It reports the breakage
rather than preventing it, so investigate the fleet rather than assuming the
artifact is bad. Override with `MCPCTL_ALLOW_SMOKE_FAILURE=1`.
Installs via nfpm:
- `/usr/bin/mcpctl` — CLI binary (bun compiled)
- `/usr/bin/mcpctl-local` — Local proxy binary (bun compiled)