michal/mcpctl
1
0
Fork 0
Code Issues Pull Requests 7 Actions Packages Projects Releases Wiki Activity

build: fail the release when smoke tests fail, and fix the SSE test that hung
Some checks failed
CI/CD / lint (pull_request) Successful in 1m11s
Details
CI/CD / test (pull_request) Successful in 1m24s
Details
CI/CD / typecheck (pull_request) Successful in 3m6s
Details
CI/CD / smoke (pull_request) Failing after 1m57s
Details
CI/CD / build (pull_request) Successful in 4m40s
Details
CI/CD / publish (pull_request) Has been skipped
Details

Browse Source 
release.sh printed `WARNING: Smoke tests failed!` and exited 0. That is how four
broken readiness probes shipped unnoticed on 2026-08-10 — the warning scrolled
past in the build log and the release reported success.

It now exits 1, with `MCPCTL_ALLOW_SMOKE_FAILURE=1` as the escape hatch. The
message is explicit that smoke runs LAST, against the installed binary: the
package is already published and installed, so the failure reports fleet
breakage rather than preventing a bad artifact.

Turning the gate on required fixing a latent hang first, or every release would
have blocked on it. `security.test.ts > /inspect SSE endpoint …` waited for a
response body that by design never ends, so it could only settle via the
socket's *inactivity* timeout — and /inspect relays every project's MCP traffic,
so during a full smoke run it is never idle. Run alone it passed and looked
flaky; run with the suite it failed every time.

httpRequest gains `headersOnly`, which resolves on the response headers and
hangs up. The assertion only ever needed the status line.

Verified: full smoke suite 158/158 (was 157/158 with this test timing out); the
gate block lifted verbatim from release.sh exits 1 with a stubbed failing smoke
run, and exits 0 reaching subsequent code under MCPCTL_ALLOW_SMOKE_FAILURE=1.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019wUmrfkVQR6CKcYKxENq7k
This commit is contained in:
Michal
2026-08-10 17:15:40 +01:00
parent cd94e855aa
commit 822c1bb047
3 changed files with 52 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
scripts/release.sh
View File

@@ -75,9 +75,28 @@ echo "==> Running smoke tests..."
export PATH="$HOME/.npm-global/bin:$PATH"
if pnpm test:smoke; then
echo "==> Smoke tests passed!"
elif [ "${MCPCTL_ALLOW_SMOKE_FAILURE:-}" = "1" ]; then
echo "==> WARNING: Smoke tests failed, continuing (MCPCTL_ALLOW_SMOKE_FAILURE=1)."
else
echo "==> WARNING: Smoke tests failed! Check mcplocal/mcpd are running."
echo " Continuing anyway — deployment is complete, but verify manually."
# This used to print a warning and exit 0. That is how four broken readiness
# probes shipped unnoticed on 2026-08-10: the warning scrolled past in the
# build log and the release reported success. A failing smoke run means
# something in the live fleet is genuinely broken — say so in the exit code.
#
# Note what this does and does not do: smoke runs LAST, against the installed
# binary, so the package is already published and installed by now. Failing
# here reports the breakage, it does not prevent it — investigate, do not
# assume the artifact is bad.
echo "" >&2
echo "ERROR: smoke tests failed — the release is published and installed, but" >&2
echo " something in the live fleet is broken. Investigate before relying" >&2
echo " on this build; do not just re-run." >&2
echo "" >&2
echo " Common causes: mcplocal/mcpd not running, a readiness probe pointing at" >&2
echo " a tool the upstream renamed, or an expired credential." >&2
echo " Override: MCPCTL_ALLOW_SMOKE_FAILURE=1 $0" >&2
echo "" >&2
exit 1
fi
echo ""