test(smoke): give the smoke-aws-docs fixture a readiness probe #107

michal · 2026-08-10T15:59:57Z

michal commented

2026-08-10 15:59:57 +00:00

Problem

health-readiness.smoke.test.ts asserts that no RUNNING server lacks a healthCheck.tool. The smoke suite's own shared fixture had none, so the suite reported its own scaffolding as a fleet regression on every run.

It is not a leak that cleanup would sweep — smoke-aws-docs is in the PROTECTED set of clean-smoke-resources.ts (five suites depend on it), so it is deliberately long-lived and failed the assertion permanently.

Fix

Mirror the production aws-docs probe — same package, so the same tool works: search_documentation with a search phrase, 300s interval to match, since the call leaves the cluster.

Live fleet fixes (mcpd config, not in this diff)

The same smoke failure covered three more, all fixed by applying config to mcpd:

server	was	now
`docs`	no `healthCheck` at all	`list_libraries`
`my-home-assistant`	`ha_config_list_areas` — no longer exists upstream	`ha_list_floors_areas`
`smoke-aws-docs`	no `healthCheck`	`search_documentation` (this diff + applied live)

Each probe was called through /api/v1/mcp/proxy before being configured, so none is a guess: ha_list_floors_areas returns 4 floors / 16 areas from Home Assistant, list_libraries answers from the docs store.

Still broken: docmost

docmost/get_workspace returns 401, and it is a genuine bad credential, not config. Diagnosed from inside the pod:

API_URL   : http://docmost.docmost.svc.cluster.local:3000/api
EMAIL     : m***@itaz.eu
PASSWORD  : set (31 chars)
POST /auth/login -> 401 {"message":"Email or password does not match"}

The endpoint is reachable and the account resolves — the stored DOCMOST_PASSWORD in OpenBao (secret/mcpctl/docmost) is simply wrong. That needs the real password; it cannot be fixed from here.

Result: 3 of 4 readiness assertions now pass; only the docmost credential remains.

## Problem `health-readiness.smoke.test.ts` asserts that no RUNNING server lacks a `healthCheck.tool`. The smoke suite's **own shared fixture** had none, so the suite reported its own scaffolding as a fleet regression on every run. It is not a leak that cleanup would sweep — `smoke-aws-docs` is in the `PROTECTED` set of `clean-smoke-resources.ts` (five suites depend on it), so it is deliberately long-lived and failed the assertion permanently. ## Fix Mirror the production `aws-docs` probe — same package, so the same tool works: `search_documentation` with a search phrase, 300s interval to match, since the call leaves the cluster. ## Live fleet fixes (mcpd config, not in this diff) The same smoke failure covered three more, all fixed by applying config to mcpd: | server | was | now | |---|---|---| | `docs` | no `healthCheck` at all | `list_libraries` | | `my-home-assistant` | `ha_config_list_areas` — no longer exists upstream | `ha_list_floors_areas` | | `smoke-aws-docs` | no `healthCheck` | `search_documentation` (this diff + applied live) | Each probe was called through `/api/v1/mcp/proxy` before being configured, so none is a guess: `ha_list_floors_areas` returns 4 floors / 16 areas from Home Assistant, `list_libraries` answers from the docs store. ## Still broken: docmost `docmost/get_workspace` returns **401**, and it is a genuine bad credential, not config. Diagnosed from inside the pod: ``` API_URL : http://docmost.docmost.svc.cluster.local:3000/api EMAIL : m***@itaz.eu PASSWORD : set (31 chars) POST /auth/login -> 401 {"message":"Email or password does not match"} ``` The endpoint is reachable and the account resolves — the stored `DOCMOST_PASSWORD` in OpenBao (`secret/mcpctl/docmost`) is simply wrong. That needs the real password; it cannot be fixed from here. Result: 3 of 4 readiness assertions now pass; only the docmost credential remains.

michal added 1 commit 2026-08-10 15:59:57 +00:00

test(smoke): give the smoke-aws-docs fixture a readiness probe

CI/CD / lint (pull_request) Successful in 1m13s

Details

CI/CD / typecheck (pull_request) Successful in 1m18s

Details

CI/CD / test (pull_request) Successful in 1m26s

Details

CI/CD / build (pull_request) Successful in 2m15s

Details

CI/CD / smoke (pull_request) Failing after 3m23s

Details

CI/CD / publish (pull_request) Has been skipped

Details

b8cedd6262

health-readiness.smoke.test.ts asserts no RUNNING server lacks a
healthCheck.tool. The smoke suite's own shared fixture had none, so the suite
reported its own scaffolding as a fleet regression — and because the fixture is
in the PROTECTED set of clean-smoke-resources.ts, it is long-lived and failed
that assertion on every run.

Mirrors the production aws-docs probe (same package): search_documentation with
a phrase, 300s interval since the call leaves the cluster.

Verified live: with this applied, "every RUNNING server has a readiness probe
configured" passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019wUmrfkVQR6CKcYKxENq7k

michal merged commit cd94e855aa into main

2026-08-10 16:00:09 +00:00

michal referenced this issue from a commit

2026-08-10 16:00:11 +00:00

Merge PR #107: smoke-aws-docs readiness probe

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: michal/mcpctl#107