mcpd never recovers a k8s STDIO instance after its container restarts — stuck ERROR while the pod is Running #114
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Symptom: a k8s STDIO server instance walks to ERROR/degraded and stays there forever, while
kubectlshows its pod 1/1 Running and the MCP server's own log says it started fine ("MCP SearXNG Server v1.15.0 connected via STDIO"). Tool calls through any project then fail with-32601 Unknown name: <server>/<tool>because mcplocal drops the upstream. Happened 3x on 2026-08-15 withsearxng.Trigger: any container restart inside the pod — observed with OOMKill (exit 137), a V8 heap abort, and a transient
npxinstall failure (truncated tarball →ERR_MODULE_NOT_FOUND, self-healed on the next kubelet retry). The pod object survives; only the container is recreated.Cause (hypothesis from behavior): mcpd holds a long-lived STDIO attach to the container. When kubelet restarts the container, that pipe dies and mcpd never re-attaches — health probes fail 3x, the instance goes ERROR, and nothing ever re-probes/reconnects. The state machine treats ERROR as terminal.
Evidence: after the container self-healed, the instance cycled STARTING → ERROR once more and then sat in ERROR for good; meanwhile the underlying SearXNG service answered
/search?q=mcp&format=jsonin 0.73s and the pod log showed a clean server start. Deleting the instance (mcpctl delete instance <id>) made the reconciler create a replacement that went RUNNING/healthy in ~60s — proving everything except mcpd's attachment was fine.Wanted: on STDIO pipe loss (or on health-fail while the k8s pod is Running/Ready), re-attach to the restarted container — or recreate the instance automatically — instead of parking in ERROR until a human deletes it.
Workaround:
mcpctl delete instance <id>; the reconciler recreates it healthy.Related: the flat 512Mi with no per-server memoryLimit knob (orchestrator.ts declares
memoryLimit, nothing populates it) is what makes Node servers restart-prone in the first place —NODE_OPTIONS=--max-old-space-size=384in server env is the current mitigation onsearxng.