feat(mcplocal): every request answers, even when the pipeline wedges
The LLM budgets bound the failure we actually hit. This is the backstop for the
ones they cannot see: a wedged plugin hook, a virtual-tool handler, an upstream
path without its own timeout.
transport.onmessage IS the whole request pipeline, and it writes to a socket
that has already been hijacked from Fastify. The SDK does not await it, so a
throw became an unhandled rejection and a hang became silence -- in both cases
the client got no response at all and waited out its own timeout. onmessage is
now structured so that reaching transport.send() is unconditional: the route
call is raced against a deadline, and the notification flush and the send each
carry their own catch, so a failure while flushing can no longer cost the client
its response.
transport.onerror was never assigned, so SDK-level transport errors were
swallowed entirely. It is now.
Shape of the answer is deliberate. A tools/call comes back as a SUCCESSFUL
result with isError and readable text naming the trace code -- the same shape
router.ts already uses for an expired _resultId, because a transport error tends
to surface as a hard failure while a tool error is something the model reads and
acts on. It also says the upstream may still be running, which is true and
matters. Other methods get a JSON-RPC error (-32001).
MCPLOCAL_TOOLCALL_DEADLINE_MS defaults to TOOLCALL_TIMEOUT_MS + 30s rather than
120s. The watchdog arms earlier in the request than mcpd's fetch does, so at
equal values it would always fire first, masking mcpd's specific
UpstreamTimeoutError with a generic message and cutting off pagination and the
rest of the post-processing. There is a test asserting the two stay ordered.
The pause queue is exempt, but not bypassed. It blocks until a human operator
releases, edits or drops a response, so the deadline SUSPENDS -- clock stopped,
then re-armed with whatever was left. A test asserts the re-arm, because an
exemption that forgot to re-arm would silently make every paused request
immortal.
End-to-end: an upstream that never settles now yields a response in under a
second carrying ⚠, the deadline, and `mcpctl trace <code>`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GqMidYEGUJG5fxeoTELBu2
2026-08-25 23:41:09 +01:00
|
|
|
import { describe, it, expect, vi, afterEach } from 'vitest';
|
|
|
|
|
import {
|
|
|
|
|
createRequestDeadline,
|
|
|
|
|
DeadlineExceededError,
|
|
|
|
|
TOOLCALL_DEADLINE_MS,
|
|
|
|
|
} from '../src/http/deadline.js';
|
|
|
|
|
import { TOOLCALL_TIMEOUT_MS } from '../src/http/mcpd-client.js';
|
|
|
|
|
|
|
|
|
|
afterEach(() => { vi.restoreAllMocks(); });
|
|
|
|
|
|
|
|
|
|
describe('TOOLCALL_DEADLINE_MS', () => {
|
|
|
|
|
it('exceeds mcpd\'s own tool-call timeout', () => {
|
|
|
|
|
// The watchdog arms earlier in the request than mcpd's fetch does. At equal
|
|
|
|
|
// values it would always fire first, masking mcpd's specific
|
|
|
|
|
// UpstreamTimeoutError with a generic deadline message and cutting off
|
|
|
|
|
// pagination and the rest of the post-processing.
|
|
|
|
|
expect(TOOLCALL_DEADLINE_MS).toBeGreaterThan(TOOLCALL_TIMEOUT_MS);
|
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
it('stays well below a typical client timeout so our error arrives first', () => {
|
|
|
|
|
expect(TOOLCALL_DEADLINE_MS).toBeLessThan(600_000);
|
|
|
|
|
});
|
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
describe('createRequestDeadline', () => {
|
|
|
|
|
it('rejects with DeadlineExceededError when the budget runs out', async () => {
|
|
|
|
|
const d = createRequestDeadline('tools/call', 40);
|
|
|
|
|
await expect(Promise.race([
|
|
|
|
|
new Promise<never>(() => { /* the hang we are protecting against */ }),
|
|
|
|
|
d.expiry,
|
|
|
|
|
])).rejects.toBeInstanceOf(DeadlineExceededError);
|
|
|
|
|
d.dispose();
|
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
it('does not fire once disposed', async () => {
|
|
|
|
|
const d = createRequestDeadline('tools/call', 30);
|
|
|
|
|
d.dispose();
|
|
|
|
|
const outcome = await Promise.race([
|
|
|
|
|
d.expiry.then(() => 'fired', () => 'fired'),
|
|
|
|
|
new Promise((r) => setTimeout(() => { r('quiet'); }, 120)),
|
|
|
|
|
]);
|
|
|
|
|
expect(outcome).toBe('quiet');
|
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
it('stops the clock while suspended, so an operator pause cannot time out', async () => {
|
|
|
|
|
const d = createRequestDeadline('tools/call', 100);
|
|
|
|
|
|
|
|
|
|
// A pause queue wait far longer than the whole deadline.
|
|
|
|
|
const held = await d.suspended(async () => {
|
|
|
|
|
await new Promise((r) => setTimeout(r, 250));
|
|
|
|
|
return 'released';
|
|
|
|
|
});
|
|
|
|
|
expect(held).toBe('released');
|
|
|
|
|
|
|
|
|
|
// Still alive afterwards, with roughly the original budget left.
|
|
|
|
|
const outcome = await Promise.race([
|
|
|
|
|
d.expiry.then(() => 'fired', () => 'fired'),
|
|
|
|
|
new Promise((r) => setTimeout(() => { r('still-running'); }, 40)),
|
|
|
|
|
]);
|
|
|
|
|
expect(outcome).toBe('still-running');
|
|
|
|
|
d.dispose();
|
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
it('re-arms after suspension rather than becoming immortal', async () => {
|
|
|
|
|
const d = createRequestDeadline('tools/call', 60);
|
|
|
|
|
await d.suspended(async () => { await new Promise((r) => setTimeout(r, 100)); });
|
|
|
|
|
// The remaining budget must still expire — suspending is not a bypass.
|
|
|
|
|
await expect(Promise.race([
|
|
|
|
|
new Promise<never>(() => { /* hang */ }),
|
|
|
|
|
d.expiry,
|
|
|
|
|
])).rejects.toBeInstanceOf(DeadlineExceededError);
|
|
|
|
|
d.dispose();
|
|
|
|
|
});
|
|
|
|
|
});
|
feat: per-server tool-call timeout, on the server resource
Completes the plan's last item. The deadline shipped with a global default;
this makes it overridable per server, where the knowledge actually lives -- a
server with genuinely slow tools declares its own budget instead of forcing the
global up for everyone.
Source of truth is the server resource, following healthCheck exactly:
Prisma column + migration (NULL keeps today's behaviour, so no existing server
changes), zod validation on create and update, the repository, an
`--tool-call-timeout` flag mirroring `--health-check-timeout`, and the apply
schema so `apply -f` accepts what `get server -o yaml` emits.
That round-trip needed care: get emits `toolCallTimeoutSeconds: null` for every
server without an override, so the apply schema is nullable, not merely
optional -- otherwise the very first `get -o yaml | apply -f` on an untouched
server would have failed validation.
Bounded at one hour. A deadline exists so a wedged call answers instead of
hanging; a value beyond an hour is indistinguishable from having none.
It reaches mcplocal through server discovery rather than the project-scoped
serverOverrides map, and is applied on EVERY sync rather than only at first
registration -- raising a server's timeout should take effect at the next
refresh, not require the upstream to be dropped and rebuilt.
The endpoint sees the wire name (`docmost_search`), so it decodes to the
canonical `server/tool` before resolving. The override is keyed by server, so
every tool on that server inherits it without being enumerated, and the
failure message quotes the deadline actually applied rather than the global.
Smoke tests (tests/smoke/bounded-failures.smoke.test.ts) cover the two
production faults against the live proxy: a stale session recovers instead of
stranding the client on an uncorrelatable 404, every request is answered, and
`mcpctl trace` responds for an unknown code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GqMidYEGUJG5fxeoTELBu2
2026-08-26 00:03:06 +01:00
|
|
|
|
|
|
|
|
describe('per-server deadline override', () => {
|
|
|
|
|
it('falls back to the global when a server declares nothing', async () => {
|
|
|
|
|
const { McpRouter } = await import('../src/router.js');
|
|
|
|
|
const router = new McpRouter();
|
|
|
|
|
expect(router.getToolCallDeadlineMs('docmost/search')).toBeUndefined();
|
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
it('returns the server override for any of its tools', async () => {
|
|
|
|
|
const { McpRouter } = await import('../src/router.js');
|
|
|
|
|
const router = new McpRouter();
|
|
|
|
|
router.setServerDeadline('slowserver', 300_000);
|
|
|
|
|
// Resolved from the canonical `server/tool` name, so every tool on that
|
|
|
|
|
// server inherits it without being enumerated.
|
|
|
|
|
expect(router.getToolCallDeadlineMs('slowserver/render')).toBe(300_000);
|
|
|
|
|
expect(router.getToolCallDeadlineMs('slowserver/export')).toBe(300_000);
|
|
|
|
|
expect(router.getToolCallDeadlineMs('other/tool')).toBeUndefined();
|
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
it('clears the override when the server stops declaring one', async () => {
|
|
|
|
|
const { McpRouter } = await import('../src/router.js');
|
|
|
|
|
const router = new McpRouter();
|
|
|
|
|
router.setServerDeadline('slowserver', 300_000);
|
|
|
|
|
router.setServerDeadline('slowserver', undefined);
|
|
|
|
|
expect(router.getToolCallDeadlineMs('slowserver/render')).toBeUndefined();
|
|
|
|
|
});
|
|
|
|
|
});
|