fix(templates): search templates stop probing with searches; add firecrawl

Readiness probes ran real searches every interval: searxng's
searxng_web_search probe fanned a query out to every engine 1,440 times a
day, billing API-key engines (braveapi, kagi), and duckduckgo's `search`
probe scraped DuckDuckGo from the same IP agents search from.

- searxng: probe searxng_instance_info, which reads /config only
- duckduckgo: probe fetch_content on example.com every 300s
- firecrawl (new): firecrawl-mcp against a self-hosted or cloud
  Firecrawl, for reading pages as main-content markdown; probe
  firecrawl_scrape on example.com every 300s

Every probe still names a readiness tool, as templates.test.ts requires;
each was run against the real package before being written down.
docs/web-search.md now pairs searxng (search) with firecrawl (reading)
and says why duckduckgo does not hold up under agent traffic.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JPjtnE6Gd343oRNtMU9Bcd

README.md

@@ -869,17 +869,21 @@ Notes:
 
 ### Web search and docs lookup
 
-The `duckduckgo`, `searxng` and `docs-mcp` templates give an agent web search and
+The `searxng`, `firecrawl`, `duckduckgo` and `docs-mcp` templates give an agent web
-version-pinned library documentation. All are self-hosted and none needs an API key.
+search, page reading and version-pinned library documentation. All can run
-`duckduckgo` needs no backing service at all:
+self-hosted without an API key. For agents, pair `searxng` (search) with `firecrawl`
+(page reading); `duckduckgo` needs no backing service but scrapes from your IP and
+gets CAPTCHA-blocked under agent traffic:
 
 ```bash
-mcpctl create server websearch --from-template duckduckgo
+mcpctl create server searxng --from-template searxng --env SEARXNG_URL=http://searxng:8080
+mcpctl create server firecrawl --from-template firecrawl --env FIRECRAWL_API_URL=http://firecrawl-api:3002
 ```
 
 See [docs/web-search.md](docs/web-search.md) for the SearXNG engine setup (including
-the `json` format setting, without which every search silently returns nothing) and
+the `json` format setting, without which every search silently returns nothing), why
-the `docs-mcp` index-persistence caveat.
+none of the search templates probes with a search, and the `docs-mcp`
+index-persistence caveat.
 
 ## Gated Sessions
 

docs/web-search.md

@@ -1,14 +1,19 @@
 # Web search and documentation lookup
 
-Three templates. All self-hosted, none needs an API key or a vendor account, and
+Four templates. All can run self-hosted without an API key or a vendor account,
-all three are deployed by mcpctl like any other server.
+and all are deployed by mcpctl like any other server.
 
 | Template | Package / image | Needs |
 |---|---|---|
-| `duckduckgo` | `duckduckgo-mcp-server` (python) | nothing |
 | `searxng` | `mcp-searxng` (node) | a SearXNG instance |
+| `firecrawl` | `firecrawl-mcp` (node) | a Firecrawl instance |
+| `duckduckgo` | `duckduckgo-mcp-server` (python) | nothing |
 | `docs-mcp` | `ghcr.io/arabold/docs-mcp-server` | nothing |
 
+For agent use, pair `searxng` (finding pages) with `firecrawl` (reading them).
+`duckduckgo` does both with no infrastructure, but it scrapes from your IP and
+does not hold up under agent traffic — see below.
+
 Search and docs are different jobs, not competing options. A search engine will
 hand you a 2023 blog post with a stale method signature; a docs index cannot
 tell you why a daemon is crash-looping. Attach both to a project that does real
@@ -17,23 +22,31 @@ engineering work.
 `docs-mcp` is the open-source replacement for Context7 / Ref.tools — same job,
 but the index lives on your infrastructure and can include private repos.
 
-## Start here: `duckduckgo`
+## No infrastructure: `duckduckgo`
 
-The only one with no infrastructure behind it. It scrapes DuckDuckGo's HTML
+The only one with nothing behind it. It scrapes DuckDuckGo's HTML endpoint
-endpoint directly, so there is no engine to run and no key to hold.
+directly, so there is no engine to run and no key to hold.
 
 ```bash
 mcpctl create server websearch --from-template duckduckgo --env DDG_SAFE_SEARCH=OFF
-mcpctl get instances | grep websearch     # RUNNING / healthy within ~a minute
+mcpctl get instances | grep websearch     # RUNNING / healthy within a few minutes
 ```
 
 Tools: `search` (`query`, `max_results`, `region`) and `fetch_content` (`url`,
 `start_index`, `max_length`) for pulling a result as markdown.
 
-The tradeoff is honest: scraping has no SLA. DuckDuckGo can change its markup or
+Fine for a person searching occasionally; not for agents. Scraping has no SLA,
-rate-limit you, and the server caps itself at 30 searches/min. Set
+and agents search in bursts — one question becomes several parallel queries.
-`DDG_SEARCH_BACKEND=curl` if bot checks start biting. When it becomes a problem,
+From a single home or office IP, DuckDuckGo starts answering CAPTCHA and stays
-move to `searxng`.
+that way, and every further request from that IP deepens the block, including
+for a SearXNG instance sharing the address. `DDG_SEARCH_BACKEND=curl` survives
+some bot checks but does not change that. Use `searxng` for agent search.
+
+The readiness probe is `fetch_content` on `https://example.com` every 300s, not
+`search`. A search probe is a DuckDuckGo scrape every interval — 1,440 a day at
+the 60s default — from the same IP your agents depend on. The trade: `healthy`
+proves the process and its outbound fetch work, not that DuckDuckGo is still
+answering you.
 
 ## Better results: `searxng`
 
@@ -79,6 +92,43 @@ fails: nearly all of them leave the JSON API off.
 The compose healthcheck probes `format=json` specifically, so a misconfigured
 instance shows up as unhealthy rather than as silently empty search results.
 
+### Probe and engines
+
+The template's readiness probe is `searxng_instance_info`, which reads the
+instance's `/config`: a pass proves SearXNG answers without sending a query to
+any engine. Do not probe with `searxng_web_search` — every interval becomes a
+real search fanned out to every engine, which bills API-key engines and feeds
+the IP blocks on scraping ones.
+
+Those blocks are the long-term problem. Scraping engines (duckduckgo, google,
+startpage, qwant) CAPTCHA a single IP under agent traffic. SearXNG also ships
+API-key engines — `braveapi`, `kagi` — that are not subject to IP reputation;
+enable at least one if agents are the main caller, and cap its spend in the
+vendor's dashboard.
+
+## Reading pages: `firecrawl`
+
+`firecrawl` is the reader to pair with `searxng`: give it a URL, get the page's
+main content as markdown. Firecrawl renders pages in a real browser, so
+JavaScript-heavy pages work where a plain fetch comes back empty.
+
+```bash
+mcpctl create server firecrawl --from-template firecrawl \
+  --env FIRECRAWL_API_URL=http://firecrawl-api:3002
+```
+
+Point it at a self-hosted Firecrawl (no key needed) or leave `FIRECRAWL_API_URL`
+unset and set `FIRECRAWL_API_KEY` for Firecrawl's cloud. The tool agents want is
+`firecrawl_scrape` with `formats: ["markdown"]` and `onlyMainContent: true`.
+
+- **Egress port.** Self-hosted Firecrawl listens on `:3002`, outside the default
+  80/443 server egress, so declare it or every call times out.
+- **Search stays off** unless the Firecrawl instance has `SEARXNG_ENDPOINT`
+  set, so `firecrawl_search` cannot quietly become a second scraper. Use the
+  `searxng` template for search.
+- **Probe:** `firecrawl_scrape` of `https://example.com` every 300s. It is a
+  real outbound fetch, hence the longer interval.
+
 ## Documentation: `docs-mcp`
 
 ```bash

src/mcplocal/src/providers/anthropic.ts

@@ -30,6 +30,15 @@ function familyOf(model: string): string | null {
   return null;
 }
 
+/**
+ * A 400 naming a sampling parameter, e.g.
+ * "`temperature` is deprecated for this model."
+ */
+function isSamplingRejection(err: unknown): boolean {
+  const msg = err instanceof Error ? err.message : String(err);
+  return msg.includes('HTTP 400') && /`?(temperature|top_p|top_k)`?/.test(msg);
+}
+
 /** Resolutions are cached this long before the models endpoint is consulted again. */
 const MODEL_CACHE_TTL_MS = Number(process.env['MCPCTL_ANTHROPIC_MODEL_TTL_MS']) || 12 * 60 * 60 * 1000;
 
@@ -40,6 +49,17 @@ export class AnthropicProvider implements LlmProvider {
   readonly name = 'anthropic';
   /** Shared across instances: the model list is account-wide, not per-provider. */
   private static readonly modelCache = new Map<string, { id: string; expiresAt: number }>();
+  /**
+   * Models that rejected a sampling parameter, learned at runtime.
+   *
+   * Anthropic removed `temperature`/`top_p`/`top_k` on the current generation
+   * (Opus 5, Sonnet 5, Opus 4.7/4.8, Fable 5) — sending one is a hard 400. A
+   * hardcoded list of which models accept it would rot exactly the way the
+   * pinned model ids did, so learn it from the API instead: the first call
+   * retries without the parameter and remembers, and every later call omits it
+   * up front.
+   */
+  private static readonly rejectsSampling = new Set<string>();
   private apiKey: string;
   private defaultModel: string;
 
@@ -66,7 +86,9 @@ export class AnthropicProvider implements LlmProvider {
     if (systemMessages.length > 0) {
       body.system = systemMessages.map((m) => m.content).join('\n');
     }
-    if (options.temperature !== undefined) body.temperature = options.temperature;
+    if (options.temperature !== undefined && !AnthropicProvider.rejectsSampling.has(model)) {
+      body.temperature = options.temperature;
+    }
 
     if (options.tools && options.tools.length > 0) {
       body.tools = options.tools.map((t) => ({
@@ -76,8 +98,19 @@ export class AnthropicProvider implements LlmProvider {
       }));
     }
 
-    const response = await this.request(body, options.signal);
+    try {
-    return parseAnthropicResponse(response);
+      return parseAnthropicResponse(await this.request(body, options.signal));
+    } catch (err) {
+      // Learn-and-retry rather than fail: a model that has dropped sampling
+      // support should cost one wasted call, once, not every call forever.
+      if (body.temperature !== undefined && isSamplingRejection(err)) {
+        AnthropicProvider.rejectsSampling.add(model);
+        delete body.temperature;
+        process.stderr.write(`[anthropic] ${model} rejects sampling params — retrying without\n`);
+        return parseAnthropicResponse(await this.request(body, options.signal));
+      }
+      throw err;
+    }
   }
 
   /**

src/mcplocal/src/proxymodel/plugins/content-pipeline.ts

@@ -137,9 +137,24 @@ function drillDownInstruction(toolName: string, resultId: string, sections: Sect
 /**
  * Declare the drill-down params on a tool's advertised input schema.
  *
- * Only `properties` is extended: `additionalProperties: false` stays as the
+ * `properties` is extended: `additionalProperties: false` stays as the
  * upstream set it, because a property listed in `properties` is allowed by
  * that keyword. Keeping it false preserves the upstream's typo protection.
+ *
+ * `required` also has to give way, or the contract is still unsatisfiable. A
+ * drill-down call carries only _resultId/_section — it never reaches the
+ * upstream, so the upstream's own required params (websearch/fetch_content
+ * requires `url`) are meaningless for it. Left in place, a client that
+ * validates arguments against the schema rejects the very call the stub just
+ * instructed the model to make ("Received tool input did not match expected
+ * schema"), and the model concludes pagination is broken.
+ *
+ * So the flat `required` becomes an alternation: either the upstream's
+ * requirements (a fresh call) or `_resultId` (a re-read of a cached result).
+ * A validator that understands `anyOf` enforces exactly that; a naive one that
+ * only looks at top-level `required` now finds none and accepts both. The
+ * upstream still rejects a fresh call that omits its required params, and the
+ * requirements stay visible to the model inside the alternation.
  */
 function withDrillDownParams(tool: ToolDefinition): ToolDefinition {
   if (GATE_TOOLS.has(tool.name)) return tool;
@@ -171,7 +186,42 @@ function withDrillDownParams(tool: ToolDefinition): ToolDefinition {
       + 'previous large result. Requires _resultId.',
   };
 
-  return { ...tool, inputSchema: { ...s, properties: props } };
+  return { ...tool, inputSchema: withDrillDownRequired({ ...s, properties: props }) };
+}
+
+/**
+ * Rewrite a flat `required` into "upstream's requirements OR a drill-down".
+ *
+ * Schemas that already carry a top-level combinator keep it: the alternation
+ * is appended to `allOf` instead of colliding with the existing `anyOf`/
+ * `oneOf`. A schema with no `required` needs nothing — every call already
+ * validates.
+ */
+function withDrillDownRequired(s: Record<string, unknown>): Record<string, unknown> {
+  const required = s['required'];
+  const upstreamRequired = Array.isArray(required)
+    ? required.filter((r): r is string => typeof r === 'string')
+    : [];
+  if (upstreamRequired.length === 0) return s;
+
+  const alternation = {
+    anyOf: [
+      { required: upstreamRequired },
+      // _resultId alone is a real call too: it re-shows the table of contents.
+      { required: ['_resultId'] },
+    ],
+  };
+
+  const next = { ...s };
+  delete next['required'];
+
+  if ('anyOf' in next || 'oneOf' in next || 'allOf' in next) {
+    const existing = Array.isArray(next['allOf']) ? next['allOf'] : [];
+    next['allOf'] = [...existing, alternation];
+    return next;
+  }
+
+  return { ...next, ...alternation };
 }
 
 /** Build a single-text-part tool result. */

src/mcplocal/tests/anthropic-model-resolution.test.ts

@@ -81,3 +81,68 @@ describe('Anthropic model resolution', () => {
     await expect(providerWith(MODELS).listModels()).resolves.toContain('claude-opus-5');
   });
 });
+
+describe('sampling-parameter rejection', () => {
+  afterEach(() => {
+    (AnthropicProvider as unknown as { rejectsSampling: Set<string> }).rejectsSampling.clear();
+  });
+
+  /** Rejects `temperature` exactly as the current Anthropic models do. */
+  function samplingStrictProvider(): { provider: AnthropicProvider; bodies: Array<Record<string, unknown>> } {
+    const provider = new AnthropicProvider({ apiKey: 'sk-ant-api-test' });
+    const bodies: Array<Record<string, unknown>> = [];
+    (provider as unknown as { request: (b: unknown) => Promise<unknown> }).request = async (b) => {
+      const body = b as Record<string, unknown>;
+      bodies.push({ ...body });
+      if (body.temperature !== undefined) {
+        throw new Error(
+          'Anthropic HTTP 400: {"type":"error","error":{"type":"invalid_request_error",'
+          + '"message":"`temperature` is deprecated for this model."}}',
+        );
+      }
+      return { content: [{ type: 'text', text: 'ok' }], stop_reason: 'end_turn' };
+    };
+    return { provider, bodies };
+  }
+
+  it('retries without temperature when the model rejects it', async () => {
+    vi.spyOn(process.stderr, 'write').mockImplementation(() => true);
+    const { provider, bodies } = samplingStrictProvider();
+
+    const result = await provider.complete({
+      model: 'claude-opus-5',
+      messages: [{ role: 'user', content: 'hi' }],
+      temperature: 0,
+    });
+
+    expect(result.content).toBe('ok');
+    expect(bodies).toHaveLength(2);
+    expect(bodies[0]!.temperature).toBe(0);
+    expect(bodies[1]!.temperature).toBeUndefined();
+  });
+
+  it('remembers, so the wasted call happens once and not forever', async () => {
+    vi.spyOn(process.stderr, 'write').mockImplementation(() => true);
+    const { provider, bodies } = samplingStrictProvider();
+    const req = { model: 'claude-opus-5', messages: [{ role: 'user' as const, content: 'hi' }], temperature: 0 };
+
+    await provider.complete(req);
+    await provider.complete(req);
+    await provider.complete(req);
+
+    // 2 for the first call (reject + retry), then 1 each — not 2 each.
+    expect(bodies).toHaveLength(4);
+  });
+
+  it('does not swallow unrelated 400s', async () => {
+    const provider = new AnthropicProvider({ apiKey: 'sk-ant-api-test' });
+    (provider as unknown as { request: () => Promise<unknown> }).request = async () => {
+      throw new Error('Anthropic HTTP 400: {"error":{"message":"max_tokens is required"}}');
+    };
+    await expect(provider.complete({
+      model: 'claude-opus-5',
+      messages: [{ role: 'user', content: 'hi' }],
+      temperature: 0,
+    })).rejects.toThrow(/max_tokens/);
+  });
+});

src/mcplocal/tests/plugin-content-pipeline-drilldown.test.ts

@@ -33,6 +33,20 @@ const STRICT_SCHEMA = {
   $schema: 'http://json-schema.org/draft-07/schema#',
 };
 
+/**
+ * Mirrors websearch/fetch_content: a required param that a drill-down call
+ * cannot supply, since the drill-down never reaches the upstream.
+ */
+const REQUIRED_SCHEMA = {
+  type: 'object',
+  properties: {
+    url: { type: 'string' },
+    site: { type: 'string' },
+  },
+  required: ['url'],
+  additionalProperties: false,
+};
+
 const BIG_PAYLOAD = 'x'.repeat(20_000);
 
 /**
@@ -74,7 +88,11 @@ interface Upstream {
   calls: Array<Record<string, unknown>>;
 }
 
-function mockUpstream(name: string, payloads: Record<string, string> = PAYLOADS): Upstream {
+function mockUpstream(
+  name: string,
+  payloads: Record<string, string> = PAYLOADS,
+  schema: unknown = STRICT_SCHEMA,
+): Upstream {
   const calls: Array<Record<string, unknown>> = [];
   const conn = {
     name,
@@ -90,7 +108,7 @@ function mockUpstream(name: string, payloads: Record<string, string> = PAYLOADS)
             tools: Object.keys(payloads).map((n) => ({
               name: n,
               description: `Retrieve ${n}`,
-              inputSchema: STRICT_SCHEMA,
+              inputSchema: schema,
             })),
           },
         };
@@ -120,7 +138,7 @@ function mockMcpdClient(): McpdClient {
   } as unknown as McpdClient;
 }
 
-function setup(opts: { gated?: boolean; payloads?: Record<string, string> } = {}) {
+function setup(opts: { gated?: boolean; payloads?: Record<string, string>; schema?: unknown } = {}) {
   const router = new McpRouter();
   router.setPromptConfig(mockMcpdClient(), 'test-project');
   router.setPlugin(createDefaultPlugin({ gated: opts.gated ?? false, providerRegistry: null }));
@@ -129,7 +147,7 @@ function setup(opts: { gated?: boolean; payloads?: Record<string, string> } = {}
     { complete: async () => '', available: () => false } as unknown as LLMProviderAdapter,
     new MemoryCache(),
   );
-  const upstream = mockUpstream('unifi-network', opts.payloads ?? PAYLOADS);
+  const upstream = mockUpstream('unifi-network', opts.payloads ?? PAYLOADS, opts.schema ?? STRICT_SCHEMA);
   router.addUpstream(upstream.conn);
   return { router, upstream };
 }
@@ -187,6 +205,63 @@ describe('content-pipeline drill-down contract', () => {
     expect(schema['additionalProperties']).toBe(false);
   });
 
+  it('does not leave the upstream\'s required params blocking a drill-down call', async () => {
+    // websearch/fetch_content requires `url`. A drill-down carries only
+    // _resultId/_section, so a client validating against a flat
+    // `required: ["url"]` rejected the exact call the stub asked for.
+    const { router } = setup({ schema: REQUIRED_SCHEMA });
+    const tool = (await listTools(router)).find((t) => t.name.endsWith('get_devices'));
+
+    const schema = tool!.inputSchema as Record<string, unknown>;
+    expect(schema['required']).toBeUndefined();
+    expect(schema['anyOf']).toEqual([
+      { required: ['url'] },
+      { required: ['_resultId'] },
+    ]);
+    // The upstream's requirement is preserved, not dropped: a fresh call still
+    // has to carry `url`.
+    expect((schema['properties'] as Record<string, unknown>)['url']).toMatchObject({ type: 'string' });
+  });
+
+  it('serves a drill-down that omits the upstream\'s required params', async () => {
+    const { router, upstream } = setup({ schema: REQUIRED_SCHEMA });
+    await listTools(router);
+
+    const stub = textOf(await callTool(router, { url: 'https://example.com' }, 3));
+    const resultId = /_resultId="([^"]+)"/.exec(stub)?.[1];
+    const callsAfterFirst = upstream.calls.length;
+
+    const page = textOf(await callTool(router, { _resultId: resultId!, _section: 'page-1' }, 4));
+
+    expect(page).toContain('x'.repeat(100));
+    expect(upstream.calls.length).toBe(callsAfterFirst);
+  });
+
+  it('leaves a schema without required params alone', async () => {
+    const { router } = setup();
+    const tool = (await listTools(router)).find((t) => t.name.endsWith('get_devices'));
+
+    const schema = tool!.inputSchema as Record<string, unknown>;
+    expect(schema['anyOf']).toBeUndefined();
+    expect(schema['allOf']).toBeUndefined();
+  });
+
+  it('composes with a schema that already carries a combinator', async () => {
+    const { router } = setup({
+      schema: { ...REQUIRED_SCHEMA, oneOf: [{ required: ['site'] }] },
+    });
+    const tool = (await listTools(router)).find((t) => t.name.endsWith('get_devices'));
+
+    const schema = tool!.inputSchema as Record<string, unknown>;
+    // The upstream's own combinator survives; ours is appended, not merged
+    // into it.
+    expect(schema['oneOf']).toEqual([{ required: ['site'] }]);
+    expect(schema['required']).toBeUndefined();
+    expect(schema['allOf']).toEqual([
+      { anyOf: [{ required: ['url'] }, { required: ['_resultId'] }] },
+    ]);
+  });
+
   it('leaves gate tools alone (they never reach the pipeline)', async () => {
     const { router } = setup({ gated: true });
     const tools = await listTools(router);

src/mcplocal/tests/smoke/mcp-client.ts

@@ -261,6 +261,21 @@ export async function isMcplocalRunning(): Promise<boolean> {
   }
 }
 
+/**
+ * Pull the drill-down id out of a paginated result or TOC.
+ *
+ * The instruction is written `_resultId="pm-abc"`, while the legacy paginator
+ * emits it as JSON (`"_resultId": "pm-abc"`). A regex matching only the JSON
+ * form silently found nothing in the current format, so drill-down checks
+ * either reported a false failure or skipped themselves as "not large enough
+ * for section-split" — both invisible in a green run. Accept either spelling
+ * and return the bare id.
+ */
+export function extractResultId(text: string): string | null {
+  const match = /_resultId"?\s*[:=]\s*"?(pm-[a-zA-Z0-9]+)/.exec(text);
+  return match?.[1] ?? null;
+}
+
 /**
  * Run an mcpctl CLI command and return stdout.
  */

src/mcplocal/tests/smoke/prompt-drilldown.test.ts

@@ -7,7 +7,7 @@
  * Requires: mcplocal running on localhost:3200, mcpd at https://mcpctl.ad.itaz.eu
  */
 import { describe, it, expect, beforeAll, afterAll } from 'vitest';
-import { SmokeMcpSession, isMcplocalRunning } from './mcp-client.js';
+import { SmokeMcpSession, isMcplocalRunning, extractResultId } from './mcp-client.js';
 
 const PROJECT_NAME = 'smoke-data';
 
@@ -90,9 +90,8 @@ describe('Smoke: Prompt section drill-down', () => {
         console.log(`  TOC preview: ${text.slice(0, 200)}...`);
 
         // Extract _resultId
-        const match = /_resultId:\s*(pm-[a-z0-9]+)/.exec(text);
+        const resultId = extractResultId(text);
-        if (match) {
+        if (resultId !== null) {
-          const resultId = match[1];
           // Extract first section id from TOC
           const sectionMatch = /\[([^\]]+)\]/.exec(text);
           if (sectionMatch) {

src/mcplocal/tests/smoke/proxy-pipeline.test.ts

@@ -17,7 +17,7 @@
 import { describe, it, expect, beforeAll, afterAll } from 'vitest';
 import { writeFile, mkdir, rm } from 'node:fs/promises';
 import { join, resolve } from 'node:path';
-import { SmokeMcpSession, isMcplocalRunning, mcpctl } from './mcp-client.js';
+import { SmokeMcpSession, isMcplocalRunning, mcpctl, extractResultId } from './mcp-client.js';
 import { ChatReporter } from './reporter.js';
 
 const PROJECT_NAME = 'smoke-data';
@@ -286,8 +286,7 @@ describe('Smoke: ProxyModel pipeline', () => {
       chat.check('Response is manageable size', text.length, (v) => v < 20_000);
 
       if (text.includes('_resultId')) {
-        const match = text.match(/_resultId:\s*(\S+)/);
+        chat.check('_resultId is present', extractResultId(text) !== null, (v) => v === true);
-        chat.check('_resultId is present', !!match, (v) => v === true);
       } else {
         chat.info('Content small enough — no pagination needed');
       }
@@ -307,14 +306,12 @@ describe('Smoke: ProxyModel pipeline', () => {
       });
       const text = result.content[0]?.text ?? '';
 
-      const match = text.match(/_resultId:\s*(\S+)/);
+      const resultId = extractResultId(text);
-      if (!match) {
+      if (resultId === null) {
         chat.info('Content not large enough for pagination — skip drill-down');
         return;
       }
 
-      const resultId = match[1]!.replace(/[^a-zA-Z0-9-]/g, '');
-
       const sectionResult = await chat.callTool('smoke-aws-docs_read_documentation', {
         url: 'https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html',
         _resultId: resultId,
@@ -401,8 +398,7 @@ describe('Smoke: ProxyModel pipeline', () => {
       chat.check('Response is manageable size', text.length, (v) => v < 20_000);
 
       if (text.includes('_resultId')) {
-        const match = text.match(/_resultId:\s*(\S+)/);
+        chat.check('Has _resultId for drill-down', extractResultId(text) !== null, (v) => v === true);
-        chat.check('Has _resultId for drill-down', !!match, (v) => v === true);
       }
     }, 60_000);
 
@@ -420,14 +416,12 @@ describe('Smoke: ProxyModel pipeline', () => {
       });
       const text = result.content[0]?.text ?? '';
 
-      const match = text.match(/_resultId:\s*(\S+)/);
+      const resultId = extractResultId(text);
-      if (!match) {
+      if (resultId === null) {
         chat.info('Content not large enough for section-split');
         return;
       }
 
-      const resultId = match[1]!.replace(/[^a-zA-Z0-9-]/g, '');
-
       const sectionResult = await chat.callTool('smoke-aws-docs_read_documentation', {
         url: 'https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html',
         _resultId: resultId,

src/mcplocal/tests/smoke/tool-drilldown.test.ts

@@ -20,9 +20,15 @@ const TOKEN = process.env['SMOKE_MCPTOKEN'];
 
 interface Tool {
   name: string;
-  inputSchema?: { type?: string; properties?: Record<string, unknown>; additionalProperties?: unknown };
+  inputSchema?: {
+    type?: string;
+    properties?: Record<string, unknown>;
+    additionalProperties?: unknown;
+    required?: unknown;
+  };
 }
 
+
 /** Tools served by the gate plugin — intercepted before the pipeline, so exempt. */
 const GATE_TOOLS = new Set(['begin_session', 'read_prompts', 'propose_prompt', 'propose_skill']);
 
@@ -97,6 +103,27 @@ describe('Smoke: tool drill-down contract', () => {
     console.log(`  ${strict.length} strict-schema tools keep additionalProperties: false`);
   });
 
+  it('required params never block a drill-down call', async () => {
+    if (!available) return;
+
+    // A drill-down carries only _resultId/_section. If the advertised schema
+    // still demands the upstream's own required params (websearch's `url`), a
+    // validating client rejects the call the stub just asked for — "Received
+    // tool input did not match expected schema".
+    // A top-level `required` is ANDed with everything else in the schema, so
+    // any entry there is unsatisfiable for a drill-down. The upstream's
+    // requirements live in an `anyOf` alternation instead.
+    const blocked = tools
+      .filter((t) => {
+        if (GATE_TOOLS.has(t.name)) return false;
+        const required = t.inputSchema?.required;
+        return Array.isArray(required) && required.length > 0;
+      })
+      .map((t) => t.name);
+
+    expect(blocked).toEqual([]);
+  });
+
   it('gate tools do not advertise drill-down params', async () => {
     if (!available) return;
 

templates/duckduckgo.yaml

@@ -5,11 +5,17 @@ packageName: "duckduckgo-mcp-server"
 runtime: python
 transport: STDIO
 repositoryUrl: https://github.com/nickclyde/duckduckgo-mcp-server
+# Readiness fetches a tiny static page instead of searching. A `search` probe is
+# a DuckDuckGo scrape every interval (1,440/day at 60s) from the same IP agents
+# search from, which is how that IP ends up answering CAPTCHA. This proves the
+# process and its outbound fetch path work, not that DuckDuckGo is answering.
 healthCheck:
-  tool: search
+  tool: fetch_content
   arguments:
-    query: "mcp"
+    url: "https://example.com"
-    max_results: 1
+    max_length: 500
+  intervalSeconds: 300
+  timeoutSeconds: 20
 env:
   - name: DDG_SAFE_SEARCH
     description: Result filtering — STRICT, MODERATE or OFF

templates/firecrawl.yaml

@@ -0,0 +1,28 @@
+name: firecrawl
+version: "1.0.0"
+description: Firecrawl MCP server for reading web pages as main-content markdown, JS-rendered pages included
+packageName: "firecrawl-mcp"
+runtime: node
+transport: STDIO
+repositoryUrl: https://github.com/firecrawl/firecrawl-mcp-server
+# Readiness scrapes a tiny static page: a pass proves the Firecrawl API answers
+# and can fetch the web, without a search. Firecrawl's own search needs a
+# SearXNG endpoint on self-hosted instances; use the searxng template for
+# search. 300s because each probe is a real outbound fetch.
+healthCheck:
+  tool: firecrawl_scrape
+  arguments:
+    url: "https://example.com"
+    formats: ["markdown"]
+    onlyMainContent: true
+  intervalSeconds: 300
+  timeoutSeconds: 60
+env:
+  - name: FIRECRAWL_API_URL
+    description: >-
+      Self-hosted Firecrawl API base URL (e.g. http://firecrawl-api:3002). Leave
+      unset only when using Firecrawl's cloud with FIRECRAWL_API_KEY.
+    required: false
+  - name: FIRECRAWL_API_KEY
+    description: API key — required for Firecrawl's cloud, optional for a self-hosted instance
+    required: false

templates/searxng.yaml

@@ -4,10 +4,12 @@ description: SearXNG MCP server for private web search and URL-to-markdown readi
 packageName: "mcp-searxng"
 transport: STDIO
 repositoryUrl: https://github.com/ihor-sokoliuk/mcp-searxng
+# Readiness reads SearXNG's /config, which proves the instance answers without
+# fanning out to its engines. A searxng_web_search probe is a real search every
+# interval (1,440/day at 60s): it burns API-key engines (braveapi, kagi) and
+# feeds the IP blocks scraping engines put on the instance.
 healthCheck:
-  tool: searxng_web_search
+  tool: searxng_instance_info
-  arguments:
-    query: "mcp"
 env:
   - name: SEARXNG_URL
     description: >-