fix(templates): search templates stop probing with searches; add firecrawl
Some checks failed
CI/CD / typecheck (pull_request) Successful in 1m17s
CI/CD / test (pull_request) Successful in 1m24s
CI/CD / lint (pull_request) Successful in 2m56s
CI/CD / smoke (pull_request) Failing after 1m54s
CI/CD / build (pull_request) Successful in 4m19s
CI/CD / publish (pull_request) Has been skipped

Readiness probes ran real searches every interval: searxng's
searxng_web_search probe fanned a query out to every engine 1,440 times a
day, billing API-key engines (braveapi, kagi), and duckduckgo's `search`
probe scraped DuckDuckGo from the same IP agents search from.

- searxng: probe searxng_instance_info, which reads /config only
- duckduckgo: probe fetch_content on example.com every 300s
- firecrawl (new): firecrawl-mcp against a self-hosted or cloud
  Firecrawl, for reading pages as main-content markdown; probe
  firecrawl_scrape on example.com every 300s

Every probe still names a readiness tool, as templates.test.ts requires;
each was run against the real package before being written down.
docs/web-search.md now pairs searxng (search) with firecrawl (reading)
and says why duckduckgo does not hold up under agent traffic.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JPjtnE6Gd343oRNtMU9Bcd
This commit is contained in:
Michal
2026-09-15 22:16:17 +01:00
parent ad00e49262
commit 61e52403a3
5 changed files with 113 additions and 23 deletions

View File

@@ -4,10 +4,12 @@ description: SearXNG MCP server for private web search and URL-to-markdown readi
packageName: "mcp-searxng"
transport: STDIO
repositoryUrl: https://github.com/ihor-sokoliuk/mcp-searxng
# Readiness reads SearXNG's /config, which proves the instance answers without
# fanning out to its engines. A searxng_web_search probe is a real search every
# interval (1,440/day at 60s): it burns API-key engines (braveapi, kagi) and
# feeds the IP blocks scraping engines put on the instance.
healthCheck:
tool: searxng_web_search
arguments:
query: "mcp"
tool: searxng_instance_info
env:
- name: SEARXNG_URL
description: >-