f614e9bb98
Measured winner from the DGX-Spark bake-off (toolsim.py, 145-tool catalog): a
curated favourite/<tool> shortlist + the full all/<server>/<tool> catalog + a
load-bearing "prefer favourite/ first" instruction nearly halved wander (37→20)
and 2.5x'd first-pick (2→5/8) vs a flat catalog. The instruction is load-bearing;
enriching descriptions did not help.
- New mcplocal plugin `favourite-index.ts`: composes AFTER gate (no-ops while
gated), reshapes the ungated upstream catalog into favourite/ + all/, injects
the instruction (onInitialize), and rewrites presented names back to canonical
server/tool in onToolCallBefore so normal routing + content-pipeline still run.
Gate/agent virtual tools pass through untouched; favourites are upstream-only.
- compose.ts: onInitialize now concatenates plugin instructions (was first-non-null)
so favindex can contribute its banner alongside the gate's.
- Per-project config `Project.favouriteIndex` {enabled, tools[], maxFavourites};
surfaced to the proxy via discovery; wired at project-mcp-endpoint when enabled.
- Usage derivation: mcpd tool-usage ranking over tool_call_trace events
(normalizing presented names → canonical), GET /api/v1/audit/tool-usage, and
`mcpctl favourites suggest|list`.
- CLI: `create project` gains --favourite/--favourite-index/--max-favourites;
favouriteIndex round-trips through get -o yaml | apply -f. Completions regenerated.
- Tests: plugin unit (presentation, rewrite routing, gated no-op, collisions),
compose merge, canonicalizeToolName, buildFavouriteIndex, + a live smoke test.
- Docs: docs/tool-presentation.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
3.4 KiB
Tool presentation: the favourite-index
Big, flat MCP tool catalogs make LLMs wander — they call wrong tools before the right one, or find it then keep exploring. mcpctl (the proxy) decides what tool list the model sees, so presentation is a lever we can pull.
We measured it. On a 145-tool catalog with a wandering model
(kubernetes-deployment/scripts/model-eval/toolsim.py), the winning presentation
— favindex — nearly halved wander (37→20) and 2.5×'d first-pick (2→5/8) vs the
flat catalog. Enriching descriptions did not help; scoping/indexing did.
(Full table + method in the feedback_llm_tool_scoping memory.)
What it does
When enabled for a project, the proxy presents tools in two namespaces plus a load-bearing instruction:
favourite/<tool>— a curated, usage-weighted shortlist of the commonly-used tools, listed first.all/<server>/<tool>— the full catalog (the escape hatch).- An
initializeinstruction: "PREFER favourite/ … use all// only if nothing in favourites fits."
The instruction is not optional — the same two namespaces without it scored materially worse in the bake-off. Curate and tell the model to check favourites first.
Favourites are a subset: leave rare/cloud tools in all/ only, so the model
falls back correctly for the long tail.
How it composes
favourite-index is a proxymodel plugin
(src/mcplocal/src/proxymodel/plugins/favourite-index.ts) composed after the
gate/default plugin:
- While a session is gated, only
begin_sessionis visible — favourite-index no-ops. Afterbegin_sessionungates, the catalog is reshaped intofavourite/+all/. - Gate/agent virtual tools (
read_prompts,propose_prompt,agent-<name>chat) pass through unchanged at the top level. Favourites are drawn only from the upstream MCP catalog. - Calls to
favourite/…/all/…are rewritten back to the canonicalserver/toolinonToolCallBefore, so normal upstream routing and the content-pipeline (pagination/sectioning) still run.
Configuring it
Per-project, stored on the project as favouriteIndex
({ enabled, tools: ["server/tool", …], maxFavourites? }). Round-trips through
get project -o yaml | apply -f -.
# enable + pin the common tools
mcpctl create project myproj --force --favourite-index \
--favourite k8s/get_pods --favourite k8s/get_pod_logs \
--favourite gitea/create_pull_request --max-favourites 15
# what the proxy will present
mcpctl favourites list --project myproj
Deriving favourites from usage
Every tool call is recorded (AuditEvent tool_call_trace). Rank them and pin
the common ones:
mcpctl favourites suggest --project myproj --top 20 --window 30
This normalizes presented names back to canonical server/tool (so all/… and
favourite/… calls collapse), ranks by call count, and prints a ready-to-run
create project … --favourite … line. ★ marks tools already pinned.
Validating a change first
Before shipping a presentation change, run it through the fake-catalog harness — it caught "enrichment doesn't help" in ~1h instead of a wasted build:
toolsim.py <model> favindex all # vs `terse all` (flat baseline)
Then confirm it reproduces through the real proxy on a tool-heavy project
(presentation + routing: src/mcplocal/tests/smoke/favourite-index.test.ts).