feat: eager vLLM warmup and smart page titles in paginate stage
- Add warmup() to LlmProvider interface for eager subprocess startup - ManagedVllmProvider.warmup() starts vLLM in background on project load - ProviderRegistry.warmupAll() triggers all managed providers - NamedProvider proxies warmup() to inner provider - paginate stage generates LLM-powered descriptive page titles when available, cached by content hash, falls back to generic "Page N" - project-mcp-endpoint calls warmupAll() on router creation so vLLM is loading while the session initializes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
75
README.md
75
README.md
@@ -150,6 +150,81 @@ mcpctl get all --project monitoring -o yaml > backup.yaml
|
||||
mcpctl apply -f backup.yaml
|
||||
```
|
||||
|
||||
## Content Pipeline (ProxyModel)
|
||||
|
||||
ProxyModel defines a **content transformation pipeline** that runs between upstream MCP servers and the client (e.g., Claude). It processes tool results, prompts, and resources through ordered stages before delivering them.
|
||||
|
||||
### Built-in Models
|
||||
|
||||
| Model | Stages | Use case |
|
||||
|-------|--------|----------|
|
||||
| **default** | `passthrough` → `paginate` (8KB pages) | Safe pass-through with pagination for large responses |
|
||||
| **subindex** | `section-split` → `summarize-tree` | Splits large content into sections, returns a summary index. Client drills down with `_resultId`/`_section` params |
|
||||
|
||||
### How `subindex` Works
|
||||
|
||||
1. Upstream returns a large tool result (e.g., 50KB of device states)
|
||||
2. `section-split` divides content into logical sections (2KB–15KB each)
|
||||
3. `summarize-tree` generates a compact index with section summaries (~200 tokens each)
|
||||
4. Client receives the index and can request specific sections via `_section` parameter
|
||||
|
||||
### Configuration
|
||||
|
||||
Set per-project (all servers use the same model):
|
||||
|
||||
```yaml
|
||||
kind: Project
|
||||
metadata:
|
||||
name: home-automation
|
||||
spec:
|
||||
servers: [home-assistant, node-red]
|
||||
proxyModel: subindex
|
||||
```
|
||||
|
||||
Override per-server within a project:
|
||||
|
||||
```yaml
|
||||
kind: Project
|
||||
metadata:
|
||||
name: monitoring
|
||||
spec:
|
||||
servers: [grafana, prometheus]
|
||||
proxyModel: default
|
||||
serverOverrides:
|
||||
grafana:
|
||||
proxyModel: subindex
|
||||
```
|
||||
|
||||
Via CLI:
|
||||
|
||||
```bash
|
||||
mcpctl create project monitoring --server grafana --server prometheus --proxy-model subindex
|
||||
```
|
||||
|
||||
### Custom ProxyModels
|
||||
|
||||
Place YAML files in `~/.mcpctl/proxymodels/` to define custom pipelines:
|
||||
|
||||
```yaml
|
||||
kind: ProxyModel
|
||||
metadata:
|
||||
name: my-pipeline
|
||||
spec:
|
||||
stages:
|
||||
- type: section-split
|
||||
config:
|
||||
minSectionSize: 1000
|
||||
maxSectionSize: 10000
|
||||
- type: summarize-tree
|
||||
config:
|
||||
maxTokens: 150
|
||||
maxDepth: 2
|
||||
appliesTo: [toolResult, prompt]
|
||||
cacheable: true
|
||||
```
|
||||
|
||||
Inspect available models: `mcpctl get proxymodels` / `mcpctl describe proxymodel subindex`
|
||||
|
||||
## Resources
|
||||
|
||||
| Resource | What it is | Example |
|
||||
|
||||
Reference in New Issue
Block a user