75 lines
2.9 KiB
Markdown
75 lines
2.9 KiB
Markdown
|
# Task ID: 29
|
||
|
|
|
||
|
|
**Title:** Implement LLM pre-processing pipeline in mcplocal
|
||
|
|
|
||
|
|
**Status:** done
|
||
|
|
|
||
|
|
**Dependencies:** 25 ✓, 27 ✓, 28 ✓
|
||
|
|
|
||
|
|
**Priority:** high
|
||
|
|
|
||
|
|
**Description:** Create the core LLM pre-processing pipeline that intercepts MCP tool calls, uses a local LLM to optimize requests before sending to mcpd, and filters responses before returning to Claude.
|
||
|
|
|
||
|
|
**Details:**
|
||
|
|
|
||
|
|
1. Create `src/mcplocal/src/llm/processor.ts` - the core pipeline:
|
||
|
|
```typescript
|
||
|
|
export class LlmProcessor {
|
||
|
|
constructor(
|
||
|
|
private providerRegistry: ProviderRegistry,
|
||
|
|
private config: LlmProcessorConfig
|
||
|
|
) {}
|
||
|
|
|
||
|
|
async preprocessRequest(toolName: string, params: unknown): Promise<ProcessedRequest> {
|
||
|
|
// Use LLM to interpret and optimize the request
|
||
|
|
const prompt = this.buildRequestPrompt(toolName, params);
|
||
|
|
const result = await this.providerRegistry.getActiveProvider().complete({
|
||
|
|
systemPrompt: REQUEST_OPTIMIZATION_SYSTEM_PROMPT,
|
||
|
|
userPrompt: prompt
|
||
|
|
});
|
||
|
|
return this.parseOptimizedRequest(result);
|
||
|
|
}
|
||
|
|
|
||
|
|
async filterResponse(toolName: string, originalRequest: unknown, rawResponse: unknown): Promise<FilteredResponse> {
|
||
|
|
// Use LLM to filter/summarize the response
|
||
|
|
const prompt = this.buildFilterPrompt(toolName, originalRequest, rawResponse);
|
||
|
|
const result = await this.providerRegistry.getActiveProvider().complete({
|
||
|
|
systemPrompt: RESPONSE_FILTER_SYSTEM_PROMPT,
|
||
|
|
userPrompt: prompt
|
||
|
|
});
|
||
|
|
return this.parseFilteredResponse(result);
|
||
|
|
}
|
||
|
|
}
|
||
|
|
```
|
||
|
|
2. Create system prompts in `src/mcplocal/src/llm/prompts.ts`:
|
||
|
|
- REQUEST_OPTIMIZATION_SYSTEM_PROMPT: instruct LLM to generate optimal queries
|
||
|
|
- RESPONSE_FILTER_SYSTEM_PROMPT: instruct LLM to extract relevant information
|
||
|
|
3. Integrate into router.ts - wrap tools/call handler:
|
||
|
|
```typescript
|
||
|
|
async handleToolsCall(request: JsonRpcRequest) {
|
||
|
|
if (this.shouldPreprocess(request.params.name)) {
|
||
|
|
const processed = await this.llmProcessor.preprocessRequest(...);
|
||
|
|
// Call mcpd with processed request
|
||
|
|
const rawResponse = await this.callMcpd(processed);
|
||
|
|
const filtered = await this.llmProcessor.filterResponse(...);
|
||
|
|
return filtered;
|
||
|
|
}
|
||
|
|
return this.callMcpd(request.params);
|
||
|
|
}
|
||
|
|
```
|
||
|
|
4. Add configuration options:
|
||
|
|
- `enablePreprocessing`: boolean
|
||
|
|
- `preprocessingExclude`: string[] (tool names to skip)
|
||
|
|
- `preferredProvider`: string (ollama, gemini, deepseek, etc.)
|
||
|
|
5. Add bypass logic for simple operations (list, create, delete)
|
||
|
|
|
||
|
|
**Test Strategy:**
|
||
|
|
|
||
|
|
1. Unit test: Request preprocessing generates optimized queries
|
||
|
|
2. Unit test: Response filtering reduces data volume
|
||
|
|
3. Unit test: Bypass logic works for excluded tools
|
||
|
|
4. Integration test: Full pipeline with mock LLM provider
|
||
|
|
5. Test error handling: LLM failure falls back to unfiltered pass-through
|
||
|
|
6. Test configuration options are respected
|
||
|
|
7. Measure: response size reduction percentage
|