Compare commits

..

11 Commits

Author SHA1 Message Date
Michal
39ca134201 fix: per-provider health checks in /llm/providers and status display
Some checks failed
CI / lint (pull_request) Has been cancelled
CI / typecheck (pull_request) Has been cancelled
CI / test (pull_request) Has been cancelled
CI / build (pull_request) Has been cancelled
CI / package (pull_request) Has been cancelled
The /llm/providers endpoint now runs isAvailable() on each provider in
parallel and returns health status per provider. The status command shows
✓/✗ per provider based on actual availability, not just the fast tier.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 02:25:06 +00:00
78a1dc9c8a Merge pull request 'feat: tiered LLM providers (fast/heavy)' (#43) from feat/tiered-llm-providers into main
Some checks are pending
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / test (push) Waiting to run
CI / build (push) Blocked by required conditions
CI / package (push) Blocked by required conditions
2026-02-25 02:16:29 +00:00
Michal
9ce705608b feat: tiered LLM providers (fast/heavy) with multi-provider config
Some checks failed
CI / lint (pull_request) Has been cancelled
CI / typecheck (pull_request) Has been cancelled
CI / test (pull_request) Has been cancelled
CI / build (pull_request) Has been cancelled
CI / package (pull_request) Has been cancelled
Adds tier-based LLM routing so fast local models (vLLM, Ollama) handle
structured tasks while cloud models (Gemini, Anthropic) are reserved for
heavy reasoning. Single-provider configs continue to work via fallback.

- Tier type + ProviderRegistry with assignTier/getProvider/fallback chain
- Multi-provider config format: { providers: [{ name, type, tier, ... }] }
- NamedProvider wrapper for multiple instances of same provider type
- Setup wizard: Simple (legacy) / Advanced (fast+heavy tiers) modes
- Status display: tiered view with /llm/providers endpoint
- Call sites use getProvider('fast') instead of getActive()
- Full backward compatibility with existing single-provider configs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 02:16:08 +00:00
Michal
0824f8e635 fix: cache LLM health check result for 10 minutes
Some checks are pending
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / test (push) Waiting to run
CI / build (push) Blocked by required conditions
CI / package (push) Blocked by required conditions
Avoids burning tokens on every `mcpctl status` call. The /llm/health
endpoint now caches successful results for 10min, errors for 1min.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 01:39:15 +00:00
Michal
9bd3127519 fix: warmup ACP subprocess eagerly to avoid 30s cold-start on status
Some checks are pending
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / test (push) Waiting to run
CI / build (push) Blocked by required conditions
CI / package (push) Blocked by required conditions
The pool refactor made ACP client creation lazy, causing the first
/llm/health call to spawn + initialize + prompt Gemini in one request
(30s+). Now warmup() eagerly starts the subprocess on mcplocal boot.
Also fetch models in parallel with LLM health check.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 01:37:30 +00:00
e8ac500ae9 Merge pull request 'feat: per-project LLM models, ACP session pool, smart pagination tests' (#42) from feat/per-project-llm-pagination-tests into main
Some checks are pending
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / test (push) Waiting to run
CI / build (push) Blocked by required conditions
CI / package (push) Blocked by required conditions
2026-02-25 01:29:56 +00:00
Michal
bed725b387 feat: per-project LLM models, ACP session pool, smart pagination tests
Some checks failed
CI / lint (pull_request) Has been cancelled
CI / typecheck (pull_request) Has been cancelled
CI / test (pull_request) Has been cancelled
CI / build (pull_request) Has been cancelled
CI / package (pull_request) Has been cancelled
- ACP session pool with per-model subprocesses and 8h idle eviction
- Per-project LLM config: local override → mcpd recommendation → global default
- Model override support in ResponsePaginator
- /llm/models endpoint + available models in mcpctl status
- Remove --llm-provider/--llm-model from create project (use edit/apply)
- 8 new smart pagination integration tests (e2e flow)
- 260 mcplocal tests, 330 CLI tests passing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 01:29:38 +00:00
17a456d835 Merge pull request 'feat: completions update, create promptrequest, LLM flag rename, ACP content fix' (#41) from feat/completions-llm-flags-promptrequest into main
Some checks are pending
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / test (push) Waiting to run
CI / build (push) Blocked by required conditions
CI / package (push) Blocked by required conditions
2026-02-25 00:21:51 +00:00
Michal
9481d394a1 feat: completions update, create promptrequest, LLM flag rename, ACP content fix
Some checks failed
CI / lint (pull_request) Has been cancelled
CI / typecheck (pull_request) Has been cancelled
CI / test (pull_request) Has been cancelled
CI / build (pull_request) Has been cancelled
CI / package (pull_request) Has been cancelled
- Add prompts/promptrequests to shell completions (fish + bash)
- Add approve, setup, prompt, promptrequest commands to completions
- Add `create promptrequest` CLI command (POST /projects/:name/promptrequests)
- Rename --proxy-mode-llm-provider/model to --llm-provider/model
- Fix ACP client: handle single-object content format from real Gemini
- Add tests for single-object content and agent_thought_chunk filtering

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 00:21:31 +00:00
Michal
bc769c4eeb fix: LLM health check via mcplocal instead of spawning gemini directly
Some checks are pending
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / test (push) Waiting to run
CI / build (push) Blocked by required conditions
CI / package (push) Blocked by required conditions
Status command now queries mcplocal's /llm/health endpoint instead of
spawning the gemini binary. This uses the persistent ACP connection
(fast) and works for any configured provider, not just gemini-cli.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 00:03:25 +00:00
6f534c8ba9 Merge pull request 'feat: persistent Gemini ACP provider + status spinner' (#40) from feat/gemini-acp-provider into main
Some checks are pending
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / test (push) Waiting to run
CI / build (push) Blocked by required conditions
CI / package (push) Blocked by required conditions
2026-02-24 23:52:31 +00:00
31 changed files with 1881 additions and 382 deletions

View File

@@ -2,10 +2,10 @@ _mcpctl() {
local cur prev words cword
_init_completion || return
local commands="status login logout config get describe delete logs create edit apply backup restore mcp help"
local commands="status login logout config get describe delete logs create edit apply backup restore mcp approve help"
local project_commands="attach-server detach-server get describe delete logs create edit help"
local global_opts="-v --version --daemon-url --direct --project -h --help"
local resources="servers instances secrets templates projects users groups rbac"
local resources="servers instances secrets templates projects users groups rbac prompts promptrequests"
# Check if --project was given
local has_project=false
@@ -78,7 +78,7 @@ _mcpctl() {
case "$subcmd" in
config)
if [[ $((cword - subcmd_pos)) -eq 1 ]]; then
COMPREPLY=($(compgen -W "view set path reset claude impersonate help" -- "$cur"))
COMPREPLY=($(compgen -W "view set path reset claude claude-generate setup impersonate help" -- "$cur"))
fi
return ;;
status)
@@ -114,7 +114,7 @@ _mcpctl() {
return ;;
create)
if [[ $((cword - subcmd_pos)) -eq 1 ]]; then
COMPREPLY=($(compgen -W "server secret project user group rbac help" -- "$cur"))
COMPREPLY=($(compgen -W "server secret project user group rbac prompt promptrequest help" -- "$cur"))
fi
return ;;
apply)
@@ -150,6 +150,15 @@ _mcpctl() {
fi
COMPREPLY=($(compgen -W "$names" -- "$cur"))
return ;;
approve)
if [[ -z "$resource_type" ]]; then
COMPREPLY=($(compgen -W "promptrequest" -- "$cur"))
else
local names
names=$(_mcpctl_resource_names "$resource_type")
COMPREPLY=($(compgen -W "$names" -- "$cur"))
fi
return ;;
help)
COMPREPLY=($(compgen -W "$commands" -- "$cur"))
return ;;

View File

@@ -3,7 +3,7 @@
# Erase any stale completions from previous versions
complete -c mcpctl -e
set -l commands status login logout config get describe delete logs create edit apply backup restore mcp help
set -l commands status login logout config get describe delete logs create edit apply backup restore mcp approve help
set -l project_commands attach-server detach-server get describe delete logs create edit help
# Disable file completions by default
@@ -28,7 +28,7 @@ function __mcpctl_has_project
end
# Helper: check if a resource type has been selected after get/describe/delete/edit
set -l resources servers instances secrets templates projects users groups rbac
set -l resources servers instances secrets templates projects users groups rbac prompts promptrequests
function __mcpctl_needs_resource_type
set -l tokens (commandline -opc)
@@ -36,11 +36,11 @@ function __mcpctl_needs_resource_type
for tok in $tokens
if $found_cmd
# Check if next token after get/describe/delete/edit is a resource type
if contains -- $tok servers instances secrets templates projects users groups rbac
if contains -- $tok servers instances secrets templates projects users groups rbac prompts promptrequests
return 1 # resource type already present
end
end
if contains -- $tok get describe delete edit
if contains -- $tok get describe delete edit approve
set found_cmd true
end
end
@@ -55,12 +55,12 @@ function __mcpctl_get_resource_type
set -l found_cmd false
for tok in $tokens
if $found_cmd
if contains -- $tok servers instances secrets templates projects users groups rbac
if contains -- $tok servers instances secrets templates projects users groups rbac prompts promptrequests
echo $tok
return
end
end
if contains -- $tok get describe delete edit
if contains -- $tok get describe delete edit approve
set found_cmd true
end
end
@@ -139,6 +139,7 @@ complete -c mcpctl -n "not __mcpctl_has_project; and not __fish_seen_subcommand_
complete -c mcpctl -n "not __mcpctl_has_project; and not __fish_seen_subcommand_from $commands" -a apply -d 'Apply configuration from file'
complete -c mcpctl -n "not __mcpctl_has_project; and not __fish_seen_subcommand_from $commands" -a backup -d 'Backup configuration'
complete -c mcpctl -n "not __mcpctl_has_project; and not __fish_seen_subcommand_from $commands" -a restore -d 'Restore from backup'
complete -c mcpctl -n "not __mcpctl_has_project; and not __fish_seen_subcommand_from $commands" -a approve -d 'Approve a prompt request'
complete -c mcpctl -n "not __mcpctl_has_project; and not __fish_seen_subcommand_from $commands" -a help -d 'Show help'
# Project-scoped commands (with --project)
@@ -157,7 +158,7 @@ complete -c mcpctl -n "__fish_seen_subcommand_from get describe delete; and __mc
complete -c mcpctl -n "__fish_seen_subcommand_from edit; and __mcpctl_needs_resource_type" -a 'servers projects' -d 'Resource type'
# Resource names — after resource type is selected
complete -c mcpctl -n "__fish_seen_subcommand_from get describe delete edit; and not __mcpctl_needs_resource_type" -a '(__mcpctl_resource_names)' -d 'Resource name'
complete -c mcpctl -n "__fish_seen_subcommand_from get describe delete edit approve; and not __mcpctl_needs_resource_type" -a '(__mcpctl_resource_names)' -d 'Resource name'
# Helper: check if attach-server/detach-server already has a server argument
function __mcpctl_needs_server_arg
@@ -196,22 +197,25 @@ complete -c mcpctl -n "__fish_seen_subcommand_from login" -l email -d 'Email add
complete -c mcpctl -n "__fish_seen_subcommand_from login" -l password -d 'Password' -x
# config subcommands
set -l config_cmds view set path reset claude claude-generate impersonate
set -l config_cmds view set path reset claude claude-generate setup impersonate
complete -c mcpctl -n "__fish_seen_subcommand_from config; and not __fish_seen_subcommand_from $config_cmds" -a view -d 'Show configuration'
complete -c mcpctl -n "__fish_seen_subcommand_from config; and not __fish_seen_subcommand_from $config_cmds" -a set -d 'Set a config value'
complete -c mcpctl -n "__fish_seen_subcommand_from config; and not __fish_seen_subcommand_from $config_cmds" -a path -d 'Show config file path'
complete -c mcpctl -n "__fish_seen_subcommand_from config; and not __fish_seen_subcommand_from $config_cmds" -a reset -d 'Reset to defaults'
complete -c mcpctl -n "__fish_seen_subcommand_from config; and not __fish_seen_subcommand_from $config_cmds" -a claude -d 'Generate .mcp.json for project'
complete -c mcpctl -n "__fish_seen_subcommand_from config; and not __fish_seen_subcommand_from $config_cmds" -a setup -d 'Configure LLM provider'
complete -c mcpctl -n "__fish_seen_subcommand_from config; and not __fish_seen_subcommand_from $config_cmds" -a impersonate -d 'Impersonate a user'
# create subcommands
set -l create_cmds server secret project user group rbac
set -l create_cmds server secret project user group rbac prompt promptrequest
complete -c mcpctl -n "__fish_seen_subcommand_from create; and not __fish_seen_subcommand_from $create_cmds" -a server -d 'Create a server'
complete -c mcpctl -n "__fish_seen_subcommand_from create; and not __fish_seen_subcommand_from $create_cmds" -a secret -d 'Create a secret'
complete -c mcpctl -n "__fish_seen_subcommand_from create; and not __fish_seen_subcommand_from $create_cmds" -a project -d 'Create a project'
complete -c mcpctl -n "__fish_seen_subcommand_from create; and not __fish_seen_subcommand_from $create_cmds" -a user -d 'Create a user'
complete -c mcpctl -n "__fish_seen_subcommand_from create; and not __fish_seen_subcommand_from $create_cmds" -a group -d 'Create a group'
complete -c mcpctl -n "__fish_seen_subcommand_from create; and not __fish_seen_subcommand_from $create_cmds" -a rbac -d 'Create an RBAC binding'
complete -c mcpctl -n "__fish_seen_subcommand_from create; and not __fish_seen_subcommand_from $create_cmds" -a prompt -d 'Create an approved prompt'
complete -c mcpctl -n "__fish_seen_subcommand_from create; and not __fish_seen_subcommand_from $create_cmds" -a promptrequest -d 'Create a prompt request'
# logs options
complete -c mcpctl -n "__fish_seen_subcommand_from logs" -l tail -d 'Number of lines' -x
@@ -227,6 +231,9 @@ complete -c mcpctl -n "__fish_seen_subcommand_from restore" -s i -l input -d 'In
complete -c mcpctl -n "__fish_seen_subcommand_from restore" -s p -l password -d 'Decryption password' -x
complete -c mcpctl -n "__fish_seen_subcommand_from restore" -s c -l conflict -d 'Conflict strategy' -xa 'skip overwrite fail'
# approve: first arg is resource type (promptrequest only), second is name
complete -c mcpctl -n "__fish_seen_subcommand_from approve; and __mcpctl_needs_resource_type" -a 'promptrequest' -d 'Resource type'
# apply takes a file
complete -c mcpctl -n "__fish_seen_subcommand_from apply" -s f -l file -d 'Configuration file' -rF
complete -c mcpctl -n "__fish_seen_subcommand_from apply" -F

View File

@@ -4,7 +4,7 @@ import https from 'node:https';
import { execFile } from 'node:child_process';
import { promisify } from 'node:util';
import { loadConfig, saveConfig } from '../config/index.js';
import type { ConfigLoaderDeps, McpctlConfig, LlmConfig, LlmProviderName } from '../config/index.js';
import type { ConfigLoaderDeps, McpctlConfig, LlmConfig, LlmProviderName, LlmProviderEntry, LlmTier } from '../config/index.js';
import type { SecretStore } from '@mcpctl/shared';
import { createSecretStore } from '@mcpctl/shared';
@@ -32,13 +32,28 @@ interface ProviderChoice {
description: string;
}
const PROVIDER_CHOICES: ProviderChoice[] = [
{ name: 'Gemini CLI', value: 'gemini-cli', description: 'Google Gemini via local CLI (free, no API key)' },
{ name: 'Ollama', value: 'ollama', description: 'Local models via Ollama' },
{ name: 'Anthropic (Claude)', value: 'anthropic', description: 'Claude API (requires API key)' },
/** Provider config fields returned by per-provider setup functions. */
interface ProviderFields {
model?: string;
url?: string;
binaryPath?: string;
}
const FAST_PROVIDER_CHOICES: ProviderChoice[] = [
{ name: 'vLLM', value: 'vllm', description: 'Self-hosted vLLM (OpenAI-compatible)' },
{ name: 'Ollama', value: 'ollama', description: 'Local models via Ollama' },
];
const HEAVY_PROVIDER_CHOICES: ProviderChoice[] = [
{ name: 'Gemini CLI', value: 'gemini-cli', description: 'Google Gemini via local CLI (free, no API key)' },
{ name: 'Anthropic (Claude)', value: 'anthropic', description: 'Claude API (requires API key)' },
{ name: 'OpenAI', value: 'openai', description: 'OpenAI API (requires API key)' },
{ name: 'DeepSeek', value: 'deepseek', description: 'DeepSeek API (requires API key)' },
];
const ALL_PROVIDER_CHOICES: ProviderChoice[] = [
...FAST_PROVIDER_CHOICES,
...HEAVY_PROVIDER_CHOICES,
{ name: 'None (disable)', value: 'none', description: 'Disable LLM features' },
];
@@ -145,6 +160,283 @@ async function defaultWhichBinary(name: string): Promise<string | null> {
}
}
// --- Per-provider setup functions (return ProviderFields for reuse in both modes) ---
async function setupGeminiCliFields(
prompt: ConfigSetupPrompt,
log: (...args: string[]) => void,
whichBinary: (name: string) => Promise<string | null>,
currentModel?: string,
): Promise<ProviderFields> {
const model = await prompt.select<string>('Select model:', [
...GEMINI_MODELS.map((m) => ({
name: m === currentModel ? `${m} (current)` : m,
value: m,
})),
{ name: 'Custom...', value: '__custom__' },
]);
const finalModel = model === '__custom__'
? await prompt.input('Model name:', currentModel)
: model;
let binaryPath: string | undefined;
const detected = await whichBinary('gemini');
if (detected) {
log(`Found gemini at: ${detected}`);
binaryPath = detected;
} else {
log('Warning: gemini binary not found in PATH');
const manualPath = await prompt.input('Binary path (or install with: npm i -g @google/gemini-cli):');
if (manualPath) binaryPath = manualPath;
}
const result: ProviderFields = { model: finalModel };
if (binaryPath) result.binaryPath = binaryPath;
return result;
}
async function setupOllamaFields(
prompt: ConfigSetupPrompt,
fetchModels: ConfigSetupDeps['fetchModels'],
currentUrl?: string,
currentModel?: string,
): Promise<ProviderFields> {
const url = await prompt.input('Ollama URL:', currentUrl ?? 'http://localhost:11434');
const models = await fetchModels(url, '/api/tags');
let model: string;
if (models.length > 0) {
const choices = models.map((m) => ({
name: m === currentModel ? `${m} (current)` : m,
value: m,
}));
choices.push({ name: 'Custom...', value: '__custom__' });
model = await prompt.select<string>('Select model:', choices);
if (model === '__custom__') {
model = await prompt.input('Model name:', currentModel);
}
} else {
model = await prompt.input('Model name (could not fetch models):', currentModel ?? 'llama3.2');
}
const result: ProviderFields = { model };
if (url) result.url = url;
return result;
}
async function setupVllmFields(
prompt: ConfigSetupPrompt,
fetchModels: ConfigSetupDeps['fetchModels'],
currentUrl?: string,
currentModel?: string,
): Promise<ProviderFields> {
const url = await prompt.input('vLLM URL:', currentUrl ?? 'http://localhost:8000');
const models = await fetchModels(url, '/v1/models');
let model: string;
if (models.length > 0) {
const choices = models.map((m) => ({
name: m === currentModel ? `${m} (current)` : m,
value: m,
}));
choices.push({ name: 'Custom...', value: '__custom__' });
model = await prompt.select<string>('Select model:', choices);
if (model === '__custom__') {
model = await prompt.input('Model name:', currentModel);
}
} else {
model = await prompt.input('Model name (could not fetch models):', currentModel ?? 'default');
}
const result: ProviderFields = { model };
if (url) result.url = url;
return result;
}
async function setupApiKeyFields(
prompt: ConfigSetupPrompt,
secretStore: SecretStore,
provider: LlmProviderName,
secretKey: string,
hardcodedModels: string[],
currentModel?: string,
currentUrl?: string,
): Promise<ProviderFields> {
const existingKey = await secretStore.get(secretKey);
let apiKey: string;
if (existingKey) {
const masked = `****${existingKey.slice(-4)}`;
const changeKey = await prompt.confirm(`API key stored (${masked}). Change it?`, false);
apiKey = changeKey ? await prompt.password('API key:') : existingKey;
} else {
apiKey = await prompt.password('API key:');
}
if (apiKey !== existingKey) {
await secretStore.set(secretKey, apiKey);
}
let model: string;
if (hardcodedModels.length > 0) {
const choices = hardcodedModels.map((m) => ({
name: m === currentModel ? `${m} (current)` : m,
value: m,
}));
choices.push({ name: 'Custom...', value: '__custom__' });
model = await prompt.select<string>('Select model:', choices);
if (model === '__custom__') {
model = await prompt.input('Model name:', currentModel);
}
} else {
model = await prompt.input('Model name:', currentModel ?? 'gpt-4o');
}
let url: string | undefined;
if (provider === 'openai') {
const customUrl = await prompt.confirm('Use custom API endpoint?', false);
if (customUrl) {
url = await prompt.input('API URL:', currentUrl ?? 'https://api.openai.com');
}
}
const result: ProviderFields = { model };
if (url) result.url = url;
return result;
}
/** Configure a single provider type and return its fields. */
async function setupProviderFields(
providerType: LlmProviderName,
prompt: ConfigSetupPrompt,
log: (...args: string[]) => void,
fetchModels: ConfigSetupDeps['fetchModels'],
whichBinary: (name: string) => Promise<string | null>,
secretStore: SecretStore,
): Promise<ProviderFields> {
switch (providerType) {
case 'gemini-cli':
return setupGeminiCliFields(prompt, log, whichBinary);
case 'ollama':
return setupOllamaFields(prompt, fetchModels);
case 'vllm':
return setupVllmFields(prompt, fetchModels);
case 'anthropic':
return setupApiKeyFields(prompt, secretStore, 'anthropic', 'anthropic-api-key', ANTHROPIC_MODELS);
case 'openai':
return setupApiKeyFields(prompt, secretStore, 'openai', 'openai-api-key', []);
case 'deepseek':
return setupApiKeyFields(prompt, secretStore, 'deepseek', 'deepseek-api-key', DEEPSEEK_MODELS);
default:
return {};
}
}
/** Build a LlmProviderEntry from type, name, and fields. */
function buildEntry(providerType: LlmProviderName, name: string, fields: ProviderFields, tier?: LlmTier): LlmProviderEntry {
const entry: LlmProviderEntry = { name, type: providerType };
if (fields.model) entry.model = fields.model;
if (fields.url) entry.url = fields.url;
if (fields.binaryPath) entry.binaryPath = fields.binaryPath;
if (tier) entry.tier = tier;
return entry;
}
/** Simple mode: single provider (legacy format). */
async function simpleSetup(
config: McpctlConfig,
configDeps: Partial<ConfigLoaderDeps>,
prompt: ConfigSetupPrompt,
log: (...args: string[]) => void,
fetchModels: ConfigSetupDeps['fetchModels'],
whichBinary: (name: string) => Promise<string | null>,
secretStore: SecretStore,
): Promise<void> {
const currentLlm = config.llm && 'provider' in config.llm ? config.llm : undefined;
const choices = ALL_PROVIDER_CHOICES.map((c) => {
if (currentLlm?.provider === c.value) {
return { ...c, name: `${c.name} (current)` };
}
return c;
});
const provider = await prompt.select<LlmProviderName>('Select LLM provider:', choices);
if (provider === 'none') {
const updated: McpctlConfig = { ...config, llm: { provider: 'none' } };
saveConfig(updated, configDeps);
log('LLM disabled. Restart mcplocal: systemctl --user restart mcplocal');
return;
}
const fields = await setupProviderFields(provider, prompt, log, fetchModels, whichBinary, secretStore);
const llmConfig: LlmConfig = { provider, ...fields };
const updated: McpctlConfig = { ...config, llm: llmConfig };
saveConfig(updated, configDeps);
log(`\nLLM configured: ${llmConfig.provider}${llmConfig.model ? ` / ${llmConfig.model}` : ''}`);
log('Restart mcplocal: systemctl --user restart mcplocal');
}
/** Advanced mode: multiple providers with tier assignments. */
async function advancedSetup(
config: McpctlConfig,
configDeps: Partial<ConfigLoaderDeps>,
prompt: ConfigSetupPrompt,
log: (...args: string[]) => void,
fetchModels: ConfigSetupDeps['fetchModels'],
whichBinary: (name: string) => Promise<string | null>,
secretStore: SecretStore,
): Promise<void> {
const entries: LlmProviderEntry[] = [];
// Fast providers
const addFast = await prompt.confirm('Add a FAST provider? (vLLM, Ollama — local, cheap, fast)', true);
if (addFast) {
let addMore = true;
while (addMore) {
const providerType = await prompt.select<LlmProviderName>('Fast provider type:', FAST_PROVIDER_CHOICES);
const defaultName = providerType === 'vllm' ? 'vllm-local' : providerType;
const name = await prompt.input('Provider name:', defaultName);
const fields = await setupProviderFields(providerType, prompt, log, fetchModels, whichBinary, secretStore);
entries.push(buildEntry(providerType, name, fields, 'fast'));
log(` Added: ${name} (${providerType}) → fast tier`);
addMore = await prompt.confirm('Add another fast provider?', false);
}
}
// Heavy providers
const addHeavy = await prompt.confirm('Add a HEAVY provider? (Gemini, Anthropic, OpenAI — cloud, smart)', true);
if (addHeavy) {
let addMore = true;
while (addMore) {
const providerType = await prompt.select<LlmProviderName>('Heavy provider type:', HEAVY_PROVIDER_CHOICES);
const defaultName = providerType;
const name = await prompt.input('Provider name:', defaultName);
const fields = await setupProviderFields(providerType, prompt, log, fetchModels, whichBinary, secretStore);
entries.push(buildEntry(providerType, name, fields, 'heavy'));
log(` Added: ${name} (${providerType}) → heavy tier`);
addMore = await prompt.confirm('Add another heavy provider?', false);
}
}
if (entries.length === 0) {
log('No providers configured.');
return;
}
// Summary
log('\nProvider configuration:');
for (const e of entries) {
log(` ${e.tier ?? 'unassigned'}: ${e.name} (${e.type})${e.model ? ` / ${e.model}` : ''}`);
}
const updated: McpctlConfig = { ...config, llm: { providers: entries } };
saveConfig(updated, configDeps);
log('\nRestart mcplocal: systemctl --user restart mcplocal');
}
export function createConfigSetupCommand(deps?: Partial<ConfigSetupDeps>): Command {
return new Command('setup')
.description('Interactive LLM provider setup wizard')
@@ -157,191 +449,16 @@ export function createConfigSetupCommand(deps?: Partial<ConfigSetupDeps>): Comma
const secretStore = deps?.secretStore ?? await createSecretStore();
const config = loadConfig(configDeps);
const currentLlm = config.llm;
// Annotate current provider in choices
const choices = PROVIDER_CHOICES.map((c) => {
if (currentLlm?.provider === c.value) {
return { ...c, name: `${c.name} (current)` };
}
return c;
});
const mode = await prompt.select<'simple' | 'advanced'>('Setup mode:', [
{ name: 'Simple', value: 'simple', description: 'One provider for everything' },
{ name: 'Advanced', value: 'advanced', description: 'Multiple providers with fast/heavy tiers' },
]);
const provider = await prompt.select<LlmProviderName>('Select LLM provider:', choices);
if (provider === 'none') {
const updated: McpctlConfig = { ...config, llm: { provider: 'none' } };
saveConfig(updated, configDeps);
log('LLM disabled. Restart mcplocal: systemctl --user restart mcplocal');
return;
if (mode === 'simple') {
await simpleSetup(config, configDeps, prompt, log, fetchModels, whichBinary, secretStore);
} else {
await advancedSetup(config, configDeps, prompt, log, fetchModels, whichBinary, secretStore);
}
let llmConfig: LlmConfig;
switch (provider) {
case 'gemini-cli':
llmConfig = await setupGeminiCli(prompt, log, whichBinary, currentLlm);
break;
case 'ollama':
llmConfig = await setupOllama(prompt, fetchModels, currentLlm);
break;
case 'anthropic':
llmConfig = await setupApiKeyProvider(prompt, secretStore, 'anthropic', 'anthropic-api-key', ANTHROPIC_MODELS, currentLlm);
break;
case 'vllm':
llmConfig = await setupVllm(prompt, fetchModels, currentLlm);
break;
case 'openai':
llmConfig = await setupApiKeyProvider(prompt, secretStore, 'openai', 'openai-api-key', [], currentLlm);
break;
case 'deepseek':
llmConfig = await setupApiKeyProvider(prompt, secretStore, 'deepseek', 'deepseek-api-key', DEEPSEEK_MODELS, currentLlm);
break;
default:
return;
}
const updated: McpctlConfig = { ...config, llm: llmConfig };
saveConfig(updated, configDeps);
log(`\nLLM configured: ${llmConfig.provider}${llmConfig.model ? ` / ${llmConfig.model}` : ''}`);
log('Restart mcplocal: systemctl --user restart mcplocal');
});
}
async function setupGeminiCli(
prompt: ConfigSetupPrompt,
log: (...args: string[]) => void,
whichBinary: (name: string) => Promise<string | null>,
current?: LlmConfig,
): Promise<LlmConfig> {
const model = await prompt.select<string>('Select model:', [
...GEMINI_MODELS.map((m) => ({
name: m === current?.model ? `${m} (current)` : m,
value: m,
})),
{ name: 'Custom...', value: '__custom__' },
]);
const finalModel = model === '__custom__'
? await prompt.input('Model name:', current?.model)
: model;
// Auto-detect gemini binary path
let binaryPath: string | undefined;
const detected = await whichBinary('gemini');
if (detected) {
log(`Found gemini at: ${detected}`);
binaryPath = detected;
} else {
log('Warning: gemini binary not found in PATH');
const manualPath = await prompt.input('Binary path (or install with: npm i -g @google/gemini-cli):');
if (manualPath) binaryPath = manualPath;
}
return { provider: 'gemini-cli', model: finalModel, binaryPath };
}
async function setupOllama(prompt: ConfigSetupPrompt, fetchModels: ConfigSetupDeps['fetchModels'], current?: LlmConfig): Promise<LlmConfig> {
const url = await prompt.input('Ollama URL:', current?.url ?? 'http://localhost:11434');
// Try to fetch models from Ollama
const models = await fetchModels(url, '/api/tags');
let model: string;
if (models.length > 0) {
const choices = models.map((m) => ({
name: m === current?.model ? `${m} (current)` : m,
value: m,
}));
choices.push({ name: 'Custom...', value: '__custom__' });
model = await prompt.select<string>('Select model:', choices);
if (model === '__custom__') {
model = await prompt.input('Model name:', current?.model);
}
} else {
model = await prompt.input('Model name (could not fetch models):', current?.model ?? 'llama3.2');
}
return { provider: 'ollama', model, url };
}
async function setupVllm(prompt: ConfigSetupPrompt, fetchModels: ConfigSetupDeps['fetchModels'], current?: LlmConfig): Promise<LlmConfig> {
const url = await prompt.input('vLLM URL:', current?.url ?? 'http://localhost:8000');
// Try to fetch models from vLLM (OpenAI-compatible)
const models = await fetchModels(url, '/v1/models');
let model: string;
if (models.length > 0) {
const choices = models.map((m) => ({
name: m === current?.model ? `${m} (current)` : m,
value: m,
}));
choices.push({ name: 'Custom...', value: '__custom__' });
model = await prompt.select<string>('Select model:', choices);
if (model === '__custom__') {
model = await prompt.input('Model name:', current?.model);
}
} else {
model = await prompt.input('Model name (could not fetch models):', current?.model ?? 'default');
}
return { provider: 'vllm', model, url };
}
async function setupApiKeyProvider(
prompt: ConfigSetupPrompt,
secretStore: SecretStore,
provider: LlmProviderName,
secretKey: string,
hardcodedModels: string[],
current?: LlmConfig,
): Promise<LlmConfig> {
// Check for existing API key
const existingKey = await secretStore.get(secretKey);
let apiKey: string;
if (existingKey) {
const masked = `****${existingKey.slice(-4)}`;
const changeKey = await prompt.confirm(`API key stored (${masked}). Change it?`, false);
if (changeKey) {
apiKey = await prompt.password('API key:');
} else {
apiKey = existingKey;
}
} else {
apiKey = await prompt.password('API key:');
}
// Store API key
if (apiKey !== existingKey) {
await secretStore.set(secretKey, apiKey);
}
// Model selection
let model: string;
if (hardcodedModels.length > 0) {
const choices = hardcodedModels.map((m) => ({
name: m === current?.model ? `${m} (current)` : m,
value: m,
}));
choices.push({ name: 'Custom...', value: '__custom__' });
model = await prompt.select<string>('Select model:', choices);
if (model === '__custom__') {
model = await prompt.input('Model name:', current?.model);
}
} else {
model = await prompt.input('Model name:', current?.model ?? 'gpt-4o');
}
// Optional custom URL for openai
let url: string | undefined;
if (provider === 'openai') {
const customUrl = await prompt.confirm('Use custom API endpoint?', false);
if (customUrl) {
url = await prompt.input('API URL:', current?.url ?? 'https://api.openai.com');
}
}
return { provider, model, url };
}

View File

@@ -196,8 +196,6 @@ export function createCreateCommand(deps: CreateCommandDeps): Command {
.argument('<name>', 'Project name')
.option('-d, --description <text>', 'Project description', '')
.option('--proxy-mode <mode>', 'Proxy mode (direct, filtered)')
.option('--proxy-mode-llm-provider <name>', 'LLM provider name (for filtered proxy mode)')
.option('--proxy-mode-llm-model <name>', 'LLM model name (for filtered proxy mode)')
.option('--prompt <text>', 'Project-level prompt / instructions for the LLM')
.option('--server <name>', 'Server name (repeat for multiple)', collect, [])
.option('--force', 'Update if already exists')
@@ -208,8 +206,6 @@ export function createCreateCommand(deps: CreateCommandDeps): Command {
proxyMode: opts.proxyMode ?? 'direct',
};
if (opts.prompt) body.prompt = opts.prompt;
if (opts.proxyModeLlmProvider) body.llmProvider = opts.proxyModeLlmProvider;
if (opts.proxyModeLlmModel) body.llmModel = opts.proxyModeLlmModel;
if (opts.server.length > 0) body.servers = opts.server;
try {
@@ -379,5 +375,31 @@ export function createCreateCommand(deps: CreateCommandDeps): Command {
log(`prompt '${prompt.name}' created (id: ${prompt.id})`);
});
// --- create promptrequest ---
cmd.command('promptrequest')
.description('Create a prompt request (pending proposal that needs approval)')
.argument('<name>', 'Prompt request name (lowercase alphanumeric with hyphens)')
.requiredOption('--project <name>', 'Project name (required)')
.option('--content <text>', 'Prompt content text')
.option('--content-file <path>', 'Read prompt content from file')
.action(async (name: string, opts) => {
let content = opts.content as string | undefined;
if (opts.contentFile) {
const fs = await import('node:fs/promises');
content = await fs.readFile(opts.contentFile as string, 'utf-8');
}
if (!content) {
throw new Error('--content or --content-file is required');
}
const projectName = opts.project as string;
const pr = await client.post<{ id: string; name: string }>(
`/api/v1/projects/${encodeURIComponent(projectName)}/promptrequests`,
{ name, content },
);
log(`prompt request '${pr.name}' created (id: ${pr.id})`);
log(` approve with: mcpctl approve promptrequest ${pr.name}`);
});
return cmd;
}

View File

@@ -1,16 +1,12 @@
import { Command } from 'commander';
import http from 'node:http';
import { execFile } from 'node:child_process';
import { promisify } from 'node:util';
import { loadConfig } from '../config/index.js';
import type { ConfigLoaderDeps, LlmConfig } from '../config/index.js';
import type { ConfigLoaderDeps } from '../config/index.js';
import { loadCredentials } from '../auth/index.js';
import type { CredentialsDeps } from '../auth/index.js';
import { formatJson, formatYaml } from '../formatters/index.js';
import { APP_VERSION } from '@mcpctl/shared';
const execFileAsync = promisify(execFile);
// ANSI helpers
const GREEN = '\x1b[32m';
const RED = '\x1b[31m';
@@ -18,13 +14,24 @@ const DIM = '\x1b[2m';
const RESET = '\x1b[0m';
const CLEAR_LINE = '\x1b[2K\r';
interface ProvidersInfo {
providers: string[];
tiers: { fast: string[]; heavy: string[] };
health: Record<string, boolean>;
}
export interface StatusCommandDeps {
configDeps: Partial<ConfigLoaderDeps>;
credentialsDeps: Partial<CredentialsDeps>;
log: (...args: string[]) => void;
write: (text: string) => void;
checkHealth: (url: string) => Promise<boolean>;
checkLlm: (llm: LlmConfig) => Promise<string>;
/** Check LLM health via mcplocal's /llm/health endpoint */
checkLlm: (mcplocalUrl: string) => Promise<string>;
/** Fetch available models from mcplocal's /llm/models endpoint */
fetchModels: (mcplocalUrl: string) => Promise<string[]>;
/** Fetch provider tier info from mcplocal's /llm/providers endpoint */
fetchProviders: (mcplocalUrl: string) => Promise<ProvidersInfo | null>;
isTTY: boolean;
}
@@ -43,34 +50,72 @@ function defaultCheckHealth(url: string): Promise<boolean> {
}
/**
* Quick LLM health check. Returns 'ok', 'binary not found', 'auth error', etc.
* Check LLM health by querying mcplocal's /llm/health endpoint.
* This tests the actual provider running inside the daemon (uses persistent ACP for gemini, etc.)
*/
async function defaultCheckLlm(llm: LlmConfig): Promise<string> {
if (llm.provider === 'gemini-cli') {
const bin = llm.binaryPath ?? 'gemini';
try {
const { stdout } = await execFileAsync(bin, ['-p', 'respond with exactly: ok', '-m', llm.model ?? 'gemini-2.5-flash', '-o', 'text'], { timeout: 15000 });
return stdout.trim().toLowerCase().includes('ok') ? 'ok' : 'unexpected response';
} catch (err) {
const msg = (err as Error).message;
if (msg.includes('ENOENT')) return 'binary not found';
if (msg.includes('auth') || msg.includes('token') || msg.includes('login') || msg.includes('401')) return 'not authenticated';
return `error: ${msg.slice(0, 80)}`;
}
}
function defaultCheckLlm(mcplocalUrl: string): Promise<string> {
return new Promise((resolve) => {
const req = http.get(`${mcplocalUrl}/llm/health`, { timeout: 45000 }, (res) => {
const chunks: Buffer[] = [];
res.on('data', (chunk: Buffer) => chunks.push(chunk));
res.on('end', () => {
try {
const body = JSON.parse(Buffer.concat(chunks).toString('utf-8')) as { status: string; error?: string };
if (body.status === 'ok') {
resolve('ok');
} else if (body.status === 'not configured') {
resolve('not configured');
} else if (body.error) {
resolve(body.error.slice(0, 80));
} else {
resolve(body.status);
}
} catch {
resolve('invalid response');
}
});
});
req.on('error', () => resolve('mcplocal unreachable'));
req.on('timeout', () => { req.destroy(); resolve('timeout'); });
});
}
if (llm.provider === 'ollama') {
const url = llm.url ?? 'http://localhost:11434';
try {
const ok = await defaultCheckHealth(url);
return ok ? 'ok' : 'unreachable';
} catch {
return 'unreachable';
}
}
function defaultFetchModels(mcplocalUrl: string): Promise<string[]> {
return new Promise((resolve) => {
const req = http.get(`${mcplocalUrl}/llm/models`, { timeout: 5000 }, (res) => {
const chunks: Buffer[] = [];
res.on('data', (chunk: Buffer) => chunks.push(chunk));
res.on('end', () => {
try {
const body = JSON.parse(Buffer.concat(chunks).toString('utf-8')) as { models?: string[] };
resolve(body.models ?? []);
} catch {
resolve([]);
}
});
});
req.on('error', () => resolve([]));
req.on('timeout', () => { req.destroy(); resolve([]); });
});
}
// For API-key providers, we don't want to make a billable call on every status check
return 'ok (key stored)';
function defaultFetchProviders(mcplocalUrl: string): Promise<ProvidersInfo | null> {
return new Promise((resolve) => {
const req = http.get(`${mcplocalUrl}/llm/providers`, { timeout: 5000 }, (res) => {
const chunks: Buffer[] = [];
res.on('data', (chunk: Buffer) => chunks.push(chunk));
res.on('end', () => {
try {
const body = JSON.parse(Buffer.concat(chunks).toString('utf-8')) as ProvidersInfo;
resolve(body);
} catch {
resolve(null);
}
});
});
req.on('error', () => resolve(null));
req.on('timeout', () => { req.destroy(); resolve(null); });
});
}
const SPINNER_FRAMES = ['⠋', '⠙', '⠹', '⠸', '⠼', '⠴', '⠦', '⠧', '⠇', '⠏'];
@@ -82,11 +127,36 @@ const defaultDeps: StatusCommandDeps = {
write: (text) => process.stdout.write(text),
checkHealth: defaultCheckHealth,
checkLlm: defaultCheckLlm,
fetchModels: defaultFetchModels,
fetchProviders: defaultFetchProviders,
isTTY: process.stdout.isTTY ?? false,
};
/** Determine LLM label from config (handles both legacy and multi-provider formats). */
function getLlmLabel(llm: unknown): string | null {
if (!llm || typeof llm !== 'object') return null;
// Legacy format: { provider, model }
if ('provider' in llm) {
const legacy = llm as { provider: string; model?: string };
if (legacy.provider === 'none') return null;
return `${legacy.provider}${legacy.model ? ` / ${legacy.model}` : ''}`;
}
// Multi-provider format: { providers: [...] }
if ('providers' in llm) {
const multi = llm as { providers: Array<{ name: string; type: string; tier?: string }> };
if (multi.providers.length === 0) return null;
return multi.providers.map((p) => `${p.name}${p.tier ? ` (${p.tier})` : ''}`).join(', ');
}
return null;
}
/** Check if config uses multi-provider format. */
function isMultiProvider(llm: unknown): boolean {
return !!llm && typeof llm === 'object' && 'providers' in llm;
}
export function createStatusCommand(deps?: Partial<StatusCommandDeps>): Command {
const { configDeps, credentialsDeps, log, write, checkHealth, checkLlm, isTTY } = { ...defaultDeps, ...deps };
const { configDeps, credentialsDeps, log, write, checkHealth, checkLlm, fetchModels, fetchProviders, isTTY } = { ...defaultDeps, ...deps };
return new Command('status')
.description('Show mcpctl status and connectivity')
@@ -95,16 +165,16 @@ export function createStatusCommand(deps?: Partial<StatusCommandDeps>): Command
const config = loadConfig(configDeps);
const creds = loadCredentials(credentialsDeps);
const llmLabel = config.llm && config.llm.provider !== 'none'
? `${config.llm.provider}${config.llm.model ? ` / ${config.llm.model}` : ''}`
: null;
const llmLabel = getLlmLabel(config.llm);
const multiProvider = isMultiProvider(config.llm);
if (opts.output !== 'table') {
// JSON/YAML: run everything in parallel, wait, output at once
const [mcplocalReachable, mcpdReachable, llmStatus] = await Promise.all([
const [mcplocalReachable, mcpdReachable, llmStatus, providersInfo] = await Promise.all([
checkHealth(config.mcplocalUrl),
checkHealth(config.mcpdUrl),
llmLabel ? checkLlm(config.llm!) : Promise.resolve(null),
llmLabel ? checkLlm(config.mcplocalUrl) : Promise.resolve(null),
multiProvider ? fetchProviders(config.mcplocalUrl) : Promise.resolve(null),
]);
const llm = llmLabel
@@ -122,6 +192,7 @@ export function createStatusCommand(deps?: Partial<StatusCommandDeps>): Command
outputFormat: config.outputFormat,
llm,
llmStatus,
...(providersInfo ? { providers: providersInfo } : {}),
};
log(opts.output === 'json' ? formatJson(status) : formatYaml(status));
@@ -148,31 +219,69 @@ export function createStatusCommand(deps?: Partial<StatusCommandDeps>): Command
return;
}
// LLM check with spinner
const llmPromise = checkLlm(config.llm!);
// LLM check + models + providers fetch in parallel
const llmPromise = checkLlm(config.mcplocalUrl);
const modelsPromise = fetchModels(config.mcplocalUrl);
const providersPromise = multiProvider ? fetchProviders(config.mcplocalUrl) : Promise.resolve(null);
if (isTTY) {
let frame = 0;
const interval = setInterval(() => {
write(`${CLEAR_LINE}LLM: ${llmLabel} ${DIM}${SPINNER_FRAMES[frame % SPINNER_FRAMES.length]} checking...${RESET}`);
write(`${CLEAR_LINE}LLM: ${DIM}${SPINNER_FRAMES[frame % SPINNER_FRAMES.length]} checking...${RESET}`);
frame++;
}, 80);
const llmStatus = await llmPromise;
const [llmStatus, models, providersInfo] = await Promise.all([llmPromise, modelsPromise, providersPromise]);
clearInterval(interval);
if (llmStatus === 'ok' || llmStatus === 'ok (key stored)') {
write(`${CLEAR_LINE}LLM: ${llmLabel} ${GREEN}${llmStatus}${RESET}\n`);
if (providersInfo && (providersInfo.tiers.fast.length > 0 || providersInfo.tiers.heavy.length > 0)) {
// Tiered display with per-provider health
write(`${CLEAR_LINE}`);
for (const tier of ['fast', 'heavy'] as const) {
const names = providersInfo.tiers[tier];
if (names.length === 0) continue;
const label = tier === 'fast' ? 'LLM (fast): ' : 'LLM (heavy):';
const parts = names.map((n) => {
const ok = providersInfo.health[n];
return ok ? `${n} ${GREEN}${RESET}` : `${n} ${RED}${RESET}`;
});
log(`${label} ${parts.join(', ')}`);
}
} else {
write(`${CLEAR_LINE}LLM: ${llmLabel} ${RED}${llmStatus}${RESET}\n`);
// Legacy single provider display
if (llmStatus === 'ok' || llmStatus === 'ok (key stored)') {
write(`${CLEAR_LINE}LLM: ${llmLabel} ${GREEN}${llmStatus}${RESET}\n`);
} else {
write(`${CLEAR_LINE}LLM: ${llmLabel} ${RED}${llmStatus}${RESET}\n`);
}
}
if (models.length > 0) {
log(`${DIM} Available: ${models.join(', ')}${RESET}`);
}
} else {
// Non-TTY: no spinner, just wait and print
const llmStatus = await llmPromise;
if (llmStatus === 'ok' || llmStatus === 'ok (key stored)') {
log(`LLM: ${llmLabel}${llmStatus}`);
const [llmStatus, models, providersInfo] = await Promise.all([llmPromise, modelsPromise, providersPromise]);
if (providersInfo && (providersInfo.tiers.fast.length > 0 || providersInfo.tiers.heavy.length > 0)) {
for (const tier of ['fast', 'heavy'] as const) {
const names = providersInfo.tiers[tier];
if (names.length === 0) continue;
const label = tier === 'fast' ? 'LLM (fast): ' : 'LLM (heavy):';
const parts = names.map((n) => {
const ok = providersInfo.health[n];
return ok ? `${n}` : `${n}`;
});
log(`${label} ${parts.join(', ')}`);
}
} else {
log(`LLM: ${llmLabel}${llmStatus}`);
if (llmStatus === 'ok' || llmStatus === 'ok (key stored)') {
log(`LLM: ${llmLabel}${llmStatus}`);
} else {
log(`LLM: ${llmLabel}${llmStatus}`);
}
}
if (models.length > 0) {
log(`${DIM} Available: ${models.join(', ')}${RESET}`);
}
}
});

View File

@@ -1,4 +1,4 @@
export { McpctlConfigSchema, LlmConfigSchema, LLM_PROVIDERS, DEFAULT_CONFIG } from './schema.js';
export type { McpctlConfig, LlmConfig, LlmProviderName } from './schema.js';
export { McpctlConfigSchema, LlmConfigSchema, LlmProviderEntrySchema, LlmMultiConfigSchema, LLM_PROVIDERS, LLM_TIERS, DEFAULT_CONFIG } from './schema.js';
export type { McpctlConfig, LlmConfig, LlmProviderEntry, LlmMultiConfig, LlmProviderName, LlmTier } from './schema.js';
export { loadConfig, saveConfig, mergeConfig, getConfigPath } from './loader.js';
export type { ConfigLoaderDeps } from './loader.js';

View File

@@ -3,6 +3,10 @@ import { z } from 'zod';
export const LLM_PROVIDERS = ['gemini-cli', 'ollama', 'anthropic', 'openai', 'deepseek', 'vllm', 'none'] as const;
export type LlmProviderName = typeof LLM_PROVIDERS[number];
export const LLM_TIERS = ['fast', 'heavy'] as const;
export type LlmTier = typeof LLM_TIERS[number];
/** Legacy single-provider format. */
export const LlmConfigSchema = z.object({
/** LLM provider name */
provider: z.enum(LLM_PROVIDERS),
@@ -16,6 +20,31 @@ export const LlmConfigSchema = z.object({
export type LlmConfig = z.infer<typeof LlmConfigSchema>;
/** Multi-provider entry (advanced mode). */
export const LlmProviderEntrySchema = z.object({
/** User-chosen name for this provider instance (e.g. "vllm-local") */
name: z.string(),
/** Provider type */
type: z.enum(LLM_PROVIDERS),
/** Model name */
model: z.string().optional(),
/** Provider URL (for ollama, vllm, openai with custom endpoint) */
url: z.string().optional(),
/** Binary path override (for gemini-cli) */
binaryPath: z.string().optional(),
/** Tier assignment */
tier: z.enum(LLM_TIERS).optional(),
}).strict();
export type LlmProviderEntry = z.infer<typeof LlmProviderEntrySchema>;
/** Multi-provider format with providers array. */
export const LlmMultiConfigSchema = z.object({
providers: z.array(LlmProviderEntrySchema).min(1),
}).strict();
export type LlmMultiConfig = z.infer<typeof LlmMultiConfigSchema>;
export const McpctlConfigSchema = z.object({
/** mcplocal daemon endpoint (local LLM pre-processing proxy) */
mcplocalUrl: z.string().default('http://localhost:3200'),
@@ -35,8 +64,8 @@ export const McpctlConfigSchema = z.object({
outputFormat: z.enum(['table', 'json', 'yaml']).default('table'),
/** Smithery API key */
smitheryApiKey: z.string().optional(),
/** LLM provider configuration for smart features (pagination summaries, etc.) */
llm: LlmConfigSchema.optional(),
/** LLM provider configuration — accepts legacy single-provider or multi-provider format */
llm: z.union([LlmConfigSchema, LlmMultiConfigSchema]).optional(),
}).transform((cfg) => {
// Backward compatibility: if old daemonUrl is set but mcplocalUrl wasn't explicitly changed,
// use daemonUrl as mcplocalUrl

View File

@@ -67,7 +67,7 @@ async function runSetup(deps: ConfigSetupDeps): Promise<void> {
describe('config setup wizard', () => {
describe('provider: none', () => {
it('disables LLM and saves config', async () => {
const deps = buildDeps({ answers: ['none'] });
const deps = buildDeps({ answers: ['simple', 'none'] });
await runSetup(deps);
const config = readConfig();
@@ -81,7 +81,7 @@ describe('config setup wizard', () => {
it('auto-detects binary path and saves config', async () => {
// Answers: select provider, select model (no binary prompt — auto-detected)
const deps = buildDeps({
answers: ['gemini-cli', 'gemini-2.5-flash'],
answers: ['simple', 'gemini-cli', 'gemini-2.5-flash'],
whichBinary: vi.fn(async () => '/home/user/.npm-global/bin/gemini'),
});
await runSetup(deps);
@@ -98,7 +98,7 @@ describe('config setup wizard', () => {
it('prompts for manual path when binary not found', async () => {
// Answers: select provider, select model, enter manual path
const deps = buildDeps({
answers: ['gemini-cli', 'gemini-2.5-flash', '/opt/gemini'],
answers: ['simple', 'gemini-cli', 'gemini-2.5-flash', '/opt/gemini'],
whichBinary: vi.fn(async () => null),
});
await runSetup(deps);
@@ -113,7 +113,7 @@ describe('config setup wizard', () => {
it('saves gemini-cli with custom model', async () => {
// Answers: select provider, select custom, enter model name
const deps = buildDeps({
answers: ['gemini-cli', '__custom__', 'gemini-3.0-flash'],
answers: ['simple', 'gemini-cli', '__custom__', 'gemini-3.0-flash'],
whichBinary: vi.fn(async () => '/usr/bin/gemini'),
});
await runSetup(deps);
@@ -130,7 +130,7 @@ describe('config setup wizard', () => {
const fetchModels = vi.fn(async () => ['llama3.2', 'codellama', 'mistral']);
// Answers: select provider, enter URL, select model
const deps = buildDeps({
answers: ['ollama', 'http://localhost:11434', 'codellama'],
answers: ['simple', 'ollama', 'http://localhost:11434', 'codellama'],
fetchModels,
});
await runSetup(deps);
@@ -148,7 +148,7 @@ describe('config setup wizard', () => {
const fetchModels = vi.fn(async () => []);
// Answers: select provider, enter URL, enter model manually
const deps = buildDeps({
answers: ['ollama', 'http://localhost:11434', 'llama3.2'],
answers: ['simple', 'ollama', 'http://localhost:11434', 'llama3.2'],
fetchModels,
});
await runSetup(deps);
@@ -163,7 +163,7 @@ describe('config setup wizard', () => {
it('prompts for API key and saves to secret store', async () => {
// Answers: select provider, enter API key, select model
const deps = buildDeps({
answers: ['anthropic', 'sk-ant-new-key', 'claude-haiku-3-5-20241022'],
answers: ['simple', 'anthropic', 'sk-ant-new-key', 'claude-haiku-3-5-20241022'],
});
await runSetup(deps);
@@ -181,7 +181,7 @@ describe('config setup wizard', () => {
// Answers: select provider, confirm change=false, select model
const deps = buildDeps({
secrets: { 'anthropic-api-key': 'sk-ant-existing-key-1234' },
answers: ['anthropic', false, 'claude-sonnet-4-20250514'],
answers: ['simple', 'anthropic', false, 'claude-sonnet-4-20250514'],
});
await runSetup(deps);
@@ -196,7 +196,7 @@ describe('config setup wizard', () => {
// Answers: select provider, confirm change=true, enter new key, select model
const deps = buildDeps({
secrets: { 'anthropic-api-key': 'sk-ant-old' },
answers: ['anthropic', true, 'sk-ant-new', 'claude-haiku-3-5-20241022'],
answers: ['simple', 'anthropic', true, 'sk-ant-new', 'claude-haiku-3-5-20241022'],
});
await runSetup(deps);
@@ -210,7 +210,7 @@ describe('config setup wizard', () => {
const fetchModels = vi.fn(async () => ['my-model', 'llama-70b']);
// Answers: select provider, enter URL, select model
const deps = buildDeps({
answers: ['vllm', 'http://gpu:8000', 'llama-70b'],
answers: ['simple', 'vllm', 'http://gpu:8000', 'llama-70b'],
fetchModels,
});
await runSetup(deps);
@@ -229,7 +229,7 @@ describe('config setup wizard', () => {
it('prompts for key, model, and optional custom endpoint', async () => {
// Answers: select provider, enter key, enter model, confirm custom URL=true, enter URL
const deps = buildDeps({
answers: ['openai', 'sk-openai-key', 'gpt-4o', true, 'https://custom.api.com'],
answers: ['simple', 'openai', 'sk-openai-key', 'gpt-4o', true, 'https://custom.api.com'],
});
await runSetup(deps);
@@ -245,7 +245,7 @@ describe('config setup wizard', () => {
it('skips custom URL when not requested', async () => {
// Answers: select provider, enter key, enter model, confirm custom URL=false
const deps = buildDeps({
answers: ['openai', 'sk-openai-key', 'gpt-4o-mini', false],
answers: ['simple', 'openai', 'sk-openai-key', 'gpt-4o-mini', false],
});
await runSetup(deps);
@@ -260,7 +260,7 @@ describe('config setup wizard', () => {
it('prompts for key and model', async () => {
// Answers: select provider, enter key, select model
const deps = buildDeps({
answers: ['deepseek', 'sk-ds-key', 'deepseek-chat'],
answers: ['simple', 'deepseek', 'sk-ds-key', 'deepseek-chat'],
});
await runSetup(deps);
@@ -275,7 +275,7 @@ describe('config setup wizard', () => {
describe('output messages', () => {
it('shows restart instruction', async () => {
const deps = buildDeps({ answers: ['gemini-cli', 'gemini-2.5-flash'] });
const deps = buildDeps({ answers: ['simple', 'gemini-cli', 'gemini-2.5-flash'] });
await runSetup(deps);
expect(logs.some((l) => l.includes('systemctl --user restart mcplocal'))).toBe(true);
@@ -283,7 +283,7 @@ describe('config setup wizard', () => {
});
it('shows configured provider and model', async () => {
const deps = buildDeps({ answers: ['gemini-cli', 'gemini-2.5-flash'] });
const deps = buildDeps({ answers: ['simple', 'gemini-cli', 'gemini-2.5-flash'] });
await runSetup(deps);
expect(logs.some((l) => l.includes('gemini-cli') && l.includes('gemini-2.5-flash'))).toBe(true);

View File

@@ -30,8 +30,6 @@ describe('project with new fields', () => {
'project', 'smart-home',
'-d', 'Smart home project',
'--proxy-mode', 'filtered',
'--proxy-mode-llm-provider', 'gemini-cli',
'--proxy-mode-llm-model', 'gemini-2.0-flash',
'--server', 'my-grafana',
'--server', 'my-ha',
], { from: 'user' });
@@ -40,8 +38,6 @@ describe('project with new fields', () => {
name: 'smart-home',
description: 'Smart home project',
proxyMode: 'filtered',
llmProvider: 'gemini-cli',
llmModel: 'gemini-2.0-flash',
servers: ['my-grafana', 'my-ha'],
}));
});

View File

@@ -26,6 +26,7 @@ function baseDeps(overrides?: Partial<StatusCommandDeps>): Partial<StatusCommand
log,
write,
checkHealth: async () => true,
fetchProviders: async () => null,
isTTY: false,
...overrides,
};
@@ -134,13 +135,23 @@ describe('status command', () => {
expect(out).toContain('✗ not authenticated');
});
it('shows binary not found error', async () => {
it('shows error message from mcplocal', async () => {
saveConfig({ ...DEFAULT_CONFIG, llm: { provider: 'gemini-cli', model: 'gemini-2.5-flash' } }, { configDir: tempDir });
const cmd = createStatusCommand(baseDeps({ checkLlm: async () => 'binary not found' }));
await cmd.parseAsync([], { from: 'user' });
expect(output.join('\n')).toContain('✗ binary not found');
});
it('queries mcplocal URL for LLM health', async () => {
saveConfig({ ...DEFAULT_CONFIG, mcplocalUrl: 'http://custom:9999', llm: { provider: 'gemini-cli', model: 'gemini-2.5-flash' } }, { configDir: tempDir });
let queriedUrl = '';
const cmd = createStatusCommand(baseDeps({
checkLlm: async (url) => { queriedUrl = url; return 'ok'; },
}));
await cmd.parseAsync([], { from: 'user' });
expect(queriedUrl).toBe('http://custom:9999');
});
it('uses spinner on TTY and writes final result', async () => {
saveConfig({ ...DEFAULT_CONFIG, llm: { provider: 'gemini-cli', model: 'gemini-2.5-flash' } }, { configDir: tempDir });
const cmd = createStatusCommand(baseDeps({

View File

@@ -15,7 +15,7 @@ describe('fish completions', () => {
});
it('does not offer resource types without __mcpctl_needs_resource_type guard', () => {
const resourceTypes = ['servers', 'instances', 'secrets', 'templates', 'projects', 'users', 'groups', 'rbac'];
const resourceTypes = ['servers', 'instances', 'secrets', 'templates', 'projects', 'users', 'groups', 'rbac', 'prompts', 'promptrequests'];
const lines = fishFile.split('\n').filter((l) => l.startsWith('complete '));
for (const line of lines) {

View File

@@ -48,6 +48,33 @@ export async function refreshProjectUpstreams(
return syncUpstreams(router, mcpdClient, servers);
}
/**
* Fetch a project's LLM config (llmProvider, llmModel) from mcpd.
* These are the project-level "recommendations" — local overrides take priority.
*/
export interface ProjectLlmConfig {
llmProvider?: string;
llmModel?: string;
}
export async function fetchProjectLlmConfig(
mcpdClient: McpdClient,
projectName: string,
): Promise<ProjectLlmConfig> {
try {
const project = await mcpdClient.get<{
llmProvider?: string;
llmModel?: string;
}>(`/api/v1/projects/${encodeURIComponent(projectName)}`);
const config: ProjectLlmConfig = {};
if (project.llmProvider) config.llmProvider = project.llmProvider;
if (project.llmModel) config.llmModel = project.llmModel;
return config;
} catch {
return {};
}
}
/** Shared sync logic: reconcile a router's upstreams with a server list. */
function syncUpstreams(router: McpRouter, mcpdClient: McpdClient, servers: McpdServer[]): string[] {
const registered: string[] = [];

View File

@@ -44,21 +44,110 @@ export interface LlmFileConfig {
binaryPath?: string;
}
/** Multi-provider entry from config file. */
export interface LlmProviderFileEntry {
name: string;
type: string;
model?: string;
url?: string;
binaryPath?: string;
tier?: 'fast' | 'heavy';
}
export interface ProjectLlmOverride {
model?: string;
provider?: string;
}
interface LlmMultiFileConfig {
providers: LlmProviderFileEntry[];
}
interface McpctlConfig {
llm?: LlmFileConfig | LlmMultiFileConfig;
projects?: Record<string, { llm?: ProjectLlmOverride }>;
}
/** Cached config for the process lifetime (reloaded on SIGHUP if needed). */
let cachedConfig: McpctlConfig | null = null;
function loadFullConfig(): McpctlConfig {
if (cachedConfig) return cachedConfig;
try {
const configPath = join(homedir(), '.mcpctl', 'config.json');
if (!existsSync(configPath)) return {};
const raw = readFileSync(configPath, 'utf-8');
cachedConfig = JSON.parse(raw) as McpctlConfig;
return cachedConfig;
} catch {
return {};
}
}
/** Type guard: is config the multi-provider format? */
function isMultiConfig(llm: LlmFileConfig | LlmMultiFileConfig): llm is LlmMultiFileConfig {
return 'providers' in llm && Array.isArray((llm as LlmMultiFileConfig).providers);
}
/**
* Load LLM configuration from ~/.mcpctl/config.json.
* Returns undefined if no LLM section is configured.
* @deprecated Use loadLlmProviders() for multi-provider support.
*/
export function loadLlmConfig(): LlmFileConfig | undefined {
try {
const configPath = join(homedir(), '.mcpctl', 'config.json');
if (!existsSync(configPath)) return undefined;
const raw = readFileSync(configPath, 'utf-8');
const parsed = JSON.parse(raw) as { llm?: LlmFileConfig };
if (!parsed.llm?.provider || parsed.llm.provider === 'none') return undefined;
return parsed.llm;
} catch {
return undefined;
const config = loadFullConfig();
if (!config.llm) return undefined;
if (isMultiConfig(config.llm)) {
// Multi-provider format — return first provider as legacy compat
const first = config.llm.providers[0];
if (!first) return undefined;
const legacy: LlmFileConfig = { provider: first.type };
if (first.model) legacy.model = first.model;
if (first.url) legacy.url = first.url;
if (first.binaryPath) legacy.binaryPath = first.binaryPath;
return legacy;
}
if (!config.llm.provider || config.llm.provider === 'none') return undefined;
return config.llm;
}
/**
* Load LLM providers from ~/.mcpctl/config.json.
* Normalizes both legacy single-provider and multi-provider formats.
* Returns empty array if no LLM is configured.
*/
export function loadLlmProviders(): LlmProviderFileEntry[] {
const config = loadFullConfig();
if (!config.llm) return [];
if (isMultiConfig(config.llm)) {
return config.llm.providers.filter((p) => p.type !== 'none');
}
// Legacy single-provider format → normalize to one entry
if (!config.llm.provider || config.llm.provider === 'none') return [];
const entry: LlmProviderFileEntry = {
name: config.llm.provider,
type: config.llm.provider,
};
if (config.llm.model) entry.model = config.llm.model;
if (config.llm.url) entry.url = config.llm.url;
if (config.llm.binaryPath) entry.binaryPath = config.llm.binaryPath;
return [entry];
}
/**
* Load per-project LLM override from ~/.mcpctl/config.json.
* Returns the project-specific model/provider override, or undefined.
*/
export function loadProjectLlmOverride(projectName: string): ProjectLlmOverride | undefined {
const config = loadFullConfig();
return config.projects?.[projectName]?.llm;
}
/** Reset cached config (for testing). */
export function resetConfigCache(): void {
cachedConfig = null;
}
export function loadHttpConfig(env: Record<string, string | undefined> = process.env): HttpConfig {

View File

@@ -13,7 +13,8 @@ import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/
import type { JSONRPCMessage } from '@modelcontextprotocol/sdk/types.js';
import { McpRouter } from '../router.js';
import { ResponsePaginator } from '../llm/pagination.js';
import { refreshProjectUpstreams } from '../discovery.js';
import { refreshProjectUpstreams, fetchProjectLlmConfig } from '../discovery.js';
import { loadProjectLlmOverride } from './config.js';
import type { McpdClient } from './mcpd-client.js';
import type { ProviderRegistry } from '../providers/registry.js';
import type { JsonRpcRequest } from '../types.js';
@@ -46,8 +47,13 @@ export function registerProjectMcpEndpoint(app: FastifyInstance, mcpdClient: Mcp
const router = existing?.router ?? new McpRouter();
await refreshProjectUpstreams(router, mcpdClient, projectName, authToken);
// Wire pagination support with LLM provider if configured
router.setPaginator(new ResponsePaginator(providerRegistry ?? null));
// Resolve project LLM model: local override → mcpd recommendation → global default
const localOverride = loadProjectLlmOverride(projectName);
const mcpdConfig = await fetchProjectLlmConfig(mcpdClient, projectName);
const resolvedModel = localOverride?.model ?? mcpdConfig.llmModel ?? undefined;
// Wire pagination support with LLM provider and project model override
router.setPaginator(new ResponsePaginator(providerRegistry ?? null, {}, resolvedModel));
// Configure prompt resources with SA-scoped client for RBAC
const saClient = mcpdClient.withHeaders({ 'X-Service-Account': `project:${projectName}` });

View File

@@ -81,6 +81,102 @@ export async function createHttpServer(
reply.code(200).send({ status: 'ok' });
});
// LLM health check — cached to avoid burning tokens on every call.
// Does a real inference call at most once per 10 minutes.
let llmHealthCache: { result: Record<string, unknown>; expiresAt: number } | null = null;
const LLM_HEALTH_CACHE_MS = 10 * 60 * 1000; // 10 minutes
app.get('/llm/health', async (_request, reply) => {
const provider = deps.providerRegistry?.getProvider('fast') ?? null;
if (!provider) {
reply.code(200).send({ status: 'not configured' });
return;
}
// Return cached result if fresh
if (llmHealthCache && Date.now() < llmHealthCache.expiresAt) {
reply.code(200).send(llmHealthCache.result);
return;
}
try {
const result = await provider.complete({
messages: [{ role: 'user', content: 'Respond with exactly: ok' }],
maxTokens: 10,
});
const ok = result.content.trim().toLowerCase().includes('ok');
const response = {
status: ok ? 'ok' : 'unexpected response',
provider: provider.name,
response: result.content.trim().slice(0, 100),
};
llmHealthCache = { result: response, expiresAt: Date.now() + LLM_HEALTH_CACHE_MS };
reply.code(200).send(response);
} catch (err) {
const msg = (err as Error).message ?? String(err);
const response = {
status: 'error',
provider: provider.name,
error: msg.slice(0, 200),
};
// Cache errors for 1 minute only (retry sooner)
llmHealthCache = { result: response, expiresAt: Date.now() + 60_000 };
reply.code(200).send(response);
}
});
// LLM models — list available models from the active provider
app.get('/llm/models', async (_request, reply) => {
const provider = deps.providerRegistry?.getProvider('fast') ?? null;
if (!provider) {
reply.code(200).send({ models: [], provider: null });
return;
}
try {
const models = await provider.listModels();
reply.code(200).send({ models, provider: provider.name });
} catch {
reply.code(200).send({ models: [], provider: provider.name });
}
});
// LLM providers — list all registered providers with tier assignments and health
app.get('/llm/providers', async (_request, reply) => {
const registry = deps.providerRegistry;
if (!registry) {
reply.code(200).send({ providers: [], tiers: { fast: [], heavy: [] }, health: {} });
return;
}
// Run isAvailable() on all providers in parallel (lightweight, no tokens burned)
const names = registry.list();
const healthChecks = await Promise.all(
names.map(async (name) => {
const provider = registry.get(name);
if (!provider) return { name, available: false };
try {
const available = await provider.isAvailable();
return { name, available };
} catch {
return { name, available: false };
}
}),
);
const health: Record<string, boolean> = {};
for (const check of healthChecks) {
health[check.name] = check.available;
}
reply.code(200).send({
providers: names,
tiers: {
fast: registry.getTierProviders('fast'),
heavy: registry.getTierProviders('heavy'),
},
health,
});
});
// Proxy management routes to mcpd
const mcpdClient = new McpdClient(config.mcpdUrl, config.mcpdToken);
registerProxyRoutes(app, mcpdClient);

View File

@@ -1,11 +1,12 @@
import type { SecretStore } from '@mcpctl/shared';
import type { LlmFileConfig } from './http/config.js';
import type { LlmFileConfig, LlmProviderFileEntry } from './http/config.js';
import { ProviderRegistry } from './providers/registry.js';
import { GeminiAcpProvider } from './providers/gemini-acp.js';
import { OllamaProvider } from './providers/ollama.js';
import { AnthropicProvider } from './providers/anthropic.js';
import { OpenAiProvider } from './providers/openai.js';
import { DeepSeekProvider } from './providers/deepseek.js';
import type { LlmProvider } from './providers/types.js';
import type { GeminiAcpConfig } from './providers/gemini-acp.js';
import type { OllamaConfig } from './providers/ollama.js';
import type { AnthropicConfig } from './providers/anthropic.js';
@@ -13,85 +14,158 @@ import type { OpenAiConfig } from './providers/openai.js';
import type { DeepSeekConfig } from './providers/deepseek.js';
/**
* Create a ProviderRegistry from user config + secret store.
* Returns an empty registry if config is undefined or provider is 'none'.
* Thin wrapper that delegates all LlmProvider methods but overrides `name`.
* Used when the user's chosen name (e.g. "vllm-local") differs from the
* underlying provider's name (e.g. "openai").
*/
export async function createProviderFromConfig(
config: LlmFileConfig | undefined,
secretStore: SecretStore,
): Promise<ProviderRegistry> {
const registry = new ProviderRegistry();
if (!config?.provider || config.provider === 'none') return registry;
class NamedProvider implements LlmProvider {
readonly name: string;
private inner: LlmProvider;
switch (config.provider) {
constructor(name: string, inner: LlmProvider) {
this.name = name;
this.inner = inner;
}
complete(...args: Parameters<LlmProvider['complete']>) {
return this.inner.complete(...args);
}
listModels() {
return this.inner.listModels();
}
isAvailable() {
return this.inner.isAvailable();
}
dispose() {
this.inner.dispose?.();
}
}
/**
* Create a single LlmProvider from a provider entry config.
* Returns null if required config is missing (logs warning).
*/
async function createSingleProvider(
entry: LlmProviderFileEntry,
secretStore: SecretStore,
): Promise<LlmProvider | null> {
switch (entry.type) {
case 'gemini-cli': {
const cfg: GeminiAcpConfig = {};
if (config.binaryPath) cfg.binaryPath = config.binaryPath;
if (config.model) cfg.defaultModel = config.model;
registry.register(new GeminiAcpProvider(cfg));
break;
if (entry.binaryPath) cfg.binaryPath = entry.binaryPath;
if (entry.model) cfg.defaultModel = entry.model;
const provider = new GeminiAcpProvider(cfg);
provider.warmup();
return provider;
}
case 'ollama': {
const cfg: OllamaConfig = {};
if (config.url) cfg.baseUrl = config.url;
if (config.model) cfg.defaultModel = config.model;
registry.register(new OllamaProvider(cfg));
break;
if (entry.url) cfg.baseUrl = entry.url;
if (entry.model) cfg.defaultModel = entry.model;
return new OllamaProvider(cfg);
}
case 'anthropic': {
const apiKey = await secretStore.get('anthropic-api-key');
if (!apiKey) {
process.stderr.write('Warning: Anthropic API key not found in secret store. Run "mcpctl config setup" to configure.\n');
return registry;
process.stderr.write(`Warning: Anthropic API key not found for provider "${entry.name}". Run "mcpctl config setup" to configure.\n`);
return null;
}
const cfg: AnthropicConfig = { apiKey };
if (config.model) cfg.defaultModel = config.model;
registry.register(new AnthropicProvider(cfg));
break;
if (entry.model) cfg.defaultModel = entry.model;
return new AnthropicProvider(cfg);
}
case 'openai': {
const apiKey = await secretStore.get('openai-api-key');
if (!apiKey) {
process.stderr.write('Warning: OpenAI API key not found in secret store. Run "mcpctl config setup" to configure.\n');
return registry;
process.stderr.write(`Warning: OpenAI API key not found for provider "${entry.name}". Run "mcpctl config setup" to configure.\n`);
return null;
}
const cfg: OpenAiConfig = { apiKey };
if (config.url) cfg.baseUrl = config.url;
if (config.model) cfg.defaultModel = config.model;
registry.register(new OpenAiProvider(cfg));
break;
if (entry.url) cfg.baseUrl = entry.url;
if (entry.model) cfg.defaultModel = entry.model;
return new OpenAiProvider(cfg);
}
case 'deepseek': {
const apiKey = await secretStore.get('deepseek-api-key');
if (!apiKey) {
process.stderr.write('Warning: DeepSeek API key not found in secret store. Run "mcpctl config setup" to configure.\n');
return registry;
process.stderr.write(`Warning: DeepSeek API key not found for provider "${entry.name}". Run "mcpctl config setup" to configure.\n`);
return null;
}
const cfg: DeepSeekConfig = { apiKey };
if (config.url) cfg.baseUrl = config.url;
if (config.model) cfg.defaultModel = config.model;
registry.register(new DeepSeekProvider(cfg));
break;
if (entry.url) cfg.baseUrl = entry.url;
if (entry.model) cfg.defaultModel = entry.model;
return new DeepSeekProvider(cfg);
}
case 'vllm': {
// vLLM uses OpenAI-compatible API
if (!config.url) {
process.stderr.write('Warning: vLLM URL not configured. Run "mcpctl config setup" to configure.\n');
return registry;
if (!entry.url) {
process.stderr.write(`Warning: vLLM URL not configured for provider "${entry.name}". Run "mcpctl config setup" to configure.\n`);
return null;
}
registry.register(new OpenAiProvider({
return new OpenAiProvider({
apiKey: 'unused',
baseUrl: config.url,
defaultModel: config.model ?? 'default',
}));
break;
baseUrl: entry.url,
defaultModel: entry.model ?? 'default',
});
}
default:
return null;
}
}
/**
* Create a ProviderRegistry from multi-provider config entries + secret store.
* Registers each provider, wraps with NamedProvider if needed, assigns tiers.
*/
export async function createProvidersFromConfig(
entries: LlmProviderFileEntry[],
secretStore: SecretStore,
): Promise<ProviderRegistry> {
const registry = new ProviderRegistry();
for (const entry of entries) {
const rawProvider = await createSingleProvider(entry, secretStore);
if (!rawProvider) continue;
// Wrap with NamedProvider if user name differs from provider's built-in name
const provider = rawProvider.name !== entry.name
? new NamedProvider(entry.name, rawProvider)
: rawProvider;
registry.register(provider);
if (entry.tier) {
registry.assignTier(provider.name, entry.tier);
}
}
return registry;
}
/**
* Create a ProviderRegistry from legacy single-provider config + secret store.
* @deprecated Use createProvidersFromConfig() with loadLlmProviders() instead.
*/
export async function createProviderFromConfig(
config: LlmFileConfig | undefined,
secretStore: SecretStore,
): Promise<ProviderRegistry> {
if (!config?.provider || config.provider === 'none') {
return new ProviderRegistry();
}
const entry: LlmProviderFileEntry = {
name: config.provider,
type: config.provider,
};
if (config.model) entry.model = config.model;
if (config.url) entry.url = config.url;
if (config.binaryPath) entry.binaryPath = config.binaryPath;
return createProvidersFromConfig([entry], secretStore);
}

View File

@@ -105,6 +105,7 @@ export class ResponsePaginator {
constructor(
private providers: ProviderRegistry | null,
config: Partial<PaginationConfig> = {},
private modelOverride?: string,
) {
this.config = { ...DEFAULT_PAGINATION_CONFIG, ...config };
}
@@ -129,7 +130,8 @@ export class ResponsePaginator {
try {
index = await this.generateSmartIndex(resultId, toolName, raw, pages);
} catch {
} catch (err) {
console.error(`[pagination] Smart index failed for ${toolName}, falling back to simple:`, err instanceof Error ? err.message : String(err));
index = this.generateSimpleIndex(resultId, toolName, raw, pages);
}
@@ -240,7 +242,7 @@ export class ResponsePaginator {
raw: string,
pages: PageInfo[],
): Promise<PaginationIndex> {
const provider = this.providers?.getActive();
const provider = this.providers?.getProvider('fast');
if (!provider) {
return this.generateSimpleIndex(resultId, toolName, raw, pages);
}
@@ -259,9 +261,12 @@ export class ResponsePaginator {
],
maxTokens: this.config.indexMaxTokens,
temperature: 0,
...(this.modelOverride ? { model: this.modelOverride } : {}),
});
const summaries = JSON.parse(result.content) as Array<{ page: number; summary: string }>;
// LLMs often wrap JSON in ```json ... ``` fences — strip them
const cleaned = result.content.replace(/^```(?:json)?\s*\n?/i, '').replace(/\n?```\s*$/i, '').trim();
const summaries = JSON.parse(cleaned) as Array<{ page: number; summary: string }>;
return {
resultId,

View File

@@ -106,7 +106,7 @@ export class LlmProcessor {
return { optimized: false, params };
}
const provider = this.providers.getActive();
const provider = this.providers.getProvider('fast');
if (!provider) {
return { optimized: false, params };
}
@@ -142,7 +142,7 @@ export class LlmProcessor {
return { filtered: false, result: response.result, originalSize: raw.length, filteredSize: raw.length };
}
const provider = this.providers.getActive();
const provider = this.providers.getProvider('fast');
if (!provider) {
const raw = JSON.stringify(response.result);
return { filtered: false, result: response.result, originalSize: raw.length, filteredSize: raw.length };

View File

@@ -7,9 +7,9 @@ import { StdioProxyServer } from './server.js';
import { StdioUpstream } from './upstream/stdio.js';
import { HttpUpstream } from './upstream/http.js';
import { createHttpServer } from './http/server.js';
import { loadHttpConfig, loadLlmConfig } from './http/config.js';
import { loadHttpConfig, loadLlmProviders } from './http/config.js';
import type { HttpConfig } from './http/config.js';
import { createProviderFromConfig } from './llm-config.js';
import { createProvidersFromConfig } from './llm-config.js';
import { createSecretStore } from '@mcpctl/shared';
import type { ProviderRegistry } from './providers/registry.js';
@@ -65,13 +65,19 @@ export async function main(argv: string[] = process.argv): Promise<MainResult> {
const args = parseArgs(argv);
const httpConfig = loadHttpConfig();
// Load LLM provider from user config + secret store
const llmConfig = loadLlmConfig();
// Load LLM providers from user config + secret store
const llmEntries = loadLlmProviders();
const secretStore = await createSecretStore();
const providerRegistry = await createProviderFromConfig(llmConfig, secretStore);
const activeLlm = providerRegistry.getActive();
if (activeLlm) {
process.stderr.write(`LLM provider: ${activeLlm.name}\n`);
const providerRegistry = await createProvidersFromConfig(llmEntries, secretStore);
if (providerRegistry.hasTierConfig()) {
const fast = providerRegistry.getTierProviders('fast');
const heavy = providerRegistry.getTierProviders('heavy');
process.stderr.write(`LLM providers: fast=[${fast.join(',')}] heavy=[${heavy.join(',')}]\n`);
} else {
const activeLlm = providerRegistry.getActive();
if (activeLlm) {
process.stderr.write(`LLM provider: ${activeLlm.name}\n`);
}
}
let upstreamConfigs: UpstreamConfig[] = [];

View File

@@ -205,12 +205,16 @@ export class AcpClient {
// Collect text from agent_message_chunk
if (update.sessionUpdate === 'agent_message_chunk') {
const content = update.content as Array<{ type: string; text?: string }> | undefined;
if (content) {
for (const block of content) {
if (block.type === 'text' && block.text) {
this.activePromptChunks.push(block.text);
}
const content = update.content;
// Gemini ACP sends content as a single object {type, text} or an array [{type, text}]
const blocks: Array<{ type: string; text?: string }> = Array.isArray(content)
? content as Array<{ type: string; text?: string }>
: content && typeof content === 'object'
? [content as { type: string; text?: string }]
: [];
for (const block of blocks) {
if (block.type === 'text' && block.text) {
this.activePromptChunks.push(block.text);
}
}
}

View File

@@ -11,39 +11,56 @@ export interface GeminiAcpConfig {
defaultModel?: string;
requestTimeoutMs?: number;
initTimeoutMs?: number;
/** Idle TTL for pooled sessions in ms (default: 8 hours) */
idleTtlMs?: number;
/** Override for testing — passed through to AcpClient */
spawn?: AcpClientConfig['spawn'];
}
interface PoolEntry {
client: AcpClient;
lastUsed: number;
queue: Promise<void>;
}
/**
* Gemini CLI provider using ACP (Agent Client Protocol) mode.
* Keeps the gemini process alive as a persistent subprocess, eliminating
* the ~10s cold-start per call. Auto-restarts on crash or timeout.
*
* Maintains a pool of persistent subprocesses keyed by model name.
* Each model gets its own `gemini --experimental-acp` subprocess with
* a serial request queue. Idle sessions are evicted after 8 hours.
*
* NOTE: Gemini ACP currently doesn't support per-session model selection,
* so all sessions use the same model. The pool infrastructure is ready for
* when vLLM/OpenAI providers are added (they support per-request model).
*/
export class GeminiAcpProvider implements LlmProvider {
readonly name = 'gemini-cli';
private client: AcpClient;
private pool = new Map<string, PoolEntry>();
private binaryPath: string;
private defaultModel: string;
private queue: Promise<void> = Promise.resolve();
private readonly requestTimeoutMs: number;
private readonly initTimeoutMs: number;
private readonly idleTtlMs: number;
private readonly spawnOverride?: AcpClientConfig['spawn'];
constructor(config?: GeminiAcpConfig) {
this.binaryPath = config?.binaryPath ?? 'gemini';
this.defaultModel = config?.defaultModel ?? 'gemini-2.5-flash';
const acpConfig: AcpClientConfig = {
binaryPath: this.binaryPath,
model: this.defaultModel,
requestTimeoutMs: config?.requestTimeoutMs ?? 60_000,
initTimeoutMs: config?.initTimeoutMs ?? 30_000,
};
if (config?.spawn) acpConfig.spawn = config.spawn;
this.client = new AcpClient(acpConfig);
this.requestTimeoutMs = config?.requestTimeoutMs ?? 60_000;
this.initTimeoutMs = config?.initTimeoutMs ?? 30_000;
this.idleTtlMs = config?.idleTtlMs ?? 8 * 60 * 60 * 1000; // 8 hours
if (config?.spawn) this.spawnOverride = config.spawn;
}
async complete(options: CompletionOptions): Promise<CompletionResult> {
return this.enqueue(() => this.doComplete(options));
const model = options.model ?? this.defaultModel;
const entry = this.getOrCreateEntry(model);
entry.lastUsed = Date.now();
this.evictIdle();
return this.enqueue(entry, () => this.doComplete(entry.client, options));
}
async listModels(): Promise<string[]> {
@@ -60,12 +77,63 @@ export class GeminiAcpProvider implements LlmProvider {
}
dispose(): void {
this.client.dispose();
for (const entry of this.pool.values()) {
entry.client.dispose();
}
this.pool.clear();
}
/**
* Eagerly spawn the default model's ACP subprocess so it's ready
* for the first request (avoids 30s cold-start on health checks).
*/
warmup(): void {
const entry = this.getOrCreateEntry(this.defaultModel);
// Fire-and-forget: start the subprocess initialization in the background
entry.client.ensureReady().catch(() => {
// Ignore errors — next request will retry
});
}
/** Number of active pool entries (for testing). */
get poolSize(): number {
return this.pool.size;
}
// --- Private ---
private async doComplete(options: CompletionOptions): Promise<CompletionResult> {
private getOrCreateEntry(model: string): PoolEntry {
const existing = this.pool.get(model);
if (existing) return existing;
const acpConfig: AcpClientConfig = {
binaryPath: this.binaryPath,
model,
requestTimeoutMs: this.requestTimeoutMs,
initTimeoutMs: this.initTimeoutMs,
};
if (this.spawnOverride) acpConfig.spawn = this.spawnOverride;
const entry: PoolEntry = {
client: new AcpClient(acpConfig),
lastUsed: Date.now(),
queue: Promise.resolve(),
};
this.pool.set(model, entry);
return entry;
}
private evictIdle(): void {
const now = Date.now();
for (const [model, entry] of this.pool) {
if (now - entry.lastUsed > this.idleTtlMs) {
entry.client.dispose();
this.pool.delete(model);
}
}
}
private async doComplete(client: AcpClient, options: CompletionOptions): Promise<CompletionResult> {
const prompt = options.messages
.map((m) => {
if (m.role === 'system') return `System: ${m.content}`;
@@ -75,7 +143,7 @@ export class GeminiAcpProvider implements LlmProvider {
})
.join('\n\n');
const content = await this.client.prompt(prompt);
const content = await client.prompt(prompt);
return {
content: content.trim(),
@@ -85,9 +153,9 @@ export class GeminiAcpProvider implements LlmProvider {
};
}
private enqueue<T>(fn: () => Promise<T>): Promise<T> {
private enqueue<T>(entry: PoolEntry, fn: () => Promise<T>): Promise<T> {
const result = new Promise<T>((resolve, reject) => {
this.queue = this.queue.then(
entry.queue = entry.queue.then(
() => fn().then(resolve, reject),
() => fn().then(resolve, reject),
);

View File

@@ -1,11 +1,13 @@
import type { LlmProvider } from './types.js';
import type { LlmProvider, Tier } from './types.js';
/**
* Registry for LLM providers. Supports switching the active provider at runtime.
* Registry for LLM providers. Supports tier-based routing (fast/heavy)
* with cross-tier fallback, and legacy single-provider mode.
*/
export class ProviderRegistry {
private providers = new Map<string, LlmProvider>();
private activeProvider: string | null = null;
private tierProviders = new Map<Tier, string[]>();
register(provider: LlmProvider): void {
this.providers.set(provider.name, provider);
@@ -20,6 +22,15 @@ export class ProviderRegistry {
const first = this.providers.keys().next();
this.activeProvider = first.done ? null : first.value;
}
// Remove from tier assignments
for (const [tier, names] of this.tierProviders) {
const filtered = names.filter((n) => n !== name);
if (filtered.length === 0) {
this.tierProviders.delete(tier);
} else {
this.tierProviders.set(tier, filtered);
}
}
}
setActive(name: string): void {
@@ -34,6 +45,42 @@ export class ProviderRegistry {
return this.providers.get(this.activeProvider) ?? null;
}
/** Assign a provider to a tier. Call order = priority within the tier. */
assignTier(providerName: string, tier: Tier): void {
if (!this.providers.has(providerName)) {
throw new Error(`Provider '${providerName}' is not registered`);
}
const existing = this.tierProviders.get(tier) ?? [];
if (!existing.includes(providerName)) {
this.tierProviders.set(tier, [...existing, providerName]);
}
}
/**
* Get provider for a specific tier with fallback.
* Resolution: requested tier → other tier → getActive() (legacy).
*/
getProvider(tier: Tier): LlmProvider | null {
const primary = this.firstInTier(tier);
if (primary) return primary;
const otherTier: Tier = tier === 'fast' ? 'heavy' : 'fast';
const fallback = this.firstInTier(otherTier);
if (fallback) return fallback;
return this.getActive();
}
/** Get provider names assigned to a tier. */
getTierProviders(tier: Tier): string[] {
return this.tierProviders.get(tier) ?? [];
}
/** Whether any tier assignments exist (vs legacy single-provider mode). */
hasTierConfig(): boolean {
return this.tierProviders.size > 0;
}
get(name: string): LlmProvider | undefined {
return this.providers.get(name);
}
@@ -46,10 +93,31 @@ export class ProviderRegistry {
return this.activeProvider;
}
/** Provider info for status display. */
listProviders(): Array<{ name: string; tiers: Tier[] }> {
return this.list().map((name) => {
const tiers: Tier[] = [];
for (const [tier, names] of this.tierProviders) {
if (names.includes(name)) tiers.push(tier);
}
return { name, tiers };
});
}
/** Dispose all registered providers that have a dispose method. */
disposeAll(): void {
for (const provider of this.providers.values()) {
provider.dispose?.();
}
}
private firstInTier(tier: Tier): LlmProvider | null {
const names = this.tierProviders.get(tier);
if (!names) return null;
for (const name of names) {
const provider = this.providers.get(name);
if (provider) return provider;
}
return null;
}
}

View File

@@ -44,6 +44,9 @@ export interface CompletionOptions {
model?: string;
}
/** LLM provider tier. 'fast' = local inference, 'heavy' = cloud reasoning. */
export type Tier = 'fast' | 'heavy';
export interface LlmProvider {
/** Provider identifier (e.g., 'openai', 'anthropic', 'ollama') */
readonly name: string;

View File

@@ -230,6 +230,77 @@ describe('AcpClient', () => {
expect(result).toBe('Part A Part B');
});
it('handles single-object content (real Gemini ACP format)', async () => {
createClient();
autoHandshake('sess-1');
await client.ensureReady();
mock.stdin.write.mockImplementation((data: string) => {
const msg = JSON.parse(data.trim()) as { id: number; method: string };
if (msg.method === 'session/prompt') {
setImmediate(() => {
// Real Gemini ACP sends content as a single object, not an array
mock.sendLine({
jsonrpc: '2.0',
method: 'session/update',
params: {
sessionId: 'sess-1',
update: {
sessionUpdate: 'agent_message_chunk',
content: { type: 'text', text: 'ok' },
},
},
});
mock.sendResponse(msg.id, { stopReason: 'end_turn' });
});
}
});
const result = await client.prompt('test');
expect(result).toBe('ok');
});
it('ignores agent_thought_chunk notifications', async () => {
createClient();
autoHandshake('sess-1');
await client.ensureReady();
mock.stdin.write.mockImplementation((data: string) => {
const msg = JSON.parse(data.trim()) as { id: number; method: string };
if (msg.method === 'session/prompt') {
setImmediate(() => {
// Gemini sends thought chunks before message chunks
mock.sendLine({
jsonrpc: '2.0',
method: 'session/update',
params: {
sessionId: 'sess-1',
update: {
sessionUpdate: 'agent_thought_chunk',
content: { type: 'text', text: 'Thinking about it...' },
},
},
});
mock.sendLine({
jsonrpc: '2.0',
method: 'session/update',
params: {
sessionId: 'sess-1',
update: {
sessionUpdate: 'agent_message_chunk',
content: { type: 'text', text: 'ok' },
},
},
});
mock.sendResponse(msg.id, { stopReason: 'end_turn' });
});
}
});
const result = await client.prompt('test');
expect(result).toBe('ok');
});
it('calls ensureReady automatically (lazy init)', async () => {
createClient();
autoHandshake('sess-auto');

View File

@@ -69,7 +69,7 @@ describe('GeminiAcpProvider', () => {
expect(result.content).toBe('padded response');
});
it('serializes concurrent calls', async () => {
it('serializes concurrent calls to same model', async () => {
const callOrder: number[] = [];
let callCount = 0;
@@ -110,6 +110,70 @@ describe('GeminiAcpProvider', () => {
});
});
describe('session pool', () => {
it('creates separate pool entries for different models', async () => {
mockPrompt.mockResolvedValue('ok');
await provider.complete({ messages: [{ role: 'user', content: 'a' }], model: 'gemini-2.5-flash' });
await provider.complete({ messages: [{ role: 'user', content: 'b' }], model: 'gemini-2.5-pro' });
expect(provider.poolSize).toBe(2);
});
it('reuses existing pool entry for same model', async () => {
mockPrompt.mockResolvedValue('ok');
await provider.complete({ messages: [{ role: 'user', content: 'a' }], model: 'gemini-2.5-flash' });
await provider.complete({ messages: [{ role: 'user', content: 'b' }], model: 'gemini-2.5-flash' });
expect(provider.poolSize).toBe(1);
});
it('uses defaultModel when no model specified', async () => {
mockPrompt.mockResolvedValue('ok');
await provider.complete({ messages: [{ role: 'user', content: 'a' }] });
expect(provider.poolSize).toBe(1);
});
it('evicts idle sessions', async () => {
// Use a very short TTL for testing
const shortTtl = new GeminiAcpProvider({
binaryPath: '/usr/bin/gemini',
defaultModel: 'gemini-2.5-flash',
idleTtlMs: 1, // 1ms TTL
});
mockPrompt.mockResolvedValue('ok');
await shortTtl.complete({ messages: [{ role: 'user', content: 'a' }], model: 'model-a' });
expect(shortTtl.poolSize).toBe(1);
// Wait for TTL to expire
await new Promise((r) => setTimeout(r, 10));
// Next complete call triggers eviction of old entry and creates new one
await shortTtl.complete({ messages: [{ role: 'user', content: 'b' }], model: 'model-b' });
// model-a should have been evicted, only model-b remains
expect(shortTtl.poolSize).toBe(1);
expect(mockDispose).toHaveBeenCalled();
shortTtl.dispose();
});
it('dispose kills all pooled clients', async () => {
mockPrompt.mockResolvedValue('ok');
await provider.complete({ messages: [{ role: 'user', content: 'a' }], model: 'model-a' });
await provider.complete({ messages: [{ role: 'user', content: 'b' }], model: 'model-b' });
expect(provider.poolSize).toBe(2);
provider.dispose();
expect(provider.poolSize).toBe(0);
expect(mockDispose).toHaveBeenCalledTimes(2);
});
});
describe('listModels', () => {
it('returns static model list', async () => {
const models = await provider.listModels();
@@ -120,7 +184,9 @@ describe('GeminiAcpProvider', () => {
});
describe('dispose', () => {
it('delegates to AcpClient', () => {
it('delegates to all pooled AcpClients', async () => {
mockPrompt.mockResolvedValue('ok');
await provider.complete({ messages: [{ role: 'user', content: 'test' }] });
provider.dispose();
expect(mockDispose).toHaveBeenCalled();
});

View File

@@ -1,5 +1,5 @@
import { describe, it, expect, vi, afterEach } from 'vitest';
import { loadLlmConfig } from '../../src/http/config.js';
import { describe, it, expect, vi, afterEach, beforeEach } from 'vitest';
import { loadLlmConfig, resetConfigCache } from '../../src/http/config.js';
import { existsSync, readFileSync } from 'node:fs';
vi.mock('node:fs', async () => {
@@ -11,6 +11,10 @@ vi.mock('node:fs', async () => {
};
});
beforeEach(() => {
resetConfigCache();
});
afterEach(() => {
vi.restoreAllMocks();
});

View File

@@ -6,13 +6,14 @@
* (node:http) and a mock LLM provider. No Docker or external services needed.
*/
import { describe, it, expect, beforeEach, afterEach, afterAll } from 'vitest';
import { describe, it, expect, vi, beforeEach, afterEach, afterAll } from 'vitest';
import { createServer, type Server, type IncomingMessage, type ServerResponse } from 'node:http';
import { McpRouter } from '../../src/router.js';
import { McpdUpstream } from '../../src/upstream/mcpd.js';
import { McpdClient } from '../../src/http/mcpd-client.js';
import { LlmProcessor, DEFAULT_PROCESSOR_CONFIG } from '../../src/llm/processor.js';
import { ResponsePaginator } from '../../src/llm/pagination.js';
import { ProviderRegistry } from '../../src/providers/registry.js';
import { TieredHealthMonitor } from '../../src/health/tiered.js';
import { refreshUpstreams } from '../../src/discovery.js';
@@ -1096,4 +1097,429 @@ describe('End-to-end integration: 3-tier architecture', () => {
}
});
});
// -----------------------------------------------------------------------
// 8. Smart pagination through the full pipeline
// -----------------------------------------------------------------------
describe('Smart pagination', () => {
// Helper: generate a large JSON response (~100KB)
function makeLargeToolResult(): { flows: Array<{ id: string; type: string; label: string; wires: string[] }> } {
return {
flows: Array.from({ length: 200 }, (_, i) => ({
id: `flow-${String(i).padStart(4, '0')}`,
type: i % 3 === 0 ? 'function' : i % 3 === 1 ? 'http request' : 'inject',
label: `Node ${String(i)}: ${i % 3 === 0 ? 'Data transform' : i % 3 === 1 ? 'API call' : 'Timer trigger'}`,
wires: [`flow-${String(i + 1).padStart(4, '0')}`],
})),
};
}
it('paginates large tool response with smart AI summaries through router', async () => {
const largeResult = makeLargeToolResult();
mockMcpd = await startMockMcpd({
servers: [{ id: 'srv-nodered', name: 'node-red', transport: 'stdio' }],
proxyResponses: new Map([
['srv-nodered:tools/list', {
result: { tools: [{ name: 'get_flows', description: 'Get all flows' }] },
}],
['srv-nodered:tools/call', {
result: largeResult,
}],
]),
});
const client = new McpdClient(mockMcpd.baseUrl, mockMcpd.config.expectedToken);
router = new McpRouter();
await refreshUpstreams(router, client);
await router.discoverTools();
// Set up paginator with LLM provider for smart summaries
const registry = new ProviderRegistry();
const completeFn = vi.fn().mockImplementation(() => ({
content: JSON.stringify([
{ page: 1, summary: 'Function nodes and data transforms (flow-0000 through flow-0050)' },
{ page: 2, summary: 'HTTP request nodes and API integrations (flow-0051 through flow-0100)' },
{ page: 3, summary: 'Inject/timer nodes and triggers (flow-0101 through flow-0150)' },
{ page: 4, summary: 'Remaining nodes and wire connections (flow-0151 through flow-0199)' },
]),
}));
const mockProvider: LlmProvider = {
name: 'test-paginator',
isAvailable: () => true,
complete: completeFn,
};
registry.register(mockProvider);
// Low threshold so our response triggers pagination
const paginator = new ResponsePaginator(registry, {
sizeThreshold: 1000,
pageSize: 8000,
});
router.setPaginator(paginator);
// Call the tool — should get pagination index, not raw data
const response = await router.route({
jsonrpc: '2.0',
id: 'paginate-1',
method: 'tools/call',
params: { name: 'node-red/get_flows', arguments: {} },
});
expect(response.error).toBeUndefined();
const result = response.result as { content: Array<{ type: string; text: string }> };
expect(result.content).toHaveLength(1);
const indexText = result.content[0]!.text;
// Verify smart index with AI summaries
expect(indexText).toContain('AI-generated summaries');
expect(indexText).toContain('Function nodes and data transforms');
expect(indexText).toContain('HTTP request nodes');
expect(indexText).toContain('_resultId');
expect(indexText).toContain('_page');
// LLM was called to generate summaries
expect(completeFn).toHaveBeenCalledOnce();
const llmCall = completeFn.mock.calls[0]![0]!;
expect(llmCall.messages[0].role).toBe('system');
expect(llmCall.messages[1].content).toContain('node-red/get_flows');
});
it('retrieves specific pages after pagination via _resultId/_page', async () => {
const largeResult = makeLargeToolResult();
mockMcpd = await startMockMcpd({
servers: [{ id: 'srv-nodered', name: 'node-red', transport: 'stdio' }],
proxyResponses: new Map([
['srv-nodered:tools/list', {
result: { tools: [{ name: 'get_flows', description: 'Get all flows' }] },
}],
['srv-nodered:tools/call', {
result: largeResult,
}],
]),
});
const client = new McpdClient(mockMcpd.baseUrl, mockMcpd.config.expectedToken);
router = new McpRouter();
await refreshUpstreams(router, client);
await router.discoverTools();
// Simple paginator (no LLM) for predictable behavior
const paginator = new ResponsePaginator(null, {
sizeThreshold: 1000,
pageSize: 8000,
});
router.setPaginator(paginator);
// First call — get the pagination index
const indexResponse = await router.route({
jsonrpc: '2.0',
id: 'idx-1',
method: 'tools/call',
params: { name: 'node-red/get_flows', arguments: {} },
});
expect(indexResponse.error).toBeUndefined();
const indexResult = indexResponse.result as { content: Array<{ text: string }> };
const indexText = indexResult.content[0]!.text;
const resultIdMatch = /"_resultId": "([^"]+)"/.exec(indexText);
expect(resultIdMatch).not.toBeNull();
const resultId = resultIdMatch![1]!;
// Second call — retrieve page 1 via _resultId/_page
const page1Response = await router.route({
jsonrpc: '2.0',
id: 'page-1',
method: 'tools/call',
params: {
name: 'node-red/get_flows',
arguments: { _resultId: resultId, _page: 1 },
},
});
expect(page1Response.error).toBeUndefined();
const page1Result = page1Response.result as { content: Array<{ text: string }> };
expect(page1Result.content[0]!.text).toContain('Page 1/');
// Page content should contain flow data
expect(page1Result.content[0]!.text).toContain('flow-');
// Third call — retrieve page 2
const page2Response = await router.route({
jsonrpc: '2.0',
id: 'page-2',
method: 'tools/call',
params: {
name: 'node-red/get_flows',
arguments: { _resultId: resultId, _page: 2 },
},
});
expect(page2Response.error).toBeUndefined();
const page2Result = page2Response.result as { content: Array<{ text: string }> };
expect(page2Result.content[0]!.text).toContain('Page 2/');
});
it('retrieves full content with _page=all', async () => {
const largeResult = makeLargeToolResult();
mockMcpd = await startMockMcpd({
servers: [{ id: 'srv-nodered', name: 'node-red', transport: 'stdio' }],
proxyResponses: new Map([
['srv-nodered:tools/list', {
result: { tools: [{ name: 'get_flows', description: 'Get all flows' }] },
}],
['srv-nodered:tools/call', {
result: largeResult,
}],
]),
});
const client = new McpdClient(mockMcpd.baseUrl, mockMcpd.config.expectedToken);
router = new McpRouter();
await refreshUpstreams(router, client);
await router.discoverTools();
const paginator = new ResponsePaginator(null, {
sizeThreshold: 1000,
pageSize: 8000,
});
router.setPaginator(paginator);
// Get index
const indexResponse = await router.route({
jsonrpc: '2.0',
id: 'all-idx',
method: 'tools/call',
params: { name: 'node-red/get_flows', arguments: {} },
});
const indexText = (indexResponse.result as { content: Array<{ text: string }> }).content[0]!.text;
const resultId = /"_resultId": "([^"]+)"/.exec(indexText)![1]!;
// Request all pages
const allResponse = await router.route({
jsonrpc: '2.0',
id: 'all-1',
method: 'tools/call',
params: {
name: 'node-red/get_flows',
arguments: { _resultId: resultId, _page: 'all' },
},
});
expect(allResponse.error).toBeUndefined();
const allResult = allResponse.result as { content: Array<{ text: string }> };
// Full response should contain the original JSON
const fullText = allResult.content[0]!.text;
expect(fullText).toContain('flow-0000');
expect(fullText).toContain('flow-0199');
// Should be the full serialized result
expect(JSON.parse(fullText)).toEqual(largeResult);
});
it('falls back to simple index when LLM fails', async () => {
const largeResult = makeLargeToolResult();
mockMcpd = await startMockMcpd({
servers: [{ id: 'srv-nodered', name: 'node-red', transport: 'stdio' }],
proxyResponses: new Map([
['srv-nodered:tools/list', {
result: { tools: [{ name: 'get_flows', description: 'Get all flows' }] },
}],
['srv-nodered:tools/call', {
result: largeResult,
}],
]),
});
const client = new McpdClient(mockMcpd.baseUrl, mockMcpd.config.expectedToken);
router = new McpRouter();
await refreshUpstreams(router, client);
await router.discoverTools();
// Set up paginator with a failing LLM
const registry = new ProviderRegistry();
registry.register(createFailingLlmProvider('broken-llm'));
const paginator = new ResponsePaginator(registry, {
sizeThreshold: 1000,
pageSize: 8000,
});
router.setPaginator(paginator);
const response = await router.route({
jsonrpc: '2.0',
id: 'fallback-idx',
method: 'tools/call',
params: { name: 'node-red/get_flows', arguments: {} },
});
expect(response.error).toBeUndefined();
const text = (response.result as { content: Array<{ text: string }> }).content[0]!.text;
// Should still paginate, just without AI summaries
expect(text).toContain('_resultId');
expect(text).not.toContain('AI-generated summaries');
expect(text).toContain('Page 1:');
});
it('returns expired cache message for stale _resultId', async () => {
router = new McpRouter();
const paginator = new ResponsePaginator(null, { sizeThreshold: 100, pageSize: 50 });
router.setPaginator(paginator);
// Try to retrieve a page with an unknown resultId
const response = await router.route({
jsonrpc: '2.0',
id: 'stale-1',
method: 'tools/call',
params: {
name: 'anything/tool',
arguments: { _resultId: 'nonexistent-id', _page: 1 },
},
});
expect(response.error).toBeUndefined();
const text = (response.result as { content: Array<{ text: string }> }).content[0]!.text;
expect(text).toContain('expired');
expect(text).toContain('re-call');
});
it('skips pagination for small responses', async () => {
mockMcpd = await startMockMcpd({
servers: [{ id: 'srv-small', name: 'smallserver', transport: 'stdio' }],
proxyResponses: new Map([
['srv-small:tools/list', {
result: { tools: [{ name: 'get_status', description: 'Get status' }] },
}],
['srv-small:tools/call', {
result: { status: 'ok', uptime: 12345 },
}],
]),
});
const client = new McpdClient(mockMcpd.baseUrl, mockMcpd.config.expectedToken);
router = new McpRouter();
await refreshUpstreams(router, client);
await router.discoverTools();
const paginator = new ResponsePaginator(null, { sizeThreshold: 80000, pageSize: 40000 });
router.setPaginator(paginator);
const response = await router.route({
jsonrpc: '2.0',
id: 'small-1',
method: 'tools/call',
params: { name: 'smallserver/get_status', arguments: {} },
});
expect(response.error).toBeUndefined();
// Should return the raw result directly, not a pagination index
expect(response.result).toEqual({ status: 'ok', uptime: 12345 });
});
it('handles markdown-fenced LLM responses (Gemini quirk)', async () => {
const largeResult = makeLargeToolResult();
mockMcpd = await startMockMcpd({
servers: [{ id: 'srv-nodered', name: 'node-red', transport: 'stdio' }],
proxyResponses: new Map([
['srv-nodered:tools/list', {
result: { tools: [{ name: 'get_flows', description: 'Get all flows' }] },
}],
['srv-nodered:tools/call', {
result: largeResult,
}],
]),
});
const client = new McpdClient(mockMcpd.baseUrl, mockMcpd.config.expectedToken);
router = new McpRouter();
await refreshUpstreams(router, client);
await router.discoverTools();
// Simulate Gemini wrapping JSON in ```json fences
const registry = new ProviderRegistry();
const mockProvider: LlmProvider = {
name: 'gemini-mock',
isAvailable: () => true,
complete: vi.fn().mockResolvedValue({
content: '```json\n' + JSON.stringify([
{ page: 1, summary: 'Climate automation flows' },
{ page: 2, summary: 'Lighting control flows' },
]) + '\n```',
}),
};
registry.register(mockProvider);
const paginator = new ResponsePaginator(registry, {
sizeThreshold: 1000,
pageSize: 8000,
});
router.setPaginator(paginator);
const response = await router.route({
jsonrpc: '2.0',
id: 'fence-1',
method: 'tools/call',
params: { name: 'node-red/get_flows', arguments: {} },
});
expect(response.error).toBeUndefined();
const text = (response.result as { content: Array<{ text: string }> }).content[0]!.text;
// Fences were stripped — smart summaries should appear
expect(text).toContain('AI-generated summaries');
expect(text).toContain('Climate automation flows');
expect(text).toContain('Lighting control flows');
});
it('passes model override to LLM when project has custom model', async () => {
const largeResult = makeLargeToolResult();
mockMcpd = await startMockMcpd({
servers: [{ id: 'srv-nodered', name: 'node-red', transport: 'stdio' }],
proxyResponses: new Map([
['srv-nodered:tools/list', {
result: { tools: [{ name: 'get_flows', description: 'Get all flows' }] },
}],
['srv-nodered:tools/call', {
result: largeResult,
}],
]),
});
const client = new McpdClient(mockMcpd.baseUrl, mockMcpd.config.expectedToken);
router = new McpRouter();
await refreshUpstreams(router, client);
await router.discoverTools();
const registry = new ProviderRegistry();
const completeFn = vi.fn().mockResolvedValue({
content: JSON.stringify([{ page: 1, summary: 'test' }]),
});
const mockProvider: LlmProvider = {
name: 'test-model-override',
isAvailable: () => true,
complete: completeFn,
};
registry.register(mockProvider);
// Paginator with per-project model override
const paginator = new ResponsePaginator(registry, {
sizeThreshold: 1000,
pageSize: 80000, // One big page so we get exactly 1 summary
}, 'gemini-2.5-pro');
router.setPaginator(paginator);
await router.route({
jsonrpc: '2.0',
id: 'model-1',
method: 'tools/call',
params: { name: 'node-red/get_flows', arguments: {} },
});
// Verify the model was passed through to the LLM call
expect(completeFn).toHaveBeenCalledOnce();
const llmOpts = completeFn.mock.calls[0]![0]!;
expect(llmOpts.model).toBe('gemini-2.5-pro');
});
});
});

View File

@@ -116,9 +116,9 @@ describe('createProviderFromConfig', () => {
{ provider: 'vllm', model: 'my-model', url: 'http://gpu-server:8000' },
store,
);
// vLLM reuses OpenAI provider under the hood
// vLLM reuses OpenAI provider under the hood, wrapped with NamedProvider
expect(registry.getActive()).not.toBeNull();
expect(registry.getActive()!.name).toBe('openai');
expect(registry.getActive()!.name).toBe('vllm');
});
it('returns empty registry when vllm URL is missing', async () => {

View File

@@ -11,6 +11,7 @@ function makeProvider(response: string): ProviderRegistry {
};
return {
getActive: () => provider,
getProvider: () => provider,
register: vi.fn(),
setActive: vi.fn(),
listProviders: () => [{ name: 'test', available: true, active: true }],
@@ -150,6 +151,25 @@ describe('ResponsePaginator', () => {
expect(text).toContain('HTTP request nodes and API integrations');
});
it('strips markdown code fences from LLM JSON response', async () => {
const summaries = [
{ page: 1, summary: 'Config section' },
{ page: 2, summary: 'Data section' },
];
// Gemini often wraps JSON in ```json ... ``` fences
const fenced = '```json\n' + JSON.stringify(summaries) + '\n```';
const registry = makeProvider(fenced);
const paginator = new ResponsePaginator(registry, { sizeThreshold: 100, pageSize: 60 });
const raw = makeLargeStringWithNewlines(150);
const result = await paginator.paginate('test/tool', raw);
expect(result).not.toBeNull();
const text = result!.content[0]!.text;
expect(text).toContain('AI-generated summaries');
expect(text).toContain('Config section');
expect(text).toContain('Data section');
});
it('falls back to simple index on LLM failure', async () => {
const provider: LlmProvider = {
name: 'test',
@@ -158,6 +178,7 @@ describe('ResponsePaginator', () => {
};
const registry = {
getActive: () => provider,
getProvider: () => provider,
register: vi.fn(),
setActive: vi.fn(),
listProviders: () => [{ name: 'test', available: true, active: true }],
@@ -189,6 +210,7 @@ describe('ResponsePaginator', () => {
};
const registry = {
getActive: () => provider,
getProvider: () => provider,
register: vi.fn(),
setActive: vi.fn(),
listProviders: () => [{ name: 'test', available: true, active: true }],
@@ -212,6 +234,7 @@ describe('ResponsePaginator', () => {
it('falls back to simple when no active provider', async () => {
const registry = {
getActive: () => null,
getProvider: () => null,
register: vi.fn(),
setActive: vi.fn(),
listProviders: () => [],
@@ -225,6 +248,58 @@ describe('ResponsePaginator', () => {
const text = result!.content[0]!.text;
expect(text).not.toContain('AI-generated summaries');
});
it('passes modelOverride to provider.complete()', async () => {
const completeFn = vi.fn().mockResolvedValue({
content: JSON.stringify([{ page: 1, summary: 'test' }, { page: 2, summary: 'test2' }]),
});
const provider: LlmProvider = {
name: 'test',
isAvailable: () => true,
complete: completeFn,
};
const registry = {
getActive: () => provider,
getProvider: () => provider,
register: vi.fn(),
setActive: vi.fn(),
listProviders: () => [{ name: 'test', available: true, active: true }],
} as unknown as ProviderRegistry;
const paginator = new ResponsePaginator(registry, { sizeThreshold: 100, pageSize: 60 }, 'gemini-2.5-pro');
const raw = makeLargeStringWithNewlines(150);
await paginator.paginate('test/tool', raw);
expect(completeFn).toHaveBeenCalledOnce();
const call = completeFn.mock.calls[0]![0]!;
expect(call.model).toBe('gemini-2.5-pro');
});
it('omits model when no modelOverride set', async () => {
const completeFn = vi.fn().mockResolvedValue({
content: JSON.stringify([{ page: 1, summary: 'test' }, { page: 2, summary: 'test2' }]),
});
const provider: LlmProvider = {
name: 'test',
isAvailable: () => true,
complete: completeFn,
};
const registry = {
getActive: () => provider,
getProvider: () => provider,
register: vi.fn(),
setActive: vi.fn(),
listProviders: () => [{ name: 'test', available: true, active: true }],
} as unknown as ProviderRegistry;
const paginator = new ResponsePaginator(registry, { sizeThreshold: 100, pageSize: 60 });
const raw = makeLargeStringWithNewlines(150);
await paginator.paginate('test/tool', raw);
expect(completeFn).toHaveBeenCalledOnce();
const call = completeFn.mock.calls[0]![0]!;
expect(call.model).toBeUndefined();
});
});
// --- getPage ---

View File

@@ -6,8 +6,18 @@ import { registerProjectMcpEndpoint } from '../src/http/project-mcp-endpoint.js'
// Mock discovery module — we don't want real HTTP calls
vi.mock('../src/discovery.js', () => ({
refreshProjectUpstreams: vi.fn(async () => ['mock-server']),
fetchProjectLlmConfig: vi.fn(async () => ({})),
}));
// Mock config module — don't read real config files
vi.mock('../src/http/config.js', async () => {
const actual = await vi.importActual<typeof import('../src/http/config.js')>('../src/http/config.js');
return {
...actual,
loadProjectLlmOverride: vi.fn(() => undefined),
};
});
import { refreshProjectUpstreams } from '../src/discovery.js';
function mockMcpdClient() {

View File

@@ -115,4 +115,105 @@ describe('ProviderRegistry', () => {
expect(models).toEqual(['anthropic-model-1', 'anthropic-model-2']);
});
describe('tier management', () => {
it('assigns providers to tiers', () => {
registry.register(mockProvider('vllm'));
registry.register(mockProvider('gemini'));
registry.assignTier('vllm', 'fast');
registry.assignTier('gemini', 'heavy');
expect(registry.getTierProviders('fast')).toEqual(['vllm']);
expect(registry.getTierProviders('heavy')).toEqual(['gemini']);
expect(registry.hasTierConfig()).toBe(true);
});
it('getProvider returns tier-specific provider', () => {
const vllm = mockProvider('vllm');
const gemini = mockProvider('gemini');
registry.register(vllm);
registry.register(gemini);
registry.assignTier('vllm', 'fast');
registry.assignTier('gemini', 'heavy');
expect(registry.getProvider('fast')).toBe(vllm);
expect(registry.getProvider('heavy')).toBe(gemini);
});
it('getProvider falls back to other tier', () => {
const vllm = mockProvider('vllm');
registry.register(vllm);
registry.assignTier('vllm', 'fast');
// Requesting heavy but only fast exists → falls back to fast
expect(registry.getProvider('heavy')).toBe(vllm);
});
it('getProvider falls back to getActive when no tiers', () => {
const openai = mockProvider('openai');
registry.register(openai);
// No tier assignments → falls back to legacy getActive()
expect(registry.getProvider('fast')).toBe(openai);
expect(registry.getProvider('heavy')).toBe(openai);
expect(registry.hasTierConfig()).toBe(false);
});
it('unregister removes from tier assignments', () => {
registry.register(mockProvider('vllm'));
registry.register(mockProvider('gemini'));
registry.assignTier('vllm', 'fast');
registry.assignTier('gemini', 'heavy');
registry.unregister('vllm');
expect(registry.getTierProviders('fast')).toEqual([]);
expect(registry.getTierProviders('heavy')).toEqual(['gemini']);
});
it('assignTier throws for unregistered provider', () => {
expect(() => registry.assignTier('unknown', 'fast')).toThrow("Provider 'unknown' is not registered");
});
it('multiple providers in same tier uses first', () => {
const vllm = mockProvider('vllm');
const ollama = mockProvider('ollama');
registry.register(vllm);
registry.register(ollama);
registry.assignTier('vllm', 'fast');
registry.assignTier('ollama', 'fast');
expect(registry.getProvider('fast')).toBe(vllm);
expect(registry.getTierProviders('fast')).toEqual(['vllm', 'ollama']);
});
it('listProviders includes tier info', () => {
registry.register(mockProvider('vllm'));
registry.register(mockProvider('gemini'));
registry.assignTier('vllm', 'fast');
registry.assignTier('gemini', 'heavy');
const providers = registry.listProviders();
expect(providers).toEqual([
{ name: 'vllm', tiers: ['fast'] },
{ name: 'gemini', tiers: ['heavy'] },
]);
});
it('disposeAll calls dispose on all providers', () => {
const disposeFn = vi.fn();
const provider = { ...mockProvider('test'), dispose: disposeFn };
registry.register(provider);
registry.disposeAll();
expect(disposeFn).toHaveBeenCalledOnce();
});
});
});