diff --git a/README.md b/README.md index b76b6d0..16ead01 100644 --- a/README.md +++ b/README.md @@ -258,15 +258,17 @@ The homelab gateway (LiteLLM in front of vLLM) already serves the Anthropic Messages API, so Claude Code can talk to it directly — no bridge: ```bash -claude-vllm # reuse the provider/model/key pi or prime-agent already uses +claude-vllm --init # write ~/.config/mcpctl/claude-vllm.jsonc (0600) +claude-vllm # use it claude-vllm --model deepseek-v4-max claude-vllm --list # show what it would use claude-vllm -- -p "summarise this repo" # args after -- go to claude ``` -It discovers the endpoint, credential, model and context window from -`~/.pi/agent`, `~/.prime/agent` or opencode's config (first hit wins) and sets -the `ANTHROPIC_*` environment Claude Code needs. +It reads its own config (shaped like `opencode.jsonc`, but a separate file), +falling back to `~/.pi/agent` or `~/.prime/agent` so it works before you have +written one. The key may be a literal in the 0600 file, `${ENV_VAR}`, or an env +var name — it is never stored in the tool. See [docs/claude-vllm.md](docs/claude-vllm.md), including why routing this through mcpctl would add surface without adding capability. diff --git a/docs/claude-vllm.md b/docs/claude-vllm.md index 72c3446..30ee014 100644 --- a/docs/claude-vllm.md +++ b/docs/claude-vllm.md @@ -43,21 +43,60 @@ sake: per-project gating of model calls, audit of prompts, or budget enforcement. Those are real features, but they are a passthrough endpoint in mcpd, not a wrapper script. -## What it discovers, and from where +## Its own config -`claude-vllm` exists only so you don't paste four exports each time. It reuses -whatever you already configured for another agent — first hit wins: +```bash +claude-vllm --init # prompts for the key, writes 0600 +claude-vllm --init --api-key '${MY_KEY}' # keep the secret in the environment instead +claude-vllm --init --base-url https://other-gateway/v1 +``` + +Written to `$XDG_CONFIG_HOME/mcpctl/claude-vllm.jsonc` (`~/.config/…`), shaped +like `opencode.jsonc` — a `provider` map with `options.baseURL` / +`options.apiKey` and a `models` map: + +```jsonc +{ + "model": "itaz/deepseek-v4-think", + "provider": { + "itaz": { + "options": { "baseURL": "https://llm.ad.itaz.eu/v1", "apiKey": "${MY_LLM_KEY}" }, + "models": { "deepseek-v4-fast": { "limit": { "context": 393216 } } } + } + } +} +``` + +**Same shape as opencode's config, but a separate file.** Reading opencode's own +config would mean a credential rotation there silently changing what Claude Code +authenticates with, and would couple two tools' configs for no reason. + +### Keys are never stored in the tool + +`apiKey` may be a literal (in a 0600 file), `${ENV_VAR}`, or a bare env var +NAME — so the secret can live in your environment or a password manager instead +of on disk. `--init` reads the key from **stdin** when `--api-key` is omitted, +keeping it out of shell history and out of the process table, where `--api-key` +would be visible to every user via `ps`. `--list` never prints more than a +10-character prefix. + +### Discovery order | Order | Source | Supplies | |-------|--------|----------| | 1 | environment (`ANTHROPIC_BASE_URL`, `ANTHROPIC_AUTH_TOKEN`, `ANTHROPIC_MODEL`) | anything already set is respected | | 2 | `--provider` / `--model` flags | provider, model | -| 3 | `~/.pi/agent` | `settings.json` → default provider/model · `models.json` → base URL, context window · `auth.json` → key | -| 4 | `~/.prime/agent` | same shape | -| 5 | `~/.config/opencode/opencode.jsonc` | `provider..options.{baseURL,apiKey}` | +| 3 | `~/.config/mcpctl/claude-vllm.jsonc` | its own config | +| 4 | `~/.pi/agent` | `settings.json` → default provider/model · `models.json` → base URL, context windows · `auth.json` → key | +| 5 | `~/.prime/agent` | same shape | -The base URL and the credential always come from the *same* source, so one -gateway's URL is never paired with another's key. +The pi/prime homes remain a convenience fallback so `claude-vllm` works before +you have written a config at all. The base URL and the credential always come +from the *same* source, so one gateway's URL is never paired with another's key. + +`--init` records the context window for **every** model it can see, not just the +active one — otherwise `--model something-else` silently falls back to Claude +Code's assumed 200k. ## What it sets, and why each one diff --git a/stack/claude-vllm b/stack/claude-vllm index d7119c0..aebb04e 100755 --- a/stack/claude-vllm +++ b/stack/claude-vllm @@ -10,22 +10,37 @@ # Discovery order (first hit wins, per field): # 1. environment already set (ANTHROPIC_BASE_URL / ANTHROPIC_AUTH_TOKEN / ANTHROPIC_MODEL) # 2. flags (--provider, --model) -# 3. ~/.pi/agent settings.json + models.json + auth.json -# 4. ~/.prime/agent settings.json + models.json + auth.json -# 5. ~/.config/opencode/opencode.jsonc (provider..options.{baseURL,apiKey}) +# 3. its own config: $XDG_CONFIG_HOME/mcpctl/claude-vllm.jsonc (~/.config/...) +# 4. ~/.pi/agent settings.json + models.json + auth.json +# 5. ~/.prime/agent settings.json + models.json + auth.json +# +# The config file is shaped like opencode's — a `provider` map with +# `options.baseURL` / `options.apiKey` and a `models` map — but it is OUR file. +# Reading opencode's own config would mean a credential rotation there silently +# changing what Claude Code authenticates with, and would tie two tools' configs +# together for no reason. The pi/prime homes remain as a convenience fallback so +# `claude-vllm` works before you have written a config at all. +# +# NO KEY IS EVER STORED IN THIS SCRIPT. `apiKey` may be a literal (in a 0600 +# config), or `\${ENV_VAR}` / a bare env var NAME, so the secret can live in your +# environment or a password manager instead of on disk. # # Usage: # claude-vllm # default provider + model, then exec claude # claude-vllm --model deepseek-v4-max # claude-vllm --provider itaz --model deepseek-v4-fast -- -p "summarise this repo" -# claude-vllm --list # show discoverable providers/models and exit +# claude-vllm --list # show what it would use (never prints the key) # claude-vllm --print-env # print the exports and exit (don't run claude) +# claude-vllm --init --api-key ... # write the config file (0600); reads stdin if omitted set -euo pipefail PROVIDER="" MODEL="" LIST=0 +INIT=0 PRINT_ENV=0 +API_KEY_ARG="" +BASE_URL_ARG="" CLAUDE_ARGS=() while [ $# -gt 0 ]; do @@ -33,6 +48,9 @@ while [ $# -gt 0 ]; do --provider) PROVIDER="${2:?--provider needs a value}"; shift 2 ;; --model) MODEL="${2:?--model needs a value}"; shift 2 ;; --list) LIST=1; shift ;; + --init) INIT=1; shift ;; + --api-key) API_KEY_ARG="${2:?--api-key needs a value}"; shift 2 ;; + --base-url) BASE_URL_ARG="${2:?--base-url needs a value}"; shift 2 ;; --print-env) PRINT_ENV=1; shift ;; -h|--help) sed -n '2,25p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;; --) shift; CLAUDE_ARGS+=("$@"); break ;; @@ -44,12 +62,27 @@ command -v jq >/dev/null || { echo "claude-vllm: jq is required" >&2; exit 1; } PI_HOME="${PI_AGENT_HOME:-$HOME/.pi/agent}" PRIME_HOME="${PRIME_AGENT_HOME:-$HOME/.prime/agent}" -OC_CONFIG="${XDG_CONFIG_HOME:-$HOME/.config}/opencode/opencode.jsonc" +CONFIG="${CLAUDE_VLLM_CONFIG:-${XDG_CONFIG_HOME:-$HOME/.config}/mcpctl/claude-vllm.jsonc}" # jq tolerates the // comments opencode.jsonc may contain only after we strip # them; harmless for strict JSON. read_json() { [ -f "$1" ] && sed 's://[^"]*$::' "$1" | jq -c . 2>/dev/null || echo '{}'; } +# Resolve an apiKey field that may be a literal, \${ENV_VAR}, or a bare env var +# NAME (the form pi's models.json uses). Keeping the indirection means the +# secret can live in the environment rather than on disk. +resolve_key() { + local raw="$1" + [ -n "$raw" ] || return 0 + case "$raw" in + '${'*'}') local n="${raw#\$\{}"; n="${n%\}}"; printf '%s' "${!n:-}" ;; + # A bare ALL_CAPS token that names a set variable is an env var reference, + # not a key: no real API key looks like that. + [A-Z_][A-Z0-9_]*) if [ -n "${!raw:-}" ]; then printf '%s' "${!raw}"; else printf '%s' "$raw"; fi ;; + *) printf '%s' "$raw" ;; + esac +} + # ── discover ───────────────────────────────────────────────────────────────── # Each agent home is tried in turn; the first one that yields a base URL wins, # and the credential is taken from that same home so we never pair one gateway's @@ -74,45 +107,107 @@ discover_from_agent_home() { FOUND_PROVIDER="$provider"; FOUND_BASE="$base"; FOUND_KEY="$key"; FOUND_MODEL="$model" FOUND_MODELS=$(jq -r --arg p "$provider" '.providers[$p].models[]?.id' <<<"$models") + # model -> context for ALL models, so --init records every limit rather than + # only the active one (Claude Code assumes 200k for anything it lacks). + FOUND_MODEL_LIMITS=$(jq -c --arg p "$provider" \ + '[.providers[$p].models[]? | select(.contextWindow) | {(.id): {limit:{context:.contextWindow}}}] | add // {}' <<<"$models") FOUND_CONTEXT=$(jq -r --arg p "$provider" --arg m "$model" \ '.providers[$p].models[]? | select(.id==$m) | .contextWindow // empty' <<<"$models") FOUND_SOURCE="$home" return 0 } -discover_from_opencode() { +# Our own config, shaped like opencode's but deliberately a separate file. +discover_from_config() { local cfg provider base key - cfg=$(read_json "$OC_CONFIG") + [ -f "$CONFIG" ] || return 1 + cfg=$(read_json "$CONFIG") provider="$PROVIDER" - if [ -z "$provider" ]; then - provider=$(jq -r '(.model // "") | split("/")[0] // empty' <<<"$cfg") - fi + [ -n "$provider" ] || provider=$(jq -r '(.model // "") | split("/")[0] // empty' <<<"$cfg") + [ -n "$provider" ] || provider=$(jq -r '.provider | keys[0] // empty' <<<"$cfg") [ -n "$provider" ] || return 1 + base=$(jq -r --arg p "$provider" '.provider[$p].options.baseURL // empty' <<<"$cfg") [ -n "$base" ] || return 1 - key=$(jq -r --arg p "$provider" '.provider[$p].options.apiKey // empty' <<<"$cfg") + key=$(resolve_key "$(jq -r --arg p "$provider" '.provider[$p].options.apiKey // empty' <<<"$cfg")") FOUND_PROVIDER="$provider"; FOUND_BASE="$base"; FOUND_KEY="$key" FOUND_MODEL="${MODEL:-$(jq -r '(.model // "") | split("/")[1] // empty' <<<"$cfg")}" FOUND_MODELS=$(jq -r --arg p "$provider" '.provider[$p].models | keys[]?' <<<"$cfg") + FOUND_MODEL_LIMITS=$(jq -c --arg p "$provider" '.provider[$p].models // {}' <<<"$cfg") FOUND_CONTEXT=$(jq -r --arg p "$provider" --arg m "$FOUND_MODEL" \ '.provider[$p].models[$m].limit.context // empty' <<<"$cfg") - FOUND_SOURCE="$OC_CONFIG" + FOUND_SOURCE="$CONFIG" return 0 } -FOUND_PROVIDER=""; FOUND_BASE=""; FOUND_KEY=""; FOUND_MODEL=""; FOUND_MODELS=""; FOUND_CONTEXT=""; FOUND_SOURCE="" -discover_from_agent_home "$PI_HOME" \ +FOUND_PROVIDER=""; FOUND_BASE=""; FOUND_KEY=""; FOUND_MODEL=""; FOUND_MODELS=""; FOUND_MODEL_LIMITS=""; FOUND_CONTEXT=""; FOUND_SOURCE="" +discover_from_config \ + || discover_from_agent_home "$PI_HOME" \ || discover_from_agent_home "$PRIME_HOME" \ - || discover_from_opencode \ || true +# ── --init: write the config file ──────────────────────────────────────────── +if [ "$INIT" = 1 ]; then + init_base="${BASE_URL_ARG:-${FOUND_BASE:-}}" + if [ -z "$init_base" ]; then + echo "claude-vllm --init: no endpoint known. Pass --base-url https://your-gateway/v1" >&2 + exit 1 + fi + init_key="$API_KEY_ARG" + if [ -z "$init_key" ]; then + # Read from stdin so the key never lands in shell history or the process + # table (where --api-key is visible to every user via `ps`). + if [ -t 0 ]; then + printf 'API key (input hidden, or pass ${ENV_VAR} to keep it out of the file): ' >&2 + read -rs init_key; echo >&2 + else + read -r init_key || true + fi + fi + [ -n "$init_key" ] || { echo "claude-vllm --init: no API key given" >&2; exit 1; } + + init_provider="${PROVIDER:-${FOUND_PROVIDER:-default}}" + init_model="${MODEL:-${FOUND_MODEL:-}}" + # Every model's limit, not just the active one — switching with --model must + # not silently drop back to Claude Code's assumed 200k. + init_models_json=$( + if [ -n "$FOUND_MODELS" ]; then + known="${FOUND_MODEL_LIMITS:-{\}}" + for m in $FOUND_MODELS; do jq -n --arg m "$m" '{($m): {}}'; done \ + | jq -s --argjson known "$known" 'add // {} | . * $known' + else + jq -n --arg m "$init_model" 'if $m == "" then {} else {($m): {}} end' + fi + ) + + mkdir -p "$(dirname "$CONFIG")" + umask 077 + jq -n --arg p "$init_provider" --arg model "$init_model" --arg base "$init_base" \ + --arg key "$init_key" --argjson models "$init_models_json" ' + { + "//": "mcpctl claude-vllm config. Shaped like opencode.jsonc, but its own file. apiKey may be a literal, \"${ENV_VAR}\", or a bare env var NAME.", + model: (if $model == "" then null else "\($p)/\($model)" end), + provider: { ($p): { options: { baseURL: $base, apiKey: $key }, models: $models } } + } | del(..|nulls)' > "$CONFIG.tmp.$$" + mv "$CONFIG.tmp.$$" "$CONFIG" + chmod 600 "$CONFIG" + echo "Wrote $CONFIG (0600)" >&2 + echo " provider: $init_provider endpoint: $init_base model: ${init_model:-}" >&2 + case "$init_key" in + '${'*'}'|[A-Z_][A-Z0-9_]*) echo " apiKey: kept as an environment reference, not a literal" >&2 ;; + *) echo " apiKey: stored in the file — readable only by you" >&2 ;; + esac + exit 0 +fi + BASE="${ANTHROPIC_BASE_URL:-$FOUND_BASE}" KEY="${ANTHROPIC_AUTH_TOKEN:-${ANTHROPIC_API_KEY:-$FOUND_KEY}}" MODEL_ID="${MODEL:-${ANTHROPIC_MODEL:-$FOUND_MODEL}}" if [ "$LIST" = 1 ]; then echo "provider: ${FOUND_PROVIDER:-} (from ${FOUND_SOURCE:-nowhere})" + echo "config: $CONFIG $([ -f "$CONFIG" ] && echo '(present)' || echo '(absent — using fallback; run --init)')" echo "endpoint: ${BASE:-}" echo "credential: $([ -n "$KEY" ] && echo "found (${KEY:0:10}…)" || echo "")" echo "default model: ${MODEL_ID:-}${FOUND_CONTEXT:+ (context ${FOUND_CONTEXT})}"