mcpctl/docs/claude-vllm.md

claude-vllm — Claude Code on the homelab LLM

The short version

claude-vllm                                   # default provider + model
claude-vllm --model deepseek-v4-max
claude-vllm --list                            # what would it use?
claude-vllm --print-env                       # the exports, without running claude
claude-vllm -- -p "summarise this repo"       # anything after -- goes to claude

Why no bridge is needed

The homelab gateway at llm.ad.itaz.eu is LiteLLM in front of vLLM, and LiteLLM already serves the Anthropic Messages API:

$ curl -s https://llm.ad.itaz.eu/v1/messages -X POST \
    -H 'x-api-key: <key>' -H 'anthropic-version: 2023-06-01' \
    -d '{"model":"deepseek-v4-fast","max_tokens":32,
         "messages":[{"role":"user","content":"say only: ok"}]}'
{"type":"message","role":"assistant","model":"deepseek-v4-fast",
 "content":[{"type":"text","text":"ok"}],"stop_reason":"end_turn", …}

That is the exact protocol Claude Code speaks. So pointing ANTHROPIC_BASE_URL at the gateway is the whole integration — no translation layer, no proxy, no mcpctl in the request path.

Why not route it through mcpctl

mcpctl's LLM layer is a client, not a server: mcpd exposes /api/v1/llms (management) and the adapters in src/mcpd/src/services/llm/adapters/ call out to Anthropic/OpenAI for gating, prompt selection and agent chat. Nothing in mcpd or mcplocal serves /v1/messages. A claude-mcpctl would therefore mean adding an Anthropic-shaped passthrough to mcpd that re-wraps what LiteLLM already does — new surface, new failure mode, no new capability.

It would only be worth it if you wanted mcpctl in the LLM path for its own sake: per-project gating of model calls, audit of prompts, or budget enforcement. Those are real features, but they are a passthrough endpoint in mcpd, not a wrapper script.

Its own config

claude-vllm --init                       # prompts for the key, writes 0600
claude-vllm --init --api-key '${MY_KEY}' # keep the secret in the environment instead
claude-vllm --init --base-url https://other-gateway/v1

Written to $XDG_CONFIG_HOME/mcpctl/claude-vllm.jsonc (~/.config/…), shaped like opencode.jsonc — a provider map with options.baseURL / options.apiKey and a models map:

{
  "model": "itaz/deepseek-v4-think",
  "provider": {
    "itaz": {
      "options": { "baseURL": "https://llm.ad.itaz.eu/v1", "apiKey": "${MY_LLM_KEY}" },
      "models": { "deepseek-v4-fast": { "limit": { "context": 393216 } } }
    }
  }
}

Same shape as opencode's config, but a separate file. Reading opencode's own config would mean a credential rotation there silently changing what Claude Code authenticates with, and would couple two tools' configs for no reason.

Keys are never stored in the tool

apiKey may be a literal (in a 0600 file), ${ENV_VAR}, or a bare env var NAME — so the secret can live in your environment or a password manager instead of on disk. --init reads the key from stdin when --api-key is omitted, keeping it out of shell history and out of the process table, where --api-key would be visible to every user via ps. --list never prints more than a 10-character prefix.

Discovery order

Order	Source	Supplies
1	environment (ANTHROPIC_BASE_URL, ANTHROPIC_AUTH_TOKEN, ANTHROPIC_MODEL)	anything already set is respected
2	--provider / --model flags	provider, model
3	~/.config/mcpctl/claude-vllm.jsonc	its own config
4	~/.pi/agent	settings.json → default provider/model · models.json → base URL, context windows · auth.json → key
5	~/.prime/agent	same shape

The pi/prime homes remain a convenience fallback so claude-vllm works before you have written a config at all. The base URL and the credential always come from the same source, so one gateway's URL is never paired with another's key.

--init records the context window for every model it can see, not just the active one — otherwise --model something-else silently falls back to Claude Code's assumed 200k.

What it sets, and why each one

Variable	Reason
ANTHROPIC_BASE_URL	The gateway. A stored OpenAI-style …/v1 base has the suffix stripped — Claude Code appends /v1/messages itself, and /v1/v1/messages 404s.
ANTHROPIC_AUTH_TOKEN + ANTHROPIC_API_KEY	The gateway takes either; which one Claude Code sends has changed between releases.
ANTHROPIC_MODEL	The model to drive the session.
ANTHROPIC_SMALL_FAST_MODEL, ANTHROPIC_DEFAULT_HAIKU_MODEL	Background/summarisation calls otherwise ask for a real Haiku the gateway does not serve, and every one 404s.
CLAUDE_CODE_MAX_CONTEXT_TOKENS	Claude Code assumes 200k for models it has no table for. deepseek-v4-* is 393k, so without this it auto-compacts at half capacity. Taken from contextWindow in the provider config.
CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS	Beta headers the gateway does not implement make it reject otherwise-fine requests.
CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC	No telemetry about a non-Anthropic endpoint.

Expected noise

⚠ claude.ai connectors are disabled because ANTHROPIC_API_KEY or another auth
  source is set and takes precedence over your claude.ai login

Unavoidable and harmless: it fires for any non-claude.ai auth, including ANTHROPIC_AUTH_TOKEN alone (verified). It means your claude.ai org connectors are not loaded for this session — which is the point of running against the homelab.

Verified

$ claude-vllm --model deepseek-v4-fast -- -p "Reply with exactly: PARITY-OK"
claude-vllm: itaz · deepseek-v4-fast · https://llm.ad.itaz.eu
PARITY-OK