mcpctl/docs/claude-vllm.md

# `claude-vllm` — Claude Code on the homelab LLM

## The short version

```bash
claude-vllm                                   # default provider + model
claude-vllm --model deepseek-v4-max
claude-vllm --list                            # what would it use?
claude-vllm --print-env                       # the exports, without running claude
claude-vllm -- -p "summarise this repo"       # anything after -- goes to claude
```

## Why no bridge is needed

The homelab gateway at `llm.ad.itaz.eu` is **LiteLLM in front of vLLM**, and
LiteLLM already serves the Anthropic Messages API:

```console
$ curl -s https://llm.ad.itaz.eu/v1/messages -X POST \
    -H 'x-api-key: <key>' -H 'anthropic-version: 2023-06-01' \
    -d '{"model":"deepseek-v4-fast","max_tokens":32,
         "messages":[{"role":"user","content":"say only: ok"}]}'
{"type":"message","role":"assistant","model":"deepseek-v4-fast",
 "content":[{"type":"text","text":"ok"}],"stop_reason":"end_turn", …}
```

That is the exact protocol Claude Code speaks. So pointing `ANTHROPIC_BASE_URL`
at the gateway is the whole integration — no translation layer, no proxy, no
mcpctl in the request path.

### Why not route it through mcpctl

mcpctl's LLM layer is a **client**, not a server: `mcpd` exposes
`/api/v1/llms` (management) and the adapters in
`src/mcpd/src/services/llm/adapters/` *call out* to Anthropic/OpenAI for gating,
prompt selection and agent chat. Nothing in mcpd or mcplocal serves
`/v1/messages`. A `claude-mcpctl` would therefore mean adding an
Anthropic-shaped passthrough to mcpd that re-wraps what LiteLLM already does —
new surface, new failure mode, no new capability.

It would only be worth it if you wanted mcpctl *in* the LLM path for its own
sake: per-project gating of model calls, audit of prompts, or budget
enforcement. Those are real features, but they are a passthrough endpoint in
mcpd, not a wrapper script.

## Its own config

```bash
claude-vllm --init                       # prompts for the key, writes 0600
claude-vllm --init --api-key '${MY_KEY}' # keep the secret in the environment instead
claude-vllm --init --base-url https://other-gateway/v1
```

Written to `$XDG_CONFIG_HOME/mcpctl/claude-vllm.jsonc` (`~/.config/…`), shaped
like `opencode.jsonc` — a `provider` map with `options.baseURL` /
`options.apiKey` and a `models` map:

```jsonc
{
  "model": "itaz/deepseek-v4-think",
  "provider": {
    "itaz": {
      "options": { "baseURL": "https://llm.ad.itaz.eu/v1", "apiKey": "${MY_LLM_KEY}" },
      "models": { "deepseek-v4-fast": { "limit": { "context": 393216 } } }
    }
  }
}
```

**Same shape as opencode's config, but a separate file.** Reading opencode's own
config would mean a credential rotation there silently changing what Claude Code
authenticates with, and would couple two tools' configs for no reason.

### Keys are never stored in the tool

`apiKey` may be a literal (in a 0600 file), `${ENV_VAR}`, or a bare env var
NAME — so the secret can live in your environment or a password manager instead
of on disk. `--init` reads the key from **stdin** when `--api-key` is omitted,
keeping it out of shell history and out of the process table, where `--api-key`
would be visible to every user via `ps`. `--list` never prints more than a
10-character prefix.

### Discovery order

| Order | Source | Supplies |
|-------|--------|----------|
| 1 | environment (`ANTHROPIC_BASE_URL`, `ANTHROPIC_AUTH_TOKEN`, `ANTHROPIC_MODEL`) | anything already set is respected |
| 2 | `--provider` / `--model` flags | provider, model |
| 3 | `~/.config/mcpctl/claude-vllm.jsonc` | its own config |
| 4 | `~/.pi/agent` | `settings.json` → default provider/model · `models.json` → base URL, context windows · `auth.json` → key |
| 5 | `~/.prime/agent` | same shape |

The pi/prime homes remain a convenience fallback so `claude-vllm` works before
you have written a config at all. The base URL and the credential always come
from the *same* source, so one gateway's URL is never paired with another's key.

`--init` records the context window for **every** model it can see, not just the
active one — otherwise `--model something-else` silently falls back to Claude
Code's assumed 200k.

## What it sets, and why each one

| Variable | Reason |
|----------|--------|
| `ANTHROPIC_BASE_URL` | The gateway. A stored OpenAI-style `…/v1` base has the suffix stripped — Claude Code appends `/v1/messages` itself, and `/v1/v1/messages` 404s. |
| `ANTHROPIC_AUTH_TOKEN` + `ANTHROPIC_API_KEY` | The gateway takes either; which one Claude Code sends has changed between releases. |
| `ANTHROPIC_MODEL` | The model to drive the session. |
| `ANTHROPIC_SMALL_FAST_MODEL`, `ANTHROPIC_DEFAULT_HAIKU_MODEL` | Background/summarisation calls otherwise ask for a real Haiku the gateway does not serve, and every one 404s. |
| `CLAUDE_CODE_MAX_CONTEXT_TOKENS` | Claude Code assumes 200k for models it has no table for. `deepseek-v4-*` is 393k, so without this it auto-compacts at half capacity. Taken from `contextWindow` in the provider config. |
| `CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS` | Beta headers the gateway does not implement make it reject otherwise-fine requests. |
| `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC` | No telemetry about a non-Anthropic endpoint. |

## Expected noise

```
⚠ claude.ai connectors are disabled because ANTHROPIC_API_KEY or another auth
  source is set and takes precedence over your claude.ai login
```

Unavoidable and harmless: it fires for *any* non-claude.ai auth, including
`ANTHROPIC_AUTH_TOKEN` alone (verified). It means your claude.ai org connectors
are not loaded for this session — which is the point of running against the
homelab.

## Verified

```console
$ claude-vllm --model deepseek-v4-fast -- -p "Reply with exactly: PARITY-OK"
claude-vllm: itaz · deepseek-v4-fast · https://llm.ad.itaz.eu
PARITY-OK
```