feat(claude): claude-vllm — run Claude Code against the homelab LLM gateway
The gateway at llm.ad.itaz.eu is LiteLLM in front of vLLM, and LiteLLM already
serves the Anthropic Messages API on /v1/messages — verified with a real
completion. So Claude Code needs no bridge: pointing ANTHROPIC_BASE_URL at it is
the whole integration.
`claude-vllm` exists only to stop you pasting four exports each time. It reuses
what another agent is already configured with — ~/.pi/agent, ~/.prime/agent, then
opencode's config, first hit wins — taking the base URL and the credential from
the same source so one gateway's URL is never paired with another's key.
Beyond the obvious ANTHROPIC_* vars it sets two that are easy to miss:
- ANTHROPIC_SMALL_FAST_MODEL / ANTHROPIC_DEFAULT_HAIKU_MODEL, or the
background and summarisation calls ask the gateway for a real Haiku it does
not serve and every one 404s;
- CLAUDE_CODE_MAX_CONTEXT_TOKENS from the provider's declared contextWindow,
because Claude Code assumes 200k for models it has no table for — and
deepseek-v4-* is 393k, so it would auto-compact at half capacity.
On `claude-mcpctl`: mcpctl's LLM layer is a client, not a server. mcpd serves
/api/v1/llms (management) and its adapters call out to providers for gating,
prompt selection and agent chat; nothing serves /v1/messages. Routing Claude
through mcpctl would mean adding an Anthropic-shaped passthrough that re-wraps
LiteLLM — worth doing only if mcpctl in the LLM path buys something of its own
(per-project gating of model calls, prompt audit, budgets), which is a mcpd
endpoint rather than a wrapper script.
Verified: `claude-vllm --model deepseek-v4-fast -- -p "..."` completes against
deepseek on the homelab, with the unknown-model context warning gone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BVwuCjuMoA13gmzYEfcrNP
This commit is contained in:
19
README.md
19
README.md
@@ -237,6 +237,25 @@ Re-sync skills on their own with
|
||||
|
||||
See [docs/opencode-extension.md](docs/opencode-extension.md) for full details.
|
||||
|
||||
### Run Claude Code on the homelab LLM
|
||||
|
||||
The homelab gateway (LiteLLM in front of vLLM) already serves the Anthropic
|
||||
Messages API, so Claude Code can talk to it directly — no bridge:
|
||||
|
||||
```bash
|
||||
claude-vllm # reuse the provider/model/key pi or prime-agent already uses
|
||||
claude-vllm --model deepseek-v4-max
|
||||
claude-vllm --list # show what it would use
|
||||
claude-vllm -- -p "summarise this repo" # args after -- go to claude
|
||||
```
|
||||
|
||||
It discovers the endpoint, credential, model and context window from
|
||||
`~/.pi/agent`, `~/.prime/agent` or opencode's config (first hit wins) and sets
|
||||
the `ANTHROPIC_*` environment Claude Code needs.
|
||||
|
||||
See [docs/claude-vllm.md](docs/claude-vllm.md), including why routing this
|
||||
through mcpctl would add surface without adding capability.
|
||||
|
||||
## Declarative Configuration
|
||||
|
||||
Everything can be defined in YAML and applied with `mcpctl apply`:
|
||||
|
||||
Reference in New Issue
Block a user