docs(reliability): fallback needs a real API key (#85)
Some checks failed
Some checks failed
This commit was merged in pull request #85.
This commit is contained in:
@@ -48,4 +48,18 @@ model actually answered**.
|
|||||||
model + upstream status/body (not "no choice").
|
model + upstream status/body (not "no choice").
|
||||||
|
|
||||||
Homelab chain: `vllm-current` (the served vLLM) → an `anthropic-fallback` server
|
Homelab chain: `vllm-current` (the served vLLM) → an `anthropic-fallback` server
|
||||||
Llm as the always-up last resort.
|
Llm as the always-up last resort. The chain is owned declaratively by Pulumi
|
||||||
|
(`kubernetes-deployment/deployments/mcpctl/llm-target.ts`, `fallbacks` arg) so a
|
||||||
|
`pulumi up` can't wipe it — mcpd *replaces* `extraConfig` on update.
|
||||||
|
|
||||||
|
**A real last resort needs an independent cloud credential.** LiteLLM only fronts
|
||||||
|
the single local vLLM, so any local model shares the same GPU-box failure as the
|
||||||
|
primary. `anthropic-fallback` points at `api.anthropic.com` and only serves once
|
||||||
|
mcpd Secret `anthropic-key` holds a genuine **`sk-ant-api03` API key** (from the
|
||||||
|
Anthropic Console). A Claude *subscription* OAuth token (`sk-ant-oat…`, what
|
||||||
|
`claude setup-token` mints) does **not** work: Anthropic gates those to the Claude
|
||||||
|
Code client — probed directly they 404 on older model ids and 429 on current ones.
|
||||||
|
mcpd's anthropic adapter does send OAuth tokens via `Authorization: Bearer` (so
|
||||||
|
auth *passes*), but the gating is server-side and unavoidable. Until a real key is
|
||||||
|
set, the fallback is wired-but-inert: an outage fails over to it and surfaces a
|
||||||
|
clear `anthropic-fallback (model) HTTP 4xx` error rather than improving uptime.
|
||||||
|
|||||||
Reference in New Issue
Block a user