feat(mcplocal): Anthropic models auto-follow the newest in their family

`claude-opus-4-20250514` was pinned as the heavy provider and had been
returning 404 on every gate ranking and every pagination title. The only
visible symptom was a fallback that looked like an ordinary one -- it surfaced
here because the degradation notice added earlier in this branch finally
printed the reason.

Checked against GET /v1/models: BOTH pins were dead. The fast tier's
claude-haiku-3-5-20241022 is gone too, so that tier had been silently 404ing
as well.

The provider's listModels() asserted "Anthropic doesn't have a models listing
endpoint" and returned four hardcoded dated ids. That endpoint does exist and
answers fine with the OAuth token this deployment uses; the hardcoded list was
simply stale, and a test pinned it in place.

So: `claude-<family>-latest` (or a bare `opus` / `sonnet` / `haiku` / `fable`)
now resolves against the live list. Exact ids pass through untouched, so
pinning still works when someone wants it.

Newest is decided by `created_at`, never by parsing the version out of the id.
That is not incidental: `claude-opus-4-5` sorts ABOVE `claude-opus-5` as a
string, and "4-5" parses as a larger minor than "5". There is a test for
exactly that trap.

Resolution is cached (12h, MCPCTL_ANTHROPIC_MODEL_TTL_MS) so it is not a
per-call network hop, and shared across instances since the model list is
account-wide. When the endpoint is unreachable it falls back to a pinned
known-good id per family and says so on stderr -- the map going stale can then
only cost availability, never correctness.

The constructor default was `claude-sonnet-4-20250514`, also retired; it now
tracks the family too.

Local config: heavy -> claude-opus-latest, fast -> claude-haiku-latest.
Verified live: opus-latest -> claude-opus-5, haiku-latest ->
claude-haiku-4-5-20251001, sonnet-latest -> claude-sonnet-5, and an exact id
passes through.

Also found while there: both anthropic entries were named "anthropic", and the
registry keys by name, so the second silently OVERWROTE the first and one
tier's model was discarded entirely. Renamed to anthropic-fast /
anthropic-heavy so NamedProvider keeps them distinct and the tier split is
real.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GqMidYEGUJG5fxeoTELBu2
This commit is contained in:
2026-08-25 23:55:44 +01:00
parent 14eb2623bc
commit 08b451b7aa
3 changed files with 237 additions and 13 deletions

View File

@@ -6,21 +6,52 @@ export interface AnthropicConfig {
defaultModel?: string;
}
/**
* Families that can be tracked with a `claude-<family>-latest` selector.
*
* The pinned value is a FALLBACK, used only when the models endpoint cannot be
* reached. Normal operation resolves against the live list, so this map going
* stale degrades availability, never correctness.
*/
const FAMILY_FALLBACK: Record<string, string> = {
opus: 'claude-opus-5',
sonnet: 'claude-sonnet-5',
haiku: 'claude-haiku-4-5-20251001',
fable: 'claude-fable-5',
};
/** `claude-opus-latest` / `opus` → `opus`; an exact model id → null. */
function familyOf(model: string): string | null {
const m = /^(?:claude-)?([a-z]+)(?:-latest)?$/.exec(model.trim().toLowerCase());
const family = m?.[1];
if (family !== undefined && family in FAMILY_FALLBACK && /latest|^(opus|sonnet|haiku|fable)$/.test(model.toLowerCase())) {
return family;
}
return null;
}
/** Resolutions are cached this long before the models endpoint is consulted again. */
const MODEL_CACHE_TTL_MS = Number(process.env['MCPCTL_ANTHROPIC_MODEL_TTL_MS']) || 12 * 60 * 60 * 1000;
/**
* Anthropic Claude provider using the Messages API.
*/
export class AnthropicProvider implements LlmProvider {
readonly name = 'anthropic';
/** Shared across instances: the model list is account-wide, not per-provider. */
private static readonly modelCache = new Map<string, { id: string; expiresAt: number }>();
private apiKey: string;
private defaultModel: string;
constructor(config: AnthropicConfig) {
this.apiKey = config.apiKey;
this.defaultModel = config.defaultModel ?? 'claude-sonnet-4-20250514';
// A dated default is a time bomb: claude-sonnet-4-20250514 is retired and
// would 404. Track the family instead.
this.defaultModel = config.defaultModel ?? 'claude-sonnet-latest';
}
async complete(options: CompletionOptions): Promise<CompletionResult> {
const model = options.model ?? this.defaultModel;
const model = await this.resolveModel(options.model ?? this.defaultModel);
// Separate system message from conversation
const systemMessages = options.messages.filter((m) => m.role === 'system');
@@ -49,14 +80,75 @@ export class AnthropicProvider implements LlmProvider {
return parseAnthropicResponse(response);
}
/**
* Turn a `claude-<family>-latest` selector into a concrete model id.
*
* Exact ids pass through untouched, so pinning still works. Selectors are
* resolved against GET /v1/models and cached, because a dated id pinned in
* config rots silently: `claude-opus-4-20250514` sat in this deployment
* returning 404 on every gate ranking and every pagination title, and the
* only visible symptom was a fallback that looked like a normal one.
*
* Newest is decided by `created_at`, never by parsing the version out of the
* id — that is what keeps `claude-opus-4-5` from beating `claude-opus-5`.
*/
async resolveModel(model: string): Promise<string> {
const family = familyOf(model);
if (family === null) return model;
const cached = AnthropicProvider.modelCache.get(family);
if (cached && Date.now() < cached.expiresAt) return cached.id;
try {
const models = await this.fetchModels();
const match = models
.filter((m) => m.id.startsWith(`claude-${family}-`) || m.id === `claude-${family}`)
.sort((a, b) => (a.created_at < b.created_at ? 1 : -1))[0];
if (!match) throw new Error(`no models found for family '${family}'`);
AnthropicProvider.modelCache.set(family, {
id: match.id,
expiresAt: Date.now() + MODEL_CACHE_TTL_MS,
});
if (cached?.id !== match.id) {
process.stderr.write(`[anthropic] ${model} -> ${match.id}\n`);
}
return match.id;
} catch (err) {
// Loud, not silent, and deterministic: an unreachable models endpoint
// must not take the provider down with it.
const fallback = FAMILY_FALLBACK[family]!;
process.stderr.write(
`[anthropic] could not resolve '${model}' (${(err as Error).message}) — `
+ `falling back to ${fallback}\n`,
);
return fallback;
}
}
async listModels(): Promise<string[]> {
// Anthropic doesn't have a models listing endpoint; return known models
return [
'claude-opus-4-20250514',
'claude-sonnet-4-20250514',
'claude-sonnet-4-5-20250514',
'claude-haiku-3-5-20241022',
];
const models = await this.fetchModels();
return models.map((m) => m.id);
}
/** GET /v1/models, newest first. Follows pagination. */
private async fetchModels(): Promise<Array<{ id: string; created_at: string }>> {
const all: Array<{ id: string; created_at: string }> = [];
let after: string | undefined;
for (let page = 0; page < 10; page++) {
const query = new URLSearchParams({ limit: '100' });
if (after !== undefined) query.set('after_id', after);
const body = await this.get(`/v1/models?${query.toString()}`) as {
data?: Array<{ id: string; created_at: string }>;
has_more?: boolean;
last_id?: string;
};
all.push(...(body.data ?? []));
if (body.has_more !== true || body.last_id === undefined) break;
after = body.last_id;
}
return all;
}
async isAvailable(): Promise<boolean> {
@@ -72,6 +164,45 @@ export class AnthropicProvider implements LlmProvider {
}
}
/** GET against the Anthropic API, same auth handling as request(). */
private get(path: string): Promise<unknown> {
return new Promise((resolve, reject) => {
const isOAuth = this.apiKey.startsWith('sk-ant-oat');
const req = https.request({
hostname: 'api.anthropic.com',
port: 443,
path,
method: 'GET',
timeout: 15000,
headers: {
...(isOAuth
? { 'Authorization': `Bearer ${this.apiKey}` }
: { 'x-api-key': this.apiKey }),
'anthropic-version': '2023-06-01',
},
}, (res) => {
const chunks: Buffer[] = [];
res.on('data', (chunk: Buffer) => chunks.push(chunk));
res.on('end', () => {
const raw = Buffer.concat(chunks).toString('utf-8');
const status = res.statusCode ?? 0;
if (status >= 400) {
reject(new Error(`Anthropic HTTP ${String(status)}: ${raw.slice(0, 200)}`));
return;
}
try {
resolve(JSON.parse(raw));
} catch {
reject(new Error('Anthropic response was not valid JSON'));
}
});
});
req.on('timeout', () => { req.destroy(new Error('models request timed out')); });
req.on('error', reject);
req.end();
});
}
private request(body: unknown, signal?: AbortSignal): Promise<unknown> {
return new Promise((resolve, reject) => {
if (signal?.aborted === true) {

View File

@@ -0,0 +1,83 @@
import { describe, it, expect, vi, afterEach } from 'vitest';
import { AnthropicProvider } from '../src/providers/anthropic.js';
/** The real payload shape, abbreviated — newest first, as the API returns it. */
const MODELS = [
{ id: 'claude-opus-5', created_at: '2026-07-24T00:00:00Z' },
{ id: 'claude-sonnet-5', created_at: '2026-06-29T00:00:00Z' },
{ id: 'claude-fable-5', created_at: '2026-06-07T00:00:00Z' },
{ id: 'claude-opus-4-8', created_at: '2026-05-28T00:00:00Z' },
{ id: 'claude-opus-4-5-20251101', created_at: '2025-11-24T00:00:00Z' },
{ id: 'claude-haiku-4-5-20251001', created_at: '2025-10-15T00:00:00Z' },
];
function providerWith(models: typeof MODELS | Error): AnthropicProvider {
const p = new AnthropicProvider({ apiKey: 'sk-ant-api-test' });
// Stub the private transport rather than the network.
(p as unknown as { get: (path: string) => Promise<unknown> }).get = async () => {
if (models instanceof Error) throw models;
return { data: models, has_more: false };
};
return p;
}
afterEach(() => {
// The cache is static — clear it so cases don't leak into each other.
(AnthropicProvider as unknown as { modelCache: Map<string, unknown> }).modelCache.clear();
vi.restoreAllMocks();
});
describe('Anthropic model resolution', () => {
it('resolves a family selector to the newest member', async () => {
await expect(providerWith(MODELS).resolveModel('claude-opus-latest')).resolves.toBe('claude-opus-5');
});
it('accepts the bare family name too', async () => {
await expect(providerWith(MODELS).resolveModel('opus')).resolves.toBe('claude-opus-5');
await expect(providerWith(MODELS).resolveModel('haiku')).resolves.toBe('claude-haiku-4-5-20251001');
});
it('picks by created_at, not by parsing the version', async () => {
// The trap: claude-opus-4-5 sorts ABOVE claude-opus-5 as a string, and
// "4-5" parses as a bigger minor than "5". Only the date is reliable.
const out = await providerWith(MODELS).resolveModel('claude-opus-latest');
expect(out).toBe('claude-opus-5');
expect(out).not.toBe('claude-opus-4-5-20251101');
});
it('leaves an exact model id alone, so pinning still works', async () => {
const p = providerWith(MODELS);
await expect(p.resolveModel('claude-opus-4-8')).resolves.toBe('claude-opus-4-8');
await expect(p.resolveModel('claude-haiku-4-5-20251001')).resolves.toBe('claude-haiku-4-5-20251001');
});
it('does not treat a dated id as a family selector', async () => {
await expect(providerWith(MODELS).resolveModel('claude-opus-4-20250514'))
.resolves.toBe('claude-opus-4-20250514');
});
it('falls back deterministically when the models endpoint is unreachable', async () => {
const err = vi.spyOn(process.stderr, 'write').mockImplementation(() => true);
await expect(providerWith(new Error('network down')).resolveModel('claude-opus-latest'))
.resolves.toBe('claude-opus-5');
// Loud, not silent.
expect(err).toHaveBeenCalledWith(expect.stringContaining('falling back'));
});
it('caches, so resolution is not a per-call network hop', async () => {
const p = providerWith(MODELS);
let calls = 0;
(p as unknown as { get: () => Promise<unknown> }).get = async () => {
calls++;
return { data: MODELS, has_more: false };
};
await p.resolveModel('claude-opus-latest');
await p.resolveModel('claude-opus-latest');
await p.resolveModel('claude-opus-latest');
expect(calls).toBe(1);
});
it('lists real models instead of a hardcoded table', async () => {
await expect(providerWith(MODELS).listModels()).resolves.toContain('claude-opus-5');
});
});

View File

@@ -271,11 +271,21 @@ describe('AnthropicProvider auth headers', () => {
expect(headers['Authorization']).toBeUndefined();
});
it('includes claude-sonnet-4-5 in model list', async () => {
it('lists models from the API rather than a hardcoded table', async () => {
// This used to assert a hardcoded list containing
// claude-opus-4-20250514 and claude-haiku-3-5-20241022 — both since
// retired, and both returning 404 in production while the provider still
// advertised them. The list now comes from GET /v1/models.
const provider = new AnthropicProvider({ apiKey: 'test' });
(provider as unknown as { get: (p: string) => Promise<unknown> }).get = async () => ({
data: [
{ id: 'claude-opus-5', created_at: '2026-07-24T00:00:00Z' },
{ id: 'claude-haiku-4-5-20251001', created_at: '2025-10-15T00:00:00Z' },
],
has_more: false,
});
const models = await provider.listModels();
expect(models).toContain('claude-sonnet-4-5-20250514');
expect(models).toContain('claude-opus-4-20250514');
expect(models).toContain('claude-haiku-3-5-20241022');
expect(models).toEqual(['claude-opus-5', 'claude-haiku-4-5-20251001']);
});
});