feat(gate): terse selection prompt (~2s on no-think model) #89

Merged

michal merged 1 commits from feat/gate-terse-selection into main

2026-07-24 20:38:33 +00:00

Author	SHA1	Message	Date
Michal	a432e9e19f	feat(gate): terse selection prompt so a no-think model answers in ~2s Some checks failed CI/CD / lint (pull_request) Successful in 1m3s Details CI/CD / typecheck (pull_request) Successful in 2m10s Details CI/CD / test (pull_request) Successful in 1m21s Details CI/CD / build (pull_request) Successful in 2m14s Details CI/CD / smoke (pull_request) Failing after 3m16s Details CI/CD / publish (pull_request) Has been skipped Details With the fast (no-think) route, gate-selection latency is now proportional to the ANSWER length (a no-think model runs ~10-13 tok/s), not hidden thinking. A verbose "reasoning" field pushed a selection to ~9s (over the 8s budget). Request compact JSON with a <=8-word reasoning → ~2s (vs ~9s prose, or ~1.2s with no reasoning). Validated live against vllm-fast (glm-4.6-reap-fast): 2.0s, valid selection JSON, zero reasoning tokens. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-24 21:38:08 +01:00