michal/mcpctl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(gate): terse selection prompt (~2s on no-think model) #89

Merged
michal merged 1 commits from feat/gate-terse-selection into main 2026-07-24 20:38:33 +00:00
Conversation 0 Commits 1 Files Changed 1 +5 -1
michal commented 2026-07-24 20:38:09 +00:00
Owner
Copy Link

Gate selection latency is now proportional to answer length on a no-think model. Compact JSON + <=8-word reasoning → ~2s (validated live vs vllm-fast/glm-4.6-reap-fast: 2.0s, valid JSON, 0 reasoning tokens). Completes the fast-selection chain.

Gate selection latency is now proportional to answer length on a no-think model. Compact JSON + <=8-word reasoning → ~2s (validated live vs vllm-fast/glm-4.6-reap-fast: 2.0s, valid JSON, 0 reasoning tokens). Completes the fast-selection chain.
michal added 1 commit 2026-07-24 20:38:09 +00:00
feat(gate): terse selection prompt so a no-think model answers in ~2s
Some checks failed
CI/CD / lint (pull_request) Successful in 1m3s
Details
CI/CD / typecheck (pull_request) Successful in 2m10s
Details
CI/CD / test (pull_request) Successful in 1m21s
Details
CI/CD / build (pull_request) Successful in 2m14s
Details
CI/CD / smoke (pull_request) Failing after 3m16s
Details
CI/CD / publish (pull_request) Has been skipped
Details
a432e9e19f 
With the fast (no-think) route, gate-selection latency is now proportional to the
ANSWER length (a no-think model runs ~10-13 tok/s), not hidden thinking. A verbose
"reasoning" field pushed a selection to ~9s (over the 8s budget). Request compact
JSON with a <=8-word reasoning → ~2s (vs ~9s prose, or ~1.2s with no reasoning).
Validated live against vllm-fast (glm-4.6-reap-fast): 2.0s, valid selection JSON,
zero reasoning tokens.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
michal merged commit 2f9e7be762 into main 2026-07-24 20:38:33 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
michal
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: michal/mcpctl#89
Delete Branch "feat/gate-terse-selection"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?