feat(cli+docs+smoke): inference-task CLI + GC ticker + smoke + docs (v5 Stage 4)

CLI surface for the durable queue:

- `mcpctl get tasks` — table view (ID, STATUS, POOL, LLM, MODEL,
  STREAM, AGE, WORKER). Aliases `task`, `tasks`, `inference-task`,
  `inference-tasks` all normalize to the canonical plural so URL
  construction works uniformly. RESOURCE_ALIASES + completions
  generator updated.
- `mcpctl chat-llm <name> --async -m <msg>` — enqueue and exit. stdout
  is just the task id (pipeable into `xargs mcpctl get task`); stderr
  carries human-readable status. REPL mode is rejected for --async
  (fire-and-forget doesn't make sense without -m).

GC ticker in mcpd: 5-min interval. Pending tasks past 1 h queue
timeout flip to error with a clear message; terminal tasks past 7 d
retention get deleted. Both queries are index-backed.

Crash fix uncovered by the smoke: when the async route doesn't await
ref.done, a later cancel/error rejected the in-flight Promise as
unhandled and crashed mcpd. The route now attaches a no-op `.catch`
so the legacy `done` semantic still works for sync callers (chat,
direct infer) without taking out the process for async ones. The
EnqueueInferOptions also gained an explicit `ownerId` field so the
async API can stamp the authenticated user on the row instead of
inheriting 'system' from the constructor's resolveOwner — without
this, every GET/DELETE from the original caller would 404 due to
foreign-owner mismatch.

Smoke (tests/smoke/inference-task.smoke.test.ts):

  1. POST /inference-tasks while no worker bound → row=pending.
  2. Bring a registrar online → bindSession drain claims and
     dispatches → worker complete()s → row=completed → GET returns
     the assistant body.
  3. Stop worker, enqueue, DELETE → row=cancelled, persisted.

docs/inference-tasks.md (new): full data model, lifecycle diagram,
async API reference, CLI examples, RBAC table, GC defaults, and the
v5 limitations / v6 roadmap. Cross-linked from virtual-llms.md and
agents.md.

Tests + smoke: mcpd 893/893, mcplocal 723/723, cli 437/437, full
smoke 146/146 (was 144, +2 new task smoke). Live mcpd verified via
manual curl: enqueue → cancel → re-fetch — no crash, owner scoping
returns 404 on foreign ids, GC ticker logs at info when it sweeps.

v5 complete: durable queue (Stage 1) + VirtualLlmService rewire
(Stage 2) + async API & RBAC (Stage 3) + CLI/GC/smoke/docs (Stage 4).

src/mcpd/src/main.ts

@@ -799,6 +799,28 @@ async function main(): Promise<void> {
     }
   }, VIRTUAL_LLM_GC_INTERVAL_MS);
 
+  // v5: InferenceTask GC sweep — pending tasks aged out of the 1 h
+  // queue timeout get flipped to error (so any caller still polling
+  // sees a clean failure instead of an indefinite "pending"); terminal
+  // tasks past 7 d retention get deleted to keep the table from
+  // growing unboundedly. Runs every 5 min — slower than the v-llm
+  // sweep because the queue's TTL granularity is coarser. Both index-
+  // backed queries return empty fast when nothing's expired.
+  const INFERENCE_TASK_GC_INTERVAL_MS = 5 * 60_000;
+  const inferenceTaskGcTimer = setInterval(async () => {
+    try {
+      const { erroredOut, deleted } = await inferenceTaskService.gcSweep({
+        pendingTimeoutMs: 60 * 60_000, // 1 h
+        terminalRetentionMs: 7 * 24 * 60 * 60_000, // 7 d
+      });
+      if (erroredOut > 0 || deleted > 0) {
+        app.log.info(`[inference-task gc] erroredOut=${String(erroredOut)} deleted=${String(deleted)}`);
+      }
+    } catch (err) {
+      app.log.error({ err }, 'Inference task GC sweep failed');
+    }
+  }, INFERENCE_TASK_GC_INTERVAL_MS);
+
   // Health probe runner — periodic MCP probes (like k8s livenessProbe).
   // Without explicit healthCheck.tool, probes send tools/list through
   // McpProxyService so they traverse the exact production call path.
@@ -834,6 +856,7 @@ async function main(): Promise<void> {
     disconnectDb: async () => {
       clearInterval(reconcileTimer);
       clearInterval(virtualLlmGcTimer);
+      clearInterval(inferenceTaskGcTimer);
       healthProbeRunner.stop();
       secretBackendRotatorLoop.stop();
       gitBackup.stop();

src/mcpd/src/routes/inference-tasks.ts

@@ -91,28 +91,25 @@ export function registerInferenceTaskRoutes(
     // through VirtualLlmService.enqueueInferTask with failFast:false so
     // the row stays pending if no worker is currently bound. Caller's
     // HTTP request returns immediately with the task id — they don't
-    // wait on ref.done.
+    // wait on ref.done. ownerId threaded through so the row carries
+    // the authenticated user's id (route layer's foreign-owner 404
+    // depends on it).
     const ref = await deps.virtualLlms.enqueueInferTask(
       llm.name,
       body.request as Parameters<typeof deps.virtualLlms.enqueueInferTask>[1],
       streaming,
-      { failFast: false },
+      { failFast: false, ownerId },
     );
+    // CRITICAL: the legacy `ref.done` promise still wires up a 10-min
+    // waiter that resolves on terminal state. The async API doesn't
+    // await it (caller polls instead), so a `cancelled`/`error` row
+    // would leave an unhandled rejection that Node escalates to
+    // `unhandledRejection` and (with our process settings) crashes
+    // mcpd. Detach with a no-op catch — the async API consumer reads
+    // terminal state via GET /:id, not via this promise.
+    ref.done.catch(() => undefined);
 
-    // Ensure the row carries ownerId from the route's authenticated user.
-    // VirtualLlmService.enqueueInferTask uses its constructor-injected
-    // resolveOwner() callback — for now that defaults to 'system';
-    // we re-fetch the row and patch ownerId so the async API surface
-    // can scope correctly. (A cleaner v6 fix is to thread ownerId
-    // through enqueueInferTask itself; out-of-scope for Stage 3.)
     const created = await deps.tasks.findById(ref.taskId);
-    if (created !== null && created.ownerId !== ownerId) {
-      // Note: this is a separate UPDATE, not part of the enqueue
-      // transaction. Race window is small (the row is brand-new) and
-      // the worst case is a stale 'system' owner — visible only via
-      // direct cross-user list, which still requires `view:tasks`.
-      // Acceptable for v5; v6 plumbs ownerId through enqueueInferTask.
-    }
 
     reply.code(201);
     return {

src/mcpd/src/services/virtual-llm.service.ts

@@ -122,6 +122,15 @@ export interface EnqueueInferOptions {
    * Default: false (durable, queues + waits up to INFER_AWAIT_TIMEOUT_MS).
    */
   failFast?: boolean;
+  /**
+   * v5: caller's user id, threaded directly into the new task row's
+   * `ownerId`. When omitted falls back to the constructor-injected
+   * `resolveOwner()` (default: 'system'). The async API route passes
+   * the authenticated user explicitly so owner-scoped lookups work;
+   * legacy callers that don't care (chat.service, direct infer) leave
+   * it undefined.
+   */
+  ownerId?: string;
 }
 
 export interface IVirtualLlmService {
@@ -419,7 +428,7 @@ export class VirtualLlmService implements IVirtualLlmService {
       tier: llm.tier,
       requestBody: request as unknown as Record<string, unknown>,
       streaming,
-      ownerId: this.resolveOwner(),
+      ownerId: options.ownerId ?? this.resolveOwner(),
     });
 
     // Try to claim + dispatch immediately if a session is up. If not,

src/mcpd/tests/inference-task-routes.test.ts

@@ -176,11 +176,14 @@ describe('Inference Task Routes (v5 Stage 3)', () => {
     expect(body.streaming).toBe(false);
     // Critical: async API must NOT use failFast — that's the entire
     // point of this endpoint over the existing /llms/<name>/infer path.
+    // Owner is also threaded through so the resulting row carries the
+    // authenticated user (otherwise foreign-owner 404 would fire on
+    // every subsequent GET/DELETE).
     expect(virtualLlms.enqueueInferTask).toHaveBeenCalledWith(
       'vllm-local',
       expect.objectContaining({ messages: expect.any(Array) }),
       false,
-      { failFast: false },
+      { failFast: false, ownerId: 'owner-1' },
     );
   });