diff --git a/docs/agentic-teams.md b/docs/agentic-teams.md new file mode 100644 index 0000000..6e65072 --- /dev/null +++ b/docs/agentic-teams.md @@ -0,0 +1,782 @@ +# Agentic Teams — Unified Design Document + +> **Status:** Consolidated design record — one authoritative document unifying +> every round of discussion. Supersedes the earlier split files +> (`agentic-teams-review.md`, `vision-fidelity.md`, `epic-task-artifact-model.md`, +> `session-summary-vision-fidelity.md`), which have been folded into this single +> living doc. +> +> This is **feature- and decision-focused**, not code. Where a technical shape +> is mentioned it is indicative; the aim is to record *what* we agreed, *why*, +> and *which decisions are still open*. +> +> The one paragraph: mcpctl should orchestrate a software-team-shaped workflow +> where human developers and autonomous agents are **peers in the same project**, +> with mcpctl as the orchestration, state, and "team lead" layer. Agents work +> real code in isolated pods, verify against a living vision, and escalate +> decisions to a single human supervisor. + +--- + +## Table of contents + +1. The vision & motivation +2. Core design principles +3. The two-level verification model (correctness + vision) +4. The person-model: agents as first-class "people" +5. The Epic / epictask / task data model +6. The task lifecycle & QA/Security/SRE stages +7. Repositories & per-session pods +8. The confidence signal +9. Escalation & the decision dashboard +10. `mcpctl exec` — the human in the pod, live +11. Artifacts & context management; script artifacts, agent-built tools & the + baseline scorecard +12. Delegation strategy — what NOT to rebuild +13. The whole system, drawn together +14. Cross-cutting concerns & hard problems +15. Consolidated open questions +16. Suggested MVP path + +--- + +## 1. The vision & motivation + +### 1a. The agentic-teams vision (rounds 1–2) + +Instead of a stateless chatbot, each **project** gains an **agentic flow**: +first-class **Tasks** and **Repositories** entities, multiple **agent personas** +(QA, security, engineers) that work real code in **ephemeral per-session pods**, +a **human-in-the-loop** escalation + approval gate, and — critically — +**persistent, per-persona memory** so agents learn and get cheaper/smarter per +task over a project's lifetime. The goal is to **replicate how the human team +works today**: standups, joint debugging/architecting sessions, autonomous solo +work when confident, and a single human supervisor clearing a queue of +escalated decisions. + +Origin (from the original idea): a flow of **Plan → Refine Plan → Draw Roadmap → +Discuss Tasks from the first roadmap position**, i.e. how a human team decomposes +and executes work. Note: we decided **against** hardcoding this as a rigid state +machine (see §5); it is the motivating scenario, not a fixed pipeline. + +### 1b. The vision-fidelity problem (round 3) + +A distinct, compounding motivation observed in real use: + +> 1. you had a grand vision +> 2. we split the grand vision into tasks +> 3. we completed all the tasks +> 4. we are left with a stick with wheels instead of a car + +Every task completed, yet no car. The diagnosis: **this is not an execution +problem — it's decomposition + verification.** Root causes: + +1. **Decomposition is lossy** — a multidimensional vision compresses into flat + imperatives; the *why* and *what-good-looks-like* get dropped. +2. **Tasks are graded mechanically** — "compiles / lint / renders," never "does + this serve the vision / is this designed / does this feel finished." +3. **Integration is nobody's job** — a car is parts *working together*, but task + systems execute a list. +4. **No feedback loop mid-flight** — the agent assumes task 3 was fine and never + re-checks against the original vision. +5. **Design/product review is a phase, not a gate** — applied at the end, when + the wrong shape is already baked. + +**Research synthesis that shapes the design:** vertical slices / walking +skeleton (decompose into user-visible `mini-wholes`, not layers — always have *a* +small car, never a pile of parts); a **two-tier Definition of Done** +(mechanical + systemic); review-gate HITL over per-tool approval; +spec-driven + **media-native** verification (judge an artifact in its native +medium — render/run, not read code); and the **critic-agent pattern** (a +non-executing persona that reviews against the vision catches drift the working +agent's confirmation bias misses). + +--- + +## 2. Core design principles (agreed) + +1. **Don't build everything from scratch.** If something is already solved well + (permissions, worktree mechanics, multi-turn agent execution, streaming), + delegate to it. Avoid rabbit holes. Explicit top priority. +2. **Agents are like people.** A persistent identity, personal memory, and the + ability to be *invited* to zero, one, or many projects. Not welded to a + single project or a single brain. +3. **Context is scoped, globally + per-project.** One global personal memory plus + *separate per-project* knowledge, so an agent doesn't cross-contaminate one + project's conventions or secrets into another. +4. **Isolation without shared live workspaces.** Agents do **not** share a live + filesystem with each other. Handoffs happen via git (source + verdicts) and an + immutable **build-artifact store** (terminology per §11). A single human *can* + share an environment with a single agent (via `exec`). +5. **Efficiency must be evidence-backed, and evidence must be honest.** Memory, + confidence, and verification scoping rely on retrievable evidence; any "still + valid" claim must be invalidated when the covered region changes — otherwise + memory becomes *false confidence*. +6. **The human is a peer and a supervisor.** A single queue of decisions; + fine-grained per-tool permission approval is delegated to the runtime; mcpctl + owns coarse-grained gates. +7. **Async, artifact-driven, human-as-fuse** (the through-line). Default flow is + asynchronous over frozen artifacts + a decision queue + disposable pods; + `exec` is the deliberate opt-in exception for live joint work. +8. **Vision is a living, versioned artifact** (round 3). There is an + authoritative Epic/VISION that tasks reference by version, so drift is + *detectable* and *auditable*. + +--- + +## 3. The two-level verification model + +This is the mental model that unifies rounds 1–3. There are **two distinct +levels of verification**, addressing the two distinct failure classes: + +| Level | What it checks | Who runs it | When | Native medium | +|---|---|---|---|---| +| **Tier 1 — Correctness/Security/Ops** | Does it build, run, pass tests, stay secure, and is the **deployment-as-code correct for this project's real infra** | Developer + QA + Security + SRE/Release personas | **Inside the task's own lifecycle** (stages before Done) | code / diff / tests / deployment manifests | +| **Tier 2 — Vision Fidelity / Design** | Does the assembled result match the vision? Is it designed, does it feel finished, does it integrate | **Design/Product critic** (review-only, non-executing) | **At the Epic level**, after tasks reach Done | render / run / walkthrough (media-native) | + +**These are different levels working at different points — they do not +replace each other.** QA/Security/SRE are correctness gates on the *task*; the +critic is a vision gate on the *Epic/whole*. Both feed the human accept gate. + +--- + +## 4. The person-model: agents as first-class "people" + +### Decisions + +- An agent exists **independently of any project** — a persistent entity with a + name, description, system prompt, and its own accumulated memory. +- An agent can **join zero, one, or many projects**, declared with + `mcpctl apply -f agent.yaml`. Intuitive shape: + + ```yaml + agents: + - name: agent001 + description: "Full-stack engineer persona" + systemPrompt: "…" + llmRef: agent001-brain # optional default/home brain + projects: [sre, payments] # many-to-many membership + ``` + +- This generalises today's model (Agent pinned to one `Llm` + one optional + `projectId`) into a **many-to-many membership** (an `AgentProject` junction, + analogous to the existing `ProjectServer` junction). The junction is also where + per-project *role / scope* (and RBAC) would hang. + +### Two lenses on the same person (both modes work) + +- `mcpctl chat agent001` — the person **on its own**: `agent.systemPrompt` + + **global personal memory**. General conversation, no project attached. +- `mcpctl chat agent001 --project sre` — the **same person, viewing project sre**: + systemPrompt **+ global personal memory (always present — it's who they are)** + **+ project-sre knowledge** (topology, coverage ledger, project Prompts) + **+ project-sre tools** (attached MCP servers as live tools) + **+ threads scoped to {person, sre}**. + +Key subtlety: **global personal memory is present in BOTH modes** — `--project` +adds a context layer, it does not create a different agent. **Threads** are +scoped to `(agent, optional project)` — same person, separate "meeting rooms." + +### Memory: global generalist + per-project context + +- **Global personal memory** — transferred expertise (methods, worldview). Owned + by the identity; travels with it. +- **Per-project knowledge** — project topology, coverage ledgers, "auth lives + here," **and operational/deployment model** (target infra, methods, tooling, + network policies, known gotchas — the SRE persona's bread and butter, §6). + Keyed to the project, retrieved in the right context only. +- **Both owned by the same identity, separately stored/scoped**, so one project's + quirks don't pollute another's judgment — like a consultant with general skills + plus per-client context. + +### The motivating example: the hotfix + +If an agent just did a **full security review of the app**, but a **tiny hotfix** +comes in, there is **no need to recheck everything — only what changed**. The +agent's own past verification of unchanged regions (pinned to a commit, still +valid) lets it **delta-check** rather than full-rescan. This is the economic +argument for memory + coverage-ledger, and it only holds if unchanged-after- +verification coverage can be *trusted* — hence non-negotiable invalidation (§14). + +### The "brain" — open knot + +Who owns the brain (which Llm powers the agent) is the one real open question. +Lean: **home brain for bare chat; the project's `llmProvider` in `--project` +mode** (portability, matches project-chat). Alternative: agent keeps its own +brain everywhere (strict persona consistency, less portable). **[OPEN — see §15]** + +--- + +## 5. The Epic / epictask / task data model + +The vision-fidelity fix needs an authoritative, versioned source of truth, so we +introduce a **vision layer** above the task layer (round 3, Jira-style +nomenclature). The through-line (async, human-as-fuse) is preserved. + +``` +Epic (the vision — its own resource) + └── epictask — a roadmap node; tasks hang off it; chains to other epictasks + └── task — the unit of work / execution (own lifecycle §6) +``` + +### Epic = the vision + +- The **back-and-forth before coding starts** *is* the vision; the Epic captures + its conclusion. Project-scoped. +- Holds the **agreed "done" criteria** (checkable outcomes + intent/quality + attributes) written at vision time, **scope-change history** (revisions, so + creep/drift is visible), and **accepted-tradeoff decisions**. +- **An Epic is "done" ONLY when:** 100% of the vision is delivered, **OR** the + user explicitly accepted tradeoffs (recorded so done is honest + auditable). + +### epictask + +- A **roadmap node**. Tasks link to an epictask; **epictasks chain to each other** + (dependencies) → visualizing all epictasks yields a **spider-web roadmap** + (dependency graph, not a flat sequence) — the planning/navigation layer. +- Lean: epictasks carry their own (decomposed) acceptance criteria so the web + shows per-node completeness. **[OPEN — §15]** + +### task + +- The unit of work, **owned by an epictask**, with the standard lifecycle (§6). +- Because tasks link to epictasks → Epic, every task can resolve **"what am I + serving and what does done-for-this look like"** by walking up the chain. + **Tier-2 (vision) DoD is reachable from every task.** + +### Reopen vs new-task vs exec (the Epic-comparison mechanism) + +After agents report done, the delivered whole is compared against the Epic: + +| Gap cause | Outcome | +|---|---| +| A task **claimed** a piece but didn't deliver it right | **Reopen** the task (directional comment back) | +| A piece was **never decomposed** into a task (breakdown gap) | **New task** under the relevant epictask | +| Hard problem / quality gap / "we couldn't fix X" | **`exec` joint session** (optional reopen/new task after) | + +The loop repeats until the Epic reaches 100%-delivered or delivered-with- +accepted-tradeoffs. **This comparison moment is the principled trigger for +`exec`** (§10). + +### Flow is NOT a rigid state machine + +Decided: do **not** hardcode Plan→Refine→Roadmap→Work as a pipeline. Tasks have +state; agents and the human move between **collaboration modes** based on +**confidence** and **collaboration need**. A rigid pipeline would force every +task (including a one-line docs typo) through the same heavyweight flow. The flow +is a set of *modes*, not a fixed sequence. + +### Collaboration modes (the "team dynamics") + +- **Standup** — on-demand, artifact-driven introspection: `mcpctl standup + --project sre` pulls each agent's per-task progress/notes since last and + summarizes; each agent's requested decisions surface as dashboard items. Not a + synchronous ritual. +- **Joint debugging / architecting** — the human works *with* an agent on a + challenge (via `exec`, §10). +- **Present architectural decisions** — agents surface decisions (→ dashboard), + not just completed work. +- **Ask the human for help** — distinguished from hot escalation: a collaboration + request, a mode of its own. **[OPEN — §15]** +- **Work alone when confident** — driven by the confidence signal (§8). + +--- + +## 6. The task lifecycle & QA/Security/SRE stages + +This refines the earlier `SelfCheck` stage into a richer set of **correctness +gates that run INSIDE the task's own loop, before "Done."** QA/Security/SRE work +with the developer *during* the task, as stages — feedback bounces the task back +to InProgress until it passes: + +``` +InProgress (developer) + → Build / Run (compile, run, smoke) + → Self-Testing (developer's own tests) + → QA Testing (QA persona checks; feedback → back to InProgress) + → Security Review (security persona checks; feedback → back to InProgress) + → SRE / Release Review (is the deployment-as-code correct for our real + infra — right target, right method, right tooling; + e.g. Kubernetes vs Portainer/EC2/local; + feedback → back to InProgress) + → Done (leaves developer's loop) + → AwaitingReview (human accept gate — your accept/reject + comment) + accept → Merged (PR merged via provider API) + reject → InProgress (you wrote a comment → developer reworks) + → Merged + post-merge failure (webhook) → Reopened (same agent; revert as fast net) +``` + +- **QA/Security/SRE are stages, not post-submit reviewers** — decided explicitly. + QA (quality) and Security (security) are Tier-1 correctness/security gates. An + **SRE/Release review stage** sits alongside them. It reviews the + **deployment-as-code** (IaC, release manifests, container/Docker config) to + ensure it matches the project's **actual deployment model and infrastructure** + — the right target (e.g. our Kubernetes cluster, not an EC2 VM or Portainer or + a local build), the right method (run in Docker, not build locally), and the + right tooling. Its feedback is directional ("no, this should go to Kubernetes, + not Portainer") and bounces the task back to InProgress like QA/Security. It + reviews the deployment *definition*; it is **not** the act of deploying. + The **Design/Product critic** (Tier-2, §3) is a different level operating at the + Epic, *not* one of these stages. + + **Why SRE must know the project's operational gotchas.** A large part of SRE + review is pre-empting failures whose root cause is an *implicit operational + rule* the developer LLM doesn't know. Example (from the user): the cluster has + **network policies**, so any new workload needs **egress and ingress rules**. + A developer LLM watching a connection fail will naturally go "cut a hole in + the floor to check the pipes" — deep-diving the network debugging path — when + the real cause is a **missing egress policy** silently blocking traffic at the + policy layer. The SRE persona, knowing the project's network-policy model, can + flag the missing egress/ingress rule in review and save the developer from the + wrong debugging path and from shipping something that fails obscurely at + runtime. This is why the SRE persona must carry rich **per-project infra + knowledge** (§4): deployment topology, targets, methods, tooling, policies, + and known gotchas. +- **PR linkage:** when a task is done, the agent **pushes and opens a PR**; the + Task stores the link (branch, commit, PR URL). The human should **not have to + go to git** — accept/reject happens inside mcpctl. +- **Merge** is a git-provider API call (e.g., Gitea), gated by CI + the human's + accept. It is **not** mcpctl executing git itself. + +### The human approval gate (decided — important) + +**NOT** Claude's per-tool manual/auto/bypass tri-state. It's a **task-level +acceptance gate**: a yes/no + comment on the *resolved outcome*, which fires the +merge. Once the Tier-1 stages pass, the task enters `AwaitingReview`; the agent +acts autonomously up to that point; the human's whole role is **one decisive +accept/reject + comment**. So **we do not rebuild a permission engine** — mcpctl +enforces coarse gates (who may transition `AwaitingReview→Merged/InProgress`, +what RBAC a session needs); the agent runtime handles fine-grained per-tool +approval. + +### Pre-submission verification — risk-scaled, not rigid + +Decided **against** a hardcoded pipeline. Verification effort **scales with task +risk** and the personas decide the level: +- **Trivial** (docs, formatting) → a light self-check may suffice. +- **Significant/risky** (auth, secrets, risk-relevant paths) → QA, Security + and/or SRE stages are engaged. +- Driven by the **confidence signal** (§8). + +--- + +## 7. Repositories & per-session pods + +### Repositories entity + +A **Repository** attaches git repos to a project and is where Tasks happen. It is +a metadata resource (which repos a project owns). **[Note: this entity was +defined in rounds 1–2; carry it forward into the Epic model as the execution +substrate for epictasks.]** + +### Per-session pods + +- **Each agent session gets its own ephemeral pod/container** (via the K8s + orchestrator mcpctl already has). The pod **is** the workspace. +- An **init container pulls all repos** the project expects → the session is + **project-aware**: right repos checked out, project Prompts, project MCP tools, + scoped secrets. +- No central worktree manager: the pod *is* the worktree (checked out to the task + branch). "Multiple agents on one repo" collapses into each pod checking out its + own branch, working, pushing. Pod dies at session end. +- **Agents do NOT share a live filesystem with each other** — buys independence + (fresh eyes) + provenance. + +### Handing work between personas — git is a feature, not a tax + +- The handoff is **immutable information about a commit** (branch + diff, base, + evidence, verdict) — not a shared live workspace. +- **Git is the audit trail + provenance:** a verdict pinned to a commit + ("QA verified 3f2ab against current main") is reproducible forever — exactly + the "records of similar work" the confidence signal needs. +- **The pod boundary buys true independence** (fresh eyes, no inherited + assumptions). +- **Reviewing a diff is closer to real review** than a shared live tree. +- One deliberate addition: an **immutable *build-artifact store*** keyed by + commit/branch for binaries/build outputs git shouldn't carry (see §11 for the + naming collision). + +### Verification session shape — cheap default, heavy only when risk-gated + +- **Lightweight review session (default):** QA/security runs a session against + the branch, ideally on a **warm base image (deps pre-installed)**. Reads the + diff, runs targeted checks, writes a verdict. No push. Cheap. +- **Heavy verification session (risk-gated only):** full clean pod checks out the + merged-to-current-main branch and runs full CI — the "verify against current + main" gate. Only for high-risk tasks (security paths, migrations, + cross-contract). + +So "separate pods per persona" ≠ "N× full CI every task." + +--- + +## 8. The confidence signal — evidence-anchored, not vibes + +This is what makes "autonomous when confident, escalate when not" *trustworthy*. +Protocol (on a **frozen task artifact**, not live chat): + +1. Frozen task artifact with all info attached (description, proposed resolution, + evidence). +2. **Retrieve K similar past tasks** (semantic similarity, weighted by recency) — + e.g. "we deployed to k8s yesterday" → similar deploys; "update this secret in + vault" → the exact same class. +3. Each attached **with description + resolution + effect** (what *actually* + happened — succeeded / follow-up / revert). +4. Ask the LLM: **"Given these K similar tasks and their outcomes, how confident + are you that THIS task will resolve without issues?"** citing which precedents. + +Refinements: +- **Any evidence can be attached, not just tasks** (docs, PRs, incident records, + QA coverage). +- **Outcome-awareness is what makes it honest** — the *effect* is load-bearing. +- **Staleness matters**: weight recent precedent higher (ties to ledger + invalidation). +- **Failures are bounded and auditable**: the decision cites its evidence, so you + can see *why* it was confident and improve retrieval. +- **Vision-fidelity integration (round 3):** confidence should ALSO score whether + the result serves the vision, not just "will it merge cleanly." The + critic/Epic-comparison (§5) is the Tier-2 half of the same honesty. + +### Thresholds drive autonomy + +``` +confidence(task, evidence[], threshold): + high → proceed solo, light verification + mid → proceed but route QA/security to delta-verify + low → ESCALATE to the human (packaged question) +``` + +### Outcome shaping (refinement) + +Make the question bidirectional/risk-aware: not just "will it go smoothly?" but +"**if it fails, where and how severe, given those precedents?**" — turning +confidence into preemptive review. + +--- + +## 9. Escalation & the decision dashboard (HITL) + +- **When confidence is low, the agent escalates a question to the user** — as a + **packaged, first-class escalation item** with full context: task, project, + proposed resolution, evidence/precedents, rationale, what it needs. +- **A dashboard** surfaces **all open questions from all agents across all + projects**, with context about each task/project. The user answers one by one — + **triaging a queue**, not being pulled into each agent's context. One human + supervises N agents across M projects from a single inbox. +- Each question is a **frozen, evidence-attached artifact** → auditable, + resumable, comparable. Actions: accept / reject-with-comment / direct / + route-to-another-agent. +- Confidence question and escalation are the **same artifact** — a low-confidence + agent packages the escalation rather than stopping. + +--- + +## 10. `mcpctl exec` — the human in the pod, live + +`mcpctl exec --project NAME AGENTNAME -- bash` must be a **first-class citizen**. +The human/agent relationship is **hybrid**: + +1. **Async, artifact-based** (confidence/escalation): most decisions vs frozen + artifacts, answered from the dashboard. +2. **Live, interactive** (`mcpctl exec … -- bash`): joint debugging / architecting + / real-time hands-on — drop into the agent's session as a shell (like + `kubectl exec` / `docker exec`). + +### Semantic (recommended) + +"Exec into a pod" should mean **into the working environment (repo/workspace)**, +NOT **into the agent's reasoning loop**: +- **A — working environment:** cloned repo, branches, tools, deps; inspect, run + commands, maybe manually fix. The agent is a separate process you can still talk + to. Robust, supports joint debugging. **Recommended default.** +- **B — attach to the agent's live session/internals:** invasive and fragile. NOT + recommended. + +Because `exec` lets a **single human share an environment with a single agent**, +it's a safe, deliberate coupling — not the multi-agent-shared-filesystem +anti-pattern. + +### The principled trigger (from §5) + +The **Epic-comparison moment** is what gives `exec` its concrete, principled +trigger: when a quality/design gap or a hard "we couldn't fix X" arises, that's +the natural time to `exec` and work it together, rather than filing more async +work or accepting a reduction. **[Note: reconciles the round-2 async decision with +this deliberate exec carve-out — name it so it doesn't conflict with LE-4.]** + +--- + +## 11. Artifacts & context management — and the "artifact" term collision + +### The collision, named + +In the **agentic-teams rounds**, an **"artifact store"** is a *storage* +mechanism: durable, server-side storage keyed by commit/branch for build outputs +and binaries git shouldn't carry. In the **prime-agent / Epic rounds**, an +**"artifact"** is a *context* mechanism: a structured object in the LLM context +(a PR, file, log) with an on-demand `expand`, never auto-injected, with an +eviction priority (user > agent > artifact-context). **These are two different +axes sharing one noun** — one is "where durable files live," the other is "what +gets lazily loaded into / flushed from the LLM's window." They must not be +conflated by reader or implementer. + +### Proposed terminology (to resolve the collision) + +- **Build-artifact store** — the durable storage for binaries/build outputs + (Wave-A meaning). Server-side, mcpctl-owned, keyed by commit/task. +- **Context-artifact** — a lazily-loaded, expandable object in the LLM context, + with eviction priority (Wave-B meaning). + +### Context-artifact philosophy (prime-agent) + +- Artifacts are **objects, not plain text** — `files|test`, logs, **PR & commits** + as structured objects with an `expand` operation, not dumped into context. +- **"Cat" → variable/artifact** — a named, retrievable reference, **not + auto-injected**. +- **Rule: context does NOT auto-populate from artifact content.** +- **Eviction / flush priority (recoverability-driven):** user's words (highest) > + agent's words > artifact-context (evict first, because artifact content is + durably stored and recoverable via `expand`). **Eviction follows recoverability.** +- Lean: an `expand`ed artifact that has been pulled in returns to + stored-only after the turn ends (safe to evict first). **[OPEN — §15]** + +### Script artifacts & agent-built tools + +Beyond `files|test` and logs, add a **`script` artifact kind** — a named, +reusable piece of executable code (e.g. "merging a PR", "deploying to the +cluster"). A script is a **tool-as-a-script**: instead of the agent re-figuring +out a recurring procedure every time ("how do we deploy?"), it encodes that +procedure once as a reusable, `expand`able artifact. + +**Scoping (nested):** a script artifact can live at **three scopes**: +- **Task-scoped** — specific to one task's work. +- **Project-scoped** — shared by everyone working in that project (e.g. the + project's canonical "deploy" or "merge PR" script). +- **Agent-scoped** — carried by a persona between projects (e.g. a security + code reviewer keeps its review tooling inside the *agent*, so it moves with + the agent to any project it joins). + +**Why agents should build their own tools (analogy to prime-agent / skills):** +like a skill but *as code*. A script is (a) a durable procedure captured once +instead of re-derived; (b) **context-saving** — because it's code, the full +procedure is not re-loaded into the prompt; it's an `expand`able/executable +artifact, so the agent invokes it rather than re-deriving it (ties to the +eviction/`expand` model above); and (c) **learned-and-reused** — once built, it +persists in the relevant scope and gets cheaper to re-use over time (mirrors +the coverage-ledger / knowledge-accumulation logic from §4/§14). + +**Incentive to build tools.** There should be a genuine motivation for an agent +to invest in tooling rather than doing things ad hoc each time. Candidates +(decide): +- building a reusable script earns the agent (or persona) **credit / proven + track-record** that raises future confidence scores (§8); +- reusing an existing script is **preferred / discounted** over ad-hoc + re-derivation; and +- a **scorecard (§11b)** can explicitly reward tool reuse / authorship. + +### 11b. The scorecard & default "basics-rules" governance + +Proposal: a **scorecard** mechanism — a "basics rules agent" that checks a +**provisioned default list of baseline rules** we want every agent to follow. +These defaults ship with the system (provisioned, not invented ad hoc). Example: +- **"Always make unit tests for everything you make."** + +The scorecard agent reviews work/productivity against these baseline rules and +scores it, surfacing violations/omissions (e.g. "missing unit tests") as +escalation/feedback. It is distinct from the Tier-1 QA/Security/SRE stages and +the Tier-2 critic — it's a **baseline-behaviour / standards gate**: a checklist +of minimum expectations rather than a judgement of correctness or vision. + +Open design points **[see §15]**: which rules are provisioned as defaults (and +where they live — inherited from the project? a global baseline?), how the +score influences autonomy/confidence vs. just reporting, and whether the +scorecard is a persona ("basics rules agent") or a passive framework the +running agents consult. + +--- + +## 12. Delegation strategy — what NOT to rebuild + +### mcpctl owns (the differentiator) + +- Task + Repo + Epic/epictask entities, task graph / roadmap. +- Project/repo awareness; per-session pod provisioning + repo-clone init. +- Session orchestration (spawn/kill, attach prompts/secrets). +- Coarse-grained RBAC → runtime binding; task lifecycle, acceptance gate, + escalation packaging, dashboard. +- Memory / confidence / coverage + vision (Epic) layer. +- **Script-artifact registry + baseline scorecard** (§11/§11b). + +### Delegate (the table stakes) + +| Piece | Decision | +|---|---| +| Multi-turn agent execution / tool loop | ✅ delegate to an agent runtime | +| Permission tri-state (manual/auto/bypass) | ✅ delegate to runtime (mcpctl does coarse gates) | +| Git worktree / branch / PR mechanics | ✅ pod-per-session + runtime | +| Streaming / REPL / approval UX | ✅ delegate (thin client) | +| Post-merge failure recovery | ⚠️ mcpctl routes/reopens; runtime/API does mechanics | + +### Agent runtime — build an interface, not a runtime + +**`AgentRuntime` interface** with operations the orchestrator needs (start +session, send message/tool, get status, request approval, push result, end +session), with **multiple implementations**: +- **#1 (recommended first): a battle-tested coding agent (e.g. Claude Code + headless) running inside the pod** — solves permission tri-state, worktree/git, + subagents, streaming, MCP client. Prompt-file/project-aware, fits + persona-as-prompt. +- **#2 (later, optional): `pi`** — attractive for its lean design and + **extensions/plugin ecosystem**, but younger; adopting it first means paying for + harness maturation, contradicting "don't rebuild." **Second adapter, not first + bet.** + +--- + +## 13. The whole system, drawn together + +``` +mcpctl (orchestrator / state / "team lead") + │ provisions session pods (existing orchestrator) + ▼ +Agent session pod (ephemeral) ← the "desk"; per-agent; isolated + │ init pulls project repos; project prompts/tools/secrets attached + ▼ +Agent runtime (delegated: Claude Code headless / pi / ChatService) ← swappable + │ edits files, runs tests, talks to humans + ▼ +git push / PR / artifact ← the contract the pod returns + ↓ +Task lifecycle (InProgress → Build/Run → Self-Test → QA → Security → SRE → + Done → AwaitingReview → (accept) Merged / (reject) InProgress) + │ QA/Security/SRE feedback→ back to InProgress + ↓ +Confidence check (evidence-anchored: retrieve similar tasks + outcomes) + ↓ low confidence +Escalation → decision dashboard (single human queue, accept/reject/comment) + ↓ accept +Merge via git-provider API (gated by current-main CI + human accept) + ↓ +Epic comparison (Tier-2 vision gate → Design/Product critic, media-native) + ↓ gaps +Reopen / new-task / exec joint session (until Epic done or tradeoffs accepted) +``` + +Layered summary: +- **Epic/epictask/task** = the vision + roadmap + work units. +- **Agent/persona** = person-like entity with global + per-project memory. +- **Pod/session** = agent's isolated workspace; `exec` lets a human join live. +- **Confidence** = evidence-anchored; thresholds drive autonomy. +- **Escalation/dashboard** = the single human decision queue. +- **QA/Security/SRE** = Tier-1 stages inside the task loop; **critic** = Tier-2 at + the Epic. +- **Merge** = current-main verified + human accept → provider API; auto-reopen on + post-merge failure. + +--- + +## 14. Cross-cutting concerns & hard problems + +1. **Retrieval quality is the crux of the confidence story.** Finding the *right* + similar tasks (semantic similarity + recency + outcomes) makes the whole + autonomous/confidence model trustworthy; weak retrieval quietly poisons it. +2. **Coverage-ledger invalidation.** Memory/coverage is only honest if "this + region is still verified" is *invalidated* when that region (or its deps) + changes — otherwise **false confidence** (agent skips a check because "it's + covered," but coverage is stale). + +Additional: +- **Memory staleness/decay** applies to confidence retrieval too (weight recent). +- **Script-artifact hygiene & trust.** Agent-authored scripts are executable code + with their own failure/security surface — they need versioning, review + (reuse of an agent's own tool = confirmation-bias risk on the *tool itself*), + and scoping boundaries (does a task-scoped script get promoted to + project/agent scope deliberately, or accidentally?). The context-savings of + "script not prompt" must not hide a broken or unsafe tool (§11). +- **Incentive integrity.** A scorecard that rewards tool-building must reward + *good, reused* tools, not just tool proliferation; otherwise agents gold-plate + scripts for score rather than reuse existing ones (§11b). +- **Retrieval / RAG failure:** notes are only as good as finding the right ones — + "I checked but checked the wrong thing" is a real failure mode in + security-relevant domains. +- **Trust boundary on agent-authored notes:** coverage ledger is + **platform-verified** (commit-pinned, change-invalidated = machine facts); + prose notes are **agent-opinion** (retrievable, never the *gate*) — avoids a + self-confirming-agent loop. +- **Tier-2 media-native verification in a headless agent** — do we need + render/screenshot infra, or can a design-aware persona judge from a structured + artifact without pixels? **[OPEN — §15]** +- **Cost:** review-only persona + integration gates add tokens/time — worth it on + product/UI work, overkill on pure backend APIs. Risk-gate it. + +--- + +## 15. Consolidated open questions + +Closed now vs. earlier rounds: **QA/Security as in-task stages** (not +post-submit); **critic as Tier-2 at Epic level** (additive, different level); +**artifact term collision** → build-artifact vs context-artifact (§11). + +Still open: + +1. **Brain ownership** — home brain for bare chat, project's `llmProvider` in + `--project` mode? (Lean yes.) Blocks the data model. +2. **Naming (LE-7)** — `epictask` spelling; and whether `task` collides with the + repo's `.taskmaster` tasks. +3. **Prompt delivery to the pod** — MCP vs files-in-container vs LLM injection. + (Affects pod image + runtime; lean: files for the coding-agent runtime.) +4. **Ask-for-help / joint problem-solving mode** — how it's structured (an + escalation tagged help-request?). +5. **`exec` carve-out naming** — reconcile the deliberate Epic-gap `exec` trigger + with the async default (§10). +6. **Epic vs epictask "done"** — do epictasks carry their own acceptance criteria + (spider-web per-node completeness)? Lean yes. +7. **Epic as a resource vs a prompt** — own resource type (RBAC + revision) vs + reuse project prompt/skill. Lean own resource. +8. **Comparison unit** — per-task or per-epictask-chain; a gap can cross linked + epictasks. +9. **Accept-tradeoffs structure** — record *what, why, who, de-scoped-by* as a + revision on the Epic. +10. **Artifact-store location + context return** — where binaries live + (server-side per LE-8/LE-5); whether an `expand`ed artifact returns to + stored-only after the turn (lean yes). +11. **Merge recovery preference** — fast revert of task-sized merge vs always + forward-fix on main (lean: revert as escape hatch). +12. **Dashboard surface** — web? TUI? CLI? +13. **Tier-2 media-native infra** — render/screenshot infra vs structured-artifact + judgment. +14. **MVP vertical-slice order** — confirm/adjust the path below. +15. **Script-artifact design** — the `script` artifact kind, its three scopes + (task/project/agent), versioning, and how reuse/`expand`/eviction apply (§11). +16. **Incentive & scorecard** — how to make tool-building genuinely worthwhile and + how the scorecard feeds autonomy vs. reporting: which baseline rules are + provisioned as defaults, where they live (global vs per-project), and whether + the “basics rules agent” is a persona or a passive framework (§11b). + +--- + +## 16. Suggested MVP path + +1. **Epic + epictask + task + Repo entities** + project linkage (Prisma models, + repository, routes, CLI). mcpctl-native, small, high clarity. (Now includes the + vision layer.) +2. **AgentRuntime adapter interface** + first runtime (existing coding agent + headless) inside a pod. Prove: pod spawn → clone → work task branch → push → + report. +3. **Task lifecycle + QA/Security/SRE stages + PR linkage + human accept gate** → + merge via provider API (current-main CI + Reopened-on-failure). +4. **Tier-2 / vision:** Design/Product critic persona + Epic-comparison → + reopen / new-task / `exec` joint session. +5. **Collaboration modes:** on-demand async standup + `exec` joint mode. +6. **Confidence-driven autonomy + escalation dashboard.** +7. **Memory + coverage ledger + context-artifact management** (invalidation- + aware). +8. **Script artifacts & baseline scorecard** — reusable `script` artifacts at + task/project/agent scope, plus the provisioned baseline-rules scorecard (§11). + +--- + +*Unified record of all rounds (agentic-teams + vision-fidelity + Epic/artifact). +Supersedes the four earlier design files. Feature/decision-focused; schema and +code intentionally indicative. Open questions in §15 are the design backlog.*