agentic-teams: epics, tasks, git forges, agent authority #139

Merged
michal merged 36 commits from feat/agentic-teams into main 2026-10-09 14:47:34 +00:00
Showing only changes of commit 5e1c6b3681 - Show all commits

782
docs/agentic-teams.md Normal file
View File

@@ -0,0 +1,782 @@
# Agentic Teams — Unified Design Document
> **Status:** Consolidated design record — one authoritative document unifying
> every round of discussion. Supersedes the earlier split files
> (`agentic-teams-review.md`, `vision-fidelity.md`, `epic-task-artifact-model.md`,
> `session-summary-vision-fidelity.md`), which have been folded into this single
> living doc.
>
> This is **feature- and decision-focused**, not code. Where a technical shape
> is mentioned it is indicative; the aim is to record *what* we agreed, *why*,
> and *which decisions are still open*.
>
> The one paragraph: mcpctl should orchestrate a software-team-shaped workflow
> where human developers and autonomous agents are **peers in the same project**,
> with mcpctl as the orchestration, state, and "team lead" layer. Agents work
> real code in isolated pods, verify against a living vision, and escalate
> decisions to a single human supervisor.
---
## Table of contents
1. The vision & motivation
2. Core design principles
3. The two-level verification model (correctness + vision)
4. The person-model: agents as first-class "people"
5. The Epic / epictask / task data model
6. The task lifecycle & QA/Security/SRE stages
7. Repositories & per-session pods
8. The confidence signal
9. Escalation & the decision dashboard
10. `mcpctl exec` — the human in the pod, live
11. Artifacts & context management; script artifacts, agent-built tools & the
baseline scorecard
12. Delegation strategy — what NOT to rebuild
13. The whole system, drawn together
14. Cross-cutting concerns & hard problems
15. Consolidated open questions
16. Suggested MVP path
---
## 1. The vision & motivation
### 1a. The agentic-teams vision (rounds 1–2)
Instead of a stateless chatbot, each **project** gains an **agentic flow**:
first-class **Tasks** and **Repositories** entities, multiple **agent personas**
(QA, security, engineers) that work real code in **ephemeral per-session pods**,
a **human-in-the-loop** escalation + approval gate, and — critically —
**persistent, per-persona memory** so agents learn and get cheaper/smarter per
task over a project's lifetime. The goal is to **replicate how the human team
works today**: standups, joint debugging/architecting sessions, autonomous solo
work when confident, and a single human supervisor clearing a queue of
escalated decisions.
Origin (from the original idea): a flow of **Plan → Refine Plan → Draw Roadmap →
Discuss Tasks from the first roadmap position**, i.e. how a human team decomposes
and executes work. Note: we decided **against** hardcoding this as a rigid state
machine (see §5); it is the motivating scenario, not a fixed pipeline.
### 1b. The vision-fidelity problem (round 3)
A distinct, compounding motivation observed in real use:
> 1. you had a grand vision
> 2. we split the grand vision into tasks
> 3. we completed all the tasks
> 4. we are left with a stick with wheels instead of a car
Every task completed, yet no car. The diagnosis: **this is not an execution
problem — it's decomposition + verification.** Root causes:
1. **Decomposition is lossy** — a multidimensional vision compresses into flat
imperatives; the *why* and *what-good-looks-like* get dropped.
2. **Tasks are graded mechanically** — "compiles / lint / renders," never "does
this serve the vision / is this designed / does this feel finished."
3. **Integration is nobody's job** — a car is parts *working together*, but task
systems execute a list.
4. **No feedback loop mid-flight** — the agent assumes task 3 was fine and never
re-checks against the original vision.
5. **Design/product review is a phase, not a gate** — applied at the end, when
the wrong shape is already baked.
**Research synthesis that shapes the design:** vertical slices / walking
skeleton (decompose into user-visible `mini-wholes`, not layers — always have *a*
small car, never a pile of parts); a **two-tier Definition of Done**
(mechanical + systemic); review-gate HITL over per-tool approval;
spec-driven + **media-native** verification (judge an artifact in its native
medium — render/run, not read code); and the **critic-agent pattern** (a
non-executing persona that reviews against the vision catches drift the working
agent's confirmation bias misses).
---
## 2. Core design principles (agreed)
1. **Don't build everything from scratch.** If something is already solved well
(permissions, worktree mechanics, multi-turn agent execution, streaming),
delegate to it. Avoid rabbit holes. Explicit top priority.
2. **Agents are like people.** A persistent identity, personal memory, and the
ability to be *invited* to zero, one, or many projects. Not welded to a
single project or a single brain.
3. **Context is scoped, globally + per-project.** One global personal memory plus
*separate per-project* knowledge, so an agent doesn't cross-contaminate one
project's conventions or secrets into another.
4. **Isolation without shared live workspaces.** Agents do **not** share a live
filesystem with each other. Handoffs happen via git (source + verdicts) and an
immutable **build-artifact store** (terminology per §11). A single human *can*
share an environment with a single agent (via `exec`).
5. **Efficiency must be evidence-backed, and evidence must be honest.** Memory,
confidence, and verification scoping rely on retrievable evidence; any "still
valid" claim must be invalidated when the covered region changes — otherwise
memory becomes *false confidence*.
6. **The human is a peer and a supervisor.** A single queue of decisions;
fine-grained per-tool permission approval is delegated to the runtime; mcpctl
owns coarse-grained gates.
7. **Async, artifact-driven, human-as-fuse** (the through-line). Default flow is
asynchronous over frozen artifacts + a decision queue + disposable pods;
`exec` is the deliberate opt-in exception for live joint work.
8. **Vision is a living, versioned artifact** (round 3). There is an
authoritative Epic/VISION that tasks reference by version, so drift is
*detectable* and *auditable*.
---
## 3. The two-level verification model
This is the mental model that unifies rounds 1–3. There are **two distinct
levels of verification**, addressing the two distinct failure classes:
| Level | What it checks | Who runs it | When | Native medium |
|---|---|---|---|---|
| **Tier 1 — Correctness/Security/Ops** | Does it build, run, pass tests, stay secure, and is the **deployment-as-code correct for this project's real infra** | Developer + QA + Security + SRE/Release personas | **Inside the task's own lifecycle** (stages before Done) | code / diff / tests / deployment manifests |
| **Tier 2 — Vision Fidelity / Design** | Does the assembled result match the vision? Is it designed, does it feel finished, does it integrate | **Design/Product critic** (review-only, non-executing) | **At the Epic level**, after tasks reach Done | render / run / walkthrough (media-native) |
**These are different levels working at different points — they do not
replace each other.** QA/Security/SRE are correctness gates on the *task*; the
critic is a vision gate on the *Epic/whole*. Both feed the human accept gate.
---
## 4. The person-model: agents as first-class "people"
### Decisions
- An agent exists **independently of any project** — a persistent entity with a
name, description, system prompt, and its own accumulated memory.
- An agent can **join zero, one, or many projects**, declared with
`mcpctl apply -f agent.yaml`. Intuitive shape:
```yaml
agents:
- name: agent001
description: "Full-stack engineer persona"
systemPrompt: "…"
llmRef: agent001-brain # optional default/home brain
projects: [sre, payments] # many-to-many membership
```
- This generalises today's model (Agent pinned to one `Llm` + one optional
`projectId`) into a **many-to-many membership** (an `AgentProject` junction,
analogous to the existing `ProjectServer` junction). The junction is also where
per-project *role / scope* (and RBAC) would hang.
### Two lenses on the same person (both modes work)
- `mcpctl chat agent001` — the person **on its own**: `agent.systemPrompt` +
**global personal memory**. General conversation, no project attached.
- `mcpctl chat agent001 --project sre` — the **same person, viewing project sre**:
systemPrompt **+ global personal memory (always present — it's who they are)**
**+ project-sre knowledge** (topology, coverage ledger, project Prompts)
**+ project-sre tools** (attached MCP servers as live tools)
**+ threads scoped to {person, sre}**.
Key subtlety: **global personal memory is present in BOTH modes** — `--project`
adds a context layer, it does not create a different agent. **Threads** are
scoped to `(agent, optional project)` — same person, separate "meeting rooms."
### Memory: global generalist + per-project context
- **Global personal memory** — transferred expertise (methods, worldview). Owned
by the identity; travels with it.
- **Per-project knowledge** — project topology, coverage ledgers, "auth lives
here," **and operational/deployment model** (target infra, methods, tooling,
network policies, known gotchas — the SRE persona's bread and butter, §6).
Keyed to the project, retrieved in the right context only.
- **Both owned by the same identity, separately stored/scoped**, so one project's
quirks don't pollute another's judgment — like a consultant with general skills
plus per-client context.
### The motivating example: the hotfix
If an agent just did a **full security review of the app**, but a **tiny hotfix**
comes in, there is **no need to recheck everything — only what changed**. The
agent's own past verification of unchanged regions (pinned to a commit, still
valid) lets it **delta-check** rather than full-rescan. This is the economic
argument for memory + coverage-ledger, and it only holds if unchanged-after-
verification coverage can be *trusted* — hence non-negotiable invalidation (§14).
### The "brain" — open knot
Who owns the brain (which Llm powers the agent) is the one real open question.
Lean: **home brain for bare chat; the project's `llmProvider` in `--project`
mode** (portability, matches project-chat). Alternative: agent keeps its own
brain everywhere (strict persona consistency, less portable). **[OPEN — see §15]**
---
## 5. The Epic / epictask / task data model
The vision-fidelity fix needs an authoritative, versioned source of truth, so we
introduce a **vision layer** above the task layer (round 3, Jira-style
nomenclature). The through-line (async, human-as-fuse) is preserved.
```
Epic (the vision — its own resource)
└── epictask — a roadmap node; tasks hang off it; chains to other epictasks
└── task — the unit of work / execution (own lifecycle §6)
```
### Epic = the vision
- The **back-and-forth before coding starts** *is* the vision; the Epic captures
its conclusion. Project-scoped.
- Holds the **agreed "done" criteria** (checkable outcomes + intent/quality
attributes) written at vision time, **scope-change history** (revisions, so
creep/drift is visible), and **accepted-tradeoff decisions**.
- **An Epic is "done" ONLY when:** 100% of the vision is delivered, **OR** the
user explicitly accepted tradeoffs (recorded so done is honest + auditable).
### epictask
- A **roadmap node**. Tasks link to an epictask; **epictasks chain to each other**
(dependencies) → visualizing all epictasks yields a **spider-web roadmap**
(dependency graph, not a flat sequence) — the planning/navigation layer.
- Lean: epictasks carry their own (decomposed) acceptance criteria so the web
shows per-node completeness. **[OPEN — §15]**
### task
- The unit of work, **owned by an epictask**, with the standard lifecycle (§6).
- Because tasks link to epictasks → Epic, every task can resolve **"what am I
serving and what does done-for-this look like"** by walking up the chain.
**Tier-2 (vision) DoD is reachable from every task.**
### Reopen vs new-task vs exec (the Epic-comparison mechanism)
After agents report done, the delivered whole is compared against the Epic:
| Gap cause | Outcome |
|---|---|
| A task **claimed** a piece but didn't deliver it right | **Reopen** the task (directional comment back) |
| A piece was **never decomposed** into a task (breakdown gap) | **New task** under the relevant epictask |
| Hard problem / quality gap / "we couldn't fix X" | **`exec` joint session** (optional reopen/new task after) |
The loop repeats until the Epic reaches 100%-delivered or delivered-with-
accepted-tradeoffs. **This comparison moment is the principled trigger for
`exec`** (§10).
### Flow is NOT a rigid state machine
Decided: do **not** hardcode Plan→Refine→Roadmap→Work as a pipeline. Tasks have
state; agents and the human move between **collaboration modes** based on
**confidence** and **collaboration need**. A rigid pipeline would force every
task (including a one-line docs typo) through the same heavyweight flow. The flow
is a set of *modes*, not a fixed sequence.
### Collaboration modes (the "team dynamics")
- **Standup** — on-demand, artifact-driven introspection: `mcpctl standup
--project sre` pulls each agent's per-task progress/notes since last and
summarizes; each agent's requested decisions surface as dashboard items. Not a
synchronous ritual.
- **Joint debugging / architecting** — the human works *with* an agent on a
challenge (via `exec`, §10).
- **Present architectural decisions** — agents surface decisions (→ dashboard),
not just completed work.
- **Ask the human for help** — distinguished from hot escalation: a collaboration
request, a mode of its own. **[OPEN — §15]**
- **Work alone when confident** — driven by the confidence signal (§8).
---
## 6. The task lifecycle & QA/Security/SRE stages
This refines the earlier `SelfCheck` stage into a richer set of **correctness
gates that run INSIDE the task's own loop, before "Done."** QA/Security/SRE work
with the developer *during* the task, as stages — feedback bounces the task back
to InProgress until it passes:
```
InProgress (developer)
→ Build / Run (compile, run, smoke)
→ Self-Testing (developer's own tests)
→ QA Testing (QA persona checks; feedback → back to InProgress)
→ Security Review (security persona checks; feedback → back to InProgress)
→ SRE / Release Review (is the deployment-as-code correct for our real
infra — right target, right method, right tooling;
e.g. Kubernetes vs Portainer/EC2/local;
feedback → back to InProgress)
→ Done (leaves developer's loop)
→ AwaitingReview (human accept gate — your accept/reject + comment)
accept → Merged (PR merged via provider API)
reject → InProgress (you wrote a comment → developer reworks)
→ Merged
post-merge failure (webhook) → Reopened (same agent; revert as fast net)
```
- **QA/Security/SRE are stages, not post-submit reviewers** — decided explicitly.
QA (quality) and Security (security) are Tier-1 correctness/security gates. An
**SRE/Release review stage** sits alongside them. It reviews the
**deployment-as-code** (IaC, release manifests, container/Docker config) to
ensure it matches the project's **actual deployment model and infrastructure**
— the right target (e.g. our Kubernetes cluster, not an EC2 VM or Portainer or
a local build), the right method (run in Docker, not build locally), and the
right tooling. Its feedback is directional ("no, this should go to Kubernetes,
not Portainer") and bounces the task back to InProgress like QA/Security. It
reviews the deployment *definition*; it is **not** the act of deploying.
The **Design/Product critic** (Tier-2, §3) is a different level operating at the
Epic, *not* one of these stages.
**Why SRE must know the project's operational gotchas.** A large part of SRE
review is pre-empting failures whose root cause is an *implicit operational
rule* the developer LLM doesn't know. Example (from the user): the cluster has
**network policies**, so any new workload needs **egress and ingress rules**.
A developer LLM watching a connection fail will naturally go "cut a hole in
the floor to check the pipes" — deep-diving the network debugging path — when
the real cause is a **missing egress policy** silently blocking traffic at the
policy layer. The SRE persona, knowing the project's network-policy model, can
flag the missing egress/ingress rule in review and save the developer from the
wrong debugging path and from shipping something that fails obscurely at
runtime. This is why the SRE persona must carry rich **per-project infra
knowledge** (§4): deployment topology, targets, methods, tooling, policies,
and known gotchas.
- **PR linkage:** when a task is done, the agent **pushes and opens a PR**; the
Task stores the link (branch, commit, PR URL). The human should **not have to
go to git** — accept/reject happens inside mcpctl.
- **Merge** is a git-provider API call (e.g., Gitea), gated by CI + the human's
accept. It is **not** mcpctl executing git itself.
### The human approval gate (decided — important)
**NOT** Claude's per-tool manual/auto/bypass tri-state. It's a **task-level
acceptance gate**: a yes/no + comment on the *resolved outcome*, which fires the
merge. Once the Tier-1 stages pass, the task enters `AwaitingReview`; the agent
acts autonomously up to that point; the human's whole role is **one decisive
accept/reject + comment**. So **we do not rebuild a permission engine** — mcpctl
enforces coarse gates (who may transition `AwaitingReview→Merged/InProgress`,
what RBAC a session needs); the agent runtime handles fine-grained per-tool
approval.
### Pre-submission verification — risk-scaled, not rigid
Decided **against** a hardcoded pipeline. Verification effort **scales with task
risk** and the personas decide the level:
- **Trivial** (docs, formatting) → a light self-check may suffice.
- **Significant/risky** (auth, secrets, risk-relevant paths) → QA, Security
and/or SRE stages are engaged.
- Driven by the **confidence signal** (§8).
---
## 7. Repositories & per-session pods
### Repositories entity
A **Repository** attaches git repos to a project and is where Tasks happen. It is
a metadata resource (which repos a project owns). **[Note: this entity was
defined in rounds 1–2; carry it forward into the Epic model as the execution
substrate for epictasks.]**
### Per-session pods
- **Each agent session gets its own ephemeral pod/container** (via the K8s
orchestrator mcpctl already has). The pod **is** the workspace.
- An **init container pulls all repos** the project expects → the session is
**project-aware**: right repos checked out, project Prompts, project MCP tools,
scoped secrets.
- No central worktree manager: the pod *is* the worktree (checked out to the task
branch). "Multiple agents on one repo" collapses into each pod checking out its
own branch, working, pushing. Pod dies at session end.
- **Agents do NOT share a live filesystem with each other** — buys independence
(fresh eyes) + provenance.
### Handing work between personas — git is a feature, not a tax
- The handoff is **immutable information about a commit** (branch + diff, base,
evidence, verdict) — not a shared live workspace.
- **Git is the audit trail + provenance:** a verdict pinned to a commit
("QA verified 3f2ab against current main") is reproducible forever — exactly
the "records of similar work" the confidence signal needs.
- **The pod boundary buys true independence** (fresh eyes, no inherited
assumptions).
- **Reviewing a diff is closer to real review** than a shared live tree.
- One deliberate addition: an **immutable *build-artifact store*** keyed by
commit/branch for binaries/build outputs git shouldn't carry (see §11 for the
naming collision).
### Verification session shape — cheap default, heavy only when risk-gated
- **Lightweight review session (default):** QA/security runs a session against
the branch, ideally on a **warm base image (deps pre-installed)**. Reads the
diff, runs targeted checks, writes a verdict. No push. Cheap.
- **Heavy verification session (risk-gated only):** full clean pod checks out the
merged-to-current-main branch and runs full CI — the "verify against current
main" gate. Only for high-risk tasks (security paths, migrations,
cross-contract).
So "separate pods per persona" ≠ "N× full CI every task."
---
## 8. The confidence signal — evidence-anchored, not vibes
This is what makes "autonomous when confident, escalate when not" *trustworthy*.
Protocol (on a **frozen task artifact**, not live chat):
1. Frozen task artifact with all info attached (description, proposed resolution,
evidence).
2. **Retrieve K similar past tasks** (semantic similarity, weighted by recency) —
e.g. "we deployed to k8s yesterday" → similar deploys; "update this secret in
vault" → the exact same class.
3. Each attached **with description + resolution + effect** (what *actually*
happened — succeeded / follow-up / revert).
4. Ask the LLM: **"Given these K similar tasks and their outcomes, how confident
are you that THIS task will resolve without issues?"** citing which precedents.
Refinements:
- **Any evidence can be attached, not just tasks** (docs, PRs, incident records,
QA coverage).
- **Outcome-awareness is what makes it honest** — the *effect* is load-bearing.
- **Staleness matters**: weight recent precedent higher (ties to ledger
invalidation).
- **Failures are bounded and auditable**: the decision cites its evidence, so you
can see *why* it was confident and improve retrieval.
- **Vision-fidelity integration (round 3):** confidence should ALSO score whether
the result serves the vision, not just "will it merge cleanly." The
critic/Epic-comparison (§5) is the Tier-2 half of the same honesty.
### Thresholds drive autonomy
```
confidence(task, evidence[], threshold):
high → proceed solo, light verification
mid → proceed but route QA/security to delta-verify
low → ESCALATE to the human (packaged question)
```
### Outcome shaping (refinement)
Make the question bidirectional/risk-aware: not just "will it go smoothly?" but
"**if it fails, where and how severe, given those precedents?**" — turning
confidence into preemptive review.
---
## 9. Escalation & the decision dashboard (HITL)
- **When confidence is low, the agent escalates a question to the user** — as a
**packaged, first-class escalation item** with full context: task, project,
proposed resolution, evidence/precedents, rationale, what it needs.
- **A dashboard** surfaces **all open questions from all agents across all
projects**, with context about each task/project. The user answers one by one —
**triaging a queue**, not being pulled into each agent's context. One human
supervises N agents across M projects from a single inbox.
- Each question is a **frozen, evidence-attached artifact** → auditable,
resumable, comparable. Actions: accept / reject-with-comment / direct /
route-to-another-agent.
- Confidence question and escalation are the **same artifact** — a low-confidence
agent packages the escalation rather than stopping.
---
## 10. `mcpctl exec` — the human in the pod, live
`mcpctl exec --project NAME AGENTNAME -- bash` must be a **first-class citizen**.
The human/agent relationship is **hybrid**:
1. **Async, artifact-based** (confidence/escalation): most decisions vs frozen
artifacts, answered from the dashboard.
2. **Live, interactive** (`mcpctl exec … -- bash`): joint debugging / architecting
/ real-time hands-on — drop into the agent's session as a shell (like
`kubectl exec` / `docker exec`).
### Semantic (recommended)
"Exec into a pod" should mean **into the working environment (repo/workspace)**,
NOT **into the agent's reasoning loop**:
- **A — working environment:** cloned repo, branches, tools, deps; inspect, run
commands, maybe manually fix. The agent is a separate process you can still talk
to. Robust, supports joint debugging. **Recommended default.**
- **B — attach to the agent's live session/internals:** invasive and fragile. NOT
recommended.
Because `exec` lets a **single human share an environment with a single agent**,
it's a safe, deliberate coupling — not the multi-agent-shared-filesystem
anti-pattern.
### The principled trigger (from §5)
The **Epic-comparison moment** is what gives `exec` its concrete, principled
trigger: when a quality/design gap or a hard "we couldn't fix X" arises, that's
the natural time to `exec` and work it together, rather than filing more async
work or accepting a reduction. **[Note: reconciles the round-2 async decision with
this deliberate exec carve-out — name it so it doesn't conflict with LE-4.]**
---
## 11. Artifacts & context management — and the "artifact" term collision
### The collision, named
In the **agentic-teams rounds**, an **"artifact store"** is a *storage*
mechanism: durable, server-side storage keyed by commit/branch for build outputs
and binaries git shouldn't carry. In the **prime-agent / Epic rounds**, an
**"artifact"** is a *context* mechanism: a structured object in the LLM context
(a PR, file, log) with an on-demand `expand`, never auto-injected, with an
eviction priority (user > agent > artifact-context). **These are two different
axes sharing one noun** — one is "where durable files live," the other is "what
gets lazily loaded into / flushed from the LLM's window." They must not be
conflated by reader or implementer.
### Proposed terminology (to resolve the collision)
- **Build-artifact store** — the durable storage for binaries/build outputs
(Wave-A meaning). Server-side, mcpctl-owned, keyed by commit/task.
- **Context-artifact** — a lazily-loaded, expandable object in the LLM context,
with eviction priority (Wave-B meaning).
### Context-artifact philosophy (prime-agent)
- Artifacts are **objects, not plain text** — `files|test`, logs, **PR & commits**
as structured objects with an `expand` operation, not dumped into context.
- **"Cat" → variable/artifact** — a named, retrievable reference, **not
auto-injected**.
- **Rule: context does NOT auto-populate from artifact content.**
- **Eviction / flush priority (recoverability-driven):** user's words (highest) >
agent's words > artifact-context (evict first, because artifact content is
durably stored and recoverable via `expand`). **Eviction follows recoverability.**
- Lean: an `expand`ed artifact that has been pulled in returns to
stored-only after the turn ends (safe to evict first). **[OPEN — §15]**
### Script artifacts & agent-built tools
Beyond `files|test` and logs, add a **`script` artifact kind** — a named,
reusable piece of executable code (e.g. "merging a PR", "deploying to the
cluster"). A script is a **tool-as-a-script**: instead of the agent re-figuring
out a recurring procedure every time ("how do we deploy?"), it encodes that
procedure once as a reusable, `expand`able artifact.
**Scoping (nested):** a script artifact can live at **three scopes**:
- **Task-scoped** — specific to one task's work.
- **Project-scoped** — shared by everyone working in that project (e.g. the
project's canonical "deploy" or "merge PR" script).
- **Agent-scoped** — carried by a persona between projects (e.g. a security
code reviewer keeps its review tooling inside the *agent*, so it moves with
the agent to any project it joins).
**Why agents should build their own tools (analogy to prime-agent / skills):**
like a skill but *as code*. A script is (a) a durable procedure captured once
instead of re-derived; (b) **context-saving** — because it's code, the full
procedure is not re-loaded into the prompt; it's an `expand`able/executable
artifact, so the agent invokes it rather than re-deriving it (ties to the
eviction/`expand` model above); and (c) **learned-and-reused** — once built, it
persists in the relevant scope and gets cheaper to re-use over time (mirrors
the coverage-ledger / knowledge-accumulation logic from §4/§14).
**Incentive to build tools.** There should be a genuine motivation for an agent
to invest in tooling rather than doing things ad hoc each time. Candidates
(decide):
- building a reusable script earns the agent (or persona) **credit / proven
track-record** that raises future confidence scores (§8);
- reusing an existing script is **preferred / discounted** over ad-hoc
re-derivation; and
- a **scorecard (§11b)** can explicitly reward tool reuse / authorship.
### 11b. The scorecard & default "basics-rules" governance
Proposal: a **scorecard** mechanism — a "basics rules agent" that checks a
**provisioned default list of baseline rules** we want every agent to follow.
These defaults ship with the system (provisioned, not invented ad hoc). Example:
- **"Always make unit tests for everything you make."**
The scorecard agent reviews work/productivity against these baseline rules and
scores it, surfacing violations/omissions (e.g. "missing unit tests") as
escalation/feedback. It is distinct from the Tier-1 QA/Security/SRE stages and
the Tier-2 critic — it's a **baseline-behaviour / standards gate**: a checklist
of minimum expectations rather than a judgement of correctness or vision.
Open design points **[see §15]**: which rules are provisioned as defaults (and
where they live — inherited from the project? a global baseline?), how the
score influences autonomy/confidence vs. just reporting, and whether the
scorecard is a persona ("basics rules agent") or a passive framework the
running agents consult.
---
## 12. Delegation strategy — what NOT to rebuild
### mcpctl owns (the differentiator)
- Task + Repo + Epic/epictask entities, task graph / roadmap.
- Project/repo awareness; per-session pod provisioning + repo-clone init.
- Session orchestration (spawn/kill, attach prompts/secrets).
- Coarse-grained RBAC → runtime binding; task lifecycle, acceptance gate,
escalation packaging, dashboard.
- Memory / confidence / coverage + vision (Epic) layer.
- **Script-artifact registry + baseline scorecard** (§11/§11b).
### Delegate (the table stakes)
| Piece | Decision |
|---|---|
| Multi-turn agent execution / tool loop | ✅ delegate to an agent runtime |
| Permission tri-state (manual/auto/bypass) | ✅ delegate to runtime (mcpctl does coarse gates) |
| Git worktree / branch / PR mechanics | ✅ pod-per-session + runtime |
| Streaming / REPL / approval UX | ✅ delegate (thin client) |
| Post-merge failure recovery | ⚠️ mcpctl routes/reopens; runtime/API does mechanics |
### Agent runtime — build an interface, not a runtime
**`AgentRuntime` interface** with operations the orchestrator needs (start
session, send message/tool, get status, request approval, push result, end
session), with **multiple implementations**:
- **#1 (recommended first): a battle-tested coding agent (e.g. Claude Code
headless) running inside the pod** — solves permission tri-state, worktree/git,
subagents, streaming, MCP client. Prompt-file/project-aware, fits
persona-as-prompt.
- **#2 (later, optional): `pi`** — attractive for its lean design and
**extensions/plugin ecosystem**, but younger; adopting it first means paying for
harness maturation, contradicting "don't rebuild." **Second adapter, not first
bet.**
---
## 13. The whole system, drawn together
```
mcpctl (orchestrator / state / "team lead")
│ provisions session pods (existing orchestrator)
▼
Agent session pod (ephemeral) ← the "desk"; per-agent; isolated
│ init pulls project repos; project prompts/tools/secrets attached
▼
Agent runtime (delegated: Claude Code headless / pi / ChatService) ← swappable
│ edits files, runs tests, talks to humans
▼
git push / PR / artifact ← the contract the pod returns
↓
Task lifecycle (InProgress → Build/Run → Self-Test → QA → Security → SRE →
Done → AwaitingReview → (accept) Merged / (reject) InProgress)
│ QA/Security/SRE feedback→ back to InProgress
↓
Confidence check (evidence-anchored: retrieve similar tasks + outcomes)
↓ low confidence
Escalation → decision dashboard (single human queue, accept/reject/comment)
↓ accept
Merge via git-provider API (gated by current-main CI + human accept)
↓
Epic comparison (Tier-2 vision gate → Design/Product critic, media-native)
↓ gaps
Reopen / new-task / exec joint session (until Epic done or tradeoffs accepted)
```
Layered summary:
- **Epic/epictask/task** = the vision + roadmap + work units.
- **Agent/persona** = person-like entity with global + per-project memory.
- **Pod/session** = agent's isolated workspace; `exec` lets a human join live.
- **Confidence** = evidence-anchored; thresholds drive autonomy.
- **Escalation/dashboard** = the single human decision queue.
- **QA/Security/SRE** = Tier-1 stages inside the task loop; **critic** = Tier-2 at
the Epic.
- **Merge** = current-main verified + human accept → provider API; auto-reopen on
post-merge failure.
---
## 14. Cross-cutting concerns & hard problems
1. **Retrieval quality is the crux of the confidence story.** Finding the *right*
similar tasks (semantic similarity + recency + outcomes) makes the whole
autonomous/confidence model trustworthy; weak retrieval quietly poisons it.
2. **Coverage-ledger invalidation.** Memory/coverage is only honest if "this
region is still verified" is *invalidated* when that region (or its deps)
changes — otherwise **false confidence** (agent skips a check because "it's
covered," but coverage is stale).
Additional:
- **Memory staleness/decay** applies to confidence retrieval too (weight recent).
- **Script-artifact hygiene & trust.** Agent-authored scripts are executable code
with their own failure/security surface — they need versioning, review
(reuse of an agent's own tool = confirmation-bias risk on the *tool itself*),
and scoping boundaries (does a task-scoped script get promoted to
project/agent scope deliberately, or accidentally?). The context-savings of
"script not prompt" must not hide a broken or unsafe tool (§11).
- **Incentive integrity.** A scorecard that rewards tool-building must reward
*good, reused* tools, not just tool proliferation; otherwise agents gold-plate
scripts for score rather than reuse existing ones (§11b).
- **Retrieval / RAG failure:** notes are only as good as finding the right ones —
"I checked but checked the wrong thing" is a real failure mode in
security-relevant domains.
- **Trust boundary on agent-authored notes:** coverage ledger is
**platform-verified** (commit-pinned, change-invalidated = machine facts);
prose notes are **agent-opinion** (retrievable, never the *gate*) — avoids a
self-confirming-agent loop.
- **Tier-2 media-native verification in a headless agent** — do we need
render/screenshot infra, or can a design-aware persona judge from a structured
artifact without pixels? **[OPEN — §15]**
- **Cost:** review-only persona + integration gates add tokens/time — worth it on
product/UI work, overkill on pure backend APIs. Risk-gate it.
---
## 15. Consolidated open questions
Closed now vs. earlier rounds: **QA/Security as in-task stages** (not
post-submit); **critic as Tier-2 at Epic level** (additive, different level);
**artifact term collision** → build-artifact vs context-artifact (§11).
Still open:
1. **Brain ownership** — home brain for bare chat, project's `llmProvider` in
`--project` mode? (Lean yes.) Blocks the data model.
2. **Naming (LE-7)** — `epictask` spelling; and whether `task` collides with the
repo's `.taskmaster` tasks.
3. **Prompt delivery to the pod** — MCP vs files-in-container vs LLM injection.
(Affects pod image + runtime; lean: files for the coding-agent runtime.)
4. **Ask-for-help / joint problem-solving mode** — how it's structured (an
escalation tagged help-request?).
5. **`exec` carve-out naming** — reconcile the deliberate Epic-gap `exec` trigger
with the async default (§10).
6. **Epic vs epictask "done"** — do epictasks carry their own acceptance criteria
(spider-web per-node completeness)? Lean yes.
7. **Epic as a resource vs a prompt** — own resource type (RBAC + revision) vs
reuse project prompt/skill. Lean own resource.
8. **Comparison unit** — per-task or per-epictask-chain; a gap can cross linked
epictasks.
9. **Accept-tradeoffs structure** — record *what, why, who, de-scoped-by* as a
revision on the Epic.
10. **Artifact-store location + context return** — where binaries live
(server-side per LE-8/LE-5); whether an `expand`ed artifact returns to
stored-only after the turn (lean yes).
11. **Merge recovery preference** — fast revert of task-sized merge vs always
forward-fix on main (lean: revert as escape hatch).
12. **Dashboard surface** — web? TUI? CLI?
13. **Tier-2 media-native infra** — render/screenshot infra vs structured-artifact
judgment.
14. **MVP vertical-slice order** — confirm/adjust the path below.
15. **Script-artifact design** — the `script` artifact kind, its three scopes
(task/project/agent), versioning, and how reuse/`expand`/eviction apply (§11).
16. **Incentive & scorecard** — how to make tool-building genuinely worthwhile and
how the scorecard feeds autonomy vs. reporting: which baseline rules are
provisioned as defaults, where they live (global vs per-project), and whether
the “basics rules agent” is a persona or a passive framework (§11b).
---
## 16. Suggested MVP path
1. **Epic + epictask + task + Repo entities** + project linkage (Prisma models,
repository, routes, CLI). mcpctl-native, small, high clarity. (Now includes the
vision layer.)
2. **AgentRuntime adapter interface** + first runtime (existing coding agent
headless) inside a pod. Prove: pod spawn → clone → work task branch → push →
report.
3. **Task lifecycle + QA/Security/SRE stages + PR linkage + human accept gate** →
merge via provider API (current-main CI + Reopened-on-failure).
4. **Tier-2 / vision:** Design/Product critic persona + Epic-comparison →
reopen / new-task / `exec` joint session.
5. **Collaboration modes:** on-demand async standup + `exec` joint mode.
6. **Confidence-driven autonomy + escalation dashboard.**
7. **Memory + coverage ledger + context-artifact management** (invalidation-
aware).
8. **Script artifacts & baseline scorecard** — reusable `script` artifacts at
task/project/agent scope, plus the provisioned baseline-rules scorecard (§11).
---
*Unified record of all rounds (agentic-teams + vision-fidelity + Epic/artifact).
Supersedes the four earlier design files. Feature/decision-focused; schema and
code intentionally indicative. Open questions in §15 are the design backlog.*