From d24c30761bb59817cea829c2b014a900d63fc641 Mon Sep 17 00:00:00 2001 From: Michal Date: Sat, 8 Aug 2026 09:03:58 +0100 Subject: [PATCH] docs: keep superseded design-round doc history Supporting documents folded into agentic-teams.md (the canonical unified design record), retained as historical context: - agentic-teams-review.md (round 2) - vision-fidelity.md (round 3 diagnosis + research) - epic-task-artifact-model.md (Epic/epictask/task + context philosophy) - session-summary-vision-fidelity.md (session narrative) All superseded by docs/agentic-teams.md, which remains the source of truth. --- docs/agentic-teams-review.md | 246 ++++++++++++++++++++++++ docs/epic-task-artifact-model.md | 156 +++++++++++++++ docs/session-summary-vision-fidelity.md | 230 ++++++++++++++++++++++ docs/vision-fidelity.md | 222 +++++++++++++++++++++ 4 files changed, 854 insertions(+) create mode 100644 docs/agentic-teams-review.md create mode 100644 docs/epic-task-artifact-model.md create mode 100644 docs/session-summary-vision-fidelity.md create mode 100644 docs/vision-fidelity.md diff --git a/docs/agentic-teams-review.md b/docs/agentic-teams-review.md new file mode 100644 index 0000000..05c3a91 --- /dev/null +++ b/docs/agentic-teams-review.md @@ -0,0 +1,246 @@ +# Agentic Teams β€” Second Review Round + +> **Status:** Second design review round. Complements and refines +> `docs/agentic-teams.md` (the primary design record). This file captures the +> loose ends surfaced by reviewing the first draft, and the decisions/recommen- +> dations that followed. +> +> Date of round: second pass. Like the base doc, this is **feature/decision- +> focused**, not code-focused. Items marked **[OPEN]** are not yet decided and +> block or shape implementation. Items under **"DECIDED in this round"** are +> resolutions reached here. + +--- + +## 0. What this round covered + +Reviewing the first draft surfaced several **loose ends, tensions, and undefined +mechanics** that would have bitten during implementation. They cluster into one +through-line question (below), then several specific items. The single most +important framing decision this round is **async-vs-sync**, because it resolves +most of the others. + +--- + +## 1. THE through-line decision: async vs sync [mostly DECIDED β€” see below] + +Most of the loose ends collapse into one question: + +> **Is the system fundamentally synchronous-team (everyoneβ€”agents and +> humanβ€”working "together" in near-real-time, waiting on each other), or +> asynchronous (frozen artifacts + a decision queue + disposable pods)?** + +**Where this landed (DECIDED in this round):** + +The system is **fundamentally asynchronous and artifact-driven**, with `exec` +being the *deliberate exception* the human opts into for joint active work. +Evidence from the whole conversation supports this: + +- Confidence checks run on **frozen task artifacts**, not live chat. +- The human supervises from a **single dashboard/inbox** of packaged decisions. +- Pods are **ephemeral and disposable** by design. +- State lives in **mcpctl's database** (not in pods). +- Agents escalate **packaged questions**, not live interjections. + +Locking this resolves loose ends 1, 2, 3 and 4 (below) together, and makes the +design **more robust** (no idle synchronous chains, no live-session +coordination, everything auditable and resumable). `exec` remains available, +but it is a **different activity** (joint active collaboration) rather than the +default way work proceeds. + +--- + +## 2. Loose ends surfaced & resolution status + +### πŸ”΄ LE-1: Do reviewers (QA/security) block the working agent, or run async? + +**The tension:** if a working agent submits and reviewers run in *their own +pods*, does the working agent **block** (idle for minutes/hours) or **end its +session** and get resumed only if rejected? + +**DECIDED (recommendation, consistent with async model):** reviewers run +**asynchronously against the frozen branch/artifact**. The working agent's pod +can end after submission. A **rejection re-spawns a working session** with the +rejection comment as new input. No idle blocking chains. + +This also means we do **not** need live multi-pod coordination β€” the handoff is +artifacts + verdicts in mcpctl's DB, not a live rendezvous between pods. + +### πŸ”΄ LE-2: Are QA/security verdicts a hard gate, or advisory? + +**The tension:** "QA checks, security looks, CI passes, then submit" implies a +**gate** (all must pass); but the confidence model and "human accept is the only +true fuse" imply they're **advisory**. + +**DECIDED (recommendation):** QA/security review is **primarily for the working +agent's self-correction** (catching issues before it reaches you) and **as +attached evidence for the human's accept decision**. Their verdicts are +**informative, not hard autonomous gates**. The **only hard autonomous gate is +CI-against-current-main**; the **human's accept remains the one true fuse.** + +Reasoning: if reviewer verdicts were hard gates, they'd effectively be an +autonomous pre-approval pipeline β€” the rigid-pipeline shape we explicitly +rejected, and the human's accept would become rubber-stamping. Keeping them +advisory preserves the human's genuine authority while still giving agents a +self-correction loop and giving the human good evidence. + +### πŸ”΄ LE-3: Did we actually decide "standup," and how does it fit the async model? + +**The tension:** "standup" was described early ("standup where we discuss +progress since yesterday") but was never designed. It also risks implying a +synchronous ritual. + +**DECIDED (recommendation):** a standup is an **on-demand, artifact-driven +instrospective flow**, not a recurring synchronous call. Example shape: + +- `mcpctl standup --project sre [--since ]` β€” pull each agent's + per-task progress/notes since the last standup and summarize. +- Each agent's **requested decisions surface as dashboard items** (one queue). +- The human reviews the summary and works the queue asynchronously. + +So standup **pulls from per-agent per-task notes** (where progress actually +lives) and **feeds the decision dashboard**. It does not require anyone to be +live simultaneously. + +### 🟑 LE-4: `exec` vs the async review model β€” clarify the boundary. + +**DECIDED (recommendation):** `exec` is for **joint active collaboration** +(debugging / architecting / real-time hands-on), *not* for inserting into a +task already under async review. A task under review is a **frozen artifact**; +`exec` is the opt-in exception when the human *chooses* to be live with one +agent. This keeps the two modes coherent and avoids expecting to `exec` into +the middle of a review pipeline. + +### 🟑 LE-5: Where does state live given pods are disposable? + +**DECIDED:** State lives in **mcpctl's database** β€” task artifacts, verdicts, +per-agent notes, coverage ledgers. **Pods are stateless workers** that read +from / write to that state. Nothing is lost on pod death; pods are cheap to +re-spawn. This is a strength of the design and worth stating explicitly. + +### 🟑 LE-6: The "brain" question β€” who powers the agent's LLM? [OPEN β†’ resolve] + +The one real open knot from the first round, still not closed. + +**Recommendation (decided-leaning):** +- **Bare chat** (`mcpctl chat agent001`): the agent's **home brain** (agent-owned + Llm default). +- **`--project` mode**: the **project's `llmProvider`** takes over (portability; + matches today's project-chat; a project can invite any agent and run it on the + project's model of choice). + +**This blocks the data model**, so it should be closed with one decision. It is +the single most important unresolved item. + +### 🟒 LE-7 (naming): "Task" may collide with the repo's Task Master workflow. + +The mcpctl "Task" resource could conceptually collide with this repo's +`.taskmaster` tasks. **Flag for a naming check** β€” e.g., `workitem` / `item` / +`assignment` β€” so the two don't confuse. Not an implementation blocker; flagged +because a second pass is the right time to settle naming. + +### 🟒 LE-8 (minor): Where does the artifact store (non-git binaries) live? + +Decided earlier that git carries source+verdicts and an **immutable artifact +store keyed by commit** carries binaries/build outputs. **Recommendation:** that +store also lives server-side (mcpctl-owned), keyed by commit/branch, so it +survives pod death and pods pull exactly what they need. Consistent with LE-5. + +### 🟒 LE-9 (open): How are project/persona prompts delivered to the pod? + +**[OPEN]** The user explicitly raised this early and we never chose between: + +- **exposed via MCP** (prompts as MCP calls), +- **as files placed in the container** (prompt files on disk in the pod), +- **LLM query injection directly** (system-prompt injection at call time). + +This affects the pod image and the runtime interface. Lean: a prompt-file-aware +coding-agent runtime β†’ *files in the container*; MCP as an alternative surface; +injection-at-call is how today's `chat` behaves. Should be closed at design time. + +### 🟒 LE-10 (open): The explicit "ask-for-help / joint problem-solving" mode + +**The user described wanting a mode where they help an agent with a challenge +and work on it together** β€” β€œyou can ask me to help you with some challenge and +we can work for it together.” This is related to but distinct from `exec`-into- +a-workspace and from hot escalation. It is not fully designed. + +Lean: an agent that is stuck or lacks knowledge can **request help** (an +escalation tagged as a help-request), routed to the dashboard or an `exec` +session, rather than the human having to discover it independently. + +--- + +## 3. Confirmed strengths worth preserving (from round-one, re-affirmed) + +While reviewing, we confirmed these hold up and should survive any refinement: + +- **Agents as persistent "people"** (global memory + per-project context + + multi-project membership). [DECIDED round one] +- **Confidence = evidence-anchored**, outcome-aware (attach similar tasks' + description + resolution + *effect*). [DECIDED round one] +- **Task-level accept gate** (one decisive human accept/reject + comment) rather + than per-tool permission β€” no permission engine rebuild. [DECIDED round one] +- **Merge gated on current-main CI + human accept** via provider API, with + **auto-reopen/revert** on post-merge failure. [DECIDED round one] +- **Coverage-ledger invalidation** is the honesty backbone of the memory. [MVP + prerequisite] +- **Delegation over reinvention** β€” build an `AgentRuntime` interface; first + runtime = battle-tested coding agent headless; `pi` as a later optional + adapter. [DECIDED round one] + +--- + +## 4. Post-merge concern β€” revisited and reinforced + +The "stuff only fails after merge to master" worry got a concrete resolution in +round one, and the async framing **reinforces** it: + +- Pre-merge acceptance must verify **against current main tip**, not the stale + base. +- The **human's accept is conditioned** on that current-main verification; if + main moved and it's stale, stay `AwaitingReview` flagged rebase-needed (agent + refreshes + re-verifies). +- Post-merge, `Merged β†’ Reopened` on a failure webhook, with **fast revert of + the task-sized merge** as the cheapest recovery. + +The async model strengthens this: because each step is a frozen artifact + state +transition, a post-merge failure is a *routable state* (auto-reopen), not a +surprise to be debugged live. + +--- + +## 5. Remaining open questions (consolidated) + +To make the design "ready to design," close these: + +1. **[LE-6] Brain ownership** β€” closed-leaning but not confirmed: home brain for + bare chat; project's `llmProvider` in `--project` mode. +2. **[LE-7] Naming** β€” is the work entity called a "Task" or something else? +3. **[MVP-scope] Which vertical slice first** β€” the base doc proposes + Task+Repo entities β†’ runtime adapter β†’ accept gate β†’ collaboration modes β†’ + confidence/dashboard β†’ memory/ledger. Confirm or adjust the order. +4. **[LE-9] Prompt delivery** β€” MCP vs files-in-container vs LLM injection (see + LE-9). +5. **[LE-10] Ask-for-help / joint problem-solving mode** (see LE-10). + +--- + +## 6. Suggested MVP path (unchanged from base doc, re-listed for completeness) + +1. Task + Repo entities + project linkage. +2. AgentRuntime adapter interface + first runtime (existing coding agent + headless) inside a pod. Prove: pod spawn β†’ clone β†’ work task branch β†’ push β†’ + report. +3. Task lifecycle + PR linkage + human accept gate β†’ merge via provider API + (current-main CI + Reopened-on-failure). +4. Collaboration modes: on-demand async standup + `exec` joint mode. +5. Confidence-driven autonomy + escalation dashboard. +6. Memory + coverage ledger (invalidation-aware). + +--- + +*This file records the second review round and its outcome. It should be read +together with `docs/agentic-teams.md`; together they form the durable design +record. The async-vs-sync through-line (section 1) is the key decision to carry +forward.* diff --git a/docs/epic-task-artifact-model.md b/docs/epic-task-artifact-model.md new file mode 100644 index 0000000..3953ae7 --- /dev/null +++ b/docs/epic-task-artifact-model.md @@ -0,0 +1,156 @@ +# Epic / Task / EpicTask + Artifact Context Model + +> **Status:** Design discussion β€” capturing decisions from the conversation. +> **Purpose:** Durably record the data model (Epic, epictask, task) and the +> IPython-style artifact/context philosophy the user is designing. Builds on +> `docs/agentic-teams.md` (round 1) + `docs/agentic-teams-review.md` (round 2). +> +> The through-line from round 2 β€” **async, artifact-driven, human-as-fuse** β€” +> is preserved here. This doc adds the *vision layer* (the Epic) and the +> *context-management layer* (artifacts + eviction priorities) that the earlier +> rounds never defined. + +--- + +## 1. The Epic β€” the vision, made a first-class thing + +An **Epic** is the agreed statement of what we want to achieve. It is the +result of the back-and-forth **before coding starts**: the discussion *is* the +vision, and the Epic captures its conclusion. It is project-scoped. + +### What an Epic holds +- The **agreed "done" criteria** β€” the checkable outcomes plus the intent/quality + attributes that define 100% delivery. Written at vision time so "compare + delivered vs vision" has something authoritative to diff against. +- Its **scope changes** over time (revisions), so creep/drift is visible too β€” + not just under-delivery. +- **Tradeoff decisions** the user makes along the way. + +### When is an Epic "done"? +An Epic is **done ONLY when one of two things is true**: +1. **100% of the vision is delivered**, OR +2. The **user explicitly accepted tradeoffs** β€” e.g. a known issue we couldn't + fix ("we accept this gap because"). The accepted-tradeoffs are recorded on + the Epic so the "done" is honest and auditable. + +### The Epic-comparison moment β†’ the joint-debug trigger +After agents report their work done, the delivered whole is **compared against +the Epic**. Two classes of gap: + +- A **listed must-have isn't shipped** β†’ reopen the under-delivering task, or + add a new task for the missing piece (see Β§3). +- A quality/design gap or a **"we couldn't fix X"** situation β†’ this is the + natural moment to **`exec`**: the user drops into the agent's workspace and + does a **joint debugging session** (the pi / Claude-Code live experience), + working the hard problem together rather than accepting a reduction or + filing more async work. + +So "deliver β†’ compare β†’ gap" routes not only to reopen/new-task, but also to +**live collaboration** β€” giving `exec` a concrete, principled trigger point +that round 2 noted was missing (LE-10 / LE-4 boundary). + +--- + +## 2. The data model β€” project resources + +Three project-scoped resources: + +``` +Epic (the vision β€” its own resource, Β§1) + └── epictask β€” a roadmap node; tasks hang off it + β”œβ”€β”€ task β€” the unit of work / execution + └── task + └── epictask ── chains to other epictasks (the roadmap web) +``` + +### `epictask` +- A roadmap node. **Tasks link to an epic task** (a task belongs-to / serves an + epic task). +- **Epic tasks chain to each other** (dependencies). Visualizing all epic tasks + yields a **spider web of roadmap** β€” not a flat sequence, but a dependency + graph. This is the navigational/planning layer. + +### `task` +- The unit of work, owned by an epictask. +- Carries the standard lifecycle from round 1 (InProgress β†’ SelfCheck β†’ + AwaitingReview β†’ accept β†’ Merged β†’ Reopened-on-failure). +- **Contains artifacts** (see Β§4). + +### Threading the vision through +Because tasks link to epictasks and epictasks belong to an Epic, every task can +resolve "**what am I serving and what does done-for-this look like**" by walking +up the chain to the Epic + its agreed "done" criteria. Tier-2 (vision) DoD is +therefore **reachable from every task**, not bolted on. + +--- + +## 3. Reopen vs new-task β€” the mechanism + +The Epic-comparison diff produces both, distinguished by *why* the gap exists: + +| Gap cause | Outcome | +|---|---| +| A task **claimed** to cover a piece but didn't deliver it right (acceptance criteria unmet) | **Reopen** the task (send back with the directional comment) | +| A piece of the Epic was **never decomposed into a task** at all (breakdown gap) | **New task** under the relevant epictask | +| A piece needs **joint effort / the fix is hard** | **`exec` joint session** (plus optionally reopen/new task after) | + +The loop repeats until the Epic reaches a terminal state: 100% delivered, or +delivered-with-accepted-tradeoffs. + +--- + +## 4. Artifacts + IPython-style context (prime-agent philosophy) + +Borrowing from +[PrimeIntellect prime-agent](https://github.com/PrimeIntellect-ai/prime-agent)'s +**"context as variables / prompt-as-a-variable"** model: + +### Artifacts are objects, not plain text +Each task can contain **artifacts**: `files|test`, logs, and also **PR and +commits** β€” but as **structured objects with an `expand` operation**, **not +stored/loaded as plain text**. + +- A PR or commit is an object; the LLM can `expand` it *on demand* to get the + detail (diff, files, messages). It is not dumped into context eagerly. +- "Catting" something **materializes it into a variable/artifact** β€” a named, + retrievable reference β€” rather than auto-injecting its full text into the + LLM context. + +**The rule: context does NOT auto-populate from artifact content.** You pull +what you need; nothing is eagerly loaded. + +### Eviction / flush priority (what to drop first) +When context must be flushed/compacted, the retention hierarchy is: + +1. **Keep the user's words the longest** β€” highest retention. +2. Next, keep what the **agent said**. +3. Next, the **artifact context** β€” and this is *safe to evict first* because + artifacts are **durably stored**; their content can always be **recovered + on demand via `expand`**. + +So eviction order is **user > agent > artifact-context**, justified precisely +by recoverability: artifact content isn't lost, it's retrievable β€” so dropping +it from live context costs nothing permanent. + +--- + +## 5. Open questions / tensions to resolve + +1. **Epic as a resource vs a "prompt"** β€” does the Epic get its own resource + type (with RBAC + `ResourceRevision`), or reuse the project `prompt`/`skill` + mechanism? Lean: own resource β€” it's the audit/source-of-truth for "is this + done," distinct from context-injection prompts. +2. **Epic-task `done` criteria** β€” do epictasks also carry their own + acceptance criteria, or is all "done" defined at the Epic level? Lean: Epic + defines overall done; epictasks carry the *decomposed* subset, so the spider + web can show per-node completeness. +3. **Artifact storage location** β€” consistent with LE-8/LE-5: server-side, + mcpctl-owned, keyed by commit/task. Confirm artifacts live outside git (for + binaries/build outputs) vs git-native for commits/PRs (which stay in the + repo/PR API and are referenced as objects, not copied). +4. **What counts as "artifact context" for eviction** β€” an `expand`ed artifact + that *has* been pulled into context: once the turn ends, does it return to + "stored-only" (safe to evict) or persist as agent-context? Lean: after the + turn it is stored-only again (evict first), because it's recoverable. +5. **Naming (LE-7)** β€” `epictask` vs `epic-task` vs `epic_task`; and whether + `task` collides with the repo's `.taskmaster` tasks. Needs a naming decision. diff --git a/docs/session-summary-vision-fidelity.md b/docs/session-summary-vision-fidelity.md new file mode 100644 index 0000000..0bc9bac --- /dev/null +++ b/docs/session-summary-vision-fidelity.md @@ -0,0 +1,230 @@ +# Session Summary β€” Vision Fidelity in Agentic Teams (full discussion) + +> **Status:** Consolidated session record β€” captures everything discussed, in +> order, so it reads as one coherent thread. +> **Date:** 2026-08-08 Β· pi session Β· mcpctl repo +> +> **IMPORTANT β€” canonical doc:** since this summary was written, +> `agentic-teams.md` was updated and is now the **single authoritative, unified +> design document** that **supersedes** this summary plus `vision-fidelity.md`, +> `epic-task-artifact-model.md` and `agentic-teams-review.md`. Read +> `agentic-teams.md` for the source of truth; this summary is the session +> narrative and highlights the sections the canonical doc added beyond it. + +--- + +## Part 1 β€” The problem statement + +We started with a recurring failure with LLM coding agents: + +> 1. you had a grand vision +> 2. we split the grand vision into tasks +> 3. we completed all the tasks +> 4. we are left with a stick with wheels instead of a car + +Every task completed, yet no car. The diagnosis (see `vision-fidelity.md` for +the full write-up): **this is not an execution problem β€” it's decomposition + +verification.** + +### Root causes +1. **Decomposition is lossy** β€” a multidimensional vision compresses into flat + imperatives; the *why* and *what-good-looks-like* get dropped. +2. **Tasks are graded mechanically** β€” "compiles / lint / renders," never "does + this serve the vision / is this designed / does this feel finished." +3. **Integration is nobody's job** β€” a car is parts *working together*, but + task systems execute a list. +4. **No feedback loop** β€” the agent assumes task 3 was fine and never re-checks + against the original vision. +5. **Design/product review is a phase, not a gate** β€” applied at the end, when + the wrong shape is already baked. + +### Research synthesis (from the write-up) +- **Vertical slices / walking skeleton** β€” decompose into `mini-wholes` + (user-visible, end-to-end slices), not layers. Always have *a* small car, not + a pile of parts. +- **Two-tier Definition of Done** β€” mechanical tier + systematic tier (matches + vision, integrates, is designed, feels finished). Tier-2 evaluated by running + the medium, not reading code. +- **Review-gate HITL over per-tool approval** β€” consistent with round-2. +- **Spec-driven + media-native verification** β€” judge an artifact in its native + medium (render/run for UI). +- **Critic-agent pattern** β€” a non-executing persona that reviews against the + vision catches drift the working agent's confirmation bias misses. + +--- + +## Part 2 β€” Grounding in the repo (the "other window") + +We confirmed this connects to real, existing machinery: + +- **mcpctl agent platform** β€” persistent agent personas, project prompts, + `ResourceRevision` (versioned log), `ResourceProposal` (review/approve + queue), personalities (same agent, different judgment via prompt overlays). +- **gstack skills** (installed skill library) β€” `/autoplan` (auto-review + pipeline: CEO β†’ Design β†’ Eng β†’ DX), `/plan-ceo-review` (grand-vision + guardian, nuclear scope challenge, 10x vision), `/plan-design-review` + (rates each design dimension 0-10, generates visual mockups, "text + descriptions are not a substitute for showing what it looks like"), `/spec`. +- **`agentic-teams.md` + `agentic-teams-review.md`** β€” the durable design + record, landed on **async, artifact-driven, human-as-fuse** (section 1 of + the review round is the through-line). + +**Key realization:** the platform reviews the *plan* hard before build, but +**nothing reviews the *built result* against the vision after implementation**, +and **no vision artifact exists as a first-class, versioned resource**. That's +the gap today's design fills. + +--- + +## Part 3 β€” Where we landed: the Epic model + +We reframed VISION.md β†’ **Epic**, in Jira nomenclature β€” and it's the thing we +discussed in *this* window's second half. Full spec in +`epic-task-artifact-model.md`. + +### The Epic is the vision +- The **back-and-forth discussion before coding starts** *is* the vision; the + Epic captures its conclusion. It's project-scoped. +- Holds the **agreed "done" criteria** (checkable outcomes + intent/quality + attributes) written at vision time, plus scope-change history (revisions) and + **accepted-tradeoff decisions**. + +### Epic lifecycle β€” "done" only two ways +An Epic reaches **done** only when: +1. **100% of the vision is delivered**, OR +2. The **user explicitly accepted tradeoffs** (e.g. an issue we couldn't fix) β€” + recorded on the Epic, so done is honest + auditable. + +**The comparison moment is the `exec` trigger.** Delivered-whole vs Epic β†’ gaps +route to: +- **reopen** under-delivering tasks, or +- **new tasks** for never-decomposed pieces, or β€” when it's a hard "we + couldn't fix X" β€” a **joint `exec` debugging session** (the pi / Claude-Code + live experience). This gives `exec` a concrete, principled trigger. + +--- + +## Part 4 β€” Data model: task / epictask / Epic + +Hierarchy (details in `epic-task-artifact-model.md` Β§2): + +``` +Epic (the vision) + └── epictask β€” roadmap node; tasks hang off it; chains to other epictasks + └── task β€” the unit of work (own lifecycle from round 1) +``` + +- **tasks link to epictasks**; **epictasks chain to each other** β†’ visualizing + all epictasks gives a **spider-web roadmap** (dependency graph, not flat). +- Walking any task up the chain reaches the Epic's "done" criteria, so + **tier-2 / vision DoD is reachable from every task**. + +### Reopen vs new-task +| Gap cause | Outcome | +|---|---| +| A task under-delivered its acceptance criteria | **Reopen** (directional comment back) | +| A piece was never decomposed (breakdown gap) | **New task** under the epictask | +| Hard problem / needs joint effort | **`exec` joint session** | + +--- + +## Part 5 β€” Artifacts + IPython-style context (prime-agent philosophy) + +Borrowed from [prime-agent](https://github.com/PrimeIntellect-ai/prime-agent)'s +**"context as variables / prompt-as-a-variable"** (details in Β§4 of the model +doc): + +- **Artifacts are objects, not plain text** β€” `files|test`, logs, and also + **PR & commits** as structured objects with an `expand` operation (not + dumped into context). +- **"Cat" β†’ variable/artifact** β€” a named, retrievable reference, **not + auto-injected**. +- **Rule: context does NOT auto-populate from artifact content.** + +### Eviction / flush priority (recoverability-driven) +1. **User's words** β€” highest retention. +2. **Agent's words** β€” next. +3. **Artifact context** β€” evict first, *because artifacts are durably stored + and recoverable via `expand`*. **Eviction priority follows recoverability.** + +The elegant win: dropping artifact-context first is *cheap* because it's +recoverable β€” most compaction schemes don't formalize this, and it turns a +hand-wavy "compaction" into a principled hierarchy. + +--- + +## Part 6 β€” Open questions / tensions (carried forward) + +1. **Epic as a resource vs a prompt** β€” own resource type (RBAC + revisions) or + reuse project prompt/skill? Lean: own resource. +2. **Epic-level vs epictask-level "done"** β€” should epictasks carry their own + acceptance criteria so the spider web shows per-node completeness? Lean: yes. +3. **Comparison unit: task-level or epictask-chain-level?** A gap can cross + multiple linked epictasks; a single "reopen" may not express "this chain is + incomplete." Decide whether comparison happens per-task or per-chain. +4. **`exec` as a deliberate carve-out** β€” the async model is default, but the + Epic-gap triggers sync `exec` *by design*. Name this so it doesn't conflict + with the round-2 async decision (LE-4). +5. **Accept-tradeoffs needs structure** β€” record *what, why, who, de-scoped-by* + as a revision on the Epic. +6. **Naming (LE-7)** β€” `epictask` spelling; and `task` vs repo's `.taskmaster` + tasks. +7. **Artifact storage** β€” server-side mcpctl-owned for binaries/builds (per + LE-8/LE-5); commits/PRs stay in git/repo and are referenced as objects. +8. **Post-build re-verification** β€” the remaining product-side gap: nothing + re-checks the rendered/built result against the reviewed mockup/vision after + implementation (the `design-review`/`qa` path is opt-in, not wired into the + Epic-comparison loop). + +--- + +## Referenced artifacts produced today + +| File | Purpose | +|---|---| +> See `docs/agentic-teams.md` β€” now the single unified document (supersedes +> `vision-fidelity.md`, `epic-task-artifact-model.md`, `agentic-teams-review.md`, +> and this summary). + +## Part 7 β€” What the canonical `agentic-teams.md` added beyond today's thread + +When reviewing the updated `docs/agentic-teams.md` (which folds in everything +we discussed), these sections were found to be **new / more evolved** than the +session thread above β€” worth reading there: + +1. **Two-level verification model (Β§3)** β€” formalised table: **Tier 1** + Correctness/Security/Ops (inside the task lifecycle) vs **Tier 2** Vision + Fidelity/Design (at the Epic, media-native). They do not replace each other. +2. **SRE/Release review stage (Β§6)** β€” Tier-1 stage inside the task loop that + reviews **deployment-as-code** (IaC, release manifests, container config) + against the project's **actual deployment model/infra** (Kubernetes vs + Portainer vs EC2 vs local). Worked example: a **missing network egress + policy** silently blocks traffic and a developer LLM deep-dives the wrong + debugging path β€” so the SRE persona must carry per-project infra gotchas. +3. **Task lifecycle expanded (Β§6)** β€” `InProgress β†’ Build/Run β†’ Self-Testing β†’ + QA β†’ Security β†’ SRE/Release β†’ Done β†’ AwaitingReview β†’ acceptβ†’Merged / rejectβ†’InProgress`, with stage feedback bouncing back to InProgress. +4. **Script artifacts (Β§11)** β€” a `script` artifact kind (tool-as-a-script, + like a skill but *as code*) at three scopes: task / project / agent. Captures + a procedure once instead of re-deriving it; context-saving via `expand`; + learned-and-reused. +5. **Baseline scorecard / "basics rules agent" (Β§11b)** β€” a provisioned + default list of baseline rules (e.g. "always make unit tests") checked by a + scorecard agent; distinct from Tier-1 QA/Security/SRE and Tier-2 critic β€” a + baseline-behaviour/standards gate. +6. **Artifact term collision (Β§11)** β€” the noun collision resolved: **build-artifact + store** (durable storage for binaries) vs **context-artifact** (lazily-loaded, + expandable object in the LLM window). Two axes, one noun; must not be conflated. +7. **Hotfix motivating example (Β§4)** β€” delta-check vs full-rescan economic + argument for memory + coverage ledger. +8. **Collaboration modes (Β§5)** β€” standup (on-demand, artifact-driven), joint + debugging/architecting, present architectural decisions, ask-the-human-for-help, + work-alone-when-confident. +9. **Consolidated open questions (Β§15)** β€” now 16 items, including brain + ownership, naming (LE-7), prompt delivery, exec carve-out, Epic vs epictask + done, Epic-as-resource, comparison unit, accept-tradeoffs structure, artifact + storage, merge recovery, dashboard surface, Tier-2 media-native infra, MVP + order, script-artifact design, incentive & scorecard. +10. **MVP path (Β§16)** β€” 8 steps now include the vision layer (Epic/epictask/task) + first, then runtime adapter, lifecycle + QA/Security/SRE, Tier-2 critic + + Epic-comparison, collaboration modes, confidence/dashboard, memory/coverage + + context-artifacts, then script artifacts + baseline scorecard. diff --git a/docs/vision-fidelity.md b/docs/vision-fidelity.md new file mode 100644 index 0000000..0098980 --- /dev/null +++ b/docs/vision-fidelity.md @@ -0,0 +1,222 @@ +# Vision Fidelity β€” why agents hand you a plank with a steering wheel, and how to fix it + +> **Status:** Discussion / design brainstorm (no code commitments). +> **Purpose:** Capture the recurring failure β€” "we split a grand vision into +> tasks, completed every task, and got *parts* instead of a *whole*" β€” diagnose +> the root causes, survey what the field already knows, and shape a concrete +> answer that fits mcpctl's existing `agentic-teams` architecture. +> +> This connects directly to `docs/agentic-teams.md`. The key claim: that doc +> designed *technical correctness* verification very well (confidence signal, +> verification sessions, acceptance gate) but **never addressed vision-fidelity, +> product, or design verification** β€” the exact axis this failure lives on. + +--- + +## 1. The failure, named precisely + +You said it almost perfectly. The meat: + +> 1. you had a grand vision +> 2. we split the grand vision into tasks +> 3. we completed all the tasks +> 4. we are left with a stick with wheels instead of a car + +This is **not an execution problem**. Every task *was* completed. The failure is +upstream (decomposition) and downstream (verification): + +- **Decomposition is lossy.** Splitting "build a car" into tasks compresses a + rich, multidimensional vision into flat imperatives ("attach wheels", + "install steering column"). The *why* and the *what-good-looks-like* don't fit + in the task form, so they get dropped. Each split is a lossy compression + step, and the errors compound. +- **Tasks are auto-graded on the wrong axis.** The generated checks are + mechanical ("compiles", "lint passes", "renders"). Nothing checks "does this + serve the vision / is this genuinely designed / does this feel finished." + Minimum-viable-task becomes the ceiling because that's all that's verified. +- **Integration is nobody's job.** A car is not wheels + chassis + steering + + engine *as a list* β€” it's those things *working together*. Task systems + execute a list; "the thing that happens when it all comes together" is an + afterthought task or nothing at all. +- **No feedback loop mid-flight.** The agent finishes task 3 and starts task 4 + assuming task 3 was right. Nothing re-contextualizes: "here is the original + vision, here is what you actually built β€” do they still match?" + +--- + +## 2. Root-cause mechanisms + +### M1. The "task" is a lossy channel +A vision is high-dimensional (function, feel, aesthetic, UX, business intent, +constraints). A task is low-dimensional (one imperative sentence + maybe a +check). Information *has* to be discarded in the split, and standard task +managers don't attach a pointer back to the vision β€” so the discard is +unrecoverable later. + +### M2. Verification defaults to the mechanical tier only +Two-tier "definition of done" is rare. Most DoD = "does it compile / pass." +The second tier β€” "is it right *for the thing we're building*" β€” is either +absent or not media-native (you can't judge a UI by reading code). + +### M3. Design/product review is a phase, not a gate +Graphic design, if present at all, is bolted on at the end ("make it pretty +after it works"). That's the wrong place β€” by then the *structure* that design +informs is already baked, and you're only re-skinning a wrong-shaped thing. + +### M4. No "whole-system" (integration) gate +Nothing periodically assembles the partial results, runs the thing, and asks +"is this converging toward a car, or drifting toward a collection of parts?" + +### M5. The vision isn't a living artifact +The vision lives in the first prompt, then decays. There's no versioned, +mutating vison document that tasks reference by version, so drift isn't even +*detectable* β€” there's nothing authoritative to diff against. + +--- + +## 3. What the field already knows (research synthesis) + +### 3a. Vertical slices / walking skeleton β€” the single most relevant idea +Software architecture has spent decades on *your exact metaphor*. The core +anti-pattern to "steering wheel on a plank" is: + +- **Walking skeleton** β€” build the thinnest *vertical* slice that touches every + layer of the whole, end-to-end, FIRST. Not "car engine" then "car chassis" + (horizontal layers), but "a single car that moves one meter" (vertical slice), + then grow it. +- **Vertical slices** β€” every increment delivers a *user-visible, working* slice + across all layers, not a completed layer. This is the Webnesday/Agile/DASA + pattern, and it directly answers your #4. + +**Implication: don't decompose the vision into tasks by layer.** Decompose into +**vertical slices, each of which is a self-contained mini-car.** Then "car" is +never far away β€” you always have *a* car, just a small one you grow. + +### 3b. Definition of Done, two-tiered +The DoD concept from Scrum already says "a task isn't done until *all* criteria +(NOT just code) pass." The field reinforces: split DoD into +- **Tier 1 mechanical** (builds, tests pass, runs) and +- **Tier 2 systemic** (matches vision, integrates, is designed, feels finished). + +Tier 2 is the one nobody implements, and the one your failure needs. + +### 3c. Human-in-the-loop as a *review gate*, not per-tool approval +Your own `agentic-teams.md` already reached the industry-validated conclusion: +avoid fine-grained per-tool approval; instead gate at the *task outcome* with a +human accept/reject + comment. The gap it didn't close: the accept/reject is a +*binary + comment*. For vision fidelity, the review gate must be able to say +**what to change and why**, and ideally be **media-native** (look at the render, +not just the diff). + +### 3d. Spec-driven development +When a system has a spec, tasks are checkable against the spec, not against a +vibe-diff. For vision fidelity, the "spec" is a **vision document with +acceptance criteria expressed as observable behaviors**, so the second tier of +DoD has something concrete to check against. + +### 3e. The "critic" / adversarial agent pattern +Multi-agent research and practice (including the "GPT critic", red-team agents, +and design-review agents) show that a **separate, non-executing agent that +reviews against the vision** catches fidelity drift that the working agent's own +confirmation bias misses. This maps cleanly onto your `agentic-teams` persona +model: a **Design/Product critic persona** that *doesn't* execute tasks but +reviews them. + +### 3f. Media-native verification +A core, under-appreciated principle: **judge an artifact in its native medium.** +- UI β†’ render / screenshot / record β†’ *look* at it +- Logic β†’ scenario playthrough +- Product β†’ user-journey walkthrough +- Motion β†’ watch it + +"Graphic design session" from your hypothesis is really this: a *verification +modality*, not a phase. You can't code-review your way to "feels designed." + +--- + +## 4. The proposed shape (how I'd answer your hypothesis) + +You guessed "a better task management system + a design session per task." I +agree with 80% but I'd reframe the 20% that matters most: + +> **The fix is not "better task management." It's (a) decomposing into vertical +> slices instead of layers, and (b) making vision-fidelity a first-class, +> media-native *review gate* β€” not a phase and not a binary accept/reject.** + +### Design principle set + +1. **Vision is a versioned, living artifact.** + A `VISION.md` per project, versioned. Tasks reference *vision version N*. + Drift = diff artifact-vs-vision, which is now detectable. (This maps to + mcpctl's `proposal`/`revision`/`prompt` resource family β€” revisions are + exactly the append-only, semver-bumping log you already built.) + +2. **Decompose into vertical slices, not layers.** + Each slice = a self-contained mini-car: touches enough layers to be + user-visible and testable end-to-end. Every increment leaves you with *a* + working (small) whole, never a pile of parts. The "car" is always present, + just small. + +3. **Two-tier DoD on every task/slice.** + - Tier 1: mechanical (builds, tests, runs) + - Tier 2: systemic (matches vision vN, integrates with siblings, is + *designed*, feels finished β€” each with concrete observable criteria). + Tier 2 is **evaluated by running the medium**, not reading code. + +4. **A Design/Product critic persona (no execute β€” only review).** + Add a persona to `agentic-teams` that does *not* edit code. Its job: after a + slice reaches `AwaitingReview`, it plays the artifact in its medium + (render/view/run), compares against VISION.md vN, and returns a **verdict + with specifics** (what to change + why), not a bare pass/fail. This slots + into the existing lifecycle as a review session between + `SelfCheck β†’ AwaitingReview`. + +5. **Integration (whole-system) gate on a cadence.** + Every N slices, run an assembly review: build all merged slices, run the + thing, ask "is this a car yet? what's the next smallest thing that makes it + more of a car?" Feed the answer back as new slices. This answers your + failure #4 directly. + +6. **Review gate feedback is directional, not binary.** + Rejecting isn't "no." It's "here's VISION vN, here's what you built, here's + the specific delta." Fed back into the task as rework, exactly like your + existing `reject β†’ InProgress (comment fed back)` β€” but with a *richer, + media-native, design-specific* comment. + +--- + +## 5. How this maps onto mcpctl / agentic-teams + +| Your existing piece | The missing piece we add | +|---|---| +| `Task` entity + lifecycle | Two-tier DoD stored on the Task; `slice` > `layer` framing | +| Verification sessions (risk-scaled) | **Media-native** verification; Tier-2 check runs the artifact, not just tests | +| Confidence signal / escalation | Confidence ALSO scores vision-fidelity, not just "will it merge cleanly" | +| Persona model (QA, security engineers) | New **Design/Product critic** persona (review-only, non-executing) | +| Accept/reject gate + comment | **Directional, media-anchored** feedback ("here's the delta vs vision vN"), not binary | +| `revision` resource (semver append-only) | Vision as a live, versioned artifact created/managed via the same mechanism | +| `proposal` queue / dashboard | Vision-fidelity escalations surface on the same dashboard as other QA | + +**Net:** the platform already has the bones of the answer. The single most +important *missing* concept is a **review-only persona whose job is to judge +the assembled artifact against a living vision document, in its native +medium, with directional feedback** β€” plus **decomposition into vertical slices** +so you're always a small car away from the real one, never a pile of parts. + +--- + +## 6. Open questions to discuss + +1. **Slices vs tasks in Task Master** β€” does Task Master's + `expand`/dependency model support the *vertical-slice* decomposition, or + does its default push you toward layers? Worth testing with a real example + ("build a car" β†’ what tasks does it actually emit today?) +2. **Where does the Design/Product critic live** in the lifecycle β€” always, or + risk-gated like QA/security? +3. **How to make Tier-2 verification media-native in a headless agent** β€” do we + need render/screenshot infra, or can a design-aware persona judge from a + structured artifact without pixels? +4. **Vision as a resource** β€” is `prompt`/`revision` the right home for + VISION.md, or does it deserve its own resource type? +5. **Cost** β€” a review-only persona and integration gates add tokens/time. Where + is it worth it (product/UI work) vs. where it's overkill (pure backend APIs)?