231 lines
11 KiB
Markdown
231 lines
11 KiB
Markdown
|
|
# Session Summary — Vision Fidelity in Agentic Teams (full discussion)
|
||
|
|
|
||
|
|
> **Status:** Consolidated session record — captures everything discussed, in
|
||
|
|
> order, so it reads as one coherent thread.
|
||
|
|
> **Date:** 2026-08-08 · pi session · mcpctl repo
|
||
|
|
>
|
||
|
|
> **IMPORTANT — canonical doc:** since this summary was written,
|
||
|
|
> `agentic-teams.md` was updated and is now the **single authoritative, unified
|
||
|
|
> design document** that **supersedes** this summary plus `vision-fidelity.md`,
|
||
|
|
> `epic-task-artifact-model.md` and `agentic-teams-review.md`. Read
|
||
|
|
> `agentic-teams.md` for the source of truth; this summary is the session
|
||
|
|
> narrative and highlights the sections the canonical doc added beyond it.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Part 1 — The problem statement
|
||
|
|
|
||
|
|
We started with a recurring failure with LLM coding agents:
|
||
|
|
|
||
|
|
> 1. you had a grand vision
|
||
|
|
> 2. we split the grand vision into tasks
|
||
|
|
> 3. we completed all the tasks
|
||
|
|
> 4. we are left with a stick with wheels instead of a car
|
||
|
|
|
||
|
|
Every task completed, yet no car. The diagnosis (see `vision-fidelity.md` for
|
||
|
|
the full write-up): **this is not an execution problem — it's decomposition +
|
||
|
|
verification.**
|
||
|
|
|
||
|
|
### Root causes
|
||
|
|
1. **Decomposition is lossy** — a multidimensional vision compresses into flat
|
||
|
|
imperatives; the *why* and *what-good-looks-like* get dropped.
|
||
|
|
2. **Tasks are graded mechanically** — "compiles / lint / renders," never "does
|
||
|
|
this serve the vision / is this designed / does this feel finished."
|
||
|
|
3. **Integration is nobody's job** — a car is parts *working together*, but
|
||
|
|
task systems execute a list.
|
||
|
|
4. **No feedback loop** — the agent assumes task 3 was fine and never re-checks
|
||
|
|
against the original vision.
|
||
|
|
5. **Design/product review is a phase, not a gate** — applied at the end, when
|
||
|
|
the wrong shape is already baked.
|
||
|
|
|
||
|
|
### Research synthesis (from the write-up)
|
||
|
|
- **Vertical slices / walking skeleton** — decompose into `mini-wholes`
|
||
|
|
(user-visible, end-to-end slices), not layers. Always have *a* small car, not
|
||
|
|
a pile of parts.
|
||
|
|
- **Two-tier Definition of Done** — mechanical tier + systematic tier (matches
|
||
|
|
vision, integrates, is designed, feels finished). Tier-2 evaluated by running
|
||
|
|
the medium, not reading code.
|
||
|
|
- **Review-gate HITL over per-tool approval** — consistent with round-2.
|
||
|
|
- **Spec-driven + media-native verification** — judge an artifact in its native
|
||
|
|
medium (render/run for UI).
|
||
|
|
- **Critic-agent pattern** — a non-executing persona that reviews against the
|
||
|
|
vision catches drift the working agent's confirmation bias misses.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Part 2 — Grounding in the repo (the "other window")
|
||
|
|
|
||
|
|
We confirmed this connects to real, existing machinery:
|
||
|
|
|
||
|
|
- **mcpctl agent platform** — persistent agent personas, project prompts,
|
||
|
|
`ResourceRevision` (versioned log), `ResourceProposal` (review/approve
|
||
|
|
queue), personalities (same agent, different judgment via prompt overlays).
|
||
|
|
- **gstack skills** (installed skill library) — `/autoplan` (auto-review
|
||
|
|
pipeline: CEO → Design → Eng → DX), `/plan-ceo-review` (grand-vision
|
||
|
|
guardian, nuclear scope challenge, 10x vision), `/plan-design-review`
|
||
|
|
(rates each design dimension 0-10, generates visual mockups, "text
|
||
|
|
descriptions are not a substitute for showing what it looks like"), `/spec`.
|
||
|
|
- **`agentic-teams.md` + `agentic-teams-review.md`** — the durable design
|
||
|
|
record, landed on **async, artifact-driven, human-as-fuse** (section 1 of
|
||
|
|
the review round is the through-line).
|
||
|
|
|
||
|
|
**Key realization:** the platform reviews the *plan* hard before build, but
|
||
|
|
**nothing reviews the *built result* against the vision after implementation**,
|
||
|
|
and **no vision artifact exists as a first-class, versioned resource**. That's
|
||
|
|
the gap today's design fills.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Part 3 — Where we landed: the Epic model
|
||
|
|
|
||
|
|
We reframed VISION.md → **Epic**, in Jira nomenclature — and it's the thing we
|
||
|
|
discussed in *this* window's second half. Full spec in
|
||
|
|
`epic-task-artifact-model.md`.
|
||
|
|
|
||
|
|
### The Epic is the vision
|
||
|
|
- The **back-and-forth discussion before coding starts** *is* the vision; the
|
||
|
|
Epic captures its conclusion. It's project-scoped.
|
||
|
|
- Holds the **agreed "done" criteria** (checkable outcomes + intent/quality
|
||
|
|
attributes) written at vision time, plus scope-change history (revisions) and
|
||
|
|
**accepted-tradeoff decisions**.
|
||
|
|
|
||
|
|
### Epic lifecycle — "done" only two ways
|
||
|
|
An Epic reaches **done** only when:
|
||
|
|
1. **100% of the vision is delivered**, OR
|
||
|
|
2. The **user explicitly accepted tradeoffs** (e.g. an issue we couldn't fix) —
|
||
|
|
recorded on the Epic, so done is honest + auditable.
|
||
|
|
|
||
|
|
**The comparison moment is the `exec` trigger.** Delivered-whole vs Epic → gaps
|
||
|
|
route to:
|
||
|
|
- **reopen** under-delivering tasks, or
|
||
|
|
- **new tasks** for never-decomposed pieces, or — when it's a hard "we
|
||
|
|
couldn't fix X" — a **joint `exec` debugging session** (the pi / Claude-Code
|
||
|
|
live experience). This gives `exec` a concrete, principled trigger.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Part 4 — Data model: task / epictask / Epic
|
||
|
|
|
||
|
|
Hierarchy (details in `epic-task-artifact-model.md` §2):
|
||
|
|
|
||
|
|
```
|
||
|
|
Epic (the vision)
|
||
|
|
└── epictask — roadmap node; tasks hang off it; chains to other epictasks
|
||
|
|
└── task — the unit of work (own lifecycle from round 1)
|
||
|
|
```
|
||
|
|
|
||
|
|
- **tasks link to epictasks**; **epictasks chain to each other** → visualizing
|
||
|
|
all epictasks gives a **spider-web roadmap** (dependency graph, not flat).
|
||
|
|
- Walking any task up the chain reaches the Epic's "done" criteria, so
|
||
|
|
**tier-2 / vision DoD is reachable from every task**.
|
||
|
|
|
||
|
|
### Reopen vs new-task
|
||
|
|
| Gap cause | Outcome |
|
||
|
|
|---|---|
|
||
|
|
| A task under-delivered its acceptance criteria | **Reopen** (directional comment back) |
|
||
|
|
| A piece was never decomposed (breakdown gap) | **New task** under the epictask |
|
||
|
|
| Hard problem / needs joint effort | **`exec` joint session** |
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Part 5 — Artifacts + IPython-style context (prime-agent philosophy)
|
||
|
|
|
||
|
|
Borrowed from [prime-agent](https://github.com/PrimeIntellect-ai/prime-agent)'s
|
||
|
|
**"context as variables / prompt-as-a-variable"** (details in §4 of the model
|
||
|
|
doc):
|
||
|
|
|
||
|
|
- **Artifacts are objects, not plain text** — `files|test`, logs, and also
|
||
|
|
**PR & commits** as structured objects with an `expand` operation (not
|
||
|
|
dumped into context).
|
||
|
|
- **"Cat" → variable/artifact** — a named, retrievable reference, **not
|
||
|
|
auto-injected**.
|
||
|
|
- **Rule: context does NOT auto-populate from artifact content.**
|
||
|
|
|
||
|
|
### Eviction / flush priority (recoverability-driven)
|
||
|
|
1. **User's words** — highest retention.
|
||
|
|
2. **Agent's words** — next.
|
||
|
|
3. **Artifact context** — evict first, *because artifacts are durably stored
|
||
|
|
and recoverable via `expand`*. **Eviction priority follows recoverability.**
|
||
|
|
|
||
|
|
The elegant win: dropping artifact-context first is *cheap* because it's
|
||
|
|
recoverable — most compaction schemes don't formalize this, and it turns a
|
||
|
|
hand-wavy "compaction" into a principled hierarchy.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Part 6 — Open questions / tensions (carried forward)
|
||
|
|
|
||
|
|
1. **Epic as a resource vs a prompt** — own resource type (RBAC + revisions) or
|
||
|
|
reuse project prompt/skill? Lean: own resource.
|
||
|
|
2. **Epic-level vs epictask-level "done"** — should epictasks carry their own
|
||
|
|
acceptance criteria so the spider web shows per-node completeness? Lean: yes.
|
||
|
|
3. **Comparison unit: task-level or epictask-chain-level?** A gap can cross
|
||
|
|
multiple linked epictasks; a single "reopen" may not express "this chain is
|
||
|
|
incomplete." Decide whether comparison happens per-task or per-chain.
|
||
|
|
4. **`exec` as a deliberate carve-out** — the async model is default, but the
|
||
|
|
Epic-gap triggers sync `exec` *by design*. Name this so it doesn't conflict
|
||
|
|
with the round-2 async decision (LE-4).
|
||
|
|
5. **Accept-tradeoffs needs structure** — record *what, why, who, de-scoped-by*
|
||
|
|
as a revision on the Epic.
|
||
|
|
6. **Naming (LE-7)** — `epictask` spelling; and `task` vs repo's `.taskmaster`
|
||
|
|
tasks.
|
||
|
|
7. **Artifact storage** — server-side mcpctl-owned for binaries/builds (per
|
||
|
|
LE-8/LE-5); commits/PRs stay in git/repo and are referenced as objects.
|
||
|
|
8. **Post-build re-verification** — the remaining product-side gap: nothing
|
||
|
|
re-checks the rendered/built result against the reviewed mockup/vision after
|
||
|
|
implementation (the `design-review`/`qa` path is opt-in, not wired into the
|
||
|
|
Epic-comparison loop).
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Referenced artifacts produced today
|
||
|
|
|
||
|
|
| File | Purpose |
|
||
|
|
|---|---|
|
||
|
|
> See `docs/agentic-teams.md` — now the single unified document (supersedes
|
||
|
|
> `vision-fidelity.md`, `epic-task-artifact-model.md`, `agentic-teams-review.md`,
|
||
|
|
> and this summary).
|
||
|
|
|
||
|
|
## Part 7 — What the canonical `agentic-teams.md` added beyond today's thread
|
||
|
|
|
||
|
|
When reviewing the updated `docs/agentic-teams.md` (which folds in everything
|
||
|
|
we discussed), these sections were found to be **new / more evolved** than the
|
||
|
|
session thread above — worth reading there:
|
||
|
|
|
||
|
|
1. **Two-level verification model (§3)** — formalised table: **Tier 1**
|
||
|
|
Correctness/Security/Ops (inside the task lifecycle) vs **Tier 2** Vision
|
||
|
|
Fidelity/Design (at the Epic, media-native). They do not replace each other.
|
||
|
|
2. **SRE/Release review stage (§6)** — Tier-1 stage inside the task loop that
|
||
|
|
reviews **deployment-as-code** (IaC, release manifests, container config)
|
||
|
|
against the project's **actual deployment model/infra** (Kubernetes vs
|
||
|
|
Portainer vs EC2 vs local). Worked example: a **missing network egress
|
||
|
|
policy** silently blocks traffic and a developer LLM deep-dives the wrong
|
||
|
|
debugging path — so the SRE persona must carry per-project infra gotchas.
|
||
|
|
3. **Task lifecycle expanded (§6)** — `InProgress → Build/Run → Self-Testing →
|
||
|
|
QA → Security → SRE/Release → Done → AwaitingReview → accept→Merged / reject→InProgress`, with stage feedback bouncing back to InProgress.
|
||
|
|
4. **Script artifacts (§11)** — a `script` artifact kind (tool-as-a-script,
|
||
|
|
like a skill but *as code*) at three scopes: task / project / agent. Captures
|
||
|
|
a procedure once instead of re-deriving it; context-saving via `expand`;
|
||
|
|
learned-and-reused.
|
||
|
|
5. **Baseline scorecard / "basics rules agent" (§11b)** — a provisioned
|
||
|
|
default list of baseline rules (e.g. "always make unit tests") checked by a
|
||
|
|
scorecard agent; distinct from Tier-1 QA/Security/SRE and Tier-2 critic — a
|
||
|
|
baseline-behaviour/standards gate.
|
||
|
|
6. **Artifact term collision (§11)** — the noun collision resolved: **build-artifact
|
||
|
|
store** (durable storage for binaries) vs **context-artifact** (lazily-loaded,
|
||
|
|
expandable object in the LLM window). Two axes, one noun; must not be conflated.
|
||
|
|
7. **Hotfix motivating example (§4)** — delta-check vs full-rescan economic
|
||
|
|
argument for memory + coverage ledger.
|
||
|
|
8. **Collaboration modes (§5)** — standup (on-demand, artifact-driven), joint
|
||
|
|
debugging/architecting, present architectural decisions, ask-the-human-for-help,
|
||
|
|
work-alone-when-confident.
|
||
|
|
9. **Consolidated open questions (§15)** — now 16 items, including brain
|
||
|
|
ownership, naming (LE-7), prompt delivery, exec carve-out, Epic vs epictask
|
||
|
|
done, Epic-as-resource, comparison unit, accept-tradeoffs structure, artifact
|
||
|
|
storage, merge recovery, dashboard surface, Tier-2 media-native infra, MVP
|
||
|
|
order, script-artifact design, incentive & scorecard.
|
||
|
|
10. **MVP path (§16)** — 8 steps now include the vision layer (Epic/epictask/task)
|
||
|
|
first, then runtime adapter, lifecycle + QA/Security/SRE, Tier-2 critic +
|
||
|
|
Epic-comparison, collaboration modes, confidence/dashboard, memory/coverage +
|
||
|
|
context-artifacts, then script artifacts + baseline scorecard.
|