223 lines
11 KiB
Markdown
223 lines
11 KiB
Markdown
|
|
# Vision Fidelity — why agents hand you a plank with a steering wheel, and how to fix it
|
||
|
|
|
||
|
|
> **Status:** Discussion / design brainstorm (no code commitments).
|
||
|
|
> **Purpose:** Capture the recurring failure — "we split a grand vision into
|
||
|
|
> tasks, completed every task, and got *parts* instead of a *whole*" — diagnose
|
||
|
|
> the root causes, survey what the field already knows, and shape a concrete
|
||
|
|
> answer that fits mcpctl's existing `agentic-teams` architecture.
|
||
|
|
>
|
||
|
|
> This connects directly to `docs/agentic-teams.md`. The key claim: that doc
|
||
|
|
> designed *technical correctness* verification very well (confidence signal,
|
||
|
|
> verification sessions, acceptance gate) but **never addressed vision-fidelity,
|
||
|
|
> product, or design verification** — the exact axis this failure lives on.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## 1. The failure, named precisely
|
||
|
|
|
||
|
|
You said it almost perfectly. The meat:
|
||
|
|
|
||
|
|
> 1. you had a grand vision
|
||
|
|
> 2. we split the grand vision into tasks
|
||
|
|
> 3. we completed all the tasks
|
||
|
|
> 4. we are left with a stick with wheels instead of a car
|
||
|
|
|
||
|
|
This is **not an execution problem**. Every task *was* completed. The failure is
|
||
|
|
upstream (decomposition) and downstream (verification):
|
||
|
|
|
||
|
|
- **Decomposition is lossy.** Splitting "build a car" into tasks compresses a
|
||
|
|
rich, multidimensional vision into flat imperatives ("attach wheels",
|
||
|
|
"install steering column"). The *why* and the *what-good-looks-like* don't fit
|
||
|
|
in the task form, so they get dropped. Each split is a lossy compression
|
||
|
|
step, and the errors compound.
|
||
|
|
- **Tasks are auto-graded on the wrong axis.** The generated checks are
|
||
|
|
mechanical ("compiles", "lint passes", "renders"). Nothing checks "does this
|
||
|
|
serve the vision / is this genuinely designed / does this feel finished."
|
||
|
|
Minimum-viable-task becomes the ceiling because that's all that's verified.
|
||
|
|
- **Integration is nobody's job.** A car is not wheels + chassis + steering +
|
||
|
|
engine *as a list* — it's those things *working together*. Task systems
|
||
|
|
execute a list; "the thing that happens when it all comes together" is an
|
||
|
|
afterthought task or nothing at all.
|
||
|
|
- **No feedback loop mid-flight.** The agent finishes task 3 and starts task 4
|
||
|
|
assuming task 3 was right. Nothing re-contextualizes: "here is the original
|
||
|
|
vision, here is what you actually built — do they still match?"
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## 2. Root-cause mechanisms
|
||
|
|
|
||
|
|
### M1. The "task" is a lossy channel
|
||
|
|
A vision is high-dimensional (function, feel, aesthetic, UX, business intent,
|
||
|
|
constraints). A task is low-dimensional (one imperative sentence + maybe a
|
||
|
|
check). Information *has* to be discarded in the split, and standard task
|
||
|
|
managers don't attach a pointer back to the vision — so the discard is
|
||
|
|
unrecoverable later.
|
||
|
|
|
||
|
|
### M2. Verification defaults to the mechanical tier only
|
||
|
|
Two-tier "definition of done" is rare. Most DoD = "does it compile / pass."
|
||
|
|
The second tier — "is it right *for the thing we're building*" — is either
|
||
|
|
absent or not media-native (you can't judge a UI by reading code).
|
||
|
|
|
||
|
|
### M3. Design/product review is a phase, not a gate
|
||
|
|
Graphic design, if present at all, is bolted on at the end ("make it pretty
|
||
|
|
after it works"). That's the wrong place — by then the *structure* that design
|
||
|
|
informs is already baked, and you're only re-skinning a wrong-shaped thing.
|
||
|
|
|
||
|
|
### M4. No "whole-system" (integration) gate
|
||
|
|
Nothing periodically assembles the partial results, runs the thing, and asks
|
||
|
|
"is this converging toward a car, or drifting toward a collection of parts?"
|
||
|
|
|
||
|
|
### M5. The vision isn't a living artifact
|
||
|
|
The vision lives in the first prompt, then decays. There's no versioned,
|
||
|
|
mutating vison document that tasks reference by version, so drift isn't even
|
||
|
|
*detectable* — there's nothing authoritative to diff against.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## 3. What the field already knows (research synthesis)
|
||
|
|
|
||
|
|
### 3a. Vertical slices / walking skeleton — the single most relevant idea
|
||
|
|
Software architecture has spent decades on *your exact metaphor*. The core
|
||
|
|
anti-pattern to "steering wheel on a plank" is:
|
||
|
|
|
||
|
|
- **Walking skeleton** — build the thinnest *vertical* slice that touches every
|
||
|
|
layer of the whole, end-to-end, FIRST. Not "car engine" then "car chassis"
|
||
|
|
(horizontal layers), but "a single car that moves one meter" (vertical slice),
|
||
|
|
then grow it.
|
||
|
|
- **Vertical slices** — every increment delivers a *user-visible, working* slice
|
||
|
|
across all layers, not a completed layer. This is the Webnesday/Agile/DASA
|
||
|
|
pattern, and it directly answers your #4.
|
||
|
|
|
||
|
|
**Implication: don't decompose the vision into tasks by layer.** Decompose into
|
||
|
|
**vertical slices, each of which is a self-contained mini-car.** Then "car" is
|
||
|
|
never far away — you always have *a* car, just a small one you grow.
|
||
|
|
|
||
|
|
### 3b. Definition of Done, two-tiered
|
||
|
|
The DoD concept from Scrum already says "a task isn't done until *all* criteria
|
||
|
|
(NOT just code) pass." The field reinforces: split DoD into
|
||
|
|
- **Tier 1 mechanical** (builds, tests pass, runs) and
|
||
|
|
- **Tier 2 systemic** (matches vision, integrates, is designed, feels finished).
|
||
|
|
|
||
|
|
Tier 2 is the one nobody implements, and the one your failure needs.
|
||
|
|
|
||
|
|
### 3c. Human-in-the-loop as a *review gate*, not per-tool approval
|
||
|
|
Your own `agentic-teams.md` already reached the industry-validated conclusion:
|
||
|
|
avoid fine-grained per-tool approval; instead gate at the *task outcome* with a
|
||
|
|
human accept/reject + comment. The gap it didn't close: the accept/reject is a
|
||
|
|
*binary + comment*. For vision fidelity, the review gate must be able to say
|
||
|
|
**what to change and why**, and ideally be **media-native** (look at the render,
|
||
|
|
not just the diff).
|
||
|
|
|
||
|
|
### 3d. Spec-driven development
|
||
|
|
When a system has a spec, tasks are checkable against the spec, not against a
|
||
|
|
vibe-diff. For vision fidelity, the "spec" is a **vision document with
|
||
|
|
acceptance criteria expressed as observable behaviors**, so the second tier of
|
||
|
|
DoD has something concrete to check against.
|
||
|
|
|
||
|
|
### 3e. The "critic" / adversarial agent pattern
|
||
|
|
Multi-agent research and practice (including the "GPT critic", red-team agents,
|
||
|
|
and design-review agents) show that a **separate, non-executing agent that
|
||
|
|
reviews against the vision** catches fidelity drift that the working agent's own
|
||
|
|
confirmation bias misses. This maps cleanly onto your `agentic-teams` persona
|
||
|
|
model: a **Design/Product critic persona** that *doesn't* execute tasks but
|
||
|
|
reviews them.
|
||
|
|
|
||
|
|
### 3f. Media-native verification
|
||
|
|
A core, under-appreciated principle: **judge an artifact in its native medium.**
|
||
|
|
- UI → render / screenshot / record → *look* at it
|
||
|
|
- Logic → scenario playthrough
|
||
|
|
- Product → user-journey walkthrough
|
||
|
|
- Motion → watch it
|
||
|
|
|
||
|
|
"Graphic design session" from your hypothesis is really this: a *verification
|
||
|
|
modality*, not a phase. You can't code-review your way to "feels designed."
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## 4. The proposed shape (how I'd answer your hypothesis)
|
||
|
|
|
||
|
|
You guessed "a better task management system + a design session per task." I
|
||
|
|
agree with 80% but I'd reframe the 20% that matters most:
|
||
|
|
|
||
|
|
> **The fix is not "better task management." It's (a) decomposing into vertical
|
||
|
|
> slices instead of layers, and (b) making vision-fidelity a first-class,
|
||
|
|
> media-native *review gate* — not a phase and not a binary accept/reject.**
|
||
|
|
|
||
|
|
### Design principle set
|
||
|
|
|
||
|
|
1. **Vision is a versioned, living artifact.**
|
||
|
|
A `VISION.md` per project, versioned. Tasks reference *vision version N*.
|
||
|
|
Drift = diff artifact-vs-vision, which is now detectable. (This maps to
|
||
|
|
mcpctl's `proposal`/`revision`/`prompt` resource family — revisions are
|
||
|
|
exactly the append-only, semver-bumping log you already built.)
|
||
|
|
|
||
|
|
2. **Decompose into vertical slices, not layers.**
|
||
|
|
Each slice = a self-contained mini-car: touches enough layers to be
|
||
|
|
user-visible and testable end-to-end. Every increment leaves you with *a*
|
||
|
|
working (small) whole, never a pile of parts. The "car" is always present,
|
||
|
|
just small.
|
||
|
|
|
||
|
|
3. **Two-tier DoD on every task/slice.**
|
||
|
|
- Tier 1: mechanical (builds, tests, runs)
|
||
|
|
- Tier 2: systemic (matches vision vN, integrates with siblings, is
|
||
|
|
*designed*, feels finished — each with concrete observable criteria).
|
||
|
|
Tier 2 is **evaluated by running the medium**, not reading code.
|
||
|
|
|
||
|
|
4. **A Design/Product critic persona (no execute — only review).**
|
||
|
|
Add a persona to `agentic-teams` that does *not* edit code. Its job: after a
|
||
|
|
slice reaches `AwaitingReview`, it plays the artifact in its medium
|
||
|
|
(render/view/run), compares against VISION.md vN, and returns a **verdict
|
||
|
|
with specifics** (what to change + why), not a bare pass/fail. This slots
|
||
|
|
into the existing lifecycle as a review session between
|
||
|
|
`SelfCheck → AwaitingReview`.
|
||
|
|
|
||
|
|
5. **Integration (whole-system) gate on a cadence.**
|
||
|
|
Every N slices, run an assembly review: build all merged slices, run the
|
||
|
|
thing, ask "is this a car yet? what's the next smallest thing that makes it
|
||
|
|
more of a car?" Feed the answer back as new slices. This answers your
|
||
|
|
failure #4 directly.
|
||
|
|
|
||
|
|
6. **Review gate feedback is directional, not binary.**
|
||
|
|
Rejecting isn't "no." It's "here's VISION vN, here's what you built, here's
|
||
|
|
the specific delta." Fed back into the task as rework, exactly like your
|
||
|
|
existing `reject → InProgress (comment fed back)` — but with a *richer,
|
||
|
|
media-native, design-specific* comment.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## 5. How this maps onto mcpctl / agentic-teams
|
||
|
|
|
||
|
|
| Your existing piece | The missing piece we add |
|
||
|
|
|---|---|
|
||
|
|
| `Task` entity + lifecycle | Two-tier DoD stored on the Task; `slice` > `layer` framing |
|
||
|
|
| Verification sessions (risk-scaled) | **Media-native** verification; Tier-2 check runs the artifact, not just tests |
|
||
|
|
| Confidence signal / escalation | Confidence ALSO scores vision-fidelity, not just "will it merge cleanly" |
|
||
|
|
| Persona model (QA, security engineers) | New **Design/Product critic** persona (review-only, non-executing) |
|
||
|
|
| Accept/reject gate + comment | **Directional, media-anchored** feedback ("here's the delta vs vision vN"), not binary |
|
||
|
|
| `revision` resource (semver append-only) | Vision as a live, versioned artifact created/managed via the same mechanism |
|
||
|
|
| `proposal` queue / dashboard | Vision-fidelity escalations surface on the same dashboard as other QA |
|
||
|
|
|
||
|
|
**Net:** the platform already has the bones of the answer. The single most
|
||
|
|
important *missing* concept is a **review-only persona whose job is to judge
|
||
|
|
the assembled artifact against a living vision document, in its native
|
||
|
|
medium, with directional feedback** — plus **decomposition into vertical slices**
|
||
|
|
so you're always a small car away from the real one, never a pile of parts.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## 6. Open questions to discuss
|
||
|
|
|
||
|
|
1. **Slices vs tasks in Task Master** — does Task Master's
|
||
|
|
`expand`/dependency model support the *vertical-slice* decomposition, or
|
||
|
|
does its default push you toward layers? Worth testing with a real example
|
||
|
|
("build a car" → what tasks does it actually emit today?)
|
||
|
|
2. **Where does the Design/Product critic live** in the lifecycle — always, or
|
||
|
|
risk-gated like QA/security?
|
||
|
|
3. **How to make Tier-2 verification media-native in a headless agent** — do we
|
||
|
|
need render/screenshot infra, or can a design-aware persona judge from a
|
||
|
|
structured artifact without pixels?
|
||
|
|
4. **Vision as a resource** — is `prompt`/`revision` the right home for
|
||
|
|
VISION.md, or does it deserve its own resource type?
|
||
|
|
5. **Cost** — a review-only persona and integration gates add tokens/time. Where
|
||
|
|
is it worth it (product/UI work) vs. where it's overkill (pure backend APIs)?
|