Proof Shape Matters

1 day ago

Passing a test is useful.

It is not enough.

In an agent merge, the system does not only need to know that something passed. It needs to know what was run, what source it was bound to, what surface it observed, what claim it is allowed to support, and what it did not measure.

Proof shape matters because a vague pass can accidentally become a broad claim.

CHANGE

SLICE

LAYOUT

FOCUS

ARIA

EVENTS

PROBE

EVIDENCE

A runtime proof should bind source, command, probe, observed signals, and claim boundary before admission.

A Pass Is Not A Claim

This is too weak:

tests passed
browser looked fine
layout unchanged
runtime proof attached

Each sentence hides the important part.

Which tests passed? Against which output? With which viewport? Which selector? Which event path? Which source hashes? Which browser state? Which claims are still false?

The merge system should not have to infer that from a transcript.

It should receive a shaped proof.

The Capsule

A runtime proof capsule is the envelope around the observation.

Output-Bound Capsule

Command: playwright test save-button.spec.tsThe exact way the observation was produced.
Probe: button:save:event-runtimeThe named surface the command was checking.
Source binding: base, worker, head, output hashesThe proof belongs to four concrete source states.
Telemetry: DOM, style, layout, event, focus, accessibilityStructured observations, not only a screenshot.
Limit: desktop fixture, save flow onlyThe proof does not become whole-app equivalence.
Route: admit runtime claim for this boundaryAdmission is scoped to the measured surface.

A proof capsule is useful because it is narrow enough to trust and explicit enough to reject.

The capsule is not ceremony.

It prevents the system from treating one observation as a universal runtime guarantee.

Evidence Needs Coordinates

Runtime evidence should have coordinates:

source path
base hash
worker hash
head hash
output hash
command
probe id
evidence hash
required signals
observed snapshots
claim flags

Without coordinates, evidence drifts.

A browser trace from the worker output might be true and still irrelevant to the final merged output. A fixture might prove a component in isolation and still not prove the route shell. A screenshot might show the right pixels and still miss focus order or accessible names.

Coordinates make those limits visible.

Proof Strength

Different observations support different claims.

SurfaceClaimRequired proofCurrent evidenceRoute

ProvedCommandThe proof is replayableExact command and environmentCommand, probe id, fixture modeAccept as evidence object

RequiredSourceThe proof belongs to this mergeFour source hashesBase, worker, head, output bindingBind to candidate

ProvedBrowserThe measured behavior heldStructured runtime telemetryDOM, computed style, layout, event, focus, a11yAdmit scoped runtime claim

MissingWhole productEverything is equivalentUnbounded product oracleNot providedKeep broad claim false

A shaped proof lets the system admit the narrow claim while refusing the broad one.

The important distinction is not pass or fail.

It is what the pass is allowed to mean.

The Mental Model

A proof is not a note that says the work is good.

It is a bounded object that says:

this exact output
under these exact conditions
produced these observations
for this exact claim
and no larger claim

When proof shape is explicit, semantic merge can trust more without guessing more.

SHAPE

SHIFT