What wisp checks beside a reply

A reply is the model’s account of its turn. wisp sets three checks of its own beside it, each from wisp’s own records, none from the reply’s word: what the turn ran, which entries the reply cites that do not exist, and, this page’s subject, which results the reply reports that no call in the turn produced. The first two are ADR 0051 and ADR 0055; the third is ADR 0060, the verification pass. All three show in the same places: muted under the reply in chat, on the turn’s end line of wisp chat --json (shown by wisp-tui), and in respond’s structuredContent over MCP.

ran: read_file ×6
cited but not in this conversation: entries 19–30 (12)
unsupported: "Ran `inspect(config)` → confirmed model settings" (no call to inspect in this turn) +9 more

The verification pass

After each reply, the pass is given the reply and the turn’s complete call list from the audit log: every tool.call with its arguments, its tool.result or error, and for a command its policy.decision and command.outcome, with each call’s whole output from the output store where the output was kept there (ADR 0050), never the slice the model saw or fetched. It answers one question: does the reply report a result that no call in the list produced? It does not judge whether the reply is right, complete, or wise.

What the readers read

The pass is a set of exact readers first, with no model. Each reads one form a result claim takes and flags it only when the list cannot support it:

A plan is not a claim: Run inspect(config) to verify…, a call form whose result is a placeholder (<scratch>), and a line that says example or e.g. are left alone. Earlier turns’ calls are a second support: a reply reporting what an earlier turn found is not flagged, though the line always says in this turn for what it checked against. Each line of the reply makes at most one claim; a reply yields at most 12.

The model step

With verification.mode set to model, when the readers find nothing and the reply has result language (a call form, or words such as ran, executed, returned, passed, failed, denied, confirmed, →), one side call asks the conversation’s model the same question, with the reply and the call list rendered within 6,000 bytes (the reply up to 2,400; each call’s arguments up to 200 and an output excerpt up to 400, head and tail; the list cut with a note at the budget). It is a fresh session with no tools, greedy sampling, and thinking off for a model that can think (ADR 0053). Its verdict is one claim, quoted, with a reason; the line then says the model found no call producing it. A failed call never fails the turn: the readers’ verdict stands and the failure is recorded. The verdict’s claim is optional: a verified verdict need not name one, and an unsupported verdict that names none is still shown, as the model named no claim with the model’s reason.

It is off by default. Measured on the on-device model on 2026-10-12 it flagged 6 of the 14 honest replies, reading plans, explanations, and “nothing ran as root” as result claims, and took 1 to 5 s a verdict (p50 1.9 s); it turns on by default only when, on the default model, the readers with it flag no honest reply and a verdict takes under 3 s (ADR 0060, “Measured, 2026-10-12”; measurements.md, verification). Turn it on for a more capable model after measuring that model with scripts/check eval compare.

When it runs, and what it costs

Set it with wisp config set verification.mode model or /config set verification.mode model (wisp.md, wisp config). A conversation whose agent has no tool-event trail (one made directly, as tests and evals make it) has no call list and checks nothing.

Where it shows

• Chat prints the line muted under the reply, after ran: and cited, only when a claim is unsupported: the first claim quoted (at most 120 characters), its reason, and +N more for the rest. A verified reply shows nothing; under every reply the line would be noise.
• wisp chat --json carries it as verified on the turn’s end line; wisp-tui shows it after cited.
• respond over MCP returns verification in structuredContent: status (verified or unsupported), method (readers or model), claims (each reader, claim, reason; reader is model for the model step’s), calls checked, and failure when the model step failed; null when nothing was checked (mcp.md). The text the caller’s model reads is unchanged.
• The audit log records response.verification when a check was made (logging.md).

What it misses

A result in other words (I checked the config and it is fine) is no form the readers read, as ran: misses a tool not named. A value the readers cannot read (macOS Ventura 13.6 after sw_vers printed 27.0, session ce87576a turn 9) is the model step’s case. The readers can flag an honest reply that quotes an output loosely enough that no three lines match; the eval’s fixtures hold the honest replies of the same sessions so the measurement counts that.

On the 17 turns of 2026-10-04 that the eval holds as fixtures, the readers alone flag eight of the nine fabricated replies and none of the eight honest ones (ReplyVerificationFixtureTests, in the gate, 2026-10-12). With the 14 hand-written cases, scripts/check eval on the on-device model on 2026-10-12 measured:

The release’s floors are on the readers, which ship on: no honest reply flagged, and at least half the fabricated turns. The model step’s figures become the floor only if it is made the default.