Verification

An agent changes something and says it works. step.verify checks that from outside the thing making the claim: it says what should become true, performs an action on a real connection, watches what the connection's senses report, and answers yes, no, unknown (could not tell) or no-fault (nothing to prove, nothing wrong) — with the evidence, and with what it could not see.

Quick Example

import { AgentHarness, connection, step } from '@noetic-tools/core';
import { mcp } from '@noetic-tools/mcp';
import { browser } from '@noetic-tools/profile-browser';
import { net, ui, verifier } from '@noetic-tools/verify';

const app = connection('app', browser);

const checkout = step.verify({
  id: 'checkout',
  subject: app,
  intent: 'Paying places an order',
  expect: [net('POST /api/orders 201').once(), ui('heading "Order confirmed"')],
  never: [net('5xx')],
  act: [step.call({ id: 'pay', connection: app, op: 'ui.tap', args: { element: 'button "Pay"' } })],
});

const harness = new AgentHarness({
  name: 'qa',
  params: {},
  connections: { app: mcp({ command: 'npx', args: ['playwright-noetic-mcp'] }) },
  verification: verifier(),
});

const result = await harness.run(checkout, undefined, harness.createContext());
console.log(result.outcome); // 'pass' | 'fail' | 'partial' | 'unknown'

Its output's outcome is pass, fail, partial, or unknown; a workflow branches on it like any value. verify never throws for a no or an unknown — those are answers.

Verification implements the Open Verification Protocol (OVP v1) through its package open-verification, pinned exactly. Noetic does not reinvent its rules: OVP's own adjudicate() decides every verdict, and a verification run is an OVP VerificationRun any OVP tool can read and check.

Installation

npm install @noetic-tools/core @noetic-tools/verify

Register the verifier once on the harness: new AgentHarness({ verification: verifier() }). A harness without one fails a verify step with VERIFY_NOT_CONFIGURED. Core owns the verify step and node and never imports @noetic-tools/verify or open-verification.

Writing a Verify

subject and witnesses take handles from connection() or harness connection names.

Matchers

The matcher builders come from @noetic-tools/verify:

ui, log, route, state, and signal work the same way, and sense('x-orders-db', 'orders.count') builds a matcher on an x- sense ({ "x-orders-db": "orders.count", "count": 1 } in JSON). A matcher for a sense the subject or a witness cannot perceive fails validation with CONNECTION_UNSATISFIED; a matcher the sense's grammar cannot parse fails with VERIFY_INVALID.

In JSON

{ "kind": "verify", "id": "checkout", "subject": "app",
  "expect": [ { "net": "POST /api/orders 201", "count": 1 }, { "ui": "heading \"Order confirmed\"" } ],
  "never":  [ { "net": "5xx" } ],
  "act": [ { "ui.tap": { "element": "button \"Pay\"" } } ] }

Steps in act run on the subject unless they name another connection. See the verify node for the full field list.

The full OVP form

expect and never lower to OVP documents: each expect entry becomes a claim with one assertion (its sense as the channel, its matcher as the predicate, a generated statement), and each never entry a blocking constraint. A verify may give claims and constraints as OVP documents directly instead — for multi-assertion claims or prose a team wants to keep — and both forms validate to the same shape. Give exactly one of expect or claims, and at most one of never or constraints. The OVP types live at @noetic-tools/types/verify, apart from Noetic's own Verdict.

How It Runs

The order is what makes the answer honest:

1. Declare. The claims are recorded before anything happens, so they count as declared before the action. A claim about an action that already ran cannot reach yes.
2. Check before. Each claim is evaluated once against the subject as it is now. If it already holds, a yes afterwards would prove nothing, so the verdict is unknown.
3. Listen. The verify starts listening on the senses its claims need and waits until the server confirms.
4. Act. act runs through the usual gates.
5. Close. Listening stops by close (see Windows).
6. Judge. Each claim is judged by OVP's adjudicate() against only the evidence, gaps, and anomalies on its own senses (see Judging).
7. Record. The run document is stored as a blob in the harness artifact store, attached to the trace, and emitted as a noetic:verification_run item.

interface VerifyOutput {
  outcome: 'pass' | 'fail' | 'partial' | 'unknown';
  verdicts: Array<{ claim: string; verdict: 'yes' | 'no' | 'unknown' | 'no-fault'; ground: string; reasons: string[] }>;
  violations: ConstraintViolation[];
  gaps: BlindSpot[];
  repairs: Repair[];
  run: BlobRef;
}

outcome is OVP's outcomeOf(run): a never that was broken makes it fail whatever the claims say, and anyone recomputing it gets the same answer. If act throws, the run is still recorded and the step fails with verify_act_failed.

With a list subject, the output is { outcome, subjects }: the worst outcome, and each connection's VerifyOutput by name. Give act as a function of the subject, so each verify acts on the connection it watches — a fixed act that names one of the subjects is VERIFY_INVALID:

step.verify({
  id: 'launch',
  subject: ['ios', 'android'],
  expect: [net('GET /api/home 200')],
  act: (subject) => step.call({ id: `open-${subject}`, connection: subject, op: 'app.launch' }),
});

A witness is refused anything with effects while act runs, however the call names its connection — a function or an expression included.

Windows

A window is the stretch of time a verdict is about. It closes by the first of:

close also takes quietMs (default 500) and timeoutMs (default 15e3).

budget-exhausted and terminated are never clean, so neither can support yes or no-fault. A settle that cannot measure quiet — a hidden tab — closes cleanly after quietMs with no events and records a gap on time, rather than failing as a timeout. If the subject is replaced mid-window, the window ends and its evidence is marked as belonging to a world that is gone.

Judging

Each claim gets its own OVP AdjudicationInput:

• evidence — observations on the claim's senses only, from the subject and witnesses;
• coverage — gaps on those senses, plus any the server flags as affecting everything;
• anomalies — those between the claim's senses;
• assertions — the claim's matchers, three-valued: a matcher that cannot be evaluated is undefined, never false;
• before — from step 2.

So a claim about the screen is never proved by an unrelated request, and a problem on log never decides a claim about ui. never entries are checked against every event as it arrives; the first that breaks one records a violation at that moment. A never on a sense the subject can only read, not listen to, adds a gap instead of passing silently.

Gaps

A gap (OVP blind spot) is something the verify could not see. Gaps on a sense a claim needs keep it from yes. They come from the server's noetic://coverage and from the harness itself:

Anomalies

Anomalies — two senses that cannot both be right — come from the server's noetic.detect, from OVP's witnessDisagreement when a witness cannot confirm what the subject claims, and from the harness when evidence belongs to a replaced subject or predates its latest edit. The harness detects nothing else; knowing what a request or a render means is the server's job. A disagreement decides a claim only when one of its senses is independent, which adjudicate() checks.

Witnesses

A witness is a connection that can look and cannot touch: a database, a payment provider's API. Listed in witnesses, it may run only its profile's operations without effects; everything else on it is refused with VERIFY_INVALID. Its senses are x- senses (x-orders-db) declared with their true independence and grade, since OVP's state and storage cannot be declared independent.

import { connection, step } from '@noetic-tools/core';
import { browser } from '@noetic-tools/profile-browser';
import { net, sense } from '@noetic-tools/verify';

const app = connection('app', browser);

const placeOrder = step.verify({
  id: 'place-order',
  subject: app,
  witnesses: ['orders'],
  expect: [net('POST /api/orders 201').once(), sense('x-orders-db', 'orders.count').count(1)],
  act: step.call({ id: 'pay', connection: app, op: 'ui.tap', args: { element: 'button "Pay"' } }),
});

Each witness closes by its own condition — by default, two reads in a row that agree — so replication lag is not mistaken for a missing write. A witness that could not look adds a gap; it is never evidence that something did not happen.

Resume

A verify that stopped before judging reopens its window on resume only if the subject and witnesses kept their identity and every needed event stream can be resumed; otherwise it starts again from Declare under the subject's resume rule. The interrupted attempt stays in the run record.

Under the Hood

@noetic-tools/types/verify defines the Verifier contract and the OVP document types, field for field as open-verification defines them, without importing it. @noetic-tools/verify implements the contract with open-verification. The published workflow JSON Schema inlines OVP's JSON Schemas, so it never points at another host. Senses are translated to OVP channel ids at this boundary.

Errors

Related Pages

• Connections -- the subject and witnesses a verify watches.
• Connection Profiles -- senses, summaries, and determinism.
• JSON Workflow Runtime -- the verify node.
• Open Verification Protocol -- the protocol verdicts follow.