Snapshots and review

A snapshot is the evidence a command retains. It might be a browser screen, a terminal’s output or a set of database rows. Your agent compares that evidence with an earlier run and checks the change against your application’s rules.

Ask your agent:

Add snapshots at the points that explain this journey. Run it, inspect the changed screens and outputs, and fix regressions within the scope of my change. Show me the evidence and anything still unresolved before preparing the PR.

Browser screens

Use a named browser snapshot after the relevant assertion:

site.expect selector="[data-testid=invitation-form]"
site.snapshot name="team-invitation" selector="[data-testid=invitation-form]"

The name identifies the checkpoint across runs. Omit selector for the viewport, or select one visible element with stable bounds. Missing or ambiguous targets fail without a viewport fallback. See the Browser plugin for all capture arguments, timeouts and animation behavior.

Here the invitation form still exists, so the assertion passes. The screenshot shows that its role selector disappeared during a copy change.

reflow / Team invitationsIllustrative example

invite-member.rflLine 5 · selected step

site.open url="/settings/team"

Illustrative flow excerpt and playback. Comments describe retained evidence and review, not extra plugin commands. Select a checkpoint line to inspect its output; preceding lines provide context.

Output · selected step

Change Summary

Update invitation copy

1 changed screen

invite-memberPassed

team-invitation/settings/team

Beforemain

Afterthis change

Snapshot delta · team-invitation Before: role selector present. After: role selector removed.

The test passed. The screen changed.site.snapshot

Agent reviewOutside scope

The role selector disappeared.

This PR changes invitation copy. Owners still need to choose a role before sending an invitation.

team-invitationteam-access.md

Command output

LocalBash and DockerUbuntu retain command results, including standard output, standard error and exit status. Wrap an application CLI command in a flow:

worker.bash command="./bin/fulfill order-1042"

Command output is retained automatically, without a terminal screenshot command. Here both versions exit successfully, but one stops returning tracking information. See LocalBash or DockerUbuntu for connection settings, command arguments and file capture.

Bash · fulfillment workerIllustrative example

fulfill-order.rflLine 4 · selected step

Illustrative flow excerpt and playback. Comments describe retained evidence and review, not extra plugin commands. Select a checkpoint line to inspect its output; preceding lines provide context.

Output · selected step

fulfill-order.mdLocalBash

worker.bash
command="./bin/fulfill order-1042"

Your application command. Its result becomes a snapshot.

Beforeexit 0

stdout.txt order: 1042 status: dispatched

tracking_number:
TRK-1042

stderr.txtempty

Afterexit 0

stdout.txt order: 1042 status: dispatched

tracking_numberMissing from output

stderr.txtempty

State and evidence changed

Exit code: 0 → 0
Standard output: Changed

Both commands succeeded. The saved output differs.

Agent’s review note

“The worker still reports ‘dispatched’, but no longer returns a tracking number. Can the shipment notification still include a tracking link?”

Evidence: before + after / stdout.txt

Database rows

A query can capture the business state behind the screen:

db.query sql="SELECT id, status, refunded FROM orders ORDER BY id" key="id" checkpoint="orders"

Use a unique, non-null key for stable row ordering. The snapshot retains query rows; your agent relates the difference to the application’s refund rules. See the PostgreSQL plugin for setup, limits and write controls.

reflow / PostgreSQL · refund checkIllustrative example

verify-refund.rflLine 4 · selected step

Illustrative flow excerpt and playback. Comments describe retained evidence and review, not extra plugin commands. Select a checkpoint line to inspect its output; preceding lines provide context.

Output · selected step

verify-refund.mddb: PostgreSQL

db.query sql="SELECT id, status, refunded
 FROM orders ORDER BY id"
key="id" checkpoint="orders"

orders / rows.jsonEarlier runCurrent run

id key	status	refunded
1041	paid	0.00
1042	refunded	25.0020.00
1043	paid	0.00

3 rows retained · amounts in USD

Retained before → after1042

refunded25.0020.00

Agent review · examplerefunds.md

Full refunds return the entire $25.00 payment.

The order says refunded. $5.00 is missing. The captured rows show $20.00 returned. Check the refund amount before merging.

Choose the evidence that matters

Command	Retained evidence
`site.snapshot name="…" selector="…"`	A named screenshot of one element; omit `selector` for the viewport.
`worker.bash command="…"`	Command output, errors and exit status.
`db.query … checkpoint="…"`	Query result rows.
Custom plugin command	The state and artifacts declared by that provider.

These are command excerpts. Declare aliases and binding names in your flow, and initialize the application plugin lock; see Write a flow. Use list_plugins to inspect the installed command schemas. Every plugin invocation returns a terminal snapshot, but only some commands capture a screen or a named review checkpoint. A click’s result does not replace an intentional screenshot of the outcome.

All visual examples on this page are illustrations. Reflow retains the exact originals from your actual execution; a comparison must not substitute an illustration or silently overwrite them.

Run, review and repair

reflow plan
reflow apply
reflow status
reflow dashboard

When earlier steps or covered source files change, the next plan automatically includes affected steps and their dependents. Apply runs that work. An unchanged, completed step can reuse its evidence; you do not need to taint it after every edit.

If the external application changed without a source or input change, request fresh evidence with reflow taint flow-name, then plan and apply. Your agent can select flow-name#step-name to start from a particular step; Reflow also includes affected dependents and the steps needed to reconstruct a browser session.

A difference is a question for your agent to investigate. It can update an obsolete test or fix a product regression, rerun the affected work and show the resulting delta. See Repair and rerun for that workflow.

Compare two runs from the CLI

A passing run can still show a broken screen. To compare the captures of a base run and a head run (or a head run against an earlier run you accepted), run:

reflow evidence diff <base-run-id> <head-run-id> --output-dir /tmp/evidence-diff

Captures pair by their retained path in the run. Screenshots compare pixel by pixel: an image is changed when more than 0.5% of its pixels differ, or when a dense region changed under that floor (one changed text line or number). The result lists the changed regions largest first, and --output-dir writes base, head and diff images. With A/A controls, a glyph-sized cluster of 12–63 changed pixels beyond and away from measured noise also counts as a small change. With two distinct same-size control images, several smaller strokes can qualify together when no existing change floor was reached: at least three compact 6–11-pixel components, at least 24 changed pixels in a 144×144 capture-pixel neighbourhood. Each component must span at least two pixels in each direction, fill at least a quarter of its box, and remain away from measured noise. This can reveal edits to small scale marks without lowering the standalone cluster floor. Isolated strokes, sparse speckles and highly fragmented images (more than 256 candidate strokes) do not qualify for this group rule. Pixel thresholds, counts and the measured noise mask remain unchanged. When a changed accessibility snapshot also contains lines whose reported node areas kept their pixels in its paired window capture, those lines appear below the remaining changes under changes with unchanged reported AX node areas. Their exact text remains listed, with visibility explicitly unconfirmed: virtual AX frames may not locate the actual glyphs, so changed text can be drawn elsewhere in the paired image. This observation proves neither that the text is drawn nor that it is absent. Unchanged area pixels alone do not classify the lines as A/A noise. A mixed snapshot and its changed window remain CHANGED. Accessibility snapshots (for example the macOS provider’s ax/*.json, which every capture also records) compare node text, ignoring layout, and list removed and added lines, so a changed label such as Largest: alpha 25.2 MB becoming Largest: . shows directly. Clock times and identifiers are normalized; values the product computes (a duration, a size, a count) are compared as captured, so took 2 s becoming took 0 min is a changed line. JSON snapshots with captured node paths also compare relative sibling order among common unique identities under the same identifiable parent. Stable reversals remain CHANGED even with identical pixels; this reports captured order, not a verified screen-reader navigation sequence. Only the specific child pair whose order varied in an A/A control or history pair is order noise. Existing text normalization and masks also apply to identities. Absolute indices, JSON serialization order, inserted/removed children and ambiguous identities do not invent reorderings. Groups over 200 children are explicitly reported as order uncompared. A window capture whose same-moment tree is unchanged is reported render-only when its changed pixels sit beside A/A noise or in text that changed. Touching measured noise cannot explain an entire changed label: at least 64 changed pixels more than four pixels from that noise keep the region a drawn change unless a trusted text redraw explains it. The measured noise mask is unchanged; smaller edge residuals can still be rendering. Redraw attribution requires the whole region to fit a text frame no taller than 96 capture pixels (two text lines). A large container frame does not locate its label. If an A/A pixel control changes the text while keeping that frame’s pixels unchanged, the frame cannot explain a redraw. Measured path noise remains noise. It stays changed when text it draws differs in a clock or identifier that no A/A control saw change, or when it changed a drawing that has no accessibility text (a gauge needle, a chart, an icon, a colour) where no control showed noise, or redrew a region in colours no control produced there (text in another ink, a warning turned red). Pixels inside or on the edge of a Browser snapshot mask box (solid #FF00FF) on either side are noise: a masked element that moved or resized is not a change. Per-step screenshots are compared only with --steps, because they are often taken mid-animation.

Steps that capture no screen compare what they retained instead: an MCP tool listing or call result, query rows, command output, and the step’s state and rendered summary. JSON compares value by value and names each change by JSON pointer, for example /items/5/annotations/readOnly: false -> true; JSON held in a string, such as an MCP tool result’s text, is compared inside it (/text#/total). Other text compares line by line; a fenced json block in a rendered summary compares as JSON (/json-block-1/...). Digests, durations and counts compare exactly; only UUIDs and wall-clock timestamps are normalized, and those differences are still counted. For an already CHANGED structured item, bounded normalized JSON leaf differences also appear below actual changes under normalized differences as captured (context only; excluded from verdict). This can show a timestamp value beside a changed output hash; it does not independently classify that timestamp as wrong. Normalization and verdicts are unchanged, and reports containing only SAME or NOISE rolling timestamps remain quiet. Artifacts beside a step’s screenshots or accessibility snapshots are bookkeeping and are listed as not compared, so a summary never reads zero without saying what it left out.

Run-specific content (a clock, a temporary path, a counter) also changes between two runs of the same code. Pass earlier runs of the same commit as A/A controls and only change beyond their noise is flagged:

reflow evidence diff <base-run-id> <head-run-id> --control base=<earlier-base-run-id>

A difference is noise only when a control changed the same place and the new value lies within what the controls produced there: one of their values, a number within their drift, or a random token of the same shape (a test-data slug, a digest of noisy content). A label edit on a line that also carries a volatile number stays changed. Numeric drift is measured at each numeric place: a stable decimal gets no last-digit allowance just because controls exist. For example, 196.6 KB becoming 196.7 KB or capacity 62.8 GB becoming 62.9 GB remains a change when controls kept those values, even beside a drifting free-space figure. A whole number one unit off may still follow a decimal in the same snapshot that moved within its own measured A/A drift; that possible derived change is listed explicitly.

An array whose elements a control showed moving or coming and going (an activity feed) compares by its elements’ identity field instead of by index. A time of day the controls saw move is noise only while it moves with the time between the two runs: a clock printed in the wrong time zone (Last scan 13:31 becoming 20:35 between runs four minutes apart) is changed. With controls, changed items are listed first: changed text first (the ones with the most differences beyond A/A noise first), each followed by the changed screenshot beside it, then other changed captures, then pixel-only ones.

A Browser snapshot also retains its page text as aria.yaml beside screenshot.png: the captured element’s accessibility snapshot (roles, names and text, masked elements left out). It compares like any accessibility snapshot, with the same A/A rules, and its CHANGED line says what changed, such as text row "Admin Built-in [20 -> 9]", row "Member Built-in [6 -> 5]"; the screenshot beside it repeats that before its pixel figures. A changed screenshot whose page text is the same or within A/A noise reads pixels only (a chart, an image, an icon, a colour or layout) and is listed after changed text, so a small changed figure is not ranked behind a noisier redraw. macOS window captures pair with their ax/<name>.json tree the same way. Page text only one run retained (one recorded before Browser kept it) is uncompared. A node line an A/A control rewrote (a member list that depends on test data) is noise when the head changed nothing in it the control did not also change; text of an element a mask paints is left out, while an element a mask box only overlaps keeps its text.

Use controls from the same environment (CLI, provider, target); a control from another viewport or browser widens the noise. The command reads evidence only; it approves no baseline.

When a compared run did not succeed, the output says so before its captures: the step it failed at with the recorded error (an assertion’s expected and received values), and the steps it did not run. A capture only the other side retained names that step, so a checkpoint the failing run never reached reads as such rather than as a silent only-base. A failed control still measures noise for the steps it passed; its capture of a step it failed counts only when the run it repeats failed that step too.

Step timing is reported apart from the capture verdicts, since a regression can change how long a screen takes to settle without changing what it shows. Each provider step’s recorded duration, and each action of a retained macOS executor timeline.json, is compared with the spread the A/A controls (--control and --history) showed for the same step. A step reads SLOWER (or FASTER) when it moved by more than a second and twice the controls’ largest difference, by more than 2x and the square of their largest factor, and beyond the other steps of the same run (a host that ran everything slower is load). An action whose run recorded its timeout reads NEAR-TIMEOUT at 80% of it or more. For example expect gigahog.breadcrumb.item.0: 74 ms -> 4294 ms (58x slower ...) heads a Step timing: 1 slower beyond A/A noise line while the summary still reads 0 changed.

Both sides must have served the code you mean to compare. When reflow apply admits new work against a local (loopback) target, it records which process listened on each target, granted and readiness origin: process ID, start time, executable SHA-256 and, for Go executables, the embedded vcs.revision and -cover setting. Apply prints it under Served build and warns when a compiled server started before the checkout last moved or was built from a different commit. evidence diff then compares the two runs’ records: same-process warns that one process served both sides (it was never restarted, so a checkout in between did not change it), same-executable warns that the compiled code was byte-identical, and different names the origins whose build changed. The record lives in the profile directory of the device that ran the apply, so runs applied elsewhere compare as unknown.

Explicit visual assertions

For a deliberate approved-image contract, the RFL snapshot checkout threshold="2%" command also compares against an approved baseline using SSIM. It reports new without one; a mismatch fails that checkpoint. The default threshold is 1%. This is a separate contract from comparing two executions.

Assert application readiness first. settle="500ms" bounds waiting for finite animations; the default is two seconds, maximum ten seconds. A timeout retains the capture and fails without comparing an unsettled image. Looping animations are not frozen or masked. See the snapshot reference.

Neither a visual difference nor a new image is permission to change the baseline. Review originals before an authorized owner accepts an expectation. Agent repair preserves the original failure; it does not turn a mismatch into a passing result by approving it. A PR can compare changed evidence without adopting a baseline.

flowKey and storyboard bindings

flowKey is the exact key declared in the comparison inventory: inspect base.checks[].key or head.checks[].key (and matching runs[].key). It is not implicitly a flow slug, UUID, file path or display name. An inventory can use login-smoke as its key; use that string only when the inventory actually declares it.

For a storyboard screen bound to that check, set flowKey: "login-smoke" and checkpoint: "org-home" only if that check captures a snapshot named org-home. checkpoint uses the authored snapshot name (the capture’s displayName in comparison reads), not the step label. Provider captures also expose a retained internal name, such as provider_snapshot:<hash>; an exact internal name is accepted too. The name must identify at most one capture on each side of the selected flow. A step move can change the internal key between sides; the storyboard retains both originals as a before/after pair. Repeated names within one side, or a name that also matches another capture’s internal key on that side, are ambiguous. Use unique snapshot names or an unambiguous internal key; Reflow never picks the first matching capture.

The review distinguishes a flow outside the comparison, a checkpoint missing from a selected flow, an ambiguous name, and an unavailable retained screenshot.

Omit both fields when no existing inventory check and named capture are known. A storyboard binding maps review evidence; it does not create a test or prove the flow ran.

Find a named provider capture

get_run_evidence lists provider screenshots using the authored checkpoint name, step and invocation number when the retained Browser manifest contains the name. Multiple captures with the same name still have separate evidence IDs. This also works for older retained runs; it does not require a new execution. Missing or unreadable metadata falls back to a step/call label without preventing image access. Labels do not change image identity, approval or checksum verification.

For volatile timestamps or branch labels, author a CSS mask on site.snapshot. See Browser snapshots for exact syntax and pixel behavior. Direct plugin captures support explicit comparisons and agent review; masking does not automatically approve a baseline or a difference.

Select knowledge for both sides

A change comparison uses the frozen knowledge reviews explicitly selected as knowledge.base and knowledge.head; the application’s current knowledge root is not attached automatically. Include the storyboard document in each review’s selected paths to show it alongside the retained captures.

Each selected knowledge review must have a repository label matching the comparison and a non-null gitCommit matching that side’s exact revision commit. When base and head commits differ, publish two knowledge reviews with their respective labels, even if both select the same unchanged corpus revision. One shared review with gitCommit: null does not satisfy either side. The labels are caller-supplied context; they do not prove Git ancestry or execution.

Inspect these labels before publishing the comparison. A published comparison’s selection is immutable: changing the current knowledge root cannot attach a map or repair its selected Git labels. A new comparison is needed to change those selections. Resolving an existing snapshot name does not require editing the map or republishing the captures.