Golden coverage suites
A coverage document is process-wide: it records everything the instrumented processes ran during a window, including startup, background jobs and other traffic. A golden coverage suite run adds per-flow attribution. One command builds an instrumented Go binary, launches it, runs your flows one at a time and snapshots every process’s counters between them. The counter deltas between snapshots are attributed to the flow, idle period or startup that produced them.
Per-flow attribution exists only in suite runs. The result is always an interval, never one number:
- lower bound: code attributed to the flow because idle time cannot explain it;
- upper bound: everything that ran while the flow ran, including background work;
- process reach: everything any process ran in the whole lifetime. This is the parent coverage document.
Suite runs measure Go servers built with
-covermode=atomic. A recipe can add one
browser JavaScript member: a
reflow.code-coverage/3 document
for the whole suite run, reported apart from Go and never blended into Go
figures. Recipe commands run only from the CLI: MCP tools can read sealed suites
but cannot start one.
What you need
Section titled “What you need”- A Go service whose processes run from your own
mainpackage and return frommainon SIGTERM (see Make SIGTERM return from main). - Adapter v2 in that
mainpackage (below), and a build with-tags reflowcoverage -cover -covermode=atomic -trimpath. - A clean, committed checkout. Untracked or ignored
.gofiles in a built package directory are refused, apart from a declared-ignored adapter. - The driver running in your application repository’s toolchain environment, so
its
go env GOVERSIONis the toolchain the recipe’s build uses. With mise, run it asmise exec -C <app repo> -- reflow coverage suite run .... A driver whosegodiffers from the build’s is refused (go_version_mismatch) before any hook or build, because covdata reads counters only with the toolchain that wrote them. - A recipe,
.reflow/coverage-suite.json. - A Reflow login and a selected collection, or
--collection <id>.
Adapter tiers
Section titled “Adapter tiers”The report names the tier of every suite run. Tiers are not comparable with each other.
committed: the adapter file is committed in the main package. Generate it withreflow coverage adapter --write <main package dir>and commit it. Its//go:build reflowcoveragetag keeps it out of normal builds.declared-ignored: you cannot commit to the repository, so the adapter file stays out of Git through an ignore rule, for example a line in.git/info/exclude. Reflow still accepts only the exact published bytes:bind --suitehashes the file and refuses anything that is not a published adapter version (adapter_unknown). An adapter that is neither tracked nor ignored is refused (adapter_untracked). A declared-ignored adapter is not in the bound tree, so it is never in the denominator, but its blocks are still checked against the binary.restart: fresh processes for every flow. It is not implemented:--segmentation restartrefusessegmentation_unsupported.
Adapter v2’s published sha256 is
d59b31d1a60dacd7fe15d4d5511578e787cbb3ec89c59192096079a485d5a9de;
reflow coverage adapter --print v2 --json prints the bytes and the hash. What
the adapter does, and every bind --suite refusal, is described in
Adapter v2 for golden suites.
In short, it exits with status 70 on a set- or count-mode build
(mode_not_snapshot_capable), serves an authenticated POST /snapshot on a
loopback port, and writes cumulative snapshots without ever resetting the
counters.
The recipe
Section titled “The recipe”reflow coverage suite run reads .reflow/coverage-suite.json (or
--recipe <file>). The sha256 of its exact bytes is recorded on the suite run.
Unknown keys refuse recipe_invalid.
{ "format": "reflow.coverage-suite/1", "label": "app-golden", "build": { "argv": ["go", "build", "-buildvcs=true", "-trimpath", "-tags", "reflowcoverage", "-cover", "-covermode=atomic", "-coverpkg=example.com/app/...", "-o", "{binary}", "./cmd/server"] }, "universe": ["example.com/app/..."], "exclusionProfile": "reflow-go@1", "exclude": ["**/testenv/**"], "roles": [ { "name": "server", "argv": ["{binary}", "serve"], "envFrom": "mise", "readiness": { "http": { "url": "http://127.0.0.1:8080/health" }, "timeoutMs": 60000 } }, { "name": "worker", "argv": ["{binary}", "worker"], "envFrom": "mise", "readiness": { "log": "worker started" } } ], "hooks": { "before": [{ "argv": ["./scripts/stop-dev-server.sh"] }] }, "shards": [ { "name": "A", "selectors": ["checkout-golden", "refund-golden"] }, { "name": "B", "selectors": ["settings-golden"] } ], "schedule": { "idleSeconds": 120, "settleMs": 10000 }}| Key | Meaning |
|---|---|
format | Always reflow.coverage-suite/1. |
label | The suite run’s label (--label overrides it). |
build | argv runs verbatim, without a shell, in cwd with env. {binary} expands to the binary path (default <work dir>/bin/<label>), or name the output with binary. |
universe | The build’s -coverpkg import-path patterns. ./..., all and std are refused. |
exclusionProfile | reflow-go@1 (see Exclusion profile). |
exclude | Your own globs, layered on the profile (up to 64). |
roles | 1 to 16 processes of the built binary. argv[0] must be {binary}, so the launched process is the instrumented one (no shell, wrapper or mise exec). env, envFrom: "mise" (the role’s mise env) and inheritEnv (names passed through) declare its environment. readiness is a loopback http URL with expected status codes, or a log regular expression one output line must match. stop sets the signal (SIGTERM or SIGINT) and its timeout. |
hooks | before and after commands (no shell), for example stopping a development server that holds the port. |
shards | One process lifetime per shard, each with its own cover directory and apply selectors. Without shards, one shard main selects every flow. |
schedule | idleSeconds (D, at least 120 s), baselineIdleSeconds (a longer first idle per process lifetime, see Periodic background work), settleMs (default 10 s), quiesceMs (default 2 s), idleEvery (K, default 10) and steps (a boundary between flow steps). See Segments. |
expectEnvDigest | The env digest of the stack you validated; suite run prints whether the run matched it, and the sealed report records the run’s env digest and that comparison (suite.envDigestMatch; a Reflow service that predates the comparison records “not compared”, and the run summary says so). |
browserJs | Optional: one browser JavaScript member (see Browser JavaScript member). |
uninstrumentedOrigins | Optional: up to 8 allowed origins of the target served by a separate program your flows need but the suite does not measure, such as a development identity provider (see Uninstrumented origins). |
Role names use lowercase letters, digits, _ and - (at most 32). The driver
sets GOCOVERDIR, REFLOW_COVERAGE_ROLE, REFLOW_COVERAGE_PROBE and the
session token itself; a recipe may not.
Env digest (reflow.coverage-env-digest/2). For each role, the variables it
receives are recorded as their names and the sha256 of each value, never the
values: the declared ones (env, envFrom, inheritEnv), and PATH, HOME,
TMPDIR, LANG, LC_ALL, USER and TZ, which a role inherits from the
driver unless it declares them. Names that describe the machine and shell running
the driver rather than the stack are left out wherever they come from, and the
digest names that list: PATH (mise env adds its tools to your own PATH),
HOME, USER, LOGNAME, SHELL, TMPDIR, PWD, OLDPWD, TERM, SHLVL
and _. The roles still receive them. LANG, LC_ALL and TZ are digested
even when inherited, because they change what the stack does: a driver with
another time zone gives another digest. Declare them in the roles to make the
digest independent of the driver’s shell.
To pin expectEnvDigest, compute the digest of the stack you validated with the
same definition: resolve each role’s environment as the driver does (its env
over mise env --json from its cwd, plus inheritEnv, plus the inherited
names above) from a shell with the driver’s LANG, LC_ALL and TZ, and digest
it. When the recipe’s roles declare exactly that stack’s environment,
suite run --json prints the value as envDigest (with envDigestRoles, the
names and value hashes per role, and envDigestExcluded); pin it, and later runs
report whether they matched. The digest does not identify the tool versions mise puts on
PATH; it does include tool variables the stack declares, such as GOROOT.
Run a suite
Section titled “Run a suite”reflow coverage suite run --golden --output-dir /tmp/app-suitereflow coverage suite run --golden --repeat 2 --shuffle 7f3a --output-dir /tmp/app-suitesuite run does the following, in order:
- Preflight. Everything it can check without running a hook or the build
comes first. It checks the recipe and its
universe, a clean tree (suite_dirty), and the toolchain: the build’sgo(asked through the build’s own argv, somise exec -- go buildanswers for mise’sgo) must be the driver’sgo env GOVERSION(go_version_mismatch). It records staleness: how many commits HEAD is behind and ahead of the localorigin/HEAD,origin/mainororigin/masterref, which it never fetches. It computes the env digest, resolves the collection, refuses a D too long for the barrier (idle_exceeds_barrier_timeout) and checksuninstrumentedOriginsagainst the selected target.--output-dirmust be new or empty; inside the worktree it must be git-ignored. The default is a fresh temporary directory. WithbrowserJsit also binds the browser coverage directory (below), so its refusals come before any hook. - Build. It runs the
beforehooks and the build, then checks the tree again (suite_dirty,head_moved). - Bind. For each repeat it runs
bind --suite --coverpkg <universe> --denominator static, then uploads the denominator object. That object is content-addressed and is not a suite run, but uploading it now surfaces a credential, scope, collection or availability fault of the suite writes before the first lifetime. The suite run itself opens in step 7. - Launch. For each shard it starts every role in recipe order, records each
pid with
record-start, waits for the adapter’s control record and start marker, then for the readiness probe. A role that exits during startup refusesrole_exited, or the adapter’s own code, such asmode_not_snapshot_capable. - Schedule. It takes the boot snapshot and holds the first idle segment.
For each apply group it runs
reflow planto find the selection closure (ancestors included), thenreflow tainton each flow so nothing is reused, thenreflow apply [--golden] --coverage-session ..., which brackets every run with snapshots. After the last group it holds the closing idle segment. - Stop. It sends SIGTERM to each role in reverse order and waits. A role
still running after
stop.timeoutMsis killed (role_killed) and has no exit counters. - Open, ingest, upload and seal. Once every lifetime of the repeat is
gathered, it opens one suite run, so a refused admission, a role that cannot
start or an interrupt leaves no open suite run behind. Only the open’s own
refusals come this late: a service that does not know the exclusion profile
or the attribution rule (
unsupported_exclusion_profile,unsupported_attribution_rule), which means a CLI newer than the service. It then ingests each lifetime’s parent document (ingest --segments --processes launched --window suite:<id>:<shard> --member <shard>, with every apply’s batch key), then the browser member, if any. Each member document has its own run window, because a window holds one document per binary and every shard runs the same binary: with the recipe above, a suite run’s documents are in the windowssuite:<id>:Aandsuite:<id>:B. It then re-derives the block table and every segment from the stored raw files, uploads them and seals the suite run. - Compare. It downloads the sealed report, checks that its bytes hash to
the sealed sha256, and derives the report again from its own evidence with
the run facts the seal recorded in it. The two must be byte-identical
(
report_mismatchnames the report sections that differ). It prints the report and the evidence directory.
--repeat N (1 to 16) makes N suite runs in one repeat group, for the variance
read. --shuffle <seed> runs one apply per shard selector in a seeded order. It
needs explicit selectors in every shard, and it reorders applies, not the runs
inside one apply. reflow apply --golden admits at most 20 flows including
ancestors, so split a larger suite into shards.
When a run stops early
Section titled “When a run stops early”Whenever suite run stops before a report is recorded, it prints the suite run
id and the exact command that resumes it,
reflow coverage suite ingest --output-dir <dir>, and says whether resuming can
help. That command resumes every repeat that is not sealed yet. Raw objects are
content-addressed, every upload is idempotent and so is the seal, so repeating
it is safe; a run the server already sealed is only compared again. A request
that meets a transient failure (a 503, 429, 502 or 504 answer, a failure the
service calls temporary, a deadline or a network error) is sent again up to 7
times over about a minute (after 1, 2, 4, 8, 15, 15 and 15 seconds, or after the
answer’s Retry-After when that is longer, at most 30 seconds), before the run
stops. That outlasts a service whose database is resuming after an idle hold.
The reads plan and apply make are retried the same way, and an apply that
still fails before anything was admitted is attempted again up to 3 times (after
15, 30 and 60 seconds) before the suite refuses backend_unavailable; each
failed attempt’s output is kept under applies/. A refusal (a 4xx answer) is
never sent again. A stop signal ends these waits. Resuming
cannot help, and the message says to run the suite again, when a lifetime was
never gathered (for example after a refused admission, a role that failed to
start or an interrupt; suite_incomplete), when the server refused the open,
when the seal failed, or when a refusal is permanent, such as
truncation_unsealable (see the seal). A stop before every lifetime
is gathered never opened a suite run, so nothing is left open on the server. A
refused browser member does not stop the run: see
Browser JavaScript member. There is no command
that fails an open suite run. An abandoned open run can fail once one of its
member documents is retired, which happens 24 hours after it opened at the
earliest, and the daily sweep fails any run still open after 7 days.
Ctrl-C and other stop signals. Roles and the plan, taint and apply
the driver runs have no terminal of their own, so Ctrl-C, Ctrl-, a SIGTERM to
suite run and a closed terminal or SSH session (SIGHUP) all reach the driver
alone. The driver starts no further apply and no further upload, passes the stop
to a running apply once (so it cancels its unfinished flows), stops every role
with its stop signal so Go writes the exit counters, runs the after hooks,
prints the suite run id and whether resuming can help (a stop before every
lifetime is gathered opened nothing: run the suite again), and exits with 128 plus the
signal’s number (130 for Ctrl-C, 143 for SIGTERM, 129 for a closed terminal). A
role that does not handle its stop signal dies without exit counters. A second
signal prints the same guidance and kills every role at once. If the driver
itself is killed with SIGKILL, roles keep running; stop them by the pids it
printed when the shard started (the reflow-launch.<pid> files of the
lifetime’s cover directory name them too).
Browser JavaScript member
Section titled “Browser JavaScript member”"browserJs": { "member": "dashboard", "exclusionProfile": "reflow-js@1", "coverDir": "/tmp/app-suite-browser", "webRoot": "client/dashboard"}The Browser plugin
records V8 coverage of the pages your flows open into its binding’s
"coverageDirectory", which must be an absolute path. Set it, in the target’s
bindings before the run, to the directory coverDir names: an absolute path, or
one relative to the repository root that Git ignores. suite run cannot read
the target’s bindings, so a mismatch shows up only at ingest, as no_profiles.
coverDir must be empty (browser_cover_dir_not_empty).
| Key | Meaning |
|---|---|
member | The member’s label: its document’s member label (in the window suite:<id>:<member>) and the suite member’s shard. It cannot be a shard name. |
exclusionProfile | reflow-js@1. |
coverDir | The Browser binding’s coverageDirectory. |
webRoot | The repository directory the dev server serves URL paths from (bind --web-root). |
exclude | Your own globs, layered on the profile. |
excludeFailedSessions | By default (and with true) only the complete browser sessions are ingested, and the failed or incomplete ones are left out and listed with their reasons. false makes any such session refuse the repeat’s browser member (see below). |
reflow-js@1 excludes **/node_modules/** (vendored), every Speakeasy SDK
directory (one holding .speakeasy/gen.lock or .speakeasy/gen.yaml) and every
tracked JavaScript or TypeScript file whose leading comment says
Code generated ... DO NOT EDIT (generated). Before any hook runs, suite run
resolves these rules and your globs over the bound tree into the document’s
exclude globs (at most 64), checks that they exclude exactly what the profile
classifies, and binds coverDir with them. It also refuses
browser_js_too_many_batches when a repeat would run more than 64 applies, the
most batch keys one document holds. Only --shuffle (one apply per selector) can
reach that.
Every browser session of every lifetime writes into coverDir. After a repeat’s
last lifetime, its contents move into that repeat’s browser directory, which
leaves coverDir empty for the next repeat. The ingest step then ingests them as
one reflow.code-coverage/3 document with its own window suite:<id>:<member>,
the member label and every apply’s batch key, and adds it as the suite run’s
browser member. If that move did not finish, the ingest refuses
browser_member_incomplete rather than seal without some sessions.
The report’s js holds that document’s id, canonical hash and lines:
found counts every instrumentable line of the files the exclude globs keep,
hit those that ran. It is one figure for the whole suite run: there is no
per-flow, per-shard or interval figure for JavaScript, and its lines are never
added to the Go figures. suite verify re-derives it from the stored document.
The summary states the share, for example “JS from 18 of 43 browser sessions; 25
left out”, beside the number of flow runs, and suite show repeats it when
sessions were left out: such a JS figure is partial, missing evidence rather
than 0%. A browser flow whose session wrote no profile also only makes js
smaller, so compare the sessions with the flow runs.
When the browser ingest is refused. A wrong coverageDirectory
(no_profiles), no complete session at all (no_complete_sessions), with
excludeFailedSessions: false a browser that never closed
(session_unfinished), or another browser refusal stops only that repeat’s
browser member. The repeat’s Go evidence
is kept and its suite run stays open and unsealed, and suite run goes on to the
next repeat. When every repeat has run, it refuses with the first repeat’s code
and lists each refused repeat. Resume with suite ingest --output-dir <dir>:
- once the cause is fixed (for example a server error), with no flag;
- with
--exclude-failed-sessions, to ingest only the complete sessions and list the failed ones; - with
--without-browser-member, to seal the Go evidence without the browser member. The report’sjsis then null. A member that was already ingested cannot be left out (browser_member_ingested).
Each flag applies to every unsealed repeat whose browser member is not ingested yet, and is kept with that repeat.
Segments
Section titled “Segments”A segment is the per-block counter delta between two snapshots of one
process lifetime. Snapshots are cumulative and never reset, so for every block
and process, boot + all segments + tail equals the exit count exactly. Reflow
checks this conservation at upload, at the seal and in suite verify.
| Segment | Span |
|---|---|
boot | Process start to readiness plus quiesceMs. |
idle (pre) | At least D, or baselineIdleSeconds when the recipe sets it, from the boot snapshot to the first run. The first plan, taint and admission run inside it. |
flow | One run, from just before a worker claims it until it is terminal. |
step | With schedule.steps, each step of a run. |
settle | settleMs after each run, in two halves: the run’s spill. |
idle (mid) | D/2 (with a baseline idle, at least a quarter of it), before every K-th further run. |
idle (post) | D, after the last run. |
tail | The last snapshot to process exit. |
D = max(idleSeconds, 120 s), the same for every repeat: no measured flow
changes it. Each apply’s barrier timeout must cover the longest idle hold at a
run’s start, the first run’s idle-pre (D + 45 s). A recipe whose D exceeds the
controller’s 900 s bound (idleSeconds over 855) refuses
idle_exceeds_barrier_timeout before anything launches. A measured flow that
runs longer than half of D has a shorter idle baseline than the method asks: it
is listed as idle_baseline_insufficient in the run summary and suite show,
and still counted. Raise idleSeconds to at least twice your longest flow to
avoid that. Expect the
idle and settle segments to add a large share of wall time; the real duration
of each segment is recorded.
The barrier and admission
Section titled “The barrier and admission”reflow coverage suite run starts a session on a loopback port and passes it to
reflow apply --coverage-session <origin>; its token travels only in the apply’s
environment. The device controller asks the session for a snapshot before each
run is claimed, after it is terminal, at both settle halves and, with steps,
between steps. Before anything is admitted, the apply refuses:
coverage_requires_serial: parallelism other than 1;coverage_reused_runs: any planned step would be reused (the driver taints the closure first);coverage_session_unavailable: the session cannot be reached or cannot hold the idle schedule;coverage_served_by_uninstrumented: a loopback listener serving the target is not one of the session’s instrumented processes, or could not be probed;coverage_uninstrumented_origin_invalid: a declared uninstrumented origin is not an allowed origin of the target, shares a port with the target URL or a readiness probe, is not among the 8 origins admission probes, or is served by one of the session’s own processes or another copy of the instrumented binary;coverage_requires_cli: the request did not come from the CLI. MCP tools and the dashboard cannot request a session.
A failed snapshot truncates segmentation. The flows keep running, no further
boundary is taken, and the tail runs from the last good snapshot to exit. The
suite run records truncatedAt. Per-flow figures exist only for segments
before that point. The suite union stays exact, because process reach comes
from the exit counters.
Uninstrumented origins
Section titled “Uninstrumented origins”Admission checks the target URL, every allowed origin and every readiness probe.
Some flows need a separate program on an allowed origin, for example a login
that redirects to a development identity provider on https://localhost:25140.
That program is not your instrumented binary, so admission refuses it. Declare
its origin in the recipe:
"uninstrumentedOrigins": ["https://localhost:25140"]Each entry is an exact loopback origin, spelled the way a browser spells an
origin: scheme, host and port, with no path or trailing slash, and a default port
left out (http://localhost, never http://localhost:80). It must be one of the
target’s allowed origins
(reflow target <url> --allow-origin https://localhost:25140), never on the
target URL’s or a readiness probe’s port, and among the first 8 loopback origins
admission probes (the target URL, then allowed origins, then readiness probes).
suite run passes the list to every apply (--coverage-uninstrumented-origin).
Admission then skips the listener on each declared origin and records its pid,
executable and sha256. Admission refuses when a declared origin is served by a
session process or by another copy of the instrumented binary, or when its
listener could not be probed. Any other listener that is not a session process
still refuses.
The sealed report’s suite.uninstrumented lists the declared origins and every
listener skipped there (executable base name, pid and sha256). suite run prints
one “declared uninstrumented, not measured” line per origin, and suite show,
the markdown export and the suite page in the app flag each one. The skipped
program’s executions are never counted. Universe packages it shares with your
binary stay in the denominator, and lines that only it runs count as uncovered.
The record names the program so that a reader can account for this. A report
without declared origins has no uninstrumented key.
Sealing a run that declares origins needs a Reflow service that copies them into
the report. An older service seals the run without them, and suite run then
refuses report_mismatch after every lifetime. The run warns about this at
start. A listener that restarts so often that the report would name more than 64
of them refuses uninstrumented_invalid at the apply that overflows.
Confounded segments
Section titled “Confounded segments”At the seal, a run in the same collection, from any profile, that the manifest
does not list and whose wall-clock interval overlaps a segment marks that
segment confounded. A confounded segment stays in the upper bound and in
process reach but is left out of the lower bound. Traffic that does not come
from Reflow runs cannot be detected, so keep anything else away from the stack
while a suite runs.
The seal
Section titled “The seal”The seal is idempotent and runs in chunks. While it is still verifying members
it answers open, and suite run seals again at once. When evidence has not
arrived yet (a raw object, segment or member) or the call ran out of time
(seal_budget_exhausted), suite run uploads what is missing and seals again
after a growing wait, up to 8 times; after that the suite run stays open and
suite ingest resumes it. It checks, in order:
- each lifetime’s column-exact block table reproduces the parent document’s
canonical hash (
blocks_mismatch); - every delta is non-negative and conservation holds (
conservation_failednames the first block); - every stored raw object hashes to its name (
raw_hash_mismatch); - every run of the suite’s batches appears in exactly one lifetime’s
manifest, with the run’s flow, version and batch (
run_missing,run_duplicated,run_mismatch). Runs that were skipped or cancelled before any worker claimed them are not measured, and runs that started in a truncated tail are unsegmented; - confounded overlap;
truncatedAtnames a lifetime’s tail.
Evidence that fails a check fails the suite run permanently with that refusal:
suite run and suite ingest refuse seal_failed and name it. Fix the cause
and run the suite again. Evidence that has not arrived yet leaves it open. The
seal then computes attribution@1 and writes the canonical report,
reflow.golden-coverage-report/1. Every surface reads this one report.
Attribution (attribution@1)
Section titled “Attribution (attribution@1)”An idle segment qualifies when it lasts at least 1 ms and at least a quarter of
the suite’s longest idle segment. The others are gaps: the short idle time
between two runs that the schedule does not hold (controller overhead, or plan,
taint and admission with --shuffle).
For each block, the idle rate r is the highest rate at which any qualifying idle segment of the suite ran it: the maximum of delta ÷ duration. Idle segments of every lifetime that shares the block table count, so the rate is conservative. Gaps never give a rate. In a flow, step or settle segment of duration d, the expected idle count is λ = r × d. A block that ran in the segment is:
- exclusive: it never ran in any idle segment (r = 0 and no gap ran it) and never at boot;
- elevated: not exclusive, and its delta is at least λ + 3√λ + 1, well above the idle rate;
- ambient: anything else.
The adapter’s own blocks (zz_reflow_coverage.go) are always ambient. Its
snapshot handler runs once per snapshot in every process, a fixed count in
every segment that is not work done by any flow.
Exclusive and elevated blocks of a run’s settle segments are that run’s spill: asynchronous work the flow started.
| Figure | Definition |
|---|---|
| Lower bound | Exclusive, elevated and spill blocks of the run’s segments that are not confounded. |
| Upper bound | Every block that ran in the run’s flow, step or settle segments, confounded or not. |
| Process reach | Every block the lifetime’s processes ran, from the parent document’s exit counters. |
| Unique | A flow’s lower bound minus every other flow’s lower bound. |
| Redundant | A measured flow with no unique statements. This is advisory: a flow can assert a contract without reaching new code. |
| Nearest | The flow whose lower bound overlaps most (Jaccard of statements). |
Exclusive does not mean only one flow ran a block: two flows can both be exclusive on shared code that never runs while idle, and elevated middleware sits in many flows’ lower bounds.
Only runs whose status is succeeded and that were not reused count toward a
flow and the suite headline. Failed, cancelled, timed-out and needs_review
runs are flagged in the report and excluded from the golden figures; their
traffic is still in process reach. A flow whose second settle half still has
exclusive or elevated blocks is flagged settle_insufficient: raise
settleMs.
Go statements, weighted as go tool cover -func weights them, are the primary
figure. Lines are secondary. The report holds integers and
numerator/denominator pairs only, and every figure cites the suite, documents,
segments and objects (with their hashes) that it comes from.
Periodic background work
Section titled “Periodic background work”Attribution subtracts what the application does while idle, using the rates it
saw in the idle segments. Background work with a period longer than an idle
hold (a sweep every 5 minutes, a schedule every 15) may never run inside one.
It then has no idle rate, and its statements are attributed to whichever flow it
happens to run in: exclusive there, or spill when it runs in a flow’s settle.
The leak rate below cannot see this, because it only holds out idle segments,
and none of them ran the work. Signs of it are a placebo flow with a lower
bound, settle_insufficient on flows that start nothing in the background, and
lower bounds that differ between repeats.
Two remedies, best used together:
- Pause the work for the suite, in the recipe’s roles or hooks, when the application lets you (stop the schedule, disable the ticker). This is the only complete remedy.
- Observe it once per process lifetime with
schedule.baselineIdleSeconds. The first idle of each lifetime is then held that long instead of D; every other idle stays at D. Choose a value longer than the work’s longest period, so that the baseline idle is certain to contain one run of it. The rate is pooled, so that one observation applies to every flow of the suite run.
What one observation does, exactly. A block’s idle rate is its count in an idle segment divided by that segment’s duration, and in a flow segment of duration d a block with count c is elevated when c ≥ λ + 3√λ + 1, with λ = rate × d. After one run of the work in a baseline idle of B seconds, λ = c × d / B:
- a block the work runs once (c = 1) is never elevated again, in any segment: λ + 3√λ + 1 is above 1 for any λ above 0. Straight-line code of the job is ambient everywhere from then on;
- a block the work runs several times per run (a loop body) is ambient only while c < c × d / B + 3√(c × d / B) + 1. With B = 450 s, a 60 s flow keeps counts up to 3 ambient and still calls 4 or more elevated; a 5 s settle half keeps only count 1 ambient. A longer baseline makes the observation certain, not stronger: it lowers the rate.
So a baseline idle removes the once-per-run part of periodic work from every flow and leaves its loops, in short segments, to be paused. It also makes the work visible: the held-out baseline idle now shows it in the leak rate.
Bounds, checked before anything launches:
baselineIdleSecondsmust be longer than D;- at most 855: one idle is one hold of the controller, which allows 900 s less a
45 s margin (
idle_exceeds_barrier_timeout). A longer baseline cannot be split over several holds; - 4 × D must be at least
baselineIdleSeconds+ 5 (baseline_idle_invalid). An idle segment is rated only when it lasts at least a quarter of the suite’s longest idle segment, and the baseline becomes the longest. Past that bound every ordinary idle would stop being rated and the leak rate would have nothing to hold out. SoidleSeconds120 allows a baseline up to 475, and a 600 s baseline needsidleSecondsof at least 152. For the same reason the idle-mid hold is raised from D/2 to a quarter of the baseline plus the margin.
The run summary and suite show print the baseline hold and what each
lifetime’s first idle measured.
Leak rate
Section titled “Leak rate”Periodic work slower than the idle segments can still be misattributed to a
flow (see above). The leak rate measures the part an idle segment saw. Each qualifying idle segment in turn is held
out and classified against the idle rate of the others, as if it were a flow;
any statement it would call exclusive or elevated has leaked. The report flags
the suite when the mean leaked statements per held-out segment exceed 1% of the
median flow’s lower bound. With fewer than two qualifying idle segments, the
leak rate is insufficient. The report lists the gaps apart, with their hit
statements (leak.gaps). A suite with no idle segment of 1 ms or more is
refused with attribution_requires_idle.
Snapshots of concurrent processes are not one consistent cut. Conservation still holds, and repeats bound the jitter. Use the variance read to see it.
Denominator
Section titled “Denominator”bind --denominator static writes reflow.coverage-denominator/1, a separate
object that is never part of the canonical document. It runs
go list -e -json over the universe with the build’s tags and platform, and
go tool cover -mode=atomic statically on every file (nothing executes). Every
tracked non-test .go file of the universe lands in exactly one class, in this
order of precedence:
| Class | Meaning |
|---|---|
excluded_by_rule | A glob rule of the exclusion profile, or one of your globs. |
generated | A // Code generated ... DO NOT EDIT. header (go-generated-header@1). |
vendored | Under a vendor/ directory. |
build_excluded | Ignored by the go tool for these tags and platform (IgnoredGoFiles, or a _, . or testdata path). |
test_only | In a package that only tests import. |
instrumented | In a package linked into the binary. |
not_linked | In a package of the universe that the binary does not link: counted at 0%, with its static statements. |
no_statements | A file with no statements, such as declarations only. |
The effective denominator is instrumented, not_linked and no_statements.
The other classes form an excluded layer, reported with their own counts and
reach. The partition must be total and disjoint (denominator_incomplete
otherwise). On every bind the static blocks of each linked package must match
the binary’s instrumentation exactly (denominator_generator_mismatch), which
also catches a binary built from an input that has since changed. Package rows
are paged, never folded.
Exclusion profile reflow-go@1
Section titled “Exclusion profile reflow-go@1”A versioned profile, recorded with its digest. Its rules are:
| Rule | Class |
|---|---|
**/vendor/** | vendored |
**/testdata/** | excluded_by_rule |
| Go generated-file header | generated |
Each recipe exclude glob | excluded_by_rule |
Generated code such as sqlc queries or protocol stubs carries the header, so it is classified as generated whatever its directory. The report lists every rule with the files it classified. The profile is a separate layer: the parent document’s own exclude globs are unchanged.
Reading a suite
Section titled “Reading a suite”Every read is read-only and pages with --cursor. With --json the answer is
the server’s, including every citation and the report’s sha256. In the local MCP
connection, query_code_coverage offers the same reads as suite_list,
suite_show, suite_flows, suite_matrix, suite_packages,
suite_uncovered, suite_lines, suite_diff, suite_variance,
suite_export and suite_covers.
| Read | What it answers |
|---|---|
suite list [--state open|sealed|failed] | Suite runs of the collection, newest first. |
suite show <suite-id> | The headline: the interval for statements, then lines; boot and ambient sets; shards; flags (failed, reused and confounded runs, settle_insufficient, truncation, the leak flag); staleness, tier, adapter and env digest. |
suite flows <suite-id> [--query <text>] | Every flow with its lower, upper and unique statements, its exclusive, elevated and spill counts, redundant, its nearest neighbour and its runs. |
suite matrix <suite-id> [--level package|file] [--flow <flow-id>] | Flow × package (or file) cells. |
suite packages <suite-id> [--class <class>] | Every package row, not_linked and excluded classes included. |
suite uncovered <suite-id> | Packages whose effective statements no process reached, most first, with not_linked packages marked. |
suite lines <suite-id> --path <file> [--from <n> --to <m>] [--flow <flow-id>] | One file’s blocks with summed exit counts, plus each flow’s hits and labels. |
suite diff <suite-id> --base <suite-id> | Changes against an earlier suite run. |
suite variance <repeat-group-id> | Agreement across the sealed repeats of one group. |
In the app, open
/<team>/code-coverage/suites/<suite-id>?collection=<collection-id>. A golden
flow’s detail page links the newest sealed suite run that measured it, and a
coverage document that belongs to a suite run names it. reflow dashboard links
has no suite kind yet.
reflow golden covers --suite <suite-id> compares each measured flow’s covers
globs with the files the suite attributed to it. Each glob is measured,
declared_unmeasured (it matches universe files the flow was not attributed)
or outside_universe (it matches nothing the suite measures, such as browser
code). Attributed files that no glob matches are listed as measured but
undeclared. This is advisory: covers drives pull-request selection, so do not
rewrite it from one suite run alone.
Comparing suite runs
Section titled “Comparing suite runs”suite diff is likeForLike only when both suite runs share the commit, tree,
binary, block table, denominator, exclusion profile and attribution rule.
Otherwise it is descriptive, with reasons such as tree_changed,
binary_changed, universe_changed, denominator_changed,
exclusion_profile_changed or attribution_rule_changed. Package and flow
deltas are always shown. When the block table, denominator and rules match, the
first page also lists block-level deltas of the suite sets and of each flow’s
lower bound, even across commits.
suite variance needs at least two sealed repeats that share one block table
(variance_needs_repeats, suite_universe_mismatch). It reports the suite and
per-flow statement Jaccard between repeats, flaky blocks (in the lower bound
of some repeats only) and order-sensitive blocks (in every repeat, but
attributed to different flows). A miss against your target is reported, never
hidden.
Export
Section titled “Export”reflow coverage suite export <suite-id> --format json > report.jsonreflow coverage suite export <suite-id> --format markdown > report.mdreflow coverage suite export <suite-id> --format lcov > suite.lcovreflow coverage suite export <suite-id> --format cobertura --flow <flow-id> > flow.xml- json: the sealed report’s exact bytes. Their sha256 is the suite run’s
reportSha256. - markdown: the identity, the interval tiles with citations, every flow, package, uncovered row and matrix cell with its citations, and an Evidence section listing every cited id and hash. It shows no percentages.
- lcov: one
TN:<flow-slug>record per measured flow with a non-empty lower bound, sogenhtmlshows each flow separately. A file’sDAlines are all its instrumented lines: the flow’s lower-bound lines carry their hit count, the others 0. - cobertura: the suite lower bound, or one
--flow, over the instrumented files.not_linkedfiles have no block positions, so they are omitted and named in a comment.
lcov and cobertura carry lines only; statements stay in json and markdown. Both
refuse export_mismatch unless they reproduce the report’s figures exactly.
Verify a report
Section titled “Verify a report”reflow coverage suite verify --output-dir /tmp/app-suitesuite verify takes the work directory of a suite run, one repeat’s directory
or its evidence directory, and runs no recipe command. It checks that
go env GOVERSION matches the recorded toolchain, then re-derives every block
table, every segment and the report sha256 from the stored raw files with
go tool covdata: the covmeta, every snapshot’s counter files, the exit
counters and the manifest. It refuses raw_object_mismatch,
blocks_mismatch, segment_mismatch or report_mismatch when anything
differs. It trusts the run facts and the confounded set that the seal recorded
in the sealed report (suite run kept them from its download); everything else
is re-derived. The browser member’s lines are re-derived from its stored
document, not from the browser profiles. It needs the evidence directory and
downloads nothing yet.
Retention and pins
Section titled “Retention and pins”Members of the latest 10 sealed suite runs, of up to 20 pinned suite runs and of
suite runs opened in the last 24 hours are exempt from the 50 clean and 20 dirty
document retention. suite pin <suite-id> and suite unpin <suite-id> need a
team owner or admin with an admin-scoped credential.
When a member document is retired anyway, its segments and raw files go with
it. The sealed report stays readable, and block-level reads (suite lines,
suite matrix --level file, block deltas and variance) answer
evidence_retired. Retiring a member of a suite run that is still open fails
that suite run (member_document_retired).
Limits
Section titled “Limits”- Per-flow attribution is Go only. The browser member is one JavaScript figure per suite run; the Browser plugin’s profiles carry no run identity.
- Any browser session that writes into
coverDirduring the run joins the browser member, so keep other applies of that binding away while a suite runs. --segmentation restartis not implemented.- Traffic from outside Reflow is not detected; only runs in the collection mark segments confounded.
- Periodic work slower than the idle segments is attributed to the flows it runs in unless a baseline idle observes it or the application pauses it; the leak rate sees only what an idle segment saw (Periodic background work).
- Gaps between runs give no idle rate. Work that ran in a gap but never in a
qualifying idle segment has an idle rate of 0: it is never exclusive, but a
flow that also runs it counts it as elevated. A
--shufflegap of a quarter of the longest idle segment or more is an ordinary idle segment. An idle segment longer than 2D (a long--shufflegap, or a first plan and admission that take longer than 2D) turns every mid-run idle hold into a gap. - Only the adapter’s blocks are treated as running once per snapshot. Your own code that runs on every snapshot would be attributed to short flows.
- The seal takes one
truncatedAtand admits runs that started in a truncated tail only for lifetimes whose tail seq equals it.suite runsends the tail seq of the lifetime whose runs are missing from its manifest. If runs are missing from two lifetimes with different tail seqs, no value can seal the run, sosuite runrefusestruncation_unsealablebefore uploading; run the suite again. suite verifyneeds the local evidence directory.reflow golden coversreads each flow’s current source, not the version the suite measured.
Command reference
Section titled “Command reference”| Command | Purpose |
|---|---|
reflow coverage adapter --print v2 | --write <main package dir> | Print or write adapter v2. |
reflow coverage suite run [--recipe <file>] [--output-dir <dir>] [--golden] [--label <label>] [--repeat <n>] [--shuffle <seed>] | Build, bind, launch, run the flows with snapshots, ingest and seal. |
reflow coverage suite ingest --output-dir <dir> | Resume the upload and seal of a suite run. |
reflow coverage suite verify --output-dir <dir> | Re-derive the segments and the report sha256 from the raw files. |
reflow coverage suite list|show|flows|matrix|packages|uncovered|lines|diff|variance | Read sealed suite runs (see Reading a suite). |
reflow coverage suite export <suite-id> --format json|markdown|lcov|cobertura [--flow <flow-id>] | Export the report. |
reflow coverage suite pin|unpin <suite-id> | Keep a sealed suite run’s documents beyond retention. |
reflow golden covers --suite <suite-id> | Reconcile flows’ covers with what the suite attributed to them. |
Every command accepts --json. Refusals are named and exit non-zero.