Introduction to software QA tooling in 2022

We want to build bug-free, high quality software that satisfies a customer need. Unfortunately, it’s not easy.

Software QA is the set of processes and practices that keep software meeting its requirements and free of defects. There are no silver bullets — no tool or strategy is optimal for every organisation. But most organisations carry significant QA inefficiency, and a wave of tooling has emerged to attack it. The right tool can drastically drop the cost of high-quality development, which makes the evaluation worth doing carefully.

This article covers three QA patterns we see inside organisations, how each works and fails, how to evaluate tooling against your context, and a map of the record/replay automation landscape — including an honest account of where reflow fits and where it doesn’t.

Strategy 1: only developers do QA

The development team gatekeeps quality. They own every change and build automated tests as part of development; the theory is that responsibility for what they build converts into product quality.

When it works. Organisations that pull this off share a culture that values quality output, hiring that embeds test-automation skill in the team, and tooling that makes writing automated tests cheap.

When it doesn’t.

1. Developer prioritization is tightly controlled, and QA is (often rightly) prioritized behind feature work. Automated testing struggles to keep pace with development.
2. Developers see quality through the happy path they built, not through a customer’s eyes.
3. Compensation and career growth attach to shipped features, rarely to QA output. Purely developer-managed QA drifts toward “ship it and move on”.

Strategy 2: a siloed QA team does QA

Specialists own QA, often isolated from the development team. They control QA priorities and handle automation plus whatever manual testing the product needs.

When it works. A QA team that establishes itself as a trusted advisor — and engages with the product from a customer-centric perspective — adds value beyond defect-finding: it aligns the product with what customers actually need.

When it doesn’t.

1. A QA team without deep product engagement can’t perform valuable QA activities.
2. Without tools to automate cost-effectively, the team is overwhelmed and becomes a liability.
3. The silo drifts naturally toward being a release bottleneck — and a scapegoat.

Strategy 3: QA personnel embedded in the dev team

The compromise position: QA specialists own quality from inside the development team, integrated into its process and able to influence work earlier.

When it doesn’t work.

1. Embedded QA gets pulled into delivery work and loses its QA focus.
2. Embedded QA inherits the development team’s pressures and biases, especially when they also build features.
3. QA personnel without automation depth become a drag on team productivity.
4. Knowledge concentrates in individuals; when they leave, it leaves with them.
5. As the organisation scales, teams specialize and split, and the embedding dissolves.

Evaluating tooling

Whichever strategy you take, tooling decides whether it scales. Without tooling, QA is manual, and manual QA scales nowhere near as well as software.

All QA tooling serves one of two aims, and every option — commercial or open-source — should be evaluated against them in your context:

Reduce the severity of bugs: deploy fast, monitor well. If a bug is fixed minutes after it’s found and the data architecture limits the blast radius, pre-release QA matters less. For early-stage products, investment in monitoring and continuous deployment usually beats investment in test automation. This flips as the product grows complex and developers churn.

Reduce the likelihood of bugs: efficient test automation. Test automation is the only method that scales to arbitrary product complexity for catching bugs before release. For products where bugs are costly and development is continuous, it is the only long-term path to quality. Commercial tools earn their keep by accumulating machine-learned knowledge of your application that an open-source harness alone can’t provide — but every SaaS automation tool creates lock-in. Evaluate against your specific needs: if your team churns or your needs outgrow the tool, your testing effort stagnates with it.

Where reflow fits

Reflow lets non-technical QA personnel build end-to-end tests in a no-code recording UI, while engineers extend it with snippets from their existing Playwright suite. The same flows record and replay both in the web dashboard and locally over a CLI — including inside CI.

• Reducing severity: point reflow at production for synthetic monitoring — low-code visual checks that recognize quickly when a site has regressed.
• Reducing likelihood: let QA personnel design and implement automation that developers then run locally and in CI, without either group blocking the other.

When it works. Teams with a mix of technical and non-technical people testing the product. The workflows are built for hand-offs in both directions: developers enhance QA flows with code snippets; QA hands developers business-specific sequences they can run against locally running software.

When it doesn’t. A product tested entirely by its development team gets little from reflow — a developer team almost always manages a versioned end-to-end framework more easily than a recorded tool. What reflow adds in that setting is observability of runs, audit records, and screenshot capture and comparison; if those don’t matter to you, look elsewhere. Reflow is also currently sold through AWS Marketplace as a usage-based subscription, which requires an AWS account.

Reflow is bootstrapped — one engineer, one engineer’s view of QA, and a growing user base whose feedback shapes it. If you evaluate it, tell us what’s missing.

The record/replay landscape

Tools in this category target code-free automated testing, so non-developers can contribute. Common features: auto-healing, visual regression testing, data-driven testing, cross-browser and responsive testing.

• reflow.io (launched 2022)
• preflight.com (launched 2021)
• rainforestqa.com (launched 2021)
• reflect.run (launched 2020)
• mabl.com (launched 2018)
• ghostinspector.com (launched 2014)
• testim.io (launched 2014)

Differentiating them

These tools share a lot; the differences that matter are capability boundaries.

• Local testing. Some run locally, some only in their cloud, some via a network tunnel. Reflow’s full record/replay process runs locally from an npm package on Windows, Mac, or Linux.
• Mobile testing. Emulation, real devices, or nothing. Reflow emulates mobile devices; it does not run on real hardware.
• Cross-browser. Reflow wraps Playwright: execution in Edge, Chrome, Firefox and Safari; recording only in Chromium-based browsers.
• Selector stabilisation. Some tools heal selectors visually, some via CSS, some both. Reflow uses both — which is also why it only supports web applications.
• Customization. Most tools allow browser JavaScript; fewer allow deep control of the execution environment. Reflow takes parameterized Playwright snippets, which can configure the test environment arbitrarily.
• Data export. Code-free tools hold your tests inside the tool; each differs in how you leave. Reflow exports all data over its CLI, extracts product screenshots from runs, and exports tests to Playwright code.
• Integrations. Reflow runs anywhere npm install reflowio runs, so it drops into any CI/CD pipeline. It has no vendor-specific integrations for JIRA, Slack, or similar.
• Cost. Many tools don’t publish pricing. Reflow sells through AWS Marketplace with public pricing metrics and cost-management controls to avoid surprise charges.
• API testing. Some tools have code-free API testing. Reflow requires snippets for it.
• Desktop applications. Some tools go beyond web UIs. Reflow doesn’t.
• Isolation. Some tools are experimenting with isolating test execution from parts of the application — first-class network mocking and replay for deterministic UI tests. Reflow snippets can use Playwright’s network APIs for this; we’re also experimenting with unit test generation from runtime execution analysis, though that isn’t live.
• Visual regression. Every tool approaches this differently. Reflow uses an SSIM-weighted pixel diff, which ignores minor cross-browser rendering noise while highlighting significant changes close together.