Many systems are in one of two states. Either they have no automated tests at all, and every release is a small act of faith, or they have a large suite that nobody trusts: slow, full of tests that fail for no reason, and somehow never covering the thing that actually broke last week.
Both states have the same cause. The tests were never chosen. They were either postponed until "later", or written to reach a number rather than to protect something. This article is about choosing: which tests to write first, which to leave for later, and how to make the ones you have actually stop bad changes from reaching your customers.
What a test is for
An automated test is a small program that checks that another program still does what it should. It is worth writing when the cost of the failure it prevents is higher than the cost of writing and keeping it.
That sentence does most of the work of choosing. A test that guards the checkout of a shop protects money every day. A test that checks the exact wording of an internal error message protects almost nothing, and it will break every time someone improves the wording. Both count the same towards a coverage percentage. They are not worth the same.
Start with the paths that earn money
Start with the paths that earn money or lose trust when they break. For most business systems the list is short:
- Sign-up and log-in. If people cannot get in, nothing else matters.
- The main transaction: checkout, booking, submitting an order, sending a quote. Whatever the system exists to do.
- Payment, and what happens after it: the order marked paid, the confirmation sent, the invoice created.
- Anything that touches other people’s data: permissions, what one customer can see of another’s records.
- Anything that produces a document someone relies on: invoices, reports for the accountant, exports.
Write one test for the ordinary, successful path of each, first. Not every edge case: the path that most users take most days. A handful of tests like that catches the majority of the failures that would actually hurt.
Why a coverage percentage is a poor target
Coverage is the percentage of the code that runs during the tests. It is easy to measure, which is why it so often becomes the target. As a target it goes wrong in two ways.
First, it counts lines, not consequences. The code that formats a date in a footer and the code that calculates a tax both count as lines. Chasing a number fills the suite with tests for whatever is easiest to test, which is rarely what matters most.
Second, running code is not checking it. A test can execute every line of a function and assert nothing useful about the result. The percentage goes up; the protection does not.
Coverage is useful as a map: it shows which important areas no test touches at all. It is a poor goal.
Unit, integration and end-to-end
Tests come at different distances from the user:
- Unit tests check one piece of logic in isolation: a price calculation, a status rule, a date conversion. They are fast and precise, and they are the right tool for rules with many cases.
- Integration tests check that parts work together: that saving an order through the API really writes it, and that the database refuses what it should refuse.
- End-to-end tests drive a real browser through a real flow: open the page, fill the form, press the button, see the result. They are slower and more fragile, and they are the only ones that prove the user can actually do the thing.
A sensible first suite has a few end-to-end tests for the money paths, integration tests where data is written, and unit tests wherever there is a rule complicated enough that a person could get it wrong. Not the other way round.
Tests that run on every change
A test suite that only runs when someone remembers to run it is documentation, not protection. Tests earn their keep when they run automatically on every change, and when a failing test stops the change from being merged.
In practice that means a pipeline, a small automated job that runs whenever someone proposes a change:
on every proposed change:
1. install dependencies
2. build
3. run the unit and integration tests
4. run the end-to-end tests for the money paths
5. if anything failed, block the merge and say which test and whyTwo rules keep it useful. It has to be fast enough that people do not route around it, so the slow end-to-end tests are kept to the paths that matter. And a failing test is never ignored "just this once". A red test that everyone ignores teaches everyone to ignore red tests.
The most valuable tests in any mature suite were not planned. They were written the day a bug was fixed.
The rule is simple: every fix arrives with a test that fails before the fix and passes after it. The test reproduces the bug first, which also proves the bug has been understood, and then it stays in the suite for good. The same defect cannot come back unnoticed, which is exactly the kind of bug that is most embarrassing to ship twice.
Over time these regression tests become a record of everything that has actually gone wrong in the system, which is a far better guide to what can go wrong than anyone’s guesses at the start.
Real devices, real browsers
An end-to-end test proves a flow works in one browser on one screen size. Your customers use many. Which ones to check is not a matter of taste: your analytics show which browsers and devices your visitors actually use. Check those, starting with the most common phone, rather than a list copied from somewhere else.
Automated tests do not replace a person looking at the screen before a release. They catch what they were written to catch; a person notices that the button is now hidden under the cookie banner on a small phone. Both belong in a release.
Reports people can read
When a test fails, the report should say what broke without anyone digging through logs: the step, the input, what was expected, what happened, and for an end-to-end test, a screenshot of the moment it failed. A report like that turns a failed test into a five-minute fix. A report that says only "assertion failed" turns it into an afternoon, and after a few of those, people stop trusting the suite.
A worked example: the checkout
Take a shop. The first end-to-end test is the one path that pays for everything else: a customer finds a product, adds it to the cart, checks out and sees the confirmation. Written with Playwright, a common tool for driving a real browser, it reads almost like the steps a person would follow:
test('a customer can place an order', async ({ page }) => {
await page.goto('/products/blue-mug');
await page.getByRole('button', { name: 'Add to cart' }).click();
await page.getByRole('link', { name: 'Checkout' }).click();
await page.getByLabel('Name').fill('Test Customer');
await page.getByLabel('E-mail').fill('[email protected]');
await page.getByLabel('Phone').fill('0888000000');
await page.getByLabel('Cash on delivery').check();
await page.getByRole('button', { name: 'Place order' }).click();
await expect(page.getByText('Thank you for your order')).toBeVisible();
});Three details make a test like this last. It finds elements the way a user does, by their role and visible name, not by a CSS class that the next redesign will rename. It checks the outcome the customer sees, not an internal detail. And it runs against its own test data, not against whatever happens to be in the database that day.
Around that one test, a handful of integration tests then check what the browser cannot see: that the order was written with the right total, that the stock went down, that the payment record exists and starts as pending.
Test data: create it, do not borrow it
Most flaky and confusing tests are really data problems. A test that passes on Monday and fails on Tuesday is often reading a product someone deleted, or an order number that already exists.
The cure is for every test to create what it needs and to start from a known state:
- A fresh database for the test run, created from the migrations, so the tests prove the schema too.
- Small factories that create a valid customer, product or order in one line, with only the fields the test cares about spelled out.
- No test that depends on another test having run first. Each one sets up its own world.
It feels slower to write at first. It is what makes a suite you can run a thousand times without surprises.
Flaky tests, and how to stop them
A flaky test passes and fails without any change to the code. It is worse than no test, because it teaches everyone that a red result means nothing. The usual causes, and what to do about each:
- Timing: the test clicks before the page is ready. Wait for the thing you need to be visible or enabled, never for a fixed number of seconds.
- Shared data: two tests change the same record. Give each test its own.
- Time and dates: a test that works except at midnight, at the end of a month, or in another time zone. Freeze the clock in tests that depend on it.
- Outside services: a courier or payment sandbox that is slow today. Test your handling of their answers with recorded responses, and keep one separate check against the real sandbox.
The rule for a flaky test is simple: fix it the day it is noticed, or remove it and write down the gap. Never leave it red.
Testing what talks to the outside world
Business systems talk to other systems: payment providers, couriers, accounting software, e-mail. Tests should not depend on those services being up, and they should not create real parcels or real charges.
- For everyday tests, replace the outside service with a stand-in that returns recorded answers, including the unpleasant ones: a timeout, a refusal, a half-completed response.
- Keep a small, separate set of tests that talk to the provider’s own sandbox, run less often, to catch the day the provider changes something.
- Test your code’s reaction to failure as carefully as its success. What happens to an order if the courier refuses the label is a question with a right answer, and it deserves a test.
Testing permissions
Permissions deserve their own tests, because a permission bug does not look like a bug. Nothing breaks; someone simply sees what they should not. The shape of the test is always the same: create two users who should not see each other’s data, and try.
- As customer A, open customer B’s order by its address. It must fail.
- As an editor, call an administrator-only action directly, not through a hidden button. It must fail.
- As a user of one site, on a system that hosts several, read another site’s records. It must fail.
These tests are cheap, and they cover the class of failure that costs the most when it reaches the news.
The first week on an untested system
For a system with no tests at all, a realistic first week looks something like this:
- Set up the pipeline, even with a single trivial test, so that tests run on every change from day one.
- Write the end-to-end test for the main money path.
- Add the two-account permission test for the most sensitive records.
- Write a regression test for the last bug that reached customers.
- From then on, every fix arrives with its test.
After a few months of that last habit, the suite covers what actually goes wrong in that particular system, which no plan made at the start could have predicted.
- Exhaustive edge cases for code that is still changing every week.
- Tests for third-party behaviour you do not control; test your handling of it instead.
- Snapshot tests of whole pages, which fail on every harmless change and train people to accept failures without reading them.
- Anything whose only purpose is to raise the coverage number.
None of these is forbidden. They are simply not first.
Releases that feel like a gamble?
Tell me what the system does and what broke last time.