Skip to content
faker.tools
All 79

Guide · 11 min read

Test Data for End-to-End Tests: Patterns for Playwright and Cypress

Most flaky end-to-end tests are flaky because of their data. Isolation, unique-per-worker records, API seeding and reproducible randomness — with code for Playwright and Cypress.

Key takeaways

  • Every end-to-end test should create the data it needs and must not depend on data another test created or changed.
  • Parallel workers collide on unique constraints. Make generated values unique per worker and per test — for example, include Playwright's workerIndex and testId in every email address.
  • Create test data through the API or directly in the database, not through the UI. Only the flow you are testing should go through the browser.
  • Use reserved values — example.com emails, 555-01xx phone numbers — so a misconfigured test environment can never message a real person.
  • Random test data is fine if it is reproducible: derive the seed from the test's title and print it on failure.

When an end-to-end test fails intermittently, the instinct is to blame timing and add a wait. Sometimes that is right. More often the test is failing because of its data: it assumed a record that another test deleted, it created a user whose email a parallel worker had created a millisecond earlier, or it computed "next Tuesday" on a Monday night in UTC.

Data problems look like timing problems because they depend on which tests ran first and on which worker. The fixes are structural, and they are the same whether you use Playwright, Cypress or anything else.

Rule 1: every test owns its data

A test should create everything it modifies and must not depend on anything another test created. That single rule removes ordering dependencies, makes tests runnable on their own, and makes parallelism safe.

Split your data into two kinds:

Kind Examples How to provide it
Reference data Countries, plans, feature flags, a product catalogue Seed once before the run, treat as read-only
Test-owned data The user who signs up, the order that is refunded Create inside the test or its fixture

If a test edits reference data, it is test-owned data, and it needs its own copy.

Rule 2: unique per worker, unique per test

Run tests in parallel and two workers will eventually generate the same email address, and the second sign-up fails on a unique constraint. Random values make the collision rare — which is worse, because rare means intermittent.

Make uniqueness structural instead. In Playwright, a fixture can build values from the worker and test identity:

// fixtures.ts
import { test as base } from "@playwright/test";

type TestUser = { id: string; email: string; password: string };

export const test = base.extend<{ user: TestUser }>({
  user: async ({ request }, use, testInfo) => {
    const email = `e2e+w${testInfo.workerIndex}-${testInfo.testId}@example.com`;
    const password = "Correct-Horse-Battery-9";

    const response = await request.post("/api/test-support/users", { data: { email, password } });
    const { id } = await response.json();

    await use({ id, email, password });

    await request.delete(`/api/test-support/users/${id}`);
  },
});

export { expect } from "@playwright/test";

Every test that asks for user gets a fresh account that no other test can see. The address uses example.com, which RFC 2606 reserves so that mail to it is never delivered — a misconfigured staging mailer cannot reach a real inbox.

In Cypress, the equivalent is a beforeEach that creates data with cy.request(), using Cypress.currentTest.title or a run-specific prefix to keep values distinct.

Rule 3: create data through the API, not the UI

Signing up through the UI in every test that needs a logged-in user is slow and couples every test to the sign-up form. If the form breaks, two hundred unrelated tests fail.

Create data through the fastest reliable path — a test-support API endpoint, an admin API, or a direct database call from a Node task — and reserve the browser for the behaviour under test. Authentication works the same way: log in once per worker through the API, save the session with Playwright's storageState, and reuse it.

Test-support endpoints need guarding: compile them out of production builds or require a secret that only CI has.

Rule 4: make randomness reproducible

Generated data finds bugs that hand-written data does not — an apostrophe in a surname, a long company name, a Unicode city. The cost is that a failure may depend on the exact values drawn. Make them reproducible by deriving the seed from the test itself:

import { faker } from "@faker-js/faker";

test.beforeEach(async ({}, testInfo) => {
  const seed = hash(testInfo.titlePath.join(" > "));
  faker.seed(seed);
  testInfo.annotations.push({ type: "seed", description: String(seed) });
});

Here hash is any stable string hash, such as FNV-1a. The seed now appears in the HTML report, and the same test always draws the same values. When you want variety across runs, include the CI run number in the hash and print it. Seeded randomness explains the technique in depth.

Rule 5: never read the clock in test data

"Created three days ago" and "expires next week" are relative to now, and now changes. Tests that build dates from the current time fail around midnight, at month ends, across daylight-saving changes, and when CI runs in a different time zone from your laptop.

Pin the clock instead. Playwright's page.clock API can install a fake clock in the browser, Cypress has cy.clock(), and your test-support API can accept an explicit "now". Generate dates relative to that fixed instant.

Stub, or use the real backend?

Neither, exclusively.

Stubbed tests — Playwright's page.route(), Cypress's cy.intercept() — are fast, deterministic and the only practical way to test rare failures such as a 503 from a payment provider. They do not tell you whether the frontend and backend still agree.

Real-backend tests catch integration failures and migrations that broke a query. They are slower and need the data discipline above.

A healthy suite has many stubbed tests for UI behaviour and edge cases, and a smaller set of real-backend journeys — sign up, check out, refund — for integration. Mocking REST APIs for frontend development covers the stubbing side.

Keep personal data out of the pipeline

End-to-end tests produce more copies of data than any other kind of test: screenshots, videos, traces, HTML reports, CI logs. Every real name or email in the test database ends up in all of them, retained for weeks and visible to everyone with CI access.

Use generated data with the right shape — the name, email and phone number generators default to reserved, non-deliverable values — and the question of what your test artefacts contain never comes up.

A checklist

  • Each test creates the data it modifies.
  • Reference data is seeded once and never modified by tests.
  • Unique fields include the worker index and test id.
  • Data is created through an API or the database, not the UI.
  • Emails use example.com; phone numbers use 555-01xx.
  • Random values are seeded, and the seed is reported on failure.
  • Dates are computed from a pinned clock.
  • The database is reset between CI runs.
  • No production data anywhere in the pipeline.

Frequently asked questions

Why are my end-to-end tests flaky?

The most common causes are data-related: tests that depend on records another test created or modified, parallel workers creating the same unique value, leftover state from a previous run, and dates computed from the current time. Fix the data isolation first; most flakiness disappears with it.

Should each test create its own data or share a seeded dataset?

Both, for different kinds of data. Read-only reference data — countries, plans, product catalogues — can be seeded once before the run. Anything a test modifies should be created by that test, so tests can run in any order and in parallel.

How do I create unique test data per Playwright worker?

Use testInfo.workerIndex (or parallelIndex) and testInfo.testId in generated values, for example e2e+w3-7f2a@example.com. Each worker and each test then gets distinct emails, usernames and slugs even when they run at the same moment.

Should I clean up test data after each test?

Clean up if your environment is long-lived or shared; otherwise prefer isolation by tenant or by a per-run prefix, and reset the whole database between CI runs. Cleanup in afterEach is skipped when a test crashes, so never rely on it for correctness.

Is it OK to use production data in end-to-end tests?

No. It exposes personal data to every tool in your test pipeline — screenshots, videos, traces and logs — and it changes underneath your tests. Use generated data that has the same shape.

Tools this guide covers

Full directory →