Skip to content
Tutorials Center
My learning

Topics

Lesson 26 of 60 ยท Test Planning and Documentation

Test Data and Test Environments

How testers create safe, realistic test data, protect privacy with masking, keep environments close to production and use test doubles, with runnable stand-ins.

A test is only as good as the data it runs on and the place it runs in. The same login test can pass on your laptop and fail in production because the database holds names with apostrophes, the mail server is slower, or a feature flag is set differently. Test data and test environments are two of the biggest hidden causes of "it works on my machine" and of tests that nobody trusts.

This lesson covers where test data comes from, how to protect real people's information, what it means for an environment to resemble production, and how test doubles stand in for services you cannot or should not call. The code is a set of stand-ins: small JavaScript programs that show the ideas. Real projects use database seeding tools, data-masking products, and service virtualization platforms, but the reasoning is the same.

Where test data comes from

There are four common sources, and most teams mix them.

Four sources of test data

Hand-made Small, precise sets for specific cases.
Generated Created by a script, often with a seed.
Production copy Realistic, but risky if not masked.
Synthetic Fake data shaped like the real thing.

Never use unmasked personal data outside production.

1 / 2

Hand-made data is best for exact cases such as the boundaries from Boundary Value Analysis: you need a user who is exactly 18 years old, not "some adult". Generated data is best when you need volume (10,000 orders to check a report) or variety. Production copies look realistic, which is exactly why they are dangerous. Synthetic data keeps the statistical shape of production without containing any real person.

Privacy and data masking

If real customer data reaches a test system, you have multiplied the places where it can leak: more people have access, security is weaker, and laptops get lost. Privacy laws and company policies usually treat test copies of personal data just like production data, so the safest rule is simple: do not copy it. When a production-like set is truly needed, mask it first.

Masking techniques include:

  • Substitution: replace a real name with a fake but plausible one.
  • Shuffling: reorder a column so values no longer match their original rows.
  • Redaction and truncation: keep only the last four digits of a card number, or only the domain of an email address.
  • Tokenization or hashing: replace a value with a stable code so joins between tables still work.

Good masking keeps the format and relationships (an email still looks like an email, one customer still owns the same orders) while removing the identity. The check is: could someone work out who this person is from what remains?

Masking is easy to get wrong. A hashed email without a secret can often be reversed by guessing common addresses, and a "masked" dataset with a unique birth date and postcode can still identify one person. Treat masking as a security task and involve the people who own data protection.

Test environments

An environment is the combination of servers, configuration, data, versions and connected services where the tests run. Most teams have several: a developer's machine, a shared integration or test environment, a staging environment that mirrors production, and production itself.

Environment parity is how closely a test environment matches production. The closer it is, the more a passing test tells you about the real release. The usual differences that bite:

  • different versions (database, operating system, browser, runtime);
  • different configuration (timeouts, feature flags, email sending turned off);
  • different data volume (100 rows in test, 50 million in production, so a slow query never shows);
  • different infrastructure (one server versus a load-balanced cluster).

You will not match production exactly, because of cost. The practical skill is to know the differences, write them down, and avoid concluding more than the environment allows. A performance result from a tiny test server says little about production capacity.

Other environment habits that save time:

  • Reset to a known state before a run, so yesterday's tests do not affect today's.
  • Own your data. If two testers share an account and one changes the password, the other's tests fail for no reason.
  • Label the environment in the bug report, as covered in Writing Excellent Bug Reports.

Test doubles for services

Real systems call other systems: a payment provider, an email service, a map API. In tests you often cannot call the real one (it costs money, sends real emails, is slow or is down). A test double replaces it. The common kinds, from simplest to smartest, are:

Double What it does
Stub Returns fixed answers to calls.
Fake A working but simplified version (an in-memory database).
Mock A stub that also records calls so you can check how it was used.
Spy Wraps the real thing and records what happened.

You met the idea of stubs and drivers in Test Levels. For manual testers, doubles often appear as a sandbox from the provider (a payment test mode with special card numbers) or a mock server the team runs. Be clear in your report when a result came from a double, because a passing test against a stub proves your code handles the stub's answers, not the real service's.

Try it: a seeded data generator

Random data makes tests flaky if each run produces something new. A seed fixes that: the same seed always gives the same "random" data, so a failure can be reproduced. This stand-in uses a tiny pseudo-random generator and builds customers, including some deliberately awkward values.

JS Seeded test data generator

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Notice the two things combined: random but reproducible data for variety, plus hand-added edge cases (an empty name, a very long name) that a generator would rarely produce on its own. Test email addresses use the reserved .example domain so nothing is ever delivered to a real person.

Try it: masking production-like data

This stand-in takes a few "production" rows (all invented) and masks them while keeping the format and the relationships. Customer 101 still owns the same order, but the identity is gone.

JS Mask personal data, keep relationships

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The two final lines are the testable idea: masking passes when relationships are intact and identities are gone. In a real project you would run similar checks over the whole masked database before releasing it to the test team.

Types of test data

Test data is not one thing. A thorough set for a field or a feature mixes several kinds, mapped to the techniques you already know.

Type What it is Example for an "age" field (18 to 65)
Valid Accepted by the rules 30
Invalid Breaks a rule 12, 70, "abc"
Boundary On or next to a limit 17, 18, 65, 66
Equivalence representative One value standing for a whole class 40
Null / empty / missing Nothing supplied "", null, field absent
Special characters and encodings Characters that break parsers O'Brien, <b>, emoji, accents
Extreme size Very long or very large 10,000 characters, 2 GB file
Historical or time-based Dates and clocks 29 February, daylight-saving change
Large volume Many rows for performance A million orders
Synthetic Invented but realistic data Generated customers
Masked production data Real structure with sensitive values replaced Names and card numbers scrambled

Choose the type from the question. Functional checks need valid, invalid and boundary values (equivalence partitioning, boundary value analysis). Negative and security checks add special characters and extreme sizes (positive and negative testing). Performance checks need large, realistic volumes.

Ways to produce it

  1. Write by hand for small, targeted sets.
  2. Generate synthetically with a script or library, using fixed seeds so the data is repeatable.
  3. Copy and mask production data to keep realistic distributions without exposing people.
  4. Use fixtures and factories in automated tests: small helper functions that build one valid object, which tests tweak.
  5. Subset and refresh large data sets so environments stay small and current.

Whichever you pick, store the data with the test, keep it under version control when possible, and document what it represents.

JS Generate test data for every type

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Whether the string "25 " should be accepted is a requirement question: the data exposes it, and you ask the owner.

Volume and privacy

Large-volume data should resemble production in size, skew (a few very busy customers) and relationships, because performance depends on all three. Never copy raw production personal data into shared environments: mask it, as shown in the masking example above, or generate synthetic data.

Putting it together

When you plan data and environments for a test cycle, ask:

  1. What data does each test need, and which tests can share it safely?
  2. Is any of it personal? If yes, is it synthetic or masked?
  3. How does this environment differ from production, and what can we not conclude as a result?
  4. Which external services are real, and which are doubles?
  5. How do we reset to a known state?

Write the answers into the test plan; they belong beside scope and schedule.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why use a seed when generating random test data?

    A seeded generator is deterministic. Re-running with the seed gives the same data, which is essential for debugging and for stable tests.
  2. 2. Which is the safest approach to personal data in a test environment?

    Real personal data should not leak into weakly protected systems. Synthetic or properly masked data stays useful without exposing people.
  3. 3. A payment sandbox accepts every special test card. What should you remember when a test passes?

    A double or sandbox imitates the service. Results are valuable but limited, and the limitation should be stated in reports.
  4. 4. Staging has 200 rows; production has 40 million. What is the main risk?

    Data volume is a classic parity gap. Slow queries, memory use and paging bugs often show only at realistic size.

Summary

  • Test data comes from hand-made, generated, copied and synthetic sources; use each where it fits, and always add edge cases.
  • Do not use unmasked personal data in test systems; mask by substitution, shuffling, redaction or tokenization while keeping format and relationships.
  • Environment parity matters: know and record how test differs from production in versions, configuration, volume and infrastructure.
  • Test doubles (stubs, fakes, mocks, spies, sandboxes) replace external services; state clearly when a result relies on one.
  • Seeds make generated data reproducible.

Next: Traceability and Test Estimation.