Skip to content
Tutorials Center
My learning

Topics

Lesson 2 of 60 ยท Testing Fundamentals

The Seven Testing Principles

Learn the seven software testing principles: defects present, no exhaustive testing, early testing, defect clustering, pesticide paradox, context, and absence-of-errors fallacy.

Over decades, testers noticed the same patterns on very different projects. Those patterns were collected into seven short statements called the testing principles. They are not laws of physics, but they are excellent guides: each one explains a common mistake and tells you what to do instead.

Keep them in mind for the rest of the track. Every technique you will learn later, from equivalence partitioning to risk-based testing, exists because of one or more of these principles. If you have not yet read the vocabulary of errors, defects and failures, start with What Is Software Testing?.

The seven principles

Defects are present Testing shows bugs, never their absence.
No exhaustive testing There are too many combinations.
Test early Start before the code exists.
Defects cluster A few modules hold most bugs.
1 / 2

1. Testing shows the presence of defects, not their absence

A failing test is proof that something is wrong. A passing test is only evidence that this specific check passed. Testing lowers the probability of hidden defects but cannot reduce it to zero.

What to do: report results as risk and coverage ("we checked these areas, not those"), never as "it is bug-free".

2. Exhaustive testing is impossible

Trying every combination of inputs, states and paths is not feasible except for trivial software. Even a tiny function has an astronomical number of possible inputs. So testers must choose, using risk, priorities and test design techniques.

The next stand-in makes the size of the problem concrete. It counts the inputs of a function that takes two 32-bit integers, and a sign-up form with a few ordinary fields.

JS How many inputs are there, really?

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Two integers already need centuries of testing, and a form with four simple fields has far more combinations than anyone could run. This is why you will learn equivalence partitioning and boundary value analysis: they pick a small number of inputs that stand for all the others.

3. Early testing saves time and money

Testing activities should start as soon as possible, ideally with the first requirement. Reviewing a requirement, a mock-up or a design finds defects before anyone has built on them, and testers who ask "how would we check this?" early often expose vague or contradictory wording. This is often called shift-left (see Testing in the SDLC). The cost curve in the previous lesson is the reason.

What to do: involve testers in requirement and design discussions, review documents, and prepare test cases while developers code.

4. Defects cluster together

A small number of modules usually contain most of the defects, a pattern often compared to the 80/20 rule. Reasons include complex logic, many changes, tight deadlines, an unfamiliar technology or a developer who was rushed in that part of the code.

What to do: track where defects are found and spend more attention on the hot spots, but do not ignore everything else.

5. Beware of the pesticide paradox

If you run the same tests again and again, they eventually stop finding new defects, just as insects become resistant to the same pesticide. The software may still be full of bugs, but your tests only look in the places already cleaned.

What to do: review and refresh your tests regularly, add new cases, vary the data and use different techniques such as exploratory testing. (Regression suites are meant to stay stable, since their job is to catch new breakage in old behavior, but even they need maintenance.)

The next stand-in shows both principle 4 and principle 5 with invented but deterministic data. First it ranks the modules of a pretend product by defects found, then it replays a fixed test suite against a product that keeps changing.

JS Clustering and the pesticide paradox

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The first table shows the cluster: the top module alone holds well over half of the defects. In the second half, the fixed suite goes on reporting PASS while one fresh idea exposes a defect the old tests never touch. (The requirement is part of the made-up story.)

6. Testing is context dependent

There is no single right way to test. A video game, a bank, an airline control system and a marketing page have different risks, so they deserve different amounts and kinds of testing. A regulated medical device needs documented, traceable testing; a prototype might need a quick exploratory session.

What to do: choose your approach from the product, the risks, the users, the technology, the regulations and the team, and never copy another project's process blindly.

7. The absence-of-errors fallacy

Finding and fixing many defects does not help if the system is hard to use, does not meet the users' needs, or solves the wrong problem. A flawless implementation of the wrong requirements is still a failed product.

What to do: test against real needs as well as written requirements. Involve users, check usability and ask whether the product is valuable, not only whether it is correct. This is the difference between verification and validation, covered in QA, QC and Testing.

Using the principles together

The principles often pull on one another. Early testing (3) and defect clustering (4) tell you where to look; the impossibility of exhaustive testing (2) tells you that you must prioritize; the pesticide paradox (5) tells you to keep your tests fresh; context (6) tells you how much rigor to apply; and principles 1 and 7 remind you what test results can and cannot say.

When someone asks "have you tested everything?", the honest answer is always "no, and here is what we chose to test and why". Principle 2 is what makes that answer professional rather than evasive.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Which principle explains why testers use techniques such as boundary value analysis?

    Because every input cannot be tried, design techniques choose a small, effective subset. Boundary value analysis picks the values where defects are most likely.
  2. 2. A regression suite has passed for six months and no new defects have been found by it, yet users report bugs. Which principle is the best fit?

    Repeating the same tests eventually stops finding new defects. Reviewing and adding fresh tests, including exploratory ones, helps.
  3. 3. A team delivers a product with almost no defects, but customers refuse to use it because it does not do what they need. Which principle is this?

    A product with few defects can still fail if it does not meet user needs. Validation checks that the right product was built.
  4. 4. Most defects found so far are in the payment module. What is the sensible reaction?

    Defects cluster, so hot spots deserve more effort, but it is a probability, not a guarantee. Other areas still need some coverage.
  5. 5. A test lead copies the exact test process of a hospital-software project onto a small marketing website. Which principle does this ignore?

    How much and what kind of testing is right depends on the risks and context, so copying a heavyweight process blindly wastes effort.

Summary

  • Testing shows the presence of defects, not their absence, so report risk and coverage, not certainty.
  • Exhaustive testing is impossible; choose tests with techniques and priorities.
  • Test early: reviews and early test design find defects when they are cheapest to fix.
  • Defects cluster, so track hot spots and spend extra effort there.
  • Refresh tests regularly to avoid the pesticide paradox.
  • Test according to context: risks, users, technology and regulation decide how.
  • A system with no known defects can still fail if it does not meet user needs.

Next up: QA, QC and Testing.