Skip to content
Tutorials Center
My learning

Topics

Lesson 55 of 60 ยท Real-World Projects

Project: Plan and Report a Release

Plan the testing of a release: rank features by risk, split the budget, run two test cycles, write the summary report and give a clear go or no-go recommendation.

Most testing work is not clever test design. It is planning what to test with limited time, running the work in organized rounds, and telling the people who decide about a release what you found. This project follows one imaginary release from the first planning meeting to the sentence "we recommend not shipping yet, and here is why".

The release is Bookshop 2.4, a small online bookstore with seven features. You are the tester and you have 48 hours of testing time. Everything in the editors is invented data, and the code is a stand-in for the spreadsheet or test-management tool a real team would use. The arithmetic, the decisions and the documents are the same as in real projects. You will use ideas from Test Strategy and Test Plan, Risk-Based Testing, Test Execution and Result Tracking and Defect Lifecycle and Quality Metrics.

From plan to decision

Plan Scope, risks, budget and exit criteria.
Prioritize Rank features by likelihood times impact.
Execute Run cycles and track every result.
Report Summarize facts and recommend go or no-go.
1 / 2

Step 1: write a one-page test plan

A plan is not a long document. It answers the questions people will ask later. For Bookshop 2.4, the plan fits on a page:

TEXT
TEST PLAN: Bookshop 2.4
Scope in:     Checkout and payment, discount codes, login and accounts,
              product search, order history, wishlist, profile page
Scope out:    Admin back office (unchanged), mobile apps (separate release)
Approach:     Risk-based. High-risk features get designed cases plus exploratory
              sessions; Medium get designed cases; Low get a smoke check.
              Two cycles: full run, then retest of fixes plus regression.
Environment:  Staging, same configuration as production, test payment provider
Data:         Seeded test accounts, no real customer data
Resources:    1 tester, 48 hours
Schedule:     Cycle 1 days 1-3, fix window days 4-5, cycle 2 days 6-7
Entry:        Build deployed to staging and smoke test passes
Exit:         No open Critical defects; no open Major defect in a High-risk
              feature; at least 95% of planned cases passed; every case for
              High-risk features executed
Risks:        New payment provider; late changes to discount rules

Notice two things. The exit criteria are written before testing starts, so nobody can bend them after seeing the results. And the plan names what is out of scope, so a missing test is a decision and not an oversight.

Step 2: rank the features by risk

You cannot test seven features equally in 48 hours, so you rank them. Each feature gets a likelihood (how probable is a fault, 1 to 5) and an impact (how bad is a failure, 1 to 5). The risk score is the product, and the score decides the depth of testing and the share of your time. Scores of 15 and above are High, 8 to 14 Medium, and below 8 Low. These cut-offs are a team decision, not a law.

JS Rank features and split the 48 hours

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The result is the heart of risk-based planning: checkout and payment get a third of your time, while the wishlist and the profile page together get about 4 hours. That is not unfair to the wishlist. It is an honest answer to "what is the worst thing that could happen if we get less testing here?" Writing the reason (why) next to each score also makes the ranking easy to challenge. If someone says the wishlist is more important than you think, you change the impact and watch the hours move.

Step 3: execute two cycles

A test cycle is one organized pass through the planned cases. Cycle 1 runs everything. The developers then fix defects, and cycle 2 retests the fixes and reruns the cases that could have been affected. Every case ends in a status: passed, failed, blocked (cannot run because something else is broken) or not run.

The stand-in below holds the invented results of both cycles. It computes the numbers that go into a status report: how many cases were executed, the pass rate, and the trend between cycles.

JS Summarize two test cycles

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The pass rate rises from 77.6% to 98.0%, which looks excellent. Look at the last detail, though. The one remaining failure is in checkout, the highest-risk feature. A pass rate is an average, and averages hide where the problems are. This is why exit criteria mention both a percentage and specific conditions.

Step 4: decide go or no-go

At the end, you compare the facts with the exit criteria from the plan. The stand-in below holds the defect list, evaluates the four exit criteria, and produces a recommendation. It then pretends the remaining Major defect is fixed and evaluates again, to show how the answer changes.

JS Check the exit criteria and recommend

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Today's recommendation is no-go, and the reason is short and checkable: a Major defect is open in checkout, the riskiest feature. If it is fixed and verified, the same rules give a "go", with three small known issues listed. The recommendation never depends on how the tester feels. It follows from rules that were agreed before testing began.

Step 5: write the summary report

The report is a page. It is read by managers who were not in the room, so it states the conclusion first, then the evidence.

TEXT
TEST SUMMARY: Bookshop 2.4, cycle 2 complete

Recommendation: NO-GO until D-103 is fixed and verified.

Why:      Exit criterion "no open Major defect in a High-risk feature" is not met.
          D-103: shipping fee is wrong for orders of exactly $50.00.
Results:  49 cases planned, 48 passed, 1 failed, 0 blocked, 0 not run.
          All 14 checkout cases were executed.
Defects:  6 found, 2 closed, 4 open (1 Major, 2 Minor, 1 Trivial).
Risks:    The payment provider is new; payment was tested in the provider's
          test mode only. Search accents (D-104) can affect non-English users.
Not tested: Admin back office and mobile apps (out of scope, see plan).
Next:     Retest D-103 and rerun the checkout regression set (about 3 hours).

Each line earns its place. "Not tested" matters because stakeholders assume everything was tested unless you say otherwise. "Next" turns the report into a plan.

Never write "the software is bug-free" or "testing is complete". As the seven testing principles say, testing shows the presence of defects, not their absence. Write what you tested, what you found, and what risk remains.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why are the exit criteria written in the test plan before testing starts?

    Criteria agreed in advance keep the go or no-go decision objective. Setting them after seeing results invites bending them to fit the date.
  2. 2. A feature has likelihood 3 and impact 4. What is its risk score and band, with High at 15 and above?

    Score is likelihood x impact = 12, which is below 15, so it is Medium. High would need 15 or more.
  3. 3. A pass rate of 98% looks great. Why can it still lead to a no-go?

    An average hides distribution. One open Major defect in checkout can matter more than many passed cases elsewhere, which is why criteria also name severity and risk.
  4. 4. Who makes the final decision to release?

    Testers provide information and a recommendation. The owner decides whether to accept the remaining risk.
  5. 5. Which line in a summary report do stakeholders most often assume without being told?

    Without an explicit 'not tested' line, readers assume everything was covered. Naming the gaps makes the remaining risk visible.

Summary

  • A good plan is short: scope in and out, approach, environment, schedule, entry and exit criteria, and risks.
  • Risk score = likelihood x impact; it decides how deeply and how long each feature is tested.
  • Run organized cycles and track passed, failed, blocked and not-run cases; remember that a percentage hides where the failures are.
  • Compare results with the exit criteria that were agreed up front, then recommend go, go with known issues, or no-go.
  • Keep the report to a page: conclusion first, then evidence, risks, what was not tested, and next steps.

Continue with Project: Run a Bug Triage Session.