Skip to content
Tutorials Center
My learning

Topics

Lesson 47 of 60 ยท Automation Concepts and Process

Testing in Agile Teams and CI/CD

How testers work in agile teams: three amigos, definition of done, testing quadrants, BDD scenarios, CI/CD pipelines and quality gates, with runnable stand-ins.

In a traditional project, testing was a phase: developers built for months, then testers received a big package and hunted for problems. Agile teams ship small pieces every week or two, so there is no time for a separate testing phase at the end. Testing becomes something the whole team does continuously, from the first conversation about a story to the moment the change reaches users. Continuous integration and delivery (CI/CD) add the machinery: every change is built and checked automatically, many times a day.

This lesson covers the habits and tools that make that work: shared conversations before coding, a clear definition of done, the testing quadrants, behavior-driven scenarios, pipelines and quality gates. The real pipeline needs a server and a repository, so the code here is a stand-in that shows the logic of a pipeline and of a plain-language scenario. This builds on testing in the SDLC and the idea of shifting left.

Quality is a team responsibility

In an agile team, testers are not a gate at the end. They bring a testing mindset into every step:

  • Refinement: asking "what if...?" questions while a story is still being shaped, which is the cheapest time to find a gap. Reviewing requirements for testability is the skill in action.
  • Planning: helping estimate the testing effort and spot risks, as in risk-based testing.
  • During the sprint: testing small increments as soon as they are ready, not after the sprint's last day.
  • Review and retrospective: sharing what was learned and improving the process.

Developers write unit tests and often more. Testers specialize in exploration, test design and spotting risks. Both care about quality, and neither can hand the other the responsibility. For the Scrum background, see Scrum overview and what is agile.

The three amigos

A powerful practice is a short meeting before development starts, with three perspectives around one user story: a business person (product owner or analyst) who knows the need, a developer who knows how to build it, and a tester who asks what could go wrong. Fifteen to thirty minutes of talking through examples exposes misunderstandings that would otherwise appear as bugs weeks later.

The outcome is not a long document but a short list of concrete examples and acceptance criteria: "a customer with a gold card gets 10 percent off orders over 50, but not on already discounted items". Those examples later become tests. See user stories for how stories are written.

Three amigos

Business What problem are we solving?
Developer How will we build it?
Tester What could go wrong?

Examples agreed early become tests later.

1 / 2

Definition of done

The definition of done (DoD) is the team's shared checklist for when a story counts as finished. Without one, "done" means different things to different people, and unfinished work, such as untested code, quietly piles up as "technical debt".

A typical DoD includes:

  • Code is written, reviewed and merged.
  • Automated unit and integration tests are written and passing.
  • Acceptance criteria are verified, including by exploratory testing.
  • No known critical or high-severity defects remain open for the story.
  • Documentation and monitoring are updated if needed.
  • The change is deployable through the pipeline.

Testers help keep the DoD honest. It differs from acceptance criteria, which are specific to one story, while the DoD applies to all of them.

Agile methods and testing challenges

Testing looks slightly different in each agile method. In Scrum, testers join sprint planning, daily scrums and reviews, and the definition of done includes tested work. Extreme Programming (XP) puts testing into the developers' practices: test-driven development, pair programming and continuous integration. Kanban limits work in progress so that testing does not pile up as a queue at the end of the board. Crystal adjusts the amount of process to team size and project criticality, and feature-driven development (FDD) builds in short feature-based iterations with inspections.

Common challenges are the same in all of them: requirements that change mid-sprint, a squeezed testing window at the end of the sprint, test automation that falls behind new features, flaky tests that fail randomly and train people to ignore failures, and over-reliance on automation at the expense of exploratory testing. Tips that help: test from the first day of a story, keep automation in the sprint's definition of done, quarantine and fix flaky tests quickly, and review the test approach in each retrospective.

The agile testing quadrants

The agile testing quadrants (popularized by Brian Marick and developed by Lisa Crispin and Janet Gregory) are a way to remember that a team needs different kinds of testing for different purposes. Two axes organize them: tests that support the team versus tests that critique the product, and tests that are technology-facing versus business-facing.

Quadrant Purpose Examples Usually
Q1: technology-facing, supports the team Guide the design of the code Unit and component tests Automated
Q2: business-facing, supports the team Confirm the behavior the business wants Acceptance tests, examples, story tests Automated and manual
Q3: business-facing, critiques the product Find what was missed Exploratory testing, usability, user acceptance Manual
Q4: technology-facing, critiques the product Check qualities such as speed and safety Performance, security, reliability Tool-supported

The quadrants are not a sequence or a rule that every project must fill each one equally. They are a checklist question: "have we thought about all four kinds of testing?" Quadrant 4 covers the lessons on performance, security and reliability.

Behavior-driven development in brief

Behavior-driven development (BDD) turns the three-amigos examples into plain-language scenarios written in a structured form called Given, When, Then:

  • Given the starting situation,
  • When something happens,
  • Then the expected outcome.

In the common Gherkin format a scenario looks like this (read-only; a real BDD tool such as Cucumber matches each line to code):

TEXT
Feature: Gold card discount

  Scenario: Gold customer gets 10 percent off a large order
    Given a customer with a gold card
    And a basket totalling 80.00
    When the customer checks out
    Then the total to pay is 72.00

  Scenario: Small orders get no discount
    Given a customer with a gold card
    And a basket totalling 40.00
    When the customer checks out
    Then the total to pay is 40.00

The value is shared language: product owners can read and challenge the scenarios, and the same text can run as an automated test. The risk is that teams treat BDD as just a test tool and skip the conversation, ending up with brittle scripts nobody reads. Keep scenarios short, about behavior and not about buttons.

Try it: a tiny Given/When/Then runner

Real tools map each sentence to a step function with a pattern. This stand-in does the same with a few lines: a list of step definitions with regular expressions, and a runner that executes the scenarios above against a small discount function.

JS Run Given/When/Then scenarios

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The third scenario fails on purpose. The rule says orders over 50 get the discount, so exactly 50 pays the full price, but the scenario claims 45.00. Either the scenario or the rule is wrong, and the run turned a vague disagreement into a concrete question the three amigos can settle. That is the real benefit of examples.

Continuous integration and delivery

Continuous integration (CI) means developers merge small changes into a shared branch often, and every merge triggers an automated build and test run. Bugs are found within minutes, while the author still remembers the change. Continuous delivery keeps the product always in a releasable state; continuous deployment goes one step further and releases every change that passes automatically.

A pipeline is the automated sequence of stages a change moves through:

flowchart LR
  A["Commit"] --> B["Build"]
  B --> C["Unit tests"]
  C --> D["Integration<br/>and API tests"]
  D --> E["Deploy to<br/>test environment"]
  E --> F["End-to-end<br/>smoke tests"]
  F --> G["Release"]

Fast, cheap checks come first; slower, broader ones run later on changes that survived.

Ordering matters: put the quickest, cheapest checks first so a broken change fails in two minutes, not thirty. This mirrors the test pyramid. A failed stage stops the pipeline, and the team's rule is to fix a red pipeline before doing anything else.

A real pipeline definition is a configuration file in the repository. This read-only GitHub Actions example shows the shape (see your first workflow and pipeline anatomy for the details):

YAML
name: ci
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm test

Quality gates

A quality gate is a rule the pipeline enforces before a change may move on. Typical gates:

  • All unit and integration tests pass.
  • Code coverage does not fall below an agreed level (remember that coverage is a signal, not a goal, and a threshold alone does not prove quality).
  • No new critical issues from static analysis or dependency scanners.
  • The performance check stays within its budget.
  • No open blocker defects for the release.

Gates turn agreements into automatic decisions, but gates can be too strict (everyone ignores them or finds workarounds) or too loose (they never stop anything). Review them in retrospectives. More on this in code quality gates.

Try it: a pipeline with a quality gate

This stand-in runs stages in order, stops at the first failure and applies gates to the results. The numbers are invented, and nothing is really built or deployed, but the control flow is the one every CI system uses. Change a value to make a stage fail.

JS Pipeline and quality gates

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The failing integration test blocks the change, and the smoke stage is skipped because nothing after a failure should run on a broken build. Fixing it and running again flips the result to green.

The tester's role in a pipeline

Automation does not make testers redundant; it changes the work:

  • Helping design the automated checks and deciding which belong at which level.
  • Exploratory testing on every increment, covering what the pipeline cannot, as in exploratory testing in practice.
  • Watching the pipeline's health: flaky tests, slow stages, gaps in coverage.
  • Testing in production safely with monitoring and feature flags, the "shift-right" side.
  • Keeping the team honest about the definition of done and about risks.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the main purpose of the three amigos meeting?

    Business, development and testing perspectives discuss concrete examples early, which prevents misunderstandings from turning into defects.
  2. 2. Which agile testing quadrant contains exploratory testing and usability testing?

    Q3 critiques the product from the business point of view and is mostly manual: exploratory, usability and user acceptance testing.
  3. 3. Why does a pipeline run fast checks like unit tests before slow end-to-end tests?

    Failing early saves time and resources. Slow, broad tests run only on changes that passed the cheap ones.
  4. 4. How does the definition of done differ from acceptance criteria?

    The DoD is the team-wide checklist for finished work. Acceptance criteria describe what a particular story must do.

Summary

  • In agile teams testing is continuous and shared; testers contribute from refinement to retrospective.
  • The three amigos agree on concrete examples early; the definition of done keeps "finished" honest.
  • The testing quadrants remind the team to cover supporting and critiquing tests, technical and business-facing.
  • BDD scenarios in Given/When/Then form turn examples into shared language and automated checks.
  • CI/CD pipelines run ordered stages with quality gates; fast checks first, red builds fixed immediately.

Next: Project: Test a Login Form.