Skip to content
Tutorials Center
My learning

Topics

Lesson 33 of 44 ยท Comparative Politics

The Comparative Method: Designs, Case Selection and Pitfalls

Learn Mill's methods, most similar and most different systems designs, how to choose cases, and why selecting on the outcome misleads, with runnable demonstrations.

If you want to know why some countries experience civil war, or why some welfare states are generous, you cannot run an experiment. But you can compare. The comparative method is the disciplined use of comparison to test causal ideas, and it rests on a logic that is older than political science: look for what changes together.

This lesson explains John Stuart Mill's methods, the two classic research designs that grew from them, how to choose cases, and the pitfalls that catch beginners and experts alike.

Two cards describing the most similar systems design and the most different systems design
Both designs narrow down possible causes by holding some things constant while others vary.

What you will learn

  • Mill's methods of agreement and difference.
  • The most similar and most different systems designs.
  • How case studies, small-N comparison and large-N statistics fit together.
  • Why selecting cases on the outcome biases results.

Key terms

Independent variable: the factor you think causes something; also called the explanatory variable.

Dependent variable: the outcome you want to explain.

Most similar systems design (MSSD): comparing cases that are alike in most respects but differ in the outcome.

Most different systems design (MDSD): comparing cases that are unlike in most respects but share the outcome.

Selection bias: a distortion that arises when the way cases are chosen is related to the outcome.

Mill's methods

In A System of Logic (1843), John Stuart Mill described methods of experimental inquiry. Two matter most for comparison.

  • The method of agreement. If two or more cases share an outcome and have only one circumstance in common, that circumstance is a candidate cause.
  • The method of difference. If a case where the outcome occurs and a case where it does not are alike in every circumstance except one, that circumstance is a candidate cause.

Mill himself thought these methods fit experiments better than social science, because real countries differ in countless ways. They still organize thinking. Adam Przeworski and Henry Teune (1970) turned them into research designs. A most similar systems design picks cases that resemble each other, say neighboring countries with similar histories and incomes, and then asks what differs when outcomes differ. A most different systems design picks very dissimilar cases that share an outcome and asks what they have in common.

Theda Skocpol's States and Social Revolutions (1979) is a famous example of this tradition. She compared the French, Russian and Chinese revolutions, looking for shared conditions in the states and international pressures they faced, and contrasted them with cases where revolution did not occur.

JS A simple method-of-difference finder

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Choosing cases

Case selection is where comparative research is won or lost. Several principles help.

  • Vary the cause, not just the outcome. You need cases where the suspected cause is present and absent, and where the outcome is present and absent.
  • Do not select only successes. Studying only revolutions, or only economic miracles, makes it impossible to see what is not distinctive about them. Barbara Geddes (1990) called this the danger of selecting on the dependent variable.
  • Consider the type of case. Arend Lijphart (1971) and Harry Eckstein distinguished case study types. A crucial case is one where a theory predicts a very clear outcome; if it fails there, the theory is in trouble. A deviant case departs from what theory expects and can suggest missing factors.
  • Watch for diffusion. Countries influence each other, so cases are not always independent. Treating neighbors that copy each other's laws as separate tests overstates the evidence.

Selection bias in action

Suppose a factor X truly raises an outcome Y on average, though with lots of noise. If you study only cases with very high Y, you throw away the cases that would show the relationship, and it looks weak or absent. The simulation below uses made-up data to demonstrate.

Python Selecting on the outcome (simulation)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The dashed line is flatter than the true line. By studying only the high-outcome cases, you conclude that X barely matters, a mistake caused by the sample, not by the world.

Small-N and large-N

Political scientists use a range of approaches. Single case studies build deep knowledge and generate theories. Small-N comparisons use Mill-style logic and process tracing, which follows the chain of events linking a cause to an outcome using detailed evidence. Qualitative comparative analysis (QCA), developed by Charles Ragin, uses set logic to study combinations of conditions. Large-N statistical studies test patterns across dozens or hundreds of countries while controlling for other factors. Each approach trades depth for breadth, and mixed methods combine them: a statistical pattern is followed up with case studies of typical and deviant cases.

Common misconceptions

"Two similar countries with different outcomes prove which factor matters." No two countries are identical; the design only narrows candidates.

"More cases are always better." More cases help only if they are comparable and independent. A large sample of poorly measured or duplicated cases can mislead.

"Case studies prove nothing." They are powerful for building theories, checking mechanisms and testing crucial cases, though less suitable for estimating average effects.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does the method of difference compare?

    If outcomes differ and cases are alike except for one factor, that factor is a candidate cause.
  2. 2. In a most different systems design, what do you look for?

    MDSD uses diverse cases that share an outcome to search for the common factor.
  3. 3. Why is selecting only successful cases a problem?

    Without cases of failure you cannot tell which features of the successes are actually causal.
  4. 4. What is a crucial case?

    Crucial cases give a tough test: passing is informative and failing counts strongly against the theory.
  5. 5. What is process tracing?

    Process tracing examines the steps of a causal mechanism within a case.

Summary

  • Mill's methods of agreement and difference underpin most-similar and most-different designs.
  • Good case selection varies the cause and the outcome and avoids selecting only successes.
  • Case studies, small-N comparison, QCA and large-N statistics each have strengths and limits.
  • Selection bias can hide real effects, as the simulation showed.

Next: Chapter review