Skip to content
Tutorials Center
My learning

Topics

Lesson 36 of 47 ยท Nudges and Choice Architecture

Evidence, Ethics and Sludge

How big are nudge effects at scale? Learn about publication bias, the nudge unit results, ethical tests for nudging, sludge, dark patterns and boosts.

Early nudge papers reported large and exciting effects. As governments and companies tried nudges at scale, a quieter picture emerged: effects exist but are typically smaller, and they vary a lot. At the same time, critics asked whether nudging is respectful of people's autonomy and whether designers can be trusted. This lesson looks at both questions: how to read the evidence on effect sizes, and what ethical tests help to separate good choice architecture from manipulation.

Bar chart: average nudge effect 8.7 percentage points in academic papers versus 1.4 in nudge unit trials
Caption: DellaVigna and Linos (2022) found an average effect of about 8.7 percentage points in published academic nudge studies but about 1.4 points in 126 trials run by two nudge units.

What you will learn

  • How publication bias can inflate the effects of published studies.
  • What the nudge unit and meta-analysis evidence says, and why experts disagree.
  • Ethical tests for nudging, and the concepts of transparency and manipulation.
  • What sludge, dark patterns and boosts are.

Key terms

Publication bias: the tendency for statistically significant or exciting results to be published more often than null results.

Effect size: how big a difference an intervention makes, in percentage points or standard-deviation units.

Nudge unit: a government or organization team that designs and tests behavioral interventions.

Dark pattern: an interface design that tricks or pressures users into choices they would not otherwise make.

Boost: an intervention that builds the person's own skills or knowledge to decide better, instead of steering.

How large are nudge effects?

Three sets of results are worth knowing.

Academic papers versus nudge units. Stefano DellaVigna and Elizabeth Linos (2022) collected the results of 126 randomized trials, with more than 23 million people, run by two large nudge units in the United States, and compared them with the published academic literature on similar nudges. The average effect in the academic papers was about 8.7 percentage points. In the nudge unit trials it was about 1.4 percentage points. Much of the gap, they concluded, could be explained by selective publication: academic studies with large and significant effects are more likely to appear in journals, while the nudge units reported all of their trials. Different interventions and populations also contributed.

A large meta-analysis and its critique. Dominik Mertens and colleagues (2022) combined hundreds of effect sizes from 212 published studies and reported a medium average effect (a standardized effect of about 0.4). Fabian Maier and coauthors (2022) argued that after adjusting for publication bias, the evidence for an overall effect becomes ambiguous or disappears, while acknowledging that some categories, such as defaults, remained promising. Barnabas Szaszi and coauthors (2022) concluded that there is no reason to expect large and consistent effects across contexts. These authors disagree on methods, and the honest reading is that nudges work on average but modestly, and the published literature probably overstates how much.

Which nudges are strongest. Even in the bias-adjusted analyses, interventions that change the structure of the choice (defaults, reducing steps) tend to be stronger than informational messages or reminders. This is consistent with the lessons so far.

Why small effects still matter

A 1.4 percentage-point change sounds small, but it can be valuable. If a letter costs a few cents to send and raises tax payment or vaccination by 1.4 points among millions of people, the cost per outcome can be lower than for many programs. The key is to compare effects with costs, and to use trials large enough to detect small effects. The second TryIt shows how many people are needed: to detect a 1.4-point change from a 10 percent baseline with 80 percent power, you need roughly 7,600 people per arm.

A publication bias simulation

Imagine the true effect of a nudge is 1 percentage point, and many teams run studies of different sizes. Smaller studies have noisier estimates. If only results with a significance test of at least 1.96 are published, the published average is far above the truth, and the smallest studies are the most inflated. The first TryIt demonstrates this with a simulation.

Ethics: where are the limits?

Nudging has attracted both enthusiasm and criticism. Some arguments and tests follow.

The case for. Choice architecture is unavoidable, so designers might as well design thoughtfully. A nudge preserves freedom of choice and can be cheap and popular. Surveys in many countries find that most people approve of transparent, health- or welfare-improving nudges, with defaults often supported when they match the person's interests.

The case for caution.

  • Whose goals? A nudge toward saving assumes saving is in the person's interest. If the person has expensive debt or a more urgent need, it may not be.
  • Manipulation. Pelle Guldborg Hansen and others have discussed nudges that work by bypassing a person's reasoning. Is a default that works because people do not notice it respectful?
  • Heterogeneity. The same nudge helps some people and harms others.
  • Slippery slope and trust. If people think nudges are covert, trust in institutions may fall.

Practical tests many practitioners use:

  1. Transparency (publicity) test: Would the designer be comfortable disclosing the nudge and its purpose? If disclosure would defeat it, be suspicious.
  2. Opt-out test: Is it easy and cheap to choose differently?
  3. Interest test: Does it serve the chooser's own goals as they themselves would judge them, and is there evidence it does so for most?
  4. Fairness test: Does it burden some groups more than others?

Sludge, dark patterns and boosts

Richard Thaler (2018) coined the contrast nudge versus sludge. Sludge is friction that stops people doing what would benefit them: complicated forms, long phone queues to cancel a subscription, hidden fees, unnecessary documentation. Cass Sunstein (2020) argued that governments should audit their own forms and procedures and remove sludge, which can be more effective than adding nudges. Sludge can be accidental (poor design) or deliberate (making cancellation hard).

Dark patterns are the commercial counterpart: pre-ticked boxes for paid extras, countdown timers that create false urgency, cookie banners that make rejecting tracking much harder than accepting it. A study of shopping websites by Arunesh Mathur and colleagues (2019) found many sites using such tricks. The behavioral tools are the same as nudges; the difference is that they serve the designer at the user's expense. Several jurisdictions now regulate some of them.

Boosts (Ralph Hertwig and Till Grรผne-Yanoff, 2017) are a different approach: instead of steering behavior, they build competence, for example teaching people simple rules of thumb for understanding risk, or training in how to read a food label. They respect autonomy by design and may have lasting effects, but they need effort from the learner and may reach fewer people. Many policies combine nudges and boosts.

Common misconceptions

"Nudges don't work because effects shrank." The evidence suggests smaller but real effects for some designs, especially structural ones.

"If it is cheap and works, it is ethical." Cost and effectiveness do not settle whether a design respects the person's own goals and is transparent.

"Publication bias means the studies are fraud." Publication bias comes from selective reporting, not necessarily dishonesty, and it can be reduced by preregistration and by publishing null results.

Try it

First, simulate publication bias. Many studies estimate an effect of true size 1 percentage point with different standard errors; only those that are statistically significant (estimate divided by standard error above 1.96) are "published". The code compares the true effect, the average over all studies and the average of the published ones. The numbers come from a simulation with assumed parameters.

Python Publication bias inflates published effects

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Second, a sample size calculator. It uses the normal approximation for comparing two proportions and shows how quickly the required sample grows as the effect you want to detect shrinks.

Python How many people do you need per arm?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Third, an ethics checklist. Answer five questions about a nudge you are considering. This is a discussion aid, not a legal or moral verdict.

A nudge ethics checklist (discussion aid)

Check your understanding

Quick check ยท 6 questions

of 6 answered You scored / 6 ยท perfect!

  1. 1. What did DellaVigna and Linos (2022) find about average effects?

    About 1.4 points in nudge unit trials versus about 8.7 in published academic papers, with selective publication a main explanation.
  2. 2. What is publication bias?

    Selective publication makes published effects larger on average than the true effect.
  3. 3. Which is a practical ethical test for a nudge?

    A transparency (publicity) test asks whether the nudge could survive being explained.
  4. 4. What is sludge?

    Examples include long forms, hidden cancellation options and unnecessary paperwork.
  5. 5. How do boosts differ from nudges?

    Boosts aim at competence and autonomy rather than steering.
  6. 6. Why might a 1.4 percentage-point effect still be worthwhile?

    Cost-effectiveness depends on the price per person reached and the value of the outcome.

Summary

  • Effects at scale are smaller than in early published studies; publication bias explains much of the gap.
  • Structural nudges such as defaults and reduced steps tend to be stronger than informational ones, but experts disagree on the overall size.
  • Useful ethical tests: transparency, easy opt-out, serving the chooser's interests and fairness.
  • Sludge and dark patterns use friction and pressure against users; boosts build skills instead of steering.

Next: Chapter 7 review