Skip to content
Tutorials Center
My learning

Topics

Lesson 4 of 46 ยท Getting Started

Replication, Ethics and Reading Research

Understand why some famous findings failed to replicate, how open science responds, and what ethics rules protect participants in social psychology studies.

In 2011 a respected Dutch social psychologist, Diederik Stapel, was found to have fabricated data in dozens of papers. In the same year, a paper in a top journal reported evidence for precognition, which most scientists doubted. Together these events made the field ask hard questions: How many published results are reliable? What should researchers do to fix the system?

This lesson explains what happened, what changed, and why research ethics matter just as much. Both topics give you tools to read any psychology headline with a clear head.

What you will learn

  • What a replication is and why it matters.
  • The main causes of unreliable findings.
  • How open science tries to fix them.
  • The ethical principles that protect participants.
  • How to read a research claim critically.

Key terms

Replication is repeating a study, ideally with new participants, to see if the result appears again. p-hacking means trying many analyses until one gives p below 0.05. Preregistration is publicly recording hypotheses and analysis plans before collecting data. Informed consent means participants agree after learning what to expect. Debriefing is explaining the real purpose of a study after it ends.

A crisis of confidence

In 2015 the Open Science Collaboration tried to replicate 100 published psychology studies. About 97 percent of the original studies had reported significant results; in the replications, roughly a third to 40 percent did, and the effects were on average about half as large. Replication rates were lower for social psychology studies than for cognitive psychology studies, though the exact percentages depend on how success is defined. This does not mean everything is false. It means that many individual published results are less certain and less large than first reported.

A chain of four steps showing how a finding earns trust: first study, replication, combined analysis and ethical protection of participants.
A single study is a clue; repeated, pooled and ethical studies build trust.

Famous cases

  • Social priming. John Bargh's 1996 study suggested that reading words related to old age made people walk more slowly. A 2012 attempt by Doyen and colleagues, using automatic timing, did not find the effect unless experimenters expected it.
  • Ego depletion. The idea that self-control is a limited resource that gets "used up" was supported by many studies, but a large multi-lab replication in 2016 (Hagger and colleagues) found an effect close to zero.
  • Power posing. Carney, Cuddy and Yap (2010) reported that standing in an expansive pose changes hormones and risk-taking. Larger replications did not support the hormonal and behavioral effects, and a lead author later said she no longer believed in them. Some evidence remains that posing may affect how powerful people feel, which is a smaller claim.
  • Facial feedback. The idea that holding a pen in your teeth (forcing a smile) makes cartoons funnier (Strack, Martin and Stepper, 1988) was not reproduced in a 17-lab project (Wagenmakers and colleagues, 2016), though later work suggests a small effect may exist. This is an example of an open debate.

Why do findings fail?

  1. Small samples. With few participants, estimates are noisy. The significant ones are often exaggerated (the "winner's curse").
  2. Flexible analysis. Choosing outcomes, dropping participants or adding data until p dips below 0.05 inflates false positives.
  3. Publication bias. Journals prefer positive, surprising results, so failures stay in the file drawer.
  4. Fraud. Rare, but damaging, as in the Stapel case.
  5. Context. Some effects depend on culture or conditions; failure to replicate may reflect hidden moderators.

The open science response

Researchers now preregister plans, share data and materials, run registered reports (journals accept the study before the results are known), and organize many-lab projects, such as the Many Labs series. Larger samples and effect-size reporting are common. The lesson for you as a reader: trust findings that have been replicated and that have clear effect sizes; be cautious with one small, surprising study.

Research ethics

Ethics rules grew from serious abuse in medical research and some dramatic psychology studies. Key documents include the Nuremberg Code (1947), the Declaration of Helsinki (1964) and the Belmont Report (1979), which names three principles: respect for persons, beneficence (do good, avoid harm) and justice (fair distribution of risks and benefits).

In practice, universities require an ethics committee (IRB in the US) to review each study. Researchers must:

  • Obtain informed consent (with extra safeguards for children).
  • Tell participants they may withdraw at any time without penalty.
  • Keep data confidential or anonymous.
  • Minimize risk and discomfort, and weigh risks against benefits.
  • Use deception only when necessary and no alternative exists, and then debrief fully.

Stanley Milgram's obedience studies and Philip Zimbardo's prison simulation, both in Chapter 5, are often cited as cases that prompted tighter rules because of the stress participants felt.

Reading a research claim: a checklist

Question Why it matters
How many participants? Small samples are noisy
Was it an experiment? Only experiments support cause
How big is the effect? Significance is not importance
Has it been replicated? Repetition builds trust
Who funded it, who benefits? Conflicts of interest
Is the headline bigger than the data? News often overstates

Common misconceptions

"Failed replication means the original was a fraud." Usually not. Chance, small samples and flexible methods explain most failures.

"Psychology is not a real science." The field's willingness to test and fix itself is a sign of scientific health.

"Deception is always unethical." It is allowed only under tight conditions, with review and debriefing.

Try it

First, see why small studies produce exaggerated effects. Second, review a fictional study design with an ethics checklist. Both are demonstrations.

Python Simulating small studies and the winner's curse

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

An ethics checklist for a fictional study

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is a replication?

    Replication tests whether a finding is reliable.
  2. 2. What is p-hacking?

    Flexible analysis inflates false positives.
  3. 3. Which is a goal of preregistration?

    Preregistration reduces flexible analysis.
  4. 4. What does debriefing involve?

    Debriefing restores trust and clarifies what happened.
  5. 5. The Belmont Report names which principle?

    Respect for persons, beneficence and justice.

Summary

  • Large replication projects found that many published effects were smaller or absent.
  • Causes include small samples, flexible analysis, publication bias and sometimes fraud.
  • Open science (preregistration, sharing, many-lab projects) improves reliability.
  • Ethics rules require consent, minimal harm, withdrawal rights, confidentiality and debriefing.
  • Read claims critically: sample size, design, effect size and replication.

Next: Chapter 1 Review