What is the replication crisis in psychology?

What is the replication crisis?

The replication crisis is the observation that a large share of published findings in psychology do not hold up when other researchers try to repeat them. A study reports a statistically significant result; a second team runs the same procedure and gets nothing. Enough of these failures accumulated that the field began describing them as a crisis^1,2.

The phrase matters beyond academia. Most popular psychology claims, about habits, willpower, priming, productivity or personality, trace back to single studies. If the underlying research is less solid than it looks, the advice built on it deserves a second look.

What caused it?

Several overlapping problems are usually blamed.

Low statistical power. Many studies simply did not include enough participants to reliably detect the effects they claimed. A study with low power can produce a significant result by chance, and a later replication with modest power can miss a real effect. Maxwell and colleagues argue that some "failures to replicate" may be artefacts of under-powered single replication attempts rather than proof that the original effect was false³.

Questionable research practices. Miller and colleagues point to p-hacking, publication bias and HARKing, hypothesising after the results are known, as central contributors¹. P-hacking means running analyses multiple ways until something crosses the significance threshold. Publication bias means significant findings get published while null results sit in file drawers. HARKing means presenting an after-the-fact observation as if it had been predicted all along. The authors argue these practices violate the American Psychological Association's ethics code, not just good statistical hygiene¹.

Over-claiming. Costa's Bayesian audit of the famous "elderly priming" study, which suggested that reading words about ageing made participants walk more slowly, found the original result corresponded to only modest Bayesian evidence. Under reasonable prior beliefs, the probability that the effect was genuine stayed below 0.5⁴. In other words, the language of the claim was far stronger than the evidence behind it.

Weak theory. Muthukrishna and Henrich argue the crisis is not purely statistical. Psychology, they say, lacks the cumulative theoretical frameworks that let researchers know whether a new finding is confirmatory or surprising. Without such frameworks, empirical programmes grow from personal intuitions and culturally biased folk theories⁵.

A mismatch between words and statistics. Yarkoni argues that many verbal hypotheses in psychology are far broader than the statistical models used to test them. Researchers generalise over participants, stimuli and settings that their models never actually included, which can inflate false-positive rates⁶.

Is it really a crisis?

Not everyone agrees on the framing.

Stroebe and Strack argue that the "crisis" partly reflects an illusion of exact replication. A study repeated at a different time, with different participants, may not be testing the same underlying construct at all. In their view, meaningful replication requires checking that the theoretical variable was actually recreated, not just copying the original procedure⁷.

Maxwell and colleagues make a related point: judging an effect from a single replication attempt is statistically fragile. They recommend Bayesian methods, meta-analysis and multiple coordinated replications before concluding that a finding has failed³. Shrout and Rodgers similarly recommend that replication efforts be based on multiple studies rather than one attempt².

The practical upshot is not that all psychology is broken. It is that single studies, especially surprising ones, carry less weight than they once appeared to.

What has changed?

The crisis has accelerated open science reforms. Shrout and Rodgers recommend preregistration, full disclosure of methods and data, more sophisticated power analysis, and reporting of nonsignificant as well as significant findings².

Hendriks reviews evidence that replication failures can reduce public trust in science, but that transparent communication about open science initiatives and proactive self-correction can soften the damage. The durable fix, she argues, requires not just new tools but a scientific culture that genuinely rewards transparency and rigorous practice⁸.

How to read psychology claims now

You do not need to become a statistician to apply the lesson. A few habits help.

First, treat a single surprising study as a hint, not a fact. If a claim rests on one small experiment, especially one with a memorable story attached, wait for independent replication before changing your behaviour.

Second, look for reviews and meta-analyses. These combine many studies and are less vulnerable to the quirks of any one result^2,3.

Third, check whether the claim matches the study. A study of university students tested in a lab does not automatically generalise to your life, your workplace or your sleep schedule⁶.

Fourth, prefer sources that show their work. Preregistration, shared data and transparent reporting are signs that the authors are trying to be checked, not just to be believed².

The replication crisis is not a reason to dismiss psychology. It is a reason to read it more carefully, and to give more weight to findings that have survived independent scrutiny than to those that have not.