Does a bigger sample size make a study more trustworthy?

Why sample size matters at all

Sample size matters because it affects how precisely a study can estimate an effect and how likely it is to detect a real difference when one exists. With too few participants, random variation can swamp the signal you are looking for. The result is a study that may miss genuine effects or produce findings that do not hold up when repeated.

But the relationship between sample size and trustworthiness is not simple. A study can have thousands of participants and still be misleading if it is badly designed, measures the wrong thing or analyses the data in a way that inflates the apparent effect. The more useful question is not "how big was the sample?" but "was the sample size appropriate for the question, and was the rest of the study done well?"

What is statistical power?

Statistical power is the probability that a study will detect an effect when one truly exists. It depends on four things: the sample size, the size of the effect you are trying to detect, the significance threshold you set, and the variability in your measurements. A practical primer on sample size determination describes these as a dynamic interrelationship: change one, and the others shift¹.

A study with low power has a reduced chance of detecting a true effect. But there is a less obvious consequence. Low power also reduces the likelihood that a statistically significant result reflects a true effect. When an underpowered study does produce a significant finding, that finding is more likely to be a false positive or an overestimate².

This is why "significant" does not equal "trustworthy". A small, underpowered study can produce a significant result that is essentially noise. A larger, well-powered study is more likely to produce a result that reflects something real.

How big do samples need to be?

There is no single number that makes a sample "big enough". The required size depends on the effect you expect, the design you are using and the precision you want. A narrative review of sample size planning for studies with multiple measurement trials shows that increasing the number of measurement trials can improve reliability and reduce the sample size needed for a desired level of precision or power³. In other words, design choices can substitute for raw participant numbers.

Different approaches to sample size planning can also produce different recommendations. A review of sample size methods in nutrition research compared three approaches: a simple rule of thumb, a method based on variance explained, and a beta weights approach tied to the smallest effect of interest. The recommended sample size differed depending on which approach was used, and each carries its own assumptions and limitations⁴.

The practical takeaway is that "big enough" is a calculation, not a feeling. It depends on what you are studying and how.

The problem: many studies are underpowered

The gap between what researchers should do and what they actually do is large. A systematic review of 266 randomised controlled trials in musculoskeletal injury research estimated the median statistical power at just 42%⁵. Among trials within statistically significant meta-analyses, fewer than one in three had at least 80% power to detect the summary effect they contributed to⁵.

The same review found that only one in four trials reported a sample size calculation. The trials that did not report one tended to be smaller and to report larger effects⁵. That pattern is a warning sign: when sample size planning is not reported, the findings may be less reliable than they appear.

This is not a problem confined to one field. A widely cited review in neuroscience found that the average statistical power of studies in that discipline is very low, with consequences including overestimated effect sizes and poor reproducibility².

What else makes a study trustworthy?

Sample size is one ingredient. A study also needs to be designed so that its results can be interpreted. A review of statistical practice in sports medicine and exercise science argues for judging sample size by the precision of estimates for clinical decisions, rather than only by statistical power for significance testing⁶. That shifts the focus from "is it significant?" to "how precise is the estimate, and is the effect practically meaningful?"

Other factors matter too. Bias from poor sampling, lack of blinding, measurement error and selective reporting can undermine a study regardless of its size⁶. A large observational study can still be confounded; a large trial can still measure an outcome badly. Sample size reduces one kind of error, but it does not touch others.

How to judge a study's sample size

When you read a study, ask a few practical questions. Was the sample size justified, or does it look arbitrary? Did the authors report a power calculation or a precision target? Is the effect they expected realistic, or did they assume something larger than what was later observed? The musculoskeletal review found that 80% of trials expected larger effects than they actually observed⁵. That gap between expectation and reality is a common source of underpowered research.

Also check whether the sample matches the population the study claims to describe. A large sample drawn from a narrow or unrepresentative group can be less trustworthy than a smaller, well-chosen one. And look at the design: a randomised controlled trial with a modest but adequate sample can be more informative than a huge observational dataset with uncontrolled confounding. If you want a refresher on what makes an RCT different, see our plain-English guide to randomised controlled trials.

What to do this week

When a headline cites a study, do not stop at the sample size. Look for three things. First, did the researchers justify their sample size, and does the effect they expected look plausible? Second, is the design appropriate for the claim being made? Third, is the result reported with an estimate of precision, such as a confidence interval, rather than just a significance test?

A sensible starting point is to treat "large sample" as a green flag, not a guarantee. A big, well-designed study is more trustworthy than a small one. But a big, badly designed study can still mislead you, and a small, carefully planned study can sometimes answer a narrow question well. Trust the combination of adequate power, honest reporting and sound design, not the sample size alone.