P-values appear throughout medical research, but they are also one of the most misunderstood statistical outputs. This article explains what a p-value actually is, how it is misused, and how to interpret it in clinical studies. You will learn why statistical significance is not the same as clinical relevance, how confidence intervals and effect sizes help, and how to avoid p-hacking and other reporting traps.
What Is a P-Value?
A p-value is the probability of observing data as extreme or more extreme than the results you obtained, assuming that the null hypothesis is true. The null hypothesis usually states there is no real difference or no true effect.
Think of it as a measure of surprise: If the null hypothesis were true, how often would random variation produce results like the ones you saw?
- A p-value is a conditional probability, not a simple yes-or-no test.
- It depends on the study design, sample size, and analysis plan.
- It does not tell you the probability that the null hypothesis is true.
- It does not tell you the probability that the treatment worked.
For example, if a trial reports p = 0.03, this means that if the treatment truly had no effect, a result this extreme would occur in about 3% of similar repeated experiments due to chance alone.
A p-value tells you how strange the data are under one assumption. It does not tell you how likely your research hypothesis is to be true.
Why the 0.05 Threshold Is Not a Magic Line
Many medical researchers treat p < 0.05 as “significant” and p ≥ 0.05 as “non-significant.” This dichotomous mindset causes serious problems in evidence interpretation.
The 0.05 threshold is a convention, not a law of nature. A result of p = 0.049 is not fundamentally more credible than p = 0.052. Small changes in the analysis can move a p-value across the threshold without changing the underlying clinical meaning.
| P-value range | Common labeling | More balanced interpretation |
|---|---|---|
| p ≥ 0.05 | Not significant | No strong evidence of an effect; the result is compatible with chance variation. |
| p < 0.05 | Significant | There is evidence against the null hypothesis, but magnitude and precision matter. |
| p < 0.001 | Highly significant | A strong statistical signal, but still not a guarantee of clinical benefit. |
| p close to 0.05 | Borderline | Use caution; confidence intervals and replication are essential. |
- Choose the alpha level before the analysis, ideally when the study protocol is written.
- Treat p-values as continuous measures of evidence rather than binary cutoffs.
- Always ask: could a different endpoint or a different subgroup change the p-value?
Common Misuses of P-Values in Medical Research
Misuse of p-values is common enough that major medical journals now encourage authors to provide effect sizes and confidence intervals alongside p-values. Knowing the patterns of misuse helps you read research with a critical eye.
- P-hacking: Trying several analyses and only reporting the one with p < 0.05.
- Subgroup fishing: Finding a “significant” effect in a subgroup after the main result failed.
- Confusing statistical significance with clinical importance: A tiny difference can be statistically significant if the sample is large enough.
- Ignoring multiple testing: Running many comparisons inflates the chance of false positives.
- Equating “not significant” with “no effect”: A non-significant result may simply mean the study lacked power.
Imagine a hypothetical trial for a blood pressure medication. The study reports p = 0.04 with a reduction of 2 mm Hg. That result is statistically significant, but for someone with severe hypertension, a 2 mm Hg drop is likely not a clinically meaningful improvement. The p-value alone would mislead you.
Statistical significance does not prove clinical importance. A small p-value can be caused by a trivial effect in a large study.
How to Interpret P-Values Correctly
To interpret p-values fairly, you need to look at the whole picture, not just the number after the letter p.
- Check whether the study was randomized and properly blinded.
- Look at the confidence intervals to see the range of plausible effect sizes.
- Look at the absolute risk reduction or the number needed to treat, not only relative changes.
- Ask whether the primary endpoint was defined before the data were collected.
- Consider whether the sample size was large enough to detect a meaningful effect.
For example, a study with p = 0.08 and a very wide confidence interval does not prove that a treatment is ineffective. It may simply be underpowered. Conversely, a study with p = 0.01 and a narrow confidence interval around a tiny effect can be statistically convincing but clinically irrelevant.
Always report exact p-values where possible. Instead of writing “p < 0.05,” write “p = 0.023” when you have the exact value. If the value is extremely small, use “p < 0.001” rather than “p = 0.000.”
P-Values, Confidence Intervals, and Sample Size
A confidence interval gives a range of values that are likely to include the true effect size. This is far more informative than a p-value because it shows both the direction and the precision of the effect.
Consider a difference in symptom improvement between two groups. A 95% confidence interval from 0.1 to 0.9 indicates a positive effect, but the interval may be too wide to support a precise clinical recommendation. A narrower interval from 0.4 to 0.6 gives more confidence in the exact magnitude.
- Large samples can make even trivial differences statistically significant.
- Small samples can produce non-significant results even when a real effect exists.
- Confidence intervals help you distinguish between an imprecise estimate and a precisely estimated null effect.
- Pre-specified sample size calculations are essential before launching a study.
If the 95% confidence interval includes the null value, this is consistent with the p-value being above 0.05. If the interval is entirely on one side of the null, the result is often statistically significant. But the interval still does not tell you whether the effect matters in real clinical practice.
Practical Tips for Medical Students and Trainees
When you read journal articles, do not skip the methods section. The most important question in evidence-based medicine is not “Is this p-value small?” but “Could this study be biased?”
- Always ask: What is the null hypothesis? What would the world look like if it were true?
- Review the study protocol or ClinicalTrials.gov entry to see if the analysis matches the pre-planned design.
- Use the phrase “statistically significant” carefully; save “clinically meaningful” for effects that change patient management.
- Do not rely on p-values alone when deciding whether a treatment works.
- Practice interpreting forest plots, confidence intervals, and number needed to treat.
As a student, you do not need to become a statistician to read medical research. But you do need to understand the basic logic of probability and the common ways authors manipulate or misinterpret it.
Alternatives and Supplements to P-Values
Modern medical research is moving toward a richer set of statistical tools. P-values are not useless, but they work best when supported by other measures.
- Effect sizes such as Cohen’s d, risk differences, and number needed to treat.
- Confidence intervals for the main outcome measures.
- Bayesian posterior probabilities that directly describe the probability of a clinically relevant effect.
- Pre-registration and open data to reduce p-hacking.
- Sensitivity analyses to show how robust the findings are to different assumptions.
You should not abandon p-values completely. Instead, make them part of a broader evidence story. A great scientific report explains the design, the measurement error, the effect magnitude, the uncertainty, and the possible biological mechanisms.
Conclusion
The p-value is a useful statistical tool, but it is only a clue. It does not stand alone as proof of a treatment’s value, and it cannot tell you what a patient should choose. In medical research, the most reliable conclusions come from pre-specified hypotheses, appropriate sample sizes, clinically meaningful endpoints, and results that are consistent across multiple studies. Read p-values with caution, interpret them alongside confidence intervals, and always keep the patient at the center of the question.
Frequently Asked Questions
Is p < 0.05 enough to say a result is true?
No. A p-value below 0.05 only indicates that the observed data are unlikely under the null hypothesis. It does not prove that the alternative hypothesis is true. The result could be a false positive due to chance, bias, or multiple testing.
What does a p-value of 0.01 mean?
A p-value of 0.01 means that, if the null hypothesis were true, you would expect to see a result this extreme in about 1% of repeated studies by random variation alone. It is stronger evidence against the null hypothesis than a p-value of 0.05, but it still does not measure the size or importance of the effect.
What is the difference between statistical significance and clinical significance?
Statistical significance refers to whether the observed difference is likely due to chance. Clinical significance refers to whether the difference is large enough to matter to
