At 12 weeks, does mean mobility-score improvement differ between home rehabilitation and usual physiotherapy?
Statistical foundations · plain-language guide
A p-value is a suspicion meter, not a truth meter.
Begin by assuming the null model. Then ask how surprising data at least this extreme would be if that model were right.
Keep this safeguard in view: a p-value is not the probability that the null hypothesis is true, and it does not tell you whether an effect is important.
One standalone worked example · explicitly fictional
Start with a real study question and two clear hypotheses.
The fictional study
A fictional trial includes 120 adults aged 65 or older who are recovering from hip fracture. Sixty are randomly allocated to a structured home rehabilitation programme and 60 to usual outpatient physiotherapy. The outcome is change in a 0 to 100 mobility score at 12 weeks, where higher scores indicate greater improvement. The planned analysis is a two-sided Welch independent-samples t-test.
Adults aged 65 or older recovering from hip fracture in this fictional trial.
Structured home rehabilitation compared with usual outpatient physiotherapy.
Change in mobility score, analysed with a planned two-sided Welch independent-samples t-test.
The population mean change is the same in both groups, so the mean difference is 0.
The population mean change differs between the groups, so the mean difference is not 0.
H₀ usually begins with no association or no difference because that gives the test a clear reference model. In an equivalence study, H₀ instead places the effect outside the pre-specified equivalence range. H₁ is the substantive competing claim that matches the study question. They are the cornerstone of the workflow because the test asks whether the observed data would be unusual if H₀ were true. The test does not prove either hypothesis.
The suspicion-meter intuition
Assume H₀, then ask how surprising the data are.
Temporarily assume the null model is correct. Measure how far the observed result lies from it, allowing for uncertainty. Then ask whether a result at least this extreme would be unusual under H₀ and the analysis assumptions. A smaller p-value raises more suspicion about that null model. It does not reveal which explanation is true.
In beginner language, “nothing meaningful is going on” is only shorthand. The real H₀ must state a precise null value, such as a mean difference of 0. Need help with magnitude and uncertainty? Open Estimates & confidence intervals.
The careful definition
Assuming H₀ and the statistical model are correct, the p-value is the probability of obtaining the observed test statistic, or one at least as extreme in the direction or directions specified by the test. The calculation depends on the test, sample size and assumptions.
Now look at the fictional study result
Estimate first. Confidence interval second. P-value third.
- 1Observed pattern
Mean improvement was 8.4 points with home rehabilitation and 5.1 points with usual physiotherapy.
- 2Estimate
The observed mean difference was 3.3 points in favour of home rehabilitation.
- 3Uncertainty
The 95% confidence interval was 0.7 to 5.9 points.
- 4P-value
p = 0.014 for the planned two-sided Welch test.
What p = 0.014 tells us
If H₀ were correct, and the independence and Welch-test assumptions held, a test statistic at least as extreme as the one observed would occur about 1.4% of the time. The data therefore raise suspicion about this null model.
It does not mean H₀ has a 1.4% chance of being true. It also does not tell us whether a 3.3-point improvement matters to patients. Judge importance from the estimate, confidence interval and a clinically justified benchmark, not from the p-value.
Keep the job of the p-value narrow
The same interpretation rule travels across methods.
No null test may be needed
If the question asks how common or how large something is, report the estimate, denominator and interval. Do not invent a p-value.
Keep size separate from compatibility
Differences use 0 as the usual null value; ratios use 1. The estimate and interval show magnitude and uncertainty. The p-value addresses the stated null model.
The model is part of the claim
A coefficient p-value is conditional on the outcome model, covariates, functional forms, sample and assumptions. It cannot choose confounders for you.
The meter is sensitive to information
Large samples can make tiny effects look unusual. Small samples can leave important effects uncertain.
Four common mistakes
What the p-value cannot tell you.
Not the probability H₀ is true
The calculation assumes H₀. It does not estimate the probability of H₀ or H₁.
Not effect size or importance
Use the estimate, units, confidence interval and clinical context to judge magnitude.
Not proof of cause
Causal conclusions depend on design, time order, bias, confounding and assumptions.
Not a magic line at 0.05
p = 0.049 and p = 0.051 provide almost the same compatibility information.
Use it well
Lead with the result, not a threshold verdict.
A clear results sentence
“At 12 weeks, mean mobility-score improvement was 8.4 points in the home rehabilitation group and 5.1 points with usual physiotherapy. The mean difference was 3.3 points (95% CI 0.7 to 5.9; two-sided Welch t-test p = 0.014). The estimate favoured home rehabilitation, but its clinical importance should be judged against a pre-specified meaningful difference.”
Self-check · A student writes “p = 0.014, so there is a 98.6% chance H₁ is true.” What is wrong?
Answer: the p-value is calculated assuming H₀ and a model. It does not assign probabilities to H₀ or H₁. Report the estimate and confidence interval first, then describe the p-value as a limited measure of compatibility with H₀ under the test assumptions.
Go deeper · authoritative medical-statistics resources
Optional direct sources for principles, misconceptions, clinical use and the mathematics behind a worked test.
ASA statement on p-values
The six core principles on interpretation, evidence and transparent reporting.
Open the ASA statementStatistical tests, P values and confidence intervals
Greenland and colleagues explain 25 common misinterpretations and how to correct them.
Read the articleStatistics at Square One: differences between means
A clinical introduction to sampling error, confidence intervals, Type I error and test interpretation.
Open the BMJ chapterDifference between two proportions
A direct worked definition of the standard error, confidence interval and null test for two proportions.
Open the NIST derivation