To analyze pre-test and post-test data from a single group, use a paired-samples t-test (or its non-parametric cousin, the Wilcoxon signed-rank test) to see whether scores changed significantly from before to after; if you have two or more groups receiving different conditions, use a mixed ANOVA or an ANCOVA instead. The mistake most graduate students make is running an independent-samples t-test on paired data — which ignores that the same people were measured twice and throws away statistical power.
Key Takeaways
- One group measured twice (before and after an intervention) calls for a paired-samples t-test when differences are roughly normal, or a Wilcoxon signed-rank test when they are not.
- Two or more groups (e.g., treatment vs. control, each measured pre and post) calls for a mixed ANOVA or an ANCOVA using the pre-test score as a covariate.
- Always analyze the difference scores (post minus pre), not the two columns independently — the pairing is the whole point of the design.
- Report an effect size alongside p: Cohen's d for t-tests, η² or partial η² for ANOVA.
- Choosing the wrong test is the single most common error in pre-post designs; StatRyx picks the correct test automatically based on your data structure.
Which test should I use for pre-test and post-test data?
The right test depends entirely on how many groups you have and whether your difference scores are normally distributed.
- One group, measured twice: Paired-samples t-test (normal differences) or Wilcoxon signed-rank test (non-normal).
- Two independent groups, each measured twice: Mixed ANOVA (within-subjects factor = time, between-subjects factor = group), or ANCOVA controlling for the pre-test.
- Three or more time points: Repeated-measures ANOVA.
The paired t-test works because it collapses your two columns into one column of difference scores, then tests whether that single column's mean differs from zero. That is why it is more powerful than an independent-samples test on the same data — it strips out the stable between-person differences that otherwise inflate your error term.
When should I use ANCOVA instead of a mixed ANOVA?
Use ANCOVA when you have a control and a treatment group and you want to compare their post-test scores while statistically adjusting for differences in their starting (pre-test) scores. ANCOVA treats the pre-test as a covariate rather than as a repeated measurement, which usually gives you more statistical power than a mixed ANOVA when groups were randomly assigned.
A practical rule: if your groups were randomized and you mainly care about the end state, ANCOVA is often the stronger, more publishable choice. If you care about the shape of change over time (did the treatment group improve more steeply?), the time × group interaction from a mixed ANOVA answers that directly.
How do I check the assumptions first?
Before trusting any p-value, verify these:
- Normality of the difference scores (not the raw scores). Use a Shapiro-Wilk test and a histogram of post-minus-pre differences. If p > .05 on Shapiro-Wilk, you can proceed with the paired t-test.
- No extreme outliers in the difference scores — a single data-entry error (e.g., a 75 typed as 750) can flip your result.
- Interval/ratio data for parametric tests. If your outcome is Likert-scale or clearly skewed, use the Wilcoxon signed-rank test.
If the normality assumption fails, don't panic — the Wilcoxon signed-rank test answers the same question (did scores change?) without requiring normal differences. If you're unsure which applies to your data, our guide on choosing between parametric and non-parametric tests walks through the decision.
Worked example: a paired-samples t-test, step by step
Suppose you run an 8-week mindfulness program and measure anxiety (on a 0–50 scale) for 30 participants before and after.
Step 1 — Compute difference scores. For each person, subtract the pre-test from the post-test. Say the mean difference is −6.4 points (anxiety dropped), with a standard deviation of differences of 9.1.
Step 2 — Check normality. Shapiro-Wilk on the 30 difference scores returns W = .96, p = .31 — normal enough, so the paired t-test is appropriate.
Step 3 — Run the test. The paired-samples t-test yields:
t(29) = −3.85, p < .001, d = 0.70, 95% CI [−9.8, −3.0]
Step 4 — Interpret each number:
- t(29) = −3.85 → the test statistic with 29 degrees of freedom (n − 1 = 30 − 1). The negative sign simply reflects that post-test scores were lower.
- p < .001 → the probability of seeing a change this large if the true change were zero is less than 0.1% — statistically significant.
- d = 0.70 → Cohen's d, a medium-to-large effect. Anxiety dropped by about 0.7 standard deviations.
- 95% CI [−9.8, −3.0] → we're 95% confident the true average drop is between 3.0 and 9.8 points. Because the interval excludes zero, the effect is significant.
Step 5 — Report it (APA 7):
A paired-samples t-test indicated that anxiety scores were significantly lower after the mindfulness program (M = 18.2, SD = 7.9) than before it (M = 24.6, SD = 8.3), t(29) = −3.85, p < .001, d = 0.70, 95% CI [−9.8, −3.0].
Running this by hand means computing difference scores, their standard deviation, the t-statistic, degrees of freedom, and the effect size — then formatting it all. StatRyx does every step above and hands you that APA sentence ready to paste into your thesis.
Pre-post test comparison: which analysis fits your design?
| Your design | Right test | What it answers | Effect size |
|---|---|---|---|
| 1 group, pre & post, normal differences | Paired-samples t-test | Did scores change? | Cohen's d |
| 1 group, pre & post, skewed/ordinal | Wilcoxon signed-rank test | Did scores change? | r (rank-biserial) |
| 2 groups, pre & post | Mixed ANOVA | Did groups change differently over time? | partial η² |
| 2 groups, randomized, outcome focus | ANCOVA (pre as covariate) | Do post scores differ, adjusting for baseline? | partial η² |
| 3+ time points | Repeated-measures ANOVA | Did scores change across time points? | partial η² |
What's the biggest mistake in pre-post analysis?
The biggest mistake is comparing pre-test and post-test as if they were two unrelated groups using an independent-samples t-test. Because the same participants appear in both columns, the measurements are correlated — ignoring that correlation discards the main advantage of a repeated-measures design and can hide a real effect. A 2019 review of psychology methods noted that mis-specified pre-post analyses remain one of the most frequent statistical errors flagged in peer review.
The second most common error is reporting only the p-value with no effect size. APA 7 expects an effect size and, ideally, a confidence interval — a p < .05 tells you the change is unlikely to be chance, but Cohen's d tells you whether it actually matters.
Stop calculating this by hand — run it free in StatRyx → [Try