Post hoc tests after ANOVA tell you which specific groups differ once the ANOVA shows that at least one difference exists: use Tukey when group variances are roughly equal, Games-Howell when they are not, and Bonferroni when you are testing only a few planned comparisons. The ANOVA itself only answers "is there a difference somewhere?" SPSS alone offers 18 post hoc procedures in its One-Way ANOVA dialog, and most students freeze at that menu. This guide explains the three you will need most often.
Key Takeaways
- A significant ANOVA F test shows that not all group means are equal. It does not show which groups differ. Post hoc tests answer that question.
- Tukey's HSD is the standard choice for all pairwise comparisons when group variances and sample sizes are similar.
- Games-Howell does not assume equal variances, so it is the safer choice when standard deviations or group sizes differ noticeably.
- Bonferroni divides your alpha level by the number of comparisons. It works well for a few comparisons and becomes overly strict for many.
- Running uncorrected t-tests on 4 groups (6 comparisons) raises your false-positive risk from 5% to roughly 26%.
Why do you need a post hoc test after ANOVA?
Suppose you compare three teaching methods. A significant ANOVA tells you the three average scores are not all the same. It does not tell you whether Method A beats B, B beats C, or A beats C.
The obvious fix is to run a t-test on every pair. The problem is that each test carries a 5% chance of a false positive, and those chances add up. With 3 groups you make 3 comparisons, and the chance of at least one false positive rises to about 1 − .95³ = 14%. With 4 groups you make 6 comparisons, and the chance rises to about 26%. This inflated risk is called the familywise error rate.
Post hoc tests solve the problem by adjusting the comparisons so that the overall false-positive risk across all of them stays at 5%. The three tests in this guide make that adjustment in different ways and under different assumptions.
What is the difference between Tukey, Games-Howell, and Bonferroni?
Tukey's HSD (Honestly Significant Difference)
Tukey compares every pair of means against a single critical distance, the "honestly significant difference." That critical value comes from the studentized range distribution, which is built specifically for comparing all pairs at once.
Tukey assumes the groups have roughly equal variances. It is the default recommendation for balanced designs because it holds the familywise error rate at exactly .05 while keeping good power.
Games-Howell
Games-Howell works like Tukey but drops the equal-variance assumption. Each pair gets its own standard error and Welch-adjusted degrees of freedom.
Use Games-Howell when:
- Levene's test is significant, or
- one group's standard deviation is roughly twice another's, or
- your group sizes are very unequal.
It pairs naturally with Welch's ANOVA. If you are unsure whether your variances are equal enough, see our guide on checking the homogeneity of variance assumption.
Bonferroni
Bonferroni runs ordinary t-tests and multiplies each p value by the number of comparisons. Equivalently, it divides alpha by that number (.05 / 3 = .0167 for three comparisons).
Bonferroni is simple and transparent. It becomes conservative quickly, though. With 10 groups (45 comparisons), each test must reach p < .0011, so real effects get missed. Bonferroni is at its best when you have a small number of comparisons planned in advance.
Tukey vs Games-Howell vs Bonferroni: key differences
| Feature | Tukey HSD | Games-Howell | Bonferroni |
|---|---|---|---|
| Assumes equal variances | Yes | No | Yes (pooled version) |
| Handles unequal group sizes | Moderately (Tukey-Kramer) | Well | Moderately |
| Best for | All pairwise comparisons | All pairwise comparisons, unequal variances | Few planned comparisons |
| Power with many groups | Good | Good | Low (very conservative) |
| Typical pairing | Classic one-way ANOVA | Welch's ANOVA | Planned contrasts |
A worked example: which post hoc test should I use?
A study compared exam scores of 45 psychology students across three teaching methods, with 15 students in each group:
- Lecture: M = 67.2, SD = 6.5
- Online: M = 70.5, SD = 7.1
- Flipped classroom: M = 74.7, SD = 7.4
Step 1: Run the ANOVA. The result was F(2, 42) = 4.31, p = .019, η² = .17.
- F(2, 42) means 2 degrees of freedom between groups and 42 within groups.
- p = .019 is below .05, so at least one mean differs.
- η² = .17 means teaching method explains about 17% of the variance in scores, which is a large effect.
Step 2: Check the variances. The standard deviations (6.5 to 7.4) are very similar, and the groups are equal in size. That points to Tukey.
Step 3: Read the Tukey output.
| Comparison | Mean difference | p (Tukey) | p (Bonferroni) |
|---|---|---|---|
| Flipped vs Lecture | 7.50 | .016 | .017 |
| Flipped vs Online | 4.20 | .24 | .32 |
| Online vs Lecture | 3.30 | .41 | .61 |
Only Flipped vs Lecture is significant. Its 95% CI for the difference is [1.27, 13.73], and Cohen's d = 1.07.
The Bonferroni column shows the same conclusion with slightly larger p values. That is the conservatism of Bonferroni in action.
If the flipped group's SD had been 14 instead of 7.4, the equal-variance assumption would fail. In that case you would report Welch's ANOVA followed by Games-Howell.
How do I report post hoc tests in APA 7?
Report the omnibus ANOVA first, then name the post hoc test and give the significant comparisons with effect sizes:
A one-way ANOVA showed a significant effect of teaching method on exam scores, F(2, 42) = 4.31, p = .019, η² = .17. Tukey HSD comparisons indicated that students in the flipped classroom (M = 74.7, SD = 7.4) scored significantly higher than those in the lecture condition (M = 67.2, SD = 6.5), Mdiff = 7.50, 95% CI [1.27, 13.73], p = .016, d = 1.07. No other pairwise differences were significant (ps ≥ .24).
Follow these APA 7 conventions:
- Italicise F, p, M, SD, and d.
- Drop the leading zero on p values and η², because they cannot exceed 1.
- Always state which correction you used.
Do I need to calculate this by hand?
No. Every major package runs these tests. SPSS, jamovi, and JASP offer Tukey and Games-Howell through menus. R and Stata require code or add-on commands.
The harder part is choosing the right test and writing it up correctly. StatRyx is an AI-powered statistical analysis tool that replaces manual SPSS workflows with automated, APA 7-formatted reporting. When you upload your data and run an ANOVA, StatRyx does four things for you:
- checks homogeneity of variance,
- picks Tukey or Games-Howell accordingly,
- reports effect sizes and confidence intervals, and
- drafts the APA paragraph above.
For the earlier decision about whether ANOVA is right at all, see our guide on choosing between ANOVA and Kruskal-Wallis.
Stop calculating this by hand — run it free in StatRyx → Try StatRyx
Frequently Asked Questions
Can I run post hoc tests if my ANOVA is not significant?
Generally, no. Conventional practice is to run post hoc tests only after a significant omnibus F. If you had specific comparisons planned before collecting data, report those as planned contrasts instead, typically with a Bonferroni or Holm correction.
Is Games-Howell always safer than Tukey?
Games-Howell is robust when variances are unequal. When variances really are equal, Tukey is slightly more powerful. If you are unsure, many methodologists recommend Games-Howell as the safer default, because its cost in power is small. StatRyx makes this choice automatically based on your variance check.
What is the difference between Bonferroni and Holm?
Holm is a step-down version of Bonferroni. It sorts the p values from smallest to largest and applies progressively less strict thresholds. It controls the familywise error rate just as well and is always at least as powerful, so it is usually preferable when you have more than a few comparisons.
Which post hoc test should I use for a Kruskal-Wallis test?
Tukey, Games-Howell, and Bonferroni-corrected t-tests are designed for parametric ANOVA. After a significant Kruskal-Wallis test, use Dunn's test with a Bonferroni or Holm adjustment for the pairwise comparisons.
Does SPSS report effect sizes for post hoc comparisons?
SPSS reports mean differences, standard errors, p values, and confidence intervals for post hoc comparisons. It does not report Cohen's d for each pair, so you would calculate d separately. StatRyx includes pairwise effect sizes in its APA-formatted output by default.