Equivalence testing (TOST) is a statistical method that lets you conclude two groups are practically the same by testing whether their difference falls within a range you decide in advance is too small to matter. A regular t-test can only fail to find a difference — it can never confirm that two things are equivalent. If you've ever written "there was no significant difference" and wanted to claim your groups were actually similar, that sentence is statistically wrong, and TOST is the fix.
Key Takeaways
- A non-significant t-test does not prove equivalence — absence of evidence is not evidence of absence. You need equivalence testing to positively support "these are the same."
- TOST stands for "Two One-Sided Tests" — you run two tests to check that the difference is both not too high and not too low to matter.
- You must set an equivalence bound (SESOI) before analysis — the smallest effect size that would still be meaningful in your field, decided in advance.
- If both one-sided tests are significant (both p < .05), you conclude equivalence — the true difference is confidently smaller than what you'd care about.
- TOST is used in drug bioequivalence, replication studies, and psychology whenever "no meaningful difference" is the actual research question.
Why can't a normal t-test prove two groups are the same?
A standard t-test asks one question: "Is there a difference?" When p > .05, all you can honestly say is that you failed to find one — not that no difference exists. This is a classic logical trap: a small sample with high variability will always return a non-significant result, even when the groups genuinely differ.
Imagine testing whether a cheaper generic teaching method works "the same" as an expensive one. You run a t-test, get p = .34, and conclude they're equivalent. But that p = .34 might just mean your study was underpowered. You never actually tested for sameness — you tested for difference and came up empty. That's the gap equivalence testing was invented to close.
When should I use equivalence testing (TOST)?
Use equivalence testing whenever your research question is genuinely "are these two things close enough to treat as the same?" Common situations include:
- Bioequivalence in pharmacology — regulators like the FDA require generic drugs to fall within roughly 80–125% of the brand-name drug's absorption. This is TOST in action, and it's the field where the method became standard.
- Replication studies — showing a new sample reproduces (is equivalent to) a published effect.
- Manufacturing and measurement — confirming a new sensor gives the same readings as the trusted one.
- Psychology and social science — arguing that two conditions produced practically identical outcomes.
If your goal is to support the null (no meaningful difference), you need TOST. If you want to see whether the same data show any difference, see our guide on the independent-samples t-test instead.
What does TOST actually mean?
TOST stands for Two One-Sided Tests. Instead of one two-tailed test, you run two separate one-sided tests against your equivalence bounds:
- Test 1: Is the difference significantly greater than the lower bound (−Δ)?
- Test 2: Is the difference significantly less than the upper bound (+Δ)?
If both tests are statistically significant (both p < .05), you can reject the hypothesis that the difference is large enough to matter — in either direction. That leaves you with the conclusion that the true difference sits inside your equivalence zone. Practically the same.
What is a SESOI, and how do I set the bounds?
The single most important step in TOST is defining your equivalence bounds, also called the SESOI — the Smallest Effect Size Of Interest. This is the point where a difference becomes small enough that you'd genuinely shrug and call the groups equivalent.
You set this before running the analysis, based on theory, prior research, or practical judgment — never by peeking at your data. A SESOI of, say, Cohen's d = 0.3 says: "Any difference smaller than a small-to-medium effect isn't worth caring about here." The bounds are then ±0.3. Choosing the SESOI is a judgment call, and being transparent about it is what makes an equivalence test credible.
A worked TOST example with real numbers
Suppose you're comparing two study techniques on an exam. You run a study with 60 students (30 per group) and get:
- Group A mean = 74.2
- Group B mean = 75.6
- Mean difference = −1.4 points
- Standard error of the difference = 1.9
Before collecting data, you decided that anything smaller than ±5 points on this exam is educationally trivial. So your equivalence bounds are −5 and +5.
Now run the two one-sided tests (df = 58):
Test 1 (lower bound):
t = (−1.4 − (−5)) / 1.9 = 3.6 / 1.9 = 1.89, p = .032 (one-sided)
Test 2 (upper bound):
t = (−1.4 − 5) / 1.9 = −6.4 / 1.9 = −3.37, p = .001 (one-sided)
Both one-sided tests are significant (p = .032 and p = .001, both < .05). You reject both "difference too large" hypotheses and conclude the techniques are statistically equivalent within ±5 points.
Notice what a regular t-test would have said here: t(58) = −0.74, p = .46 — non-significant, but that alone proves nothing. TOST turns "we found nothing" into a positive, defensible claim.
How do I report a TOST result in APA 7?
Report the equivalence bounds, the more decisive of the two one-sided tests (the larger p-value, since both must clear .05), and ideally the 90% confidence interval of the difference. APA 7 example:
"The two conditions were statistically equivalent within a bound of ±5 points. The more stringent one-sided test was significant, t(58) = 1.89, p = .032. The 90% CI for the mean difference [−4.58, 1.78] fell entirely within the equivalence bounds."
The 90% CI trick is the elegant shortcut: if the 90% confidence interval of your difference lies entirely inside the equivalence bounds, the TOST is significant at .05. That single interval tells the whole story at a glance.
Regular test vs equivalence test: key differences
| Feature | Standard t-test | Equivalence test (TOST) |
|---|---|---|
| Research question | Is there a difference? | Is the difference too small to matter? |
| Null hypothesis | Difference = 0 | Difference is ≥ the bound (in either direction) |
| What p < .05 means | A difference exists | The groups are equivalent |
| Can prove "same"? | No | Yes |
| Requires bounds/SESOI? | No | Yes — set in advance |
| Confidence interval used | 95% two-sided | 90% for the .05 TOST |
How StatRyx runs TOST for you
Setting the correct bounds, running two one-sided tests, and pulling the 90% confidence interval by hand is exactly where beginners slip — using the wrong CI (95% instead of 90%) or forgetting that both tests must be significant. StatRyx handles the mechanics: you enter your groups, set your SESOI in raw units or Cohen's d, and it runs both one-sided tests, checks the 90% CI, and returns a plain-language verdict plus an APA 7-formatted write-up ready to paste into your thesis. Because StatRyx is AI-powered, it also flags when equivalence testing — not a plain t-test — is the analysis your question act