A Type I error is a false positive — you conclude there's an effect when there isn't one — and a Type II error is a false negative — you miss a real effect that's actually there. If you've ever stared at a p-value wondering whether you can trust it, or worried your "non-significant" result just means your study was too small, you're wrestling with exactly these two errors.
Key Takeaways
- A Type I error (false positive) means rejecting a true null hypothesis — claiming an effect exists when it doesn't. Its probability equals your significance level, α (usually .05).
- A Type II error (false negative) means failing to reject a false null hypothesis — missing a real effect. Its probability is called β, and statistical power equals 1 − β.
- The two errors trade off: lowering your α (being stricter about false positives) raises your risk of a false negative, and vice versa.
- Small samples cause most Type II errors. If a study is underpowered, a real effect can easily hide in the noise and produce a non-significant result.
- Report the direction of your conclusion honestly. A non-significant result is not proof of "no effect" — it may just reflect low power.
What is a Type I error in plain language?
A Type I error happens when you decide something is going on, but nothing actually is. Imagine a smoke alarm that shrieks when you're just making toast — that's a false positive. In research terms, you reject the null hypothesis (the "nothing is happening" assumption) when the null was actually true.
The probability of a Type I error is set by you, in advance, as your significance level (α). When you use p < .05, you're accepting a 5% chance of a false positive on any given test. That's a deliberate risk: roughly 1 in 20 times, you'll flag an effect that's really just random noise.
What is a Type II error in plain language?
A Type II error is the opposite mistake: a real effect exists, but your study fails to detect it. Picture a smoke alarm with a dead battery — the house is genuinely on fire, but the alarm stays silent. That's a false negative. In statistics, you fail to reject the null hypothesis even though the null is false.
The probability of a Type II error is called β. Unlike α, you don't set β directly — it depends on your sample size, the true size of the effect, and how much variability is in your data. The flip side of β is statistical power (1 − β): the probability your study will catch a real effect. Most journals now want power of at least .80, meaning an 80% chance of detecting a true effect if one exists.
Type I vs Type II error: the key differences
| Type I error | Type II error | |
|---|---|---|
| Nickname | False positive | False negative |
| What happens | Reject a true null | Fail to reject a false null |
| Real-world analogy | Alarm rings, no fire | Fire rages, no alarm |
| Probability symbol | α (alpha) | β (beta) |
| You control it by | Setting your significance level | Sample size, effect size, power |
| Typical target | .05 | β ≤ .20 (power ≥ .80) |
| Medical example | Diagnosing a healthy patient as sick | Missing a disease in a sick patient |
The single most important thing to understand: these two errors pull against each other. If you make your test stricter to avoid false positives (say, using p < .01 instead of p < .05), you make it harder to detect real effects — which raises your Type II error rate. Balancing them is the whole art of study design.
A worked example with real numbers
Suppose you're testing whether a new study-skills workshop improves exam scores. Your null hypothesis is "the workshop makes no difference."
You run an independent-samples t-test comparing 30 students who attended the workshop against 30 who didn't. You set α = .05 in advance.
Scenario A — you find a difference: t(58) = 2.41, p = .019, d = 0.62. Since p < .05, you reject the null and conclude the workshop helped. But there's always a residual risk this result is a fluke — a Type I error. Your α of .05 means that if the workshop truly did nothing, you'd still see a "significant" result like this about 5% of the time.
Scenario B — you find nothing: t(58) = 1.28, p = .206, d = 0.33. You fail to reject the null. But notice the effect size, d = 0.33 — that's a small-to-moderate real difference that your sample was too small to confirm. This is the danger zone for a Type II error. With only 30 per group, your power to detect a d of 0.33 is roughly 35%, meaning you'd miss a real effect this size about two-thirds of the time. The correct interpretation isn't "the workshop doesn't work" — it's "this study couldn't confirm an effect."
That gap between "we found no effect" and "we found no evidence of an effect" is where careless conclusions get published — and later fail to replicate.
How do I reduce both types of error?
You can't eliminate both at once, but you can shrink your Type II risk without inflating Type I risk by increasing your sample size. A bigger sample gives you more power to detect real effects while keeping α fixed at .05. Running a power analysis before you collect data tells you how many participants you need to hit 80% power for the effect size you care about.
The other levers:
- Effect size: Larger true effects are easier to detect, so Type II errors are less likely for big effects.
- Measurement precision: Less noisy measures reduce variability, boosting power.
- One-tailed vs two-tailed tests: A directional (one-tailed) test has more power but only if you've justified the direction in advance.
What you should not do is quietly lower your α threshold after seeing your data, or run many tests and report only the significant ones — both inflate your real Type I error rate far above 5%.
Which error matters more for my study?
It depends entirely on the cost of being wrong. In a cancer-screening test, a Type II error (missing a real tumour) can be fatal, so you'd tolerate more false positives to catch every real case. In a criminal trial, the system is deliberately built to minimise Type I errors (convicting the innocent) even at the cost of more Type II errors (acquitting the guilty) — that's what "beyond reasonable doubt" means.
For most academic research, the field defaults to α = .05 and power ≥ .80, which weights false positives as roughly four times more costly than false negatives. But if you're studying something where a missed effect is dangerous, argue explicitly for a higher power target in your methods section.
When you run your analysis in StatRyx, the platform reports your p-value, effect size, and confidence interval together — so you can see not just whether a result is significant, but whether a non-significant one might be hiding a real effect due to low power. If you're weighing which test protects you best, our guide on choosing the right statistical test walks through the assumptions that affect your error rates.
Stop calculating this by hand — run it free in StatRyx → Try StatRyx
Frequently Asked Questions
Is a Type I or Type II error worse?
Neither is universally worse — it depends on the consequences. A Type I error (false positive) is worse when acting on a false result is costly, like approving an ineffective drug. A Type II error (false negative) is worse when missing a real effect is dangerous, like failing to detect a disease. Most research defaults to treating false positives as the more serious risk, setting α at .05.
What is the relationship between Type II error and statistical power?
Statistical power is exactly 1 minus the Type II error rate (β). If your power is .80, then β is .20, meaning you have a 20% chance of missing a real effect. Increasing power — usually by increasing sample size — directly reduces your Type II error rate.