ROC Analysis

Evaluate the discriminatory ability of a diagnostic test or predictive model by plotting sensitivity against specificity.

Receiver Operating Characteristic (ROC) analysis evaluates how well a continuous test score or model prediction discriminates between two binary outcomes (e.g., disease present vs. absent). The ROC curve plots sensitivity (true positive rate) against 1 − specificity (false positive rate) across all possible cutoff thresholds. The Area Under the Curve (AUC) summarises overall discriminatory ability: AUC = 0.5 indicates no discrimination (chance); AUC = 1.0 indicates perfect discrimination.

ROC analysis is foundational in clinical medicine (evaluating biomarkers), machine learning (model evaluation), and psychology (assessing signal detection). The optimal cutoff is typically identified using the Youden index (maximising sensitivity + specificity − 1), though the optimal balance depends on the relative costs of false positives and false negatives.

StatRyx generates the ROC curve, computes AUC with 95% CI (DeLong method), identifies the optimal Youden cutoff with its sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) — all in APA 7 format.

The ROC curve plots sensitivity against 1 − specificity at every possible cutoff of a continuous score. The area under the curve (AUC) summarises discrimination: 0.5 is chance, 1.0 is perfect separation. The Youden index (sensitivity + specificity − 1) picks the cutoff that balances the two. In practice, the right cutoff also depends on whether a missed case or a false alarm costs more.

When to use the ROC Analysis

You want to evaluate how well a continuous variable discriminates between two groups or outcomes
You need to identify the optimal cutoff threshold for a diagnostic test
You want to compare two or more tests or models on the same dataset
You validate a screening questionnaire against a clinical diagnosis
You want to report how well a logistic regression model's predicted probabilities separate the two outcomes

When not to use it

Your outcome has more than two categories — use Multiclass AUC or one-vs-rest ROC analysis instead
You only need sensitivity and specificity at a pre-specified cutoff — use A simple 2×2 diagnostic accuracy table instead
Your outcome is continuous — use Linear regression (R²) or correlation instead

Worked example

Cutoff	Sensitivity	Specificity	PPV	NPV	Youden Index
45.2	.78	.76	.81	.72	.54

How to report it (APA 7)

ROC analysis demonstrated good discriminatory ability of the biomarker for disease classification, AUC = 0.83 (95% CI [0.76, 0.90]). The optimal cutoff (Youden index = 0.54) was 45.2 ng/mL, yielding sensitivity = 0.78 and specificity = 0.76.

Frequently asked questions

Is ROC analysis free on StatRyx?

Yes. You can run it without an account (3 free analyses), and a free account adds 10 more — a one-time allowance, no payment needed. Every result includes the APA 7 write-up. Paid plans (Student from $9/month) add more analyses and Word, PDF and Excel export.

What is a good AUC value?

AUC 0.7–0.8 is considered acceptable, 0.8–0.9 is excellent, and > 0.9 is outstanding. AUC = 0.5 means the test performs no better than chance.

Can I compare two ROC curves?

Yes. StatRyx uses the DeLong method to statistically compare AUCs from two diagnostic tests on the same sample.

How do I report a ROC analysis in APA 7?

Report the AUC with its 95% CI and p-value, then the chosen cutoff with its sensitivity and specificity: "The scale discriminated cases from controls, AUC = .83, 95% CI [.76, .90]; a cutoff of 12 gave a sensitivity of .81 and a specificity of .74."

What is the difference between sensitivity and specificity?

Sensitivity is the share of true cases the test correctly identifies (true positive rate). Specificity is the share of non-cases it correctly rules out (true negative rate). Lowering the cutoff raises sensitivity and lowers specificity — the ROC curve shows that trade-off at every cutoff.

Run a ROC Analysis on your data →