Receiver Operating Characteristics
Receiver Operating Characteristic (ROC) curves are a useful way to interpret
sensitivity and specificity levels and to determine related cut scores. ROC curves are a
generalization of the set of potential combinations of sensitivity and specificity possible
for predictors (Pepe, Janes, Longton, Leisenring, & Newcomb, 2004). ROC curve
analyses not only provide information about cut scores, but also provide a natural
common scale for comparing different predictors that are measured in different units,
whereas the odds ratio in logistic regression analysis must be interpreted according to a
unit increase in the value of the predictor, which can make comparison between
predictors difficult (Pepe, et al., 2004). An overall indication of the diagnostic accuracy
of a ROC curve is the area under the curve (AUC). AUC values closer to 1 indicate the
screening measure reliably distinguishes among students with satisfactory and
unsatisfactory reading performance, whereas values at .50 indicate the predictor is no
better than chance (Zhou, Obuchowski, & Obuschowski, 2002).
Compute a ROC Curve in SPSS
To compute a ROC curve in SPSS (v. 16), the following procedures should be
followed:
1. Your screening measure should use a standard score
2. Your outcome measure should be recoded into a dichotomous variable of “Not atrisk” – “0”, and “At-Risk” – “1”
1
3. Under the top menu option “Analysis”, select “ROC Curve”
4. You will have a dialog box that says ROC Curve. In this box, you should drag
your screening measure to the box that says “Test Variable”. You should drag
your dichotomized outcome variable into the “State Variable” box.
5. In the box that asks for the “Value of the State Variable”, put “1” – (you are
trying to predict which kids are at risk).
6. In the box labeled “Display” check the options for “ROC Curve” “with Diagonal
reference line” and “Standard Error and Confidence Interval”
7. In the upper right corner of the ROC Curve dialog box, you’ll see the word
“Options” – click on it.
8. A new dialog box should open called “ROC Curve Analysis; Options”. In this
box, under “Test Direction” select the appropriate option. You will likely need to
select “Smaller test result indicates more positive test” – this means that a lower
score on the screen instrument means the student is more likely to be at risk.
9. Select “Continue” and you should be returned to the ROC Curve dialog box,
select “Ok” and the analysis will run.
10. The statistics of interest will be:
a. Case Processing Summary - this gives the total N, as well as the number
of students at risk and not at risk
b. ROC Curve Graph – gives a visual depiction of the screening measure’s
utility
c. Area Under the Curve table – provides the AUC along with the standard
error & confidence interval
2
Compute a ROC Curve in Excel
To compute a ROC curve using Excel, consult the following website to download a
template: http://www.analyseit.com/products/method_evaluation/roc.aspx?gclid=COba5eWAw5cCFRxNagodZ3sxSw
3
References
Pepe, M., Janes, H., Longton, G., Leisenring, W. & Newcomb, P. (2004). Limitations of
the odds ratio in gauging the performance of a diagnostic, prognostic, or screening
marker. American Journal of Epidemiology, 159, 882-890.
Zhou, X. H., Obuchowski, N. A., & Obushcowski, D. M. (2002). Statistical methods in
diagnostic medicine. Wiley & Sons: New York.
4