AbstractID: 7339 Title: Statistical Power of ROC and FROC Experiments in Studies
Involving Lesion Localization
Statistical power determines the probability of detecting real differences between imaging
modalities and the cost in terms of readers and cases of conducting the study. As such it
is an important consideration in an observer performance study. It is likely that the
neglect of location information in nodule detection studies, as analyzed by the Receiver
Operating Characteristic (ROC) method, can compromise power. The Free–response
Receiver Operating Characteristic (FROC) method considers the location information but
its usage has been discouraged, since the FROC analysis model neglects correlations
between events on the same image. However, testing of the statistical powers of the two
methods has not yet been conducted. This study compared the statistical power of ROC
and AFROC methodologies using simulations. A new model (XFROC, for Extended
FROC) was developed for the decision variable sampling, which included intra-image
correlations. The model was used to simulate data for ROC and AFROC analysis. Six
readers and 200 cases, half of which contained one signal, were simulated for each trial.
Two hundred trials were run equally split between the Null Hypothesis (NH) and the
Alternative Hypothesis (AH). In each case the quasi-continuous ratings were analyzed by
the DBM method. The net separation of the NH and AH distributions was calculated. It
was found that in virtually every case the AFROC method yielded higher power. We
conclude that the FROC method can yield higher power than ROC and greater use of this
methodology is warranted.