RN Comprehensive Predictor® 2013
and NCLEX-RN® Readiness
INTRODUCTION
The purpose of the ATI RN Comprehensive Predictor® 2013 is twofold. The first purpose is
to provide students and educators with a numeric indication of the likelihood of passing
the NCLEX-RN® at the student’s current level of readiness. The second purpose is to guide
remediation efforts based on the exam content missed. This is achieved by providing a listing of
topics related to missed items in the individual and group score reports. To provide the numeric
indication of NCLEX-RN readiness, ATI engages in an extensive validation process involving a
statistical comparison of student performance on the RN Comprehensive Predictor 2013 and
actual NCLEX-RN first attempt pass/fail status.
Table 1 displays the results of this process. Each student’s individual score (expressed as a
percentage correct) is associated with a probability of passing the NCLEX-RN. The relationship
between Predictor scores and probability of passing rests on these assumptions.
1. The students taking the assessment are at or near completion of an RN nursing program and are
about to sit for the NCLEX-RN. Students with a significant amount of instruction to receive
before taking the NCLEX-RN can be expected to underperform on the Predictor.
2. The probability of passing refers to students’ first NCLEX-RN attempt after taking the Predictor.
After repeated attempts, it is expected that most students will eventually pass, but these
attempts are outside the realm of the Predictor.
3. The typical expectations of students taking a standardized test are met (e.g., students are
motivated to perform, no cheating has occurred, test is given under standardized
conditions in a proctored environment).
To the extent that these assumptions are not met, the validity of the prediction and of the test
scores themselves can be in question.
© 2014 Assessment Technologies Institute ®, Inc.
1
RN Comprehensive Predictor® 2013
and NCLEX-RN® Readiness
EXPECTANCY TABLE OF INDIVIDUAL PERCENTAGE CORRECT SCORE
AND PREDICTED PROBABILITY OF PASSING NCLEX-RN
Table 1
RN COMPREHENSIVE PREDICTOR
2013 INDIVIDUAL SCORE
PREDICTED PROBABILITY OF PASSING
THE NCLEX-RN
80.0% to 100.0%
77.3% to 79.3%
74.0% to 76.7%
72.0% to 73.3%
70.0% to 71.3%
68.7% to 69.3%
66.7% to 68.0%
65.3% to 66.0%
63.3% to 64.7%
60.0% to 62.7%
54.0% to 59.3%
0.0% to 53.3%
99%
98%
96% to 97%
94% to 95%
91% to 93%
89% to 90%
84% to 87%
80% to 82%
73% to 78%
59% to 71%
31% to 56%
1% to 28%
DEVELOPMENT OF THE PROBABILITY OF PASSING NCLEX-RN
EXPECTANCY TABLE
SAMPLE
The data for the expectancy table (Table 1) were collected during the norming phase of the
Predictor development. In the spring of 2013, ATI client institutions were asked if they would be
willing to administer the assessment to their RN students who were near graduation and expected
to sit for the NCLEX-RN in the summer of 2013. One hundred and three (103) RN nursing
programs volunteered and provided a total of 3,142 students who sat for the RN Comprehensive
Predictor. The original norms (means and percentile ranks) for the assessment were set based
on this sample of 3,142. These norms are initially reset based on students who have taken the
Predictor under live conditions after a sufficient sample has been accumulated.
All 3,142 students taking the assessment were asked if they would be willing to allow their
instructors to share their NCLEX-RN first attempt results with ATI. ATI only asked instructors for
the NCLEX-RN results of students who specifically agreed to provide them. Complete data from
1,642 students were obtained for the expectancy table development. The reported NCLEX-RN pass
rate for this sample was 90.3%. The first-attempt pass rate for the 2013 version of the NCLEX-RN
was 83.0% at the time this analysis was conducted (NCSBN, 2013). In order to run the analysis on
a group as representative of national NCLEX-RN performance as possible, 936 participants were
randomly selected from the group of 1,642 to have a sample pass rate of 83.0%. One outlier was
© 2014 Assessment Technologies Institute ®, Inc.
2
RN Comprehensive Predictor® 2013
and NCLEX-RN® Readiness
removed from the dataset for a total of 935 participants. The breakdown of this final sample by
region and program type is shown in Table 2. The score distribution for this sample is displayed
in Figure 1.
LOGISTIC REGRESSION RESULTS
The expectancy table was developed from this final sample of 935 using a statistical procedure
called logistic regression. Unlike standard regression, this procedure is specifically designed to
deal with the case of a continuous predictor variable (RN Comprehensive Predictor score) and a
categorical outcome variable (NCLEX-RN pass/fail status) (Thompson, 2006). For this sample, a
statistically significant relationship was found between Predictor scores and NCLEX-RN pass/fail
status (Model X2 = 202.0, df = 1, p < .001). This finding of statistical significance indicates there
is a high likelihood that a relationship exists between Predictor scores and NCLEX-RN results
in the greater population of nursing students. However, the statistical significance finding does
not indicate the strength of the relationship or the accuracy of the assessment in predicting
NCLEX‑RN results.
For this sample, a Nagelkerke R2 of 0.325 was found, indicating a moderate strength of
relationship between Predictor scores and NCLEX-RN pass/fail status. Another way to
conceptualize this R2 value is as a 32.5% reduction in predictive error by using the Comprehensive
Predictor versus using the pass rate of the sample alone (Menard, 1995). This R2 value, in
conjunction with the finding of statistical significance, is evidence that the Predictor can be
expected to perform as a useful indicator of NCLEX-RN readiness.
© 2014 Assessment Technologies Institute ®, Inc.
3
RN Comprehensive Predictor® 2013
and NCLEX-RN® Readiness
PERCENTAGE BREAKDOWN OF FINAL EXPECTANCY TABLE SAMPLE BY
REGION AND PROGRAM TYPE (N = 935)
Table 2.
PROGRAM TYPE
REGION
I. Connecticut, Maine, Massachusetts, New Hampshire,
Rhode Island, Vermont
II. New York, New Jersey
III. Delaware, Maryland, Pennsylvania, Virginia, West
Virginia, District of Columbia
IV. Alabama, Florida, Georgia, Kentucky, Mississippi, North
Carolina, South Carolina, Tennessee
V. Illinois, Indiana, Ohio, Michigan, Minnesota, Wisconsin
VI. Arkansas, Louisiana, New Mexico, Oklahoma, Texas
VII. Iowa, Kansas, Missouri, Nebraska
VIII. Colorado, Montana, North Dakota, South Dakota, Utah,
Wyoming
IX. Arizona, California, Hawaii, Nevada
X. Alaska, Idaho, Oregon, Washington
Total
TOTAL
ADN
BSN
Diploma
0.0%
0.3%
0.0%
0.3%
2.9%
12.2%
0.5%
2.6%
0.8%
0.6%
4.2%
15.5%
19.1%
6.3%
0.0%
25.4%
3.7%
11.3%
4.2%
0.0%
6.8%
4.7%
3.8%
0.0%
0.0%
4.9%
0.0%
0.0%
10.5%
20.9%
7.9%
0.0%
7.9%
2.0%
63.4%
2.1%
3.1%
30.2%
0.1%
0.0%
6.4%
10.2%
5.2%
100.0%
80
Frequency
60
40
20
Mean = 69.14
Std Dev = 7.49
N = 935
0
30
40
50
60
70
80
90
100
% Correct Individual Score
Figure 1. RN Comprehensive Predictor 2013 score distribution for final expectancy table sample
© 2014 Assessment Technologies Institute ®, Inc.
4
RN Comprehensive Predictor® 2013
and NCLEX-RN® Readiness
PREDICTION ACCURACY
The following information refers to the predictive accuracy of the test for this sample. When
evaluating the usefulness of the ATI Predictor or any other similar tool, it is overall predictive accuracy as
shown in Table 3 (i.e., 85.1%, the percentage of students who performed as predicted) that is of greater
value than the percentage of either outcome predicted. For example, Table 3 shows that 96.4% of those
who passed were predicted to do so. This 96.4% value has historically been thought of as the
predictive accuracy, but as will be shown this is not really the case.
Table 3.
PREDICTED AND ACTUAL NCLEX-RN PASS/FAIL OUTCOMES
Actual Fail
Actual Pass
N
Correct Prediction Percentage
PREDICTED
FAIL
48
28
76
63.2%
PREDICTED
PASS
111
748
859
87.1%
N
PREDICTED
PERCENTAGE
30.2%
96.4%
159
776
935
85.1%
Table 3 displays the predicted and actual pass/fail classifications from the analysis. The following
information highlights the meaning of some of the percentages on this table:
Number of examinees predicted to fail and did fail the NCLEX-RN
Number of examinees predicted to fail the NCLEX-RN
=
48
76
= 63.2%
Number of examinees predicted to pass and did pass the NCLEX-RN
=
Number of examinees predicted to pass the NCLEX-RN
748
859
= 87.1%
48 + 748
76 + 859
= 85.1%
Number of examinees the Predictor predicted correctly
Number of examinees that passed or failed the NCLEX-RN
=
In other words, 63.2% of those who were predicted to fail did so, and 87.1% of those predicted
to pass did so. In total, the Predictor correctly predicted the pass and failure status of 85.1%
of the examinees. Although prediction accuracy is often reported as the 96.4% value under
the “Predicted Percentage” column, this refers only to the percentage of students who passed
NCLEX‑RN that were predicted to do so, i.e.:
Number of examinees the Predictor predicted to pass and did pass
the NCLEX-RN
Number of examinees predicted to pass the NCLEX-RN
=
748
= 96.4%
776
© 2014 Assessment Technologies Institute ®, Inc.
5
RN Comprehensive Predictor® 2013
and NCLEX-RN® Readiness
This value has very little practical meaning and should not be used to determine the value of a
Predictor. There are two reasons for this.
1. Notice that the denominator of the fraction (776) counts the number of examinees who
passed the NCLEX-RN, not the number of examinees predicted to pass. If a Predictor
predicted that every student would pass the NCLEX-RN, the numerator would be equal to
776 instead of 748, and the formula would give a “prediction accuracy” of 100%.
2. This value does not take into account examinees that were predicted to or did fail the
NCLEX -RN. It only accounts for those who passed.
Again, the actual NCLEX-RN pass/fail status was accurately predicted for 85.1% of the sample.
Although classification tables such as Table 3 have become widely accepted as the way to evaluate
predictor test accuracy, these tables do not take into account that students are not actually scored
as “likely to pass” or “likely to fail” on the ATI Predictor; they are given a numeric probability of
passing (see Table 1). This probability tells the student how likely they are to pass, rather than
simply if they are likely to pass.
The logistic regression procedure bases the predicted classifications on a probability of passing
the NCLEX-RN. Any student found to have a probability of passing greater than or equal to .50
is classified as “predicted to pass.” Any student who has a probability less than .50 is classified
as “predicted to fail.” Because the classifications in Table 3 are based on probabilities, a reasonable
number of misclassifications are to be expected. For example, in a group of 10 students, each with a
passing probability of 0.80, all would be classified as “predicted to pass.” However, the probability of
0.80 indicates the expectation that 8 of the 10 actually will pass, and the remaining 2 of the 10 will
be expected to fail. These two students who failed, but were expected to pass would be “misclassified”
in Table 3.
The pass/fail classifications illustrated in Table 3 are strictly for the purpose of evaluating the
RN Comprehensive Predictor in this study. Please note that no “likely to pass” or “likely to fail”
classification is reported to students or faculty, only the probabilities contained in the expectancy
table (Table 1). It is recommended that programs using the Predictor consider the predicted
probability of passing as a measure of NCLEX-RN readiness, and direct remediation efforts
accordingly.
© 2014 Assessment Technologies Institute ®, Inc.
6
RN Comprehensive Predictor® 2013
and NCLEX-RN® Readiness
RN COMPREHENSIVE PREDICTOR BENCHMARKING
RECOMMENDATIONS
Because the purpose of the RN Comprehensive Predictor 2013 is to predict performance on
the NCLEX-RN, the expectancy table data in Table 1 is likely to be the most relevant source of
information for making decisions concerning student performance. As such, the first step in
setting a Predictor standard should be to consider what an acceptable level of risk is for a student
to fail the NCLEX-RN. For example, if a program felt that meeting a proposed standard required at
least an 82% chance of passing the NCLEX-RN on the first attempt, the corresponding individual
score for the Predictor would need to be 66.0% (see Table 1). Note that this is only an example
and not meant as a suggested standard.
Before finalizing a Predictor standard, it is necessary to examine its likely impact. If data from
previous administrations of the RN Comprehensive Predictor 2013 are available, a program can
apply the proposed standard to the scores and see how many students met it. Assuming this pass
rate “reality check” is not far out of line with the program’s expectations, the proposed standard
can be finalized. It might be necessary for the program to re-evaluate what is considered an
acceptable level of risk if too many students are failing at the proposed standard.
Programs without previous administrations of the RN Comprehensive Predictor 2013 can still
evaluate likely impact using percentile rank data available from ATI. This percentile rank data
is contained in the RN Faculty Resource Guide, which is available on the ATI website to ATI
clients. In the case of the example above, an individual score of 66.0% currently corresponds
with a national percentile rank of 40. This means that approximately 40% of the students in the
norming sample scored at or below 66.0%. A program evaluating this proposed standard would
need to consider how far above or below the national average their students have historically
been. Programs above the national average could probably expect that fewer than 40% would fail
to meet the proposed standard.
A few cautions about setting a standard for the Predictor are necessary.
1. The Predictor pass rate should not dictate the standard; it should only act as a “reality
check.” The criterion is NCLEX-RN readiness, not the number of students passing the
Predictor. If most of the students in question are truly unprepared for the NCLEX-RN it
is perfectly reasonable for most of them to fail the program’s standard on the Predictor.
However, the Predictor standard should not be excessively difficult for students who really
are prepared for NCLEX-RN.
2. The Predictor standard should be re-evaluated on a regular basis. Regardless of whether the
initial standard was set using previous data or percentile ranks, it is advisable to check the
Predictor pass rate at least once per semester and adjust the standard if need be.
© 2014 Assessment Technologies Institute ®, Inc.
7
RN Comprehensive Predictor® 2013
and NCLEX-RN® Readiness
3. Data from earlier versions of the Predictor are not applicable or appropriate to use when
setting the 2013 Predictor standard. RN Comprehensive Predictor 2010 has its own
expectancy table and is not equated to RN Comprehensive Predictor 2013.
4. The expectancy table is only applicable to the RN Comprehensive Predictor, not to other
ATI assessments. Probability of passing NCLEX-RN does not pertain to the Content
Mastery Series®, TEAS, or any other ATI assessment.
5. The RN Comprehensive Predictor is not recommended for high-stakes uses (e.g.,
graduation, course grade) without an extensive and well-documented validation process.
REFERENCES
Menard, S. (1995). Applied Logistic Regression Analysis. Thousand Oaks, CA: Sage.
National Council of State Boards of Nursing (2013). https://www​.ncsbn.org/ Table_of_Pass_
Rates_2013​.pdf
Thompson, B. (2006). Foundations of Behavioral Statistics: An Insight Based Approach. New York,
NY: Guilford Press.
© 2014 Assessment Technologies Institute ®, Inc.
8