FORECASTING NURSING STUDENT SUCCESS AND FAILURE ON THE
NCLEX-RN USING PREDICTOR TESTS
by
Lawrence A. Santiago
Copyright 2013
A Dissertation Presented in Partial Fulfillment
of the Requirements for the Degree
Doctor of Education
University of Phoenix
ABSTRACT
A severe and worsening nursing shortage exists in the United States. Increasing numbers
of new graduate nurses are necessary to meet this demand. To address the concerns of
increased nursing demand, leaders of nursing schools must ensure larger numbers of
nursing students graduate. Prior to practicing as registered nurses in the United States,
graduates of nursing schools must pass the National Council Licensure Examination for
Registered Nurses (NCLEX-RN). Various companies, such as the Assessment
Technologies Institute (ATI) and Kaplan have created NCLEX-RN predictor tests that
report candidates’ chances of passing the NCLEX-RN. ATI created a test called the RN
Comprehensive Predictor and Kaplan created an NCLEX-RN predictor examination
called the Kaplan Readiness Test. Students with less than optimal scores on the predictor
can remediate to improve their knowledge of nursing, critical thinking, and test-taking
skills. The intent for the ATI RN Comprehensive Predictor and the Kaplan Readiness
Test is to predict the probability of success on the NCLEX-RN. The focus of the
quantitative study was the ability of the aforementioned examinations to predict both
success and failure on the NCLEX-RN. The overall TEAS score did not have a
significant relationship with NCLEX-RN results. However, the ATI TEAS Math score
was significantly higher (p = .005) for students who passed the NCLEX-RN. Data
analysis showed no significant relationship between the Kaplan NCLEX-RN Readiness
Test and NCLEX-RN results (p > .05). A significant relationship between the ATI RN
Comprehensive Predictor and the NCLEX-RN scores existed in both the total sample (p
= .001) and the BSN subsample (p = .001). In the MSN student subsample, all 37
students passed the NCLEX on the first attempt.
iii
DEDICATION
This work is dedicated to my parents, Mildred and Larry. Thank you for helping
me become who I am today.
ACKNOWLEDGMENTS
I would like to take this opportunity to thank all of the individuals who have
supported me in this doctoral program. First of all, I would like to thank God for all of
the blessings He has given me in this life. Through the past four years, my family’s love,
support, and encouragement has sustained me through this journey. My wife Myla has
been by my side through each course, each residency, and each stage of the dissertation
process. Asawa, mahal na mahal kita! A special thanks to my son Joshua. You are the
best son a father could have.
I would also like to thank my dissertation committee. Dr. Phan, thank you for
your continued inspiration throughout the dissertation process. Dr. Tidwell, thank you
for sharing your expertise and experience with me. Finally, Dr. McAtee, thanks for being
so reliable over the years.
I would also like to thank the nursing department, including Dr. Susan
Drummond, Dr. Geneva Oaks, Jeanette Russell, and Lynne Bosch. Thanks for helping
with my study and answering my many emails.
Finally, I would like to thank my colleagues for their steadfast support: Millicent
De Jesus, Caroline Alfonso, Jeffrey Woods, and Karen Silva. Thanks for always being
there for me! I cannot forget the support of my director, Peachy Hain. You have truly
been my cheerleader through this process.
ii
TABLE OF CONTENTS
Contents
Page
List of Tables ......................................................................................................... vi
Chapter 1: Introduction ............................................................................................1
Problem Statement .......................................................................................2
Purpose of the Study ....................................................................................6
Significance of the Problem .........................................................................7
Significance of the Study .............................................................................7
Nature of the Study ......................................................................................9
Research Questions ....................................................................................11
Hypotheses .................................................................................................12
Theoretical Framework ..............................................................................13
Definition of Terms....................................................................................14
Assumptions...............................................................................................15
Scope, Limitations, and Delimitations .......................................................16
Summary ....................................................................................................17
Chapter 2: Review of the Literature.......................................................................19
Title Searches, Articles, Research Documents, and Journals ....................20
Bloom’s Taxonomy and Revision .............................................................20
Nursing Shortage .......................................................................................22
Societal Need .............................................................................................25
History of Nursing Licensure Examinations..............................................26
NCLEX-RN ...............................................................................................26
iii
Quality of Nursing Programs .....................................................................27
Predictive Validity .....................................................................................28
High-Stakes Testing ...................................................................................29
Test Anxiety ...............................................................................................31
Computerized Testing ................................................................................31
Costs of NCLEX-RN Failure .....................................................................32
Historical NCLEX Predictor Research ......................................................32
Current NCLEX Predictor Research ..........................................................33
Beyond the NCLEX-RN ............................................................................40
Attitudinal Change .....................................................................................41
Gaps in the Literature.................................................................................41
Summary ....................................................................................................42
Chapter 3: Research Methods ................................................................................44
Research Method and Design Appropriateness .........................................44
Population ..................................................................................................45
Sampling ....................................................................................................45
Geographic Location ..................................................................................46
Data Collection ..........................................................................................46
Instrumentation ..........................................................................................47
Validity and Reliability ..............................................................................48
Data Analysis .............................................................................................49
Research variables .........................................................................49
Logistic regression analysis ...........................................................49
iv
Summary ....................................................................................................49
Chapter 4: Results ..................................................................................................51
Research Questions ....................................................................................51
Data Analysis .............................................................................................52
Testing of the Hypotheses ..........................................................................54
Findings......................................................................................................57
Testing of the Hypotheses ..........................................................................58
Chapter Summary ......................................................................................60
Chapter 5: Conclusions and Recommendations ....................................................61
Purpose of the Study ..................................................................................61
Research Questions ....................................................................................62
Summary of Key Findings .........................................................................63
Implications................................................................................................64
Limitations .................................................................................................65
Suggestions for Future Research ...............................................................66
Significance................................................................................................67
Practitioner Recommendations ..................................................................68
Conclusion .................................................................................................69
References ..............................................................................................................71
Appendix A: Data Collection Tool ........................................................................83
Appendix B: Permission to Use Premises Form ....................................................84
Appendix C: Letter of Collaboration Among Institutions .....................................85
v
LIST OF TABLES
Table 1: Frequency Counts for Selected Demographic Variables (N = 251) ........53
Table 2: Descriptive Statistics for Selected Variables (N = 251) ..........................53
Table 3: Distribution of Predicted and Actual NCLEX Scores and Pass Rates for 2007
Predictor (n = 162) .................................................................................................54
Table 4: Distribution of Predicted and Actual NCLEX Scores and Pass Rates for 2010
Predictor (n = 57) ...................................................................................................55
Table 5: t Test Comparisons Based on NCLEX Outcome for Selected Variables 56
Table 6: Relationship Between Outcome of ATI Screening Test and NCLEX Outcome
................................................................................................................................58
vi
Chapter 1
Introduction
A severe and worsening nursing shortage exists in the United States (American
Association of Colleges of Nursing [AACN], 2011) and more new-graduate nurses are
necessary to meet this demand. To address the concerns of increased nursing demand,
larger numbers of nursing students must graduate from nursing schools (Roa, Shipman,
Hooten, & Carter, 2011). Prior to practicing as registered nurses in the United States,
graduates of nursing schools must pass the National Council Licensure Examination for
Registered Nurses (NCLEX-RN), which is the national registered nurse licensing
examination (National Council of State Boards of Nursing [NCSBN], 2011).
Various companies such as the Assessment Technologies Institute (ATI) and
Kaplan have created NCLEX-RN predictor tests that report a candidate’s chances of
passing the NCLEX-RN (ATI, 2012; Kaplan, n.d.). Leaders at ATI created a test called
the RN Comprehensive Predictor (ATI, 2012) and leaders at Kaplan created an NCLEXRN predictor examination called the Kaplan Readiness Test (Kaplan, n.d.). Students with
less than optimal scores on the predictor can remediate to improve their knowledge of
nursing, critical thinking, and test-taking skills. The intent for the ATI RN
Comprehensive Predictor and the Kaplan Readiness Test is to predict the probability of
success on the NCLEX-RN. The focus of the current quantitative study is the ability of
the aforementioned examinations to predict both success and failure on the NCLEX-RN.
Leaders at ATI also created the Test of Essential Academic Skills (TEAS).
Educators at various schools of nursing use this test as part of the admissions
requirements. Students at these schools, including the school in the current study, must
1
achieve a certain score on the test, determined by each school, to be eligible for
admission to their school of nursing (ATI, 2011). The purpose of the test is to ensure
admitted students are more likely to be successful in nursing school and to pass the
NCLEX-RN than those students who scored below the indicated passing mark on the
TEAS. The focus of the current quantitative study was also on the ability of the TEAS to
predict success and failure on the NCLEX-RN.
Chapter 1 contains the problem and purpose of the quantitative, retrospective
study. The study focus was on the ability of the TEAS, the ATI RN Comprehensive
Predictor, and the Kaplan Readiness Test to forecast the outcome of NCLEX-RN results
for the total population of registered nurse candidates from a school of nursing in
California. Due to reasons unknown to the researcher, only 219 of the 251 graduates
took the ATI RN Comprehensive Predictor, and only 100 of the 251 graduates took the
Kaplan NCLEX-RN Readiness Test. Literature on NCLEX-RN predictor accuracy of
nursing board examination failure is lacking. Therefore, determining the accuracy of
these standardized tests to predict success and failure might assist in establishing the
degree of usefulness for prospective registered nurses. Research such as this quantitative
study could lead to earlier NCLEX-RN success and alleviation of the nursing shortage.
When nursing students pass the NCLEX-RN earlier, they will be able to obtain
employment and lessen the nursing shortage.
Problem Statement
A gap exists in the literature regarding the ability of the ATI RN Comprehensive
Predictor or the Kaplan NCLEX-RN Readiness Test to forecast failure on the NCLEXRN accurately. Twenty percent of nursing schools in the United States will not allow
2
students to graduate and receive their nursing degree unless they pass the predictor test
(National League for Nursing [NLN], 2012). A review of the EBSCOhost and ProQuest
databases revealed no studies on the predictive accuracy of the Kaplan NCLEX-RN
Readiness Test. EBSCOhost and ProQuest are use-for-fee research database providers
that allow students and researchers to view articles from thousands of peer-reviewed
journals (EBSCO Industries, 2012; ProQuest, 2012). The quantitative, retrospective
study involved examining the ATI TEAS scores, the ATI RN Comprehensive Predictor
scores, and the Kaplan Readiness Test scores for the total population of graduates from a
nursing school in California. The purpose of the study was to determine the extent to
which these standardized tests can accurately predict both passing and failure on the
NCLEX-RN.
Background of the Problem
The nationwide nursing shortage continues to worsen. As baby boomers age, the
need for nurses to care for this population will increase (AACN, 2011). The U.S. Census
Bureau (2006) defined baby boomers as Americans born between 1946 and 1964. Baby
boomers include Americans born in foreign countries who immigrated to the United
States (U.S. Census Bureau, 2006). As of 2006, the U.S. Census Bureau reported that
77,980,296 baby boomers resided in the United States. The nursing shortage is even
worse in California because the state has mandatory nurse-to-patient ratios (Aiken et al.,
2010). These ratios include five patients per nurse in a medical/surgical acute setting and
two patients per nurse in an intensive/critical care setting (California Department of
Public Health, 2010). Nurses in California care for one less patient on average than other
states (California Department of Public Health, 2010). Because of this mandatory ratio, it
3
is more difficult to staff hospitals in California with sufficient numbers of nurses than in
other states (California Department of Public Health, 2010). California is the only state
in the United States with mandatory nurse ratios (American Nurses Association, 2011).
Insufficient staffing decreases patient safety, increases work-related stress and
injuries for nurses, and causes nurses to leave the profession (AACN, 2011). Large
numbers of registered nurses will retire over the next 25 years, contributing to a projected
shortage of 265,000 nurses by 2025 (Buerhaus, Staiger, & Auerbach, 2009). The demand
for registered nurses will continue to increase, which will create the largest shortage of
nurses since the 1960s (Buerhaus et al., 2009). The average life expectancy in the United
States is 77.8 years, and the United States will experience a substantial increase in its
elderly population over the next 30 years, from now until 2033 (Orman & Guarneri,
2009). By 2030, all baby boomers will be over 65 years old (Buerhaus et al., 2009).
Increasing the number of prospective registered nurses who can successfully pass the
NCLEX-RN will assist in alleviating the nursing shortage.
The NCLEX-RN passing standard has increased in recent years, which could
potentially place more graduate students at risk for failure (Carrick, 2011). Although
nursing school leaders in California should always ensure their graduating students are
ready to pass the NCLEX-RN and join the nursing workforce, the worsening shortage
further indicates the importance of first-time NCLEX-RN success (AACN, 2011). The
purpose of an NCLEX-RN predictor test is to identify readiness for students to sit for the
exam and areas of weakness to assist with remediation.
A significant number of nursing schools in the United States use NCLEX-RN
predictor tests to determine whether students can receive approval to take the NCLEX4
RN (Spurlock & Hunt, 2008). The primarily reason for designing exit examinations is to
predict readiness for the NCLEX-RN. Many schools use the exit exam as a high-stakes
test. Nichols and Berliner (2008) defined high-stakes testing as “the practice of attaching
important consequences to high stakes test scores” (p. 41). High-stakes nursing tests
require students to pass before they can obtain approval to take the NCLEX-RN
(Harding, 2010). If nursing instructors are failing students based on the results of a highstakes test, then the test must accurately predict failure on the NCLEX-RN (Giddens,
2009; Spurlock & Hanks, 2004). A limited number of NCLEX-RN predictor studies
have shown prediction of failure rates to be less than 50% accurate (Benefiel, 2011;
Spurlock & Hanks, 2004). Potentially, some students held back would pass the NCLEXRN with the first attempt. Students capable of passing the NCLEX-RN should be able to
sit for the test, pass the national board examination, and begin their careers as registered
nurses (Spurlock & Hanks, 2004).
Nursing leaders should identify variables that affect NCLEX-RN pass rates
(Ukpabi, 2008). An NCLEX-RN predictor test can help students to determine the
knowledge and concepts that were deficient in their ability to pass the NCLEX-RN
(Ukpabi, 2008). For the ATI RN Comprehensive Predictor, nursing instructors have the
option to provide students with a breakdown of the specific topics for which the students
require remediation (ATI, 2011). Remediation should be the primary purpose of a
predictor examination, not being a singular measure of progression in a nursing program
(Ukpabi, 2008).
5
Purpose of the Study
The purpose of the quantitative study was to determine if the TEAS, the ATI RN
Comprehensive Predictor, and the Kaplan NCLEX-RN Readiness Test could accurately
predict both success and failure on the NCLEX-RN at one nursing program in Southern
California. The quantitative study took place at one nursing program in the Southern
California region. This program offers both a baccalaureate degree program (bachelor of
science in nursing, BSN) and an accelerated entry level master’s degree program (master
of science in nursing, MSN). If these tests can accurately predict student outcomes on
the NCLEX-RN, they will be valuable tools for selecting students for remediation
(Uyehara, Magnussen, Itano, & Zhang, 2007). Faculty can identify students at risk for
failing the NCLEX-RN and develop an individualized tutoring and mentoring program
for them (Uyehara et al., 2007). The interventions are one method of increasing the
nursing school’s first-time NCLEX-RN pass rate (Uyehara et al., 2007).
The quantitative study involved a statistical analysis that measured the ability of
the students’ ATI TEAS results, their ATI RN Comprehensive Predictor results, and their
Kaplan Readiness Test results to predict both success and failure on the NCLEX-RN.
The school required students to take all three examinations. Due to reasons unknown to
the researcher, only 219 of the 251 graduates took the ATI RN Comprehensive Predictor.
In addition, only 100 of the 251 graduates took the Kaplan NCLEX-RN Readiness Test.
Because the purpose of the study was to determine the extent to which the predictor tests
can individually forecast success on the NCLEX-RN, a quantitative, retrospective study
design was appropriate. Because the study involved comparing the NCLEX-RN result to
the NCLEX-RN predictor result, a retrospective design was necessary.
6
Significance of the Problem
Although employment of registered nurses is increasing rapidly, the growth might
not be enough to meet the increasing demand (Bureau of Labor Statistics, 2012). In
2011, leaders at the AACN reported 135,000 registered nurse vacancies. Of all registered
nurse positions in the United States, 8.1% were vacant. This is an alarming statistic
because the nursing shortage persists in a time of economic recession (AACN, 2011). As
nurses retire, the number of newer nurses entering the profession is not sufficient to meet
the demand (Buerhaus et al., 2009). Nursing schools must ensure they graduate every
student who is able to safety practice as a registered nurse. The number of elderly will
dramatically increase because the baby boomer generation is aging (Buerhaus et al.,
2009). More nurses will be necessary to care for the older population (Bureau of Labor
Statistics, 2009).
If nursing school prorgression policies prevent a large number of nursing students
capable of passing the NCLEX-RN from graduating, this group of prospective nurses is a
delayed resource because these individuals will not be able to assist in alleviating the
nursing shortage in a timely manner. More research on NCLEX-RN predictor failure
accuracy is necessary because of the insufficient number of studies on this topic. In
addition, a review of the literature revealed published studies on the Kaplan NCLEX-RN
Readiness Test, which opened an opportunity for further research to validate this test.
Significance of the Study
The quantitative study involved comparing student results of the TEAS test, the
ATI RN Comprehensive Predictor, and the Kaplan Readiness Test to their result on the
NCLEX-RN. The findings might assist nursing leaders with developing policies for
7
remediation. Although other researchers have measured the accuracy of exit exams to
predict passing the NCLEX-RN, few researchers have sought to measure the accuracy of
these exams to predict failure (Abbott, 2008; Adamson & Britt, 2009; Landry, Davis,
Alameida, Prive, & Renwanz-Boyle, 2010). Prediction to pass relates to the result of the
predictor test. If the predictor test result is 60%, the prediction is that the student will
have a 60% chance of passing the NCLEX-RN and a 40% chance of failing the NCLEXRN (ATI, 2011; Kaplan, n.d.). A lack of literature exists on the topic of NCLEX-RN
predictor failure. Also, a review of literature did not reveal any studies measuring the
predictive accuracy of the Kaplan NCLEX-RN Readiness Test.
The predictor tests provide scores after students complete the examinations. A
table is provided for faculty that attaches the scores to percentages that indicate the
chances of passing the NCLEX-RN if the students were to take it that day. If a highstakes test cannot accurately predict failure on the NCLEX-RN, passing such a test
should not be a requirement for graduation from the nursing program (Harding, Rateau,
& Heise, 2011). The primary purpose of the proposed study is to examine the ability of
the ATI RN Precdictor Test and the Kaplan NCLEX-RN Readiness Test to determine
failure on the NCLEX, as well as passing.
Although it is important for educators to determine if students are ready for the
NCLEX-RN examination, it is critical to choose a valid research design to confirm these
predictions. This research design includes the ability of the research instruments to
predict NCLEX-RN success as well as failure. Through careful scrutiny of research
design, researchers can implement an accurate study (Burns & Grove, 2009). Using the
appropriate research design will further validate the findings of the study and increase the
8
ability of the results to apply to other academic settings. The next section will provide a
brief overview of the chosen research design for the study.
Nature of the Study
The quantitative, retrospective study involved examining the predictive accuracy
of the ATI TEAS test scores, the ATI RN Comprehensive Predictor scores, and the
Kaplan Readiness Test scores to forecast both passing and failing the NCLEX-RN at one
school of nursing in Southern California. Retrospective studies are nonexperimental, and
no manipulation of the subjects occurs (Burns & Grove, 2009). In a retrospective study,
a researcher attempts to link an event to another event that occurred prior to the first one
(Burns & Grove, 2009). The current study involved linking NCLEX-RN results to
NCLEX-RN predictor tests taken before the students’ graduation. A multiple regression
analysis served to determine if a relationship exists between the test scores and the
NCLEX-RN outcomes. The software used to perform the analysis was Statistical
Package for the Social Sciences (SPSS) 21.
A retrospective, quantitative design was the ideal method to compare standardized
test data to the NCLEX-RN. Retrospective studies involve measuring current
circumstances that might have a relationship with past circumstances (Polit & Beck,
2008). The current study involved analyzing NCLEX-RN scores with ATI TEAS scores,
ATI RN Comprehensive Predictor scores, and Kaplan Readiness Test scores to determine
the extent of their relationship. Qualitative research designs were not appropriate for the
study.
A Southern California baccalaureate degree nursing program was the site for the
retrospective study. Data collection consisted of examining archived data. The data
9
included ATI TEAS scores, ATI RN Comprehensive Predictor scores, and Kaplan
NCLEX-RN Readiness Test scores. Prospective nursing students take the TEAS before
admission to nursing school and take the exit tests during the last semester of the nursing
program. The researcher collected the data on the campus. He received archived student
test results and he recorded the results onto the Data Collection Tool (see Appendix A).
Archived data analysis is cost-effective, accessible, and available (Creswell, 2005). Data
collection proceeded in an organized, systematic manner that maximized the
generalization of the findings (LoBiondo-Wood & Haber, 2008).
The data gathered were from one nursing program in Southern California after
receiving approval from the school’s institutional review board (IRB) as well as the IRB
from University of Phoenix. All applicants to the nursing school must take the TEAS to
receive consideration for admission to the nursing school. All students take both the ATI
RN Comprehensive Predictor and the Kaplan Readiness Test prior to graduation. Due to
reasons unknown to the researcher, only 219 of the 251 graduates took the ATI RN
Comprehensive Predictor. In addition, only 100 of the 251 graduates took the Kaplan
NCLEX-RN Readiness Test. The study involved measuring only the first attempt of
these predictor examinations. Students who fail one or both of these examinations must
enter a remediation course. After the course, the student will retake the test. Students
must pass this examination to graduate. The study involved comparing results data from
five student cohorts (2008-2011) to the students’ pass–fail data on the NCLEX-RN.
Student consent was not a requirement because the study involved analyzing existing data
sets. No personal contact with any nursing students or graduates took place. Data
collection occurred during a review of retrospective files at the school of nursing. The
10
use of codes rather than student names served to protect students’ identity. The
researcher consistently matched all scores with the correct code number.
Research Questions
The research project included two questions. The questions were appropriate to
the study and highlighted the dependent and independent variables. The study involved
determining the extent to which three standardized nursing exams could predict both
success and failure on the NCLEX-RN at one nursing school in Southern California.
1. How are ATI RN Comprehensive Predictor and Kaplan Readiness Test
examination scores (independent variable) related to results on the NCLEX-RN
(dependent variable)?
Recent studies have shown the ability of NCLEX-RN predictor examinations to forecast
success on the NCLEX-RN accurately (Harding et al., 2011; McGahee, 2010; Ukpabi,
2008). The predictor tests examined included the Health Education Systems, Inc. (HESI)
Exit Exam, the ATI RN Comprehensive Predictor test, and the Educational Resources
Inc. (ERI) test. The ERI test was an NCLEX-RN predictor test that no longer exists
(Bondmass, Moonie, & Kowalski, 2008). Lacking in the literature were studies on the
ability of NCLEX-RN predictor exams to forecast failure on the NCLEX-RN accurately.
The current study involved analyzing both success and failure predictions. A multiple
regression analysis assisted with the study (LoBiondo-Wood & Haber, 2008). In some
schools of nursing, including the school in the current study, students must pass an
NCLEX-RN predictor examination prior to graduation and prior to receiving permission
to sit for the nursing board examination. If the predictor test cannot accurately forecast
failure on the NCLEX-RN, the progression policy might not be fair to the students. The
11
focus of the second research question was NCLEX-RN failure and NCLEX-RN predictor
accuracy.
2. How are nursing school admissions test scores (independent variable) related
to results on the NCLEX-RN (dependent variable)?
Researchers recently examined the relationship between high scores on
standardized nursing school admissions examinations and positive results on the
NCLEX-RN, but a gap exists in the literature regarding the relationship between low
scores on the admissions exams and negative results on the NCLEX-RN. The two
research questions served as guides during data collection. The focus of the study was
the results of the ATI TEAS test, the ATI RN Comprehensive Predictor test, and the
Kaplan Readiness Test compared with the NCLEX-RN results. A statistical analysis
assisted in determining the answers to the study questions.
Hypotheses
Parallel to the research questions were the following null and alternative
hypotheses:
H10: The ATI RN Comprehensive Predictor test score is not significantly related
to results on the NCLEX-RN.
H1a: The ATI RN Comprehensive Predictor test score is significantly related to
results on the NCLEX-RN.
H20: The Kaplan Readiness Test score is not significantly related to results on the
NCLEX-RN.
H2a: The Kaplan Readiness Test score is significantly related to results on the
NCLEX-RN.
12
H30: The ATI TEAS nursing school admissions test score is not significantly
related to results on the NCLEX-RN.
H3a: The ATI TEAS nursing school admissions test score is significantly related
to results on the NCLEX-RN.
The study determined the extent to which the ATI TEAS test, the ATI RN
Comprehensive Predictor, and the Kaplan Readiness Test could foretell both success and
failure on the NCLEX-RN. The preceding hypotheses were appropriate for the
quantitative, retrospective nature of the study.
Theoretical Framework
For both nursing schools and the NCSBN, one of the primary educational goals is
for students to develop didactic and clinical competency. Bloom’s taxonomy has
historically served as a guide for faculty to determine what is the appropriate level at
which a nursing student must function. Bloom, Engelhart, Furst, Hill, and Krathwohl
(1956) first developed the taxonomy to classify learning outcomes. Nursing faculty
typically focus on the cognitive domain of Bloom’s taxonomy because this framework is
the basis of the formulation for the NCLEX-RN test plan (NCSBN, 2011). NCLEX-RN
predictor test plans must be congruent with Bloom’s taxonomy or they risk a decrease in
compatibility with the NCLEX-RN.
Although the original 1956 publication of Bloom’s taxonomy was not research
based, the 1999 revision was evidence based. Research conducted after the 2001
publication further validated the concepts. Nasstrom (2009) conducted a study in which
the findings indicated Bloom’s revised taxonomy is an acceptable tool for understanding
mathematical concepts. Bumen (2007) conducted a study that involved comparing the
13
initial 1956 taxonomy to the revision. Bumen found the revised taxonomy was superior
to the initial taxonomy when applied to lesson planning skills.
Bloom’s taxonomy is an excellent guide for teaching students about the nursing
process (Duan, 2006). This is particularly true for the taxonomy’s cognitive domain.
Nursing faculty often use these concepts to develop nursing program and course
objectives. The levels of the taxonomy such as knowledge and comprehension are
appropriate for beginning nursing students, and instructors gradually integrate the higher
concepts such as application and analysis into instruction (Duan, 2006). For test
questions, instructors should be aware to which cognitive level the question relates. Test
developers must weave the elements of the taxonomy into NCLEX-RN predictor test
plans, as the pattern of the actual NCLEX-RN aligns with Bloom’s taxonomy.
Definition of Terms
Some of the terminology in the study was specific to undergraduate nursing
education. Such terms are not common knowledge to society. This section includes a
discussion of the terms.
Bachelor of science in nursing (BSN) program: A BSN program is a 4-year
higher education program that prepares students to become licensed registered nurses
(Chitty & Black, 2011). Graduates are eligible to sit for the NCLEX-RN. The sample in
the current quantitative study included graduates from a BSN program.
ATI RN Comprehensive Predictor: Students typically take this test toward the
end of the nursing program (ATI, 2011). This examination is computerized and has 180
items. Test developers at ATI developed this examination. The quantitative study
included the ATI RN Comprehensive Predictor as a predictor variable.
14
National Council Licensure Examination for Registered Nurses (NCLEXRN): The NCLEX-RN is a computer adaptive test (NCSBN, 2011). The computer-based
examination assesses for minimum competency levels for candidates seeking to become
registered nurses in the United States. Test developers at the NCSBN strive to create a
test that will ensure safe nursing practice.
California Board of Registered Nursing (BRN): The California Board of
Registered Nursing (BRN, 2011) is the government organization that approves and
monitors nursing programs in the state of California.
Test of Essential Academic Skills (TEAS): The TEAS is a standardized
examination typically given either prior to admission to nursing school or at the
beginning of a nursing program (ATI, 2011). The examination has 170 questions,
including 55 English and language use items, 30 science items, 45 mathematics items,
and 40 reading items.
Assumptions
Assumptions are present in all research studies (Neuman, 2003). Identifying the
assumptions in a study will assist the researcher to better comprehend the various
concepts in the study (Neuman, 2003). Several assumptions existed in the proposed
quantitative study. The first assumption was that the data collection would be accurate.
The rationale for this assumption included the researcher following protocol for
retrospective data collection. The second assumption was the execution of the data
analysis would be correct. The rationale for this assumption included the researcher
performing an accurate data analysis. The third assumption was that the administration
of the ATI TEAS test, the ATI RN Comprehensive Predictor, and the Kaplan Readiness
15
Test would be appropriate to the nursing students. The fourth assumption was that the
nursing students in the study would follow the same or a similar curriculum during
nursing school.
Assumptions about the research design also existed. One assumption was no
unknown or unacknowledged variables influenced the study. An example of this was the
assumption that all students from the nursing program studied had fully met the
admission criteria required by the institution. Another assumption was the admission
criteria from the nursing program were similar to other programs in the state of
California.
Scope, Limitations, and Delimitations
The scope of the study included graduate nurses from a prelicensure nursing
program in Southern California. The population included program graduates from the
past 10 cohorts. These include seven cohorts from the traditional baccalaureate program
and three cohorts from the entry level master’s program. The data collected were the
archived results of the students’ TEAS test, ATI RN Comprehensive Predictor, Kaplan
Readiness Test, and NCLEX-RN. Data collected were from the graduating cohorts
between 2006 and 2012. Although NCLEX-RN scores exist, the study involved
collecting only pass–fail data. The dependent variable was the outcome of the NCLEXRN.
The sample was a limitation because the nursing students all came from the same
school. The program was relatively new as it began in 2006; therefore, the data gathered
were from 10 graduating cohorts of students. Also, the sample was a convenience
sample. A convenience sample is a group of individuals chosen at the discretion of the
16
researcher (Burns & Grove, 2009) and does not involve an attempt to capture an accurate
representation of the population. Selecting the sample did not involve randomization,
which is the assignment of study participants by chance, rather than at the discretion of
the researcher (Burns & Grove, 2009). The scores of students with previous health care
careers remained part of the sample. The data collected were for the total population of
graduated cohorts.
Delimitations are the defined boundaries of a study (Burns & Grove, 2009). The
data collection tool used in the study was one such delimitation. The geographical
location of the study was also a delimitation (Burns & Grove, 2009). The study took
place at one baccalaureate nursing school in Southern California.
Summary
The nursing shortage in the United States is severe and worsening. By 2020, the
national shortage could increase to more than 1 million full-time equivalent registered
nurses (Health Resources and Services Administration, 2010). NCLEX-RN predictor
examinations forecast both success and failure on the board examination. Some nursing
schools mandate that students pass predictor examinations to progress or graduate. If the
predictor test is not accurate, then it should be a determination of competence for nursing
students. The purpose of the quantitative retrospective study was to compare the results
of NCLEX-RN predictor tests to actual NCLEX-RN results on 10 cohorts of
undergraduate nursing students at a Southern California university. The dependent
variable was the NCLEX-RN results. The independent variables were the predictor test
results.
17
Chapter 2 will include a literature review on NCLEX predictor research. Also
included will be discussion on the study’s chosen theoretical framework, Bloom’s
taxonomy. The chapter also contains a discussion on the history of nursing shortages in
the United States, as well as the current nursing shortage.
18
Chapter 2
Review of the Literature
The purpose of the retrospective, quantitative study was to determine the extent to
which the ATI TEAS test, the ATI RN Comprehensive Predictor, and the Kaplan
Readiness Test could accurately forecast both success and failure on the NCLEX-RN.
The proposed study included one baccalaureate degree and entry-level master’s degree
nursing program in Southern California. A program requirement is that students achieve
a set passing score on the TEAS prior to admission to the program. Students must also
pass both the ATI RN Comprehensive Predictor and the Kaplan Readiness Test to
graduate and take the NCLEX-RN.
A high first-time student pass rate on the NCLEX-RN is crucial to validate a
successful nursing program (BRN, 2011). The BRN (2011) mandates that schools
maintain at least a 75% first-time pass rate. Approximately one third of nursing schools
in the United States use NCLEX-RN predictor tests to determine whether students can
graduate from their programs (NLN, 2012). Exit examinations, including the ATI RN
Comprehensive Predictor and the Kaplan Readiness Test, primarily predict readiness for
the NCLEX-RN and do not predict failure on the test (Spurlock & Hunt, 2008). Many
schools use the exit exam as a high-stakes test. High-stakes tests such as NCLEX-RN
predictor examinations require students to pass before they can receive approval to take
the NCLEX-RN (Spurlock & Hunt, 2008).
The focus of this chapter is a review of peer-reviewed publications in which
researchers have addressed the nursing shortage and NCLEX predictor examinations.
The literature review consists of six sections. The first section includes a review of
19
nursing shortages throughout the history of the United States and the societal need for
nurses. The second section includes a discussion on the history of nursing licensure
examinations in the United States. The third section contains a description of the concept
of predictive validity and recent studies that include a measure of the predictive validity
of an examination. The fourth section will contain a review of historic and recent
research from peer-reviewed publications pertaining to NCLEX predictor tests. The fifth
section will include a discussion of gaps in the literature. The sixth section will consist of
a discussion on leadership.
Title Searches, Articles, Research Documents, and Journals
An electronic literature search using EBSCOhost provided articles published after
2007. The search included the following keywords: NCLEX, HESI, ATI, Kaplan,
nursing, student, shortage, and Bloom’s taxonomy. The search yielded 70 referenced
articles and documents. Of the total references in the study, the dates for 85% were after
2007.
Bloom’s Taxonomy and Revision
One of the chief directives of the NCSBN is for nursing students to develop
didactic and clinical competency (NCSBN, 2011). Since the 1950s, Bloom’s taxonomy
has functioned as a tool for nursing faculty to establish the appropriate cognitive level to
which a student must perform. Bloom, Englehart, Furst, Hill, and Krathwohl (1956)
initially developed the taxonomy to categorize learning outcomes. Nursing faculty
typically focus on the cognitive domain of Bloom’s taxonomy because the NCLEX-RN
test plan is formulated on this framework (NCSBN, 2011). To maximize the predictive
20
ability of an NCLEX-RN predictor test, Bloom’s taxonomy is necessary to assist in
formulating its test plan.
In 1999, Anderson and Krathwohl (2001) revised the taxonomy to include the
following levels: remember, understand, apply, analyze, evaluate, and create. In the
cognitive domain, three of the six levels changed. The levels are no longer hierarchical
in nature (Ari, 2011). Nursing faculty expect that students will gradually master each
level, beginning with remembering (Duan, 2006). Remembering involves the ability to
recall knowledge from memory, such as with anatomical sites and names of medications.
The next level is understanding, wherein the learner can interpret meaning from
information. If a patient has chest pain, the student with understanding can state that the
patient possibly had a heart attack. Applying is the next level. A nursing student who
masters this level can successfully write a care plan. If a patient complains of
constipation, the student can assess for signs and symptoms of constipation, diagnose the
patient, set a goal to restore normal bowel function, and implement interventions to
relieve constipation. Analyzing is the next level. A student is able to separate a concept
into individual parts and learn how the parts relate to each other. A student is also able to
determine the overall purpose of these parts. Evaluation is the next level, wherein a
student is able to make decisions and determine if the intervention is effective or
ineffective. The final level is creating. A student is able to put the pieces together and
create a logical structure, discuss a patient’s plan with care with the patient’s family,
perform discharge planning, and produce discharge teaching instructions for the patient.
The NCSBN (2011) reported, “NCLEX-RN and NCLEX-PN examinations may
include items written at various cognitive levels. The majority of items are written at the
21
application or higher levels of cognitive ability” (p. 11). This section includes a
discussion on both the 1956 work by Bloom and the 1999 update. Because Bloom’s
taxonomy is the basis of the NCLEX-RN test plan, nurse educators should also consider
lining up Bloom’s instructional theory with their own curriculum (Duan, 2006).
NCLEX-RN predictor test developers must also base their test plan and test items on
Bloom’s taxonomy. Writing instructional outcomes using the keywords from Bloom’s
taxonomy, such as analyze and evaluate, will help students understand the level of
competency at which they must function to meet the objectives of the particular course
(Boland, 2009). This practice will create a congruency between the nursing school’s
curriculum and the NCLEX-RN test plan. Using Bloom’s taxonomy will also increase
similarity between NCLEX-RN predictor tests and the actual NCLEX-RN.
In addition to the six cognitive levels of Bloom’s taxonomy, the 1999 update
includes another four kinds of knowledge. The additions were factual, conceptual,
procedural, and metacognitive elements (Anderson & Krathwohl, 2001). The elements
allow instructors to engage students in activities that, while maintaining focus on the
cognitive domain, include affective and psychomotor aspects. Using this array of
elements in instruction will prepare students for the NCLEX-RN and for becoming safe,
effective, and caring nurses. In the clinical area, instructors must determine at what
cognitive level the student is functioning. Senior nursing students should demonstrate
evidence of thinking at the analysis, application, and evaluation levels (Duan, 2006).
Nursing Shortage
Several nursing shortages have occurred since the post-World War II era (1945present). In 1947, the Hill-Burton Act became law (Buerhaus, 1987). The law allowed a
22
large increase in the number of hospitals in the United States. The Hill-Burton Act
greatly increased the demand for nurses (Buerhaus, 1987) and significantly influenced the
modern health care system, especially regarding in-hospital patient care.
The U.S. government has taken various steps to alleviate the nursing shortages
throughout history. In 1964, President Johnson signed the Nurse Training Act into law
(Yett, 1975). The law enabled nurses to receive hospital-managed loans by which
nursing students would receive funding for school. The nurses would pay back the loans
interest-free over 10 years after completing nursing school, as long as they worked fulltime as a registered nurse. The intent for the law was to entice potential nursing students
to enter school who otherwise would not be able to afford nursing school.
Buerhaus (1991) analyzed hospital wage increases during times of nursing
shortage compared to times of adequate staffing. The intent for the increases was to
entice licensed nurses who were not working. Hospitals offered wage increases during
times of shortage. Hospitals that could not afford such wage increases hired lower cost
employees, such as licensed practical nurses and nurses’ aides.
The current nursing shortage began in 2000 (Chitty & Black, 2011). Several
factors make the current shortage unique compared to previous crises. There are 77
million baby boomers compared to 44 million people in the next generation following,
known commonly as Generation X (Americans born between 1965 and 1985; AACN,
2011). Increasing health care resources will be necessary for baby boomers as they age.
Typically, as a person ages, they will require more health care needs, including
hospitalizations (Buerhaus et al., 2009). In 2006, there were 37 million people over 65 in
23
the United States (Federal Interagency Forum on Aging Related Statistics, 2009). This
number will more than double by the year 2030.
Another unique issue of the current nursing shortage is that the number of nursing
schools in the United States is inadequate to supply the health care system with the
number of nurses needed (Buerhaus et al., 2009). For the 2008-2009 school year, 36,511
qualified students applied for admission to nursing programs in California, and the
programs did not accept 61.7% of the applicants (BRN, 2011). The statistics clearly
indicated the number of applicants far exceeded the seats available. The situation is
disturbing, considering the high demand for nurses in health care (Buerhaus et al., 2009).
Increasing numbers of nurses are leaving the profession compared to the past. The stress
and intensity of the work causes many nurses to leave their jobs (Chitty & Black, 2011).
Nurses also experience a number of physical ailments rendering them unable to work,
including back, knee, and wrist injuries (Chitty & Black, 2011). Technological advances
have also caused some nurses to leave their jobs because they do not feel comfortable
adjusting to changes such as computerized documentation (Chitty & Black, 2011).
Technologies include computerized documentation, electronic medication storage, and
barcode medication administration. Rapid advances are challenging for some older
nurses (Chitty & Black, 2011).
Several solutions to alleviate the current nursing shortage exist. One such
solution was the implementation of nurse–patient ratios to reduce work-related stress and
improve patient care (Tevington, 2011). The nurse–patient ratio had the opposite effect
to some degree, as a decreased nurse–patient ratio requires more nurses for each hospital
unit (Tevington, 2011). Another solution for nursing shortages has been for hospitals to
24
recruit nurses from foreign countries, although the cost of recruiting foreign nurses is
high and the result is not always positive (Ross, Polsky, & Sochalski, 2005).
Societal Need
The largest contributing factor to the nursing shortage over the next 20 years will
be the baby boomer population in the United States (AACN, 2011). As this population
ages, it will become a large obligation for the American health care system that will
require a greater number of registered nurses in the health care workforce. Schools of
nursing must ensure they will graduate the largest number of qualified students possible
(Giddens, 2009).
In addition to aging patients, registered nurses are also aging. In 2007, the
average age of registered nurses in the United States was 43.7 years old (AACN, 2011).
By the year 2025, many nurses from the baby boomer generation will retire. These
nurses need replacing by greater numbers of nurses from succeeding generations. By
2025, the United States might experience a nursing shortage of 260,000 nurses, which is
double the number of nurses needed in 2007 (AACN, 2011). Adequate nursing staffing
is essential to patient safety.
Kane, Shamliyan, Mueller, Duval, and Wilt (2007) performed a meta-analysis of
28 studies that focused on the relationship between registered nurse staffing and patient
deaths and complications. Kane et al. found when registered nurses are present on patient
units, length of stay on intensive care units and postsurgical units was significantly less.
To address the concerns of increased nursing demand and an aging workforce, nursing
schools must have larger numbers of nursing students graduate (Roa et al., 2011).
Nursing programs should not have attrition policies that fail potentially capable students.
25
Through increased support, counseling, guidance, and remediation, nursing faculty can
guide greater numbers of nursing students to graduate nursing school and pass the
NCLEX-RN the first time (Uyehara et al., 2007).
History of Nursing Licensure Examinations
Before 1944, every state had its own nursing licensure examination (MatassarinJacobs, 1989). Beginning in 1944, nursing graduates sat for the State Board Test Pool
Examination developed by the NLN. During this time, each state maintained its own
pass rate, but this become problematic when nurses moved from one state to another
(Birnbach, 1982). By 1978, the NCSBN began to oversee the State Board Test Pool
Examination, redesigned the test to include criterion referencing, and set a standardized
passing score at 1600. The NCSBN renamed the examination to the NCLEX-RN. In
1988, the examination changed to a pass–fail format (Matassarin-Jacobs, 1989).
NCLEX-RN
To work as a registered nurse in the United States, each nursing school graduate
must pass the NCLEX-RN (NCSBN, 2011). The intent for the NCLEX-RN was to
ensure nurses in the United States can practice safe patient care (NCSBN, 2011). The
examination initially began as a paper and pencil examination. Since 1994, the NCLEXRN has been a computerized adaptive test, which is a test that adapts to the ability level
of the examinee (NCSBN, 2011).
Every 3 years, the NCSBN reexamines the rigor of the NCLEX-RN. Since 1994,
the NCSBN has modified the NCLEX-RN four times, based on a practice analysis
survey. Aucoin and Treas (2005) studied the practice analysis survey, as the results of
the survey lead to modifications on the NCLEX-RN Test Plan. The survey has a return
26
rate of approximately 30%. Aucoin and Treas found the practice analysis is a useful tool
for nurse educators, as it can make clear the necessary changes to nursing education. The
authors recommend that nursing educators adjust curricula to include more information
on topics such as advanced cardiac life support and electrocardiograms (Aucoin & Treas,
2005).
The NCLEX-RN passing standard has increased in recent years. The NCSBN
assesses passing candidates’ ability to perform safely in health care settings and uses
surveys to educators and employers to determine the passing standard (NCSBN, 2011),
which could potentially place more graduate students at risk for failure. Carrick (2011)
used two theories to identify interventions for the NCLEX-RN candidates: systems
theory and the student’s approach to learning theory. Although nursing instructors might
feel the need to increase the difficulty of questions on their class examinations, it is
important to first assess the students’ learning needs. Although assessment testing is
common, research to improve NCLEX-RN prediction is inadequate. Instructors must
acknowledge that personal or situational issues might influence students’ test-taking
abilities (Spurlock & Hunt, 2008). In addition to assessment testing, instructors must use
a variety of methods to assist students at risk for failure.
Quality of Nursing Programs
State boards of nursing often measure the quality of a nursing program by its
NCLEX-RN pass rate. The National League for Nursing Accrediting Commission and
the Commission on Collegiate Nursing Education cite the NCLEX-RN as primary
evidence for achieving program outcomes (Morrison, 2005). Some have questioned the
wisdom of only counting the NCLEX-RN first-time pass rate in the integrity of the
27
nursing program. This practice has led to the implementation of strict progression
policies in some nursing programs (Giddens, 2009), including the requirement of passing
standardized examination that predict success on the NCLEX-RN prior to progressing to
the next semester of classes. Giddens (2009) noted this practice is potentially unfair and
could be an unethical policy. She recommended boards of nursing and accrediting bodies
consider the possibility of including the second NCLEX-RN attempt when judging the
integrity of a nursing program. Morrison (2005) noted the developers of the NCLEX-RN
did not design the test as an evaluation tool for the quality of a nursing program or its
curriculum.
Predictive Validity
Predictive validity is the value of a test that determines a person’s achievement on
a future measure (Polit & Beck, 2008). The study involved measuring the predictive
validity of the ATI TEAS, the ATI RN Comprehensive Predictor, and the Kaplan
Readiness Test. The proposed study will focus on the aforementioned examinations’
ability to predict success and failure on the NCLEX-RN.
Ali, Zaman, and Alamgir (2011) measured the predictive validity of admissions
tests for dental schools in Pakistan and found significance between the admissions test
and the academic achievement of the dental students. Students with higher scores on the
admissions test had better grades and a lower attrition rate than students with lower
scores. Ali et al. advocated for the inclusion of additional variables for admissions
consideration, including live interviews. Additional admissions criteria would increase
the likelihood of admitting the best students for each program (Ali et al., 2011).
28
Pishghadam and Khosropanah (2011) measured the predictive validity of the
English Language Teacher Competency Test. The test assessed examinees’ ability to
read, write, and speak the English language. English teachers in Iran took the test. The
test scores were compared to the final examination scores of each teacher’s students. The
statistical analysis revealed a significant relationship between the English Language
Teacher Competency Test and the students’ final examination scores.
High-Stakes Testing
Tests have been a vital part of human society for thousands of years (Madaus &
Russell, 2009). A high-stakes test is an examination in which the outcome holds great
significance for the examinee. In the United States, 12% of nursing schools, including
the school in the current study, require students to pass an NCLEX-RN predictor test
before allowing them to take the NCLEX-RN (NLN, 2012; Spurlock & Hunt, 2008).
Twenty percent of schools have policies that will not allow students to graduate and
receive their nursing degree unless they pass the predictor test (NLN, 2012). About one
third of nursing schools in the United States require registered nursing students to achieve
a minimum score on a predictor test to progress in the nursing program (NLN, 2012).
The NCLEX-RN predictor examinations are not the only high-stakes tests.
Exams such as the SAT and the Graduate Record Examination are also high-stakes tests
(Duncan & Stevens, 2011). The SAT is a part of the admission requirements for most
undergraduate universities in the United States, and the Graduate Record Examination is
an admission requirement for most U.S. graduate schools (Duncan & Stevens, 2011).
Duncan and Stevens (2011) surveyed public school teachers in Texas about their attitudes
toward high-stakes testing. Of the respondents, 92% believed that high-stakes tests
29
forced them to prioritize tested subjects over nontested material. Seventy-five percent of
the respondents believed the high-stakes tests did not measure instructional effectiveness
accurately (Duncan & Stevens, 2011). Duncan and Stevens (2011) believed standardized
testing should continue but teachers should not have to limit the curriculum to material
that appears on the standardized examination.
In the United States, tests often become a remedy to solve educational as well as
societal dilemmas (Madaus & Russell, 2009). High-stakes testing promotes American
values, including competitiveness, accountability, and standardization. However, highstakes tests often compel teachers to convert lectures into test preparation sessions
(Madaus & Russell, 2009). Creators of predictor tests do not take random circumstances
into account when determining test design and scoring. Such circumstances include
student illness, or movement and noises occurring within the testing environment.
In February 2012, the NLN Board of Governors published The Fair Testing
Imperative in Nursing Education on their website. Leaders at the NLN became aware of
the pressure schools of nursing feel to generate high first-time NCLEX-RN pass rates
(NLN, 2012). An association exists between a high first-time pass rate and the need to
ensure safe nursing practice for the public. The NLN’s primary concern is the use of
standardized predictor tests to prevent nursing students from taking the NCLEX-RN.
Because of this concern, leaders at the NLN developed national fair testing guidelines to
ensure educators and administrators follow evidence-based practices for progression and
graduation.
The first guideline addresses the ethics of high stakes testing. The NLN leaders
implore nurse educators to use predictor tests that are well-researched and validated
30
across each nursing course. They must also ensure the tests lack age, gender, ethnic,
religious, sexual, or regional bias (NLN, 2012).
Faculty members must recognize they have the primary responsibility to ensure
their students can practice nursing competently, not pass the standardized test (NLN,
2012). Also, the NLN leaders recommend multiple means for assessing competence and
knowledge. Nursing faculty must use test results to evaluate the curriculum continually
and as a guide to improve instruction, not just as a tool to measure student achievement.
Faculty members must review all information regarding the comprehensive test prior to
implementing policies on high stakes testing. Students must be fully aware of the
purpose of the test and the consequences of not passing a high-stakes test (NLN, 2012).
Test Anxiety
Test anxiety is a prevalent issue among college students (Neuderth, Jabs, &
Schmidtke, 2009). Low academic performance and psychological problems often
accompany the phenomenon. High-stakes tests often generate feelings of severe anxiety
for examinees. It is not the test that creates this anxiety but the stakes linked with the test
scores (Madaus & Russell, 2009).
Computerized Testing
Computerized testing has increased exponentially since the 1990s (Madaus &
Russell, 2009). By the 21st century, standardized tests had combined with advanced
technology (Madaus & Russell, 2009). Many standardized tests are computerized,
including the predictor tests measured in the current study and the NCLEX-RN (ATI,
2011; Kaplan, 2011; NCSBN, 2011). The next section will include a review of the
existing literature on NCLEX predictors.
31
Costs of NCLEX-RN Failure
A recent update of the NCLEX-RN occurred in April 2010. The result stemmed
from a national practice analysis survey on the current practices of new graduate nurses
(NCSBN, 2010). The survey included evaluations of 155 nursing bedside interventions.
Also, health care administrators expressed the need for greater competence from entrylevel nurses. Leaders at NCSBN updated the NCLEX-RN again in April 2013.
Roa et al. (2011) estimated the cost of NCLEX-RN failure for nursing school
graduates could be as much as $11,426. The cost includes the loss of salary for a
minimum period of the months in California and remediation costs (BRN, 2012). About
15,000 new graduate nurses in the United States fail the NCLEX-RN each year (NCSBN,
2009) on the first attempt.
Historical NCLEX Predictor Research
Deardorff, Denner, and Miller (1976) looked at 5 years of associate degree in
nursing (ADN) graduates at one school of nursing who graduated between 1969 and
1974. Deardorff et al. compared scores on the State Board Examination (SBE) with
scores on the NLN Achievement Tests using correlation analysis and multiple regression.
The tests consisted of examinations on each nursing subject. The Medical, Pediatric, and
Postpartum tests accurately predicted success on the SBE.
Washburn (1980) performed a study on one nursing diploma program. The
sample consisted of 166 graduates. She also found a significant correlation between the
SBE and specific NLN Achievement Tests: Pediatric, Psychiatric Nursing, and Nursing
Care of Patients I, II, and III. Bell and Martindill (1976) also tested the predictive
accuracy of the NLN Achievement Tests to predict success on the SBE by analyzing the
32
test results of 101 students from one BSN program in Houston, Texas. Bell and
Martindill found the Pediatric and Obstetric test scores were the best predictors of the
students’ outcome on the SBE.
Fowles (1992) compared the Mosby Assess Test results to NCLEX-RN results for
192 graduates of one BSN program. Using multiple regression analysis, Fowles (1992)
determined that the Mosby Assess Test accurately predicted NCLEX-RN success. She
recommended using the data to help at-risk students enter intervention programs.
Current NCLEX Predictor Research
Simon, McGinniss, and Krauss (2013) examined predictor variables for NCLEXRN results, NLN-readiness examination results, and nursing program success. The
researchers analyzed student records at one urban nursing school in the northeastern
United States. Initially, grade point average (GPA), biology grades, and chemistry grades
seemed to be predictive of success in the nursing program, but the researchers found the
content from the core nursing curriculum overrode these findings. The first nursing
course, called Nursing 1, was highly predictive of success on the NLN-readiness
examination. Simon et al. (2013) also found transfer students had higher success rates on
the NCLEX-RN than the traditional students. The transfer students tended to be older,
second-career adults.
Grossbach and Kuncel (2011) conducted a meta-analysis on 31 independent
samples with 7,159 study participants. The participants consisted of baccalaureate
nursing students. The focus of the meta-analysis was the ability of 13 different predictors
to forecast student performance on the NCLEX-RN. Effective predictors included
SAT/ACT results and baccalaureate nursing course GPA. The second year nursing GPA
33
predictor success was comparable to standardized NCLEX-RN predictor tests such as the
NLN and Mosby tests.
Bondmass, Moonie, and Kowalski (2008) compared nursing students’ scores on
the Nursing Entrance Test (NET) and ERI standardized examinations and their results on
the NCLEX-RN. The NET is a standardized nursing school admissions test, and the ERI
is a standardized NCLEX-RN predictor test. Because of a low first-time pass rate, the
leaders at one study school instituted the NET and ERI tests in the hopes of increasing the
pass rates and having more graduate students achieve their goal of becoming registered
nurses. The results revealed an increase in NCLEX-RN pass rates after introducing the
standardized testing program. Bondmass et al. found the NET and ERI scores
significantly correlated to NCLEX-RN success, whereas the students’ GPAs did not.
Bondmass et al. did not examine the ability of the NET and ERI to predict failure on the
NCLEX-RN, but focused on the probability statistics for success on the NCLEX-RN.
Leaders at many nursing schools in the United States use NCLEX-RN predictor
tests to determine whether students can graduate from their programs. Spurlock and Hunt
(2008) performed a study in which they demonstrated that one school’s HESI Exit Exam
results did not accurately predict success on the NCLEX-RN. If the test cannot reliably
predict both success and failure on the NCLEX-RN, it should not be a means of
determining success in nursing school (Spurlock & Hunt, 2008).
Nibert, Young, and Adamson (2006) performed multiple studies measuring the
ability of the HESI Exit Exam to accurately predict success on the NCLEX-RN. The
HESI Exit Exam is a standardized NCLEX-RN predictor test produced by Elsevier.
Nibert et al. disclosed the predictive accuracy of the HESI Exit Exam to determine
34
passing the NCLEX, but not the accuracy of the exam to determine failure. Spurlock and
Hanks (2004) noted the positive predictive value of the HESI Exit Exam is only 19%,
which means approximately 81% of the students predicted to fail actually passed the
boards.
McGahee (2010) sought to determine the factors that contributed to success on the
NCLEX-RN. The studied variables were SAT or ACT scores, prerequisite nursing
course grades, nursing course grades, and standardized assessment tests. McGahee
examined two cohorts of students from one nursing school. The science course grades
and the RN Assessment test score positively related to success on the NCLEX-RN.
Morris and Hancock (2008) performed a study to determine if the HESI Exit
Exam helped improve outcomes at a new nursing program. The study had two research
questions: (a) How did the new curriculum impact learning as measured by the HESI Exit
Exam and (b) Is there a relationship between the HESI Exit Exam score and the first-time
pass rate on the NCLEX-RN? Morris and Hancock examined data from two cohorts of
students. The data included the results of the HESI Exit Exam and students’ NCLEX-RN
results. A relationship did exist between HESI Exit Exam results and NCLEX-RN
results. All students who scored 900 or higher on the HESI Exit Exam also passed the
NCLEX-RN.
Ukpabi (2008) performed a study to identify the variables attributed to success on
the NCLEX-RN at a baccalaureate nursing program. The researcher looked at one cohort
and examined 18 variables, including ATI tests and several tests created by the NLN.
The other variable was the students’ GPA. The ATI and NLN tests were the strongest
35
predictors of success on the NCLEX-RN. The tests can assist leaders in schools of
nursing to predict student success and improve quality in their respective programs.
Alameda et al. (2011) measured the relationship between the NCLEX-RN and the
ATI RN Comprehensive Predictor. The researchers examined two different versions of
the examination: Version 3.0 and Form A and B. Form A and B are two versions that
intend to predict results from the 2007 NCLEX-RN test plan, and the basis of Version 3.0
is the 2007 test plan. Alameda et al. analyzed the test results from 589 students,
including 367 students for Version 3.0 and 222 students for Form A and B. On Version
3.0, the test result accounted for 3% of the course grade. On Form A and B, the test
result increased to 10% of the course grade. Alameda et al. discovered the Form A and B
group averaged a higher score on their exam than the Version 3.0 group. The researchers
attributed this to the higher stakes of the Form A and B test.
Harding et al. (2011) measured the ability of the HESI midcurricular examination
to predict nursing school success, success on the HESI Exit Exam, and success on the
NCLEX-RN. Harding et al. used a retrospective descriptive correlational design to
examine the data and found a positive correlation between the midcurricular exam and
the exit exam. The midcurricular exam had a predictive value of 95% for passing the
NCLEX-RN. These findings might help guide the remediation policy for nursing schools
concerning the HESI midcurricular exam.
Tipton et al. (2008) studied variables that possibly predicted success in the
nursing program and on the NCLEX-RN. The study involved examining entrance test
scores, nursing course grades, and GPA for five cohorts of students from one nursing
school. The students who had higher nursing course grades were more successful on the
36
NCLEX-RN than those students with lower nursing course grades. The entrance test
scores and GPA had no association with student success.
Abbott (2008) examined predictors of NCLEX-RN success for nursing students in
an accelerated bachelor degree in science program. Abbott looked at preprogram degree,
course grades, and the results of the HESI Exit Exam and compared these variables to the
students’ NCLEX-RN results. The students who had preprogram science degrees
performed better in program courses and on the NCLEX-RN than the other students. The
students who performed well on the HESI were more likely to pass the NCLEX-RN.
Adamson and Britt (2009) and their associates published several studies regarding
the accuracy of the HESI Exit Exam. In previous studies, the researchers focused on the
accuracy of the first HESI Exit Exam to predict NCLEX-RN success, but did not
examine the ability of subsequent exit exam attempts to predict NCLEX-RN success.
Adamson and Britt examined the accuracy of the first, second, and third attempts. The
first attempt was 96.44% accurate and the second attempt was 92.94% accurate.
However, the third attempt was only 82.5% accurate. Adamson and Britt concluded that
students who required three attempts to pass the HESI had greater risk for failing the
NCLEX-RN than those who passed following the first or second attempt.
Murray, Merriman, and Adamson (2008) measured the ability of the HESI
Admission Assessment Test (A2) to predict nursing school success. Murray et al.
examined A2 results and course grades for 286 students from two different nursing
programs and found that the A2 reliably predicted student success in both the associate’s
degree in nursing and the bachelor of science in nursing programs studied. The A2 was a
37
more reliable predictor than the students’ preprogram GPA. The A2 can be a preprogram
determinant of admission for prospective nursing students.
Lavandera et al. (2011) sought to determine the effectiveness of the HESI tests to
predict first-attempt nursing licensure. The researchers compared HESI test results to
NCLEX-RN results at one private university in the southeastern United States. They
excluded those students who took the NCLEX-RN more than 140 days after graduation,
as the lag time can create an increased chance for failure. Using logistic regression to
determine predictive accuracy, the researchers found that the HESI Exit Exam accurately
predicted success on the NCLEX-RN. Nurse educators can use the results of the HESI
Exit Exam to determine necessity for academic remediation prior to sitting for the
NCLEX-RN. Lavandera et al. acknowledged that the HESI Exit Exam is a modest
predictor of failure on the NCLEX-RN at best. They found that students who received a
single D or F on one nursing, science, or mathematics course were more likely to fail the
NCLEX-RN than those students who failed the HESI Exit Exam.
Zweighaft (2011) evaluated the effectiveness of the HESI specialty exams to
predict success on the HESI Exit Exam and the NCLEX-RN at 63 nursing programs
across the United States, consisting of six diploma programs, 31 ADN programs, and 26
BSN programs. Of the programs, 43 used both specialty exams and the exit exam, while
20 used only the exit exam. Zweighaft found that the schools that used specialty exams
scored significantly higher scores on the exit exam than the schools that did not use
specialty exams. Also, some of the HESI specialty exams strongly associated with
NCLEX-RN success, including the Medical/Surgical exam, the Fundamentals exam, the
Maternity exam, and the Critical Care exam (Zweighaft, 2011). Zweighaft recommended
38
using the results of the specialty exams to determine which students required remediation
to ensure the best possible readiness for the HESI Exit Exam and the NCLEX-RN.
Kleber (2010) compared exit exam results to National Council Licensure
Examination for Practical/Vocational Nurses (NCLEX-PN) results for 411 students from
14 practical nursing programs throughout the state of Kentucky. Using a logistic
regression model, Kleber found the first-attempt exit exam result was significant in
predicting success on the NCLEX-PN. Subsequent attempts on the exit exam were not
significant predictors. Kleber suggested using the exit exam results as a tool to assist atrisk students. These students should receive resources and interventions to increase
chances of success on the licensure examination.
Benefiel (2011) examined five nursing program and 11 preadmission variables at
two Central California community colleges to determine which variables significantly
predicted results on the NCLEX-RN. These variables included TEAS, ATI
Comprehensive Predictor, and NCLEX-RN results. Benefiel (2011) analyzed the data
using logistic regression for four different models. The first model included the
traditional and contract students, as well as the overall TEAS score. The model had
92.3% accuracy. The second model was the same as the first, except using the TEAS
subject test scores instead of the overall score. This model had 91.7% accuracy. The
focus of the third model was the LVN to RN students minus the TEAS variables. This
model had 100% accuracy. The fourth model included all students minus the TEAS
variables. This model had 90.2% accuracy. The variables did not significantly predict
NCLEX-RN failure, with Model 4 having the highest predictive accuracy at 47.1%
(Benefiel, 2011).
39
The Fair Testing Imperative in Nursing Education
In February 2012, the NLN Board of Governors published The Fair Testing
Imperative in Nursing Education on its website. The NLN became aware of the pressure
schools of nursing feel to generate high first-time NCLEX-RN pass rates (NLN, 2012).
A high first-time pass rate is associated with the need to ensure safe nursing practice for
the public. The NLN’s primary concern is the use of standardized predictor tests to
prevent nursing students from taking the NCLEX-RN. Because of this concern, the NLN
developed national fair testing guidelines to ensure educators and administrators follow
evidence-based practices for progression and graduation.
The guidelines address the ethics of high stakes testing. The NLN implores nurse
educators to use predictor tests that are well-researched and validated across each nursing
course. They must also ensure the tests are free of age, gender, ethnic, religious, sexual,
or regional bias (NLN, 2012).
Beyond the NCLEX-RN
Valiga and Ironside (2012) were critical of the current rigor in educational nursing
research. A gap exists in the literature that measures outcomes between prelicensure
nursing programs and postlicensure practice settings. Measurement of competence
should go beyond the NCLEX-RN and NCLEX-RN predictor tests. Valiga and Ironside
called for a national agenda focusing on nursing education research. More
comprehensive research on postlicensure competence will better assist nursing faculty to
ensure prelicensure education will lead to postlicensure safe and effective patient care.
40
Attitudinal Change
Carr (2011) acknowledged her students did not believe their university’s first-time
NCLEX-RN pass rate was a priority for them. Students tended to be satisfied with
graduating nursing school or postponed sitting for the NCLEX-RN even though the
chances of passing decrease the longer a student waits (Eich & O’Neill, 2007). However,
in 2002, the Long Island University School of Nursing’s first-time NCLEX-RN pass rate
averaged only 73.2%. The school had to raise the pass rate or face loss of accreditation
from the New York State Education Department. The education department required
annual reports and a strategic plan to demonstrate how the school would improve the pass
rate. School leaders implemented a standardized exit examination and subject tests, but
the exams did not improve the pass rate. When the school began to require students to
pass the exit exam before graduation, and the pass rate increased to 93%. However,
because the New York State Education Department felt a single criterion for graduation
was not acceptable. The policy changed so that the exit exam was only part of the
graduation requirements. Carr (2011) believed educating students about the importance
of taking the NCLEX-RN early made a difference in their scores. Despite the school’s
stoppage of high-stakes testing, the school’s pass rate remained high in 2011.
Gaps in the Literature
An examination of the extent to which the ATI TEAS test, the ATI RN
Comprehensive Predictor, and the Kaplan Readiness Test can forecast both success and
failure on the NCLEX-RN was the focus of the conducted study. A gap in the literature
existed in the exploration of the ability of NCLEX predictor tests to forecast failure on
the NCLEX-RN. Many researchers have examined the ability of NCLEX predictor tests
41
to forecast success on the NCLEX-RN, but few have focused on their ability to predict
failure on the NCLEX-RN. Few researchers have examined the predictive validity of the
Kaplan Readiness Test.
Many schools use NCLEX-RN predictor tests to determine if students should pass
nursing school, so a failure on the predictor test should have a strong relationship with a
failure on the NCLEX-RN (Spurlock & Hunt, 2008). The study involved determining the
ability of the ATI TEAS test, the ATI RN Comprehensive Predictor, and the Kaplan
Readiness Test to predict success and failure on the NCLEX-RN at one location.
Summary
Several nursing shortages have occurred in the modern American hospital era, but
the United States is approaching a shortage unlike any previous one. The baby boomer
generation is aging, and every baby boomer will be 65 or older by 2030. Nursing schools
should aspire to achieve a high first-time NCLEX-RN pass rate. Tests that accurately
predict both success and failure on the NCLEX-RN would help nursing students and
faculty achieve high first-time NCLEX-RN pass rates.
Leaders in the U.S. government, hospitals, and schools of nursing are
continuously involved in a search for solutions to the national nursing shortage.
Increasing nursing school graduation rates is one such solution (Uyehara et al., 2007).
Decreased attrition will also assist in alleviating the nursing shortage.
Chapter 3 will contain details of the design of the study, the appropriateness of the
study design, and how the data collection occurred for the retrospective, quantitative
study. Chapter 3 will include a description of the methodology selected to collect
quantitative data from archived data at one school of nursing in Southern California. The
42
study involved comparing the results of the students’ ATI TEAS, ATI RN
Comprehensive Predictor, and Kaplan Readiness Test scores to the first-time results of
the NCLEX-RN.
43
Chapter 3
Research Methods
The purpose of the retrospective quantitative study was to determine the degree of
accuracy that the ATI TEAS test, the ATI RN Comprehensive Predictor, and the Kaplan
Readiness Test can predict both success and failure on the NCLEX-RN at one nursing
program in Southern California. The study included three associated hypotheses. The
study is unique because the focus was on measuring the ability of two NCLEX-RN
predictor tests and one admission test to forecast both success and failure on the NCLEXRN. Although numerous studies exist that measure the accuracy of NCLEX predictor
tests, none include a focus on the ability of the tests to predict failure on the NCLEX-RN.
Given the fact that the leaders of some nursing programs use these predictor tests as a
graduation requirement, the gap in the literature should be addressed.
This chapter will include a discussion on the methodology selected to collect
quantitative data from archived data of the last 10 graduating cohorts of a nursing
program. The chapter also includes a discussion on the appropriateness of a
retrospective, quantitative design for the study, followed by a description of the design
and data collection. This chapter contains six sections: (a) research method, (b) research
questions, (c) study population, (d) confidentiality, (e) geographic description, and (f)
methodology of the data collection.
Research Method and Design Appropriateness
The quantitative design for the study was appropriate because the research
involved analyzing variables to gauge the strength of a relationship between them. The
retrospective approach was also appropriate for the study. Retrospective studies involve
44
examining a phenomenon that exists in the present that has a relationship with
phenomena from the past (Polit & Beck, 2008). Researchers examine an outcome from
the present and determine if a relationship exists with the past phenomenon. The study
involved examining the results of the NCLEX-RN and comparing them to the results of
the ATI TEAS test, the ATI RN Comprehensive Predictor, and the Kaplan Readiness
Test. Students took all these tests prior to the NCLEX-RN.
The data collection stage of the study included the cognitive portion of Bloom’s
taxonomy. The designers of the NCLEX-RN formulated the test from the cognitive
process categories of Bloom’s taxonomy. The NCLEX-RN test plan is the basis of both
the ATI RN Comprehensive Predictor and the Kaplan Readiness Test.
Population
The target population for the retrospective, quantitative study included graduates
of a baccalaureate degree nursing program in Southern California. The college has a
health sciences division that includes a nursing program. The program has full approval
by the California BRN and accreditation by the Commission on Collegiate Nursing
Education. Graduates of the program are eligible to sit for the NCLEX-RN after
successfully completing all classes and requirements. The requirements include passing
the ATI RN Comprehensive Predictor and the Kaplan Readiness Test. The demographics
of the nursing program are similar to those of other schools of nursing in the state of
California.
Sampling
The only source of data was archived records of the last 10 graduating cohorts of
the nursing program. Approval from the University of Phoenix’s IRB was necessary to
45
proceed with the study. A representative of the participating school signed a permission
to use premises form (see Appendix B) and a letter of collaboration to provide evidence
of cooperation with the study (see Appendix C).
The University of Phoenix IRB provided approval for this research prior to the
beginning of data collection. The researcher held and will continue to hold all collected
information in the strictest confidentiality. No names appeared on data collected,
computer files, or presentations. Three years after the completion of the study, the
researcher will shred all student data and erase all electronic files.
Geographic Location
The geographic location for the study was on the campus of a private university in
Southern California. This university has a baccalaureate nursing program that has been
in existence since 2006. During this time, students had to achieve a set score on the
TEAS examination to be admitted to the nursing program and had to pass both the ATI
RN Comprehensive Predictor and the Kaplan Readiness Test to graduate from the
program and take the NCLEX-RN.
Data Collection
Researchers accumulate research data for quantitative studies through an
organized procedure that includes details about the information they will collect and how
to collect it. Researchers usually cannot collect data from the entire population under
study, and therefore will study a representative sample of the group (Polit & Beck, 2008).
Collection of data consisted of an organized review of students’ archived records from
the past seven cohorts of baccalaureate graduates and the past three cohorts of entry-level
MSN graduates from the nursing program. The predictor variables included results from
46
the ATI TEAS test, results of the ATI RN Comprehensive Predictor, and results from the
Kaplan Readiness Test (see Appendix C). The study involved comparing the predictor
results to the actual NCLEX-RN result, which is a pass or fail result. Students take the
ATI TEAS test prior to admission, and the results are a part of the admission qualification
process. The results include reading, math, and English scores. Students take the ATI
RN Comprehensive Predictor and the Kaplan Readiness Test prior to graduation and
must achieve a minimum passing score on each test prior to sitting for the NCLEX-RN.
The University of Phoenix IRB provided permission to perform the study. In
addition, the dean of nursing signed a permission to use premises letter and a personal
letter of permission. The nursing program data technician created a code for each student
name to maintain compliance with student data privacy regulations. The nursing
program data technician was not part of the study and is an employee of the nursing
school. No student names or identifying data were a part of the data collected. All
nursing student data were deidentified, provided to the researcher, and analyzed. All data
will remain confidential, and all records will be secure for 3 years after completion of the
study. After the 3 years, the researcher will destroy all records.
Instrumentation
The instrument used for data collection was simple, organized, and systematic.
The instrument was a table created specifically for collecting data for the study. The only
data entered into the table were nominal, ordinal, and interval data. For each item in the
study, a clear definition exists (see Appendix A).
47
Validity and Reliability
Internal validity ensures the independent variable caused the outcome, rather than
other factors (Polit & Beck, 2008). Researchers must develop approaches that will
prevent other aspects outside a study causing the outcome. The study included measures
to make certain the interval validity is intact.
Selecting the entire population of graduates in the nursing program eliminates
selection bias, which is one of the greatest threats to internal validity (Polit & Beck,
2008). Mortality is another threat to internal validity, but was not an issue in the study
because of the retrospective nature of the study. All nursing graduates who took the
NCLEX-RN took the other predictor tests previously. The study did not provide a direct
benefit to any company, institution, or individual during the study period.
External validity involves the ability to apply the observed relationships in a study
to various settings and conditions (Polit & Beck, 2008). Evidence gained from a
controlled study with external validity could apply to real-world settings. The researcher
addressed this concept in the study.
The university that was the setting for the study had a curriculum that was fully
compliant with the guidelines set forth by the California BRN. The design of the study
did not require any changes to the nursing program environment. The curriculum at the
school is similar to other curricula throughout the state of California, so the results of the
study will be applicable to other schools in California with a similar nursing program
environment (BRN, 2011). Because the study included only archival data, the results
were generalizable to similar nursing schools in California, and because the study sample
48
did not involve human subjects, subject bias was not a consideration. The reactions of
subjects were not a threat to external validity in the study.
Data Analysis
The study involved a statistical analysis to determine the significance of research
findings. The analysis in the study served to determine if research findings are
significant by confirming either the hypothesis or the null hypothesis. The research
design serves to determine the type of analysis performed (Burns & Grove, 2009). The
study included SPSS 21 for all statistical analyses.
Research variables. The research variables included the ATI TEAS reading
score, the ATI TEAS math score, the ATI TEAS English score, the ATI Comprehensive
Predictor score, and the Kaplan Readiness Test score. The study included point biserial
correlation to determine the level of significance between the aforementioned variables
with the pass and fail rates on the first-time results of the NCLEX-RN test.
Logistic regression analysis. Logistic regression analysis helped to determine
the accuracy of predictor variables related to the actual results of the first-time NCLEXRN test. “Logistic regression analyzes the relationship between multiple independent
variables and a dependent variable and yields a predictive equation” (Polit & Beck, 2008,
p. 629). A logistic regression converts the probability of an event occurring into odds
between 0 and 1. The higher the number is, the greater is the probability of an event
occurring (Polit & Beck, 2008).
Summary
The quantitative study involved examining nursing students’ ATI TEAS test
scores, ATI RN Comprehensive Predictor scores, and Kaplan Readiness Test scores and
49
the ability to predict students’ success and failure on the NCLEX-RN. Chapter 3
contained a description of the study method, design, and data collection process. The
retrospective design was the quantitative method chosen. The study involved performing
a statistical analysis with the data collected from the school of nursing. Chapter 4 will
include the methodology implemented in this study. The chapter also includes a
discussion on the design, study, variables, data collection, statistical analyses, and
limitations.
50
Chapter 4
Results
A gap existed in the literature regarding the ability of the ATI RN Comprehensive
Predictor or the Kaplan NCLEX-RN Readiness Test to accurately forecast failure on the
NCLEX-RN. The purpose of the quantitative study was to add this information into the
literature. The independent variables included the baccalaureate and ATI TEAS scores,
the ATI RN Comprehensive Predictor scores, and the Kaplan NCLEX-RN Readiness
Test scores of the entry-level master’s students. The dependent variable was the BSN
and MSN students’ first-time results on the NCLEX-RN. The examination included
archived records from 251 BSN and MSN student records.
Chapter 4 will include a description of the screening procedures and data
collection of the study, a synopsis of the statistical analyses used to address the research
questions, and the results of the analysis in the quantitative study. The study involved
analyzing all the independent and dependent variables. Analysis results will then be
aligned with the hypotheses.
Research Questions
The research project included two questions. The questions were appropriate to
the study and highlighted the dependent and independent variables. The study involved
determining the extent to which two standardized nursing exams can predict both success
and failure on the NCLEX-RN at one nursing school in Southern California.
1. Are NCLEX-RN predictor examination scores (independent variable)
significantly related to results on the NCLEX-RN (dependent variable)?
51
2. Are nursing school admissions test scores (independent variable) significantly
related to results on the NCLEX-RN (dependent variable)?
The associate dean of the university under study granted permission to use the
premises and the name of the university (see Appendix A) as well as permission to use
the data (see Appendix B). Data analysis included SPSS 21. The level of statistical
significance was set at p = .05 to determine if a significant relationship existed between
the independent variables and first-time success or failure on the students’ first attempt
on the NCLEX-RN.
Data Analysis
The purpose of the quantitative study was to determine if the TEAS, the ATI RN
Comprehensive Predictor, and the Kaplan NCLEX-RN Predictive Exit Test could
accurately predict both success and failure on the NCLEX-RN at one nursing program in
Southern California. The data gathered for the study came from 251 students.
Table 1 shows the frequency counts for selected variables. The sample included
more female students (90.0%) than male students (10.0%). All but three (98.8%) had
U.S. citizenship. The most common racial and ethnic categories were Caucasian
(62.2%), Asian/Pacific Islander (15.5%), and Hispanic (14.7%). For the entire sample,
90.8% of the students passed the NCLEX on their first attempt (see Table 1).
Table 2 shows the descriptive statistics for selected variables. The statistics
included the student’s age (M = 27.73, SD = 5.26), the TEAS total score (M = 82.40, SD
= 4.59), and the ATI score (M = 71.59, SD = 6.76). The table also includes the seven
TEAS subscale scores (see Table 2).
52
Table 1
Frequency Counts for Selected Demographic Variables (N = 251)
Variables and category
Gender
Female
Male
Citizenship
Other
United States
Race/ethnicity
African American
Asian/Pacific Islander
Caucasian
Hispanic
Other
NCLEX outcome
Failed
Passed
n
%
226
25
90.0
10.0
3
248
1.2
98.8
11
39
156
37
8
4.4
15.5
62.2
14.7
3.2
23
228
9.2
90.8
Table 2
Descriptive Statistics for Selected Variables (N = 251)
Variable
M
SD
Low
High
Age
27.73
5.26
22.00
61.00
TEAS total score
82.40
4.59
69.30
93.50
Reading
91.25
5.93
54.80
100.00
Inferences
87.82
12.69
33.30
100.00
Math
78.10
9.91
44.40
100.00
Diagrams
77.93
15.83
33.30
100.00
Science
75.84
8.20
53.30
93.30
Science reasoning
68.50
20.26
20.00
100.00
English
83.43
5.71
63.30
100.00
a
ATI score
71.59
6.76
55.30
89.30
a
For this variable, n = 219 because school leaders did not administer it some semesters.
Table 3 shows the distribution of predicted and actual NCLEX scores and pass
rates for the students who took the 2007 ATI RN Comprehensive Predictor. For all but
one of the predicted score ranges, the students in the sample either had similar or better
pass rates than expected based on the predictions provided by ATI (2007; see Table 3).
53
Table 3
Distribution of Predicted and Actual NCLEX Scores and Pass Rates for 2007 Predictor
(n = 162)
RN Comprehensive Predictor
score range (%)a
≥82.0
79.3-81.3
76.0-78.7
74.0-75.3
72.0-73.3
70.7-71.3
68.7-70.0
67.3-68.0
65.3-66.7
62.0-64.7
56.7-61.3
0.0-56.0
Predicted probability of
passing the NCLEX (%)
99
98
96-97
94-95
91-93
89-90
85-88
82-84
76-80
63-73
38-60
1-36
Actual nb
10
19
17
8
27
14
12
7
10
11
10
1
Actual pass
rate (%)c
100.0
100.0
100.0
100.0
100.0
100.0
81.8
90.9
78.9
78.9
82.4
100.0
Table 4 displays the distribution of predicted and actual NCLEX scores and pass
rates for students who took the 2010 ATI Comprehensive Predictor. For all but one of
the predicted score ranges, the students in the present sample had either similar or better
pass rates than expected based on the predictions provided by ATI (2010; see Table 4).
Testing of the Hypotheses
Null Hypothesis 1 was as follows: The ATI RN Comprehensive Predictor test
score is not significantly related to results on the NCLEX-RN. Table 4 shows the chisquare tests used to test the relationship between the two variables. This table includes
the chi-square tests for the total sample (n = 219), the traditional subsample (n = 182),
and the accelerated subsample (n = 37). A significant relationship existed in both the
total sample (p = .001) and the traditional subsample (p = .001). In the accelerated
54
student subsample, all 37 students passed the NCLEX on the first attempt. This
combination of findings provided support to reject Null Hypothesis 1 (see Table 4).
Table 4
Distribution of Predicted and Actual NCLEX Scores and Pass Rates for 2010 Predictor
(n = 57)
RN Comprehensive Predictor Predicted probability of
Actual pass
score range (%)a
passing the NCLEX (%) Actual nb
rate (%)c
≥80.7
99
2
100.0
78.0-80
98
7
100.0
74.7-77.3
96-97
4
100.0
72.0-74.0
94-95
9
89.0
70.0-71.3
91-93
6
100.0
68.7-69.3
89-90
7
100.0
66.0-68.0
84-88
6
100.0
64.7-65.3
81-82
3
100.0
62.7-64.0
75-79
5
100.0
59.3-62.0
63-73
4
75.0
53.3-58.7
37-60
4
75.0
0.0-52.7
1-34
0
100.0
a
NCLEX predictions based on data supplied by ATI RN Comprehensive Predictor 2010
Proctored Assessment. bNumber of students in current study who passed the NCLEX but
had a predicted score in that specific range. cNCLEX pass rate for students with a
predicted score in that specific range.
Null Hypothesis 2 was as follows: The Kaplan NCLEX-RN Readiness Test score
is not significantly related to results on the NCLEX-RN. To examine this hypothesis, an
analysis of variance (ANOVA) test was suitable for the 100 students who took the
Kaplan NCLEX-RN Readiness Test. The ANOVA yielded no significant differences
between Readiness Test results and NCLEX-RN results (p < .05). These findings
provided support to fail to reject Null Hypothesis 2.
Null Hypothesis 3 was as follows: The ATI TEAS nursing school admissions test
score is not significantly related to results on the NCLEX-RN. Table 5 displays the t
55
tests for independent means comparing whether students passed the NCLEX on their first
attempt with the TEAS total score along with the seven TEAS subscale scores. Only the
TEAS Math score was significantly higher (p = .005) for students who passed the
NCLEX. This combination of findings provided limited support to reject Null
Hypothesis 3.
Table 5
t Test Comparisons Based on NCLEX Outcome for Selected Variables
Variable and outcome
n
M
SD
rpb
t
p
Age
.07 1.12 .27
Fail
23 26.57
3.13
Pass
228 27.85
5.42
TEAS Score
.10 1.60 .11
3.35
Fail
23 80.95
Pass
228 82.55
4.67
Reading
.02 0.29 .78
Fail
23 90.91
4.71
Pass
228 91.28
6.05
Inferences
.03 0.41 .68
Fail
23 88.85 12.37
Pass
228 87.72 12.74
Math
.18 2.86 .005
Fail
23 72.55
8.01
Pass
228 78.66
9.93
rpb = Point biserial correlations used as a measure of the strength of the relationship.
Also in Table 5 is the t test for independent means comparison of the students’
ATI score with whether they passed the NCLEX on their first attempt. Students who
passed had significantly higher ATI scores (p = .001). The table also includes the pointbiserial correlation (rpb) used as a measure of the strength of that relationship. The
correlation (rpb = .27, rpb2 = .073) found that the ATI score accounted for 7.3% of the
variance in whether the student passed the NCLEX on the first attempt.
56
In conclusion, the study included data from 251 students to attempt to determine
if the TEAS, the ATI RN Comprehensive Predictor, and the Kaplan NCLEX-RN
Predictive Exit Test could accurately predict both success and failure on the NCLEX-RN.
The overall NCLEX pass rate for this sample was 90.8% (see Table 1). Hypothesis
testing resulted in supporting Hypothesis 1 (ATI test with the NCLEX test result; see
Tables 4, 5, and 6). It was not possible to test Hypothesis 2 (Kaplan score with NCLEX
test result) due to limited available data. Hypothesis 3 (TEAS scores with NCLEX test
result) received limited support, with only the Math subscale score being significantly
higher for successful NCLEX students (see Table 6). The final chapter will contain a
comparison of these findings to the literature, as well as conclusions, implications, and a
series of recommendations.
Findings
The sample in the quantitative study consisted of 251 BSN and MSN students: 45
traditional BSN students who graduated in 2009, six entry-level MSN students who
graduated in 2009, 48 traditional BSN students who graduated in 2010, 19 entry-level
MSN students who graduated in 2010, 52 traditional BSN students who graduated in
2011, 13 entry-level MSN students who graduated in 2011, and 38 traditional BSN
students who graduated in 2012. Of the students, 89% passed the NCLEX-RN on their
first attempt.
The study involved calculations for the minimum, maximum, mean, and standard
deviations for the ATI TEAS scores, the ATI RN Comprehensive Predictor scores, and
the Kaplan NCLEX-RN Readiness Test scores. The ATI TEAS scores ranged from a
low of 25 to a high of 100, with a mean ATI TEAS score of 50. The ATI RN
57
Comprehensive Predictor scores ranged from a low of 25 to a high of 100, with a mean
ATI RN Comprehensive Predictor score of 50. The Kaplan NCLEX-RN Readiness Test
scores ranged from a low of 25 to a high of 100, with a mean Kaplan NCLEX-RN
Readiness Test score of 50.
Testing of the Hypotheses
Null Hypothesis 1 was as follows: The ATI RN Comprehensive Predictor test
score is significantly related to results on the NCLEX-RN. Table 6 displays the chisquare tests used to test the relationship between the two variables. This table includes
the chi-square tests for the total sample (n = 219), the traditional subsample (n = 182),
and the accelerated subsample (n = 37). A significant relationship was found in both the
total sample (p = .001) and the traditional subsample (p = .001). In the accelerated
student subsample, all 37 students passed the NCLEX on the first attempt. This
combination of findings provided support to reject Null Hypothesis 1 (see Table 6).
Table 6
Relationship Between Outcome of ATI Screening Test and NCLEX Outcome
Failed
n
%
a
Passed
n
%
Sample and ATI outcome
Total sample (n = 219) b
Fail
16 16.0
84 84.0
Pass
3
2.5
116 97.5
Traditional subsample only (n = 182) c
Fail
16 18.0
73 82.0
Pass
3
3.2
90 96.8
Accelerated subsample only (n = 37) d
Fail
0
0.0
11 100.0
Pass
0
0.0
26 100.0
a
Passing the ATI screening test meant achieving at least the 90th percentile. bTotal
sample: χ2 (1, n = 219) = 12.46, p = .001. Cramer’s V = .24. cTraditional subsample: χ2
(1, n = 182) = 10.59, p = .001. Cramer’s V = .21. dAccelerated subsample: no chi-square
or Cramer’s V test because all passed the NCLEX.
58
Null Hypothesis 2 was as follows: The Kaplan Predictive Exit Test score is
significantly related to results on the NCLEX-RN. Given that less than half the students
took the Kaplan test, the researcher did not test this hypothesis.
Null Hypothesis 3 was as follows: The ATI TEAS nursing school admissions test
score is not significantly related to results on the NCLEX-RN. To examine this
hypothesis, Table 5 displayed the t tests for independent means comparing whether
students passed the NCLEX on their first attempt with the TEAS total score along with
the seven TEAS subscale scores. Only the TEAS Math score was significantly higher (p
= .005) for students who passed the NCLEX. This combination of findings provided
limited support to reject Null Hypothesis 3.
Table 5 also showed the t test for independent means between the students’ ATI
score and whether they passed the NCLEX on their first attempt. Students who passed
had significantly higher ATI scores (p = .001). The table included the point-biserial
correlation (rpb) used as a measure of the strength of that relationship. The correlation
(rpb = .27, rpb2 = .073) found that the ATI score accounted for 7.3% of the variance in
whether students passed the NCLEX on their first attempt.
This study included data from 251 students to determine if the TEAS, the ATI RN
Comprehensive Predictor, and the Kaplan NCLEX-RN Predictive Exit Test could
accurately predict both success and failure on the NCLEX-RN. The overall NCLEX pass
rate for this sample was 90.8% (see Table 1). Hypothesis testing revealed support for
Hypothesis 1 (ATI test with the NCLEX test result; see Tables 4 and 5). Testing of
Hypothesis 2 (Kaplan score with NCLEX test result) did not occur due to limited
available data. Hypothesis 3 (TEAS scores with NCLEX test result) found limited
59
support, with only the Math subscale score being significantly higher for successful
NCLEX students (see Table 5).
Chapter Summary
Chapter 4 contained the findings for the statistical analyses. The statistical
analyses determined if the predictor variables had significant relationships between
passing and failing the NCLEX-RN examination on the students’ first attempt. The
predictor variables included ATI TEAS scores, ATI RN Comprehensive Predictor scores,
and Kaplan NCLEX-RN Readiness Test scores. Chapter 5 contains a discussion of the
research findings, interpretations, and conclusions. The chapter also includes
recommendations for future research studies.
60
Chapter 5
Conclusions and Recommendations
Chapter 5 contains a discussion of the research findings, interpretations, and
conclusions. Also reviewed will be findings from other studies that are in the line with
the results from the current study. The chapter also includes recommendations for future
research.
The severe and worsening nursing shortage in the United States has increased the
need to examine progression policies in American schools of nursing. The specific
problem discussed in this study was the use of NCLEX-RN predictor examinations as
high-stakes tests to ensure satisfactory first-time NCLEX-RN pass rates at schools of
nursing in the United States. If predictor examinations do not correctly predict failure on
the NCLEX-RN, then leaders of nursing schools should not hold nursing students back
based on the results of an inaccurate predictor test. To help alleviate the severe and
worsening nursing shortage, nursing schools in the United States must graduate all
students who can demonstrate safe and effective clinical practice in the classroom and
clinical settings.
Purpose of the Study
There is a gap in the literature regarding the ability of the ATI RN Comprehensive
Predictor or the Kaplan NCLEX-RN Readiness Test to forecast failure on the NCLEXRN accurately. The purpose of the study was to determine if the ATI TEAS, the ATI RN
Comprehensive Predictor, and the Kaplan NCLEX-RN Readiness Test could accurately
predict both success and failure on the NCLEX-RN at one baccalaureate degree nursing
program in Southern California. The independent variables were BSN students’ ATI
61
TEAS scores, ATI RN Comprehensive Predictor scores, and Kaplan NCLEX-RN
Readiness Test scores. The dependent variable was the BSN students’ first-time results
on the NCLEX-RN. The archived records examined were from 251 BSN and entry-level
MSN students.
Research Questions
1. Are NCLEX-RN predictor examination scores (independent variable)
significantly related to results on the NCLEX-RN (dependent variable)?
2. Are nursing school admissions test scores (independent variable) significantly
related to results on the NCLEX-RN (dependent variable)?
Hypotheses
The study also included six hypotheses.
H10: The ATI RN Comprehensive Predictor test score is not significantly related
to results on the NCLEX-RN.
H1a: The ATI RN Comprehensive Predictor test score is significantly related to
results on the NCLEX-RN.
The study involved testing Hypothesis 1 using ANOVA to determine if a significant
relationship existed between the ATI RN Comprehensive Predictor test and NCLEX-RN
results.
H20: The Kaplan Readiness Test score is not significantly related to results on the
NCLEX-RN.
H2a: The Kaplan Readiness Test score is significantly related to results on the
NCLEX-RN.
62
The study involved testing Hypothesis 2 using ANOVA to determine if a significant
relationship existed between the Kaplan Readiness Test and NCLEX-RN results.
H30: The ATI TEAS nursing school admissions test score is significantly related
to results on the NCLEX-RN.
H3a: The ATI TEAS nursing school admissions test score is not significantly
related to results on the NCLEX-RN.
The study involved testing Hypothesis 3 using ANOVA to determine if a significant
relationship existed between the ATI TEAS nursing school admissions test and NCLEXRN results.
Summary of Key Findings
Null Hypothesis 1 was as follows: The ATI RN Comprehensive Predictor test
score is not significantly related to results on the NCLEX-RN. Table 6 showed the chisquare tests used to test the relationship between the two variables. This table includes
the chi-square tests for the total sample of students who took the ATI RN Comprehensive
Predictor (n = 219), the traditional subsample (n = 182), and accelerated subsample (n =
37). A significant relationship existed in both the total sample (p = .001) and the BSN
subsample (p = .001). In the MSN student subsample, all 37 students passed the NCLEX
on the first attempt. This combination of findings provided support to reject Null
Hypothesis 1.
The quantitative method was appropriate for identifying the ability of the
independent variables to predict success and failure on the NCLEX-RN. The findings
offered substantiation to reject Null Hypothesis 1 for the ATI RN Comprehensive
63
Predictor. The higher the students scored on the ATI RN Comprehensive Predictor, the
more likely they were to pass NCLEX-RN on the first try.
Null Hypothesis 2 was as follows: The Kaplan Predictive Exit Test score is not
significantly related to results on the NCLEX-RN. Given that less than half the students
took the Kaplan test, the stud did not involve testing this hypothesis.
Null Hypothesis 3 was as follows: The ATI TEAS nursing school admissions test
score is not significantly related to results on the NCLEX-RN. Table 5 displayed the t
tests for independent means comparing whether the students passed the NCLEX on their
first attempt with the TEAS total score along with the seven TEAS subscale scores. Data
analysis revealed the TEAS Math score was significantly higher (p = .005) for students
who passed the NCLEX. This combination of findings provided limited support to reject
Null Hypothesis 3.
The higher the student scored on the TEAS mathematics test, the more likely they
were to pass NCLEX-RN on the first try. Because ATI does not directly measure the
relationship between TEAS scores and NCLEX-RN results, nursing faculty and
administrators should find these data to be helpful to their programs. These results are
similar to other studies, such as one conducted by Ukpabi (2008). Ukpabi (2008)
performed a study to identify the variables attributed to success on the NCLEX-RN at a
baccalaureate nursing program. The ATI and NLN tests were the strongest predictors of
success on the NCLEX-RN.
Implications
This study resulted in numerous contributions to the literature. Few published
researchers have studied the predictive ability of the Kaplan Readiness Test. This study
64
involved examining the predictive ability of both the ATI tests and the Kaplan predictor
at one nursing school. Data from the study can assist in shaping admission and
progression policies for nursing programs that use these predictor tests.
The 219 students who took the ATI RN Comprehensive Predictor, both 2007 and
2010 versions, had similar or better pass rates than the rates predicted by ATI for all but
one of the predicted score ranges (ATI, 2007, 2010). One hundred students failed the
ATI RN Comprehensive Predictor on the first attempt. Of these 100 students, 84 passed
NCLEX-RN on the first attempt, which translated to 84% passing. The ATI scale
predicted these students would have only a 71% chance of passing NCLEX-RN. These
statistics call into question the predictive ability of the ATI RN Comprehensive Predictor
to forecast failure on the NCLEX-RN.
Data analysis showed no significant relationship between the Kaplan NCLEX-RN
Readiness Test and NCLEX-RN results (p > .05). However, because only 100 students
took the Kaplan Readiness Test, further study will be necessary to validate findings.
Nursing administration and faculty should use the Kaplan Readiness Test with caution
until researchers can independently validate its validity.
The overall TEAS score did not have a significant relationship with NCLEX-RN
results. However, the ATI TEAS Math score was significantly higher (p = .005) for
students who passed the NCLEX-RN. Nursing administration and faculty should take a
close look at this score for future applicants.
Limitations
The study included several limitations. The study had a nonrandomized sample.
The data collected were from only one nursing program in California. The students all
65
had gone to school under the same support system. Another limitation was the different
numbers of students who passed the first attempt of the NCLEX-RN versus those who
failed the first time. In this study, 200 students were in the NCLEX-RN pass group,
whereas only 19 students were in the fail group. The fail group size might not have been
large enough to accurately measure the prediction of failure on the NCLEX-RN. These
factors can limit the generalizability of the findings of this study to other nursing
programs across the United States. The study did not involve accounting for individual
differences among the students. The differences included age, ethnicity, gender, and
socioeconomic status.
Suggestions for Future Research
The results of the current study are in agreement with the literature that the
NCLEX predictor examinations often predict NCLEX-RN success accurately. The study
has also found that the accuracy of failure prediction is significantly less accurate.
However, researchers should replicate this type of study at many nursing schools across
the United States. Future researchers should study differences in predictive accuracy
between students of different ethnicities. Researchers should also study student test
results from ADN, BSN, and direct-entry MSN programs, as well as the differences in
test results for students of various age groups.
Another gap in the literature is the NCLEX predictors’ ability to predict a
successful career as a registered nurse. A researcher should conduct a longitudinal study
on NCLEX predictor results and a nurse’s patient care outcomes. The study would
require collaboration between the nursing school and the facilities where the graduates
work.
66
Significance
The significance of the quantitative study was the contribution of evidence-based
practice for restructuring progression policies in nursing schools across the United States.
In February 2012, the NLN Board of Governors published The Fair Testing Imperative in
Nursing Education on its website. The NLN became aware of the pressure schools of
nursing feel to generate high first-time NCLEX-RN pass rates (NLN, 2012). An
association exists between high first-time pass rates and the need to ensure safe nursing
practice for the public. The NLN’s primary concern is using standardized predictor tests
to prevent nursing students from taking the NCLEX-RN. Because of this concern, the
NLN developed national fair testing guidelines to ensure educators and administrators
follow evidence-based practices for progression and graduation.
The guidelines address the ethics of high-stakes testing. Leaders at the NLN
implore nurse educators to use predictor tests that are well-researched and validated
across each nursing course. They must also ensure the tests are free of age, gender,
ethnic, religious, sexual, or regional bias (NLN, 2012).
First-time NCLEX-RN pass rate is the measure used to determine the quality of
nursing schools in the United States. However, the school’s attrition rate is not often a
measure of quality. The first-time NCLEX-RN pass rate for California schools of
nursing is readily available on the California BRN website. The only other statistic
provided on the site is the number of students who sat for the NCLEX-RN in that
particular academic year. The website does not include the attrition rate. Government
authorities and regulatory agencies should report nursing school attrition rates to
67
encourage nursing schools to reduce their attrition rates. Remeasuring the NCLEX-RN
passing standard every 3 years may not be enough to ensure the safety of the public.
“Just as grades are not the only result of the college experience, the environment
is not the only cause of learning. Measuring only the components of students and
structure overlooks the complete equation: input plus structure equals outcomes”
(Cornell, 1985, p. 357). Structure (NCLEX-RN results) should not be the sole
determinant of the quality of a nursing program. Researchers have not extensively
studied the consequences of low NCLEX-RN pass rates for nursing programs (Spurlock,
2013). Although measuring the input level of a nursing student is more complicated than
measuring the structure, its value toward the safety of the public cannot be undermined.
Practitioner Recommendations
Nursing faculty should create course examinations in a similar style to the
NCLEX-RN. Consistently performing well on course examinations but struggling on
NCLEX predictor tests could demonstrate that the course examinations are dissimilar in
style or content to the predictor test. Faculty should focus on graduating high-quality
nurses who can alleviate the nursing shortage. The focus should not be on passing any
one test that could lead to increased attrition rates.
Nursing school administrators strive to obtain a high first-time NCLEX-RN pass
rate. However, they typically do not seek to maintain lower attrition rates. The first-time
NCLEX-RN pass rate for California nursing schools is easily available on the California
BRN website. Including the attrition rate on the website would present a clearer picture
of the school’s quality. In this scenario, nursing school administrators would probably
pay greater attention to their attrition rate. Recommendations include a decreased
68
reliance on NCLEX-RN predictors to determine readiness for the NCLEX-RN. Another
recommendation is the inclusion of the percentage of examinees who passed the NCLEXRN on the second attempt.
Another contributing factor to reliance on NCLEX-RN predictors is the inability
for nursing instructors to write items that are comparable to NCLEX-RN questions. Even
experienced faculty members have difficulty developing reliable and valid test questions
to adequately evaluate student learning (Halstead, 2013). Most nursing faculty would
benefit from attending item-writing workshops, such as those offered by NCSBN
(NCSBN, 2013). Just as hospitals use continuous quality improvement for patient
outcomes, faculty must continuously improve testing and remediation procedures. If
more nursing faculty would learn to write test items that are similar to NCLEX-RN items,
there would be fewer reasons to rely on NCLEX-RN predictors and administer them as
high-stakes tests. Many nursing programs do not rely on high-stakes tests to achieve
excellent NCLEX-RN pass rates (Spurlock, 2013). More research is necessary to
determine the reasons for their success.
Conclusion
The study involved investigating the effectiveness of the ATI TEAS test, the ATI
RN Comprehensive Predictor examination, and the Kaplan Readiness Test to predict
NCLEX-RN outcomes. The findings of this study might help to reexamine progression
policies at undergraduate schools of nursing. The results of the study demonstrated that
the NCLEX predictors studied were not accurate in predicting failure on the NCLEX-RN.
Researchers should study students from various schools to obtain a larger sample size and
69
should examine the differences between students of various ages, ethnicities, genders,
and socioeconomic statuses.
70
References
Abbott, A. A. (2008). Predictors of success on National Council Licensure Examination
for registered nurses for accelerated baccalaureate nursing graduates. Nurse
Educator, 33, 5-6. doi:10.1097/01.NCN.0000336453.62659.06
Adamson, C., & Britt, R. (2009). Repeat testing with the HESI exit exam-sixth validity
study. CIN: Computers, Informatics, Nursing, 27(6), 393-397.
doi:10.1097/NCN.0b013e3181c1367d
Aiken, L. H., Sloane, D. M., Cimiotti, J. P., Clarke, S. P., Flynn, L., Seago, J. A., . . .
Smith, H. L. (2010). Implications of the California nurse staffing mandate for
other states. HSR: Health Sciences Research, 45, 904-921. doi:10.1111/j.14756773.2010.01114.x
Alameda, M. D., Prive, A., Davis, H. C., Landry, L., Renwanz-Boyle, A., & Dunham, M.
(2011). Predicting NCLEX-RN success in a diverse student population. Journal of
Nursing Education, 50(5), 261-267. doi: 10.3928/01484834-20110228-01
Ali, A., Zaman, A., & Alamgir. (2011). Correlation between admission criteria and
academic performance of students in dental colleges of Khyber Pukhtunkhawa.
FWU Journal of Social Sciences, 5, 1-16. Retrieved from
http://www.fwu.edu.pk/downloads/
American Association of Colleges of Nursing. (2011). Nursing shortage fact sheet.
Retrieved from
http://www.aacn.nche.edu/Media/FactSheets/NursingShortage.htm
71
American Nurses Association. (2011). Nurse staffing plans and ratios. Retrieved from
http://www.nursingworld.org/MainMenuCategories/PolicyAdvocacy/State/Legislative-Agenda-Reports/State-StaffingPlansRatios
Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching,
and assessing: A revision of Bloom’s taxonomy of educational objectives. New
York, NY: Longman.
Ari, A. (2011). Finding acceptance of Bloom’s revised cognitive taxonomy on the
international stage and in Turkey. Educational Sciences: Theory & Practice, 11,
767-772. Retrieved from http://www.edam.com.tr/kuyeb/en/default.asp
Assessment Technologies Institute. (2011). Comprehensive assessment and review
program. Retrieved from https://atitesting.com/Solutions/DuringNursingSchool
/ComprehensiveAssessmentAndReviewProgram.aspx
Aucoin, J. W., & Treas, L. (2005). Assumptions and realities of the NCLEX-RN. Nursing
Education Perspectives, 26(5), 268-271. Retrieved July 24, 2011, from
http://www.nln.org/nlnjournal
Bell, J. A., & Martindill, C. F. (1976). A cross-validation study for predictors of scores
on state board examinations. Nursing Research, 26(7), 278-281. Retrieved
August 2, 2011 from
http://journals.lww.com/nursingresearchonline/pages/default.aspx
Benefiel, D. (2011). Predictors of success and failure for ADN students on the NCLEXRN (Doctoral dissertation). Available from ProQuest Dissertations and Theses
database. (UMI No. 3456526)
72
Birnbach, N. (1982). The genesis of the nurse registration movement in the United
States, 1893-1903. Ed.D. diss., Teachers College, Columbia University.
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956).
Taxonomy of educational objectives: The cognitive domain. New York, NY:
Longman.
Boland, D. (2009). Developing curriculum: Frameworks, outcomes, and competencies. In
D. M. Billings & J. A. Halstead (Eds.), Teaching in nursing: A guide for faculty
(pp. 137-153). St. Louis, MO: Saunders.
Bondmass, M., Moonie, S., & Kowalski S. (2008). Comparing NET and ERI
standardized exam scores between baccalaureate graduates who pass or fail the
NCLEX-RN. International Journal of Nursing Education Scholarship, 5, Article
16. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/18384274
Buerhaus, P. (1987). Not just another nursing shortage. Nursing Economic$, 5(6), 267279. Retrieved from http://www.nursingeconomics.net
Buerhaus, P. (1991). Economic determinants of the annual number of hours worked by
registered nurses. Medical Care, 29, 1181-1195. Retrieved from
http://journals.lww.com/lww-medicalcare/pages/default.aspx
Buerhaus, P., Staiger, D. O., & Auerbach, D. I. (2009). The future of the nursing
workforce in the United States: Data, trends, and implications. Boston, MA:
Jones & Bartlett.
Bumen, N. T. (2007). Effects of the original versus revised Bloom’s taxonomy on lesson
planning skills: A Turkish study among pre-service teachers. Review of
Education, 53, 439-455. doi:10.1007/s11159-007-9052-1
73
Bureau of Labor Statistics. (2012). Occupational outlook handbook (OOH), 2012-13
edition. Retrieved from http://www.bls.gov/ooh/healthcare/registered-nurses.htm
Burns, N., & Grove, S. K. (2009). The practice of nursing research: Appraisal, synthesis,
and generation of evidence (6th ed.). Philadelphia, PA: W. B. Saunders.
California Board of Registered Nursing. (2011). 2009-2010 annual school report.
Retrieved from http://www.rn.ca.gov/pdfs/schools/schoolrpt09-10.pdf
California Department of Public Health. (2010). Nurse to patient ratio. Retrieved from
http:// www.cdph.ca.gov/services/DPOPP/regs/Pages/N2PRegulations.aspx
Carr, S. M. (2011). NCLEX-RN pass rate peril: One school’s journey through curriculum
revision, standardized testing, and attitudinal change. Nursing Education
Perspectives, 32(6), 384-388. Retrieved February 12, 2012 from
http://www.nln.org/nlnjournal
Carrick, J. A. (2011). Student achievement and NCLEX-RN success: Problems that
persist. Nursing Education Perspectives, 32(2), 78-83. doi:10.5480/1536-502632.2.78
Chitty, K. K., & Black, B. P. (2011). Professional nursing: Challenges and concepts (6th
ed.). Maryland Heights, MO: Saunders Elsevier.
Cornell, G. (1985). The value-added approach to the measurement of educational
quality…measuring student gains. Journal of Professional Nursing, 1(6), 356363.
Creswell, J. W. (2005). Educational research: Planning, conducting, and evaluating
qualitative and quantitative research (2nd ed.). Upper Saddle River, NJ: Pearson,
Merrill, Prentice-Hall.
74
Deardorff, M., Denner, P., & Miller, C. (1976). Selected National League of Nursing
Achievement Test scores as predictors of state board examination scores. Nursing
Research, 25, 35-38. Retrieved August 2, 2011 from
http://journals.lww.com/nursingresearchonline/pages/default.aspx
Duan, Y. (2006). Selecting and applying taxonomies for learning outcomes: A nursing
example. International Journal of Nursing Education Scholarship, 3, Article 10.
doi:10.2202/1548-923X.1154
Duncan, B. A., & Stevens, A. (2011). High-stakes standardized testing: Help or
hindrance to public education. National Social Science Journal, 36(2), 35-43.
Retrieved from http://www.nssa.us/journals/pdf/NSS_Journal_36_2.pdf
EBSCO Industries. (2012). About us. Retrieved from http://www.EBSCO.com
Eich, M., & O’Neill, T. (2007). NCLEX delay pass rate study. Retrieved from
http://www.ncsbn.org
Federal Interagency Forum on Aging Related Statistics. (2009). Data sources on older
Americans 2009. Hyattsville, MD: Author.
Fowles, E. R. (1992). Predictors of success on NCLEX-RN and within the nursing
curriculum: Implications for early intervention. Journal of Nursing Education,
31(2), 53-57. Retrieved from http://www.healio.com/journals/jne
Giddens, J. (2009). Changing paradigms and challenging assumptions: Redefining quality
and NCLEX-RN pass rates. Journal of Nursing Education, 48(3), 123-124.
doi:10.3928/01484834-20090301-04
75
Grossbach, A., & Kuncel, N. R. (2011). The predictive validity of nursing admission
measures for performance on the National Council Licensure Exam: A metaanalysis. Journal of Professional Nursing, 27, 124-128. doi:
10.1016/j.profnurs.2010.09.010
Halstead, J. A. (2013). The NLN’s fair testing imperative and implications for faculty
development. Nursing Education Perspectives, 34(2), 72. Retrieved August 7,
2013 from http://www.nln.org/nlnjournal
Harding, M. (2010). Predictability associated with exit examinations: A literature review.
Journal of Nursing Education, 49(9), 493-497. doi:10.3928/01484834-2010073001
Harding, M., Rateau, M., & Heise, J. L. (2011). Efficacy of a midcurricular examination
for predicting nursing student academic success. CIN: Computers, Informatics,
Nursing, 29(10), 593-598. doi:10.1097/NCN.0b013e3182066458
Health Resources and Health Administration. (2010). HRSA study finds nursing
workforce is growing and more diverse. Retrieved from
http://bhpr.hrsa.gov/healthworkforce/rnsurveys/rnsurveyinitial2008.pdf
Kane, R. L., Shamliyan, T. A., Mueller, C., Duval, S., & Wilt, T. J. (2007). The
association of registered nurse staffing levels and patient outcomes: Systematic
review and meta-analysis. Medical Care, 45, 1195-1204. Retrieved July 24,
2011, from http://journals.lww.com/lww-medicalcare
Kaplan. (n.d.). Kaplan-LWW NCLEX-RN Integrated Testing. Retrieved from
http://www.kaptest.com/NCLEX/Deans-and-Directors/RN-Programs/rntesting.html
76
Kleber, M. A. (2010). Determination of variables which predict success on the National
Council Licensure Examination (NCLEX-PN) (Doctoral dissertation). Available
from ProQuest Dissertations and Theses database. (UMI No. 3451104)
Landry, L. G., Davis, H., Alameida, M. D., Prive, A., & Renwanz-Boyle, A. (2010).
Predictors of NCLEX-RN success across 3 prelicensure program types. Nurse
Educator, 35(6), 259-263. doi:10.1097/NNE.0b013e3181f7f1c9
Lavandera, R., Whallen, D. M., Perkel, L. K., Hackett, V., Molnar, D., Steffey, C., . . .
Harris, J. (2011). Value-added of HESI exam as a predictor of timely first-time
RN licensure. International Journal of Nursing Education Scholarship, 8, 1-12.
doi:10.2202/1548-923X.2152
LoBiondo-Wood, G., & Haber, J. (2008). Nursing research: Methods and critical
appraisal for evidence-based practice. St. Louis, MO: Mosby.
Madaus, G., & Russell, M. (2009). The paradoxes of high stakes testing: How they affect
students, their parents, teachers, principals, schools, and society. Charlotte, NC:
Information Age.
Matassarin-Jacobs, E. (1989). The nursing licensure process and the NCLEX-RN. Nurse
Educator, 14(6), 32-35. Retrieved from
http://journals.lww.com/nurseeducatoronline/pages/default.aspx
McGahee, T. W. (2010). NCLEX-RN success: Are there predictors? Southern Online
Journal of Nursing Research, 10(4), Article 13. Retrieved from
http://snrs.org/publications/SOJNR_articles2/Vol10Num04Art13.html
Morris, T., & Hancock, D. (2008). Program exit examinations in nursing education:
Using a value added assessment as a measure of the impact of a new curriculum.
77
Educational Research Quarterly, 32(2), 19-29. Retrieved from
http://www.erquality.org
Morrison, S. (2005). Improving NCLEX-RN pass rates through internal and external
curriculum evaluation. In M. H. Oermann & K. Heinrich (Eds.), Annual review of
nursing education (Chap. 5, pp. 77-94). New York, NY: Springer.
Murray, K., Merriman, C., & Adamson, C. (2008). Use of the HESI admission
assessment exam to predict student success. CIN: Computers Informatics
Nursing, 26(3), 167-172. Retrieved from
http://www.nursingcenter.com/library/JournalArticle.asp?Article_ID=818352
Nasstrom, G. (2009). Interpretation of standards with Bloom’s revised taxonomy: A
comparison of teachers and assessment experts. International Journal of Research
and Method in Education, 32, 39-51. doi:10.1080/17437270902749262
National Council of State Boards of Nursing. (2011). 2011 NCLEX Examination
Candidate Bulletin. Retrieved from
https://www.ncsbn.org/11_NCLEX_Candidate_Bulletin.pdf.pdf
National Council of State Boards of Nursing. (2012). 2013 NCLEX Examination Test
Plan. Retrieved from http://www/ncsbn.org
National Council of State Boards of Nursing. (2013). Test development webinar.
Retrieved from https://www.ncsbn.org/4282.htm
National League for Nursing Board of Governors. (2012). The fair testing imperative in
nursing education. Retrieved from
http://www.nln.org/aboutnln/livingdocuments/pdf/nlnvision_4.pdf
78
Neuderth, S., Jabs, B., & Schmidtke, A. (2009). Strategies for reducing test anxiety and
optimizing exam preparation in German university students: A preventionoriented pilot project of the University of Wurzburg. Journal of Neural
Transmission, 116, 785-790. doi: 10.1007/s00702-008-0123-7
Neuman, W. L. (2003). Social research methods: Qualitative and quantitative
approaches (5th ed.). Boston, MA: Allyn and Bacon.
Nibert, A. T., Young, A., & Adamson, C. (2006). Predicting NCLEX success with the
HESI Exit Exam: Fourth annual validity study. Computers, Informatics, Nursing:
CIN, 20(6), 261-267. doi: 10.1097/01.NCN.0000336439.16918.8b
Nichols, S. L., & Berliner, D. C. (2008). Why has high-stakes testing so easily slipped
into contemporary American life? Education Digest, 74(4), 41-47. Retrieved from
http://www.eddigest.com/
Orman, J. M., & Guarneri, C. E. (2009). United States population projections: 2000 to
2050. Retrieved from
http://www.census.gov/population/www/projections/analytical-document09.pdf
Pishghadam, R., & Khosropanah, F. (2011). Predictive validity of the English Language
Teacher Competency Test. International Journal of Education, 3, 1-19. Retrieved
from http://profdoc.um.ac/ir/articles/a/1022785.pdf
Polit, D. F., & Beck, C. T. (2008). Nursing research: Generating and assessing evidence
for nursing practice (8th ed.). Philadelphia, PA: Lippincott Williams & Wilkins.
ProQuest. (2012). Library types. Retrieved from http://www.ProQuest.com
Roa, M., Shipman, D., Hooten, J., & Carter, M. (2011). The costs of NCLEX-RN failure.
Nurse Education Today, 31, 373-377. doi: 10.1016/j.nedt.2010.07.009
79
Ross, S. J., Polsky, D., & Sochalski, J. (2005). Nursing shortages and international nurse
migration. International Nursing Review, 52(4), 253-262. Retrieved July 24,
2011 from http://onlinelibrary.wiley.com/journal/10.1111/%28ISSN%2914667657
Simon, E. B., McGinniss, S. P., & Krauss, B.J. (2013). Predictor variables for NCLEXRN readiness exam performance. Nursing Education Perspectives, 34, 18-24.
Retrieved August 7, 2013 from http://www.nln.org/nlnjournal
Spurlock, D. R., & Hanks, C. (2004). Establishing progression policies with the HESI
Exit Examination: A review of the evidence. Journal of Nursing Education,
43(12), 539-545. Retrieved July 24, 2011 from
http://www.healio.com/journals/jne
Spurlock, D. R., & Hunt, L. A. (2008). A study of the usefulness of the HESI Exit Exam
in predicting NCLEX-RN failure. Journal of Nursing Education, 47(4), 157-166.
Retrieved July 24, 2011 from http://www.healio.com/journals/jne
Spurlock, D. (2013). The promise and peril of high stakes tests in nursing education.
Journal of Nursing Regulation, 4, 4-8. Retrieved June 8, 2013 from
http://jnr.metapress.com/content/u3uk17m21h014161/
Tevington, P. (2011). Mandatory nurse-patient ratios. MEDSURG Nursing, 20(5), 265268. Retrieved February 27, 2012 from http://www.medsurgnursing.net
Tipton, P., Pulliam, M., Beckworth, C., Illich, P., Griffin, R., & Tibbitt, A. (2008).
Predictors of associate degree nursing students’ success. Southern Online Journal
80
of Nursing Research, 8, Article 2. Retrieved from
http://snrs.org/publications/SOJNR_articles2/Vol08Num01Art02.html
Ukpabi, C. V. (2008). Predictors of successful nursing education outcomes: A study of
the North Carolina Central University’s nursing program. Educational Research
Quarterly, 32(2), 30-40. Retrieved July 24, 2011 from
http://eric.ed.gov/?id=EJ847443
U.S. Census Bureau. (2006). Selected characteristics of baby boomers 42 to 60 years old
in 2006. Retrieved from http://www.census.gov/population/www/socdemo/age
/2006%20Baby%20Boomers.pdf
Uyehara, J., Magnussen, L., Itano, J., & Zhang, S. (2007). Facilitating program and
NCLEX-RN success in a generic BSN program. Nursing Forum, 42, 31-38.
Retrieved July 24, 2011 from
http://onlinelibrary.wiley.com/journal/10.1111/%28ISSN%291744-6198
Valiga, T. M., & Ironside, P. M. (2012). Crafting a national agenda for nursing education
research. Journal of Nursing Education, 51, 3, 6. doi:10.3928/0148483420111213-01
Washburn, G. (1980). Relationship of achievement tests scores and state board
performance in a diploma nursing program. Doctoral dissertation, Indiana
University at South Bend.
Yett, D. E. (1975). An economic analysis of the nurse shortage. Lanham, MD: Lexington
Books.
81
Zweighaft, E. L. (2011). The impact of the Elsevier/HESI specialty exams on student
performance on the National Council Licensing Exam for registered nurses
(Doctoral dissertation). Available from ProQuest database. (UMI No. 3459906)
82
Appendix A
Data Collection Tool
83
Appendix B
Permission to Use Premises Form
84
Appendix C
Premises, Recruitment, and Name Use Permission
85
Author Biography
Lawrence Santiago has been a registered nurse for 16 years. He has been a nursing
instructor for nine years, including four years of teaching undergraduate nursing students.
He has also taught BSN and MSN students for University of Phoenix for five years.
Lawrence obtained his Bachelor of Science in Nursing degree from Azusa Pacific
University. He received a Master of Science in Nursing degree from Cal State University
Dominguez Hills. He has completed an Ed.D. program at University of Phoenix. He is
currently an Education Program Coordinator at Cedars-Sinai Medical Center in Los
Angeles.
He has been married for ten years and they have an eight year old son.
86