High-Stakes
Assessment:
Using Evidence for
Admission Policy
Decisions
Cheryl Shaffer PhD, RN
Associate Dean
School of Nursing
Background
O First-time licensure pass rates have been used as a
quality indicator for nursing program success.
O The Accreditation Commission for Education in
Nursing (ACEN) require evidence of achievement of
the licensure examination pass rates being at or
above the national mean.
Background
O The National Council of State Boards of Nursing
publishes a new NCLEX (Licensure Examination)
test plan and passing standard every three years.
O The latest version of the NCLEX-RN test plan (April
2013) will remain in effect until March 31, 2016.
The Problem
O Nationwide, the competency level for safe
and effective entry-level nursing practice
has significantly and permanently changed.
O As such, the April 2013 passing standard
rise was the largest single increase ever in
the history of the RN licensure exams.
First time NCLEX Pass Rates
School of Nursing
2011
National
Average
87.89%
80.56%
New York
Average
84.05%
2012
90.34%
82.96%
87.15%
2013 *
83.04%
73.02%
78.14%
2014 **
84.69 %
67.67 %
82.34%
* 2010 and 2013 test plan
** Only January graduates
Previous Admission Criteria
O 3.0 cumulative grade point average for: ENG101,
PSY101, BIO130 and BIO132 (and SOC 101 for
Advanced Placement Program)
O A minimum grade of C must be earned for BIO130
and BIO132
O All prerequisite courses may be retaken only once to
achieve a higher grade. The second grades will be
used for admission consideration.
Previous Admission Criteria
O Admission to the nursing programs has
always been competitive
O Based on the NCLEX results, it was
hypothesized the current criteria for
admission was no longer an accurate
predictor of program success
Repeated Course Withdrawals
O Approximately 25% of admitted students
withdraw or repeat their science courses
O Incentives to withdraw, opposed to course
completion from such prerequisite courses
may be attributed to their use as
preadmission criteria.
O Students understand that final averages are
crucial in competitive selections.
Repeated Course Withdrawals
O Through regression analysis, repeated attempts in
preadmission science courses were found to be a
significant predictor of NCLEX-RN outcome in SCCC
graduates from 2010 and 2011 (N=335)
O There was a significant, negative correlation between the
number of science course attempts and passing the
NCLEX-RN on the first attempt (r= -0.284, p=0.000).
O Repeating science course grades were found to be a
significant predictor of NCLEX success, accounting for 25
percent of the total variance (F 1, 164 = 5.732, R2 = 0.254, p
= 0.000).
Shaffer, C., McCabe, S. (2013). Evaluating the predictive validity of preadmission academic criteria: high stakes
assessment. Teaching and Learning in Nursing, 8 (4), 157-161.
Repeated Course Withdrawals
O 82 students did not graduate May 2014 due
to withdrawal from a nursing course or
achieving a grade less than a C
O Of those 82 students, 42 % withdrew 1 or
more times from a Science prerequisite
Review of the Literature
O Nursing program admission criteria in the
literature and other SUNY community colleges
and universities on Long Island was reviewed.
O The most common admission criterion was
administration of the entrance exam, the
Assessment Technologies Institute (ATI) Test of
Essential Academic Skills (TEAS V).
O This result was combined with pre-requisite
course grades and the use of a scoring rubric.
The TEAS V
O Computerized test, 209 minutes, 150 multiple
choice items
O Measures students basic or entry-level program
readiness in the major content areas of Reading,
Mathematics, Science and English/Language
usage.
O The TEAS V score report will identify students
overall academic preparedness necessary to
support learning of nursing course content.
The TEAS V
O Academic Preparedness Level category of:
O Exemplary- high level
O Advanced-high level
O Proficient-moderate level
O Basic- low level
O Developmental-very low level
Assessment Project
O June 2014, all Fall 2014 (N=225) newly
admitted students took the TEAS V post
admission
O Students individual performance profile can
was used to guide additional preparation for
the nursing program over the summer.
Results
Academic
Preparedness
At-Risk Area
Qualified for
Admission
Unqualified
for Admission
Reading
Mathematics
Science
English
Language
72%
32%
11%
20%
30%
12%
Results- Math
Percent of Students
Deficient
Numbers and Operations
Identify decimal placement
Compare rational numbers
Divide fractions
Solve word problems
Algebraic Applications
Add, subtract, multiply polynomials
solve equations or inequalities
Measurement
Convert quantities from 1 scale to another
64%
61%
57%
55%
62%
62%
Nursing Students
Computation Expectations
O Read a dosage calculation word problem
O Set up dosage calculation in fractional form
O Ratio and proportion method
O Cross multiplying fractions
O Substitute converted units into proportion
O Metric system and weight measurement
conversions
O Round to nearest tenths and hundredths
Results- Reading
Percent of Students
Deficient
Paragraph Comprehension
Identify characteristics of different passages
61 %
Identify topic, main idea, supporting details,
themes
68 %
Analyze and evaluate use of text structure
55 %
Results- English/Language
Percent of
Students
Deficient
Grammar and Word Meanings in Context
Apply subject-verb agreement rules
Use of grammar for stylistic effect and enhance
clarity
Use context clues to determine meaning of
words
Use dialogue correctly
Spelling and Punctuation
Capitalization
65 %
60%
61%
58%
80%
Nursing Students
Writing Expectations
O Using APA style for reference page for
various sources
O (textbook, periodicals, internet sources),
O Proper use of punctuation or grammar,
organization of ideas
O Apply evidence in the literature to patient
assignments
Results- Science
Percent of Students
Deficient
Life Science
Identify parts of a cell
Explain changes in DNA
Explain heredity information
Compare and contrast chromosomes and genes
67 %
68 %
62 %
61 %
Physical Science
Explain physical and chemical properties
Balance and identify chemical reactions
Distinguish among energies
63 %
62 %
56 %
Scientific Reasoning
Recognize and analyze alternative explanations and models
70 %
Nursing Students
Science Expectations
O Build on prerequisite courses to apply
knowledge of the human body anatomy and
physiology, and principles of microbiology to
learn concepts of pathophysiology
O Development of critical thinking, problem
solving, analysis and synthesis to provide
patient and family nursing care
Common Themes
O Students demonstrated ability of subject domain
knowledge and comprehension
O Students were deficient in ability to:
O
O
O
O
O
O
O
O
O
Apply
Convert
Evaluate
Explain
Solve
Distinguish
Analyze
Compare and contrast
Develop alternative explanations
Use of TEAS V Data for
Decision Making
O Development of a proposal to change admission policy
O Prioritize curricular goals, identify and correct gaps
O Improve teaching of higher level cognitive processes
O Allocate resources
O Interdisciplinary collaboration
New Admission Policy
( in addition to prerequisite course work)
O All* RN program applicants are required to take
the pre-admission Test for Essential Academic
Skills (TEAS V) and achieve a minimum of
Proficient Level
O Within an academic year (September 1 through
August 31), applicants may only take the TEAS V
twice, and must register and pay all fees each
time.
Answers to FAQ
O TEAS V is now offered at the Sayville Center
O Cost $56
O Directions on how to register is on the School of
Nursing Website
O Prospective students are advised use the TEAS
study guide manual and take practice assessments
online prior to taking the exam
Expected Outcomes
O Admitted students will possess the
academic preparedness for program
success
O Students with lower level of academic
preparedness can focus on areas to be
strengthened before applying to program
O Take additional course
O Academic labs
O Review course