Closing the Loop on Student Learning
Assessment
Mohua Bose, Ph.D
Director of Outcomes Assessment
Mary Beth Hanner, Ph.D, RN, ANEF
Provost and Chief Academic Officer
Excelsior College, Albany, NY
Presentation at MSCHE, December 12, 2011
Purpose
 Describe a framework that addresses
assessment of student learning in a nontraditional degree completion institution.
 Discuss improvements based on
assessment results at the program and
institutional level.
 Describe the process of facilitating an
assessment culture.
Background
 Excelsior College founded in 1971 (previously
Regents College).
 1998 Chartered as a private, independent
college.
 Large student body (approximately 33,000
students).
 Distance education, flexible learning format.
Philosophy:
“What you know is more important that where or
how you learned it”
Background
 Valuing prior learning is a central premise
for adult learners (Green, 1998).
 Acceptance of a variety of credit sources:
Examples:
 Regionally accredited institutions
 ACE/PONSI approved courses
 Exams – CLEP, DANTE,EC
 EC portfolio assessment for prior learning
 Online courses at Excelsior and preferred providers
 Residency requirement (presently minimal,
historically none).
Background
 Accreditation




Middle States (all programs)
NLNAC (specialized accreditation - Nursing)
ABET (specialized accreditation - Technology)
IACBE (specialized accreditation - Business)
Challenge:
Who is a typical student?
Career oriented working adult
College 2
College 1
Average age:
39 year
College 3
At a distance
Excelsior
College
College 5
College 4
No first time, full time students.
Credits transferred from other approved sources ranges from 25% to
75% of total credits required for the degree program.
Accountability and Continuous Improvement
OAIR Steering Committee
School of
Business
& Tech
School of
Liberal
Arts
OAIR
Office
School
of Health
Sciences
School
of
Nursing
Institutional Assessment Plan for Student Learning
(IAPSL)
Institutional Assessment Plan for
Student Learning (IAPSL)
 Provides a framework for assessment of learning
outcomes at course, program and institutional
level.
 Addresses the key question “How effectively do
we deliver on our mission and make a distinctive
impact relative to our resources?” (Collins, 2011).
 IAPSL link:
http://www.excelsior.edu/web/excelsior-theinstitution
Assessment Process at Schools
Establish clear
learning
outcomes
Provide
learning
opportunities
Use the Results
Assess Student
Learning
Step 1: Establish Learning Outcomes
 Ultimate responsibility to determine learning
outcomes for a program rests with the faculty.
 Instructional/Adjunct faculty receive an online
faculty training that orients them to EC. The
orientation includes an introduction to learning
outcomes assessment.
 Development of online courses requires
completion of a structured template that involves
program, course and module outcomes.
Step 1: Establish Learning Outcomes
(continued)
 A raised awareness of the importance of
learning outcomes assessment at the College
due to OAIRSC and OAIR offices. Senior level
personnel significantly involved (eg. Deans).
 In addition to OAIR, an assessment director
within the schools work with the faculty.
 Continuous and evolving process in defining
learning outcomes.
Step 2: Create Learning Opportunities
Monitoring the learning opportunities
provided to students:
 Course revisions based on:
Centralized student course evaluations
Faculty evaluation of courses
Faculty curriculum committee
 Curriculum reviews
Example: Curriculum Review
 Committee of 3 reviewers (some members are
completely external to the college).
 2-3 days including two teleconferences.
 Reviewers examine curricular structure, topical
areas covered, course descriptions.
 Provide feedback on curriculum rigor,
curriculum currency, and program outcomes
attainability and currency.
 Feedback from the external curriculum review is
taken back to program faculty.
Step 3: Assessing Student Learning
 Direct assessments of program outcome
measures for all programs using:
o Capstones, Portfolios, or Exams (theory or
performance)
o State licensing exams (e.g. NCLEX)
 Direct assessments of general education
outcomes using:
o ETS Proficiency Profile & VALUE Rubrics
Step 3: Assessing Student Learning
 Also have indirect assessments:
 Alumni surveys
 Supervisor surveys
Centralized for all degree programs
Discipline Specific Example on
Closing the Loop
Associate Degree in Nursing
NCLEX pass rates
 Consistently at or above national average
over a decade.
 Dipped below for one quarter in national
and state pass rates by approximately 1%.
 Conducted analysis of subscales on the
NCLEX exam.
 Examined the CPNE exam.
Associate Degree in Nursing
 Analysis of the data revealed that students
scored lower on
o Knowledge of adolescent growth and
development
o Care of the ill adolescent
 After faculty review – new curriculum
requirements.
 Growth and Development across the
Lifespan.
Associate Degree in Nursing
 After approximately 2 years
• Scores exceeded both state and national
levels
• NCLEX scores in overall exam and
subscales
 Remained up for a number of years.
 Continuing to monitor.
General Education Example
General Education
 Assess Writing
Outcome: Write with grammatical
competence, clarity, and coherence
 Administered standardized assessment:
ETS Proficiency Profile
Reading
Writing
Mathematics
Critical thinking
General Education
 ETS Proficiency Profile administered
annually 2009, 2010, and 2011.
 Results pertaining to Writing show that
Excelsior students at the Associate’s and
Bachelor’s level have performed at or
above the ETS norms for writing.
General Education
 College decided to use multiple measures
to assess writing.
 Use of VALUE rubrics in writing
 Capstone papers from Bachelor’s degree
programs across the College used.
 Independent panel of reviewers.
 Data collection completed, shows
variations in student performance.
 Closing the loop takes time.
Key Steps in Implementing
Outcomes Assessment
 Institutional support and commitment.
 Creation of a college-wide oversight
committee to set a relevant and strategic
agenda for assessment of student learning
and institutional performance.
 A visionary and guiding document for
implementation of outcomes assessment
College-wide.
Key Steps in Implementing
Outcomes Assessment
 Training and implementation of an
assessment management tool/database.
 Centralized assessment model supported
by a distributed model of assessment staff.
 Showcase assessment results - Quarterly
Dean’s presentations at Steering
Committees.
 Bi-annual assessment audits.
 Continuous and evolving process!