Student Learning Outcomes Workshop
University of Tennessee Chattanooga – September 24, 2012
Jillian Kinzie, Indiana University, Center for Postsecondary Research
Assessment
Assessment is an ongoing process aimed at understanding and improving student learning. It involves:
• Making program objectives and student learning outcomes explicit and public
• Setting appropriate criteria and high expectations
• Systematically gathering, analyzing, and interpreting evidence to
determine how well programs and services match those
expectations and standards
• Using the resulting information to document, explain, and improve
• Demonstrating student learning gains and contributions to student
success
• Showcasing department/program strengths
• Identifying areas where the department/program can improve
Assessment needs to be authentic and meaningful to those orchestrating it.
There should be furthered through a feeling of “ownership” about assessment activities. Assessment
should ideally work to provide a deep understanding of the connection between institutional,
departmental/major programs and course level learning and the effects on students. Assessment should
additionally and simultaneously help improve the student experience based on such tangible factors as
performance, retention, and graduation, and such less-tangible factors as student health, citizenship, and
overall lifelong learning and well-being.
At its best, assessment reflects the ethic of inquiry that informs academic and faculty work – it brings
scholarly habits of research and inquiry into the work of teaching and learning.
Assessment generally occurs at 3 levels: Institutional, Program or Department, and the course/classroom
level.
Institutional Level = identified set of learning outcomes expected of all students; assessment provides
broad picture of students program on outcomes; additionally, campuses employ assessments of overall
quality of the student experience; activities usually directed at central level, but results can inform
department/program and class level and including in and out of classroom learning environment.
Program/Department Level = articulation of program goals and learning outcomes at the department
or major level – What should our majors know and be able to do? What kind of experiences do our
students have and what do we know about what makes a difference to post-college outcomes? Locally
developed approaches to assessment, including portfolios, presentations, senior projects, and even
Major Field Tests, can be used; Assessment activities can be powerful educational experiences for
students, and provide information to departments for determining needed improvements.
Course/Classroom Level = assessment of course level student learning and quality of course
experiences – always integral to classroom teaching! Approaches can be simple, and formative – asking
students about the “muddiest point” – or complex involving pre-post tests of central course concepts,
or using rubrics across multiple sections of a course.
1
Program Goals & Outcomes: Program goals are broad statements that describe the overarching
long-range intended goals of the academic unit/program. Describes the primary function of academic
unit/program; key activities involved; and student competencies upon graduation from the academic
unit/program. Goals may align with respective professional organization and program-specific accrediting
bodies.
Program Goals & Outcomes should describe:
1. What should a graduate of your program know, do, and value?
2. What program experiences contribute to producing and supporting students in your program?
Checklist to determine if the goal statements are appropriate.
 Do your goals describe desired performance?
 Are your goals consistent with your institutional and program mission?
 Are your goals aligned with program values and key experiences?
Student Learning Outcomes (SLOs):
Student Learning Outcomes (SLOs) are specific,
measurable statements that use action verbs to articulate the knowledge, skills, and perceptions students
should gain or improve through engagement in the academic program or learning experience during a
specific time frame.
Distinguishing Course (SLOs) from Program (SLOs)?
• Program student learning outcomes (SLOs) identify what students should learn as a result of their entire
educational experience. For example “Upon completion of the XXXX program, students will be able to
discuss the processes of urbanization and modernization in the developing world.”
• Course student learning outcomes (SLOs) identify what students should learn as a result of taking a
specific course. For example “Upon completion of COUR XXX, students will be able to analyze cities as
products of modernization, as expressions of various processes, such as investment and employment.”
• Program and course student learning outcomes (SLOs) are connected via disciplinary and institutional
outcomes e.g., content knowledge/major, general education competencies, accreditation standards.
• Although every course student learning outcome (SLO) does not have to reflect every program,
institutional and disciplinary student learning outcome, course student learning outcomes (SLOs) should
contribute in some combination to those outcomes. Mapping of course student learning outcomes
(SLOs) to program, institutional and disciplinary student learning outcomes (SLOs) helps assure there
are integrated and layered opportunities for students to develop the desired outcomes.
Adapted from: Maki, P.L. (2004). Assessing for learning: Building a sustainable commitment across the institution. Sterling, VA:
Stylus.
Purpose and Use of SLOs:
• SLOs identify skills that our students are learning. Are these the skills that we want them to learn? Are
these the skills that we are teaching them?
• SLOs enable students to demonstrate what they are learning or have learned (both in- and outside of
the classroom).
• SLOs help faculty, chairs/department heads, deans, administrators, and staff understand how to better
facilitate student learning through continuous quality improvement.
2
To consider when developing student learning outcomes (SLOs):
1. Focus on the end result – student learning.
“Upon completion of __________, the student will be able to _______”
2. Determine a limited number of student learning outcomes that are most applicable to either the
course or program. For program student learning outcomes, a general guideline would be 3-5.
3. Use concrete action verbs that result in observable behavior that can be measured (see table
below). Avoid verbs that call for students’ behavior which cannot be easily measured (e.g. exposed
to, familiar with, learn, understand or know).
Cognitive
Learning
A
Remembering:
can the student recall or
d
a
remember
the information?
p
Understanding:
can the student explain
t
e or concepts?
ideas
d
Applying:
can the student use the
information
in a new way?
f
r
o
Analyzing:
can the student distinguish
m
between the different parts?
:
Creating:
can the student create new
a
product
or point of view?
.
Examples of Action Verbs:
define, duplicate, list, memorize, recall, repeat,
reproduce state
classify, describe, discuss, explain, identify, locate,
recognize, report, select, translate, paraphrase
choose, demonstrate, dramatize, employ, illustrate,
interpret, operate, schedule, sketch, solve, use, write.
appraise, compare, contrast, criticize, differentiate,
discriminate, distinguish, examine, experiment,
question, test.
assemble, construct, create, design, develop, formulate,
write.
Affective
Learning
a
appreciate, accept, attempt, challenge, defend, dispute,
join, judge, praise, question, share, support
Psychomotor
Learning
A
write, manipulate, react
P
l
o
l
t
o Colleges (http://www.alamo.edu/pac/assessment/development/AssessmentResources.aspx)
b. Bloom, B.S. (Ed.) (1956). Taxonomy of educational objectives: The classification of educational goals: Handbook I,
cognitive domain. New York: Longmans, Green.
c. University of Louisville, Office of Academic Planning & Accountability, Institutional Effectivenes (Gilchrist, Goldstein & Bays)
Bloom’s Taxonomy of Cognitive Skills Action Verb List
4. Ensure that the student learning outcome (SLO) can be measured and that learning is being
demonstrated.
3
Examples of Student Learning Outcomes (SLO)
Student learning outcomes that are general and hard to measure1…
… will value exercise as a stress reduction tool.
… will be able to develop and apply effective problem solving skills that would enable one to
adequately navigate through the proper resources within the university.
… will demonstrate ability to resolve personal conflicts and assist others in resolving conflicts.
… will demonstrate effective communication, such as facilitating conversations about social issues.
Student learning outcomes that are specific and relatively easy to measure1 …
… will be able to explain how exercise affects stress.
… will be able to identify the most appropriate resource that is pertinent to their university concern.
… will be able to assist roommates in resolving conflicts by helping them negotiate agreements.
… will demonstrate the ability to analyze and respond to arguments about racial discrimination.
1
Adapted from University of Rhode Island, Office of Learning & Outcomes Assessment (assess@uri.edu)
Checklist to ensure that your learning outcomes are adequately stated: Do/Are the SLO statements:
 Aligned to the mission (and goal) statements
 Indicate the level and type of competence that is required of graduates of a program
 Distinctive and specific to the program
 Framed in terms of the program rather than individual courses or individual students
 Simply stated so outcomes requiring different assessment methods are not bundled into 1
statement
 Describe intended outcomes not actual outcomes
 Focus on the learning results, not the learning process
 Stated so that more than one measurement method could be used
 Measurable and there are available resources for measurement
 Useful to identify areas to improve
 Include concrete action verbs rather than passive verbs
METHODS: Traditionally, assessment methods have been categorized as either direct or indirect. These
two classifications are based on the distinction between assessing SLOs and student experience. Direct
assessors evaluate the competence of students in the program. Indirect assessors are concerned with
students’ experiences, opinions, or perceptions, rather than knowledge and skills (national, local and
program surveys).
A Resource for methods: “Measuring Quality Inventory” sponsored by AIR, ACE, and NILOA
http://applications.airweb.org/surveys/
4
DIRECT MEASURES
Performance-Based
• Capstone course assignments
• Capstone projects
• Case studies, hypothetical situation
responses
• Minute papers
• Course-embedded questions
• Portfolio assignments (standard across
program)
• Assessment of papers, projects with
standard rubric
• Research papers
• Performance appraisal of in class exercises
• Expert evaluation
Other
• Observations by evaluator (not the teacher)
• Peer evaluation of practical skills using rubric
• Clinical practice or internship skill
assessment
Examinations and Tests
• Standardized Exams & Tests
• National Test
• State Test
• Local Examinations and Tests
• Local Tests
• Pre-post test
• Test-embedded questions (across
several course sections)
•
Licensure Exams
(Adapted from the University of Central Florida Academic Program Assessment Handbook; Suskie, 2009; Maki, 2004)
Blending Direct & Indirect Evidence: Use a matrix to categorize each assessment tool as being direct or
indirect depending whether instruments are used to evaluate a student’s abilities & knowledge of academic
support services (direct) or reflect perception of academic activities (indirect).
Example 1: Matrix of Direct & Indirect measures
Targets and Timeframe: When defining the outcomes,
it is important to describe where you would like to be
within a specified time period (e.g., 10% improvement in
exam scores within one year, 90% satisfaction rating for
next year, 10% improvement in student communication
performance within two years). Also, determine what
standards are expected from students in your program.
For some learning outcomes, you may want 100% of
graduates to achieve them. This may be unrealistic for
other outcomes. You may want to determine what
proportion of your students achieve a specific level (e.g.,
80% of graduates pass the written portion of the
standardized test). If you have previously measured an
outcome, it is helpful to use this as the baseline for setting
a target for next year.
To use SLO assessment data to facilitate student learning through quality improvement:
• Involve your faculty in planning how to best use results to improve teaching and learning.
• Use results to demonstrate accountability by sharing results internal & externally.
• Address disappointing assessment in positive and developmental manner.
5