WRITING MEASURABLE
STUDENT LEARNING
OUTCOMES
Dr. Timothy S. Brophy
Director of Institutional Assessment
University of Florida
TODAY’S GOALS
 Describe and explain SACS accreditation expectations for
academic program assessment
 Identify and apply steps for developing measurable student
learning outcomes
 Develop and/or refine student learning outcomes for your
degree program
WHAT IS SACS AND WHY IS IT
IMPORTANT TO BE ACCREDITED?
 SACS-COC = the Southern Association of Colleges
and Schools’ Commission on Colleges
 SACS is the Federally -approved accrediting body for
southern region of the US
 SACS develops policies and standards that
operationalize Federal Regulations
 Federal Student Aid is tied to our reaffirmation –
without accreditation we lose this important
funding source
Selected Student Financial Aid Data for University of Florida Undergraduate Students for the
Three Most Recently Available Years
Academic Year
2008-09
All undergraduate students
Pell Grants
Federal Loans
2009-10
$
30,894,352
$
41,996,944
$
46,733,613
$
60,969,318
$
59,795,534
$
59,470,923
Full-time, first-time
degree/certificate-seeking
undergraduate students
Federal Grants
2010-11
$106,204,536
$
6,512,300
$
10,400,395
$
11,826,734
Pell Grants
$
5,471,084
$
8,097,580
$
9,014,885
Other Federal Grants
$
1,041,216
$
2,302,815
$
2,811,849
$
6,651,786
$
7,457,421
$
7,362,376
Federal Loans
Source: IPEDS Student Financial Aid Component
HISTORICAL CONTEXT
THEN:
2003 Self Study
THE
TRANSITION:
2003-present
NOW:
Data–driven
Continuous
Improvement
Volume 1:
Six required
sections
Evolving
Standards and
Expectations
Compliance
Volume 2:
International
Focus report
Paradigm shift
Quality
Enhancement
Student
Learning
SCOPE OF THE SACS PRINCIPLES
12 Core
Requirements
35
subcomponents
14
Comprehensive
Standards
73
subcomponents
9 Federal
Requirements
SACS STANDARD 3.3.1.1
3.3.1 - The institution identifies expected
outcomes, assesses the extent to which it
achieves these outcomes, and provides
evidence of improvement based on analysis of
the results in each of the following areas:
(Institutional effectiveness)
3.3.1.1 educational programs, to include
student learning outcomes
WHAT SACS EXPECTS
Completion of at
least one complete
SLO assessment
cycle
Evidence and
documentation of
compliance with
all requirements
and standards
Documented
evidence that the
“improvement
cycle” is complete
THE UF ASSESSMENT PROCESS
Modify and
Improve
Interpret and
Evaluate the Data
Assessment
Planning
Implement the
Plan and Gather
Data
Assessment
Institutional
Effectiveness
Establish Goals
and Outcomes
ACADEMIC ASSESSMENT PLAN
STUDENT LEARNING OUTCOMES AT UF
 March 29, 2007 – Board of Governors regulation 8.016,
“Academic Learning Compacts”
 “Research indicates that university students are served
best when students and faculty fully engage in a
teaching-learning partnership, and this partnership is
all the more meaningful if it is made as clear as
possible to students what it is they will learn and how
program faculty will assess that learning. Therefore, the
Board has determined that universities must develop
“Academic Learning Compacts” and related assessment
processes to define and demonstrate student
achievement in baccalaureate degree programs in the
State University System.”
 BOG Regulation 8.016, revised 1-19-12, “Student Learning
Outcomes Assessment”
IDENTIFYING SLOS AND
PROGRAM GOALS
DEFINITIONS
Student Learning
Outcomes
(SLOs)describe
student learning –
what students will
know and be able to
do as a result of
completing a UF
program
Program Goals do
not describe student
learning – instead,
they describe
programmatic
elements, such as
admission criteria,
acceptance and
graduation rates, etc
OUTPUTS OR OUTCOMES?
DEFINITIONS
Outputs describe
and count what we
do and whom we
reach, and represent
products or services
we produce.
Processes deliver
outputs; what is
produced at the end
of a process is an
output.
An outcome is a
level of performance
or achievement. It
may be associated
with a process or its
output. Outcomes
imply measurement
- quantification - of
performance.
OUTCOMES AND OUTPUTS: WHAT IS THE
DIFFERENCE?
This distinction is important, especially in the development and review of
Student Learning Outcomes.
We seek to measure outcomes as well as their associated outputs; however,
SLOs focus on outcomes.
For example, while we produce a number of new graduates (the output), it is
critical that we have a measure of the quality of the graduates as defined by
the college or discipline (the outcome).
Effective Student Learning Outcomes describe, in measurable terms, these
quality characteristics by defining our expectations for knowledge, critical
thinking, and communication for UF undergraduates, and knowledge, skills,
and professional behaviors for graduate and professional students,.
CHARACTERISTICS OF EFFECTIVE
STUDENT LEARNING OUTCOMES
 Focus on what students will know and be able to do.
 All disciplines have a body of core knowledge that students must learn to
be successful as well as a core set of applications of that knowledge in
professional settings.
 Describe obser vable and measureable actions or behaviors.
 Effective SLOs present a core set of observable, measureable behaviors.
Measurement tools vary from quizzes and tests to complex rubrics .
 The key to measurability: an active verb that describes a
observable behavior, process, or product
 A framework for developing SLOs: Bloom’s Taxonomy (see Table
3 in your Student Learning Outcomes handout)
VERBS AND PHRASES THAT COMPLICATE
MEASURABILIT Y
 Understand
 An internal process that is indicated by demonstrated behaviors – OK for
ALCs but not recommended for program or course SLOs
 Appreciate; value
 Internal processes that are indicated by demonstrated behaviors closely
tied to personal choice
 Become familiar with
 Focuses assessment on “becoming familiar,” not familiarity
 Learn about, think about
 Not observable; demonstrable through communication or other
demonstration of learning
 Become aware of, gain an awareness of
 Focuses assessment on becoming and/or gaining – not actual awareness
 Demonstrate the ability to
 Focuses assessment on ability, not achievement or demonstration of a
skill
DIRECT OR INDIRECT
ASSESSMENT?
DEFINITIONS
Direct assessments
of student learning
are those that
provide for direct
examination or
observation of
student knowledge
or skills against
measurable
performance
indicators.
Indirect
assessments are
those that ascertain
the opinion or selfreport of the extent
or value of learning
experiences
DEVELOPING MEASURABLE SLOS:
A THREE-LEVEL MODEL (CARRIVEAU, 2010)
Program Learning Goal Level – programs establish learning goals for the degree
these are described in the Academic Learning Compact, Program mission, or Catalog entry
Program-level Student Learning Outcome
these describe what students will do to demonstrate they have met the learning goals
Course-level Student Learning Outcome
these are determined by the faculty and specify course-level, observable products or demonstrations
This model allows you to develop assessments that measure the outcomes, and that then connect
directly to the program learning goals
LEVEL 1: ESTABLISHING LEARNING
GOALS FOR THE DEGREE
Learning Goals – these are found in the
Academic Learning Compact in the
description of the major
Example: Materials Science and
Engineering
The major enables you to develop an
understanding of materials systems and
their role in engineering. Emphasis is
placed on the ability to apply knowledge
of mathematics, science and engineering
principles to materials science and
engineering; to design and conduct
experiments, as well as to analyze and
interpret data; and to design a system,
component or process to meet desired
needs within realistic constraints such as
economic, environmental, social, political,
ethical, health and safety,
manufacturability and sustainability.
Source: 2012-13 UF Undergraduate Catalog,
https://catalog.ufl.edu/ugrad/current/engineering/alc/materials-science-andengineering.aspx
LEVEL 1: LEARNING GOALS BASED ON
THE MSE ALC
Students who complete the MSE degree will:
Understand materials systems and their role in
engineering
Apply knowledge of mathematics, science and
engineering principles to materials science and
engineering to design and conduct experiments, as
well as to analyze and interpret data
Design a system, component or process to meet
desired needs within realistic constraints such as
economic, environmental, social, political, ethical,
health and safety, manufacturability and sustainability
LEVEL 2 – PROGRAM STUDENT
LEARNING OUTCOMES FOR MSE
Content Knowledge
•Apply knowledge of mathematics, science and engineering principles to
materials science and engineering.
•Design and conduct materials science and engineering experiments and
analyze and interpret the data.
Critical Thinking
•Design a materials science and engineering system, component or process
to meet desired needs within realistic economic, environmental, social,
political, ethical, health and safety, manufacturability and sustainability
constraints.
Communication
•Communicate technical data and design information effectively in speech
and in writing to other materials engineers.
MSE: CONNECTING GOALS TO OUTCOMES
Goal
ALC Learning Goals:
Understand materials systems and their
role in engineering
Design a system, component or process to
meet desired needs within realistic
constraints such as economic,
environmental, social, political, ethical,
health and safety, manufacturability and
sustainability
Student Learning Outcomes:
Design a materials science and engineering
system, component or process to meet
desired needs within realistic economic,
environmental, social, political, ethical,
health and safety, manufacturability and
sustainability constraints.
Communicate technical data and design
information effectively in speech and in
writing to other materials engineers
SLO
MSE: CONNECTING GOALS TO OUTCOMES
Level 1
ALC Learning Goals:
Understand materials systems and
their role in engineering
Apply knowledge of mathematics,
science and engineering principles to
materials science and engineering to
design and conduct experiments, as
well as to analyze and interpret data
Student Learning Outcomes:
Apply knowledge of mathematics,
science and engineering principles to
materials science and engineering.
Design and conduct materials
science and engineering experiments
and analyze and interpret the data
Level 2
CONNECTING PROGRAM SLOS TO COURSES
MSE CURRICULUM MAP
Additional
Assess-ments
SLOs
Content
Knowledge
EMA3050
EMA3066
EMA4714
#1
I
R
A
#2
EMA3080C
EMA3066
EMA4223
EMA4714
#3
I
R
A
Communication
EMA3080C
EMA3013C
EMA3513C
#4
I
R
A
EMA4714
Senior exit
survey
I
Critical
Thinking
EMA3513C
R
A
Assessments in the boxes marked A are conducted using specific
homework, exam, or assignment questions aligned with that SLO.
Source: 2011-12 MSE Academic Assessment Plan
Senior exit
survey
Senior exit
survey
Senior exit
survey
LEVEL 3 – COURSE LEVEL SLOS
These are determined by the faculty to teach
the course
However, these should directly relate to the
program SLOs
WRITING STUDENT LEARNING OUTCOMES
(use with the UF Student Learning Outcome Guide)
1. Review the Academic Learning Compact. List the learning
goals for the program that are in the ALC.
2. Review the current SLOs for your area with your program
faculty for recency, relevance, and rigor.
3. Examine the SLOs for the Knowledge Type (see Table 1) and
Cognitive Processes level (see Table 2) they engage. The
majority of the SLOs should be in the upper three levels of
the Cognitive Processes Dimension – Analyze, Evaluate, and
Create. The Taxonomy template in Figure 1 may help with
this process.
4. Cross-reference your SLOs with the list of verbs/actions
associated with their corresponding cognitive dimension
levels (see Table 3), and replace any “verbs and phrases to
avoid” with appropriate verbs from Table 3.
5. Write the SLO concisely and clearly.
ADDITIONAL RESOURCES
 Carriveau, R. (2010). Connecting the dots – Developing
student learning outcomes and outcomes -based assessments.
Denton, TX: Fancy Fox Publications
 Suskie, L. (2009). Assessing student learning: A common
sense guide (2 nd ed.) San Francisco, CA: Jossey -Bass.
 Walvoord, B. (2010). Assessment clear and simple: A practical
guide for institutions, depar tments, and general education
(2 nd ed.). San Francisco, CA: Jossey -Bass.
You can always find our UF -specific Institutional Assessment
resources at our website , http://assessment.aa.ufl.edu /