Assessment of Student Learning
An Integrated View
Why Assess?
“When we care about something , we measure it.” Diane Halpern
Why Assess?
…to improve student learning.
Why Assess?
…to improve student learning.
“Assessment is an ongoing process aimed at understanding
and improving student learning…systematically gathering,
analyzing, and interpreting evidence to determine how well
performance matches those expectations and standards …
using the resulting information to … improve performance …
assessment can help us focus our collective attention, examine
our assumptions, and create a shared academic culture
dedicated to assuring and improving the quality of higher
education” (Angelo, AAHE Bulletin, November 1995, p. 7).
Why Assess?
…to improve student learning.
is an ongoing process aimed at understanding
and improving student learning…systematically gathering,
analyzing, and interpreting evidence to determine how well
performance matches those expectations and standards…using
the resulting information to … improve performance …
assessment can help us focus our collective attention,
examine our assumptions, and create a shared academic
culture dedicated to assuring and improving the quality of
higher education” (Angelo, AAHE Bulletin, November 1995, p. 7).
“Assessment
Multiple Assessment Stakeholders
General Education SLO
Assessment
Course-specific
SLO Assessment
Program-specific
SLO Assessment
What is an SLO?
Student Learning Outcome (SLO) – An outcome that
describes what a student is expected to learn as a result
of participating in academic activities or experiences.
* Operational phrasing of learning outcomes is a crucial part of the process *
SLOs focus on knowledge gained, skills and abilities acquired or demonstrated, and attitudes or values changed.
Where do SLOs come from?
University Vision/Mission/Strategic Priorities
The University is committed to assist students in their search for knowledge and understanding and to
prepare them with the attitudes, skills, and habits of lifelong learning in order to assume responsibility in a
democratic community and to be useful members of a global society.
College Mission & Goals
The College mission is to…deepen understanding of behavioral, social, cultural, and environmental issues.
Program-specific Goals
Students will exercise critical thinking, problem solving strategies, and scientific methodology in their
academic studies and everyday lives.
Course-specific Goals
“Students will be able to effectively critique ‘lay persons’ and ‘technical’ article on the same topic.”
(Method: Embedded; Evaluation Criteria: Rubric; Benchmark: 70% of students rated as at least “adequate”)
Note.
Accredited programs usually have their own competency-based/SLO requirements.
GE-related SLOs may also have to be taken into consideration.
Selecting which SLOs to Assess
Consider the stakeholders
 General Education / Pathway SLOs
 Program-specific SLOs
 Course-specific SLOs
 Key questions to consider
I.
What knowledge, skills, and values is a graduate from the
program expected to acquire?
II. How to determine whether the student has achieved some
minimum standard of performance?
The Big Picture
Creating a Curriculum Matrix
• Cross index courses and program SLOs
 Specifies the courses where competency for each SLO is “introduced”, “practiced”, and
“demonstrated”
 Guides which courses are appropriate SLO assessment targets
 Helps to ‘close the loop’
• Should specify where SLO competencies are
Introduced
Practiced
Demonstrated
Criminal Justice (BA)
Curriculum Matrix
Criminal Justice
Course
Outcome 1
Critical
Thinking
Outcome 2
Oral
Communication
Outcome 4
Content
Knowledge
Outcome 5
Understand
Diversity
Outcome 6
Understand
Ethics
I
I
I
I
Demonstrate (D)
I
POLS 250
Introduce (I)
POLS 257
Practice (P)
P
P
POLS 259A
P
P
P
POLS 352
P
P
P
POLS 353
P
P
P
P, D
P, D
I, P
P
P
POLS 459D
P, D
SOCI 384
P
Electives
P
POLS 331
I
Outcome 3
Effective
Writing
P, D
P
I
D
P
P
PHIL 332
P
P
CJ & the Individual
P
P
I
CJ & Society
P
P
P
Theoretical
Foundations
D
P
Courts & CJ
System
D
D
BA in Criminal Justice curriculum matrix. Shown with permission.
P, D
Economics (BA)
Curriculum Matrix
Goal:
Student Learning Outcomes
1.
Be able to apply the microeconomic
theory to explain and predict price
changes and economic behavior in
individual markets.
I = Introduce
P = Practice
C = Competency
Courses
101, 102, 103
301, 330, 335, 340, 253/333/ 355, 360, 365,
370, 375, 410
435, 440, 450, 462, 465, 466, 470, 495
I
P
2. Be able to apply mainstream
macroeconomic theory to explain and
predict events in the aggregate economy,
including roles played by fiscal and
monetary policies.
101, 102, 103
302, 330, 70, 375, 376, 410
431, 470, 483, 495
I
P
C
3. Be able to identify economic issues and
problems, gather data needed to
evaluate them, and analyze the data to
gain insights into economic behavior and
formulate possible solutions.
301, 302
335, 340, 352/333, 360, 365, 376, 389
431, 435, 440, 450, 462, 465, 495
466, 470, 481, 482, 483, 495
I
P
C
4. Be able to communicate with written and
spoken word in the discipline of
economics.
101, 102, 103
301, 302, 305, 307, 320, 340, 352/333, 355,
360, 365, 370, 376, 399, 410
431, 435, 440, 450, 462, 465, 466, 481,
483, 495
I
P
BA in Economics curriculum matrix. Shown with permission.
C
C
Operationalizing SLOs
“write clear, correct, and appropriate sentences
and paragraphs, avoiding major grammatical
and semantic problems.”
What are your thoughts about this SLO?
Operationalizing SLOs
“write clear, correct, and appropriate sentences
and paragraphs, avoiding major grammatical
and semantic problems.”
Is vague (what constitutes ‘appropriate’?)
Is an example of ‘double barreling’
Rubric evaluation of an embedded assignment is
the most likely method and evaluation
What do you want to know?
• Three broad SLO categories
 Knowledge SLOs (most commonly assessed)
 Bloom’s Taxonomy for example of Knowledge domains to assess
 Skill SLOs (quite commonly assessed)
 Writing and Oral communication are typical Skill SLOs
 Values / Attitude SLOs (least commonly assessed)
 Attitude surveys have been successfully used
It’s all about specificity
• Data targeted for assessment should align
with competencies specified in the SLO
 Rubrics, rubrics; everywhere rubrics
• Information should be ‘actionable’
 Does evaluation of the data provide information
as to where improvements can be made?
• Bottom line… can you close the loop?
Setting Achievement Standards
 Benchmark acceptable standard of
achievement (e.g., based on rubric)
Percentage meeting some a priori level of
achievement (e.g., 70% of students achieving
at least ‘adequate’ on rubric; 70% of students
correctly answering embedded question, etc.)
Thinking about Student Learning Outcomes:
Closing the Loop
Assessment Methods and Data
• Embedded Assessment
 Target exam items, assignments, key course components
 Common final (not just overall grade; target exam concepts)
• Pre- and Post-test comparisons (“big idea” items)
• Third Party Assessments
 Professional exams
• Portfolios
• Attitude Surveys
Assessment Schedule
• Develop an overall SLO plan
 Align with program’s 5 year review or other review cycle
• Consider evaluating more than one SLO per
AY cycle
 A single artifact (e.g., final paper) can be used to assess
more than one SLO
Final Thoughts
• Clearly operationalize your SLOs
• What you measure should allow you to “Close
the Loop”
• Assess all SLOs within a review cycle
• Make it a point to share assessment results
during department meetings and/or with other
stakeholders
“If we care about something,
then we should measure it.”
Diane Halpern