Making It Meaningful:
Authentic Assessment for
Intentional Education
David W. Marshall, PhD
Joanna M. Oxendine, MEd
Overview
 Two Cultures: A Contrast in Emphasis
 Defining Outcomes
 Principles of Meaningful Assessment
 Making Meaningful Change
Two Cultures: A
Contrast in Emphasis
CCSF’s Assessment Cycle
The Assessment Cycle
Strategize
Program
Improvement
Write
Outcomes
Identify
Assessment
Measures
Analyze
Results
Gather
Evidence
Perception of the Assessment Cycle
SUBMIT
REPORTS
Write
Outcomes
Identify
Assessment
Measures
PACKAGE
RESULTS
Gather
Evidence
 Sees accreditation as an end in itself
The Culture of
Compliance
Students become unimportant
elements of the assessment process
 Seeks information on what accreditors want to
see
 Worries about whether what they have matches
accreditors’ expectations
Another View of the Assessment Cycle
Plan for
Intentional
Improvements
Articulate
Intended
Outcomes
Determine
Extent to
Which
Outcomes
Were Met
Work Toward
Intended
Outcomes
Gather
Evidence
 Is student-centered
The Culture of
Intentionality
Students become the primary focus of
the assessment process
 Seeks information about how well students are
learning and/or how well various areas of
the college are supporting the student
experience
 Reflects on what we teach or do and how we
teach or do it
 Accepts (some) responsibility for student
learning and the student experience
 Experiments with new strategies for student
success
Evidenced-Based Change
in the Context of a Culture
of Intentionality
What is Evidence-Based Change?
Evidence-based change is:
An approach to organizational activities and decision-making
that uses information about the effectiveness of those
activities and decisions to implement strategic and targeted
revisions towards increased impact of the organization’s key
goals.
What is Evidence-Based Change?
Evidence-based change is:
An approach to organizational activities and decision-making
that uses information about the effectiveness of those
activities and decisions to implement strategic and targeted
revisions towards increased impact of the organization’s key
goals.
What is Evidence-Based Change?
Evidence-based change is:
An approach to organizational activities and decision-making
that uses information about the effectiveness of those
activities and decisions to implement strategic and targeted
revisions towards increased impact of the organization’s key
goals.
What is Evidence-Based Change?
Evidence-based change is:
An approach to organizational activities and decision-making
that uses information about the effectiveness of those
activities and decisions to implement strategic and targeted
revisions towards increased impact of the organization’s key
goals.
The Organization’s Key Goals??
Plan for
Intentional
Improvements
Articulate
Intended
Outcomes
Determine
Extent to
Which
Outcomes
Were Met
Work Toward
Intended
Outcomes
Gather
Evidence
Defining Shared Purpose
An approach to organizational activities and decision-making
that uses information about the effectiveness of those
activities and decisions to implement strategic and targeted
revisions towards increased impact of the organization’s key
goals.
An Analogy
Those who:
 Pray/Teach
 Fight/Defend
 Farm/Provide Food
Mutuality
Taking the Analogy Too Far
Defining Outcomes
 Student Learning Outcomes
Two Types of
Outcomes
Outcomes indicate means by which
success in achieving goals might be
defined.
 Statements that describe the knowledge, skills,
attitudes, and habits of mind that students take
with them from a learning experience (adapted
from Suskie, 2009)
 Service Area Outcomes
 Statements that describe the intended results of
specific programmatic, operational, or
administrative activities
 Two types at CCSF
 Student Service Outcomes: statements that identify
what a student will be able to know and/or do after
receiving a service and/or participating in student
support programs
 Administrative Unit Outcomes: statements directly
related to the college’s operations and non-academic
services that support the learning environment
Intentionality and Outcomes
From the student perspective, outcome articulation:
Two Types of
Outcomes
Outcomes indicate means by which
success in achieving goals might be
defined.
 Makes explicit for students what is expected of
them so they know what to expect.
 Removes the need for students to intuit what
learning is expected.
From the college’s perspective, outcome
articulation:
 Directs efforts to evaluate how successful
activities have been.
 Helps to identify what kind of information to
collect.
Outcomes and the Two Cultures
CULTURE OF COMPLIANCE
CULTURE OF INTENTIONALITY
 Rarely communicates outcomes to students
 Makes outcomes visible to students
 Files outcomes with the appropriate office
or administrative oversight entity
 Incorporates outcomes into practice
 Sticks with what has always been done
 Works on outcomes assessment for an
accreditation cycle
 Assesses outcomes appropriately
 Uses outcomes for ongoing conversations
about effectiveness and continual
improvement
Outcomes and the Two Cultures
CULTURE OF COMPLIANCE
CULTURE OF INTENTIONALITY
 Rarely communicates outcomes to students
 Makes outcomes visible to students
 Files outcomes with the appropriate office
or administrative oversight entity
 Incorporates outcomes into practice
 Sticks with what has always been done
 Works on outcomes assessment for an
accreditation cycle
 Assesses outcomes appropriately
 Uses outcomes for ongoing conversations
about effectiveness and continual
improvement
Principles of Meaningful
Assessment
1. What do we want students to know, understand, and be
able to do?
What are the intended results of our programmatic,
operational, or administrative activities?
A Process of
Questions
The Culture of Intentionality opens a
clear approach to assessment.
2. Where do students learn what we expect them to learn?
How do we accomplish what we set out to do?
3. How well did students learn what we expected them to
learn?
How well did we do what we intended to do?
4. How do we know how well they learned what we expected
them to learn?
How do we know how well we did what we expected to
do?
5. What does our evidence tell us? How can we use what our
evidence tells us to make appropriate decisions?
1. What do we want students to know, understand, and be
able to do?
What are the intended results of our programmatic,
operational, or administrative activities?
A Process of
Questions
The Culture of Intentionality opens a
clear approach to assessment.
2. Where do students learn what we expect them to learn?
How do we accomplish what we set out to do?
3. How well did students learn what we expected them to
learn?
How well did we do what we intended to do?
4. How do we know how well they learned what we expected
them to learn?
How do we know how well we did what we expected to
do?
5. What does our evidence tell us? How can we use what our
evidence tells us to make appropriate decisions?
1. What do we want students to know, understand, and be
able to do?
What are the intended results of our programmatic,
operational, or administrative activities?
A Process of
Questions
The Culture of Intentionality opens a
clear approach to assessment.
2. Where do students learn what we expect them to learn?
How do we accomplish what we set out to do?
3. How well did students learn what we expected them to
learn?
How well did we do what we intended to do?
4. How do we know how well they learned what we expected
them to learn?
How do we know how well we did what we expected to
do?
5. What does our evidence tell us? How can we use what our
evidence tells us to make appropriate decisions?
1. What do we want students to know, understand, and be
able to do?
What are the intended results of our programmatic,
operational, or administrative activities?
A Process of
Questions
The Culture of Intentionality opens a
clear approach to assessment.
2. Where do students learn what we expect them to learn?
How do we accomplish what we set out to do?
3. How well did students learn what we expected them to
learn?
How well did we do what we intended to do?
4. How do we know how well they learned what we expected
them to learn?
How do we know how well we did what we expected to
do?
5. What does our evidence tell us? How can we use what our
evidence tells us to make appropriate decisions?
1. What do we want students to know, understand, and be
able to do?
What are the intended results of our programmatic,
operational, or administrative activities?
A Process of
Questions
The Culture of Intentionality opens a
clear approach to assessment.
2. Where do students learn what we expect them to learn?
How do we accomplish what we set out to do?
3. How well did students learn what we expected them to
learn?
How well did we do what we intended to do?
4. How do we know how well they learned what we expected
them to learn?
How do we know how well we did what we expected to
do?
5. What does our evidence tell us? How can we use what our
evidence tells us to make appropriate decisions?
1. What do we want students to know, understand, and be
able to do?
What are the intended results of our programmatic,
operational, or administrative activities?
A Process of
Questions
The Culture of Intentionality opens a
clear approach to assessment.
2. Where do students learn what we expect them to learn?
How do we accomplish what we set out to do?
3. How well did students learn what we expected them to
learn?
How well did we do what we intended to do?
4. How do we know how well they learned what we expected
them to learn?
How do we know how well we did what we expected to
do?
5. What does our evidence tell us? How can we use what our
evidence tells us to make appropriate decisions?
Making Meaningful
Change
If an outcome meets expectations at the standard:
 Take no action
Responding to
the Results
Assessment may find that student
learning or a program meets
expectations at the determined
standard for some outcomes.
 Identify this area as a program strength
 Consider increasing expectations outlined in the outcome
 Raise the standard of attainment
 Consider surveying students about their experience of the
program or service area
If results for an outcome do not meet expectations:
 ASK QUESTIONS!!
Responding to
the Results
Assessment may find that student
learning or a program does not meet
expectations at the determined
standard for some outcomes.
 Are our outcomes aligned with our expectations?
 What aspect of our program or activities might be in need of
revision?
 Are our teaching and learning methods in need of revision?
 With whom might we need to work to strengthen our
outcomes?
 What crazy idea might we come up with to tackle the problem?
CCSF Progress on NEW ACCJC STANDARD for
Disaggregated SLO Data
CRN: 31251
Daphne:
CRN:Met
31252
Jerome:
Daphne:
CRN:
Developing
Met
31253
Janet:
Jerome:
Not
Daphne:
assessed
CRN:
Developing
Met
31254
Ben:Janet:
Met
Jerome:
Not
Daphne:
assessed
CRN:
Developing
Met
31255
Su-Lin:
Ben:Developing
Janet:
Met
Jerome:
Not
Daphne:
assessed
CRN:
Developing
Met
31256
Ahmed:
Su-Lin:
Ben:
NotDeveloping
Janet:
assessed
Met
Jerome:
Not
Daphne:
assessed
Developing
Met
Ahmed:
Su-Lin:
Ben:
NotDeveloping
Janet:
assessed
Met
Jerome:
Not assessed
Developing
Ahmed:
Su-Lin:
Ben:
NotDeveloping
Janet:
assessed
Met Not assessed
Ahmed:
Su-Lin:
Ben:
NotDeveloping
assessed
Met
Ahmed:
Su-Lin:
NotDeveloping
assessed
Ahmed: Not assessed
TOTAL ACROSS ALL SECTIONS
Meets the outcome: 287 (74%)
Developing the outcome: 74 (20%)
No evidence: 21 (6%)
PROGRAM
IMPROVEMENTS!
MEETS THE SLO:
Online: 45 (85%)
Face-to-Face: 242 (74%)
MEETS THE SLO:
BOG Waiver: 192 (78%)
No BOG Waiver: 65 (72%)
MEETS THE SLO:
Morning: 146 (78%)
Afternoon: 96 (68%)
MEETS THE SLO:
Male: 161 (77%)
Female: 123 (73%)
BIO 9 CurricUNET Data Processed by Office of Research (ARGOS)
SLO: Critique scientific information in the media for accuracy and reliability.
287 Meets SLO
(74%)
Male:
161 (77%)
Female:
123 (73%)
Morning: 146
(78%)
Afternoon: 96
(68%)
BOG Waiver:
192 (78%)
No BOG Waiver: 65
(72%)
Online:
45 (85%)
Face-to-face: 242
(74%)
74 Developing
SLO (20%)
Male:
37 (22%)
Female:
36 (17%)
Morning: 34
(18%)
Afternoon: 53
(23%)
BOG Waiver:
57 (21%)
No BOG Waiver: 15
(18%)
Online:
5 (9%)
Face-to-face: 67
(20%)
21 No evidence
of SLO (6%)
Male:
8 (5%)
Female:
13 (6%)
Morning: 10
(5%)
Afternoon: 12
(8%)
BOG Waiver: 18(5%)
No BOG Waiver: 3
(4%)
Online:
3 (6%)
Face-to-face: 22
(7%)
For meaningful change to occur…
Faculty and staff must be the driving force and they must work collectively!
 Faculty and staff:
 know the curriculum or systems in which they work.
 are responsible for seeing that outcomes are met.
 will be the ones who innovate practice.
 The administration:
 creates space for the work to occur.
 fosters the culture of intentionality.
Plan for
Intentional
Improvements
Articulate
Intended
Outcomes
Responding to
the Results
Students benefit from an institution’s
thoughtful response to an honestly
undertaken attempt to determine a
program’s strengths and weaknesses.
Determine
Extent to
Which
Outcomes
Were Met
Work Toward
Intended
Outcomes
Gather
Evidence
To Conclude
With an approach that strives for student-centered intentionality, compliance becomes a matter
of simply completing forms and writing narratives, thereby accreditation takes care of itself.
Plan for
Intentional
Improvements
Articulate
Intended
Outcomes
Determine
Extent to
Which
Outcomes
Were Met
Work Toward
Intended
Outcomes
Gather
Evidence