How to Develop and/or
Update A Simple and Clear
Assessment Plan
Office of Academic Program
Assessment
Spring 2013
Overview






Introduction
Definition of Basic Terms
Differences Between Goals and Outcomes
Criteria for Evaluating Goals and Outcomes
Using Rubrics in PLOs
The Next Step: Developing the Assessment
Plan
Handouts
1.
2.
3.
4.
5.
6.
PowerPoint slides
WASC Rubric for Assessing the Quality of
Academic Program Learning Outcomes
Examples of Goals and Outcomes
Curriculum map (Sociology)
Rubric (BA Nutrition and Food)
VALUE Rubric on Critical Thinking
Learning Goal
“a high-level, very general statement of learning
expected of graduates, aligned with the
institution’s mission, vision, and values (more
specific learning outcomes are derived from
goals)”
(WASC Draft 2013 Handbook of Accreditation, p. 45)
Learning Outcome
“a concise statement of what the student should know
or be able to do. Well-articulated learning outcomes
describe how a student can demonstrate the desired
outcome; verbs such as “understand” or “appreciate”
are avoided in favor of observable actions, e.g.,
“identify,” “analyze.” Learning outcomes can be
formulated for different levels of aggregation and
analysis. Student learning outcomes are commonly
abbreviated as SLOs, course learning outcomes as
CLOs, program learning outcomes as PLOs, and
institution-level outcomes as ILOs.”
(WASC Draft 2013 Handbook of Accreditation, p. 48)
Assessment Plan
Assessment—an ongoing, iterative process
consisting of four basic steps:
1.defining learning outcomes
2.choosing a method or approach and then
using it to gather evidence of learning
3.analyzing and interpreting the evidence
4.using this information to improve student
learning
(WASC Draft 2013 Handbook of Accreditation, p. 40)
Qualities of Goals
(adapted from Graves, 2000)






General but not vague
Transparent to all stakeholders (no jargon)
Realistic
Simple—usually not multifaceted
Something the program addresses
Stated in terms of the learner
Qualities of Outcomes






(adapted from Graves, 2000)
More specific than goals
Directly relate to goals
 hierarchical relationship
 cause-effect relationship
Focus on what students learn and/or
associated processes
More objectives than goals
Stated in terms of the learner
Transparent to all stakeholders
More Qualities of Outcomes
(WASC Rubric for Assessing the Quality of
Academic Program Learning Outcomes)

Complete & well-organized
 Both discipline-specific and institution-wide
outcomes
 Clearly distinguish between levels (e.g.,
undergraduate versus graduate)
 Have explicit criteria statements developed
by faculty (e.g., rubrics)
Example:
BA General Communication
GOAL:
 Students will communicate effectively in a
variety of contexts
SUB-GOALS:
1. Demonstrate proficiency in oral
communication
2. Demonstrate proficiency in written
communication
Example:
BA General Communication
Learning Outcomes for Sub-Goal 1
(Demonstrate proficiency in oral communication)
1.determine presentation needs in different
situations
2.correctly use visual aids
3.make appropriate language choices
4.use proper structure
5.effectively deliver presentations
Example:
BA General Communication
Learning Outcomes for Sub-Goal 2
(Demonstrate proficiency in written communication)
1.implement a variety of style sheets
2.use thesis statements
3.use appropriate organizational strategies
4.apply transitions
5.include appropriate evidentiary support material
6.employ grammar conventions
The Importance of Verbs
(Mager, 1975, cited in Brown, 1995)
Multiple Interpretations
 to know
 to understand
 to really understand
 to appreciate
 to fully appreciate
 to grasp the significance
of
 to enjoy
 to believe
 to have faith in
Fewer Interpretations
 to write
 to recite
 to identify
 to sort
 to solve
 to construct
 to build
 to compare
 to contrast
Criteria for Evaluating Goals
Have program goals been specified?
1. Is each goal comprehensible?
2. Is each goal justified?
– represents institutional goals
– represents what faculty and/or discipline feel to
be important
3. Is each goal stated in terms of the learner?
4. Will all goals actually be addressed in the program?
5. Can all goals be achieved by students in the
program?
Criteria for Evaluating Outcomes
Have program learning outcomes (PLOs) been
developed?
1. Does the program specify clear PLOs?
2. Are there distinct differences in the level of
specificity between program goals and program
learning outcomes?
3. How will achieving program learning outcomes help
to reach program goals?
4. Are PLOs achievable within the curriculum?
Criteria for Evaluating Outcomes
5. Have each of the following been considered in
developing PLOs?:
– Do PLOs focus on the key knowledge, skills and
values students learn in the program?
– Are the verbs in PLOs clear and unambiguous?
Do they describe the observable behaviors by
which students will demonstrate learning?
– Are the criteria for performance on each PLO
explicit and transparent? (Are there benchmarks?)
– How and where can you observe each PLO?
– What expectations do you have for student
achievement for each PLO?
Rubrics for PLO Specification

Especially for skills-based outcomes
 Rubrics allow faculty to agree on explicit criteria
statements that


describe observable behaviors
differentiate levels of performance
• e.g. , Cultural and global awareness/sensitivity
including demonstrated understanding, respect
and
• support of multiple perspectives from other
disciplines, societies, individuals, groups, and
cultures
Example Rubric
BA in Nutrition and Food
Awareness of the integration of the
different concentrations in FACS and their
importance to the relationships between
humans and their diverse environments
as individuals and groups as a whole.
Rubric Development


Time-consuming
Look at existing rubrics
e.g., VALUE rubrics can be used ...

without change
 with revisions
 as a heuristic
Group Discussion
Your program includes “critical thinking” as a
learning goal and has decided to assess it this
year. You have not yet specified PLOs or criteria
for this goal.
 Look at the VALUE rubric
1. How useful would this rubric be to your program
for establishing PLOs and explicit criteria?
2. Choose one dimension. How would you need to
revise the rubric to represent how that
dimension is realized in your program?

Debrief
1. What did you learn about the process
of specifying PLOs and criteria from
doing this?
2. What did you learn from trying to adapt
the rubric?
The Next Step:
Sustainable Assessment Plans
Good Assessment Plans:
 Focus on the program (e.g., majors and
concentrations) rather than courses
 Have a limited number of goals and outcomes
 Are developed collaboratively
 Are ongoing
 Are manageable
 Use multiple methods of data collection
 Anticipate how results will be used and lead to
improvement
Comprehensive Assessment
Planning
Include both content area knowledge and
skills/competencies
Multi-year plan with 1-2 outcomes assessed
each year
Direct and indirect methods
Capstone student projects (direct)
Student exit surveys: self-reporting on all
outcomes
Writing an effective
assessment Plan
Outcome
Assessed
and/or
Assessment
Question
Method(s) to Collect
Evidence
Brief description of
what will be collected,
how, and by whom.
(Enter one method
per row; add rows as
needed.)
LO 1
Capstone
Assignment (Direct)
LO 1
Exit Survey
(indirect)
Method to Analyze/
Evaluate*
Brief description of
how the evidence
will be analyzed or
evaluated and by
whom. (When
applicable, include
scoring criteria or
rubric in an
Appendix.)
Include sample
size, scoring
criteria, expected
outcomes, etc.
Timeline & Status
List the
semester/dates
when the evidence
will be collected
and evaluated.
Lead Team
members
List the name(s) of
those who will
oversee collecting,
analyzing,
reporting, and
using results.
From UH-Manoa Assessment Template: http://http://manoa.hawaii.edu/assessment/resources/templates.htm
The Next Step:
Sustainable Assessment Plans
What should be assessed??
 IPP definition of Programs
 What appears on the diploma
 Undergraduate majors/concentrations
 Graduate majors/concentrations
Do we have to assess all of these
separately??
The Next Step:
Sustainable Assessment Plans
Issues
• too many outcomes, not enough time
• too many programs, not enough time
Too Many Programs
—Not Enough Time
An Example: Biological Sciences Undergraduate
programs
 1,449 students (excluding pre-major)
 BA Biological Sciences
 BS Biological Sciences (7 concentrations)
 Common curriculum for all concentrations: two LD
and two UD courses
How can we assess so many programs??
Learning Goals/Outcomes
Students will demonstrate the ability to
communicate in the Biological Sciences
Written Communication
Oral Communication
Information Literacy
Students will demonstrate the ability to apply
the process of science
Critical Thinking
Quantitative Reasoning
BA & BS-General: Curriculum
Map
Skills & Competencies
Learning Goal 1: Students will demonstrate the
ability to communicate in the Biological Sciences
LO’s/Cour
ses
Bio 1
Bio 2
Bio 100
Bio 184
Bio 121
Bio 139
Bio 160
Bio 188
Electives
Written
Communication
I
I,D
D
D
D
M
Various
Oral
Communication
I
I,D
D
D
Various
Information
Literacy
I
I,D
D
D
M
Various
Learning Goal 2: Students
will demonstrate the ability to
apply the process of science
Critical
Thinking
I
I,D
D
D
D
D
D
M
Various
Quantitative
Reasoning
I
D
D
D
M
Various
The problem
Shared courses: Bio 1, 2, 100, 184
Other courses: problematic!
Bio 188: Evolution
Required by 4 of 7 concentrations
Will capture 70% of students
Five Year Assessment Plan
L.O. / Year
1
Written
Communication
Bio
188/capst
one
Oral
Communication
Capstone
Information
Literacy
Critical Thinking
Quantitative
Reasoning
Content
Knowledge
2
3
4
5
Bio
188/caps
tone
Bio
188/caps
tone
Bio
188/caps
tone
Bio
188/capst
For more information
• Please contact Office of Academic
Program Assessment
– Amy Liu (amyliuus@hotmail.com)
– Julian Heather (jheather@csus.edu)
– Shannon Datwyler (datwyler@csus.edu)
– Elizabeth Strasser (strasser@csus.edu)