Strategies from the
Assessment Trenches:
A Self-Help Clinic
presented by AURA
2005 CELT Conference
1
Topics & Format
Topics
• Program Mission, Goals & SLO Statements
• Course Alignment Matrix
• Developing & Implementing an Assessment Plan
Format
• We introduce the topic.
• Groups discuss progress & problems.
• Debrief & share strategies.
2005 CELT Conference
2
Your Program Mission, Goals & Student
Learning Outcome Statements
Mission: a holistic view of the values and philosophy of the
department/program
Goals:
general statements about knowledge, skills, attitudes,
and values expected of graduates
SLOs:
clear, concise statements that describe how students
can demonstrate their mastery of program goals
Courtesy of Dr. Mary Allen
2005 CELT Conference
3
Group
Assignment
- SLOs
• Share your program’s SLOs.
• What challenges did you face
in crafting your SLOs?
• How did you overcome the
challenges? What lingering
concerns remain?
• Prepare to report on a few
key ideas, problems, or
strategies.
2005 CELT Conference
4
The Outcomes/Course Alignment Matrix
Course SLO 1
Stage 1 – Ensuring a
Cohesive Curriculum
In which courses are which
outcomes introduced,
practiced, mastered?
151
I
177
P
201
Stage 2 – Developing an
Assessment Plan
What student artifacts can be
used to measure progress?
SLO 2 SLO 3 SLO 4
I
I
P
I
277
P
P
340
M - oral
M
presentation
477
P - term
paper
2005 CELT Conference
5
Group
Assignment –
The Matrix
• Create a draft of your
program’s Outcomes/Course
Alignment Matrix.
• If your program has a (draft)
matrix, share it.
• Prepare to report on a few
key ideas, problems, or
strategies.
2005 CELT Conference
6
Developing/Implementing an Assessment Plan
An assessment plan should be meaningful, manageable, and
sustainable.
For each SLO, ask:
• How and where will the SLO be assessed?
• How will you collect, organize, and analyze the data?
• When and how often will this be done? By whom?
• Who will reflect on the results?
• What will ensure that results are used to implement program
changes to improve student learning?
2005 CELT Conference
7
Elements of An Effective Assessment Plan
What are the building blocks needed ?





Program Mission
Goals
Student Learning Outcomes (SLOs)
Course Alignment Matrices
Plan to Assess Individual SLOs … such as …
2005 CELT Conference
8
Assessment of SLOs
• Direct Assessment Methods:
–
–
–
–
Published Tests
Local Exams
Embedded Assignments & Activities
Portfolios
• Indirect Assessment Methods:
– Surveys
– Interviews
– Focus Groups
2005 CELT Conference
9
Assessment of SLOs
• Direct Assessment Methods are Essential:
– Examine Student Work …
• Indirect Assessment Methods are Supportive:
– Examine Student / Alumni Perceptions …
• Effective Assessment Programs Use
Multiple Measures … Triangulate
2005 CELT Conference
10
Measurement Using Rubrics
• What is a rubric?
– “Explicit schemes for classifying products or
behaviors into categories that vary along a
continuum” Mary Allen, WACS/AAHE workshop, Sept. 2004
– Holistic (along one dimension)
– Analytic = traits or characteristics of the
product or behavior
– Similar to grading criteria sheets that some
faculty use
2005 CELT Conference
11
Example Writing Rubric (Partial)
2005 CELT Conference
12
Example of Rubric:
SLO: IT Proficiency
2005 CELT Conference
13
Steps To Develop a Rubric
• Define the SLO and pick the work sample
• Involve the potential evaluators, faculty who assign the
student work, external stakeholders, (if possible)
• Brainstorm the traits or characteristics of the goal that you
want to measure
– Specific traits lead to actionable results
• Determine the number of discrete categories that you want
to have (generally 3-6)
• Brainstorm the descriptions of each category for each trait
• Refine the traits/descriptions avoid “and” clauses
2005 CELT Conference
14
The Next Steps
To collect, organize, and analyze the data …
• When and how often will this be done?
• By whom?
• Who will reflect on the results?
• How will they measure success?
• What will ensure that results are used to
implement program changes to improve
student learning?
2005 CELT Conference
15
A Caution:
Assessment Plans in Early Stages are Good at
Collecting Data … but Lack a Plan for How to:
 Organize, Summarize the Data
 Systematically Reflect on Results
 Ensure that Results are used to Effect Change
and Improve Student Learning
• Effective Assessment Programs are Both
Manageable and Sustainable
2005 CELT Conference
16
Evaluation and Use of Data for
Continuous Improvement
•Assessment Data Must be Evaluated
Systematically, Periodically
– Recommend Annual Collection / Assessment /
Action Plan
•Modify Curriculum and Pedagogy to Improve Student
Learning …
– Typical Actions:
• Modify Content, Sequencing of Course(s)
• Modify Pedagogical Approaches
2005 CELT Conference
17
Evaluation and Use of Data for
Continuous Improvement
• The Entire Process (Goals, Outcomes, Assessment
Measures) Must be Evaluated Periodically
– Recommend Five Year Re-evaluation to
synchronize with FYR process:
• Are we measuring reasonable work samples?
• Is our process meaningful? Are we better informed about
what our students are learning or not?
• Is our process sustainable? Can we be more efficient and
still have valid measures?
2005 CELT Conference
18
The Assessment Plan Design Process
• Wherever you begin developing your Assessment
Program will NOT be the ideal place to start …
• You Must Begin Somewhere …
• Iteration will save you ...
– Continuous Refinement of Methods
– Consensus and Team Building
– Alignment of Program Resources with Program
Goals and Faculty Values …
2005 CELT Conference
19
Group
Assignment –
Assessment
Planning
• Share your program’s SLO
assessment plan:
– What SLO(s) do you plan to
assess this year?
– How do you plan to assess it?
– What is the biggest hurdle you
face in the assessment process?
• Prepare to report on a few key
ideas, problems, or strategies.
2005 CELT Conference
20
AURA’s Advice
• Don’t get trapped in a quest
for the perfect assessment
plan. Try something!
• Do involve the faculty.
• Don’t forget to use your
results.
• Do plan to repeat the process.
• Don’t hesitate to ask for help.
• KISS
2005 CELT Conference
21