When, How, and How Much?
A Guide to Designing and Implementing a Departmental Assessment Plan
(Based on Barbara Walvoord’s Assessment Clear and Simple)
Contextual Note
On April 19, Western’s Accreditation and Assessment Advisory Committee (AAAC)
is bringing Barbara Walvoord to campus to present an assessment workshop for
departments. An assessment specialist known for keeping assessment “Clear and
Simple,” Walvoord’s workshop will help departments create simple, sustainable
plans “…useful to the department for its own goals, and consonant with regional
and professional accreditation guidelines.”
Some departments believe it will be simpler to complete their assessment plans
during winter term. To this end we present the following guidelines and standards,
drawn from Dr. Walvoord’s book Assessment Clear and Simple.
Working Definitions
Accreditation is a process of peer-review by which university practices are evaluated
against a set of published standards.
Assessment is the systematic collection of performance data in order to inform decisions
and make improvements. Academic assessment involves three key steps: goals,
information, and action.
Direct Assessment Measures evaluate student performance of a learning objective, rather
than reports or perceptions of such achievement. A common direct measure would be a
capstone paper or project.
Indirect Assessment Measures evaluate reports or perceptions of student achievement of
a learning objective. A common indirect measure is a survey.
Steps in the Assessment Cycle
Walvoord breaks assessment into three simple steps:
Step 1: Goals (Also called “outcomes”)
Departments decide what they want their students to learn.
Step 2: Information
Departments research whether their graduates have successfully learned what they
wanted them to learn. Walvoord recommends that departments use both direct and
indirect assessment measures when gathering information.
Step 3: Action (Also called “closing the loop”)
Departments use their research findings to make changes and decisions that improve
student learning.
NOTE: This guide addresses Step 2, providing guidance and minimum standards for
researching whether students have successfully learned what a department wants them to
learn. This assumes that the department has previously established its goals, or program
learning outcomes.
Orienting questions: When, How and How Much?
When?
A department can collect information on student learning in classes while students are
learning, or at the end of the program in a capstone course, concluding exam, or practicum.
While it is ideal to collect information both during and at the conclusion of an academic
major, concluding assessments are the simplest way of demonstrating whether graduating
students demonstrate that they have met the program learning outcomes.
How?
To oversimplify the matter, there are two principle ways to directly assess whether
students have learned what a department wanted them to learn: standardized licensing or
professional exams, and classroom work (papers, portfolios, exams).
Standardized licensing or professional exams.
Departments can use licensing or professional exams to assess outcomes that are
directly measurable via such exams.
Classroom Work: Papers, Portfolios and Exams.
Papers and Portfolios.
A capstone or senior seminar provides a particularly rich opportunity for direct
program assessment. Papers permit the assessment of many outcomes; capstone
papers place that assessment at the end of a program, permitting faculty to
determine whether graduates ultimately did or did not learn the knowledge and
abilities they wanted them to learn.
Walvoord suggests that a committee of faculty evaluate a sample of capstone papers,
portfolios or assignments against a rubric describing departmental standards for
each outcome, or against anchor papers that provide examples of work that is “at
departmental standard” for particular outcomes.
Faculty findings are summarized and shared with the department, which discusses
where students have and have not met the program learning outcomes. Based on
this information, the department plans program improvements.
Exams
Alternatively, faculty can look for final exam assignments and questions that permit
assessment of departmental outcomes. These can be evaluated against rubrics or
anchor answers which indicate whether a departmental outcome has been achieved.
Faculty volunteers can collect the results of this assessment to share with the
Department, which discusses where students have and have not met program
outcomes and what steps can be made to make improvements to improve learning.
How Much?
Walvoord states that one direct assessment activity, such as those described above, which
involves a fair representation of department members and which covers all departmental
outcomes at the conclusion of a program, is largely sufficient for accreditation standards.
Combining this concluding assessment with an indirect assessment, such as a satisfaction
survey, will typically meet all regional requirements for program assessment.