Outcome Assessment Reporting for
Undergraduate Programs
Stefani Dawn and Bill Bogley
Office of Academic Programs, Assessment & Accreditation
Faculty Senate, April 11, 2013
Assessment is…
• “…the systematic collection of information
about student learning, using the time,
knowledge, expertise, and resources available,
in order to inform decisions about how to
improve learning.”
– B. Walvoord, Assessment Clear and Simple, 2004
Assessment 1-2-3
1. OUTCOMES (“Students will be able to…”)
2. EVIDENCE
–
–
–
Direct measures (exams, papers, performances…)
Indirect measures (surveys, placement…)
Qualitative and quantitative indicators
3. IMPROVEMENT & VALIDATION
–
–
What does success look like?
How do we know?
How well are students interpreting and responding to
our expectations for their learning?
Assessment at OSU: Highlights
1997
2000
2005
2011
2012
• Assessment of learning in Student Affairs
• Assessment for professional accreditation
• Undergraduate reports (BEAVERS Learning Outcome
Tracking System)
• Graduate program and Bacc Core assessment
• On the right track, keep going (NWCCU)
NOW: Next Steps for Undergraduate
Program Assessment
• Revised Assessment Reporting Template
– Clarify what is needed; provide guidance &
examples
– Reinforce the fundamental role of program
outcomes
– Endorsed by Executive Committee, Curriculum
Council, University Assessment Council
• Initiate institution-level feedback & synthesis
New Assessment Report
Template
• Features
– Spreadsheet file format
– Tabs: Introduction; Instructions; Example; Report
Year; Add new years over time
– In the report:
The information that
needs to be aligned with
outcomes is aligned
in a single table
New Assessment Report
Template
• Process Clarifications
– Each year one or more of the program outcomes
must be measured such that ALL outcomes have
been assessed in a period of 5 years.
• A plan must be in place to do this – the plan is simply
communicated in the report table in this reporting
document.
– There needs to be at least 1 direct assessment per
outcome.
• If an outcome only has indirect assessment, then an
explanation is requested.
New Assessment Report
Template
• Process Clarifications
– As programs develop their assessment
infrastructure, we encourage solid, in-depth
assessment of 1 or 2 outcomes; do it well rather
than trying to assess too many outcomes at once.
– Timeline:
• This template will be used for the September 15, 2013
submission.
3. Program Outcomes Matched
with Measures and Results
Question
Example
List your program level student learning
(SLO)
Design, implement, monitor, and adjust an individualized, evidencebased, pharmaceutical care plan that will maximize patient safety
and therapeutic efficacy.
Is this SLO/SSO new or revised since the
last year you reported on it?
No
What year will you be reporting on this
outcome?
2012
List the measures/methods/instruments
used to assess the outcome.
Pharmaceutical Care Lab Clinical Cases to develop a care plan (3rd
year); Advanced Pharmacy Practice Experience (APPE) Preceptor
Evaluation on specific care plan elements (4th year)
Are the measure(s)/methods/instruments D
direct (D) or indirect (I)?
What benchmarks are you using to
determine if the outcome has been
satisfactorily met by the students?
Clinical cases - 95% of the class will receive at least a
75% score on the first attempt; APPE - 95% of the
students will receive at least a 70% on each of the
element scores
3. Program Outcomes Matched
with Measures and Results
Question
Example
What do the data show about student
learning for each outcome?
100% of the students passed both direct assessments
at the defined benchmark. This is an improvement
from 92% last year for the clinical cases and 90% for
the PPEs.
Course level changes
Last year we implemented course level changes in the
Pharmaceutical Care Lab to provide students with more
opportunities to practice writing and receive feedback on their
Care Plans. The success of this change is indicated in the
student success rate - nothing else was changed (assessments,
rubrics, and PPEs remained the same, students were just
provided more opportunities for practice and receive
feedback).
Program/degree level changes
None at this point
3. Program Outcomes Matched
with Measures and Results
Question
Example
Process unit used to reflect on the data
The typical process is that the data collected are presented to
the curriculum committee, who then identifies and prioritizes
action items and responsible parties. Those action items are
shared with the dean's executive committee who presents
questions to the curriculum committee, approves the action
items and/or allocates resources as needed.
Assessment plans for the upcoming year
We plan to assess learning outcome #4 next year. Although this
outcome is assessed each year in the course, because we are
focusing on this program level assessment of the outcome, the
course instructors will meet and develop the specific exam questions
for the assessment and then have the questions reviewed by three
faculty from the curriculum committee to ensure the questions map
well to the outcome and are clearly written and understandable. For
open-ended questions, the results will be graded by two faculty and
scores averaged to represent the student score for those test items.
The results will be presented to the curriculum committee.
Report Feedback
& Unit Meetings
• Over the next two months and/or summer:
• Consultations/feedback on last year’s reports
• Orientations to the new form
• Available Support
• Addressing specific feedback items
• Preparing for the new report template
• Targeted consultations – e.g. direct vs. indirect
assessments, matching outcomes with assessments,
organizational process ideas
• 1-on-1 meetings, special sessions at departmental
meetings or retreats, etc.
Questions?
• Stefani.Dawn@oregonstate.edu (7-0919)
• Bill.Bogley@oregonstate.edu (7-3506)