A Simple and Effective Programmatic Assessment Process for Continuous Quality Improvement Lisa Lebovitz, JD
Assistant Dean of Academic Affairs & Assessment
Objectives of Programmatic Assessment
1. Ensure coverage of science content and competencies/outcomes through curriculum mapping
2. Confirm academic progress and achievement by student cohorts and individuals
3. Provide faculty with student feedback on course content, delivery and teaching pedagogy
Mapping as a Tool to Visualize and Quantify the PharmD Curriculum
Purpose is to assure appropriate depth, breadth and complete coverage of ACPE Appendix B content and the School’s terminal performance outcomes (TPOs). Assessment tools are evidence‐based, visually simple representations for faculty content experts to see where their material is situated in the curricular structure. I=introduced, R=reinforced, E=emphasized
Student Performance & Achievement
Purpose is to determine student achievement at defined levels of the professional competencies, in aggregate and at the individual student level, and to assure comparable academic performance between campuses and longitudinally.
Assessment tools include grade distributions provided to faculty, and class ranks provided to students.
Course1 Course2 Course3 Course4 Course5
Class Rank – Distributed to Students via email
This assessment of your academic achievement is being provided to you to promote private
reflection and to develop self-awareness about your academic standing compared to your
peers. More than academic standing plays into what a makes an outstanding student
pharmacist, but enhanced academic performance can give you an edge in landing that allimportant first position after graduation.
«NAME»
(«YEAR») «class» «campus»
Based on data obtained from Banner at the end of the fall 2013 semester:
•Your fall 2013 semester GPA is «termGPA» out of 4.00 (class average «classtermGPA»/4.00)
•Your cumulative GPA is «cumGPA» out of 4.00 (class average «classcumGPA»/4.00)
•Currently you rank «rank» out of «count» students in your class, «standing»
Please share this data with your faculty advisor and mentors as you are considering ways to
enhance your academic performance, residency preparedness and employability.
Sincerely,
Richard Dalby, Ph.D.
Professor and Associate Dean for Academic Affairs
University of Maryland School of Pharmacy
Course and Instructor Evaluations
Purpose is to ensure that teaching and learning methodologies are leading to effective and efficient learning experiences.
Assessment tools are simple and easily interpretable graphs and data of student feedback on course content, delivery and teaching pedagogy. NOTE that “All FA13 Courses” refers to all courses taught by the School of Pharmacy within this academic term. Percent positive responses = (Strongly Agree + Agree) / (Strongly Agree + Agree + Disagree + Strongly Disagree). Data for responses of Unable to Comment were excluded from the mean and the %positive.
Standard Course Evaluation Questions
Q#
1
2
3
4
5
6
7
8
9
10
questionText
This course's Blackboard site was well organized.
Class‐wide issues that arose were effectively addressed by course managers.
The types of learning activities in this course were valuable to my understanding of the material.
I was challenged to apply the information in this course.
The amount of work in this course was reasonable.
The pace of this course was reasonable.
Graded assessments (including exams) related to the course objectives.
Graded assessments (including exams) accurately measured my understanding of the course content.
I understand how the material in this course contributed to my professional development.
I have achieved the stated course goals/expected outcomes.
Individual dots represent the average student response for each full time faculty member teaching in one or more Fall 2013 courses for the question, “This instructor taught the material in a way that was conducive to learning”. The red bar labelled “AVERAGE”
represents the overall average for all full‐time faculty (n=71) and part‐time (n=57) instructors teaching in didactic PharmD, Masters and graduate courses during the Fall 2013 semester. The grey shaded area represents ±2 standard deviations from the mean. CLOSING THE LOOP
•
The course managers and department vice chairs for academic
affairs review and sign off on course results (not instructor results),
each noting their interpretation of the data and any plans for
improvement.
•
The Office of Academic Affairs retains course evaluation
documentation and provides general feedback to students during class
meetings.
•
The Curriculum Committee conducts an iterative course review
process that generates discussions between representatives of the
committee and the current course managers regarding course
evaluations, and documentation of any changes to the course as a
result of the student feedback.
•
The Assessment Committee reviews overall course evaluation data
as part of the metrics for the Education goals of the school’s strategic
plan.
Benefits
•
Letter grade distributions -- Faculty can easily monitor student campus
cohort academic performance in each course and across courses.
Academic performance between campuses is comparable, with annual
cumulative grade point averages by class cohort varying no more than a
few hundredths of a point. When minor statistical differences are
detected they are investigated and corrective action is taken.
•
Instructor evaluations -- Save time for course managers to edit and
distribute results from each course to each instructor. Instructors have a
summary of the entire semester’s teaching at a glance to review with
their mentors overall performance across courses with varying content
and pedagogy, and plan for professional development.
•
Course evaluations -- Students know their opinions are heard. Course
managers and the Curriculum Committee can review student perceptions
longitudinally.
•
Class rank -- Students need class rank for residency applications.
Transferability and Effectiveness
This process is easily transferable to other colleges and schools of
pharmacy, regardless of the type of academic records and
registration system and online survey tool for course evaluations.
Skill in Excel is necessary to produce the graphs and summaries
each semester.
Keys to success in building a culture of assessment:
•Issuing frequent reminders to students to assure high response
rates on course evaluations
•Providing feedback to students by faculty and administrators
•Benchmarking individual and course performance to mean data
•Requiring faculty and administrators to interpret and document their
interpretation of course evaluation data