Assessment of Student Knowledge in Individual Content Areas for
Nine Integrated Drugs and Disease Courses
S. W. Zito, M. E. Gillespie, A. C. Marziliano, G. M. LaPan
Discussion (Con’t)
Pearson Product-Moment Correlation Coefficients
Introduction
The College’s entry-level Pharm.D. curriculum contains a sequence of nine Drugs
and Disease courses (D&Ds) that integrates didactic content from Pathophysiology,
Medicinal Chemistry, Pharmacology and Therapeutics. The D&D courses are
delivered in a block sequence over 3 semesters with 3 D&Ds offered each semester
(Fall and Spring of P-2 and Fall of P-3). Currently the D&Ds are, sequentially:
Infectious Disease, Skin and Connective Tissue, Cardio-Renal 1, Cardio-Renal 2,
Respiratory System, Nervous System, Neoplasia, Endocrine and Reproductive
Systems, and Gastro-Intestinal and Genital-Urinary Systems. Ongoing faculty
concerns about issues related to student competence in discipline-specific didactic
knowledge within the integrated course sequence resulted in a formal assessment
study of knowledge retention.
Intro to Pharmacology
Intro to Pharmaceutical Care
D&D EndoRepro
D&D EndoRepro
D&D GI/GU
D&D GI/GU
D&D Neoplasia
D&D Neoplasia
D&D Respiratory
D&D Respiratory
D&D Neuro
D&D Neuro
D&D CR-2
D&D CR-2
D&D CR-1
D&D CR-1
D&D Skin
D&D Skin
D&D Infectious
D&D Infectious
0
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.4
Pearson Correlation: Intro Pharmacology/Pharmacology
Pearson Correlation: Intro PC/Therapeutics
Methods
Summative methods to establish a benchmark for the achievement of competence in
each of the four didactic components of the integrated course sequence include:
•Evaluation of student performance on embedded content–specific examination
questions
•Pearson product-moment correlation coefficients between grades achieved in the
discipline-specific introductory courses and in each didactic area of the integrated
courses
•Evaluation of student performance determined by sorting introductory course
grades into quartiles and comparing them to whether the student performance
improved, decreased or remained the same in discipline-specific content across
the D&Ds.
Students take a midterm and final examination in each D&D. The examinations
consist of 50 multiple choice questions (5 Pathophysiology and 15 questions related
to each of the other content areas). Student test scores in each didactic area were
graded using Scantron ParScore software. Pearson-Product-Moment Correlation
Coefficients were calculated between discipline-specific introductory courses
(General Pathology, Introduction to Pharmaceutical Care, Introduction to Medicinal
Chemistry, and Introduction to Pharmacology) and student scores in the respective
didactic areas using MS Excel data analysis software. Calculation of student
success ratios were also calculated using MS Excel software.
Intro Medicinal Chemistry
General Pathology
D&D EndoRepro
D&D EndoRepro
D&D GI/GU
D&D GI/GU
D&D Neoplasia
D&D Neoplasia
D&D Respiratory
D&D Respiratory
D&D Neuro
D&D Neuro
D&D CR-2
D&D CR-2
D&D CR-1
D&D CR-1
D&D Skin
D&D Skin
D&D Infectious
D&D Infectious
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.0
Pearson Correlation: Intro MedChem/MedChem
0.1
0.1
0.2
0.2
0.3
0.3
0.4
0.4
Pearson Correlation: Pathology/Pathophysiology
Introductory Course Quartile Comparison
5th Year Intro to MedChem/D&D MedChem
4th Year Intro MedChem/D&D MedChem
90%
80%
100%
70%
90%
60%
80%
50%
70%
40%
60%
30%
50%
20%
40%
10%
30%
20%
%Improved/Same
10%
% Worse
0%
%Improved/Same
1st
0%
1st
2nd
3rd
% Worse
2nd
3rd
4th
4th
Results
Average Student Grades on Embedded Content-Specific Questions and
Average Grades in Discipline-Specific Introductory Courses
4th Year -Pathology/D&D Pathophysiology
5th Year Pathology/D&D Pathophysiology
100%
90%
80%
80%
70%
70%
60%
60%
50%
50%
40%
40%
30%
20%
10%
0%
30%
20%
10%
1st
%Improved/Same
2nd
% Worse
%Improved/Same
0%
% Worse
1st
2nd
3rd
3rd
4th
4th
Discussion
1. Student performance on examinations
Pathophysiology
Med Chem
100.00%
90.00%
80.00%
70.00%
60.00%
50.00%
40.00%
30.00%
20.00%
10.00%
0.00%
100.00%
90.00%
80.00%
70.00%
60.00%
50.00%
40.00%
30.00%
20.00%
10.00%
0.00%
c. The average grades on the therapeutic questions across all 9 D&Ds ranged
from 70% to 79% demonstrating that this key competency is being acquired
by the students.
2. Pearson Product-Moment Correlation Coefficients
a. There were no strong correlations between the grades in the Introductory
courses and grades in the discipline-specific questions across all nine D&Ds.
The range was 0.07 to 0.42.
b. There was a very poor correlation (<0.2) for General Pathology and
Pathophysiology in D&D Cardio-Renal 1 and 2 (0.15 and 0.07) as well as for
D&D Skin (0.15).
Additionally, Introduction to Pharmacology and
Pharmacology in D&D Skin (0.14) showed a poor correlation.
c. There was a poor correlation for Introduction to MedChem and Medchem
in D&D of GI/GU (0.16).
d. These poor results may indicate a need to review the content of the
Introductory courses to identify areas that need improvement.
3. Introductory Course Quartile Comparison
a. The students who do well in the Introductory courses (quartiles 1 or 2)
show a higher probability of doing poorly in the discipline-specific content of
the D&Ds (quartiles 3 or 4). The converse is also true. The cause of this is
not clear but may involve a number of factors: grade inflation in the
Introductory courses; students have difficulty adapting to an integrated teamtaught course or the Introductory material is not relevant to the D&D content.
b. Sometimes these reversals are quite dramatic. For example almost 90%
of all quartiles do worse in D&D 4th Year Pathophysiology and in 4th Year
MedChem.
c. On the other hand, the 5th year cohort tended to improve. For example
there are dramatic improvements in 5th Year Pathophysiology and in 5th Year
Med Chem. This could possibly indicate that after 6 D&Ds, the students who
struggled with the Introductory courses have improved study habits and
strategies for the D&Ds.
a. Overall, average student grades on discipline-specific questions were modestly
lower than the average grade in the corresponding Introductory courses.
Pathophysiology (71% vs 82%); MedChem (71% vs 82%); Pharmacology (78%
vs 82%) and Therapeutics (74% vs 82%).
b. Students performed poorly in some discipline-specific areas:
Pathophysiology in D&D Cardio-Renal 1 and 2 (58% and 54%) and D&D in
Neoplasia (54%). This may be reflective of the difficulty of the pathology of
these two systems as well as the relative small number of questions allocated
on the examinations (10%).
Pharmacology in D&D of Cardio-Renal 2 system (63%). This may also be
reflective of the difficulty of the pharmacology of the various drug classes.
MedChem in D&D of Infectious diseas (ID) (63%). ID is the first D&D in the
sequence and students may be struggling to adapt to the blocked program and
to team teaching.
Conclusions
Student performance is reassuring since in general, we see no specific
discipline area in which the students are consistently doing poorly.
However, there are D&Ds where students are under achieving: Pathophysiology
in D&D Cardio-Renal 1 and 2 and Neoplasia; Pharmacology in Cardio-Renal 2
and MedChem in Infectious Disease. Whether this is due to poor preparation in
Introductory courses is hard to tell since there were only weak Pearson
correlations between Introductory courses and the D&Ds. This can also be seen
in the Introductory course quartile comparisons. Student performance is in flux
with the student’s ability to do better over the D&D sequence. Students who did
extremely well in the Introductory courses can stay the same or decrease in
performance whereas those students who did poorly in the Introductory courses
are more likely to improve their grades.
The results are indicative of a complex dynamic at work:
-Students have to adapt to factors related to team teaching.
-Students have to adapt to the rapid pace of content delivery as well as
delivery styles of different faculty.
-Depending on the specific D&D, students may opt to spend less time studying
a certain discipline-specific content area in favor of one they perceive as
more important/difficult in order to pass the course…a logical strategy.
Taken together these observations suggest that judging future student
performance by using Introductory course metrics is fraught with
difficulties. However, we now have baseline data which can be shared with the
Pharmacy Curriculum Committee, Assessment Committee and the faculty who
teach the Introductory and D&D courses. Changes to content delivery and
teaching styles can now be monitored for their effectiveness in achieving
maximum learning outcomes for the largest number of students.