Integrating A Problem-Based Learning
Approach Into Large Sections of GraduateLevel Introductory Biostatistics Courses
Patrick D. Kilgo
Emory University
Department of Biostatistics and Bioinformatics
Southern Regional Council on Statistics
June 6th, 2011
Core Course Setting
 Graduate level biostatistics courses with associated lab components
 All incoming Master’s degree candidates in public health are
required to take BIOS 500:
 Descriptive statistics
 Probability
 Common hypothesis tests
 5 large sections – approximately 85 students per section
 Two classes per week – 80 minutes per class
 Follow-up regression course (BIOS 501) is optional
 Linear regression / ANOVA
 Logistic regression
 Survival analysis
What’s The Problem?
RETENTION
 It is common for our students to have forgotten almost
everything in the intervening month between Fall and Spring
semesters
 Thesis season: By their second year, the average student
has:
 Forgotten most of the statistical concepts they once “knew”
 Has forgotten how to apply concepts and statistical tests and
also the programming necessary to accomplish their analysis
 Resorted to roaming the halls of the third floor, beckoning
any statistical-looking person for help
Problem-Based Learning (Duch, 2001)
 We learn and retain when solving a problem ourselves
 “Complex, real world problems are used to motivate
students to identify and research the concepts and
principles they need to know to work through those
problems”
 Small learning teams are used to collectively acquire,
communicate and integrate information
 “Instructor is no longer the sage on the stage but
rather is the guide on the side.”
Problem-Based Learning Objectives
(Duch, 2001)
 Think critically and be able to analyze and solve real-
world problems
 Find, evaluate and use appropriate learning resources
 Work cooperatively in teams and small groups
 Demonstrate versatile and effective communication skills,
both verbal and written
 Use content knowledge and skills acquired at the
university to become continual learners.
Previous PBL Biostatistics Courses
 Carolyn Boyle, Mississippi State, Journal of Statistics
Education v.7, n.1 (1999)
 Applied in an animal science setting
 18 veterinarian students
 8 cases over two semesters
 The only published account of PBL in biostatistics
Goals – Excellence In These Areas …

Generating descriptive statistics
 Choosing the appropriate analysis approach when faced with a research
problem
 Interpreting findings from research studies
 Writing reports and communicating results of research findings following a
statistical analysis
 Thinking through analytical problems and subsequently designing studies
 Working in groups to solve research problems
 Beginning the statistical thinking/planning for your Master’s thesis
 Discussing statistical analysis with other faculty, students and employers
Extra Resources Required for PBL
 Departmental Support - $$$$$$$$
 3 additional experienced “co-instructors”
 Though some disagree, I still believe that this task is
beyond the capabilities of the average TA
 3 additional classrooms
 Patience and flexibility on the part of the lab instructors
 A ton of my time
General Framework of My PBL Class
 No more tests or homework
 No required textbook: I asked them to find any statistical text for
reference
 4 “cases” (problems) over the course of the semester
 Mondays: Lecture (Taught by Kilgo)
 Wednesdays: 4 PBL “breakout” sections of size 20 where cases are
worked on in groups of 4-5. (Taught by Kilgo and 3 co-instructors)
 Teach them deeper, not wider
Deeper, Not Wider
 Half as many lectures = must be efficient
 Before I presented a topic I asked myself three questions:
 How likely are the students to encounter this topic in practice?
 How likely is the average student to remember this topic in
three weeks?
 Will they be taught this topic in their introductory
epidemiology course?
 Sample of topics omitted:





Many probability axioms and concepts (~1 lecture)
Bayes rule (~1 lecture)
Binomial and Poisson distributions (~2-3 lectures)
Nonparametric tests (~1-2 lectures)
Several statistical tests – McNemar’s Test, ANOVA (~2 lectures)
First Implementation
 Fall 2009, a class of 72 first-year, first semester Global Health
students
 Non-majors
 PBL co-instructors:
 Lisa Elon – fellow faculty-level colleague
 Laura Ward – staff senior biostatistician
 Jeff Switchenko – 5th year doctoral student
 Open-ended, real-life, interesting problems in public health and
medicine
 Individual deliverables, even though group work was
encouraged
 Data analysis report with emphasis on methods, results, conclusions,
limitations
Cases
 Case 1 – Designing a study to determine whether data
collected from Automatic Crash Notifiers in cars can
be used to determine the need for Level I trauma care
 No data in this case
 Thought experiment
 Case 2 – Were players accused in the Mitchell Report of
taking steroids better offensive performers?
 Students had to make a descriptive case one way or the
other.
 Outliers, multiple observations per player, skew, etc.
Cases
 Case 3 – Validation of an experimental testing device
designed to diagnose pre-Alzheimer’s disease
 Data management, t-tests, assumption violations, experimental
design issues
 Real-life problem – collaboration between Emory and GA Tech
 Case 4 – Smallpox Vaccine Trial
 Chance to compare modern methods to Jenner’s method
 Students had to read Jenner’s original paper
 Chi-square tests, odds ratios, Interactions (non-homogeneity)
Final Project
 Students proposed a personalized final project in the middle of
the semester
 Could be anything from a research interest to a personal
interest:
 What is the effect of maternal iron supplements on neo-natal iron
levels?
 Do women think mustaches are more sexy when they are ovulating?
 Students asked for specific variables, guessed at their
distribution and hypothesized about group differences.
 Instructors generated datasets for them so that they were
studying something that is interesting to them in a context they
are familiar with.
First Semester PBL Evaluation …
How comfortable are you with the following
…?
Question
Non-PBL Class
N=74
4.4 / 5.0
PBL Class
N=57
4.6 / 5.0
p-value
Generating descriptive stats
4.12 (0.84)
4.16 (0.65)
0.78
Choosing the right analysis
3.64 (0.82)
4.00 (0.79)
0.012
Interpreting your findings
3.88 (0.64)
4.04 (0.69)
0.18
Writing/Communicating results
3.58 (0.72)
4.00 (0.89)
0.003
Thinking through problems / study design
3.27 (0.90)
3.98 (0.74)
<0.001
Working in groups to solve problems
3.82 (0.84)
4.23 (0.87)
0.008
Beginning the planning for your thesis
2.97 (1.02)
3.60 (0.96)
<0.001
Discussing statistics with other faculty
3.53 (0.74)
3.74 (0.92)
0.15
Feedback From Students
If I could go back in time to the beginning of
the semester with a choice of class formats I
would …
1)Choose the lecture-only format (4/57)
2)Choose the problem-based format (48/57)
3)Be indifferent towards the format (5/57)
First Semester Growing Pains
 Timing of cases / lectures / labs
 Should have taken a TA when one was offered
 Workload distribution – most of the assignments
came due later in the semester
 Different approaches from different co-instructors
First Semester Pleasant Surprises
 Students liked SAS
 Very positive course feedback
 Students having an easier time working with faculty on
projects
 Many requests for a third course offering
 Was as much a class in research writing and organization as
it was biostatistics – their scientific writing greatly
improved over the semester
Second Implementation – BIOS 501
Spring 2010
 Only three cases – no final project
 Case 1 – Predicting traffic deaths using 1964 NHTSA-type data
 Linear regression, transformation, skew, outliers, missing data,
validation, confounding.
 Case 2 – The evaluation of off-pump CABG compared to on-pump
CABG with respect to major adverse outcomes
 Logistic regression, lots of covariates, confounding, fitting of
associative models, graphics, independent risk factor identification,
interactions
 Case 3a - Survival Analysis –The Role of Race and Race Mismatch in
Determining Survival in Pediatric Heart Transplant Patients
 Case 3b – The Effect of ICU LOS on Long-Term Survival
 KM curves, Cox proportional hazards regression, confounding, etc.
Conclusions - Feedback From Students
 Very positive in general
 Complaints include:




Workload distribution
Time-consuming
Learning material/working on cases concurrently
“I got an 800 on the math GRE and I’m struggling in your class …
I felt like I would have done better in the traditional section”
 BIOS 500
 Fall 2009: 4.6/5.0
 Fall 2010 4.2/5.0
 BIOS 501
 Spring 2010
 Spring 2011
4.7/5.0
4.7/5.0
Likert Scale Question: I learned a lot in this course …
THANK YOU!