Qualitative Methods in
Educational Research
AARON W. BERNARD, MD
JONATHAN FISHER, MD
Objectives
 Define qualitative research
 Contrast with quantitative research
 Describe how to conduct qualitative research by
using examples in the literature
Quantitative Research
 Test well-specified hypothesis concerning some
predetermined variables
 Answer questions like;
 Whether……..
 How much….
Quantitative Educational Research
 Attempts to study links
between teaching factors
and learning outcomes
 Improved Medical Student
Satisfaction and Test
Performance with a
Simulation-based Emergency
Medicine Curriculum: A
Randomized Controlled Trial.
Ten Eyck. Ann Emerg Med. 2009;54:684-691
Qualitative Research
 Qualitative research questions tend not to ask
whether or how much but rather;



What
Why
How
 Qualitative research offers insight into social,
emotional, and experiential phenomena.
 Qualitative reports do not typically generate answers
but rather generate narrative accounts, explanations,
conceptual frameworks.
Qualitative Research
 Qualitative research is inquiry-guided, inductive, and
descriptive.
 Similar to an anthropologic or sociologic descriptive
study.
Data Collection Methods
 Field observations
 Interviews
 Acad Med. 2009.
 Document analysis
 Teach Learn Med. 2004.
Data Analysis
 Grounded theory
 Coding
 Iteration
 Theoretical saturation
Qualitative Educational Research

Acad Emerg Med 2005;12:856-861.
What Do Emergency Medicine Learners Want from Their Teachers? A
Multicenter Focus Group Analysis. Acad Emerg Med 2005.
 Background
 Effective teaching practices in the ambulatory setting have
been documented
 The ED is a different type of learning environment
 A qualitative study to determine what EM teachers consider
important has been done1
 What do EM learners consider important to effective
EM teaching?

Quantitative methods vs.. qualitative methods
Bandiera . Ann Emerg Med 2005; 45:253-261
What Do Emergency Medicine Learners Want from Their Teachers? A
Multicenter Focus Group Analysis. Acad Emerg Med 2005.
 Methods

Focus groups at Five EM programs in Ontario


Multiple sites adds validity
Medical student, EM Resident, Off-service resident, Junior
FRCP, Senior FRCP
Purposeful sampling
 28 participants total

What Do Emergency Medicine Learners Want from Their Teachers? A
Multicenter Focus Group Analysis. Acad Emerg Med 2005.
 Methods

Semi-structure interviews by a trained moderator

Tape-recorded and transcribed by a third party

Participants used terms like “junior” to identify themselves to
allow the data to be matched to type of participant
What Do Emergency Medicine Learners Want from Their Teachers? A
Multicenter Focus Group Analysis. Acad Emerg Med 2005.
 Methods/Data Analysis
 Transcripts were reviewed by two independent assessors and a
coding framework was derived using grounded theory methods
 Novel ideas were assigned a new teaching code or a new
qualifier as they emerged
 When a new code was added the entire transcript was reviewed
 The two codes that emerged were compared and consensus
was reached
 Agreement on 41 coded items and disagreement on 9

5/9 were subthemes
What Do Emergency Medicine Learners Want from Their Teachers? A
Multicenter Focus Group Analysis. Acad Emerg Med 2005.
What Do Emergency Medicine Learners Want from Their Teachers? A
Multicenter Focus Group Analysis. Acad Emerg Med 2005.
 Data analysis
 The transcripts were then reviewed against the final code by
one investigator to check for frequency of codes
 A second investigator reviewed 20 percent of the transcript to
allow for an estimation of interrater reliability

Computer software exists to help with this!
What Do Emergency Medicine Learners Want from Their Teachers? A
Multicenter Focus Group Analysis. Acad Emerg Med 2005.
 Summary
 Qualitative methods were most appropriate for this
investigation
 Focus group interviews are a common method to obtain
data
 Purposeful sampling (low numbers, key players)
 Grounded theory to create code
 Frequency of themes to assess importance of themes to
subject
Qualitative Educational Research

Academic Medicine 2010; 85:124-133
Medical student professionalism narratives: a window on the informal
and hidden curriculum. Acad Med 2010
 Background
 The informal and hidden curriculum drives
professionalism development
 There is a lack of evidence regarding what students
experience.
 Objective
 To understand students experiences as they relate to an
understanding of professionalism
Medical student professionalism narratives: a window on the informal
and hidden curriculum. Acad Med 2010
 Methods
 Students on IM required to post 2 narratives over 2 months
 Retrospective review of narratives June – November 2007
 272 narratives from 135 students
Medical student professionalism narratives: a window on the informal
and hidden curriculum. Acad Med 2010
 Methods/Data Analysis
 Thematic content analysis
 Immersion/crystallization method
 Reading and rereading narratives until themes emerge
 Code was created with the first 50 narratives
 A narrative could be coded more then once
Medical student professionalism narratives: a window on the informal
and hidden curriculum. Acad Med 2010
Reliability

1.
2.
One author created the code based on the first 50 narratives
then two others independently reviewed the work.
Created focus groups of students to review themes
Medical student professionalism narratives: a window on the informal
and hidden curriculum. Acad Med 2010
 Two major domains
 14 themes
 63.4% positive, 29.1%
negative, 7.5 % hybrid
Medical student professionalism narratives: a window on the informal
and hidden curriculum. Acad Med 2010
Direct representative quotes
 Smith AK, Fisher J. Ann Emerg Med. 2009.
Direct representative quotes
Direct representative quotes
Medical student professionalism narratives: a window on the informal
and hidden curriculum. Acad Med 2010
 Summary
 Qualitative methods were most appropriate
for this investigation
 Document analysis is a common method to
obtain data
 Methodological steps to improve validity is
important
 The use of direct representative quotes
Qualitative Educational Research

Teach Learn Med.
2010;22(1):28-32
Medical student participation in a disaster seminar and drill: brief description of
activity and report of student experiences. Teach Learn Med 2010
 Background
 Disaster training is relevant to medical students who as
physicians may be asked for guidance during training or real
events.
 Medical school offer little formal curriculum
 Objective
 Provide 4th year medical students with a didactic and clinically
experience in disaster medicine
 Evaluate the effectiveness through lecture evaluations and a
qualitative analysis of focus groups
Medical student participation in a disaster seminar and drill: brief description of
activity and report of student experiences. Teach Learn Med 2010
 Methods
 43 students
 4 hour didactic session the evening before the drill
 Training in the use of a standardized assessment tool for
disaster medicine drills
 Evaluation and Focus Group
 5 point likert scale surveys regarding the didactics
 2 Focus groups after the drill

Semi-structured interviews
Medical student participation in a disaster seminar and drill: brief description of
activity and report of student experiences. Teach Learn Med 2010
 Methods/Data Analysis
 Median, means, SD for the survey data
 Notes were taken in real time
 One researcher performed the qualitative analysis and created
the thematic domains of the code
Medical student participation in a disaster seminar and drill: brief description of
activity and report of student experiences. Teach Learn Med 2010
 28 of the 42 students
participated in both the
didactic and the drill
 17 agreed to participate in
the focus group
 Themes


Changes in self-perceived
attitude toward DM
Changes in student’s ability to
apply this knowledge in a
simulated setting.
Medical student participation in a disaster seminar and drill: brief description of
activity and report of student experiences. Teach Learn Med 2010
 Effective use of qualitative research to create a
descriptive study of an educational curriculum that is
publishable.
 Combination of qualitative and quantitative methods
Final Conclusions
 Qualitative research is a tool that can be used for educational





investigations.
Qualitative research is inquiry-guided, inductive, and descriptive.
Data usually comes from field observations, interviews, or document
analysis.
Grounded theory is used to review the data to create themes and
subthemes. This is known as coding.
Frequency of themes is often used as a marker of importance of themes
to subjects studied
Various techniques can be used to increase reliability of the analysis
References







Tehaerani A, O’Brien B, Masters DE, et al. Burden, responsibility, and reward: preceptor
experiences with the continuity of teaching in a longitudinal integrated clerkship. Acad
Med. 2009;84(10 Supp):s50-s53.
Alford CL, Currie DM. Introducing first-year medical students to clinical practice by
having them “shadow” third-year clerks. Teach Learn Med. 2004;16(3):260-263.
Thurgur L, Bandiera G, Lee S, et al. What do emergency medicine learners want from
their teachers? A multicenter focus group analysis. Acad Emerg Med 2005; 12:856-861.
Karnieli-Miller O, Vu R, Holtman M, et al. Medical students’ professionalism narratives: a
window on the informal and hidden curriculum. Acad Med. 2010;85:124-133.
Giacomini MK, Cook DJ, et al. Users’ guides to the medical literature XXIII. Qualitative
research in health care A. Are the results of the study valid? JAMA. 2000;284:357-362.
Smith AK, Fisher J, Schonberg MA, et al. Am I doing the right thing? Provider
perspectives on improving palliative care in the Emergency Department. An Emerg Med.
2009;54:86-93.
Kaji AH, Coates WC, Fung C-C. Medical student participation in a disaster seminar and
drill: brief description of activity and report of student experiences. Teach Learn Med.
2010;22(1):28-32