1
“Quality” in qualitative
research
Key Concepts in Qualitative Research
Adapted from:
Roller, M. R., & Lavrakas, P. J. (2015). Applied
qualitative research design: A total quality
framework approach. The Guilford Press, New
York.
2
Four Pillars of Qualitative Research
 Credibility
– Data completeness (scope) and accuracy (data gathering)
 Analyzability
– Completeness and accuracy of the analysis (processing
and verification)
 Transparency
– Full disclosure of the procedures used for the study and
analyses (reporting)
 Usefulness
– “Doing something” with the findings, making a difference
Credibility: Completeness
(Scope)
• Scope: sampling: does the study represent the population of
humans and/or documents it is investigating?
 Must clearly identify and create a list of the target population (who you
want to understand)
 Use a systematic sampling process to select participants (or to choose
times and locations in the case of observational studies)
 Determine sample size (in qualitative research, this changes based on
the data gathering: reduced from plan if low variability; may add cases
if there is a lot of variability)
 Gain Access/Cooperation
 Build rapport/gain trust
 Assure confidentiality
 Explain material (if any) and non-material benefits
3
Credibility: Accuracy (in
Data Gathering)
• Is the data representative of what you claim to be
measuring?
 Mode of data collection
 Options: In-person, traditional mail, landline telephone, email,
Internet, mobile device.
 Consider how mode of data collection will affect quality
 Create data collection tools
– E.g., Interview guide, focus group questions, content analysis coding forms,
ethnography observation grids
– Substance: depth and breadth of the topical content (how many questions?)
and form (how will they be worded, ordered?)
– Important to pilot test tools
4
5
Additional Threats to Credibility
 Evaluate data collectors as a source of bias and
inconsistency
 Of particular concern in qualitative research where researcher
is central to data collection, analysis, and interpretation
– Personal beliefs, expectations, preferences can affect decisions made
throughout the research process
– May perform data collection inconsistently, creating variability in the
data
– Because of this, must reflect on and report all aspects of data gathering
that might have affected data quality/findings
 Evaluate participants as a source of bias and
inconsistency
6
Pillar 2: Analyzability
• Data must be processed and verified completely and
accurately.
 Processing: converting “raw” data into a usable format
Changing format of the data
– (e.g., if record a focus group, must covert audio to text, called
transcription)
Identification of and correcting errors and missing data
– (e.g., look for inconsistencies, may find a question that wasn’t
asked, or failure to code something)
7
Analyzability
• Verification: look for explanations that support or
contradict early themes
 Peer debriefing: allow expert peers to review data and
results to identify biases/errors in data gathering and
analysis that introduced inaccuracies or led to
weak/incorrect interpretations
 Reflexive journal: record events, thoughts, decisions,
feelings on process and how researcher may have affected
outcomes; review these to see if they influenced
interpretations
8
Analyzability: Verification:
Triangulation
• Contrast and compare data from other sources
 Data Triangulation
 Compare data on same issue from different data source
– E.g., client responses compared to therapist responses
 Method Triangulation
 Use multiple methods in a study (e.g., interviews vs. focus
group vs. observation)
 Investigator triangulation (Reliability)
 Multiple people collecting and analyzing data.
9
Analyzability: Verification
• Member Checking:
Some researchers may consult with participants
after analysis to ensure accuracy and
completeness
Question-answer validity
Interviewer/focus group moderator engages in realtime paraphrasing of comments to confirm or
clarify intended meaning
– E.g., check in with a paraphrase – does this accurately
reflect your views?
10
Pillar 3: Transparency
• Completeness and disclosure in the final
document
“Thick description”: provision of rich details of
the phenomena, and context in which they
occurred, not just facts
Allows reader to decide if they would have reached
similar interpretations
Evaluate degree to which outcomes can be
transferred (apply to other areas)
11
Pillar 4: Usefulness
• To what degree is the study useful?
How can the study be interpreted, acted upon or
applied in the “real world”?