Introduction to Research Methods
Session 5: Overcoming Bias and Confounding
Prof. David Taylor
Director of Emergency and General Medicine Research, Austin Health
Chair, Austin Health Human Research Ethics Committee
Overview:
Bias
•
•
•
•
selection bias
measurement bias
identifying bias
minimising bias
Confounding
•
•
•
•
common confounders
controlling for confounders
minimising confounding
residual confounders
Bias in a Research Study:
Research Principle
• To determine the relationship between a risk factor
(exposure) and a disease (or outcome) of interest
e.g. smoking - lung cancer
overweight - diabetes
polio vaccination - disease eradication
Bias in a Research Study:
Bias
• the way a study is designed or carried out that can result
in an incorrect conclusion about the relationship between a
risk factor and the disease
• leads to an incorrect estimate of the effect of a risk factor
on the development of a disease
• two major types: 1. Selection bias 2. Measurement bias
• cannot be controlled:
• in the analysis of a study
• by increasing the sample size
Selection Bias:
Selection Bias
• Invariably, we enroll a sample of the „whole population‟
e.g. mail survey
100 patients for an RCT
• We assume that the sample is representative of the
„whole population‟
• Selection bias occurs when there is a difference between the
people enrolled and the „whole population‟
• The results may be differ from those had a truly representative
sample had been enrolled
Selection Bias:
• If substantial selection bias occurs, we can not
generalise the results to the whole population
Example:
• in 1965, a survey was done by telephoning a sample of
the general population from the phone book
• only wealthy people in those days had telephones
• wealthy people were over-represented
• the results were biased
Selection Bias:
Example
• You enroll volunteers to study the frequency of hormone
replacement therapy (HRT) in women aged over 45 years
• People who volunteer for studies are often different:
• more health conscious
• better-educated
• more likely to be on HRT
• This method is likely to suggest that a high proportion of
women are on HRT
• Alternative: sample women from the electoral roll
Selection Bias:
Recruitment methods:
• advertisements
• invitations of noticeboards (back of the dunny door)
• mail / on-line surveys
All tend to attract:
•
•
•
•
the interested
those with an agenda
the disgruntled
the research participant „tragic‟ (the frequent flyer)
Selection Bias:
Convenience sampling
• Sampling at a time that is convenient for the researchers
(or potential participants)
• Is there a difference between patients at convenient and
inconvenient times?
Examples:
• Prevalence of drug/alcohol use among ED patients
• Patient satisfaction with ED care
• Aim to sample 24 hours/day, 7 days/week
• Pilot data to justify your convenience sampling
Selection Bias:
Refusals
• Patients who refuse are often different:
•
•
•
•
sensitive issues e.g. life style/risk factors
language/reading difficulties
cultural issues
previous experience
• Aim to minimise the refusal rate:
• friendly, non-confrontational approach
• anonymity
• non-judgemental
Selection Bias:
Loss to follow up
• Many causes:
•
•
•
•
Death
Falsification of contact details
Decision to withdraw
Moved
• Selection bias often introduced
e.g. deaths will affect known (reported) survival rates
• Aim to minimise loss to follow up
• Triangulation of contact details
• Other sources of data
Selection Bias:
The „Healthy Worker‟ Effect
• may occur when the general population is used as the
comparison group
• the overall death/disease rate in workers is lower than
that of the general population
• workers are healthier members of the community (the sick don‟t
work), so they are selected on the basis of their health status
• the general population includes those who are too ill to be
employed
Selection Bias:
Randomised controlled trials
• Selection bias less of a problem
• Inclusion/exclusion criteria are clearly defined
• Patients are randomised to the study groups
• Beware:
• refusal to participate after randomisation
• withdrawals from the study
Selection Bias:
Minimising Selection Bias
• Was the study population clearly defined?
• What were the inclusion and exclusion criteria?
• Were refusals, losses to follow-up, etc kept to a minimum?
Cohort and Intervention studies:
• Are groups similar except for exposure/intervention status?
• Is the follow-up adequate? Is it similar for all groups?
Case-control studies:
• Do the controls represent the whole population?
• Selection of cases or controls influenced by exposure status?
Measurement Bias:
Measurement (information) Bias
• Occurs when there is inaccuracy in measurement of:
• Data items
• Classification of disease
• Risk factor level
• Errors in measurement may occur because of the:
• Data collection tools (pure measurement bias)
• Investigators (observer bias)
• Study participants (responder bias)
Measurement Bias:
Pure Measurement Bias
• Relates to the accuracy of data collection tools
Examples:
• non-validated questionnaire
• poorly calibrated instruments
• non-standard instruments across sites
Measurement Bias:
Observer bias
• Variability in data collection by the researcher(s)
• Conscious or unconscious
• Minimised by:
• Blinding
• Not knowing the study hypothesis
• Objectivity (not subjectivity) of data items
• Should be evaluated:
• Comparisons with other sources of data
• Inter-observer tests (kappa scores)
Measurement Bias:
Responder Bias
• Conscious or unconscious
Types:
• Comprehension bias:
• Misunderstanding
• Language
• Instructions
• Prevarication bias:
• Aiming to please
• Embarrassment (self-protection)
• Deliberate deceit
Measurement Bias:
Minimising Measurement Bias
•
•
•
•
•
•
•
Were the exposures/outcomes of interest clearly defined?
Were the measurements as objective as possible?
Was the study “blinded” as much as possible?
Were the observers/interviewers rigorously trained?
Were clearly written protocols used for data collection?
Were the study subjects randomised to interviewers?
Was information provided by the patient validated against
any existing records?
• Were any strategies built into the study design to assess the
magnitude and direction of the bias?
Confounding:
Confounding factor:
• is associated with the exposure (risk) under study and
independently affects the risk of developing the disease
• may itself affect the observed outcome if unequally distributed
between study groups
• may offer an alternative explanation for any association found
between a risk factor and a disease
• a wrong conclusion about an association may be made
• must be taken into account when collecting and analysing data
• accounted for mainly in the study design
Confounding:
Example:
Fact:
Fact:
Conclusion:
More people are stung by jelly fish during summer
Jelly fish are more abundant during summer
The excess of stings in summer is related to the
abundance of jelly fish
Confounder: ??
Confounding:
Example:
Fact:
Fact:
Conclusion:
More people are stung by jelly fish during summer
Jelly fish are more abundant during summer
The excess of stings in summer is related to the
abundance of jelly fish
Confounder: More people swim in summer
-increased risk (exposure) of the disease (outcome)
Confounding:
• To be a confounder, it must be:
• associated with the risk factor under study
• a risk factor for the disease being investigated
• A potential confounder is believed to have a real effect on
the risk of the disease under investigation
Example: Lung cancer prevalence across counties
• a direct causal link with the disease
e.g. smoking and lung cancer
• a proxy measures of more direct unknown causes
e.g. age, socioeconomic status, occupation
Confounding:
Common (universal) confounders
• need to be considered in almost every study
•
•
•
•
Age
Gender
Ethnicity
Socioeconomic status
Example:
•
•
•
•
Age is associated with increased rates of many diseases
If one group is older then the other, then it will be at increased risk
So, this relationship would be confounded by age
The risk may be attributed to the older group‟s exposure, not age
Overview:
• Consider Table 1 of just about every RCT publication
• Demonstration that potential confounding variables are
equally distributed between the study groups
• Universal confounders
• age, gender, ethnicity, s-e status
• Study specific confounders
• Cardiac studies: obesity, cholesterol, hypertension, activity
• Dementia studies: alcohol use, drug use
• Pancreatic cancer studies: smoking, caffeine, occupation
RCT comparing three
drug regimens for the
sedation of acutely
agitated ED patients
Confounding:
Controlling for Confounding
• There are several ways to “control for” the effect of
confounding on the exposure-disease relationship
• To control for confounding during the design of the
study, there are three possible alternatives:
• randomisation
• restriction
• matching
Confounding:
Randomisation
• the cornerstone of a randomised controlled trial
• equally distributes potential confounding factors
between the control and intervention groups
• factors are then equally common to all groups
• factors no longer are associated with the exposure (the
treatment) and cannot confound the relationship
Note:
• Randomisation equally distributes unknown confounders
equally between exposure groups, as well
• Randomisation does not always distribute equally
Confounding:
Restriction
• Restricting the participants to one level of a potentially
confounding variable
Example:
In a study exploring the association b/w coffee and pancreatic
cancer, restrict all cases and controls to be non-smokers
-smoking could no longer be a confounding factor
• restriction is quite straightforward
• limitation:
• can only control for those few variables that are restricted
• numerous restrictions makes enrolment near impossible
Confounding:
Matching
• Matching subjects on potential confounding variables
• Ensures that these variables are evenly distributed
• Especially useful in case-control studies
• a case of a certain gender and age group
• select a control to „match‟ the gender and age group
• Match for variables that are potential confounders for the
study outcome (disease) of interest
Confounding:
To control for confounding in the analysis phase
of a study, you can use:
• stratification
• statistical modeling (regression)
Confounding:
Stratification
• the effect of potential confounders can be assessed
within separate strata of the confounding variable
Example:
• a study is examining whether age is confounding the
relationship between exercise and heart disease
• the study groups are divided into separate age groups
• within each age group, the association between exercise
and heart disease is evaluated
• the results from each strata can be „merged‟
Confounding:
Statistical modeling
• Multiple regression models offer the benefit of controlling
for multiple confounders simultaneously
• In analysing data where there may be several confounding
factors:
• begin with stratification on each individual factor
• assess for both confounding and effect modification
• incorporate the important factors into the final model
Confounding:
Residual confounding
• only possible to control for confounders in the analysis if
data on confounders were collected
• extent of control depends on the accuracy of the data
Example:
non-differential (random) misclassification of exposure to a
confounder will tend to underestimate the effect of the confounder and
so will decrease the amount to which confounding can be controlled.
• the association will persist even after adjusting for the
confounder because of residual confounding.
Summary:
Selection Bias
• Recruited the wrong patients
• Results will not be generalisable
Measurement Bias
• Mistakes in the accuracy of data collection
• Address before any data is collected
Confounding
• Factors that may offer another explanation for the results
• Need to be considered and controlled for