Study Design
Study Design
• Experimental studies
• Prospective observational studies
• Retrospective observational studies
• Exercise
Experimental
Studies
Introduction
• In many cases, we want to compare two (or more) conditions or groups of individuals
• Does a certain drug reduce migraine frequency?
• Do smokers have an increased risk of HBP?
• Do men and women spend different amounts of time exercising?
• Exactly how we make these comparisons and what results mean depend on the design of
the study
• How subjects sampled
• How groups are formed
• How data collected
Introduction
• Question may be causal or associational
• Causal: Does exposure to second-hand cigarette smoke increase the risk of lung
cancer?
• Associational: Are patients who present with disease A more likely to develop
disease B later in life?
• A does not cause B, but it indicates it is more likely
Introduction
• Experimental studies go by many names:
• Experiments
• Randomized studies
• Randomized controlled trials (RCTs)
• Main idea: each individual’s exposure to treatment/ intervention is under the
researcher’s control and is chosen by a random mechanism
• Example: coin flip
Randomization
• The process by which subjects get assigned to groups
• Treatment: active intervention
• Control: no intervention
• If everyone has a 50/50 chance of being in treatment group, then two groups will be similar
• Say there are 1000 subjects, 400 men and 600 women
• How many men do you expect in the treatment group? Control group?
• How many women in each group?
• Groups are balanced
• Measured and unmeasured variables
Randomization
• Theory:
• Treatment and control groups should be similar at baseline (before
treatment is assigned)
• Any differences in the outcome are likely due to treatment
• Results can be interpreted as the effect of the treatment (causal)
Adherence
• In a randomized trial, participants may not adhere to their assigned group
• People in treatment group may not take their assigned treatment
• Causes two problems:
• Don’t know who adhered and who did not
• Some tools to help track, rarely perfect
• Those who did adhere will be different from those who didn’t
• No longer balanced
• What to do?
Adherence
• Choice of analysis depends on research question
• Clinical efficacy: how well the treatment works when protocol is followed
(regulatory approval)
• Biological
• Real-world effectiveness: changes doctor can expect to see in patient given
treatment (expected change in practice)
• Biological + Psychological/Social
Adherence
• Intention-to-treat (ITT): Estimate effect of treatment assignment on outcome
• Not the causal effect of treatment
• Estimates real-world effectiveness
• Depends on adherence level
• Include all patients
• Analyze all patients randomized to treatment as treated, patients randomized to
control as untreated
• Limitations
• Misclassification → underestimate efficacy
Adherence
• Per-protocol (PP): Estimate effect of treatment on those who actually took it
• Causal effect of treatment
• Estimates clinical efficacy
• Remove any subjects who did not adhere to assigned treatment
• Compare outcomes in adherent groups
• Limitations
• Loss of sample size
• Introduce selection bias, confounding
Observational studies
Prospective
observational studies
(Cohort studies)
Retrospective
observational studies
(Case-control studies)
Introduction
• Not all studies can be randomized
• Time to follow up can be prohibitively long
• Diseases may take years or decades to develop
• Practicality
• How to ensure adherence to designated group
• Ethical concerns
• Cannot assign people to a treatment/intervention we expect to do harm
• Cannot keep people from taking a treatment known to be effective
• Bodies in place to ensure ethical standards (e.g., FDA, SLU IRB)
Introduction
• An alternative to experimental studies
• Exposure not under researcher control, but naturally occurring
• Air pollution, heat waves
• Smoking, binge drinking
• Racial/ ethnic or gender differences
• Government policies
• Researchers observe but do not actively alter subject’s daily lives
Confounding
• Results are not causal due to potential confounding variables
• Variables that (1) are not balanced between the exposed and comparison
group and (2) influence the outcome of interest
• Ex: Studying association between air pollution and health
• People with low SES more likely to live in polluted areas
• Also more likely to have worse health
SES
Air pollution
?
Health
Confounding
• Can account for some confounding using multivariate analyses
• Stratification: Make comparisons within specific subgroups, then
combine results across groups
Sex
SES
Males
Low
Males
Moderate
Males
High
Females
Low
Females
Moderate
Females
High
Avg risk in
low pollution
Avg risk in
high pollution
Confounding
• Never know if you have controlled for all possible confounding
• In example, adjusted for sex and SES. Any other variables of interest?
• May not be able to collect new variables after the fact
• Conclusions depend on assumptions about variable relationships
• No unmeasured confounders
• Proposed mathematical relationships correct
Study designs
• Two broad categories of observational studies are prospective and retrospective studies
• Prospective studies: subjects are recruited, and data collected before outcome occurs
• Often called cohort studies
• Ability to follow people over time
• More accurate measurements
Study designs
• Retrospective studies: subjects are recruited, and data collected after outcome has
occurred
• Quicker
• Data subject to memory, records, and recall bias
• One example: case-control studies
• Take a sample of people with outcome and sample of people without outcome
• Useful for rare diseases (e.g., Zika)
• Can get a large number of cases without having to take a large sample and wait
years
Summary
Summary
Experimental studies
Observational studies
Terminology
Treatment/ intervention
Exposure
Control
Comparison
Blocking
Stratification
Design
Prospective
Prospective or retrospective
Interpretation
Causal
Associational
Potential problems
Non-adherence/non-compliance
Confounding
Dropout
Dropout (prosp) or recall bias (retrosp)
Summary
• Random sampling
• Results can generalize to the population
• Major concerns: non-response, response biases
• Random assignment (randomization)
• Results can be interpreted causally
• Major concerns: adherence, dropout
• Statistical tools for mitigating biases, but not fool-proof
• Depend on unknowable assumptions
Exercise
Problem 1. 1.10
Problem 1. 1.10
(a) Each row of the data matrix represents a different participant from which data
regarding smoking habits was collected from.
(b) There were 1691 participants included in the study.
(c)
 age: numeric, continuous
 Sex: categorical; nominal
 marital status: categorical, not ordinal
 income: categorical, ordinal
 smoke: categorical, not ordinal
 amtWeekends, amtWeekdays: numeric, discrete
Problem 2. 1.41
Problem 2. 1.41
a. This is an example of a randomized experiment.
b. Explanatory variable: Fruit/vegetable consumption group
Response: Psychological well-being
c. Can be generalized to young adults, the group the sample was taken from
d. Yes. Since participants were randomized to groups, results can be interpreted as causal.
e. Could say something about “actually providing the fruit/vegetables to them” “positive
association was found between young adults eating fresh fruits/vegetables and
improved psychological well-being”. (other possible answers)
Designing
studies:
practice
• Questions:
• Problem 1: Effect of new
medication on COVID
recovery.
• What is the research question?
• What would be the
exposure/treatment and
outcome(s)?
• Experimental or observational?
• Who to recruit?
• How to ensure adherence/followup?
• What other information to
collect?
Next week
Descriptive statistics