Lecture 2
Study Designs:
Overview, analytic
techniques and measures
of association/effect
Larry Holmes, Jr.,
Jobayer Hossain
Research Statistics Lecture Series
Nemours, DE - October, 2008
Research STATS
What do you see? Error!!! Bias!!!
Lecture Objectives
At the end of this presentation
participants will be expected to:
– Identify and describe the types of
designs used to conduct clinical studies
– Understand the selection of study
subjects or participants in research
– Identify the measure of association or
effect
– Estimate relative risk and odds ratio from
cohort and case-control studies
respectively
Why Design Strategies?
Inappropriate study designs generate
validity issues that cannot be corrected
by statistics, however sophisticated the
model we use
“No intelligent person would
underestimate the importance of
mathematics in science or question the
necessity for its correctness, but it cannot
bring truth out of error. If it is applied to
truth it will produce truth, and if it is applied
to error it will almost certainly produce
error” – Dingle H, 1958
Design or Sampling?
Research Feasibilities: Health Care Outcome Approaches
Management
Patient
Outcome Measures
Health Care Provider
• prognostic factors
• medical specialty
• survival
• knowledge/interest learning
about cancer
• medical
coverage/insurance
• recurrence
• coping strategies
• access to clinical trials
• quality of life
• support group
availability
Treatment/Diagnostic
• treatment recommendation
• treatment given
• new disease
• treatment
complications
• health related quality
of life
• disease-specific
impairments
• appropriateness of treatment
• satisfaction with care
• pain/symptom management
• financial & family
burden
Study Conceptualization: Framework
Inferences Re
Conceptual Ho
Theory/
Knowledge/
Problems
Conclusions and
Interpretations
Synthesis & Hunches
Conceptual
Hypotheses
Inferences Re
Operational Ho
Study Design
Operational
Hypotheses
Empirical
Findings
Data Collection
Data Analysis
Observations/
Data
Research conceptualization: Schematic representation of
the natural history of disease
Outcome
Death
E
Clinical
Disease
Chronicity
B
A
C
D
Clinical Horizon
STAGE OF
DISEASE
Residual
Disability
SUSCEPTIBILTY
TISSUE
CHANGES
LEVEL OF
PREVENTION
MODES OF
INTERVENTION
PRESYMPTOInapparent or
MATIC
CLINICAL
DISEASE
Subclinical
Pre-Pathogenesis
Primary
Health Promotion
Specific
Protection
Pathogenesis
Secondary
Detection
Early
Diagnosis
Prompt
Treatment
Tertiary
Treatment and
Rehabilitation
Limitation of
Disability
DISABILITY
OR RECOVERY
Resolution or
Sequelae
Research/Study Design
Descriptive
To: Describe an experience,
programs, treatment,
unusual observation
Examples:
1) Case Reports
2) Case series
3) Ecological study
Analytic
Examine etiology, efficacy
Observational
Association of cause and effect
Comparison between 2 treatments
Experimental?
Evaluate the efficacy of
a therapeutic , or other
interventions
Examples:
Examples:
1) Cross-sectional
2) Case-control
3) Cohort
1)
2)
Clinical Trial
Community trial
Meta-analysis?
Combination of studies
Ecological Study Design
The strategy is based on determining whether those
ecological units with a high frequency of exposure also
tend to be the groups with a high frequency of health
outcome occurrence.
Measure group rates of the health outcome and exposure
prevalence for the same population group, but not
necessarily using the same source of data.
Types of comparisons:
– Geographical or group comparisons, e.g., countries or
administrative units.
– Temporal comparisons
– Combines geographic and temporal comparisons
– Time-series studies.
Cross-Sectional Study
Cross-sectional studies typically conducting a medical survey in a
community or defined population – orthopedic hospital. Key steps in
conducting a cross-sectional medical survey are:
– To identify the base population
– To choose a sampling design and sampling frame for selecting the
study participants
– To measure exposure (spine fusion) and health outcome status
(post-operative pancreatitis) on the study subjects.
A critical aspect of conducting a medical surveys is the logistical plans for
examining the subjects and obtaining and processing any biological
specimens.
Cross-sectional studies may also be based on existing records. For
example, a study using data from Fracture Registry to evaluate the
association between osteopena or ontogenesis imperfecta and repeated
fractures
Cross-sectional design
Sampled or surveyed population
Exposure – risk
present (eg.
Developmental
motor delay) – (n?)
Disease –
thoraco
lumbar
kyphosis
(TLK)
progressi
on (n1 ?)
Nondisease
(n2?)
Prevalence (PE)
– n1/(n1 + n2)
Non-exposure –
risk absent (eg. No
developmental
motor delay) – (n?)
Diseas
e – TLK
progres
sion
(n?)
Nondisease
– (n?)
Prevalence (PU)
– n1/(n1 + n2)
Relative prevalence – PE/PU - Using 2x2 table a/(a+b) / c(c+d). Prevalence odds
ratio – ad/bc
Measuring Association in a Cross-Sectional Study
Simplest case is to have a dichotomous
outcome and dichotomous predictor
variable
Everyone in the sample is classified as
diseased or not and having the risk factor
or not, making a 2 x 2 table
The proportions with disease are
compared among those with and without
the risk factor
2 x 2 table for association of disease and exposure
Disease
Yes
Yes
No
No
a
b
a+b
c
d
c+d
a+c
b+d
N = a+b+c+d
Prevalence ratio of disease in exposed and unexposed
Disease
Yes
Yes
No
a
b
PR =
No
c
d
a
a+b
c
c+d
Case-Control Study


Is a study that starts with the identification of
persons with the outcome of interest and a
suitable control group of persons without the
outcome.
The relationship of an exposure to the outcome is
examined by comparing those with and without the
outcome with regard to how frequently the
exposure is present, or if quantitative, the levels of
exposure in each of the groups
Syn: Case-referent Study; Retrospective Study;
What is a case-control study?
Case-control study is a method of
sampling a population in which cases of
disease are identified and enrolled, and
a sample (controls) of the population
that produced the cases is identified
and enrolled.
Exposures are determined for
individuals in each group.
What is a case-control study?
(Cont’d)
Case-control study has two groups (case
group and control group): one group has the
disease of interest (cases) and a comparable
group is free from the disease (controls).
The case-control study identifies possible
causes of disease by finding out how the two
groups differ with respect to exposure to the
study factor of interest and other factors.
Whatever selection factors in the referral system affected
admissions of cases to a certain hospital would also affect the
admission of hospital controls.
Characteristics of the
Case-Control Study
A single point of observation.
Defined by presence or absence of
the outcome.
Exposure is determined
retrospectively.
Does not directly provide incidence
data.
Design of a Case-Control Study
Hallmark of the case-control study is
that it begins with people with the
disease (case) and compares them to
people without the disease (control).
This is in contrast to the design of a
cohort study
Design of a case-control study
Design of a case-control study. A, Starting with cases
and controls.
Design of a case-control study. B, Measuring
exposure in both groups.
Design of a case-control study. C, Expected findings if
the exposure is associated with disease.
Design of case-control studies
First Select
Cases (With
Disease)
Controls (Without
Disease)
Then Measure Past Exposure
Were exposed
a
b
Were not exposed
c
d
a+c
b+d
a/(a+c)
b/(b+d)
Total
Proportions exposed
Odds Ratios in
Case-Control Study
Odds of an event is defined as the ratio
of the number of ways the event can
occur (P) to the number of ways the
event cannot occur (1-P).
Odds= _P__
1-P
If probability of winning n= 60%
Then, odds of winning = 60%/(1-60%)
= 60%/40% = 1.5.
Odds ratio in a cohort study
Odds ratio in a case-control study.
Odds Ratio From a Case-Control Study
First, Select
Then, Measure
Past Exposure
CHD Cases
Controls
Smokers
112 (a)
176 (b)
Nonsmokers
88 (c)
224 (d)
200 (a + c)
400 (b + d)
56%
44%
Totals
Proportions
smoking
cigarettes
Analysis in Case-Control Studies
Odd Ratio (OR) – crude and adjusted (from multivariate
statistical analysis)
OR ~= RR when disease is rare and cases/controls representative
of all cases/population with regard to exposure.
Interpretation of an Odds
Ratio (OR)
OR=1 implies no association.
Assuming statistical significance:
– OR = 2 suggests cases were twice as likely as
controls to be exposed, or cases were 2 times
more likely to be exposed than controls.
Above Example: Patients with CHD were 1.62
times more likely to be smokers than controls
without CHD.
– OR < 1 suggests a protective factor.
What is a cohort?
A cohort is defined as a population
group, or subset thereof, that is
followed over a period of time.
The term cohort is said to originate
from the Latin cohors, which referred to
one of ten divisions of an ancient
Roman legion.
What is a cohort? (cont’d)
Cohort group members experience a
common exposure associated with a
specific setting (e.g., an occupational
cohort or a school cohort) or they
share a non-specific exposure
associated with a general
classification (e.g., a birth cohort—
being born in the same year or era).
COHORT STUDIES
Follow-up studies, longitudinal studies, and incidence
studies
Cohort study is the method in which
subsets of a defined population can be
identified who are, have been, or in the
future may be exposed or not exposed
to a factor or factors hypothesized to
influence the probability of occurrence
of a given disease or other outcome.
Historical Cohort Studies
Retrospective Cohort Studies.
When the cohort(s) are assembled from
past records. Follow-up can be either in the
present or from more recent past records.
Historical cohort studies may study either
general population cohorts or specific
exposure cohorts.
Structure of a Cohort Study
DISEASE
EXPOSED
NONCASES
SOURCE
POPULATION
NO DISEASE
RANDOM
SAMPLING
DISEASE
NON-EXPOSED
NO DISEASE
PREVALENT
CASES
FOLLOW-UP
OUTCOME
Design of cohort study
TIME
Exposure
Study group
Comparison
Group
Disease
Present
a
Absent
b
Present
c
Absent
d
Present
Absent
Disease rate for exposed
=
for nonexposed
Relative Risk
=
a
a+b
= c
c+d
a
a+ b
c
c+ d
Design of a Cohort Study
Start with a group of subjects who
lack a positive history of the
outcome of interest and are at risk
for the outcome.
Include at least two observation
points: one to determine exposure
status and eligibility and a second
(or more) to determine the number
of incident cases.
Design of a Cohort Study
(cont’d)
Permit the calculation of incidence
rates.
Can be thought of as going from
cause to effect.
Involve the collection of primary
data.
If we observe an association between an exposure
and a disease or another outcome, the question is:
Is the association causal?
Design of a cohort study.
Prospective Cohort Study
Purely prospective in nature;
characterized by determination of
exposure levels at baseline (the
present), and follow-up for occurrence
of disease at some time in the future.
Figure 9-6. Time frame for a hypothetical
prospective cohort study begun in 2004.
Advantages of Prospective
Cohort Studies
Enable the investigator to collect
data on exposures; the most direct
and specific test of the study
hypothesis.
The size of the cohort is under
greater control by the investigators.
Advantages of Prospective
Cohort Studies (cont’d)
Biological and physiological assays can be
performed with decreased concern that the
outcome will be affected by the underlying
disease process (e.g., cholesterol levels or
nutrient levels).
Direct measures of the environment (e.g.,
indoor radon levels, electromagnetic field
radiation, cigarette smoke concentration)
can be made in order to define exposures
precisely.
Retrospective Cohort Study
Despite substantial benefits of
prospective cohort studies,
investigators have to wait for cases
to accrue.
Retrospective cohort studies make
use of historical data to determine
exposure level at some baseline in
the past.
Time frame for a hypothetical retrospective
cohort study begun in 2004.
Figure 9-8. Time frames for a hypothetical
prospective cohort study and a hypothetical
retrospective cohort study begun in 2004.
Advantages of Retrospective
Cohort Studies
A significant amount of follow-up may
be accrued in a relatively short period
of time.
The amount of exposure data collected
can be quite extensive and available to
the investigator at minimal cost.
Prospective cohort study (Framingham Heart Study)
5,889 subjects aged 30 or older (mean age 55 years,
54% women) included
Free of heart failure at the baseline examination (in
1976-79, and offspring in 1979-83)
Followed them up to the year 2001 (mean 14 years)
To ascertain the occurrence of new heart disease
Calculate the cumulative incidence of heart disease,
age-adjusted incidence
Calculate the relative risk (hazard ratio) of heart
disease in those obese subjects compared to those
with normal weight.
Results: RR = 1.39 (95% CI: 1.12-1.72) for overweight,
and 1.98 (1.54-2.56) for obese subjects, compared with
subjects with normal weight (reference group).
Smoking and Coronary Heart Disease (CHD): A
Hypothetical Cohort Study of 3,000 Cigarette
Smokers and 5,000 Nonsmokers
CHD Does
CHD
Not
Develops Develop
Smoke
cigarettes
84
2,916
Totals
3,000
Do not smoke
cigarettes
87
4,913
5,000
Incidence
per 1,000
per Year
28.0
17.4
Relative Risk (RR) = 28.0/17.4 = 1.61.
Risk Calculation in a Cohort Study
Then Follow to See Whether or not the outcome occurs
First Select
Disease
Develops
Disease
Does Not
Develop
Totals
Incidence
Rates of
Disease
Exposed
a
b
a+b
a/(a+b)
Not
exposed
c
d
c+d
c/(c+d)
Incidence in exposed =
a/(a+b)
Incidence in non exposed =
c/(c+d)
Relative Risk in Cohort Study
Relative Risk (RR) - provides information
on the relative effect of the exposure on
the disease.
RR= _risk in exposed____
risk in nonexposed
Indicates how many times higher or
lower the disease risk is among the
exposed as compared to the
unexposed.
Is commonly used in etiologic/incidence
research.
Relative Risk (RR) of a Disease
If RR = 1 Risk in exposed equal to risk in
nonexposed (no association)
If RR > 1 Risk in exposed greater than risk in
nonexposed (positive association;
possibly causal)
If RR < 1 Risk in exposed less than risk in
nonexposed (negative association;
possibly protective)
Relative merits: cohort studies
Advantages
Clear temporal
relationship
Least susceptible to
some forms of bias
Can examine multiple
predictors of outcome
Efficient for rare
exposures
Useful when RCT
infeasible, unethical
Disadvantages
No control over
predictor (vs. RCT)
– confounding
Inefficient for rare or
long-latent diseases
Loss to follow-up
threatens validity
Potential bias in
outcome ascertainment
Relatively resourceand time-intensive
Clinical trial design
Study population
Treatment
Yes
No
Random
assignment by
investigator
No treatment
(usual care, placebo)
follow-up
period
Yes
No
outcome of
interest
Estimate of effect is rate (risk) of outcome in treatment vs.
control (e.g. risk[treatment]/risk[control])
Relative merits: clinical trials
Advantages
strong claims for
causality
control of most bias,
confounding
tight control on
exposure/treatment
high internal validity
possible to examine
multiple outcomes
Disadvantages
time consuming
expensive, resource
intensive
compliance, drop-out
sometimes severe
ethical constraints
may not mirror practice
generalizability may be
limited (i.e. selection
bias)
Hierarchy of Evidence
Meta-analysis
Hierarchy of Research Design Quality
– Randomized, prospective, community and/or clinical
trial (Cochrane and Campbell collaborations)
– Cohort studies
– Case-control studies
– Cross-sectional studies (prevalence studies)
– Descriptive/ecologic (correlation) studies
– Case series/case reports
– Individual evidence (personal experience/expert opinion)
Questions