STUDY DESIGN
Noura Maher
Bsc.pharma ,PharmD ,BCPS ,HTA,ALX
fellowship
Learning Objectives
Define, compare, and contrast the concepts of internal and external validity, bias, and confounding in
clinical study design, and identify potential sources in clinical trials.
Outline the hierarchy of evidence generated by various study designs and compare and contrast the
advantages and disadvantages of various study designs.
Compute, define and evaluate odds, odds ratio, risk/incidence rate, risk ratio/relative risks (RRs), and
other risk estimates.
Compute and evaluate number needed to treat and number needed to harm. Define point and period
prevalence, incidence rate, prevalence rate, absolute risk difference, and RR difference.
Describe the various steps of the professional writing and peer‐review processes.
New drug ,,,,New outcomes
Assumption, hypothesis testing.
Association /relation vs Causality
Comparator vs standard of care or Placebo.
Interventional(experimental) vs observational.
Descriptive vs analytical.
Retrospective vs prospective.
Time horizon
Prospective Begin in the present and progress forward, collecting
data from subjects whose outcomes lie in the future.
Retrospective Begin and end in the present (a major backward
look to collect information about events that occurred in the past).
Outline the hierarchy of evidence generated by various study designs and
compare and contrast the advantages and disadvantages of various study
designs.
Research study design is a framework, or the set of methods and procedures used to collect and
analyze data on variables specified in a particular research problem.
Research study designs are of many types, each with its advantages and limitations.
The question is determined by:
• The nature of question.
• The goal of research.
• The availability of resources.
Types of study
Hierarchy of
clinical
evidence
Level of evidence
Proposed
revision to
hierarchy of
clinical
evidence.
Evidence
based
medicine
EBM
Experimental studies
Analytical studies
Interventional studies
Randomized
Controlled
Trial
RCT
New : antiDM sasgliptin
Control : saxagliptin
Research Question :is it right higher HA1C reduction
with new ?
Outcomes :HA1C
Significance of outcomes (P value ,95% CI)
Randomization
Each participant in the sample group have equal chance to be assigned to treatment
group or control group.
Controlled
all the steps
are controlled
by the
researcher
control group
Hallmark of clinical research.
Study design seeks to minimize bias through randomization of
subjects, blinding of participants, and analytic approach.
RCT
This should leave at least two groups that differ only in study
treatments.
Experimental design to test the effects of an intervention compared
with either placebo or the established standard of care (treatment
or process of care); allows for description and causality
BLINDING
Single
Double
Triple
Blinding
SINGLE
BLIND
DOUBLE
BLIND
TRIPLE
BLIND
Doubledummy
OPEN
LABEL
single-blinding
In a single-blind trial, only
one of the three categories
of individuals (usually
participant rather than
investigator) is unaware of
the intervention assignment.
For example, the participant
is blinded to the treatment
allocation, whereas the trial
investigator and assessors
are aware of the
intervention.
Double-blinding
In a double-blind trial, both the participants in the study (subjects)
and those involved in the assessment (investigators and assessors) are
unaware of the randomization schedule.
For studies involving investigational agents, such as phase III clinical
trials, at least two double-blind trials are typically required for the
drug to be approved by the FDA.
triple-blinding
The most objective type of trial design is the triple-blind study. A triple-blind
design is an extension of the double-blind study where patients and investigators
are blinded. In triple blinding, an external group of individuals who are involved in
monitoring the outcomes of the study are also kept unaware of the treatment
allocations.
For example, data may be sent to the review board in a blinded fashion as
Treatment A and Treatment B. An advantage of the triple-blind study is that all
individuals (patients, investigators, and monitors) involved in the allocation,
evaluation, design, and monitoring are able to objectively review the study
results.
Open label
The least objective type of design is the open label study. In an open label study,
all participants, investigators, and assessors are aware of the treatment
allocation. This type of design should be avoided in studies that involve subjective
assessments or outcome measurements comparing different treatments due to
the potential for patient reporting and investigator bias.
The open label design is usually restricted to early pharmacokinetic studies, or
phase I trials, where objective information about drug exposure (such as
absorption, distribution, metabolism, and excretion) and safety is learned about
the investigational agent.
Blinding
Double-dummy:
• Two placebos necessary to match active and control therapies
• Example when we compare the difference between iv infusion vs oral drug
 Treatment group iv infusion drug +placebo oral form
 Control group iv infusion placebo (saline) +oral drug
Open-label:
• Everyone is aware of subject assignment to active/control
Double-dummy
Two placebos necessary to match active and control therapies
RCT(parallel vs Crossover) interpersonal
difference , check if applicable or not ?
parallel
Crossover
PARALLEL
 All members of the group receive the treatment/placebo over the
duration of the study. This is the most common design that is used for
phase III comparative trials, where patients with a given disease are
randomized to receive either the experimental drug or the standard
therapy and followed over a specified period.
Advantages of a parallel study include strength of the design and
shorter time needed to conduct the trial.
Disadvantages of this type of design usually requires a large sample
size when compared to crossover design.
CROSSOVER
In a crossover design, each subject receives all of the interventions
based on a specified sequence of events.
For example, if two interventions are to be studied, subjects will be
randomized to a treatment sequence, such as “AB” or “BA.”
In this case, patients in the AB group receive Treatment A first, and
then cross over to Treatment B after the appropriate washout period.
For patients in the BA group, subjects receive Treatment B first, and
then cross over to Treatment A.
CROSSOVER
Advantages of this type of paired design can have more statistical
power with fewer subjects than a parallel design, since each subject
undergoes both treatments.
Disadvantages of crossover studies include the need to enroll patients
with more stable disease states. They take longer to conduct due to
washout periods and often have a higher dropout rate than parallel
designs.
Crossover studies are susceptible to problems
related to carryover and washout
Carryover effect Refers to outcomes that remain or linger after the first
treatment phase (such as Treatment A) is completed. These effects can
carry over into Treatment B, which alters the baseline and subsequently
alters the true treatment effect of Treatment B
Washout period which is the time needed for the outcomes of
Treatment A to dissipate prior to beginning Treatment B. This is
particularly problematic for studies involving drugs with slow
elimination or long half lives, where washout periods are typically
extended over weeks or months
Factorial Design
 Factorial randomized trials are designed to evaluate multiple
interventions in a single experiment.
 Factors to be studied can include multiple dose levels and multiple
drug regimens.
 For example, in the simplest 2 × 2 factorial design, patients would be
assigned to one of the two dose levels (i.e., 100 mg or 200 mg) and
one of the two drug choices (Drug A or Drug B).
Factorial Design
RCT
Advantages:
Determines a causal relationship; provides a direct measure of risk; controls for selection bias and
confounding; blinding; detailed information on subjects; may allow for subgroup analyses.
Disadvantages:
Expensive
 time-consuming.
can be complex and difficult to conduct; adherence to study protocol can be poor.
incomplete follow-up of study subjects; guidelines/practice may have changed since the initiation of
the trial.
subjects may differ from the population of interest (external validity)
• To
While historically considered the gold standard for research, many limitations
exist to application and interpretation of randomized controlled trials in the ICU
Heterogeneous population: The assumption that all patients would benefit
from a treatment is unlikely.
Gaps in understanding: Prognostic markers and definitions are often limited,
raising concern for inappropriate inclusion and/or exclusion.
Therapeutic optimization often unknown: Preclinical data is often highly
different from the population studied (healthy humans or rodent models)
limiting confidence in extrapolation.
Clinical equipoise: Uncertainty surrounding the intervention in question may be
subjective, and opinions on what would be considered the standard of care may
differ at local, national, and international levels in the critically ill. An ethical
approach to the comparator arm must therefore be carefully considered
Trial enrichment is a mechanism in which a
specific subgroup of a population who have
higher likelihood of a response to an intervention
are selected over those who may not
Trial
enrichment
Prognostic enrichment selects for patients with
a higher likelihood of disease-related interest
(i.e., mortality)
Predictive enrichment selects for those patients
who are more likely to respond to a treatment
based on underlying biologic mechanisms or
physiology (i.e., inflammatory barkers in ARDS)
Prognostic enrichment &Predictive enrichment
Adaptive trials
Increasing attention has been given to the use of adaptive trial designs. Such
trials seek to increase flexibility in clinical trials by allowing for trial modification.
Prespecified rules dictate modifications upon scheduled interim evaluations of
the data as the trial is ongoing.
Examples of adaptive designs include: Continual reassessment method; groupsequential method; sample size re-estimation; multi-arm, multi-stage population
enrichment; biomarker adaptive; adaptive randomization; adaptive doseranging; and seamless phase I/II or II/II trials.
Adaptive trials seek to increase efficiency of any type of prospective trial and to
minimize potential harm to patients.
Adaptive trials
Adaptive Design
One of the most state-of-the art ways to design clinical
trials is using adaptive designs. In adaptive designs, the
conditions of a study or analysis plan are prospectively
planned to be modified over time based on the results
of preliminary analysis at interim time points.
One potential advantage of this method over traditional
designs is the ability to reduce the number of subjects to
be enrolled in the active intervention group when
compared to the original sample size calculation.
Challenges
Challenges to adaptive trials include
 the complexity of statistical interpretation
 lack of knowledge of the scientific community about these designs
 difficulty in communication of the results.
Pharmacokinetic and Pharmacodynamic trials
1. Studies designed to evaluate effects of unique disease states on
clinically relevant pharmacokinetic and pharmacodynamic
parameters
2. Trial designs can be prospective based on predefined sampling
strategies or retrospective based in routine clinical care laboratory
samples
3. Large study size not needed to define properties in unique patient
populations
4. Population based pharmacokinetic and pharmacodynamic studies
can help define novel clinical dosing strategies in unique patient
populations
Noninferiority
trial
Superiority vs.
Equivalence vs.
Non‐inferiority
A superiority trial:
•Designed to detect a
difference between
experimental treatments.
•Typical design in a clinical
trial.
An equivalence trial:
•Designed to confirm the
absence of a meaningful
difference(s) between
treatments, neither better nor
worse (both directions).
•The example is a
bioequivalence trial.
A non-inferiority trial:
Designed to investigate whether a
treatment is not clinically worse
(not less effective than stated
margin, [non-inferiority margin] or
inferior) than an existing treatment.
a. It may be the most effective, or
it may have a similar effect
b. b. Useful when placebo
administration
Superiority vs. Equivalence vs.
Non‐inferiority
Define, compare, and contrast the concepts
of internal and external validity, bias, and
confounding in clinical study design, and
identify potential sources in clinical trials
internal and external
validity
Validity
Internal
 Does the study design adequately test the hypothesis?
 Are the study methods sound?
External
 Is the study population representative of the clinical setting?
 Are the study findings generalizable outside the study setting?
 Can the study be replicated in clinical practice?
Internal validity vs external validity
bias
Bias
Bias: Systematic error leading to an estimate of association in the study
population that varies from the source population.
a. Selection bias: Systematic selection of subjects that leads to an
imbalance, or an advantage, in favor of one cohort over the other
b. Observation bias: Observers (research team) are aware of the research
purpose and allow this knowledge to influence interpretation of results.
c. Recall bias: Methodological error that is introduced in survey research
when the participant is asked to provide recall of a past event
d. Misclassification bias: Inappropriately categorizing a group of patients
with, or without, the disease/ syndrome
Bias
e. Immortal time bias: Occurs when populations studied include an
exposed and unexposed group without a predefined time-zero. This
results in a delay before a subject is considered treated. Given that the
subjects must survive up until the time of first exposure, the time
before exposure is considered immortal time
f. Measurement bias: When information collected for use as a study
variable is inaccurate
g. Confounding variables: Extraneous variables that influence both the
dependent and the independent variable, affecting how the overall
result can be interpreted
Selection bias (sampling)
Observation
bias
Observation bias (Observers)
Recall bias
Recall bias
Immortal time bias
Approaches to control for bias
Study design:
 Randomization
 matching
Analysis:
 Multivariable analysis
matching potential confounding variables with a controlled cohort
confounding
Confounding variables
Confounding
variables
How to overcome confounding ?
Residual confounding occurs when unmeasured variables may affect
overall outcome results.
This can only be avoided with randomization.
Confounding by indication occurs when a contributor to the outcome
is present in those at a high risk and is an indication for intervention.
This results in care differences between the exposed and nonexposed
groups that may be based on differences in indication for exposure
Sensitivity analysis
These may be used to determine robustness of study findings in
different analysis methods or presence of unmeasured confounding.
They may also be used to detect misclassification.
SIMPLE RANDOMIZATION
BLOCK RANDOMIZATION
RANDOMIZATION
STRATIFIED RANDOMIZATION
ADAPTIVE RANDOMIZATION
Simple Randomization
01
02
03
involves the use of a
random number
generator to allocate
participants to study
groups.
This is equivalent to
flipping a coin in a case
of two treatment
groups, where heads
receives Treatment A
and tails receives
Treatment B.
This is the simplest form
of randomization, but it
is also susceptible to
flaws.
Simple Randomization
For example, this approach can lead to unequal
numbers of subjects assigned to two groups in
studies with a small sample size.
Having unequal numbers in each group
can influence the distribution of baseline
characteristics across the two groups.
Simple Randomization
Block randomization
refers to the process of dividing potential study subjects into a specified number of
“blocks” to be randomized at the beginning of the trial, as a means of ensuring that
the number of subjects in each treatment group will be equal.
Here, the total number of subjects to be enrolled in the study is divided into a series
of “blocks.” Each block has the same number of subjects assigned from each group.
If there are two treatments and one patient is assigned to each treatment, then the
size of each block is 2.
Once all blocks are assigned, the study will have equal numbers of subjects in each
treatment group.
Stratified randomization
is the process of randomization that
ensures balance of participants for
predefined strata based on
prognostic factors such as disease
severity among the study groups.
In addition to balancing the number
of subjects in each treatment group,
it is important to make sure that the
subjects enrolled in each group have
equal numbers of each stratum or
level of a factor, such as age with
strata of old and young patients.
Stratified randomization
 For example, if treatment Group A has more patients over the age of 60 than
treatment Group B, the group containing older individuals may have a different
response than the younger group. Thus, the trial may not be valid due to an imbalance
in patient age.
 To achieve balance within each group, a stratified randomization process is employed.
This involves stratification of patients and randomly assigning patients in each stratum.
Stratified randomization
 This results in groups that are balanced to account for characteristics
of the study population, such as age, gender, race, and disease
severity. Here, it is important that baseline measurements be taken
before randomization, especially in smaller trials.
 In very large randomized trials, stratification is not usually required
because the risk of imbalanced groups is less likely.
Stratified randomization
Adaptive Randomization
 refers to the process of assigning patients to a treatment group generally
based on previous success of the treatment as the trial progresses. Here, the
probability of being assigned to a group changes based on the responses of
the prior patients.
 For example, the ratio of experimental versus control may change from 1:1 to
randomly assign patients to the arm in which the treatments are more
favorable.
 These methods of adaptive randomization are not widely accepted in the
scientific community, and their value has not been fully determined.
Adaptive Randomization
Group or Cluster Randomization
Group or Cluster Randomization Design In a cluster randomization
design, a specific group of subjects is selected for randomization. For
example, patients enrolled in a clinic or hospital.
These groups would then be randomly selected to a specific
sequence of treatments or study procedures.
All members of a cluster would receive the same treatment
sequence.
Group or Cluster Randomization
Cluster randomized trials
1. Randomize groups (clusters) of subjects to either control or
intervention groups
Cluster
2. Require a large sample size given the intra cluster correlation
randomized
coefficient and resulting design effect
trials
3. May result in recruitment bias, more baseline imbalance between
groups, loss of entire clusters, or inappropriate analyses
 Useful in system or group-level intervention (i.e., medication diluent
change for entire ICU),
Cluster
when individual randomization is not possible,
randomized
and to prevent contamination, or

the phenomenon which occurs when providers or subjects learn
trials
about the intervention during the study period and start adopting it as
standard of care, whether within the treatment group or not
Observational studies
Observational Studies
Observation of clinical practice
 no intervention is tested
Describes associations between phenomena
Case control
Cohort
Cross sectional
Observational
(Case-control
study )
Observational (Case-control study )
 Hypothesis generating and retrospective design
 Provides cost-effective means to determine the association between
the risk factor and the outcome of interest.
 The case group that experienced the outcome are compared with the
control group that did not experience the outcome to identify the
differences and risk factors for developing the outcome of interest.
Observational (Case-control study )
Potential for selection bias and confounding because study proceeds from
effect (subjects selected based on outcome of interest) to cause of effect.
Cases and controls are representative of the population with the disease and
should be chosen in a way to minimize selection bias. Controls are randomly
selected out of the same population from which the cases arose
Odd Ratio (OR)
Prospective or retrospective
design
Observational
(Cohort study)
Observational study of a given
population over a given time to
determine the relationship
between exposure and outcome
Describes the natural progression
of a disease or syndrome
Cohort studies
Cohort studies
Observational (Cohort study)
Example: Delirium, defined a positive Confusion Assessment Method
for the ICU examination, is an independent predictor of mortality in
mechanically ventilated patients (JAMA 2004;291:1753- 62).
Cohort of mechanically ventilated patients observed during ICU stay
for presence of delirium.
Determination of outcome: ICU mortality, 6-month mortality.
Regression models to determine whether delirium is predictive of 6month mortality, controlled for covariates.
Relative risk(RR)
Absolute risk
reduction
(ARR)
relative risk reduction (RRR)
Prospective cohort
Advantages: Can control for confounding factors to
a greater extent, easier to plan for data collection
Disadvantages: More expensive and time-intensive,
loss of subject follow-up, difficult to study rare
diseases/conditions at a reasonable cost
Retrospective cohort
Advantages:
Less expensive and time-consuming; no loss to follow-up, ability to investigate issues
not amenable to a clinical trial or ethical or safety issues
Disadvantages:
Only as good as the data available, little control of confounding variables through
Non-statistical approaches, recall bias
Cross
sectional
study
Prevalence of the diseases
Prevalence of the diseases
Prevalence of the diseases
Incidence
rate
Prevalence vs incidence
Descriptive studies
Case
reports/case
series
 Describes the
experience in
treating a single
patient or the
cumulative
experience in
treating a series of
patients
 Typically, novel or
rare patient
population or
intervention
Propensity score matching
a. This method in theory allows for design and analysis of
observational studies to mimic aspects of a randomized controlled
trial.
b. The propensity score itself distributes observed baseline covariates
similarly between treated and untreated subjects to create a certain
number of matched sets.
c. Caution should be exercised as propensity scores ensure balance
only in observed covariates, where randomization ensures balance
between both observed and unobserved covariates.
Propensity score matching
Challenges with observational studies
Missing data are frequently a problem within observational research
and may be classified as missing completely at random, missing at
random, or missing not at random.
 Missing completely at random data are truly random and casedependent and are at decreased risk for introduction of bias
 Missing at random indicates data that are absent, and the absence is
related to other patient data (e.g., correlation with age), and
therefore may increase risk of bias.
Missing not at random indicates data that are absent, but the
absence is not related to one of the above
Challenges with observational studies
There are multiple ways to handle missing data (e.g., imputation),
but the method should be defined a priori given that significant
amounts of missing data may introduce bias.
 Confounding variables must be handled in a manner that can be
controlled during analysis (i.e., regression models)
Meta-analysis
Meta-analysis: a systematic approach to the identification and abstracting of
critical information from research reports.
Outcomes from a meta-analysis may include a more precise estimate of the
treatment effect or risk factor for disease than individual studies contributing
to the pooled analysis.
1. Provides examination of heterogeneity or variability in responses
2. May be applied even when the included studies are small, and
substantial variation exists in the issues studied, research methods, study
subjects, and other factors that may affect the overall findings.
Meta-analysis
3. Many meta-analyses combine results into a best estimate with
statistical confidence bounds meant to summarize what is known
about the clinical problem in question.
4. Pooled results may incorporate the biases of individual studies and
embody new sources of bias (e.g., publication bias).
Thanks for your
attention