Fundamentals of Biostatistics
Lecture 2
1. Clinical Trials
2. Validity/Reliability
3. Assessing Evidence
Randomized Clinical Trials (RCT)
 Randomization first used by RA Fisher in
agriculture expts in 1920s
 First clinical trial using randomization in 1931
by Anderson on use of sanocrysin on TB
patients. Also, first trial using blinding.
 Placebo first used in RCT in 1938 in cold
vaccine trial.
Randomized Clinical Trials (RCT)
Fundamental Point:
A properly planned clinical trial is a
powerful expt’l technique for assessing
effectiveness of a drug or intervention.
Defn: A clinical trial is a prospective study
comparing the effect of intervention(s)
against a control in humans.
Randomized Clinical Trials (RCT)
Phases of clinical trials:
Phase I: Determine tolerance of a compound in
humans (i.e., how large a dose can be given until
unacceptable toxicity?).
Phase II: Evaluation of biologic activity or effect,
and estimate rate of adverse events.
Phase III: Definitive comparative trial. Designed to
determine effectiveness and its role in clinical
practice.
Randomized Clinical Trials (RCT)
Terminology
Efficacy: How well an intervention works in an
ideal setting.
Effectiveness: How well an intervention works
in actual practice.
Phase IV: Evaluation of long-term saftey of an
intervention believed to be effective in phase III
trials.
Randomized Clinical Trials (RCT)
Why are clinical trials needed?
Because evaluating the effectiveness of a
treatment using uncontrolled observations
is very difficult, since other factors
affecting treatment outcome may not be
balanced in treatment groups.
Randomized Clinical Trials (RCT)
Advantages of RCT:
 Groups more comparable b/c
confounding variables are balanced
 Ability to detect small effects
 Most stat tests based on assumption of
random allocation of pts to trt groups
(validity)
Randomized Clinical Trials (RCT)
Disadvantages of RCT:
 Expensive and time consuming
 Subject pool may not be representative
 Effective treatment may be withheld
 Expose pts to dangerous drugs
Randomized Clinical Trials (RCT)
What is the question?
 Each clinical trial must have a primary question.
 The primary, as well as secondary questions, must
be carefully selected, clearly defined, clinically
relevant, and stated in advance.
 This includes the choice of response variable (ie.,
true clinical endpoint or surrogate endpoint).
Randomized Clinical Trials (RCT)
Study population
 The study population is a subset of the
population with the condition or characteristics
of interest defined by the eligibility criteria.
 This population should be defined in advance,
stating unambiguous inclusion (eligibility) criteria.
 The impact that the inclusion criteria will have on
study design, ability to generalize, and participant
recruitment must be considered.
Randomized Clinical Trials (RCT)
Randomized control studies
 comparative studies with an intervention
group and a control group
 the assignment of the participant to a
group is determined by the formal process
of randomization.
Randomized Clinical Trials (RCT)
Randomized control studies . . .
 Sound scientific investigation almost
always demands that a control group be
used against which the new intervention
can be compared.
 Randomization is the preferred way of
assigning participants to control and
intervention groups.
Randomized Clinical Trials (RCT)
Randomization
 Randomization tends to produce study
groups that are:
1. comparable with respect to known and
unknown risk factors
2. Removes investigator bias in the allocation
and treatment of patients
3. Guarantees statistical tests will have valid
significance levels.
Randomized Clinical Trials (RCT)
Randomization . . .
 Simple randomization is easiest to
understand and use, but randomization
can also be blocked, stratified, adaptive,
etc.
 Randomization is best accomplished by
an independent central statistical unit.
Randomized Clinical Trials (RCT)
Blinding (Masking)
 Ideally, a clinical trial should use a doubleblind design to avoid potential problems
with bias during data collection and
assessment.
 If using a double-blind design is not
feasible, a single-blind design and other
measures to reduce bias should be used.
Randomized Clinical Trials (RCT)
Baseline Assessment

Relevant baseline data should be measured in all
study participants before the start of intervention.

These baseline measurements can be used to
determine eligibility (if obtained prior to
assignment to treatment group).

Can be also used to determine if the randomization
produced identical groups (if not, can be used as
covariates for adjustment of imbalance).
Randomized Clinical Trials (RCT)
Data Analysis

Excluding randomized participants or observed
outcomes from analysis and sub-grouping on the basis
of outcome or response variables (including nonadherence) can lead to biased results of unknown
magnitude or direction.

Including all randomized participants in the analysis, in
the group they were assigned, is the intent-to-treat
principle.

The intent-to-treat analysis is viewed as the most valid
approach (least susceptible to bias).
Randomized Clinical Trials (RCT)
Intent-to-Treat Principle
Once randomized . . .
. . . Always analyzed!
Randomized Clinical Trials (RCT)
Covariate Adjustment
 While randomization eliminates bias, it does not
guarantee comparable baseline characteristics of
patients in different treatment groups in a
particular trial.
 Baseline balance is not a requirement for obtaining
valid variables.
Randomized Clinical Trials (RCT)
Covariate Adjustment . . .
 Imbalance will matter only if characteristic is
related to patient outcome, ie., it is prognostic.
 When randomization leads to chance baseline
imbalance, estimates of treatment effects will be
biased when using unadjusted analysis.
Validity and Reliability
Validity
Represents the degree to which a measurement
represents a true value.
Reliability
A measure of the reproducibility of a result or
observation. ie., How closely do repeated
measurements on the same object agree?
Validity and Reliability
 Errors can be caused by a lack of either
validity or reliability
 Validity and reliability are related. If a
measure is unreliable it is not capable of
producing valid results.
 Better reliability is necessary, but not
sufficient for validity.
Threats to Study Validity
Bias
• Selection
• Information
• Confounding
Regression to the Mean
Threats to Study Validity
Selection Bias
Distortion of effect estimate because of
(i) manner subjects selected: biased sampling,
(ii) selective losses: loss to follow-up and nonresponses.
(Case-control studies are particularly
susceptible)
Threats to Study Validity
Information Bias
Distortion of effect estimate when
measurement of exposure or disease is
systematically inaccurate.
(i) misclassification bias - incorrect classification
of exposure or disease,
(ii) recall bias: accuracy of self-reported data
varies
across comparison groups.
Threats to Study Validity
Other common types of bias:
(i) Investigator/Patient bias
(ii) Measurement error
Many forms of bias can be eliminated or reduced by
using randomization and blinding (preferably doubleblinding).
Threats to Study Validity
Confounding
Type of bias occurring when effect of
exposure mixed-up with one or more
extraneous variables.
• can create appearance of exposure-disease
relationship, when none exists.
• can hide true nature of exposure-disease
relationship.
Threats to Study Validity
Confounding . . .
• due to presence of key relationships
between extraneous variable(s) and both
exposure and disease.
• confounding variables are risk factors for the
disease, or a correlate of a causal factor.
• confounding variables are associated with
exposure of interest.
Threats to Study Validity
Confounding . . .
Addressing the Problem:
• At the design stage we can use:
– Randomization
– Matching
• At the analysis stage we can use:
– Stratified analysis (Mantel-Haenszel Methods)
– logistic regression
Threats to Study Validity
Regression to the Mean (RTM)
tendency of extreme observations by
chance to move closer to the mean when
repeated
What is the danger of ignoring RTM?
Causality may erroneously be inferred
Threats to Study Validity
Regression to the Mean (RTM) . . .
Examples:
• Patients having higher than average
cholesterol levels at initial screening, have
lower levels on repeat.
• Acute pain patients seek help when symptoms
severe, and any change is likely to be
improvement => useless treatment may
appear effective.
Threats to Study Validity
Regression to the Mean (RTM) . . .
RTM is a function of:
• Correlation between pre- and post-treatment
values.
• How extreme the pre-treatment values are.
Threats to Study Validity
Regression to the Mean (RTM) . . .
RTM implies that if we select subjects because
they appear abnormal on some test, AND
do nothing to them,
they will seem to improve when retested, thus
treatment effects become confounded with RTM.
Threats to Study Validity
Regression to the Mean (RTM) . . .
Four ways to minimize RTM:
1. Increasing reliability of screening test
2. Testing each subject twice, and requiring
all
tests be extreme before entry into
study.
3. Adjust for the correlation of pre/post
measures
Types of Reliability
1. Interobserver – agreement among
observers
(Kappa, Intraclass correlation)
2. Test-Retest – stability, does the same
measure give the same results repeatedly?
3. Parallel form – Two parallel test forms with
different items are correlated.
4. Split-half- Split individual measure into two
Ten Steps for
Evaluating Evidence
1.
2.
3.
4.
5.
6.
Be skeptical
Don’t rely on biological plausibility
Reliable info requires comparison
Ensure cause precedes the effect
Post-trial questions maybe unreliable
Pre-trial question should be specific and
clinically relevant
Ten Steps for
Evaluating Evidence . . .
7. Discovering small effects requires
randomization
8. Be wary of conflicts of interest
9. Non-specific exposure effects can be
important
10. Unblinded examiners may introduce bias.