CHP CHEAT SHEET
SUMMARY
Done by: Grace Lum
1
CONTENTS
S/N
2
3
4
5
6
7/8
9
CONTENTS
Descriptive studies
Case reports, case series, ecological
Analytical studies
Cross-sectional, cohort, case-control
Measures of disease frequency
Nominal, ordinal scale
Incidence, cumulative incidence
Prevalence, point prevalence, period prevalence
Incidence vs prevalence
Rate
Incidence density/incidence rate
Cumulative incidence vs incidence density
Describing variation in continuous data
Mean, median, mode
Long ordinal data
Range, quartiles
Variance, standard deviation
Understanding and using interventional studies
Uses
Basic principles – test and control groups, comparability, unbiased
measure
Treats to validity – loss to followup, non-compliance, contamination
Advantages
Disadvantages
Ethical issues
Phases of trials
Community trials
Measures of risk and association
2x2 contigency table
Relative risk (RR) / Risk ratio (for cohort study)
Rate ratio
Risk ratio vs rate ratio
Attributable risk (AR)
Relative risk vs attributable risk
Population attributable risk (PAR)
Population attributable risk perent (PAR%)
Characteristics of cohort/survival study
Kaplan-Meier estimate
Logrank test
Odds and odds ratio (for case-control study)
Pearson correlation coefficient (r)
Bias
Sources of bias
Classification of bias
Bias, by studies
PAGE
4
4
5
5
5
5
5
6
6
7
7
7
7
8
8
9
9
9
9
10
10
11
11
11
11
12
12
12
12
12
13
13
13
13
14
14
15
2
10/11 Assessing sampling errors
Sampling distribution of sample means; Standard Error of the Mean
Confidence interval
Hypothesis testing (p value)
Type I and II errors
Tests of significance
12
Confounders
Confounders – criteria, detection, measures against
Effect modifiers
Causality and criteria for establishing causality
13
Understanding and using diagnostic and screening tests
True positive, false positive, true negative, false negative
Sensitivity, specificity, positive predictive value, negative predictive value
Sensitive tests vs Specific tests
Predictive value
Screening tests vs Diagnostic tests
Guidelines for screening programmes
Outcomes of screening programmes
14
Understanding and using meta-analyses
Methods of summarising research findings
Meta-analyses
Forest plots
Heterogeneity
Publication bias
Pooled analysis
Limitations of meta-analysis
Tutorial learning points
Critical evaluation of study methodology
Clinical evaluation of clinical trial
Critical evaluation of diagnostic/screening test
Clinical decision making (including meta-analysis)
Guidelines on evaluation an overview/ systematic review/ meta-analysis
16
16
17
17
17
18
19
19
20
20
20
20
21
21
21
22
22
22
23
23
24
24
25
25
26
26
26
3
2 - DESCRIPTIVE STUDIES
Study
Details
Case reports
1 or 2 patients; describe
rare/unusual clinical events
Advantages
Highly detailed, can use
sophisticated methods
Disadvantages Cannot estimate frequency;
there is bias and chance
Case series
10-100 patients
Ecological
Study unit is population
Absence of
comparison
group
Cheap; can generate
hypotheses
Ecologic fallacy: factor and
disease may not be related at
individual level
3 - ANALYTICAL STUDIES
Analytical studies are used to evaluate (casual) relationships between risk factor and disease.
Study
Details
Advantag
es
Disadvan
tages
Cross-sectional
 At specific point in
time
 Risk factor and
disease measured
concurrently
 Simple, rapid,
inexpensive
 Can study wide
range of factors
 Not good for rare
conditions
 Cannot establish
cause-effect
relationship
 Length bias (study
of survivors) –
more likely to
catch people who
survive
Cohort
 Population no outcome;
ask exposure
 Follow-up and compare
outcome between exposed
and unexposed
 Least susceptible to bias
 Highest level of evidence
for association among
observational studies
 Can describe natural
history of disease
 Calculate cumulative
incidence/incidence rate
 Direct measure of risk
 Can tell cause-effect
(temporal sequence)
 Not good for rare
conditions
 Not feasible when long
latent period between
exposure and disease
Case-control
 Start with cases and
controls from same
population base
 Capture new cases in
study period
 Quick, less resourceintensive
 Can be used for rare
diseases
 Can be used for diseases
with long latent period



Prone to bias (recall bias,
selection bias)
Cannot establish causeeffect relationship
Does not measure risk
directly, but calculates
odds. The odds ratio is an
approximation of the
relative risk in exposed vs
non-exposed.
4
4 – MEASURES OF DISEASE FREQUENCY
Categorical variables


NOMINAL SCALE
o Mutually exclusive with no logical order
o Eg. Gender, race
ORDINAL SCALE
o Mutually exclusive with logical order
o Difference between ranks is not equal (subjective)
o Eg. Cancer staging, anxiety score
Measurement (continuous) variables




Ratio – divide one quantity by another; no units
Proportion – a ratio in which numerator is included in the denominator; no units
Incidence – frequency of new occurrences of a disease, injury or death during period of
study.
Cumulative incidence – frequency of new cases over a specific period of time.
o
𝐼
𝑁
𝐶𝐼 =
o
o
o


I = # of new cases during follow-up
N = # of disease-free subjects at start of follow-up
Most common (direct) way to estimate risk – probability an individual will
experience a health status change over a specific follow-up period.
o Disadvantages
 Does not reflect dynamic changing population
 Does not allow subjects to be followed for different time periods
Prevalence – frequency of disease in a population at a point in time.
o Cross-sectional study; no follow up
Point prevalence – frequency of disease at given point in time.
o


𝐶
𝑁
o C = # of observed cases at time t
o N = Population size at time t
Period prevalence
o

𝑃=
𝑃𝑃 =
𝐶+𝐼
𝑁
o C = # of prevalent cases at beginning of time period
o I = # of incident cases that develop during that period
o N = size of population for same time period
Incidence vs prevalence
o P = I x D (duration of disease)
o If incidence ↓, prevalence may ↑ due to treatment prolonging life.
Rate – how rapidly health events are occurring in a population. Needs time (person-years,
etc)
5

Incidence Density/ Incidence Rate – individuals in the population at risk studied for varying
lengths of time
o Incidence density rate = Number who get disease at specific period (I)/ Summation
of length each person is at risk (PT)
o I = # of new cases during follow-up
o PT = total time that disease-free individuals in the cohort are observed over the
study period
Cumulative incidence
Acute diseases with restricted risk period
Fixed populations
Direct measure of risk
Incidence density
Chronic disease with extended risk period
Dynamic populations
Can make comparisons between populations
with different observation time
Not a direct measure of risk
6
5 – DESCRIBING VARIATION IN CONTINUOUS DATA

Measurement (continuous) variables
o Infinite number of values; equally spaced
o Eg. height, weight, temperature, BP
o Ratio scale – 0 has no meaning (eg. height)
o Interval sale – 0 is important (eg. Celsius)
Measures of central
tendency
Mean
Definition
Advantages
Disadvantages
Average
Widely used
Median
Exactly half
Insensitive to outliers
Mode
Most frequent
-
Overly sensitive to
outliers
Less sensitive to actual
numerical values
Rarely used

Long ordinal data
o Ordinal data graded on long scale, may be treated as continuous data
o Eg. pain on scale of 1-10
o But, not truly continuous data
 Finite number of values
 Gaps in continuum
 Spacing between categories is not numerically equivalent
Measures of spread of
measurement data
Range
Quartiles
Variance
𝑠2 =
∑(𝑥 − 𝑥̅ )2
𝑛−1
Standard Deviation (s)


Details
Advantages
Disadvantages
Difference between highest and
lowest observed values
Interquartile range – middle 50% of
data set; eliminate outliers
Combines all values in data set to
produce measure of spread
Easy to
compute
Less sensitive
to outliers
Greatly influenced
by outliers
Difficult to
calculate
Square root of variance. Greater
spread, greater SD.
In a normal distribution,
o 68% of scores are within 1 S.D. of mean
o 95% of scores are within 2 S.D. of mean
 Eg. 95% of all the 1000 babies lie within 3.25 +/- (2xSD) kg
 Describes group you measured. Cannot be used for entire population – use
standard error.
o 99% of scores are within 3 S.D. of mean
Skewness
o Positive – scores cluster toward lower end of scale
o Negative – scores cluster toward upper end of scale
7
6 – UNDERSTANDING AND USING INTERVENTIONAL STUDIES
Uses of interventional studies
o
o
Evaluate efficacy of
 Therapeutic agents/ surgical approaches
 Prophylactic agents
 Preventive measures
Test aetiologic hypothesis and provide conclusive evidence of a cause-effect relationship
Gold standard

Randomised, double-blind, controlled clinical trial
Basic principles



Test and control groups
o Appropriate selection – applicable to real life. Affects generalisability of results to all
patients commonly seen in clinical practice.
 Type of cases (severity, pathology)
 Contraindications
 Ability and willingness to comply
o Control group
 To ensure results not due to
 Natural history of disease
 Biological variation – ‘regression to the mean’
 Effect of being studies – ‘hawthorne effect’
o Compared with respect to outcome
Baseline comparability
o Through randomisation
 Remove selection bias
 Equalises baseline differences – reduce confounders (maximises likelihood
that both groups will be similar in characteristics that may influence
outcome to treatment)
Unbiased measured of outcomes
o More than one end-point can be used. Must be scientifically valid, clinically relevant.
o Determine side-effects
o Masking (blinding); placebo
 Double-blind is gold standard
 Patient – prevent placebo effect (expectation causes change in
status)
 Doctor – prevent selection bias, prevent data gathering bias
 Single blind – subject unaware
 Double blind – subject and observer unaware
 Triple blind – data analyst also unaware
o Difference observed due to
8




Inherent differences in relevant characteristics of groups
Chance variation
Difference in way groups were managed, or differences in compliance
True difference in effects of treatment
Threats to validity


Loss to followup
o Drop out – affect validity of study and sample size
 If high dropout rate, problem! People who dropout may be different from
people who stay in - bias
 Minimise: pre-test for compliance, provide incentives, frequent contact,
measuring adherence via pill counts etc, not stretching trial for longer than
necessary
Non-compliance and contamination
o Omit medication
o Take medicine not intended for them
o Intention to treat analysis
 In real life, will the treatment work?
 Considering that people may drop out or drop in in real life as well
 So, compare according to group randomised to, whether they take or not
Advantages



Known and unknown prognostic factors balanced between groups
Blinding can reduce bias
Effect of dosage can be examined
Disadvantages





Expensive
Selection criteria may limit generalisability
Long follow-up period may be required
Compliance not assured
Ethical issues
Ethical issues



Each of treatment options should be equally acceptable under current knowledge
o Sufficient evidence and doubt of treatment’s efficacy
o No treatment options should be known to be inferior to another based on previous
randomised studies
No patient should be denied appropriate treatment as a result of participation, or refusal to
participate
Risk-benefit ratio must be favourable
o Potential benefits to subjects and to society justify the risks
o Continuous monitoring; stop study if need be
9

Informed consent
o Informed they are part of the study, made aware of treatment options, risks and
benefits, nature of randomisation, and that they will be selected to receive any of
the options.
Phases




Phase I = safety and dosimetry
Phase II = preliminary information on efficacy
Phase III = defininte evidence of efficacy
Post marketing surveillance = detect uncommon side effects
Community trials



Entire community serves as experiment/control unit – eg. 3 city study
For diseases influenced by wide range of factors like lifestyle, social behaviour, environment
Limitations
o Expensive
o Small sample size
o Possible contamination
o Communities may not be comparable
10
7/8 – MEASURES OF RISK AND ASSOCIATION
We are most interested in associations that are not necessary and not sufficient, but nevertheless
associated in an average person.
Table showing association between risk and disease occurrence
Risk factor
present
Risk factor absent
Total
Disease
present
a (?%)
Disease
absent
b (?%)
Total
a+b (?%)
c (?%)
a+c (?%)
d (?%)
b+d (?%)
c+d (?%)
a+b+c+d (100%)
RELATIVE RISK / RISK RATIO (For Cohort Study)


Cumulative incidence of disease among those with risk factor = a/(a+b)
o This is the risk of developing disease if risk factor present
Cumulative incidence of disease among those without risk factor = c/(c+d)
o This is the risk of developing disease if risk factor absent
𝑎/(𝑎+𝑏)

Relative risk = risk ratio = 𝑐/(𝑐+𝑑)

Eg. Smokers are 2.1 times as likely to die from heart disease than non-smokers
RATE RATIO




How fast a disease occurs – faster it occurs, higher the risk. Indirect way to measure relative
risk.
Exposed
Unexposed
Total
New cases
I1
I0
I
Person-time PT1
PT0
PT
Average rate = I/PT
o I = number of new cases
o PT = person-time over follow-up
Rate ratio = (I1/PT1) / (I0/PT0)
Eg. Mortality rate among hypertensive is 3.5 times the mortality rate among normotensives
RISK RATIO VS RATE RATIO




Computes relative risk
Cumulative incidence ratio = CIexposed/CIunexposed (RISK RATIO)
Incidence density ratio = IDexposed/IDunexposed (RATE RATIO)
Relative risk = ratio of risk/rate for one group to risk/rate of another group
11
ATTRIBUTABLE RISK / EXCESS RISK






Risk difference
o Risk in exposed – risk in non-exposed = a/(a+b) – c/(c+d)
Measures excess risk on absolute scale
Tells how much reduction in disease there will be if exposure is removed.
Qn: How many cases of disease can you attribute to the exposure?
Gives idea of health impact on society if you get rid of exposure - depends on how common
the disease is
Eg. If AR = 0.004, for 1000 smokers, 4 more oral cancers than 1000 non-smokers
RELATIVE RISK VS ATTRIBUTABLE RISK


Relative risk: measures strength and direction of an association
Attributable risk: reflects potential public health consequences of an exposure
POPULATION ATTRIBUTABLE RISK (PAR)






Risk (total) – Risk (unexposed) = (a+c)/(a+b+c+d) – c/(c+d)
Also equals AR x PF (prevalence of exposure in population [ (a+b)/(a+b+c+d) ])
Tells how much reduction in disease there will be if exposure is removed, in the population
Takes into account proportion of risk factor in population
Qn: How many cases of disease can you attribute to the exposure, in the population?
Eg. If smoking is eliminated, the mortality of heart disease decreases by 9 per 100 patients
POPULATION ATTRIBUTABLE RISK PERCENT (PAR%)




Qn: What % of total risk of disease is due to the risk factor?
PAR% = [ Risk (total) – Risk (unexposed) ] / Risk (total) x 100%
Also, PAR% = (PF)(RR-1) / [1 + (PF)(RR-1)] x 100%
o Where PF = prevalence of exposure factor
Eg. If smoking is eliminated, 35% of deaths in population of patients with heart disease can
be prevented.
COHORT/ SURVIVAL STUDY

Characteristics
o Well-defined time of entry
o Well-defined endpoint
o Time in between = survival time
12
o

Censored observation = no longer observable. (eg. still alive at study closure, death
from other causes, or loss to follow up.) Time = censored survival time
Kaplan-Meier estimate / Product-limit estimate
o dt = number of events (death) at time t
o nt = number of people at risk at time t (remove deaths and censored observations)
Probability of survival at that point of time t = Pt = 1 −
o
Survival function S(t) is the probability of surviving beyond time t


𝑑𝑡
𝑛𝑡
o
S(t) = (1 −
𝑑1
)
𝑛1
(1 −
𝑑2
)...
𝑛2
(1 −
𝑑𝑡
)
𝑛𝑡
 Values should be taken each time an event occur (someone dies)
Logrank test
o Test of clinical significance
o Compare survival distributions between groups of clinical interest
o Survival distributions can be displayed using a Kaplan-Meier survival plot
o Takes into account information provided by censored observations
ODDS (For Case-Control Study)





Odds = proportion/ 1-proportion = risk of having something / risk of not having something
For case-control, must use odds as cannot find incidence, cannot compute risk or risk ratio
Formula
o Odds of lung cancer given smoking = (a/a+b)/(b/a+b) = a/b
o Odds of lung cancer given non-smoking = (c/c+d)/(d/c+d) = c/d
o Odds ratio = (a/b)/(c/d) = ad/bc
Odds ratio from case-control study of rare disease is estimate of relative risk of the disease
How to write:
o Lung cancer patients have 4 times the odds of being smokers than those without
lung cancer. The odds of lung cancer in smokers is 4 times the odds in non-smokers.
o Fulfills the criteria
 The cases are incident cases (need to include all new cases over the entire
time period; prevalent cases = bias)
 The controls come from same source in population that cases come from.
 The disease is rare (low incident rate).
o Therefore, odds ratio is a good estimate of the relative risk.
o Thus, the risk of lung cancer in smokers is 4 times the risk in non-smokers.
PEARSON CORRELATION COEFFICIENT




For linear relationship only. r varies from -1 to 1.
o >0.8 very good; 0.5-0.8 good; 0.25-0.5 bad; <0.25 very bad
Must eyeball data on scatter plot  could be sigmoid or otherwise.
If skewed measurement data (not normal) or ordinal data (eg. Cancer staging), use
Spearman rank correlation coefficient
r2 is the proportion of variability in y explained by the variability in x.
o The higher the correlation, the greater changes in x can explain changes in y.
o Eg. r2=0.25. 25% of variability in y is explained by the variability in x.
13
9 – Bias


Any systematic error in design, conduct, or analysis of study that results in distorted
estimate of exposure’s effect on risk of disease
Any association can be due to
o Casual relationship (true result)
o Confounders
o Chance (random error)
o Bias (systematic error)
Sources of bias



Before study – literature review, study design
During study – selection of participants, data collection
After study – analysis, publication
Classification of bias
Bias
Details
Found in
Solutions
INFORMATION BIAS – Method for collecting information introduces error
Exposure
Recall bias
Affected persons respond
CaseUse cohort study
identificat (participants)
differently about prior
control
ion bias
exposures compared to
study
unaffected persons.
 Affected over-recall
 Unaffected under-recall
Interviewer Interviewers probe for exposure CaseMask interviewer;
bias
in cases
control
Train interviewers and tape
(interviewers)
study
interviews;
Standardise interviews
Disease
Surveillanc Some exposures will have more
Identical procedures for
identificat e bias
medical surveillance, thus more
identification of diseases
ion bias
likely to have subclinical disease
(standardise doctor visits)
diagnosed.
Observer
Observer who knows exposure
Cohort
bias
status may be biased when
study
assigning disease outcome (for
borderline cases)
SELECTION BIAS – Biased estimation of cases or controls (case-control studies), or of exposed or
unexposed (cohort study). Method of participant selection distorts exposure-outcome r/ship.
Selection bias
Case
Cases are controls should
 Cases may not
be from comparable
represent all individuals control
populations; cases should
with disease of interest
represent all individuals
in community
with disease; controls
 Controls may not
should represent all
represent all individuals
individuals without disease
without disease of
interest in community
 Selection of cases and
controls affect results
14
Admission bias
Non-response bias
OTHER BIASES
Temporal bias
Analytic bias
Publication bias
Admission rates of exposed and
unexposed cases and controls
differ – eg. more obese, more
likely to be admitted
People who respond are
different from people who don’t
respond – reflects attitude
Hospital
based
case
control
Population-based casecontrol studies;
Don’t release hypothesis to
emergency doctor
Do not know if exposure or
outcome occurred first
Crosssectional
study
Cohort study
Only articles with positive
results published
Bias, by studies



Case-control studies
o Recall bias
o Interviewer bias
o Selection bias
Cohort studies
o Observer bias
o Lost to follow up bias – individuals lost to follow-up different from those who are
not
o Non-response bias???
o Problems in measuring exposure
Clinical trials (interventional studies)
o Disease identification bias – observer bias
o Lost to follow up bias – those lost may have different exposure-disease relationship
compared to those who remain
o Selection bias – if subjects are not randomised
o Non-compliance bias – stop taking or take agent assigned to other group
o Observer bias
15
10/11 – ASSESSING SAMPLING (RANDOM) ERRORS



Truth = observed – errors
Systematic errors – have pattern. Can do something to remove it if you can identify pattern.
Random (sampling errors) – no pattern
Indices of sampling errors


Confidence interval (Null:1 for ratio; 0 for difference)
o Preferred approach
o More informative
o Easy to understand
Hypothesis testing
o Less informative
o Many variations/test
o Obsession with p<0.05 = statistically significant
Sampling distribution of sample means and standard error











If systemic error minimal, difference between pop. mean and sample mean = sampling error.
Unfortunately, in real life we may not know population mean.
Thus, do sampling distribution of sample means – if you do alot of sampling, you’ll get
population mean most of the time (normal distribution)
Mean of the sampling distribution of sample means is population mean (μ).
Standard deviation = measure of degree of sampling variation
Standard deviation of sampling distribution of sample means = standard error of the mean
(SEM)
𝑆𝐸𝑀 ≅
𝑆𝐷
√𝑛
o To reduce sampling error, make n (sampling size) as large as possible
68% of all observations lie between μ +/- 1 SEM
95% of all observations lie between μ +/- 2 SEM
o Ie. There is a 95% chance that the sample mean is within μ +/- 2 SEM
o Eg. if SEM=0.5, sample mean = 3.23, we are 95% confident that μ is between 2.23
and 4.23 (95% confidence interval)
99% of all observations lie between μ +/- 3 SEM
SD vs SEM
o 95% variation of sample = mean +/- 2 SD
o 95% confidence interval of mean of whole universe = mean +/- 2 SEM
Confidence interval




Eg. RR (95% CI) = 0.75 (0.64-0.87)
I am 95% confident that the true population mean lies between 0.64-0.87
At best, there is a 36% reduction in risk
At worst, there is a 13% reduction in risk  reduction in risk still there!
16
Hypothesis testing (p value)

p<0.05 – there is less than 5% chance that the observed difference is due to sampling error.
o But, statistically significant does not mean clinically significant and vice versa
o Confidence intervals are more informative
o Statistical significant difference means that difference is unlikely to be due to
random (sampling) error, but there could still be systematic error.
Type I and II errors


Type I (α) error  say there is an association, when there is actually none. (False positive)
o Reduce by ↓ p value.
Type II (β) error  say there is no association, when there is actually one. (False negative)
o Reduce by increasing sample size.
Tests of significance

Comparing means / measurement data that is normally distributed
o Comparing 2 unmatched means
Unpaired t test
o Comparing 2 matched means
Paired t test
o Comparing 3 or more means
One-way analysis of variance

Comparing medians / ordinal data, or measurement data that is not normally distributed
o Comparing 2 unmatched medians
Mann-Whitney U test
o Comparing 2 matched medians
Wilcoxon’s Sign Rank test
o Comparing 3 or more medians
Kruscal Wallis test

Comparing proportions
o Comparing 2 unmatched proportions
o Comparing 2 matched proportions
o Comparing 3 or more proportions
Fisher’s χ2 test
McNemar’s χ2 test
χ2 test (Pearson’s)
*p value lower for measurement data (parametric tests), higher for ordinal data (non-parametric)
-- parametric tests compare actual values, non-parametric tests compare ranks (↓ differences)
*p value lower for paired test (only considers variance of difference), higher for unpaired test (more
variability due to intra-group variance)
*p value based on sample size (small size, p value ↑) and variance (small variance, p value ↓)
*Matched – the 2 sets of data are matched (eg. same people before and after exposure)
Comparing more than 2 means – probably unmatched



Do not perform multiple t-tests
o Multiple tests each have 5% chance of sampling error  3 tests ↑ p value to 15%
Perform post-hoc tests
o Eg. bonferroni method – 0.017 per test  p value overall still 5%
o Other methods: Duncan, Tukey, Scheffe
Genomic data: 10-7
17
12 – Confounders
Criteria for confounder (X)



X is a risk factor for disease
X is associated with exposure
X must not be in intermediate step in casual path between exposure and disease
How to detect confounders




Collect data on all possible confounders
Evaluate association between possible risk factors (smoking) and confounders (age)
o Eg. Is age related to smoking?
 Smokers are older as 50% of subjects above age 20 are smokers but only
10% of subjects less than age 20 are smokers
Evaluate association between confounders (age) and disease (bladder cancer)
o Eg. is age related to disease status?
 Older people have higher risk of bladder cancer as only 38% of individuals
less than 20 years have bladder cancer, but 71% of individuals above 20
years have bladder cancer
Stratify data by possible confounder (age)
o For participants age <20 years, calculate odds ratio
o For participants age >20 years, calculate odds ratio
o Get Mantel-Haenszel ‘Adjusted Odds Ratio’.
 If this = 1.0
 Thus, after stratification by the confounder (age), there is no relationship
between smoking and bladder cancer for individuals within the age strata
<20 years and for individuals within the age strata >20 years. Both stratified
odds ratios are different from crude odds ratio, thus spurious relationship
was explained entirely because of confounding by age.
 In addition, the Mantel-Haenszel ‘adjusted’ OR is different from the crude
OR. Bladder cancer cases were more often smokers not because they were
cases, but because they were older.
Measures to control confounding


Prior to data collection
o Restriction – involve only those with certain characteristics (strata) (eg. old persons)
o (case-control study) Matching – cases and controls matched by potential
confounder, so that age distribution of cases and controls similar
o (clinical trial) Randomisation – 2 groups have high chance of being comparable for
confounders both known and unknown.
After data collection
o Stratification – evaluate association between exposure and disease for different
strata of potential confounder
18
o
o
Direct and indirect standardisation – adjust for confounding in 2 or more
populations by use of standard population or standard rates
Multivariate analysis – sophisticated statistical techniques such as multiple linear
regression and logistic regression analysis – allow control of several confounders
simultaneously
Effect modifiers




Relationship between exposure and outcome is modified by a third variable
We do not seek to control or eliminate effect modification
Occurs when difference in risk/rate between those with and without the risk factor is not
homogenous in strata formed by the effect modifier
o When stratified by the effect modifier, odds ratios are different
Confounder vs effect modifiers
o Both involve 3rd factor
o Both may be evaluated using stratified analysis
o Confounding: stratification removes confounders. Compare crude and stratified
ORs.
o Effect modification: stratification does not change anything. Compare stratified ORs.
o Can be confounder alone, effect modifier alone, or both
Causality



Approaches to studying etiology of disease
o Animal models
o In vitro systems
o Epidemiologic studies in human populations
Henle-Koch’s Postulates (criteria for establishing casual relationships between organisms
and disease)
o Organism always found with disease
o Organism not found with other diseases
o The organism, isolated and cultured through several generations, produces disease
in experimental animals
Criteria for establishing casualty (Hill’s criteria)
o Temporal relationship: exposure precedes outcome in time
o Biologic plausibility: consistency with existing knowledge from animal studies, etc
o Consistency: findings comparable with other studies in different
populations/circumstances
o Strength of association: strong association (high odds ratio) has less bias
o Dose-response relationship: as exposure increases, risk of disease increases
o Specificity of association: single exposure leads to single outcome; a lack does not
rule out causality
o Effects of removing exposure: removal of exposure reduces risk of disease
19
13 – UNDERSTANDING AND USING DIAGNOSTIC AND SCREENING TESTS
Test result
Disease (defined using the gold standard)
Present
Absent
Positive
True positive (a)
False positive (b)
Negative
False negative (c)
True negative (d)
Diagnostic tests are simpler, less expensive and less invasive than the gold standard test.
Characteristics
Sensitivity
Specificity
Positive predictive
value
Negative predictive
value
Definition
Proportion of disease correctly classified
(“positivity in disease”)
Proportion of non-diseased correctly
classified (“negativity in health”)
Likelihood of disease in person with a
positive test
Likelihood of no disease in person with a
negative test
Formula
𝑎
(𝑎 + 𝑐)
𝑑
(𝑏 + 𝑑)
𝑎
(𝑎 + 𝑏) ∗
𝑑
(𝑐 + 𝑑) ∗
Sensitivity vs Specificity



How accurate is the test?
Trade-off between the two
Eg. cut-off point to designate ‘abnormal’ result (eg. blood sugar levels for DM)  choice of
cut-off determines if test is very sensitive (low cut-off point) or very specific (high cut-off
point)
More SENSITIVE test
Low false negative
SNout  rule out condition (excluding)
Use if benefits of detection/treatment is great
Important for screening test (when cost of false
negative is high)
More SPECIFIC test
Low false positive
SPin  rule in condition (confirming)
Use if risks of further treatment is great
Important for diagnostic test (when cost of
treatment is high eg. chemotherapy)
Predictive value



Predictive value = what are the implications of results for my patient?
Predictive value of test varies according to prevalence of disease
Eg. Hospital has higher positive predictive value than Community
20
Screening tests vs Diagnostic tests
Characteristic
Purpose
Costs associated with false
negative result
Costs associated with false
positive result
Sensitivity vs specificity
Screening test
Classify persons into those
likely or unlikely to have the
disease
False reassurance
Diagnosis postponed, higher
morbidity
Labelling/anxiety
Risks associated with treatment
Sensitive test
Diagnostic test
Confirm a diagnosis
Requires further testing
Labelling/ anxiety
Risks associated with treatment
Specific test
Guidelines governing introduction of screening programmes (from tutorial)






Natural history of disease
o Is there a latent stage? Where asymptomatic, but can detect before critical point
o Critical point = point in natural history of disease before which therapy may be more
effective
Importance of disease
o Large numbers; or increase in disease incidence
o Serious morbidity or mortality
o Great burden of suffering
Treatment
o Available and acceptable treatment in early stages of disease to slow or stop it.
Screening test
o Acceptability
o Safety
o Sensitivity and specificity
Cost-benefit ratio
o Cost of test
o Cost of follow up process if false-positive results found
Need for repeated screening
o Affected by length of latent phase (shorter length – screen more often)
Outcomes of screening programmes (from tutorial)





Reduce mortality
Reduce case-fatality rate
Reduce complications of disease
Reduce recurrences or metastases
Improve quality of life
21
14 – UNDERSTANDING AND USING META-ANALYSES
Methods of summarising research findings


Non quantitative
o Narrative review – consensus statements, expert reviews
 Problems: subjective, no formal rules – can have selective inclusion of
studies
o Systematic review – rules for evaluating quality of article
Quantitative
o Vote counting
 Problems: ignores sample size, research design, effect size
o Meta-analysis (‘analysis of analyses’)
o Pooled analysis – individual data pooled together
Meta-analyses




Benefit – structured
Components
o Qualitative – assess quality using predetermined criteria
o Quantitative – statistical integration of results
o May be integrated – overall results weighted by quality score
Steps
o Define clinical/research question
o Identify relevant articles
 Specify inclusion criteria – randomised, controlled trial, etc
o Evaluate quality of each study according to methodology
 Criteria for appraisal of study quality – method and adequacy of
randomisation, blinding protocols, completeness of follow-up, cointerventions, etc.
o Summarise results quantitatively
 Overall measure of effect
 May analyse by subgroups if relevant
o Discuss possible biases, and implications of results (self-critique)
Understanding the findings
o Overall effect – forest plots and summary estimates
o Are results from various studies too different to be combined? - Heterogeneity
o Is there significant publication bias? – Funnel plots
Forest plots



Provide information on results from each study, and the summary/ combined effect
Risk ratio and confidence limits for each study plotted on the same scale.
Area of each black square reflects weight of study, usually proportionate to size of the
sample
22


Log scale used so that risk ratios of the same magnitude equidistant from 1, and confidence
intervals symmetrical about point estimate
Last one combines all into a single RR and CI.
Heterogeneity




Variations in results across trials, beyond the effect of chance
Due to
o Clinical diversity (participants, intervention, outcomes differ)
o Methodological differences (design or conduct of study)
o Statistical heterogeneity (excessive or unexpected variation in results)
Tests for heterogeneity
o Are the individual study results likely to reflect a single underlying effect, or different
effects?
o Many different statistical tests
o Most tests give p value.
 If p > 0.05, differences assumed to be consequence of sampling chance
o Some tests quantify level of heterogeneity across studies
 Eg. 25% of variation is due to heterogeneity rather than due to chance
Dealing with heterogeneity
o Fixed effects model
 Assume variability is exclusively due to random variation
 Each study weighted by sample size
 Used when not enough heterogeneity to be a concern
o Random effects model
 Assume different underlying effect to reflect the uncertainty
 Gives wider confidence intervals
 Most appropriate if there is substantial heterogeneity between studies
Publication bias


Problem with identifying relevant articles
o Grey literature – unpublished or awaiting publication
o Fugitive literature – published but difficult to locate eg. conference proceedings
o File-drawer manuscripts – unpublished as expected to be rejected
o Foreign language studies
Use funnel plots
o X axis – increasing treatment effect
o Y axis – sample size – a measure of precision
o Towards the top (high sample size), results should be less varied
o Empty funnel – no publications at 0 treatment effect as these are less likely to be
published
o Asymmetrical plots
 Could be that the truth IS this way – not a proof of bias
 Not very helpful if studies are clinically or methodologically diverse, or if
there are few studies.
23
Pooled analysis







Aggregates original data from several studies
More precise
Measurements of exposures and outcomes standardised; adjustment for confounders done
consistently
Can be very informative when meaningfully stratified eg. compare ethnic groups
Improved statistical power
Does not overcome biases – garbage in garbage out
Limited by availability of data from primary authors
Limitations of meta-analysis





Publication bias (non-reporting of negative or non-significant study results)
Temptation to integrate all studies regardless of quality
Cannot change original study (warts and all)
Other key findings might be neglected
Methodological issues still unresolved eg. how much heterogeneity is allowed
24
TUTORIAL LEARNING POINTS
Critical evaluation of study methodology







If confidence interval includes 1  not statistically significant
Multivariate RR = adjusted for smoking/other factors
Trend = dose-dependent response
Recruitment of subjects
o Cohort study  start with group of people without disease
o Homogenous population – lifestyle, etc same. Hopefully only 1 exposure different.
Need not represent whole world, but those who drink and those who don’t should
be otherwise homogenous – to minimise confounders.
 Results can be generalised to all population, unless some nurse factor
interacts with coffee (effect modifier)
Baseline characteristics table
o Shows you whether there are confounders – where exposed and unexposed differ
Interpreting results
o Eg. With reference to people who don’t drink coffee, those who drink 1/day have a
0.60 risk of pancreatic cancer. 95% confidence interval is 0.38-0.94, which does not
include 1, thus it is statistically significant.
o Eg. Trend test  downward trend could be by chance as p-value = 0.35 > 0.05 – not
statistically significant.
o Most RR not significant; no dose-dependent relationship  coffee drinking is not a
risk factor for pancreatic cancer
nd
2 case; interpreting results
o Eg. In males, using 0 cups as reference group, those who drink <2, 2-3, and 4+ cups
have RR of pancreatic cancer of 1.07, 1.45, 2.08 respectively. However, none are
statistically significant because 95% confidence interval includes 1.
Clinical evaluation of clinical trial


Stratified randomisation
o Stratify, then perform randomisation within each strata
o Not usually performed as simple randomisation can already mimimise confounders
o Use when some confounder is extremely striking
Treatment effect
𝐶𝑜𝑛𝑡𝑟𝑜𝑙 𝑒𝑣𝑒𝑛𝑡 𝑟𝑎𝑡𝑒−𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝑒𝑣𝑒𝑛𝑡 𝑟𝑎𝑡𝑒
𝐶𝑜𝑛𝑡𝑟𝑜𝑙 𝑒𝑣𝑒𝑛𝑡 𝑟𝑎𝑡𝑒
o
𝑅𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑟𝑖𝑠𝑘 𝑟𝑒𝑑𝑢𝑐𝑡𝑖𝑜𝑛 (𝑅𝑅𝑅) =
o
 This is more effective number.
𝐴𝑏𝑠𝑜𝑙𝑢𝑡𝑒 𝑟𝑖𝑠𝑘 𝑟𝑒𝑑𝑢𝑐𝑡𝑖𝑜𝑛 (𝐴𝑅𝑅) = 𝐶𝑜𝑛𝑡𝑟𝑜𝑙 𝑒𝑣𝑒𝑛𝑡 𝑟𝑎𝑡𝑒 − 𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝑒𝑣𝑒𝑛𝑡 𝑟𝑎𝑡𝑒
 This is an arbitrary number; difficult to interpret
o
𝑁𝑢𝑚𝑏𝑒𝑟 𝑛𝑒𝑒𝑑𝑒𝑑 𝑡𝑜 𝑡𝑟𝑒𝑎𝑡 (𝑁𝑇𝑇) = 𝐴𝑅𝑅
1


Number of patients you need to treat before one gets a fully favourable
result
Precision of estimate of treatment effect
o Measured by 95% confidence interval
25

o More narrow; more precise
Will results help in caring for my patients?
o Similar to patients you see?
o Consider side effects, recurrence of symptoms
o Cost-benefit analysis
Critical evaluation of diagnostic/screening test

Why does cardiologist have higher positive predictive value than school doctor?
o Higher sensitivity and specificity
o Prevalence of disease higher for cardiologist than school doctor
o Thus, not good indicator; but most direct advice you can give to patients when
interpreting their results
o Sensitivity and specificity are more stable indices
Clinical decision making (including meta-analysis)




Evidence based medicine  practice
Effectiveness in changing over-prescription of antibiotics
o Depends on how well doctor can communicate his point to change:
 I – ideas
 C – concerns
 E – expectations; of the patients
o Follow clinical practice guidelines  no grounds for patient to sue you
Type of healthcare system can affect management of patient’s disease
Communication
o Diagnosis/ disease + prevention
o Prognosis – allay fears or prepare for the worst
o Complications – treatment, occupation etc. Pre-empt patients.
Guidelines on evaluation an overview/ systematic review/ meta-analysis



Are the results of the study valid?
o Primary guides
 Did the overview address a focussed clinical question?
 Criteria for inclusion of articles appropriate?
o Secondary guides
 Were important, relevant studies missed?
 Was validity of included studies appraised?
 Were assessments of studies reproducible?
 Were results similar from study to study?
What are the results?
o What are the overall results?
o How precise were the results?
Will the results help my patients?
o Can be applied to my patient care?
o Were all clinically important outcomes considered?
o Are the benefits worth the harms and costs?
26