How to appraise
Medical Journal
(Therapy)
นพ.บริบรู ณ ์ เชนธนากิจ
หน่วยเวชศาสตรฉุ
์ กเฉิน
ภาควิชาเวชศาสตรครอบครั
ว
์
คณะแพทยศาสตรมหาวิ
ทยาลัยเชียงใหม่
์
What is EBM?
Patient
Concerns
EBM
Best research
evidence
Clinical
Expertise
Critical appraisal
Are the result valid?
What are the results?
How can I apply the result to patient care?
Are the result valid?
ประเด็นทีต
่ องประเมิ
นวา่ RCT นั้นน่าจะมี BIAS
้
หรือไม่
• Did Intervention & Control groups start
with same prognosis?
• Was prognostic balanced maintained as
the study progressed?
• Were the groups prognostically balanced
at the study’s complettion?
Are the result valid?
• Did Intervention & Control groups start
with same prognosis?
•Were patients randomized?
•Was randomization concealed?
•were patients similar in respect to known
prognostic factor
• Was prognostic balanced maintained as the study progressed?
• Were the groups prognostically balanced at the study’s completion?
THERAPY STUDY: Are the results of the trial valid?
What question did Randomization
the study ask?
Patients –
Intervention - Comparison -
(Internal Validity)
Outcome(s) -
1a. R- Was the assignment of patients to treatments randomised?
What is best?
Centralised computer randomisation is ideal and often
used in multi-centred trials. Smaller trials may use an
independent person (e.g, the hospital pharmacy) to
“police” the randomization.
This paper: Yes No Unclear
Where do I find the information?
The Methods should tell you how patients were allocated
to groups and whether or not randomisation was
concealed.
Comment:
1b. R- Were the groups similar at the start of the trial?
independent person (e.g, the hospital pharmacy) to
“police” the randomization.
This paper: Yes No Unclear
Comment:
concealed.
Randomization
1b. R- Were the groups similar at the start of the trial?
What is best?
Where do I find the information?
If the randomisation process worked (that is, achieved The Results should have a table of "Baseline
comparable groups) the groups should be similar. The Characteristics" comparing the randomized groups on a
more similar the groups the better it is.
number of variables that could affect the outcome (ie.
There should be some indication of whether dif ferences age, risk factors etc). If not, there may be a description of
between groups are statistically significant (ie. p values). group similarity in the first paragraphs of the Results
section.
This paper: Yes No Unclear
Comment:
2a. A – Aside from the allocated treatment, were groups treated
Are the result valid?
• Did Intervention & Control groups start with same prognosis?
• Was prognostic balanced maintained as
the study progressed?
1.To what extent was the study blinded?
• Were the groups prognostically balanced at the study’s completion?
results. Patients should also be analysed in the groups will need to read the results section to clarify the number
to which they were randomised – ‘intention-to-treat
and reason for losses to follow-up.
analysis’.
This paper: Yes No Unclear
Comment:
Blindness
3. M - Were measures objective or were the patients and clinicians kept
“blind” to which treatment was being received?
What is best?
It is ideal if the study is ‘double-blinded’ – that is, both
patients and investigators are unaware of treatment
allocation. If the outcome is objective (eg., death) then
blinding is less critical. If the outcome is subjective (eg.,
symptoms or function) then blinding of the outcome
assessor is critical.
This paper: Yes No Unclear
Comment:
Where do I find the information?
First, look in the Methods section to see if there is some
mention of masking of treatments, eg., placebos with the
same appearance or sham therapy. Second, the
Methods section should describe how the outcome was
assessed and whether the assessor/s were aware of the
patients' treatment.
between groups are statistically significant (ie. p values). group similarity in the first paragraphs of the Results
section.
Co-intervention/Confounders
This paper: Yes
No Unclear
Comment:
2a. A – Aside from the allocated treatment, were groups treated
equally?
What is best?
Apart from the intervention the patients in the different
groups should be treated the same, eg., additional
treatments or tests.
This paper: Yes No Unclear
Comment:
Where do I find the information?
Look in the Methods section for the follow-up schedule,
and permitted additional treatments, etc and in Results
for actual use.
2b. A – Were all patients who entered the trial accounted for? – and
Are the result valid?
• Did Intervention & Control groups start with same prognosis?
• Was prognostic balanced maintained as the study progressed?
• Were the groups prognostically balanced
at the study’s completion?
• Was follow up complete?
• Were patients analyzed in the group to which
they were randomized?
What is best?
Apart from the intervention the patients in the dif ferent
groups should be treated the same, eg., additional
treatments or tests.
This paper: Yes No Unclear
Comment:
Where do I find the information?
Look in the Methods section for the follow-up schedule,
and permitted additional treatments, etc and in Results
for actual use.
Intention to treat analysis
2b. A – Were all patients who entered the trial accounted for? – and
were they analysed in the groups to which they were randomised?
What is best?
Where do I find the information?
Losses to follow-up should be minimal – preferably less The Results section should say how many patients were
than 20%. However, if few patients have the outcome of andomised (eg., Baseline Characteristics table) and how
interest, then even small losses to follow-up can bias the many patients were actually included in the analysis. You
results. Patients should also be analysed in the groups will need to read the results section to clarify the number
to which they were randomised – ‘ intention-to-treat
and reason for losses to follow-up.
analysis’.
This paper: Yes No Unclear
Comment:
3. M - Were measures objective or were the patients and clinicians kept
“blind” to which treatment was being received?
What is best?
Where do I find the information?
Critical appraisal
Are the result valid?
What are the results?
How can I apply the result to patient care?
What are the result?
• How large was the treatment effect?
• How precise was the estimate of
treatment effect?
Critical Appraisal for Therapy
Articles
Treatment effect
What were the results?
1. How large was the treatment effect?
Most often results are presented as dichotomous outcomes (yes or not outcomes that happen or don't happen)
and can include such outcomes as cancer recurrence, myocardial infarction and death. Consider a study in which
15% (0.15) of the control group died and 10% (0.10) of the treatment group died after 2 years of treatment. The
results can be expressed in many ways as shown below .
What is the measure?
What does it mean?
Relative Risk (RR) = risk of the outcome in the The relative risk tells us how many times more likely it is that
treatment group / risk of the outcome in the control an event will occur in the treatment group relative to the control
group.
group. An RR of 1 means that there is no difference between
the two groups thus, the treatment had no effect. An RR < 1
means that the treatment decreases the risk of the outcome.
An RR > 1 means that the treatment increased the risk of the
outcome.
In our example, the RR = 0.10/0.15 = 0.67
Since the RR < 1, the treatment decreases the risk of death.
Absolute Risk Reduction (ARR) = risk of the The absolute risk reduction tells us the absolute dif ference in
outcome in the control group - risk of the outcome the rates of events between the two groups and gives an
group.
group. An(yes
RR orof not
1 means
that there
is no difference
between
Most often results are presented as dichotomous outcomes
outcomes
that happen
or don't happen)
the myocardial
two groups infarction
thus, the treatment
no effect.
An RR
<1
and can include such outcomes as cancer recurrence,
and death.had
Consider
a study
in which
the treatment
decreases
the riskof oftreatment.
the outcome.
15% (0.15) of the control group died and 10% (0.10)means
of thethat
treatment
group died
after 2 years
The
An RR > 1 means that the treatment increased the risk of the
results can be expressed in many ways as shown below.
outcome.
InWhat
our example,
RR = 0.10/0.15 = 0.67
Since
RR < it
1, the
treatment decreases the risk of death.
is thethemeasure?
Whatthedoes
mean?
Absolute
Risk(RR)
Reduction
= risk ofinthe
risktells
reduction
us the
absolute
Relative Risk
= risk of(ARR)
the outcome
the The absolute
relative risk
us howtells
many
times
moredifference
likely it is inthat
outcome
the control
- risk of the
outcome
rates of
the twogroup
groupsrelative
and gives
treatmentingroup
/ risk ofgroup
the outcome
in the
control the
an event
willevents
occur between
in the treatment
to theancontrol
ingroup.
the treatment group. This is also known as the indication
effect. Anbetween
ARR of
group. An ofRRtheof baseline
1 meansrisk
thatand
theretreatment
is no difference
absolute risk difference.
0themeans
that there
no difference
between
the twoAngroups
two groups
thus,isthe
treatment had
no effect.
RR < 1
thus,
treatment
had nodecreases
effect. the risk of the outcome.
meansthethat
the treatment
In our example, the ARR = 0.15 - 0.10 = 0.05 or The
absolute
benefitthatofthe
treatment
is aincreased
5% reduction
in the
An RR
> 1 means
treatment
the risk
of the
5%
death
rate.
outcome.
Relative
Risk Reduction
(RRR) = absolute
is the decreases
complementtheofrisk
the ofRRdeath.
and is
In our example,
the RR = 0.10/0.15
= 0.67 risk The
Sincerelative
the RRrisk
< 1,reduction
the treatment
reduction / risk of the outcome in the control
probably the most commonly reported measure of treatment
Absolute Risk Reduction (ARR) = risk of the The absolute risk reduction tells us the absolute difference in
group. An alternative way to calculate the RRR is effects. It tells us the reduction in the rate of the outcome in the
outcome in the control group - risk of the outcome the rates of events between the two groups and gives an
to subtract the RR from 1 (eg. RRR = 1 - RR)
treatment group relative to that in the control group.
in the treatment group. This is also known as the indication of the baseline risk and treatment effect. An ARR of
In our example, the RRR = 0.05/0.15 = 0.33 or The treatment reduced the risk of death by 33% relative to that
absolute risk difference.
0 means that there is no difference between the two groups
33%
occurring in the control group.
thus, the treatment had no effect.
Or
RRR = 1 - 0.67 = 0.33 or 33%
In our example, the ARR = 0.15 - 0.10 = 0.05 or The absolute benefit of treatment is a 5% reduction in the
Number Needed to Treat (NNT) = inverse of the The number needed to treat represents the number of patients
5%
death rate.
Treatment effect
Most often results are presented as dichotomous outcomes (yes or not outcomes that happen or don't happen)
reduction
/ risk ofsuch
the outcome
controlrecurrence,
probably
the most
commonly
measurea ofstudy
treatment
and can include
outcomesinasthecancer
myocardial
infarction
and reported
death. Consider
in which
group.
An
alternative
way
to
calculate
the
RRR
is
effects.
It
tells
us
the
reduction
in
the
rate
of
the
outcome
in
the
15% (0.15) of the control groupTreatment
died and 10% (0.10) of the treatment
group
died
after
2
years
of
treatment.
The
effect
toresults
subtract
the
RR
from
1
(eg.
RRR
=
1
RR)
treatment
group
relative
to
that
in
the
control
group.
can be expressed in many ways as shown below.
In our example, the RRR = 0.05/0.15 = 0.33 or The treatment reduced the risk of death by 33% relative to that
33%
occurring in the control group.
What is the measure?
What does it mean?
Or
RRR = 1 - 0.67 = 0.33 or 33%
Relative Needed
Risk (RR)to =Treat
risk of(NNT)
the outcome
The number
relative risk
tellstoustreat
howrepresents
many timesthemore
likely
it is that
Number
= inverseinofthethe The
needed
number
of patients
treatment
/ risk ofasthe1 /outcome
an event
occurwithin the
the experimental
treatment group
relative
to thetocontrol
ARR
and isgroup
calculated
ARR. in the control we
need towilltreat
therapy
in order
group.
group. An1 bad
RR outcome
of 1 meansandthatincorporates
there is nothedifference
prevent
durationbetween
of
the two groups
the treatment
no effect. An
RR < 1
treatment.
Clinicalthus,significance
can had
be determined
to some
meansbythatlooking
the treatment
decreases
of the outcome.
extent
at the NNTs,
but alsothebyriskweighing
the NNTs
An RR >any1 means
the treatment
risk of the
against
harms that
or adverse
effectsincreased
(NNHs) ofthetherapy.
outcome.
In our example, the NNT = 1/ 0.05 = 20
We
would need to treat 20 people for 2 years in order to
In our example, the RR = 0.10/0.15 = 0.67
Since the1 death.
RR < 1, the treatment decreases the risk of death.
prevent
Absolute
Risk Reduction
risk ofestimate
the The of
absolute
reduction tellseffect?
us the absolute difference in
1. How
precise(ARR)
was =the
therisktreatment
outcome
in
the
control
group
risk
of
the
outcome
the
rates
of
events
between
the
two
groups
and
gives
an
The true risk of the outcome in the population is not known and the best we can do is estimate the true risk based
Critical appraisal
Are the result valid?
What are the results?
How can I apply the result to patient
care?
Apply to patient care?
• Were the study patients similar to my
patient?
• Were all clinically important outcomes
considered?
• Are the likely treatment benefits worth
the potential harm and costs?
“ขอซั
ก
ถาม”
้
สวัสดี
22