Critically Appraising the
Evidence
資料評讀
Best Evidence depends
on the type of Question
0 What are the phenomena/problems?
0 Observation
0 What is frequency of the problem? (Frequency)
0 Random (or consecutive) sample
0 Does this person have the problem? (Diagnosis)
0 Random (or consecutive) sample with gold standard
0 Who will get the problem? (Prognosis)
0 Follow-up of inception cohort
0 How can we alleviate the problem? (Therapy)
0 Randomized controlled trial (RCT)
Best evidence is not always from RCTs.
for an individual randomized trial
Critically Appraising the
Evidence for its (VIP)
• Validity (closeness to the truth)
• Impact (size of the effect)
• Applicability (usefulness in our clinical
practice)
Validity
closeness to the truth
研究的結果可信嗎?
Validity
(closeness to the truth)
0 Was there a fair start
0 開始是否公平
0 Was there a fair race
0 過程是否公平
0 Some finer points
0 其他細項
Validity
(closeness to the truth)
0 Was there a fair start
1. Was the assignment of patients to treatment randomized?
2. Was the randomization concealed?
3. Were the groups similar at the start of the trial?
0 Was there a fair race
4. Was follow-up of patients sufficiently long and complete?
5. Were all patients analyzed in the groups to which they were
randomized?
0 Some finer points
6. Who was blinded: Were patients, clinicians and study
personnel kept blind to treatment?
7. Were groups treated equally, apart from the experimental
therapy?
Fair Start
起始點的公平
0 Patient randomized
0 病患的分配（治療組、控制組）是隨機的
0 Randomization concealed
0 隨機的過程是隱藏的；隨機分派中心
0 病患、治療者、評估者都不知道分組的狀況
0 Groups similar
0 各組病患（治療組、控制組）在治療之前的特徵（主要
的預後因子，除了治療本身之外）都是相似的
Fair Race
過程公平
0 Follow-up sufficiently long and complete
0 CONSORT (consolidated standards of reporting trials)
statement
0 Were all patients analyzed in the groups to which they
were randomized?
Follow-Up and Drop out
0 Drop out:
0 無法忍受副作用
0 治療無效
0 Assign the drop-outs to the worst case outcome
0 >80% follow-up acceptable
Follow up Duration
0 Sufficiently long to see a clinical effect
0 Study duration
0 Expected duration of treatment
0 Sometimes, study stopped early when a large benefit
is seen
0 Look for the sample size and effect size
Patients Analyzed
in the Assigned Groups?
0 “Intention-to-treat (ITT) analysis” 意向分析，意圖治
療分析
0 無論是否接受介入，都應該維持隨機分派後的狀態
0 應該評估所有受試者的預後
0 所有接受隨機分派的受試者都應該納入分析
0 “Per-protocol (PP) analysis”依計畫書分析
0 only patients who complete the entire clinical trial are
counted towards the final results
0 只有完整接受治療到最後，才納入分析
Blinding
0 Patients
0 Treating clinician
0 Study personnel
0 Evaluating personnel
0 Surgery studies
0 Blinded assessment
0 Article should explicitly state who was blinded
Treat Equally,
Apart From the Experimental Therapy
0 Contamination:
0 The control group receivedthe same therapy as the
experimental group
0 Co-intervention:
0 When one group or the other received different medical
care based upon their group assignment
Validity
(closeness to the truth)
0 Was there a fair start
1. Was the assignment of patients to treatment randomized?
2. Was the randomization concealed?
3. Were the groups similar at the start of the trial?
0 Was there a fair race
4. Was follow-up of patients sufficiently long and complete?
5. Were all patients analyzed in the groups to which they were
randomized?
0 Some finer points
6. Who was blinded: Were patients, clinicians and study
personnel kept blind to treatment?
7. Were groups treated equally, apart from the experimental
therapy?
RAM-bo
0 研究族群是否具有代表性（Representative）?
0 隨機選擇（random selection）／連貫性／起始點病人
群
0 或者，如果是比較性的，組別間是否可以比較?
0 隨機分派（random allocation）／調整
0 是否有足夠的確認與追蹤（Ascertainment/follow up
）?
0 反應率／追蹤率／確認＞80%
0 結果的估計值（Measurement）是否公正?恰當?
0 使用盲法（blinded）或客觀的（objective）估計
Impact
size of the effect
結果是否重要
Impact
Size of the Effect
0 The magnitude of the treatment effect
0 治療效果的強度
0 However precise is the estimate of the treatment
effect?
0 治療效果的預估: 信賴區間
Treatment Effect
0 Outcome
0 Important outcome: mortality
0 Surrogate outcome: BMD, risks of fracture
0 Composite outcome:
0 Methods to report the results
Methods to Report the Results
0 Control Event Rate (CER)
0 對照組事件發生率
0 Experimental Event Rate (EER)
0 實驗組事件發生率
0 Relative risk reduction (RRR)
0 相對風險比率差
0 relative benefit increase，relative risk increase
0 Absolute risk reduction (ARR)
0 絕對對風險比率差
0 absolute benefit increase，absolute risk increase
Outcome
No Outcome
Sum
Experimental
A
B
A+B
Control
C
D
C+D
0 Control Event Rare (CER) = C/(C+D)
0 Experimental Event Rate (EER) = A/(A+B)
0 Relative Risk Reduction = |CER-EER| /CER
0 Absolute Risk Reduction = |CER-EER|
0 Number Needed to Treat (NNT) = 1 /ARR
Number Needed to Treat (NNT)
0 NNT = 1/ARR
0 需要被治療的病人數目
0 在一段實驗期間內，使一位病人達到實驗組治療之有
意結果（或預防產生一個不良結果）所需治療的病人
數目
0 為減少一個不良結果所需治療的病人數目
Number Needed to Harm
NNH
0 NNH= 1/ARI (absolute risk increase)
Cost and Effect - NNT and NNH
from Statin Stroke Prevention Trial
0 For rhabdomyolysis
0 Control event rate (CER): 0.03%
0 Experimental event rate (EER): 0.05%
0 Absolute risk increase (ARI) = |0.03%-0.05%| = 0.02%
0 Number Needed to Harm (NNH) = 1/ARI = 1/0.02%=5000
NNT vs Follow-Up Time
不同的試驗，追蹤時間不同，如何比較
0 假設相對風險下降固定
0 Relative Risk Reduction = |CER-EER| /CER
0 NNT = 1/ARR = 1/ |CER-EER|
0 如果追蹤時間變為為原本的 t 倍
0 CERt = CER * t ;
EERt = EER * t
0 NNTt = 1/ARRt = 1/|CERt-EERt| = 1/(|CER-EER|*t) = NNT/t
0 t = hypothetic time / observed time
0 NNThypothetical = NNTobserved x (observed time/hypothetic time)
However precise is the estimate
of the treatment effect?
治療效果的預估: 信賴區間
信賴區間
0 95% Confidence interval
0 實際值有95% 的機會是在這個信賴區間內的
Applicability
usefulness in our clinical practice
研究的結果是否可以套用在我的病人身上
適用性
0 Is our patient so different from those in the study that
its results cannot apply
0 病患是否與研究受試者明顯不同
0 Is the treatment feasible in our setting
0 治療是否可行，健保
0 What are our patient’s potential benefits and harms
from the therapy
0 好處與壞處
0 What are our patient’s values and expectations for
both the outcome we are trying to prevent and the
treatment we are offering
0 病患的價值觀與期待
Patients vs
Study Population
0 Inclusion and exclusion criteria
0 通常不會如此考慮
0 Sociodemographic status
0 Pathobiology
Potential Benefit and Harms
0 Estimate our patient’s unique benefits and risks
0 Patient’s individual CER (control event rate) or
“patient’s expected event rate” (PEER)
0 From NNT (number needed to treat ) and NNH
(number needed to harm)
Patient’s Expected Event Rate
PEER
0 Set to the overall control event rate (CER) of the study
0 Set to the CER of subgroup, similar to our patient
0 According to the clinical prediction guide, assign PEER
0 From other paper, assign PEER
0 NNT= 1/(PEER x RRR)
0 NNH = 1/(PEER x RRI)
0 Relative Risk Reduction (RRR)= |CER-EER| /CER
= ARR / CER
0 Absolute Risk Reduction (ARR)= |CER-EER|
= CER x RRR
0 Number Needed to Treat (NNT) = 1 /ARR
=1/(|CER-EER|) = 1 / (CER x RRR)
0 NNT= 1/(PEER x RRR)
0 NNH = 1/(PEER x RRI)
From NNT / NNH
0 To estimate our patient’s risk of the outcome event
relative to that of the average control patient
0 Decimal fraction (ft / fh)
0 如果病人比起control 有兩倍的風險 ft = 2
0 如果病人比起control 只有一半倍的風險 ft = 0.5
0 病人的 NNT = NNT / ft
0 同理
0 病人的 NNH = NNH / fh
Individualized Benefits and Risks
個人化的利與弊
0 Clinical decision analysis: 費時
0 Likelihood of being helped and harmed (LHH)
0 LHH = ARR (absolute risk reduction) ÷ ARI (absolute
risk increase)
0 = (1/NNT) ÷ ( 1/NNH)
0 LHH = [(1/NNT) × ft] ÷ [( 1/NNH) × fh]
Patients’ Values and Expectations
病人的價值與期望
0 Ask patient to make value judgment about the relative
severity of the outcome (disease treated, adverse
event)
0 severity factor = s 因子
0 = 疾病的嚴重性 / 副作用嚴重性
0 LHH = [ (1/NNT) × ft × s] ÷ [( 1/NNH) × fh]
0 severity factor = s 因子
0 = 疾病的嚴重性 / 副作用嚴重性
0 例如 以 statin 藥物 來預防中風
0 中風：0.025
0 橫紋肌溶解症：0.95
0 S = 0.95/0.025 = 38
0 中風為橫紋肌溶解症38倍的嚴重
0 LHH = [ (1/NNT) × ft × (1-Uevent) ] ÷ [( 1/NNH) × fh ×
(1-Utoxicity)]
0 1-Utility = Disutility = Harm
一個實際的例子
Statin vs Stroke Prevention
Outcome
(stroke)
No outcome
Experimental
444
9825
Control
585
9682
0 CER (control event rate) = 585 / (585+9682) = 0.057
0 EER (experimental event rate)= 444 / (444+9825) =
0.043
0 RRR (relative risk reduction) = (0.057-0.043)/0.057 =
24.6%
0 ARR (absolute risk reduction) = (0.057 – 0.043) =
0.014
0 NNT = 1/0.014 = 71.42  72
http://ktclearinghouse.ca/cebm/practise/ca/calculators/statscalc
Outcome
(rhabdomyolysis)
No outcome
Experimental
5
10264
Control
3
10264
0 CER (control event rate) = 3 / (3+10264) = 0.00029
0 EER (experimental event rate)= 5 / (5+10264) =
0.00049
0 RRI (relative risk increase) = (0.000490.00029)/0.00029 = 69.0%
0 ARI (absolute risk increase) = (0.00049 – 0.00029) =
0.0002
0 NNH = 1/0.0002 = 5000
個人化的NNT與 NNH
0 Decimal fraction (ft / fh)
0 病人的 NNT = NNT / ft
0 如果病人比起control 有兩倍的風險 ft = 2
0 NNT=72/2=36
0 如果病人比起control 只有一半倍的風險 ft = 0.5
0 NNT=72/0.5=144
0 病人的 NNH = NNH / fh
0 如果病人比起control 有兩倍的風險 ft = 2
0 NNH=5000/2=2500
0 如果病人比起control 只有一半倍的風險 ft = 0.5
0 NNT=5000/0.5=10000
Likelihood of being helped and harmed
(LHH)
0 LHH = (1/NNT) ÷ ( 1/NNH)
0 = (1/72) ÷ (1/5000) = 69.4 ≒ 70
0 Statin 治療 使他受益的機會為受害的機會的 70 倍
0 LHH = [(1/NNT) × ft] ÷ [( 1/NNH) × fh]
再加上病患的價值觀
0 LHH = [ (1/NNT) × ft × s] ÷ [( 1/NNH) × fh]
0 severity factor = s 因子
0 例如 以 statin 藥物 來預防中風
0 中風：0.025
0 橫紋肌溶解症：0.95
0 S = 0.95/0.025 = 38
0 中風為橫紋肌溶解症38倍的嚴重
0 LHH = [(1/72) x 38] ÷ (1/5000) = 2638.8