D:\533556871.doc
EPI-820 Evidence-Based Medicine
Homework # 3: DIAGNOSIS
Dr. Mat Reeves BVSc, PhD
In this homework you will use several programs in an Excel spreadsheet to illustrate some general principles of
diagnostic testing. Simply download the file (HWK3_ebmdxlab.xls) from the EPI-820 web site. On a separate page
complete the answers for each of the 5 separate exercises and return to me (for the most part, I am looking for just a
few short words on each question).
Directions:
Open Excel for Windows (you should be able to use any version 5.0 or higher)
Open Excel file ‘HWK3_ebmdxlab.xls’ file.
EXERCISE 1:
Directions:
Click on first worksheet or page labeled ‘Exercise 1'
This test has a Se = 0.60 and a Sp = 0.90. The first column shows a range of prevalence figures (prior probabilities)
- from 0.01 to 0.99. I have calculated the PV’S for each prevalence. The results are shown in the accompanying
graph.
1.
What are Series 1 and Series 2?
2.
What principle is being demonstrated in the figure?
EXERCISE 2:
Directions:
Click on second worksheet or page labeled ‘Exercise 2'
There are now 4 different tests, each of which has a different Se (from 0.60 to 0.90) but the same Sp (0.90). For
each test, I have calculated the PVP and PVN for each prior probability on two separate graphs (A and B). These
results are shown in two accompanying figures that plot the PVP and PVN against the Prevalence, respectively.
1.
What do you notice about the relative position of each of the 4 lines (each of which represent a different
test) on the two graphs?
2.
How does changing the Se affect the PVP and PVN graphs?
3.
Experiment by now changing the values of Sp for the 4 tests but keep the Se constant (say, at 0.9). How
does changing the Sp affect the PVP and PVN graphs?
4.
What is going on here? - What principle is being demonstrated?
EXERCISE 3: Se/Sp or LR Approach to Bayes’ Theorem
1
D:\533556871.doc
Directions:
Click on third worksheet or page labeled ‘Exercise 3'
This exercise is designed to show that a LR approach to Bayes theorem will produce the same results as using Se/Sp.
The only values that should be changed in this example are those of Se, Sp and Prevalence (in cells B5-B7)!!!
Explore how the PVP and PVN values change after altering any one of the three values (Se, Sp, Prev).
1.
Define PVP and PVN:
PVP is.....
PVN is....
Now, explore how changing the Se, SP and Prev values (B5-B7) affects the pre-test odds and LR+ and LR- (cells
B12-14).
2.
Define LR+ and LR- for a dichotomous test. How are they related to Se and Sp?
LR+ is....
LR- is....
3.
How does the post-test probability for a positive test (LR+) and the post-test probability for a negative test
(LR-) relate to PVP and PVN?
EXERCISE 4: Bayes’ Theorem and Use of the LR
Directions:
Click on fourth worksheet or page labeled ‘Exercise 4'
This exercise uses data from an article by Giard and Hermans (Arch Pathol Lab Med, 114:852-4, 1990) that reports
likelihood ratios for the interpretation of fine needle aspiration cytology (FNAC) when used in the diagnostic
workup of breast cancer. The first table present the LRs for all 4 FNAC test outcomes.
1. How were these LRs calculated? Reproduce one of the 4 figures?
2. Interpret this LR value - what does it mean?
3. Look at the first graph (go to cell B36):
a) What does it show?
b) Which test result contains the most clinical information?
c) Does the unsatisfactory test tell you anything useful? Why?
d) Which test result rules-in disease?
e) Which test result rules-out disease?
f) In what region of the graph do you see the greatest change between the pre-test and post-test
probabilities?
Move down one page to cell A53.
Effect of Combining Results into Only 2 Categories.
We are now going to dichotomize these results - there are now only two test outcomes Malignant/Suspicious, OR
Unsatisfactory/Benign (Normal). Based on the original data, fill in the cells in the first part of this table (i.e., cells
2
D:\533556871.doc
C57, C60, C63, D57, D60 and D63).
4. What are the new LR values? What happened to them?
5. Look at the next graph (go to cell A85):
a) What does it show? What has happened to the curves?
b) Which test result contains the most clinical information?
c) Which contains the least clinical information?
d) Does a result of Malignant/Suspicious always rule-in disease?
e) Does a normal test result always rule-out disease?
f) What principle is being illustrated in this example?
6. Go back to the 2 x 2 table and alter the values in cells C57, C60, C63, D57, D60 and D63, such that the truepositive rate and the false-positive rate are now both 0.8.
a) What are then new LR values?
b) What has happened to the graph? Why?
EXERCISE 5: Example of Use of Logistic Regression Model to Calculate LRs!
Directions:
Click on fifth worksheet or page labeled ‘Exercise 5'
This worksheet is an example of how logistic regression models can be used to estimate the LR for a combination of
test data. Two models are presented (see Reeves et al., Prev Vet Med, 13:155-72, 1992) the first of which is used to
estimate the likelihood of the need for surgical exploration for horses presenting with acute abdominal pain. The
variables (or tests) used are age, sex, breed, rectal findings, frequency of abdominal pain, strength of the peripheral
pulse, and frequency of abdominal sounds.
Look at the co-efficients from the model in cells E9-E26. These are used to calculate the LR for any given
permutation of parameters.
1. Go to cell I5. This is the LR for surgery. Describe in English what it means.
2. Using the codes that are described below (see cells H20-H34), explore the effect of changing some of the
variables while keeping others constant.
a) What happens to the LR for surgery?
b) Describe the specific effects of any two of the variables on the LR for surgery.
3. Note also where the prior probability of surgery is entered (cell I13). How does changing this affect the post-test
probability?
Repeat this process for the Prognosis Model, which is used to estimate the likelihood of survival of horses with
acute abdominal pain. Once again the coefficients are shown in E45-E61. The variables or tests used are different
in this model that now includes age, sex, breed, strength of the peripheral pulse, heart rate (HR), packed cell volume
(PCV), and the need for surgical or medical intervention.
4. Using the codes that are described in cells H63-H75, explore the effect of changing some of the variables while
keeping others constant.
a) What happens to the LR and post-test probability?
b) Describe the specific effects of any two of the variables on the LR.
3
D:\533556871.doc
EXERCISE 6: Hit the Web!!!
Check out some of these Web sites:
The EBM site of David Sackett’s at Oxford:
http://cebm.jr2.ox.ac.uk/
Some great stuff here!!
Check out the Tool Box and the sections of Se/Sp, LRs, NNT’s etc.
The McMaster Site:
http://hiru.mcmaster.ca/ebm/workshop
Information on their annual EBM course.
http://www.cche.net/ebcp/ebcpbiblio.htm
Great reference resource for EBM
http://www.cche.net/usersguides/main.asp
The complete JAMA published user guides to EBM series on-line!!!!
A Bayesian Calculator:
http://www.intmed.mcw.edu/clincalc.html
A real life Bayesian calculator that gives you PVP, PVN, LR’s etc - great for checking your work!
4