A&E QMP Practical 1 Worksheet
The Bias Detectives
You are trying to find out about the prevention of stroke and run a basic Medline search.
You find what looks like a great article; it’s a meta-analysis and will take some of the
grind out of your literature search – it seems as if it’s got everything you need to know
could be in this one paper! Hooray!
Meta-analysis:
Corvol, J-C, Bouzamondo, A, Sirol, M, Hulot, J-S, Sanchez, P, Lechat, P. (2003).
Differential Effects of Lipid-Lowering Therapies on Stroke Prevention. A Metaanalysis of Randomized Trials. Arch Intern Med, 163, 669-676.
Download and read the abstract, method and results and answer the following questions:
1. What did this study aim to find out?
2. Who was carrying it out?
3. What are its main conclusions?
4. Can you see any problems with either this method or the findings?
Check your clues with the detective superintendent…
Step 2
A while later, feeling rather pleased with your first find, you are surfing the net, looking
for another article when you accidentally hit the wrong journal link and end up looking at
the following article:
Essay:
Smith, R. (2005). Medical Journals Are an Extension of the Marketing Arm of
Pharmaceutical Companies. PLoS Med, 2(5): e138:
You realise that you have stumbled on a fascinating essay by Richard Smith (ex-editor of
the BMJ and executive of the BMJ Publishing Group). Intrigued you read on…
You want to know more and by simply looking at who has cited this article, you find the
following article tells you when the journals started changing their attitude to information
on author and trial funding:
News article:
Kingsley, D. (12.09.2001). Medical journals crack down on drug money. News in
Science, ABC Science Online, Australia. Accessed on 10.09.08 from:
http://www.abc.net.au/science/news/stories/s363575.htm
You are totally appalled by what you have just read and can’t really believe it:
Discuss with your fellow detectives:
1. What is the main issue that Richard Smith is describing here?
2. Who is to blame for this? Is this failure of the pharmaceutical companies, the journals,
or the whole system?
3. What do you think about this? Have you come across “conflict of interest” before (e.g.
in your ethics learning)?
4. Would this type of influence affect the validity of a trial and is this therefore a bias?
5. How might you check a trial for this kind of effect?
Check your clues with the detective superintendent…
Step 3
So, you were not aware of the extent of the influence that the pharmaceuticals have on
trials that are published. You feel hoodwinked and cheated. In a fit of disbelief, you do
another search and find two reasonably well-known and well-cited articles and construct
a table to compare and contrast them.
Tip: You may need to look deeper than their abstracts to find the real clues...
Abstract 1:
Bloomfield Rubins, H, Davenport, J, Babikian, V, Brass, LM, Collins, D, Wexler, L,
Wagner, S, Papademetriou, V, Rutan, G, Robins, SJ, for the VA-HIT Study
Group.(2001). Reduction in Stroke With Gemfibrozil in Men With Coronary
Heart Disease and Low HDL Cholesterol. The Veterans Affairs HDL Intervention
Trial (VA-HIT).Circulation, 103, 2828.
LINK:
http://circ.ahajournals.org/cgi/content/abstract/103/23/2828?ijkey=71708eda67097254ac
187eff4f6a1779f1f5064f&keytype2=tf_ipsecsha
Abstract 2:
Heart Protection Study Collaborative Group. (2002). MRC/BHF Heart Protection Study
of cholesterol lowering with simvastatin in 20,536 high-risk individuals: a
randomised placebo-controlled trial. The Lancet, 360, 7-22.
DOI:10.1016/S0140-6736(02)09327-3
LINK:
http://sfx.nun.unsw.edu.au:9003/sfx_local?issn=00995355&date=2002&volume=360&issue=9326&spage=7
CLUE TABLE FOR STEP 3
Clues
Journal this study was first published in
Journal impact rating
Year study published
Before or after 2001 (new journal position on
conflict of interest)? (Y/N)
Authors’ positions, place of work and funding
Trial / author position funding compromising?
(Y/N and why)
No. of citations since publication*
Do you think this is significant?
(Sig / Not Sig)
Subject selection method adequate? (Y/N and
why)
Consent given before randomisation? (Y/N and
why)
The “trial drug” is compared to what? (Placebo
or another drug that has known similar effects?)
Subject allocation concealed?
Good randomisation method? (Y/N and why)
Abstract 1
Abstract 2
Blinding – list all types used
Adequate blinding used?
(Y/N and why)
Outcome measurement method adequate? (Y/N
and why) Are these ‘soft’ or hard outcome
measures?
State the dosage used for each of the main drugs
(incl. comparison if there is one)
Dosage used for drugs is correct and equivalent
quality of drug used?
(Y/N and why)
Follow- up adequate (Y/N)
Intention to treat analysis carried out? (Y/N)
Present results as:
Relative Risk Reduction (RRR)
Or Absolute Risk (AR)? Work them out if you can if
they are not available (SeeCEBM Glossary
http://www.cebm.utoronto.ca/glossary/index.htm#te_arr ).
Side effects / adverse reactions – results shown
and discussed?
Does the way these results are presented make a
difference to how you perceived these results?
* To find out how many citations there have been = use the link (if present) on the abstract page, if not use Web of Science
Check your clues with the detective superintendent…
Step 4
Phew! That was a lot of reading! So what have you discovered?
First, take a look at the table and discuss with your fellow detectives what you can see.
Next, compare and contrast the clues from the three studies. Then answer the questions
below:
What do you think about:
1. the journals in which these studies were published?
2. the trial designs?
2. who the authors are and how they are funded?
4. the conclusion of each article?
5. the potential biases present in each?
In conclusion, (to get your tick from the detective superintendent):
Which is the most transparent /least transparent article?
Which is the best run trial / worst run trial?
Reflect on what you have found = what new clues would you look out for when reading a
paper in the future?
Check your conclusion with the detective superintendent…
TICK BOX
Step 5
Now you are convinced about the influence of the pharmaceutical companies, but are
worried about your previous findings from the meta-analysis. To find out whether the
meta-analysis that you read and believed earlier has any truth in it, you run another search
find and read the following study:
Bero, L., Oostvogel, F., Bacchetti, P., and Lee, K. (2007). Factors Associated with
Findings of Published Trials of Drug–Drug Comparisons: Why Some Statins
Appear More Efficacious than Others. PLoS Medicine, 4(6), e184. Accessed on
10.09.08 from: http://medicine.plosjournals.org/perlserv/?request=getdocument&doi=10.1371/journal.pmed.0040184&ct=1
1. Does this confirm what you have already discussed or raise further questions?
2. What are some of these biases called that you have re-discovered today? Find out their
definitions.
Check your conclusion with the detective superintendent…
Step 6
If you have 10 mins left before you have to rush off to do more detecting elsewhere, carry
out a Google search to locate one or two major pharmaceutical websites. See if you can
locate a code of ethics or publication policy.
Now, take a look at the home pages for the journals for the three articles above = see if
you can found out what rules they have regarding: acknowledgements, funding sources
and disclosure of conflicts of interest.
Is Richard Smith right? Are the journal articles that we read and use in our practice still
tainted by bias due to the way the trials are published?
Are there still loopholes through which pharmaceutical companies and other invested
parties can alter the way we practice?
Check your final conclusions with the detective superintendent and you are free to go
off and sleuth elsewhere!