How to smell a rat: Examples of scientific and
regulatory failure in medicine
Dr Aubrey Blumsohn
Presentation at Radstats
British Library, London
24 February 2012
Nature of science – and how we define scientific fraud
Data – and access to data
Regulators (watchdogs)
Individual responsibility in science
Science
Data
Regulators
Individual responsibility
History
• Einstein
– Insanity: doing the same thing over and over
again and expecting different results.
What is Science?
• UK Science Council (2009)
– Spent a year working out a new definition of science.
– Claim as the first "official definition of science" ever
published.
•
“Science is the pursuit of knowledge and
understanding of the natural and social world
following a systematic methodology based on
evidence”
What is Science? – Mertonian Norms
• Communalism
– Everyone can see what’s going on
• Universalism
– Doing of it not restricted to special people
• Disinterestedness
– An arms length attitude.
• Skepticism (Organized Skepticism) – Claims exposed to critical scrutiny. Awkward questions.
Merton, Robert K. (1973), "The Normative Structure of Science",
Science demands awkward people
It takes all sorts of people
Example 1: “Mr Nobody”
and Journal of the American Medical Association (JAMA)
Letter #1 to JAMA
Letter #2 To BMJ
Scientific deception
- and its mis-definition
The Classics
William Summerlin (Sloan Kettering, 1984) faked transplantation experiments in white mice
by blackening patches of their skin with a pen
Vijay Soman (Yale, 1980) a diabetologist wrote 12 papers based on invented data. His coauthor Felig had to resign
The Classics
Robert Slutsky (Radiologist, University
of California). Published 137 papers
between 1978 and 1985 -- one every 20
days. Many faked.
John Darsee (Cardiologist, Harvard,
1981). Observed falsifying data. His boss,
Eugene Braunwald ignored the problem.
Later investigations discovered more than a
100 fraudulent papers
In these odd cases, the stories,
as told, usually minimise
the involvement of senior
colleagues, University
administrators, Hospitals
and regulatory bodies who
hid the misconduct before it
was eventually exposed
The impression
• The choice of cases implicitly defines what we
mean by scientific fraud/misconduct
– Fraud is rare
– It involves a particular type of activity
(painting a mouse)
– Usually involves an aberrant (male) acting alone
(or duping otherwise honourable collaborators)
“Problems of scientific misconduct are rare - the
product of psychopathic behaviour’ originating in
temporarily deranged minds”
President of the National Academy of Sciences
(at Congressional Hearings)
And the solution
• Is to create yet more procedures and
guidance
• And for Universities to create robust, secret,
legalistic procedures to investigate these
rare rogues
• Only one problem.
As Arnold
Schwarzenegger once said:
“it's bullshit—all of it”
The reality
• Not rare
• Scientific misconduct
often not an individual
activity – but a group
activity (or involves
group collusion)
Group-think
• Hannah Arendt (1965)
– "rule by nobody".
• No single individual feels responsible for what is
ultimately done.
• Indeed many people have gone to great lengths to
make sure that this exact state of affairs achieved
–
–
–
–
Legislators
Ethics Committees
Trade bodies
Regulators
• System designers responsible for fraud?
The reality
• The offender is often
not a lone white male
psychopath
• Corporate fraud
The reality
• Corporation acting at
the University
interface
The reality
• or a journal editor
The reality
• or a regulator ….
• or government itself
The mechanisms also differ
• The mechanism of
misconduct is also
usually not direct
fabrication of findings
but something far
more nuanced
Defining pornography
• "hard-core pornography"
was hard to define, but "I
know it when I see it“
Potter Stewart
(Associate Justice of the
US Supreme Court)
Misconduct
• Misconduct is nothing to do with the correctness of
conclusion
– To err is human, and scientific!!!
•
•
•
•
It has to do with behaviours
Behaviours which could disturb the scientific record
Not binary
We need to get away from ludicrous discussion about
restrictive definitions and start discussing behaviours
– Openly
Example 2: The fearful junior
• Approached by a Junior Doctor
• In order to gain her pathology registration she had to submit her
dissertation
• She could not write up her dissertation or prepare the main area
of work for publication because her supervisor had made this
impossible – for years…..
• She could only present the secondary aspects
• The work involved looking at physiological changes in healthy
volunteers randomised to atorvastatin or placebo
• The study showed that atorvastatin induced a diabetes like state
• Her supervisor was funded by the manufacturer of atorvastatin
Example 3: Does Seroquel cause
diabetes and weight gain
• But there was a cloud on the horizon.
The F.D.A. was beginning to look into the
issue of Diabetes in patients on Atypical
Antipsychotics and wrote to Astrazeneca
• Then somebody noticed the obvious that
they had more than just 18 months of data
in these subjects
Example 4 – Famovir and Herpes
• Famvir is used for treatment of herpes
zoster
• There was a nine year delay before partial
publication of findings which had adverse
implications for the sponsor
What are data?
Data is not the same
as representation of
that data
Getting a hamburger when what
you need is the original cow
Example 5: Vioxx APPROVE Trial
New York Times May 31, 2006
Look at the language
Merck & Co. acknowledged that it
misidentified a statistical method
Merck has contended that the
study shows
The company said in a statement
yesterday
Merck has insisted that its 18-month
argument holds up
while outside scientists have cast doubt on
the company's interpretation.
Merck research chief Peter Kim said
the company stood by its analysis
Example 6: Vioxx AVANTAGE
Trial
• On Oct. 21, 1999 a 73-year-old female
participant dies in a clinical trial
• The trial was Merck’s Advantage trial of
Vioxx
• She died a few minutes after calling her son
to tell him she felt short of breath
• The Advantage trial was a very short
(12-week) clinical trial
• eight people taking Vioxx suffered heart
attacks or sudden cardiac death in the trial
(compared with only one not on the drug)
• The difference was statistically significant,
but Merck never showed the data that way
• In the published study, the “author”
(Dr. Lisse, University of Arizona), the 8
deaths became` 5, a difference that did not
reach statistical significance.
• The paper never mentioned the other
cardiac deaths of patients taking Vioxx,
including the 73-year-old woman.
• Dr. Lisse said that while he was listed as the
paper's first author, Merck actually wrote the
report.
"Merck designed the trial, paid for the trial, ran the
trial," Dr. Lisse said.
"Merck came to me after the study was completed
and said, 'We want your help to work on the
paper.' The initial paper was written at Merck, and
then it was sent to me for editing."
• Results from the trials were also supposed
to be blinded when coded
• During coding, Dr. Eliav Barr, a Merck
employee judged that the woman “had
probably died of a heart attack."
• Dr. Reicin a Merck
executive immediately
shot back :
"I think this should be
called an unknown cause
of death." A few hours
later, she wrote, "I would
prefer unknown cause of
death so we don't raise
concerns."
• In November 2001, an F.D.A. reviewer who
had examined all the company's data
concluded in a report to other agency
officials that Merck had cheated. But the
F.D.A. never told anyone either.
• Dr. Lisse, the study "author" said he had
never heard of the case of the woman who
died, until told of it by the New York
Times.
Example 7: Vioxx - Fake Journals
• Merck paid Elsevier to create a number of fake
scientific journals
• This came to light 2009 in the context of a civil
suit filed by Graeme Peterson, who suffered a
heart attack in 2003 while on Vioxx
• These were designed in such a way that the
average medical reader would “mistake the
publication for a genuine peer reviewed medical
journal”
• The articles supported Merck Drugs
• One of the members of Australasian Journal
of Bone and Joint Medicine's "Honorary
Editorial Board," Peter Brooks, a
rheumatologist said:
he didn't recall who asked him to serve on
the board, …. "You get involved in a whole
bunch of things at this level".
• Elsevier issued at Press release and an
apology
– We “published a series of sponsored article compilation
publications, on behalf of pharmaceutical clients, that were made
to look like journals and lacked the proper disclosures. This was
an unacceptable practice, and we regret that it took place.”
• But what of the academics involved, and
their institutions?
REGULATORS AND SYSTEMS
•
•
•
•
•
•
Medical Writers (Ghost writers)
Medical Journals
MHRA (UK Drug regulator)
Medical Ethics system
UK Research Integrity Office
General Medical Council
The illusion of due diligence
• "We're looking for the illusion
of due diligence, Mr. Pope.
Two criminal acts successfully
prosecuted -- it gives us that
illusion."
Medical Writers
(and Ghostwriters)
“When the time comes for publication, the medical writer writes the
publication in consultation with the named author(s). This usually begins
with discussions between the medical writer and authors(s) about the main
features of the paper, after which the medical writer writes the first
draft. The draft is then revised (sometimes many times) after review by
the author(s). Thus although the authors do not actually write the paper,
they do have control over its content, which is far more important.
Naturally, they have access to the report of the study and its data
tables, so they can verify that the paper is accurate. They often don't
have access to the raw data, but it is worth repeating that raw data are
of very little use when writing a paper. Results of a study need
summarising and analysing before they mean anything, and it is those
summaries and analyses that are important for proper interpretation of a
study, not the raw data themselves. “
Adam Jacobs – European Medical Writers Association
BMJ 21 Nov 2004
Journals
Motto – Which? Magazine
No advertising, no bias, no hidden agenda. Just expert advice from an
independent source
Motto – BMJ
“We protect our reputation by careful balance of editorial and advertising, which
means your messages will always stand out.”
(http://group.bmj.com/group/advertising)
The Medicines and
Healthcare Regulatory
Agency (MHRA)
MHRA
• Has absolutely nothing to say on the problem of
ghostwriting
• Has nothing to say on the problem of denial of
access to data (to authors or the scientific
community in general)
• Have no apparent concerns nor any apparent remit
to deal with events at Sheffield University
• Even suggested that it might be “illegal” for
authors to obtain the randomisation codes in a
study they are authoring because commercial
companies “own” those codes.
Risedronate
• FOI:
– Did the MHRA receive and scrutinise in any
way the raw data relating to the three key
clinical trials which led to approval of
Risedronate
– Has the MHRA got the randomisation codes
Risedronate
• FOI response:
– We cannot tell you because part of the response is
commercially confidential
• Reply: The questions required answers of “yes” or
“no”. Which parts of the Yes or No are
commercially confidential
• Eventual FOI response: The MHRA did not
scrutinise or retain any actual raw data before
licensing Risedronate. They simply accepted what
the company told them.
The MHRA (HOC SC Report 2005)
• "practices have developed which act against
the public interest."
• The MHRA relies on company data
presented as reports, in its decision whether
or not to licence a drug. Raw data is very
rarely analysed.
The MHRA (HOC SC Report 2005)
• “The MHRA Chairman suggested that trust
underpinned the stance of the MHRA towards
the companies they regulate. We inferred that
this extended to the routine acceptance of
companies’ summaries of the results of tests
on their drugs as true reflections of the raw
data on which they were based.”
• “Evidence that regulatory agencies are
acquiescing in biased later publication of the
information they hold.”
The MHRA (HOC SC Report 2005)
• “Lack of transparency has played a major
part in allowing failings to continue.”
• “100% of the funding for the MHRA comes
from the industry it regulates”
• "In view of the failings of the MHRA, we
recommend a fundamental review of the
organisation."
The General Medical Council
• Body that “regulates” doctors
• A science-based profession
• Might have thought they would take
scientific integrity seriously
• Long history of a very lax approach to
scientific fraud
– Particularly when it involves very senior
academics
Dr David Hurlstone
•
•
•
•
Sheffield Teaching Hospitals NHS Trust
Bowel cancer research
2006 BUPA Foundation Research Award
2007 Medical Futures Innovation Oncology &
Cancer Care Award
• GMC - dishonesty in three research studies
published in 2007 & 2008 (not retracted)
• Suspended from Medical Register for 3 months
• On return to work retained Clinical Excellence
Award
What did he do?
• In three publications
– Submitted authors’ consent forms to journal with false
signatures of collaborators
– Studied patients without ethical approval
– Created consent forms for patient records in retrospect
– Stated falsely that patients had been randomised
– Made false statements about microscopy having been carried
out
– Stated falsely that two blinded specialists had exampled
samples – they did not exist
– Stated falsely that an independent nurse participated in the
study and that the study endoscopist was blinded
• And so on…..
GMC and Richard Eastell
• 3 publications intended by Procter and Gamble
(1992-1995)
• Authors actively denied randomisation codes by
sponsor
• Eastell prepared to go along regardless
• Prepared to induce a colleague to do the same
• False conclusions in all manuscripts
• Research Dean of a Medical School
R&D Director of Sheffield NHS Trust
How to get out of this?
• Didn’t write the false statement
The “medical writer” wrote that as well
• I don’t need raw data as a scientist because I would not
know how to analyse it (and presumably neither would my
coauthors). It is very hard to analyse data.
• The statement was in 2002.
– In 2002 I had a watershed moment when I realised this kind of
activity was inappropriate.
– I would never have done it again after 2002, and I will never do
this sort of thing again.
• Legal manoeuvres to avoid having the GMC panel
examine any information subsequent to this 2002 paper
and repeated events
The GMC response
• 2002 statement
–
–
–
–
–
Data are raw data
Data are having the data
It was false
It was misleading
It was negligent
• Epiphany in 2002
• 2003/2004/2005???
• No punishment - “it would interfere
with his ability to get research
funding”!
• This type of behaviour has no impact
on patient care (according to the GMC)
Dr Guirong Jiang
• Radiologist
• Working as a postdoctoral fellow at the University
of Sheffield
• One of the most experienced radiologists in
the world in terms of diagnosis and definition of
vertebral fracture
• Several complaints about Richard Eastell 2007-2010
– Including his wish to author in the absence of data
– Apparent secret contracts involving Eastell, Sheffield University and
Commercial Companies preventing publication
– Jiang not permitted to see contracts apparently preventing her right
to publish findings she believed to be true based on studies in which
she was involved