survey
Science Fraud:
The Hard Figures
One in twelve medical scientists in Flanders admits to making up or ‘massaging’ data in order for it to match a hypothesis. And almost six in twelve
see such fraudulent practices happening around them. They identify high
publication pressure as one of the causes.
Eos 24
I
n November and December 2012 Belgian
science journalist Reinout Verbeke (editor
of Eos Magazine) spread an anonymous
survey on fraud and pressure to publish
among scientists of the Medical Science faculties of all Flemish universities. The result
was 315 completely filled in questionnaires.
It is the first quantitative survey on science
fraud in Flanders. Psychiatrist and researcher Joeri Tijdink (VU University Medical
Center Amsterdam) collaborated on the survey. He did another sounding in 2011 in the
Netherlands, before the scandal surrounding
Diederik Stapel had broken out – the social
psychologist who had made up data and experiments. For years nobody had been on to
him. Stapel and his unsuspecting doctoral
students and co-authors even made top magazines with their fictitious studies. Luckily
though, such large-scale fraud is rather rare.
The results of the Flemish survey are striking. Of the 315 participating scientists, four
(1.3%) admit to having made up data at least
once in the last three years. If what they say
is true, this probably concerns fraud that is
still undiscovered. 23 respondents (7.3%)
admit to having selectively removed data or
results to make research match a hypothesis,
so-called ‘data massaging’. Overall, about 8%
of the Flemish medical scientists admits to recently having made up and/or massaged data.
The figures are worse than the international
average. A meta-analysis of 18 scientific studies on fraud by Daniele Fanelli showed that
on average 2% of all scientists (from different
fields of study) admitted to having done similar practices at least once (PloS ONE, 2009).
Why are the results among Flemish respondents even worse? “That doesn’t surprise me,
because we are talking about medical scientists”, says American journalist and fraud
expert Ivan Oransky from RetractionWatch.
com. “Cooperating with the pharmaceutical
industry gains researchers financial rewards.
That could pressurise scientists to cut corners.” André Van Steirteghem, a pioneer in
reproductive medicine and secretary of the
Committee on Publication Ethics (COPE),
thinks there is something else at play. “There
spread still. “Surveys have their limits”, says
Daniele Fanelli. “Many cheaters won’t admit
to having done it, or will falsely assume they
have a clean conscience.”
Fraudulent Colleagues
But ask scientists if they see similar practices
happening around them and there will less
hesitation to answer. Fanelli’s meta-analysis
showed that an average of 14% of the respondents had noticed made up or manipulated data in colleagues’ research. Here, too, Flemish
figures are much higher: 47% has witnessed
such practices in their direct surroundings
or heard about them firsthand. In this case,
reality will probably be less harsh: “When it
comes to reporting other people’s fraud, the
In certain labs there seem to be a
‘culture’ in which malpractice is
commonplace
is a significant lack of openness on fraud and
malpractice at Flemish universities. This survey asked scientists about their perceptions
for the very first time. They were able to vent
their feelings. I think that explains the high
figures in Flanders.” We can even suspect
malpractices in Flanders to be more wide-
Mohammed Ali effect comes in play”, Fanelli
says (Ali found himself morally better than the
others, Ed.). “You always think the others are
just marginally less reliable and less honest
than yourself. That explains in part why the
questions about the researchers’ colleagues
resulted in many more positive answers than
Fraud: black area (last three years)
102%
98,7%
96%
Did you fabricate
and / or selectively
removed data to
match hypothesis?
92,7%
80%
76%
64%
32%
16%
0%
56,2%
no
48%
no
24%
43,8%
no
no
7,3%
1,3%
yes
yes
self
witnessed in direct
surroundings
Fabrication of data
yes
Flemish
survey
7,9%
yes
yes
self
witnessed in direct
surroundings
Remove data to make
research match hypothesis
Eos 25
Other surveys
(meta-analysis
Fanelli, 2009)
2%
yes
Did you witness
fabrication and / or
selective removal
of data to match
hypothesis?
Flemish
survey
47,3%
yes
Other surveys
(meta-analysis
Fanelli, 2009))
14,1%
yes
Discovered Fraud
The number of well-founded complaints since the instalment of the commissions of scientific integrity (CWI) at the Flemish universities. As a comparison: Dutch journalist Frank van Kolfschooten has counted 35 well-founded cases (at 20 research institutes) in the Netherlands since 2005, 26
of which were penalised. (Book: Ontspoorde wetenschap).
Plagiarism
Fraud
Authorship
Other*1
Remark
University of Antwerp (2008-2012)
1
0
0
0
sanction: deprived of doctorate
University of Ghent (2011-2012)
1
1
0
0
sanction: none / under consideration
University of Leuven (2007-2012)
-
-
-
-
12 well-founded complaints, no further information
1
0
0
0
sanction: reprimand in writing
0
0
0
University
Free University of Brussels*2
2012)
(2002-
University of Hasselt*3
0
Other research institutes: VIB: no information, VITO (2008-2012): 0, INBO: 2x conflict of interest, sanctions: limitation of issuing project advice, BIRA: 0,
KBIN (2005-2012): 0, SOMA: 0, IMEC (2010-2012): 0, KMI: 0
*1e.g. conflict of interest *2 VUB has procedures to handle integrity complaints, but is now planning the instalment of a commission *3 In 2013 a commission was installed at UHasselt. No complaints
filed before.
the questions about their own practices.” We
cannot rule out the possibility that several researchers from a certain lab all refer to the same dubious colleague. Still, the high amount
of recently recorded or declared fraud – 24%
witnessed the ‘making up of data’ happening
around them, 44% witnessed colleagues
‘massage data’ – could mean that in certain
labs there is a ‘culture’ in which malpractice is
commonplace. In an anonymous survey many
researchers point to fraud by colleagues. But
they are too little inclined to officially report
it – considering the few complaints filed with
the universities’ commissions of scientific integrity (cf. ‘Discovered fraud’).
Plagiarism
Plagiarism – using someone else’s ideas without their consent or without citing correctly –
is a delicate topic, too. Many universities see it
as one of the ‘big three’, in addition to making
up and manipulating. It is a violation of intel-
lectual property as well. Still, you could say
that from a scientific point of view, plagiarism
is less harmful than the other two: it does not
disrupt scientific knowledge as such. Few
medical scientists (0.6%) admit to recently
ence. That comprises, among others, salami
slicing, dividing research into as many separate science articles as possible, or the shotgun approach, including as many variables as
possible in the hope that one will show so-
46% of professors think that pressure to
publish makes science ‘ill’
having committed plagiarism, but – remarkably – one in three (35%) sees it occurring in
their direct surroundings. Unnoticed stealing
of texts, parts of texts or ideas is more difficult today than it used to be. Many specialist
journals use plagiarism software, which scans
science articles entirely for copied passages.
Grey Area
We also probed for different practices situated in the grey area – so-called sloppy sci-
mething interesting. The selective reporting
of research that ‘works out’ or that confirms
your own previous research also belongs to
this grey area (cf. ‘Fifty shades of fraud’).
Dropping obeservations or data based on a
gut feeling that they were not right, is a dubious practice, too. In our survey 27% of the
respondents admit to having done it in the
past. A 2005 study among more than three
thousand American medical researchers yielded 15% yesses (Brian Martinson, Nature).
Fraude: grey area (last three years)
96,8%
100%
79,8%
73,5%
60%
42,9%
26,5%
20%
no
77,9%
63,1%
57,1%
36,9%
no
20,2%
no
yes
yes
yes
self
witnessed in direct
surroundings
Dropping observations or data
based on a gut feeling that they
were not right
self
no
no
yes
witnessed in direct
surroundings
Overlooking others’ use of flawed
data or questionable interpretation
of data
Eos 26
3,2%
yes
22,1%
no
yes
self
witnessed in direct
surroundings
Failing to present data that
contradicts one’s own previous
research
One in five Flemish medical scientists tolerate colleagues’ inadequate data or dubious
interpretations every now and then (13% in
Martinson’s study). To the question whether
they had ever withheld data that contradicted
some of their own previous research, 3% of
the Flemish medical scientists answered affermatively (6% in Martinson’s study).
Medical scientists very often do research in
cooperation with pharmaceutical companies.
4% of the respondents state that they have
changed the methodology or results of a study after being pressured by financiers – less
than the 16% in Martinson’s study. But 18%
of the respondents in the flemish survey has
recently witnessed a colleague cave in to the
pressure of a financier.
“Grey area practices are dangerous, it’s a
slippery slope”, says Joeri Tijdink, who conducted our survey. “If you start by forgetting
a citation just once, more serious ‘offences’
are just behind the corner. There is no penalty
for not following academic ethical standards.
Especially if you only do research and seldom
cooperate with other scientists who can keep
an eye on your activities such practices can be
tempting. One tap on the delete key and a test
subject is gone from your dataset…”
Publication Pressure
One of the causes of the malpractices is the
pressure to publish. Publication of research
articles is crucial for scientists’ careers and
for the esteem of the research groups to which
they belong. Individual researchers are ranked
according to the production of papers and their
‘impact scores’. That ranking is important for
promotions at the university or for acquiring
research funding. The publish-or-perish culture increases the pressure on researchers to
publish a lot and also adds to competition
between them. That could benefit the efficiency and productivity of science research. It is
95,9%
82,3%
neen
17,7%
4,1%
ja
neen
ja
self
witnessed in direct
surroundings
Changing the methodology or
results in respons to pressure
from a funding source
Fifty Shades of Fraud
Fraud comprises a wide array of malpractices. The most blatant forms of fraud include
‘making up data’, ‘manipulating data to make
it match a hypothesis’ and plagiarism (black
area). In addition, there are several ‘dubious
research practices’ (grey area), such as double
publishing (self-plagiarism), withholding undesirable research results, undeserved authorships or dividing research into as many separate science articles as possible (salami slicing).
Frequency
Seriousness
of
icts
val erad no
t
n
o
em th
le
ha
es
e r ypo or
at c h, c ter
oub
al r
ctivh a h ging a
a thsearc gy af …
s, d ing…
t
e
Ide
p
l
a
i
d
re dolo iers
sh lic
, se atc ), for dat
ing us
c
hor i s
old revio etho finan
risme it mdata ng up
h
aut, salam
a
i
h
m
p
d
t
i
k
y
g
g
g
ve
Wi own ta or ed b
Pla o ma agin mak
ser lishin
a sur
e
s
s
t
’
d
d
s
e
b
a
a
Un pu
on ging pres
dat is (m
ng
s
bei
ch
ear
one of the reasons for the significant increase
in the number of science papers that the last
decennia have seen: every twelve years the
number of articles doubles. In 2012 the medical database PubMed registered more than
1.1 million new articles, that equals as many
as 126 per hour. The total number of papers
published last year, in all fields of study, is
estimated to be twice as high. The number
of specialist journals increased as well: from
5,000 in 1997 to 8,281 in 2012.
But the pressure to publish and the intense
competition among scientists also have negative effects. Researchers are urged to produce
‘publishable’ results, which possibly comes at
the cost of the quality and integrity of their research. Specialist journals with a high impact
predominantly publish ‘positive’ research:
results that confirm the predetermined hypothesis. “Negative research is equally valuable
from a scientific point of view, but it isn’t sexy
for the specialist journals and their readers”,
says Joeri Tijdink. “That is why some scientists interfere with the data to improve the
outcome and thereby increase their chances at
being published. That goes from small manipulations – leaving out a test subject – to grave
violations, such as photoshopping images of
experiments or making up complete datasets.”
Such practices are harmful for science, because other studies will sooner or later rely
or build on the results. And medical research
based on such unreliable research results,
presents a possible danger to test subjects in
Eos 27
clinical studies and to patients.
rotten Science
A majority of the Flemish medical scientists
experiences publication pressure. If we only
take professors into account, 66% thinks the
pressure to publish is “excessive”, a number
that is higher than in Joeri Tijdink’s identical
survey held in the Netherlands at the end of
2011 (54%).
The answers of Flemish medical professors
also show a noticeable degree of scepsis.
58% thinks that pressure to publish urges
some colleagues to (un)consciously manipulate data (compared to 33% of Dutch medical
professors). The pressure also makes them
question the value of science. 57% is of the
opinion that pressure to publish leads to serious doubts as to the validity of global research results (Netherlands: 38%). And 46%
thinks that pressure to publish makes science
downright ‘ill’ (Netherlands: 25%).
“I always thought that Dutch medical professors had a cynical view on science”, says Tijdink. “But now that I see the Flemish figures,
I must say I am rather shocked. Are Flemings
more cynical towards science or are there
other factors at play? Maybe the lower science
budgets play a part. There might be more of a
‘sports star culture’ in the Netherlands: a higher encouragement of science and a higher
budget. That means that the ‘sports stars’ –
professors – can focus more on science and
are less subjected to publication pressure.
Publication pressure
1 2 3 4 5
Flemish medical researchers (%)
(Based on the Publication Pressure Questionnaire of Joeri Tijdink, VU University Medical Center Amsterdam)
‘I think the pressure to
publish has become
excessive’
72,1 16,5 11,4
I agree
I don’t agree
neutral
‘The publication pressure puts my relations
with fellow researchers
under pressure’
59,1 17,5 23,5
I agree
I don’t agree
neutral
The Flemish survey also shows that young
researchers – doctoral students, doctors and
post-docs – are exposed to higher pressure
to publish than professors. “In a way, that
makes sense: their job security and career
depend even more heavily on the number of
publications they can boast”, says Tijdink.
Lavish with Authorships
An easy way to make your publication list
longer is to hitch a ride on someone else’s
research as a co-author. “Pressure to publish
adds significantly to that temptation”, says
Luca Consoli, assistant professor in Science
and Society at Radboud University in Nijmegen. Our survey shows that it is the order of
the day among medical scientists: 69% admits to having in the past three years added
‘I suspect that in some colleagues publication pressure leads
to (un)intentional data
manipulation ’
‘On a worldwide
scale, publication
pressure causes serious doubts on the
validity of research
64,4 19,7 15,9
I agree
I don’t agree
neutral
61,3 22,9 15,9
I agree
I don’t agree
neutral
the name of at least one author to a study that
that person had not contributed to. And 85%
witnessed colleagues doing it. “Science is collective work,” says Consoli, “so it is not all
that strange for someone to say: you’ve given
me the idea for this paper, so that makes you
a valid co-author.”
“The head of the lab is often put as last author on a paper, while he or she has very
often not even read the paper, or only has a
vague notion of what it is about”, says Consoli. That is what we call honorary authors. A
recent report that looked into six big medical
science journals showed that 25% of the articles mentioned honorary authors. It is often a
win-win game, because for young researchers
adding a big name as honorary author helps
get the paper through the peer review proces,
The Survey
This survey is a first quantitative study of
publication pressure and fraud in Flanders.
We opted for medical science because it
has a strong impact on the public weal. In
November and December 2012 we asked
the deans of the Medical Science Faculties
in Flanders to mail an extensive anonymous
survey to their academic staff. Two weeks
later a reminder was sent.
The survey consisted of 14 proposition that
measured the level of publication pressure
experienced (on a Likert scale from 1 to 5),
and a questionnaire that assesses the level
of fraud and other dubious practices recently carried out by the scientists themselves
and by people in their direct surroundings.
The questions were based on surveys on
scientific literature done before. From the
2,548 medical scientists that we e-mailed
we received 484 (19%) surveys, 315 (12%)
of which had been completely filled in. The
survey was sent to the researchers by the
universities themselves (which perhaps lowered the feeling of anonymity).
55.6% of the respondents was affiliated to
a university hospital. Almost equally many
men (54%) and women (46%) took part.
We included all types of researchers: assistants in training (3%, 9), doctoral students (21%, 66), doctors/post-docs (45%,
141) and professors (31%, 99). 32.1% of
the researchers were from University of
Leuven, almost as many from University of
Ghent (30.5%), University of Antwerpen accounted for 19.7%, Free University of Brussels for 7,6% and University of Hasselt for
4.1%.
Reinout Verbeke and Joeri Tijdink (VU University
MC Amsterdam) are preparing a scientific paper of
this survey.
Eos 28
‘The urge to publish
makes science ill ’
52,4 21,9 25,7
I agree
I don’t agree
neutral
in which a number of colleagues assess the
research for publication in a specialist journal. The heads of the lab, in turn, can benefit
from a longer list of publications for fundraising purposes. To discourage authors from
hitching, more and more specialist journals
ask all authors to confirm their authorship in
writing and state what their share was in the
paper in question.
Self-plagiarism – publishing the same research results in different peer reviewed journals – is yet another way to quickly lengthen
your list of publications. 4.7% of the respondents admits to having done it themselves,
40% witnesses colleagues doing it. Double
publishing is reprehensible, but, like undeserved authorships, it is situated in the grey
area of malpractices.
Debate
In our survey Flemish medical researchers
make mention of fraud and dubious research
practices without hesitation. For many aspects their scores are worse than those in
similar surveys in other countries. Has publication pressure and the demand of specialist
journals for positive and spectacular results
become so high that it jeopardises the scientific value of many studies? This survey can
be a starting point to open the debate, and,
where possible, improve or thoroughly revise
the scientific system.
This article was produced with the support
of the Pascal Decroos Fund for Investigative
Journalism.