Editors’ impact on improving the accuracy of references: randomised comparison of
standard practice, brief reminder or educational intervention
Kristina Fister, MD1,2
Ana Marusic, MD, PhD3
Andrew Hutchings, MSc4
Josipa Kern, PhD1
Matko Marusic, MD, PhD3
1 Andrija Stampar School of Public Health, Zagreb, Croatia
2 BMJ, London, UK
3 Croatian Medical Journal, Zagreb University School of Medicine, Zagreb, Croatia
4 Health Services Research Unit, London School of Hygiene & Tropical Medicine, London,
UK
Corresponding author:
Kristina Fister
Roger Robinson editorial registrar
BMJ Editorial
BMA House
Tavistock Square
London WC1H 9JR, UK
Tel: +44 (0)20 7383 6307
Fax: +44 (0)20 7383 6418
E-mail: kfister@bmj.com
Editors’ impact on improving the accuracy of references: randomised comparison of
standard practice, brief reminder or educational intervention
Word count: 308 excluding title, subtitles and table
Objective:
To investigate whether editors’ intervention prompts authors to reduce the number of errors
in references of manuscripts they submit for publication.
Design:
We randomly allocated 75 consecutive manuscripts accepted for publication in a general
medical journal to one of three interventions. The Editor-in-chief returned manuscripts to the
authors for final changes with a covering letter corresponding to the intervention group:
standard practice (prompting authors to acknowledge required changes, with no specific
mention of the references section), brief reminder (standard practice plus a sentence
prompting authors to pay special attention to the accuracy of references) or educational
intervention (standard practice plus a paragraph highlighting the importance of the accuracy
of references and a copy of reference citation formats recommended by the International
Committee of Medical Journal Editors). Blinded to the intervention, errors were classified as
technical (e.g. punctuation) or substantive (e.g. misspelled author’s last name) compared with
the Manuscript Editor's gold standard. Differences in the accuracy and rate of errors in
revised manuscripts between standard practice and the other interventions were examined
using logistic and Poisson regression. We adjusted for ICMJE reference format and errors in
the original manuscript. The manuscripts were the unit of analysis and included as random
effects.
Results:
We considered 2035 pairs of references after excluding 12 references deleted from original
manuscripts and 79 added to revised manuscripts. The percentage of completely accurate
references and references without technical and substantive errors in the original manuscripts
were 14.9%, 30.1% and 42.7% respectively. Small but statistically significant improvements
in completely accurate and technically correct references were observed in the educational
intervention group compared with standard practice (table). These improvements were also
better than in the brief reminder group (p=0.02 and p<0.01 respectively). No statistically
significant differences were observed for improvement in substantive errors.
Conclusion:
Editors’ educational intervention can produce small improvements in the accuracy of
references as supplied by authors.
Number (%) of completely
accurate references
Standard practice
(25 manuscripts, n=720)
Before
After
Brief reminder
(25 manuscripts, n=613)
Before
After
Educational intervention
(25 manuscripts, n=702)
Before
After
109 (15.1)
94 (15.3)
100 (14.2)
139 (19.8)
195 (27.8)
240 (34.2)
308 (43.9)
337 (48.0)
643 (0.92)
583 (0.83)
105 (14.6)
95 (15.5)
Number (%) of references
257 (35.7)
251 (34.9)
161 (26.3)
162 (26.4)
with no technical errors
Number (%) of references
274 (38.1)
283 (39.3)
286 (46.7)
302 (49.3)
with no substantive errors
Total number (number per
reference) of substantive
743 (1.03)
715 (0.99)
484 (0.79)
445 (0.73)
errors
Improvement in accuracy of references: odds ratio (95% confidence interval; p-value)
Completely accurate
reference
3.18 (0.37-27.4; p=0.29)
references
References without technical
reference
1.09 (0.28-4.26; p=0.90)
error
References without
reference
2.75 (0.96-7.91; p=0.06)
substantive error
Improvement in rate of errors: rate ratio (95% confidence interval; p-value)
Substantive errors per
reference
0.86 (0.71-1.03; p=0.09)
reference
19.4 (2.65-142; p<0.01)
4.31 (1.30-14.4; p=0.02)
3.34 (0.72-15.4; p=0.12)
0.85 (0.71-1.01; p=0.06)