Warwick Evidence
Standard Operating Procedure
for the Use of Abstracts
Warwick Evidence
ID number: WE UA NW V2
Norman Waugh
March 2016
Division of Health Sciences
Warwick Medical School
The University of Warwick
Signed ...................................................................................Date................................
Signed ...................................................................................Date................................
Signed ...................................................................................Date................................
Signed ...................................................................................Date................................
Page 1 of 4
Warwick Evidence Standard Operating Procedure for the Use of Abstracts
WE UA NW V2
(Version 1 Feb 2013)
Contents
1. Introduction.................................................................................................................... 3
2. Problems........................................................................................................................ 3
3. Conclusion ...................................................................................................................... 4
Page 2 of 4
Warwick Evidence Standard Operating Procedure for the Use of Abstracts
WE UA NW V2
(Version 1 Feb 2013)
1. Introduction
We need to consider how best to use conference abstracts for several reasons. One is that many
studies, even RCTs, are never published in full. Lee et al (PLOS Medicine 2008/5/issue 9/e191)
reported that over half of all supporting trials for FDA-approved drugs had not been published 5 years
after approval.
In these cases, we may have no data other than that in the abstracts, unless the protocol has been
published (probably unlikely in the case of Pharma trials) or we get data from an industry submission.
Another reason is that abstracts often appear years before the full publications, so may be the only
evidence available.
There is a monograph from SHTAC on six breast cancer drugs looking at time to full publication of
conference abstracts. Of 41 publications from 18 RCTs, only four publications (from three RCTs)
reported the same outcomes in abstract and full publication. The mean delay was nine months, with a
range of five to 19 months. The SHTAC report also found 11 trials where abstracts had not been
followed by full publication after a range of 3 months to 38 months, with mean of 16.5 months.
2. Problems
1. The term “abstract” covers several possibilities;
 a 250 word abstract in a journal supplement or conference website, and nothing else
 a 250 word abstract but with further detail on a poster or slides
 a 250 word abstract of the results but with a full protocol available
 a set of abstracts with different results – for example one with main results, one with quality
of life, others with cost-effectiveness, side-effects, patient preference. Sometimes it can be
difficult to link abstracts if there can be no authors in common, and no trial registration
number or other way of linking them.
 an abstract that updates previous full publications, and may only add numbers – for example
an 11 year follow-up paper when the 10-year results were published in full.
2. Some abstracts are never published in full, and there is publication bias, in that the more positive
studies are more likely to be published, and to appear earlier. So including old abstracts might counter
publication bias.
3. In some appraisals of very new drugs, most or all published evidence may be from conference
abstracts (and presentations). We may have unpublished papers in industry submissions but be unable
to quote those.
4. We also need to consider;
 the TAR value of a report. In a full TAR we have more time to look for and include abstracts.
In an STA time is limited.
 the quality and quantity of evidence published in full. If there is a lot, we can ignore abstracts
 consistency within reports – i.e. if we include abstracts in a treatment section should we do so
in the screening section? We should be consistent within sections but not necessarily between
sections, because the volume and quality of published evidence may vary. If evidence on
Page 3 of 4
Warwick Evidence Standard Operating Procedure for the Use of Abstracts
WE UA NW V2
(Version 1 Feb 2013)
treatment is sparse but there is a lot on screening, we might include abstracts in the treatment
section but not the screening one.
5. The level of detail means that quality assessment is usually not possible.
6. We could also argue that they are not admissible because they have not been peer reviewed, but
that may over-emphasise the value of peer review.
Some would argue that we should exclude all abstracts on the grounds of paucity of data, inability to
quality assess, and the frequency of changes between abstracts and full publication. However that
would be too extreme an approach.
3. Conclusion
The use of abstracts should be considered on a case by case basis, depending on the factors mentioned
above, and most importantly, whether there are full publications.
Options include;
 Exclude abstracts from meta-analyses but deal with in narrative fashion in Discussion,
perhaps with a section, called “emerging evidence”
 Include in main clinical effectiveness review including meta-analyses, especially if there is a
published protocol.
 Include in clinical effectiveness review only as a sensitivity analysis.
 Write to authors for full data – probably too time-consuming and poor response rate.
 Not using abstracts at all.
Page 4 of 4
Warwick Evidence Standard Operating Procedure for the Use of Abstracts
WE UA NW V2
(Version 1 Feb 2013)