See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/47659551
Critique of an Economic Evaluation Using the Drummond Checklist
Article in Applied Health Economics and Health Policy · November 2010
DOI: 10.2165/11584400-000000000-00000 · Source: PubMed
CITATIONS
READS
7
13,688
1 author:
Christopher M Doran
Central Queensland University
223 PUBLICATIONS 3,628 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
Alcohol economics View project
Impact assessment View project
All content following this page was uploaded by Christopher M Doran on 14 April 2016.
The user has requested enhancement of the downloaded file.
This material is the copyright of the original publisher.
Unauthorised copying and distribution is prohibited.
Terms and Conditions for Use of PDF
The provision of PDFs for authors' personal use is subject to the following Terms & Conditions:
The PDF provided is protected by copyright. All rights not specifically granted in these Terms & Conditions are expressly
reserved. Printing and storage is for scholarly research and educational and personal use. Any copyright or other notices
or disclaimers must not be removed, obscured or modified. The PDF may not be posted on an open-access website
(including personal and university sites).
The PDF may be used as follows:
• to make copies of the article for your own personal use, including for your own classroom teaching use (this includes
posting on a closed website for exclusive use by course students);
• to make copies and distribute copies (including through e-mail) of the article to research colleagues, for the personal use
by such colleagues (but not commercially or systematically, e.g. via an e-mail list or list serve);
• to present the article at a meeting or conference and to distribute copies of such paper or article to the delegates
attending the meeting;
• to include the article in full or in part in a thesis or dissertation (provided that this is not to be published commercially).
EDITORIAL
Appl Health Econ Health Policy 2010; 8 (6): 357-359
1175-5652/10/0006-0357/$49.95/0
ª 2010 Adis Data Information BV. All rights reserved.
Critique of an Economic Evaluation Using
the Drummond Checklist
Christopher M. Doran
This material is
the copyright of the
original publisher.
Unauthorised copying
and distribution
is prohibited.
National Drug and Alcohol Research Centre, University of New South Wales, Sydney, New South Wales,
Australia
In this issue of Applied Health Economics and
Health Policy, Sladkevicius et al.[1] report the results of an economic evaluation of establishing a
neonatal screening programme for phenylketonuria (PKU) in Libya. The authors undertook a
cost-effectiveness analysis (CEA) and a return on
investment analysis. Against a background of
limited resources and rational decision making,
economic evaluations have been used more frequently over the past 2 decades as pressures to
control healthcare spending have increased.[2]
The central purpose of an economic evaluation is
to compare the relative value of different interventions in creating better health, longer life or a
return on investment.[3] The results of such evaluations are typically summarized as a ratio, where
the denominator reflects the gain in health from
an intervention (measured as units natural to the
programme at hand, such as life-years gained, for
a CEA or converted to monetary terms for a costbenefit analysis [CBA]), and the numerator reflects the cost of obtaining that health gain. A
variant of CEA, cost-utility analysis (CUA) measures the gain according to health-state preference scores or utility weights, thus providing an
opportunity to measure the quality of life-years
gained (or disability-adjusted life-years averted),
not just the crude number of years.[3]
Given the rise in popularity of the use of economic evaluations, guidelines have been developed
to improve the methodological rigour, and hence
comparability, of each evaluation. Two of the more
commonly cited references are the Washington
Consensus Panel on Cost-Effectiveness in Health
and Medicine[2] and a textbook by Drummond
et al.[3] Both provide recommendations for the
conduct of economic evaluations in order to improve their quality. Drummond et al.[3] developed a
checklist for assessing economic evaluations, which
provides useful guidance for those undertaking
their own study. It is in the general context of these
guidelines that the contribution by Sladkevicius
et al.[1] is critiqued.
1. Was a Well Defined Question Posed
in Answerable Form?
The Libyan healthcare system is faced with
resource constraints, with only 3.5% of total
GDP being allocated to health. Libya’s population is characterized by large families, high
maternal and paternal age and high rates of
consanguineous marriages, which are thought to
increase the prevalence of disabling and expensive inherited metabolic diseases.[1] With the
evident success of neighbouring countries in initiating neonatal screening, primarily for PKU,
which can lead to a patient having a healthy life,
the Libyan Government is considering a neonatal
screening programme for PKU. Against this rationale, the authors’ research question is timely,
relevant and clearly stated as an examination of
the cost effectiveness of establishing a national
neonatal screening programme for PKU in Libya
from a societal perspective.
2. Was a Comprehensive Description of
the Competing Alternatives Given?
The authors provide an excellent overview of
the current management of PKU in Libya and
Doran
358
model the proposed screening intervention on
a protocol produced by the Medical Advisory
Panel of the National Society for Phenylketonuria,[4] following adaptation to reflect the Libyan
infrastructure.
3. Was the Effectiveness of the
Programme Established?
was based on protocol-driven requirements and
discussions with clinicians and screening equipment suppliers. Few details are reported in the
text on the actual quantity of resources utilized,
although some reference is made to quantities in
the tables. Life-years gained were calculated for
both current practice and the proposed intervention using reported life expectancies. Productivity
costs were calculated using the human capital
approach that combines life-years gained with
average annual salary and adjusted for the unemployment rate. The authors also measured transfer
payments by taking into account losses in taxation
and social security contributions from both patients and carers and disability allowances paid to
patients. Although the inclusion of transfer payments is a debatable practice,[2,3] all assumptions
used by the authors are clearly outlined.
This material is
the copyright of the
original publisher.
Unauthorised copying
and distribution
is prohibited.
The authors based their results on a decision
model constructed in Microsoft Excel. The model
was populated with data collected from experts,
families of patients with PKU and secondary data
sources. The primary outcome used in the economic evaluation was the incremental number of
life-years gained. The time-horizon of the model
comprised the shelf life of the screening equipment
and the lifetime of patients with PKU. In a sensitivity analysis, the authors varied, among other
things, the potential incidence of PKU and life
expectancy of treated PKU patients.
4. Were All the Important and Relevant
Costs and Consequences for Each
Alternative Identified?
Sladkevicius et al.[1] adopted a social perspective, which is the gold standard in economic evaluation. They provide a comprehensive overview
of healthcare and social care resources related
to screening and treatment of PKU. However,
the consideration of non-health resource use was
limited to the institutional cost of those PKU
patients with mental complications. The outcome
of interest for the CEA was life-years gained,
while the return on investment analysis was calculated as the ratio of the size of investment to the
size of savings over the 15 years of screening and
over the lifetime of those diagnosed with PKU.
5. Were Costs and Consequences
Measured Accurately in Appropriate
Physical Units?
An understanding of resource utilization relating to current practice was elicited by interviews with experts and families of PKU patients.
Resource use related to the proposed intervention
ª 2010 Adis Data Information BV. All rights reserved.
6. Were Costs and Consequences Valued
Credibly?
A base year of 2007–8 was chosen and all costs
are presented in $US. Unit costs of each resource
item are presented and varied in the sensitivity
analysis. Assumptions underpinning the valuation of outcomes are clear and appropriate.
7. Were Costs and Consequences
Adjusted for Differential Timing?
Costs incurred in the second and subsequent
years were discounted at a rate of 3% per annum.
The text makes no mention of discounting transfer payments or indirect costs; however, table V
indicates that all costs were discounted. Lifeyears were not discounted in the base case but this
assumption was tested in the sensitivity analysis.
8. Was an Incremental Analysis of Costs
and Consequences of Alternatives
Performed?
The authors report both an incremental costeffectiveness ratio of -$US15 500 per life-year
gained (cost savings combined with health benefits) and a return on investment of 1 : 1.9, sugAppl Health Econ Health Policy 2010; 8 (6)
Editorial
gesting that, for every $US1 invested, society
would gain $US1.9.
9. Was Allowance Made for Uncertainty
in the Estimates of Costs and
Consequences?
Over the last decade, improvements in computing technology have enabled extensive testing
of the sensitivity of cost-effectiveness results to
variations in assumptions around the input cost
and epidemiological variables. The authors conducted appropriate and rigorous sensitivity and
uncertainty analysis; however, they do not present results on a cost-effectiveness plane or an
acceptability curve.
359
Second, uncertainty modelling is crucial to
examining the robustness of results to variations
in parameters, model specification and time horizons. In particular, economic and epidemiological modelling is now used extensively to broaden
the time horizon.[5]
In summary, economic evaluation is an acceptable and useful tool to assist decision makers
in allocating resources across competing healthcare interventions. The rise in popularity of the
use of economic evaluations has fostered the
development of guidelines to improve the methodological rigour and comparability of evaluations. It is hoped that better adherence to these
guidelines will improve the calibre of such appraisals and encourage more effective communication
between researchers, clinicians and policy makers.
This material is
the copyright of the
original publisher.
Unauthorised copying
and distribution
is prohibited.
10. Did the Presentation and Discussion
of Study Results Include All Issues of
Concern to Users?
The authors provide a discussion of their findings in the context of study limitations and
existing literature. They also address the contentious area of including transfer payments and
adopting the human capital approach in valuing
lost productivity. The extensive uncertainty analysis lends support to the robustness of results.
The key findings are presented and explained in a
manner that policy makers can comprehend. The
authors also couch the findings in terms of feasibility to policy makers.
By considering the Sladkevicius et al.[1] study
in the context of guidelines for appraising economic evaluation, the intention has been to provide the
reader with an insight into the methodological
challenges underpinning such an analysis. Two
important aspects are worth reflecting on.
First, the perspective adopted for an economic
evaluation needs to reflect the purpose of the research. A social perspective is considered the ideal
since it aims to capture the full extent of resource
use associated with competing healthcare interventions, thus providing a more accurate estimate
of the true economic cost to society.
ª 2010 Adis Data Information BV. All rights reserved.
View publication stats
Acknowledgements
No sources of funding were used to prepare this editorial.
The author has no conflicts of interest that are directly relevant to the content of this editorial.
References
1. Sladkevicius E, Pollitt RJ, Mgadmi A, et al. Cost effectiveness of establishing a neonatal screening programme for
phenylketonuria in Libya. Appl Health Econ Health Policy
2010; 8 (6): 407-20
2. Gold MR, Siegel JE, Russell LB, et al. Cost-effectiveness in
health and medicine. New York: Oxford University Press, 1996
3. Drummond M, O’Brien B, Stoddart G, et al. Methods for the
economic evaluation of health care programmes. Oxford:
Oxford University Press, 1987
4. The National Society for Phenylketonuria (NSPKU). Management of PKU: a consensus document for the diagnosis
and management of children, adolescents and adults with
phenylketonuria. London: NSPKU, 2004 [online]. Available
from URL: http://www.nspku.org/Documents/Management
%20of%20PKU.pdf [Accessed 2009 Jun 17]
5. Liew D, McNeil J, Peeters A, et al. Epidemiological modelling
(including economic modelling) and its role in preventive
drug therapy. Med J Aust 2002; 177: 364-7
Correspondence: Associate Professor Christopher M. Doran,
Health Economist, National Drug and Alcohol Research
Centre, University of New South Wales, Sydney, NSW 2052,
Australia.
E-mail: c.doran@unsw.edu.au
Appl Health Econ Health Policy 2010; 8 (6)