OHTAC decision determinants: systematic review Ann-Sylvia Brooker, MSc, PhD Joanna Bielecki, MISt Murray Krahn, MD, MSc April 14, 2015 Saskatoon, SK Research Objective and Research Questions: The objective of the literature review is to examine the decisionmaking methodologies applicable to the health technology appraisal process (decision-making process). Research Questions : • ***What decision criteria are used during the decision-making process? • What methods are used to evaluate the decision criteria (e.g. assigned weights, ranked, rated)? • What decision-making methods are used to integrate these criteria in order to develop recommendations regarding funding decisions concerning health technologies (e.g. are decision rules used?). Inclusion Criteria • Information from national and international health technology organizations and insurance agencies making policy recommendations or funding coverage decisions concerning the technology. • Published research describing the decision-making methods of HTA organizations. • What is • What should be • Also – criteria for resource allocation Exclusion Criteria • Primary research evaluating the effectiveness of specific aspects of the decision-making process. • Theoretical/methodological papers that discussed only one decision criteria (e.g. HTA and ethics) These articles were forwarded for consideration to the relevant OHTAC sub committees . • Information regarding HTA agency operations. • Information regarding sources of evidence. • Literature that described the details of evaluating a technology. Literature Search • Search in published academic literature; between February 2007 and March 2013. • Reference list of published articles. • HTA agency website search. • An email was sent to every member of INA-HTA re: decision criteria or decision-making processes. Results • 1479 abstracts were scrutinized from academic literature. • 18 members of INAHTA responded to the email. The response rate was 32% (18/56). 4/18 (22%) did not have relevant documentation in English or French. Another 5/18 (28%) responded that their agency had an advisory role only. However, 4 organizations had relevant documentation. • Results • 26 documents are included in this review. • Documents from US, Canada, Alberta, Ghana, Germany, Australia, South Africa, Singapore, Chile, UK, USA, Netherlands, Brazil, Scotland, New Zealand, England. (English speaking) Author/Year Title of Publication Duclos et al., 2012 Developing evidence-based immunization recommendations and GRADE Goetghebeur et al., 2012 Bridging Health Technology Assessment (HTA) and Efficient Health Care Decision Making with Multicriteria Decision Analysis (MCDA): Applying the EVIDEM Framework to Medicines Appraisal Decision Criteria -Epidemiologic features of the Disease -Clinical characteristics of the targeted disease -Vaccine and immunization characteristics -Economic considerations -Health-system considerations -Social impacts -Legal considerations -Ethical considerations -Disease Impact -Context of Intervention -Intervention outcomes -Type of benefit -Economics -Quality of Evidence Evaluation of Criteria For effectiveness: GRADE (however, risk of bias checklist from Cochrane) For economic & costeffective ness: WHO For adverse effects: Cochrane An MCDA estimate for the technology was calculated by combining scores and weights for each criterion. Decision-Making Method The committee makes recommendations by consensus. The weighting exercise is not designed to be prescriptive. The framework is designed to support health care decision making by stimulating reflection and exchange and making the thinking process more explicit. Author/Year Title of Publication Decision Criteria Evaluation of Criteria Decision-Making Method Husereau et al., 2010 Priority setting for health technology assessment at CADTH -Alternatives -Budget impact -Clinical impact -Disease burden -Economic impact -Available Evidence Weights are given as follows: -Alternatives (.081) -Budget impact (.143) -Clinical impact (.258) -Disease burden (.216) -Economic impact (.167) -Available Evidence (.135) Background report prepared by 2 researchers provides scores to these criteria. But, final decision is made by a committee. Jehu-Appiah et al., 2008 Balancing equity and efficiency in health priorities in Ghana: the use of multi-criteria decision analysis -number of potential beneficiairies -severity of disease -cost-effectiveness -poverty reduction -vulnerable population But also social/ethical acceptability Intervention complexity Ranking of criteria as follows (in descending order of importance): 1. Vulnerable populations They suggest a framework that combines quantitative and non-quantitative analytical criteria (e.g. ethical/social responsibility and intervention complexity) and that final approval should occur after a number of elaborations by different advisory panels 2. Cost-effectiveness 3. Severity of disease 4. Number of potential beneficiaries 5. Poverty reduction Additional “nonquantifiable” criteria: -ethical/social acceptability -intervention complexity Author/Year Kreis et al., 2011 Lopert, 2009 Title of Publication From evidence assessments to coverage decisions? The case example of glinides in Germany Evidence-Based Decision-Making within Australia’s Pharmaceutical Benefits Scheme Decision Criteria Evaluation of Criteria -needs -costs -safety and effectiveness -services are: adequate, expedient, cost-effective and, do not exceed what is necessary. -Efficacy -Safety -Quality -Cost-effectiveness -Clinical need -Uncertainty in costeffectiveness -Total cost to PBS -Ability to constrict a restriction -Potential for adverse outcomes -Affordability of drug -Rule of Rescue A drug can be excluded based on inexpediency. First, if the on the basis of relevant studies, a drug is inferior to a comparable therapy option. Or, second, studies demonstrating a benefit are lacking. No fixed weight for these factors. The factors will be of greater or lesser importance in different situations. Decision-Making Method A specific decision making method were not found in this article. An expert committee deliberates on the evidence and provides its recommendation to the Minister. The Minister may veto a positive recommendation HTA agency Country PHARMAC New Zealand Decision Criteria (a) the health needs (b) the particular health needs of Maori and Pacific peoples; (c) the availability and suitability of existing medicines, and related products and related things; (d) clinical benefits/risks; (e) costeffectiveness (f) the budgetary (g) the direct cost to health service users; (h) the Government’s priorities for health funding, (i) other criteria as PHARMAC thinks fit. Evaluation of Criteria Pharmac gives weight to each criterion as PHARMAC considers appropriate. Decision-making method PHARMAC makes decision after reviewing evidence and consulting with public, groups, that may be affected by its proposals, and with expert advisory committees. HTA agency Country Decision Criteria NICE England -ICER -certainty of the ICER estimate -certainty of the health related quality of life measure -benefits beyond those captured in the QALY measure -whether technology is a “life-extending treatment at the end of life” -aspects that relate to nonhealth objectives of the NHS (e.g. whether a substantial proportion of savings/ benefits occur outside the NHS and PSS.) Evaluation of Criteria The appraisal committee uses different decision rules depending on whether the ICER is (1) less that £20,000 per QALY gained (2) in the range of £20,000 to £30,000 per QALY gained, and (3) above a most plausible ICER of £30,000 per QALY gained. Decision-making method The Appraisal Committee does not use a precise maximum acceptable ICER above which a technology would automatically be defined as not cost effective or below which it would. But a stronger case must be made for an ICER (2) in the range of £20,000 to £30,000 per QALY gained and even more for an (3) ICER above a most plausible ICER of £30,000 per QALY gained. Summary of Decision Criteria Decision Criteria Efficacy- Potential benefit of the intervention (mortality, morbidity, PRO) Terms Health benefit, potential health gain in terms of mortality (saving life, life expectancy gains, average lifeyear benefit per patient, prolongation of disease-free survival); morbidity (health benefit, enhanced health outcome, relative advantage, incremental health gain); patient-reported outcomes (quality of life, number of ***** QALYs gained per patient, disability adjusted life years, relative value to patient). Overall gain in quality of care. Health benefits relative to current standard therapy. Safety of the Side (adverse) effects, unintended consequences, safety intervention and tolerability, risks, risk management, harm, risk of *** event, risk of toxicity compared with standard therapy. External impact of Impact on patient’s family, possible harms to others, intervention infectious disease involved, population effect (positive or * negative), herd immunity, public health interest, social impact, social benefit, prevention of ill health, prevention. Need (clinical) Treatment alternatives, comparative intervention limitations (unmet needs), availability of alternative *** treatments, availability of effective alternative treatments, availability of preventative measures, clinical need, emergencies and need. Disease determinants Factors responsible for the persistence of the burden. Disease burdenclinical ** Prevalence of disease, incidence of disease, number of patients, severity of disease, impact of disease/condition on quality of life, number of potential beneficiaries, indirect beneficiaries, Decision criteria - Terms Decision Criteria Terms Quality of evidence (re: effectiveness research) *** Relevance of evidence/genera lizability/ effectiveness in real practice * Availability of evidence, strength of evidence, consistency of findings, quality of data, choice of end points, validity of data, certainty, precision of effect, selection of studies, proof, scientific evidence, time of assessment in technology development, therapy mechanism of action. Relevance of evidence, representativeness of patients (studies vs. real world), representativeness of technology user (e.g. skill of surgeon or health care practitioner in studies vs. real world), representativeness of context (e.g. acute vs long term care; country differences), response rate, patient compliance, level of generalization, effectiveness in real practice, evidence of effectiveness. Decision Criteria Ethics and moral issues *** Vulnerable and needy populations Decision Terms criteria - Terms Consistency with societal values, moral consequences of HTA, ethical implications, rule of rescue. Vulnerable populations (e.g. age, gender, geography, ethnicity, indigenous populations), life extending treatment for end of life, social groups with high risk and/or increased vulnerability, age of ** targeted group, population equity, positive poverty reduction. Human dignity Human integrity and dignity, basic human rights, meets patient’s basic needs. Patient autonomy Patient autonomy, patient preference. (e.g. and patient patient-centered healthcare? Is there patient & preference ** public involvement?) Equity, fairness Equity, fairness, health equity, equality, and justice distributive justice, formal justice, procedural * justice, social justice, addressing health status inequalities at population level, geographical equity, equity of access, timeliness of access. Utility Utility, utilitarianism. Solidarity Solidarity, collectivism, cohesion. Cultural aspects Cultural and religious convictions. Decision Criteria Terms Disease burden-cost ** Cost to treat disease, cost to prevent disease, national cost of the disease/condition to the health care system. Opportunity costs Efficiency / value for money for patient. Opportunity costs to the population. Maximizing impact on health for a given level of resource compared to available alternatives for this patient group (e.g. cost-effectiveness, cost-utility, cost per QALY, cost-effectiveness utility curves, cost consequence analysis.). Could include comparisons of interventions with different objectives (e.g. psycho-therapy vs. pain meds). ***** Quality of evidence (re: efficiency & cost estimates) * Cost per patient. Financial/budget impact-costs of intervention * Financial impactsavings of intervention Costs (benefits) of externalities Uncertainty in QALYs, possible benefit/harms not included in the QALY (i.e. non-health benefits, social benefits) Cost per patient, unit cost. Budget impact, affordability, operating and start-up costs, national medical costs per year, financial impact on government. Cost-savings, national savings in terms of costs of absences per year, savings in terms of medical costs. Costs of externalities such as: impact on patient’s family, possible harms to others, infectious disease involved, population effect (positive or negative), public health interest, social impact, social benefit, prevention of ill health, prevention. Decision Criteria Priorities: national, local level etc. Stakeholder interests and pressures; political aspects. Decision criteria - Terms Terms National priorities, local priorities (does it meet a local health need? public expectations?), international priorities, strategic direction. Advocacy, pressure from patient groups, pressure from physician groups, producer interests, recommendations made by other countries, clinical expert opinions; political pressure, political impact. Decision Criteria Terms Feasibility (at the organizational level and at the system level) System requirements, physical environment, system capacity, local capacity, ability to implement, implementation, organization’s structure, organizational burden, logistics, process, wellorganized, feasibility of delivery; all enablers and barriers to diffusion within the health system infrastructure (operational, capital, human resources, legislative, regulatory) including ease of integration into local community, system integration, acceptability. Flexibility, reversibility, revisability, ability to evaluate, provision to revision. *** Flexibility of implementation * Ensuring adequate quality and sustainability of intervention Appropriate use of intervention, appropriateness, appropriate setting/level of service, sustainability, longevity.