M&E Framework for Programmes for Most-at-Risk Populations Regional Workshop on the Monitoring and Evaluation of HIV/AIDS Programs February 14 – 24, 2011 New Delhi, India What is a Concentrated Epidemic? Epidemics are concentrated if transmission is mostly confined to individuals with high-risk behaviors and those in their sexual and drug use networks, and where protecting these groups would protect society more broadly Epidemics are generalized if transmission occurs mainly outside vulnerable groups and would continue despite effective vulnerable group interventions Who are Most-at-Risk Populations (MARP)? These are sub-populations with elevated levels of HIV-related risk behavior and where there is potential for HIV transmission Populations may include: Female sex workers (FSW) Clients of FSW Injecting drug users (IDU) Men who have sex with men (MSM), including male sex workers (MSW) What are HIV-related High-risk Behaviors? Unprotected sex with multiple-partners Unprotected sex with a partner who has multiple partners Unprotected anal sex with a male partner Sharing injecting equipment …occurring in the presence of HIV Purpose of framework A “road map” or logic model for planning appropriate assessment, monitoring and evaluation in concentrated and low-level epidemic settings Can be used to bring stakeholders together to compile and analyze data in order to plan more effective intervention and improve programs Can be used as a capacity-building tool to identify strengths and weaknesses in planning, implementing and using strategic information at national and subnational levels Understanding the problem and potential response: 1. What is the problem? The Answer Describes the nature, magnitude, and course of the overall HIV epidemic and related sub-epidemics Data Sources HIV prevalence from surveillance Estimates of the size of affected MARP populations Proportion of infections in MARP and projected course of the epidemic from modeling HIV Prevalence in IDU and female sex workers 100 Drug Injectors Percent HIV-Positive 90 Female sex workers 80 70 68 64 59 55 60 52 47 50 39 40 30 20 10 9 3 3 8 8 0 Guangxi, Mumbai, India China (2002) (2002) Jakarta, Indonesia (2003) Kathmandu Valley, Nepal (2003) Source: AIDS in Asia: Face the Facts. MAP Report 2004 Bangkok, Thailand (2003) Haiphong, Vietnam (2002) Source: Strategizing Interventions among MSM in the Greater Mekong Sub-region (GMR) CDC-GAP/USAID-RDM/FHI-APD Workshop, 2004 Understanding the problem and potential response: 2. What are the contributing factors? The Answer Describes determinants of HIV infection and contributing factors at the structural, community, and individual level Data Sources Rapid assessments and situation analysis Knowledge, attitude, and behavior surveys; Epidemiologic studies Source: PLACE in Central Asia: A regional strategy to focus AIDS prevention in Almaty and Karaganda, Kazakhstan; Osh, Krygystan; Tashkent, Uzbekistan. 2002 Understanding the problem and potential response: 3. What interventions are effective? The Answer Identifies appropriate and effective interventions to address the problem and the sub-populations affected Data Sources Review of existing information on what interventions are efficacious and effective Operations research, evaluation studies, and other special studies Understanding the problem and potential response: 4. What interventions and resources are needed? The Answer Identifies the specific interventions and resources that are needed to mount an effective and comprehensive response Data and Sources Needs analysis, costing, Response analysis including an assessment of current programming and estimated coverage Monitoring and evaluating the response: 5. What are programs doing and are they doing it right? The Answer Assesses whether project activities are being implemented as planned and assesses the value of what a project or programme has achieved in relation to its planned activities and objectives. Data Sources Routine data from project monitoring systems, special process evaluations including assessments of service quality Yearly Product Retail Sales & Outlets 1,400,000 250,000 Male Condom 1,200,000 Lubricant Sachets Female Condom 1,000,000 Male Condoms 200,000 150,000 800,000 2,255 600,000 1,926 1,896 400,000 1,171 50,000 890 200,000 100,000 - 2002 Source: PSI Burma, 2007. 2003 2004 2005 2006 Lubricants & Female Condoms Outlets Example: Assessing Quality and Use Use of multiple components of a minimum package of interventions by FSW project beneficiaries Percent of FSW Clients 100 Peer Outreach and Education above plus Condoms 80 60 above plus Targeted Media 40 above plus VCT 20 above plus STI 0 Vientiane Yangon Gejiu Source: MEASURE Evaluation & IPSR, 2006 Monitoring and evaluating the response: 6. Are programs being implemented as planned and are they reaching the target population? The Answer Determines whether or not the project is reaching its target population Data Sources Monitoring of program outputs and estimates of coverage based on 1) project records aggregated across partners combined with estimates of population size or 2) surveys. Example: Monitoring Coverage Percentage (%) Exposure to various types of NGO-related information sources about HIV/AIDS among FSWs is high and increasing over time, Terai Highway Districts, Nepal, 1998 – 2002. 100 90 80 70 60 50 40 30 20 10 0 87 84.5 98.3 97.3 93.8 90 78.5 83.3 75.8 58.3 58.5 60.3 69 43.8 24 1998 1999 Mass m edia* Individual-based Com m unication*** Source: FHI Bangkok 2000 2001 2002 Group-based Com m unication** Source: FHI, BSS Monitoring and evaluating the response: 7. Are interventions effective? The Answer Examines program outcomes (e.g. HIV-related risk behaviors) and determines whether or not changes are attributable to interventions Data Sources Outcome evaluation studies with control or comparison groups, operations research, health services research, formative research, and other special studies. Example: Outcome Evaluation The Sonagachi Project of Calcutta Intervention: community-based intervention to increase condom use among sex workers Design: randomized multiple group community trial conducted over a 15 month period Results: Condom use increased by 39% in the intervention group compared to 11% in the control group. Consistent condom use increased by 25% in the intervention group compared to 16% in the control group. Source: Basu, Janaand Rotheram-Borus, 2004 Monitoring and evaluating the response: 8. Are collective efforts impacting the epidemic? The Answer – Assesses whether or not there are changes in HIV-related risk behavior and HIV prevalence in the population targeted by programs, and whether or not these changes are plausibly linked to overall program effort Data Sources – Monitoring outcomes (e.g. risk behavior) and impact (e.g. HIV prevalence) using surveys and surveillance, triangulated analysis Example: Data Triangulation 7000 100 90 6000 80 5000 70 4000 60 50 3000 40 2000 30 20 1000 0 10 1999 2000 2001 2002 489 2750 252 2032 5809 2582 4187 6191 Condoms distributed (x10,000) STI referrals Intervention contacts Consistent Condom Use 80.3 HIV prevalence 33.2 0 HIV prevalence and condom use (Percent) Intervention, referral, condoms (Number) HIV prevalence among brothel-based sex workers declined in Cambodia and condom use increased. Over the same period interventions among sex workers tripled and achieved high coverage Condoms distributed (x10,000) STI referrals Intervention contacts Consistent Condom Use HIV prevalence 89.8 31.5 28.8 Sources: Cambodia BSS 1999 and 2001, HSS 1999, 2000 and 2002, F HI Cambodia monitoring data for women’s health interventions Source: FHI Bangkok Challenges to Collecting Data among MARP Political and/or legal environment may lead to discrimination, threat of prosecution if attention is brought to members of the subpopulation Stigma may keep populations hidden making them hard to reach for programs or M&E Ethical Considerations Follow a “do no harm” approach Maintain the confidentiality of client information Respect privacy and anonymity Informed consent Data collection should go hand-in-hand with programming MEASURE Evaluation is a MEASURE project funded by the U.S. Agency for International Development and implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill in partnership with Futures Group International, ICF Macro, John Snow, Inc., Management Sciences for Health, and Tulane University. Views expressed in this presentation do not necessarily reflect the views of USAID or the U.S. Government. MEASURE Evaluation is the USAID Global Health Bureau's primary vehicle for supporting improvements in monitoring and evaluation in population, health and nutrition worldwide.