Benchmarks of Fairness for
Health Sector Reform in
Developing Countries: Overview
and Latin American Applications
Norman Daniels
PIH, HSPH
Ndaniels@hsph.harvard.edu
Santiago, Chile, Jan 16, 2004
Historical Development of the
Benchmarks
• 1993 Clinton Task Force
• 1996 Benchmarks of Fairness for Health Care
Reform – Oxford University Press.
• Pilot work in Pakistan, 1997
• 1999-2000 Adaptation: Pakistan, Thailand,
Colombia, Mexico: Daniels, Bryant et al Bulletin
of WHO, June 2000
• 2001-3 Demonstration Phase: Mexico, Portugal,
Pakistan, Thailand; Vietnam Cameroon, Ecuador,
Nicaragua, Guatemala, Chile, Yunnan (China),
Sri Lanka, Bangladesh, Zambia
–The Adapted Benchmarks
–
–
–
–
–
–
–
–
–
1. Intersectoral public health
2. Financial barriers to equitable access
3. Nonfinancial barriers to access
4. Comprehensiveness of benefits, tiering
5. Equitable financing
6.Efficacy,efficiency,quality of health care
7. Administrative efficiency
8. Democratic accountability, empowerment
9. Patient and provider autonomy
Connections to social justice
• Equity
– B1Intersectoral public health, B2-3 Access,
B4Tiering, B5 Financing
• Democratic Accountability
– B8, B9Choice
• Efficiency
– B6 Clinical Efficacy and quality
– B7 Administrative efficiency
Structure of BMs
• B1-9 Main Goals
– Criteria -- Key aspects
• Sub criteria-- main means or elements
• Evidence Base + Evaluation
– Indicators
– Scoring Rules
WHO Framework vs BM
WHO
BM
Scope
Cross national
Nat, subnat
Objective
Current perform
Reform eval
Purpose
Motivate
Deliberate
Product
Index, ranks
Scores
Who uses
National pol mk
Various
Requires
Good info
Info, tr. people
Problems
Inform change?
Subjectivity?
Overlap
Move to reforms
complementary
B1: Intersectoral Public Health
• Degree to which reform increases per cent of
population (differentiated) with: basic nutrition,
adequate housing, clean water, air, worplace
protection, education and health education
(various types), public safety and violence
reduction
• Info infrastructure for monitoring health status
inequities
• Degree reform engages in active intersectoral
effort
B2: financial barriers to access
• Nonformal sector
– Universal access to appropriate basic package
– Drugs
– Medical transport
• Formal Sector Social/Private Insurance
–
–
–
–
Encourages expansion of prepayment
Family coverage
Drug, med transport
Integrate various groups, uniform benefits
B3: Nonfinancial barriers to
access
• Reduction of geographical maldistribution
of facilities, services, personnel, other
• Gender
• Cultural -- language, attitude to disease,
uninformed reliance on traditional
practitioners
• Discrimination -- race, religion, class,
sexual orientation, disease
B6: Efficacy, efficiency and
quality of health care
• Primary health care focus
– Population based, outreach, community participation, integration
with system, incentives, appropriate resource allocation
• Implementation of evidence based practice
– Health policies, public health, therapeutic interventions
• Measures to improve quality
– Regular assessment, accreditation, training
B8: Democratic accountability
and empowerment
• Explicit public detailed procedures for evaluating
services, full public reports
• Explicit deliberative procedures for resource
allocation (accountability for reasonableness)
• Fair grievance procedures, legal, non-legal
• Global budgeting
• Privacy protection
• Enforcement of compliance with rules, laws
• Strengthening civil society (advocacy, debate)
Why is evidence base important?
• Evidence base makes evaluation objective
• Making evaluation objective means:
– Explicit interpretation of criteria
– Explicit rules for assessing whether criteria met and the degree to
which alternatives meet them
• Objectivity provides basis for policy deliberation
– Gives points of disagreement a focus that requires reasons and
evidence
Evidence Base: Components
• Adapted Criteria--convert generic benchmarks into
country-specific tool
– Reflect purpose of application
– Reflect local conditions
• Indicators
– Outcomes
– Process
– revisability
• Scoring rules
– Connect indicators to scale of evaluation
– Specify in advance
Process of selecting indicators
• Clarity about purpose
• Type of criterion determines type of indicator
– Outcomes vs process indicator appropriate
– Standard vs invented for purpose
– Requires clarity about mechanisms of reform
•
•
•
•
Availability of information
Consultation with experts
Final selection in light of tentative scoring rules
Further revision in light of field testing
Scoring Benchmarks
Reform relative to status quo
-5
0
Or use qualitative symbols, --- or +++
+5
Scoring Rules: General Points
• Map indicator results onto ordinal scale of
reform outcomes
• Final selection of indicators should be done
as scoring rules are developed, so
refinements can be made
• Scoring rules should be adopted prior to
data collection to increase objectivity, but
may have to be revised in light of problems
Two approaches to evidence
• Thailand: survey of
• Guatemala,
various groups judging
Cameroon: team
based on discussion of
evaluation based on
evidence
indicators, scoring
rules
• Strengths: range of
views, involvement of • Strengths: clarity
larger groups
about evidence base
for evaluation
• Weakness: vaguer
basis for judgment?
• Weakness: trained
team, narrow input
Guatemala, Ecuador:
Stage 1: Theoretical adaptation
• Conceptualizing public health
– The set of actions implemented through a health care system which
includes personal, collective, environmental and health promotion
interventions. The delivery of services can be through public or
private providers (with public funding) and its design and
evaluation concerns providers, financers (public and private) and
regulators.
• Output:
– Working document with specific version adapted to the context of
Guatemala and Ecuador
Adapted benchmarks
Defined by Daniels et al (2000)
Benchmark I: Intersectorial Public Health
Benchmark II: Financial barriers to
equitable access
Benchmark III: Non financial barriers to
access
Benchmark IV: Comprehensiveness of
benefits and tiering
Benchmark V: Equitable financing
Benchmark VI: Efficacy, efficiency and
quality of care
Benchmark VII: Administrative efficiency
Benchmark VIII: Democratic accountability
and empowerment
Benchmark IX: Patient and provider
autonomy
Adaptation to Public Health
Benchmark I: Intersectorial public health
Benchmark II: Universal access to public health
interventions
Preventive services, Curative services
Social protection against catastrophic illness
Reduction of financial barriers
Reduction non-financial barriers.
Benchmark III: Equitable and sustainable financing
Equity in health financing
Sustainability in public financing
Benchmark IV: Ensuring the delivery of effective
public health services
Technical quality (standard treatment guidelines)
Efficiency (relation between inputs and outputs)
User satisfaction
Benchmark V: Accountability
Social participation, community involvement in the
evaluation and monitoring of inequities in health care
delivery and resource allocation
Stage 2: Data collection and
data analysis tools
• Intervention level: Province/Department
– Decentralization transferred policy-implementing
responsibilities and resources to the sub-national level.
Development of tools and field testing follows from the
provincial to the municipal level.
• Outputs:
– Data collection: questionnaires (quantitative &
qualitative) to assess criteria and indicators for each
benchmark
– Data analysis: index to assess inequities, health
expenditures analysis through proxies (drug
consumption), excel database.
Stage 3: Field testing
• Outputs:
– Data collection tools for benchmarks I to V.
Examples of application
• Starting with an analysis of inequities in the
delivery of basic health care services and
inequities in the distribution of basic
resources.
INDEX OF PRIORITY FOR
HEALTH SERVICE (IPSS)
IPSS= (Ciin-CDxin ) +
Ciin
(Ciap-CDxap )+ (Cips-CDxps ) Va
Ciap
Cips
3
IPSS= Index of priority for health services
Ciin= Ideal coverage for immunization (100%)
CDxin= Immunization coverage for district X
Ciap= Ideal coverage for antenatal care (100%)
CDxap= Antenatal coverage for district X
Cipss=Ideal coverage for supervised deliveries (100%)
CDxps=Coverage of supervised deliveries for district X
Va= Sum of three values
NOTES: The coefficient will go from 0.01 up to 0.99
The higher the value, the higher the priority for the delivery of basic
services to the population
INDEX OF RESOURCES
IR = (GPDx X 0.4 ) + (MDx X 0.3)+ (FDa X 0.3)
GPDa
MDa
FDx
IR= Index of resources
GPDx= per capita expenditure district x
GPDa= District with the highest per capita expenditure
MDx= Medical staff per population for district x
MDa= District with the highest number of medical staff/pop
FDa= District with the highest number of health facilities per
population (district with the lowest number of inhabitants per
health facility)
FDx= health facility per population in district x
Indexes
DISTRICTS
SAN MIGUEL
CUBULCO
GRANADOS
SAN JERONIMO
PURULHA
EL CHOL
RABINAL
SALAMA
IPSS
0.51
0.47
0.38
0.36
0.33
0.33
0.28
0.15
IR
0.29
0.34
0.81
0.38
0.59
0.55
0.47
0.34
IPSS VERSUS IR
0.90
0.80
0.70
0.60
0.50
IPSS
IR
0.40
0.30
0.20
0.10
0.00
SAN MIGUEL
CHICAJ
CUBULCO
GRANADOS
SAN JERONIMO
PURULHA
EL CHOL
RABINAL
SALAMA
Examples of application
• Benchmark II: Universal access to integrated public
health services
• Definition of integrated public health: the delivery of
services related to curative, preventive and health
promotion, as well as services for both, transmittable and
non-transmittable diseases and chronic diseases. An
integrated effort should include some forms of protection
against catastrophic diseases.
CRITERIA
INDICATORS
RESULTS
Access to the curative services % of population receiving the services N/A
included in the basic package of at any of the three subsystems (public,
services
social security and private) with
public funding
Access to preventive services % of population receiving the services N/A
included in the basic package of at any of the three subsystems (public,
services
social security and private) with
public funding
The provision of services aimed % health facilities at the district level
at non-transmittable, chronic offering services for the following
and degenerative diseases
problems: diabetes, hypertension,
cardiovascular diseases, screening
cervical cancer
Actions implemented aimed to
protect the individuals against
the socio-economic
consequences of catastrophic
illnesses
42%
(5 facilities
from a total of
12)
% of health districts or municipalities 0%. This type
that have a catastrophic disease fund of benefit
for their population
does not exist
in the area
CRITERIA
INDICATORS
RESULTS
Reduction
of % health facilities in a given district in which
financial barriers the population contributes with cash or in kind
resources to the delivery of basic health care
services (both curative and preventive)
0% (interviews to
health authorities
100% (focus groups
with community
members)
Reduction of non- •% of health personnel (by category) that speak
financial barriers the local indigenous language
• % of health staff (by category) who are
women
• % of health facilities offering services in a
schedule that is appropriate to the occupation
and schedules of the local population (24 hours
emergency; OPD services offered until late
evening)
• % of first level health facilities that
experienced shortage of basic resources during
last year (equipment, drugs, medical staff)
30% (see table & graph
for distribution)
59% (see table &graph
for distribution)
25% (3 out of 12
facilities)
(pending tabulation)
Instrument #1b: Human Resources (feed analysis of nonfinancial barriers and inequities in the distribution of health
personnel)
PERSONNEL
TOTAL WOMEN SPEAK LANGUAGE
Doctors
12
2
0
Nurses
16
16
2
Auxiliary Nurses
17
17
2
Rural health technicians
9
0
3
Institutional facilitators
4
1
2
Community facilitators
12
4
12
PERSONNEL
Community facilitators
CATEGORY
Instiutional facilitators
Rural health technicians
SPEAK LANGUAGE
WOMEN
TOTAL
Auxiliary Nurses
Nurses
Doctors
0
2
4
6
8
10
NUMBER
12
14
16
18
Lessons learned
• Benchmarks and their potential contribution
to the analysis of inequities
– Start by analyzing inequities in the delivery of
basic health services and inequities in the
distribution of basic resources
– From here the benchmarks can help to explain
the factors that may be related to the observed
inequities
Lessons learned
• Difficulties of transferring concepts into practice
– Identifying and assessing indicators for accountability,
social participation, intersectorial work, etc.
• Limitations related to health information systems
– Existing system collects mainly traditional information
(health service production) and has little flexibility to
introduce new indicators (intersectorial work and others)
Lessons learned
• Skills in research team
– Actors at sub-national levels require skills development
• Qualitative research
– Potential users and data collectors have little experience & skills
for qualitative research
• Planning cycle
– The benchmarks approach seems more useful as an approach that
helps the planning cycle: evaluate existing situation-design
interventions-implement-evaluate. Issues related to equity and
social justice within the health system can be addressed in each of
the stages of the planning cycle.
Ecuador
• Team members:
– 12 people representing the following
institutions: Universidad Nacional de Loja,
PAHO-Ecuador, ALDES, Universidad de
Cuenca, Fundación Eugenio Espejo, Harvard
School of Public Health (USA) Liverpool
School of Tropical Medicine (UK)
Work carried out during the year
2003
• 5 workshops (two days per workshop)
• 9 work-meetings (one day or less)
• Outputs:
– Adapted version to Ecuador of the generic
matrix (Daniels et al 2000) with specific
indicators for each benchmarks’ criteria
– Development of data collection instruments to
assess indicators
Adaptation of generic matrix
• Followed simmilar process to Guatemala
• Exchange of ideas and indicators between
the Guatemalan team and the Ecuadorian
team.
• Adaptation in Ecuador emphasize the
assessment of recent health policies:
national health system law, free MCH
services law
Field application (Jan-April
2004)
• Two provinces: Azuay y Canar
• 25 health facilities (11 MoH 7 social
security; 7 NGO’s; 1 local government.
• In addition, a household survey that will
allow to investigate socio-economic
inequalities and its relation with access to
reproductive health and MCH services.
Expected use of findings (field
application)
• Inform local government health plans
• Inform advocacy groups in Azuay and
Canar
• Field testing of the benchmarks approach as
a tool that can aid the monitoring and
evaluation of health policy implementation
APHA
• Thailand
• Guatemala
• Cameroon
Later
• Zambia--HIV/AIDS
• Yunnan, China-rural
reform
• Ecuador, public health,
comprehensive
• Vietnam-comprehensive
reform
• Pakistan- community use
• Chile, Nicaragua, Sri
Lanka, Nigeria
(ACOSHED), Bangladesh
Plans for Benchmarks
• Research Network for all sites, other efforts
at monitoring reform
• Funding for country level projects using
adapted benchmarks
• Coordination with WHO, regional
organizations of WHO, World Bank,
USAID