Monitoring, Analyzing & Evaluating HRH
Policies in the Region of the Americas
Lima, Peru
November 14‐16, 2012
Overview
•
•
•
•
•
•
•
Background & Context
Caribbean Case Study
A Regional Collaborative Approach The Process of HRH Analysis & Evaluation
Challenges & Lessons Learned
First Steps: A Regional Example
Moving Forward
Background & Context
• Second Regional Goals Measurement
• Develop Regional HRH policy monitoring and evaluation models
• While progress has been made a number of HRH goals may not be reached by 2015
• Aging populations, aging workforces
• Reduction in country resources and capacities
• Out migration, rural distribution, slow growth
Regional HRH Forecast
Laying the Foundation
REGIONAL HRH POLICY RTR
MONITORING & EVALUATION
Road Map Implementation, 2012‐17
ROAD MAP IMPLEMENTATION, 2012‐17
High Level Meeting, Caribbean HRH Road Map (2012)
Kampala Declaration (2008) CARICOM CCH III (2008)
Twenty Regional Goals for Health (2007)
Global Health Alliance (2007) HRH Data Management Project (2007)
WHO World Health Report (2006) Health Agenda for Americas (2007)
Joint Learning Initiative (2004) Toronto Call to Action (2005) Joint Learning Caribbean Case Study
• 20 HRH Goals Review
• Roadmap for Strengthening the Caribbean Health Workforce
• Interview Survey and Inventory on HRH Governance
Caribbean 20 HRH Goals Summary
120
Percentage of Goals achieved by Region 100
High
4 ‐ RNs to MDs
9 ‐ Recruit PHC
80
1 – Supply
3 – PHC teams
12 ‐ Foreign cred.
16 ‐ Labour negot.
20 ‐ Accreditation
60
40
Low
6 – Urban/rural gap
11 ‐ Self‐sufficiency
13 ‐ Secure posts
20
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Challenge 1 Challenge 2 Challenge 3 Challenge 4 Challenge 5
Goal 1 – HRH density ratio of 25 per 10,000
CARIBBEAN HRH
REGIONAL GOALS
SUMMARY
CHALLENGE ONE
95%
Goal 5 ‐ Establish a HRH unit
53%
60%
Goal 2 ‐ Proportion of PHC physicians will exceed 40% of the total medical workforce
88%
Goal 4 ‐ Ratio of Nurses: 100%
Physicians is at least 1:1
Goal 3 ‐ Develop Primary Health Care teams with broad competencies
Caribbean HRH Roadmap
The primary thrust of the Road Map is to strengthen HRH planning, management and provider skills in the region in order to create a more stable, efficient and effective health workforce that can meet the needs of an aging population with a growing incidence of chronic diseases by broadening regional access to PHC and public health services.
Caribbean HRH Road Map Context
National
Regional
HRH ROAD MAP
Health Information Service
Financing Systems Delivery
Caribbean Health Workforce:
Renewal Process
Retain
Renew
7
Re-recruit
8
6
Re-vitalize
1
5
4
Re-deploy
3
Re-invest
2
Re-educate
Re-employ
Strengthening the Foundation
First Steps: Enhancing Workforce Capacity
EDUCATION
& TRAINING
GOVERNANCE
Caribbean
Health
Work
Force
HRH Capacity & Skills Development
EDUCATION
E
D
U
TRAINING
HRH Planners
‐ Legislation & regulation
‐ Policies & programs
‐ Data systems & monitoring
‐ Communications & partnerships
‐ Research & evaluation
‐ Evidence‐based planning
‐ Funding Health
‐
H R H
Care Providers
Personnel Managers ‐ Primary health care
‐ Health, safety & security
INFORMATION
SUPPORT
‐ Labour relations
‐ Assessment & deployment
‐ Professional development
‐ Performance management
‐ Succession planning
‐ Incentives & compensation
‐ Recruitment & retention
‐ Public health
‐ Chronic care
‐ Health promotion
‐ Disease prevention
‐ Community‐based
‐ Integrated teams
‐ Task‐shifting
‐ Competency‐based
deployment
GOVERNANCE
G
O
E
R
MANAGEMENT
N
A
N
C
SERVICE E
PROVISION
Road Map – The Journey
ROAD MAP FOR STRENGTHENING THE CARIBBEAN HEALTH WORKFORCE, 2012‐2017 Context Components/Milestones Outcomes POPULATION HEALTH ISSUES STABILITY of Health Workforce HRH Health HRH Learning Database Epidemiology EFFICIENCY of Service Observatory Network Aging Population Delivery Child Health HRH ISSUES Mental Health EFFECTIVENESS of Education Health Outcomes HRH Shortages Poverty/Education Rural Supply Violence/Crime GOVERNANCE L Growing Needs EDUCATION & Attrition Develop & Implement TRAINING Emigration HRH Plans Health & Safety HRH Policies Training Workplace Productivity Improvements HEALTH SYSTEM HRH Inventories ISSUES Managerial Skills HIS Training Policy Development Staff Management Provider Competence Inefficiency Costs & Capacity Poor Access Service Quality Leadership Management HRH PLANS
Community Based Integrated PHC Teams – Enhanced Skills ‐ Task Shifting – Competency Based Deployment ‐ Needs Based Planning – Health Promotion – Disease Prevention – Chronic Care ENABLERS Legislation Programs Policies Management Funding Planning Partnerships Communications Evaluation/Monitoring Impacts WORKFORCE
Optimal Competent Aligned with Needs Stable Distributed Flexible Accountable HEALTH SYSTEM Efficiency Quality Equity Access Administration Sustainability POPULATION Improved Access Appropriate Access Better Quality Better Informed Better Health 2012 2013 ‐ 2017 2020+ HRH Road Map Final Destination
To achieve and maintain a stable and optimally trained, deployed and distributed health workforce with the capacity to provide responsive, timely, efficient and effective health care services to meet the identified health needs of the population in a way that is flexible and innovative as well as affordable and sustainable.
HRH Planning & Management Survey & Inventory
OBJECTIVES
In support of the Caribbean HRH Roadmap implementation process:
Short Term
•To determine the role and scope of ‐ and capacity for ‐ HRH planning and management of Ministries of Health within the Caribbean region;
•To identify legislative and regulatory mechanisms that govern HRH within the countries of the CARICOM;
•To identify current gaps, challenges and areas requiring priority attention;
Medium Term
•To help identify HRH areas that would most benefit from additional skills and training;
•To help identify areas were the country governance mechanisms might be developed or strengthened through regional collaboration; and,
•To support the exploration of regional planning mechanisms to best support national HRH planning activities.
HRH Survey Results: “Single Biggest Challenges”
•
•
•
•
•
•
•
HRH Database
HRH Unit
Redefine roles & skill mix
Capacity & resources
Inability to implement plans
Qualified staff
Program Evaluation
•
•
•
•
Recruitment & retention schemes
Identify training needs
Link HRH to Health plan
Salaries & Working Conditions
Inter‐Ministry HRH Management
•
•
•
Rural distribution
Staff burnout
Chronic vacancies
•
A Regional Collaborative Approach
More effective HRH policies will achieve:
•More stable, optimal workforce
•Greater efficiency in health care delivery
•Improve geographic, professional, intra‐
professional and public‐private distribution
•Enhance quality of care
•Health supportive motivating work environment that maximize productivity
HRH Monitoring & Evaluation
HRH Governance D
E
F
I
N
I
T
I
O
N
"GOVERNANCE is about the rules
that distribute roles and responsibilities among government, providers and beneficiaries and that shape the interactions among them. It encompasses authority, power, and decision‐making in the institutional arenas of civil society, politics, policy, and public administration." Role of Governance
RULES & REGULATIONS
L
E
G
I
S
L
A
T
I
O
N
Standards & Guidelines
AUTHORITY ENGAGEMENT
HRH DECISION‐
PLANNING & MAKING POWER
MANGEMENT
OVERSIGHT
LEADERSHIP Practices & Principles
MANDATED ROLES & RESPONSIBILITIES
P
O
L
I
C
I
E
S HRH Policy Support Framework
Legislation
Regulations
Programs
Strategic Plans
Operational Plans
Partnerships
Communications
KTE
Data Systems
HRH Policy
Finance
Incentives
Education
Leadership
Management
Vision
Mandate
Research
Evaluation
HRH Decision‐Making Arena Program Commitments
Shared/Mandated Responsibilities
Fiscal Space
Allocation of Resources
Stakeholder Interests
Competing Priorities
Sustainability
? ? ? Timing
Long‐Term
Labour Negotiations Party Platform
Strategic Plan Issue Profile
Government Priorities
Other Areas
Other Departments
Deficit(s)
Data / Research / Information / KTE
HRH Policy Development & Implementation
•
•
•
•
•
•
•
•
•
Lack of explicit HRH policies hinders progress
Need comprehensive policies & strategies
Need to be integrated, and pro‐active
Involve stakeholders in all components
Long‐term political support critical
Social & cultural environments representation
Assessment of financial implications of policies
Capacity of country to mobilize needed resources
45 of 57 HRH crisis countries HRH poorly implemented
HRH Action Framework (GHWA)
.
Program Evaluation Framework
Logic Model in Evaluation What do you want to know?
How will you know it?
EVALUATION: check and verify
Logic model and common types of evaluation Needs/asset
assessment:
What are the
characteristics, needs,
priorities of target
population?
What are potential
barriers/facilitators?
What is most
appropriate to do?
Process evaluation:
Outcome evaluation:
Impact evaluation:
How is program
implemented?
To what extent are
desired changes
occurring? Goals met?
To what extent can
changes be attributed to
the program?
Who is benefiting/not
benefiting? How?
What are the net
effects?
What seems to work?
Not work?
What are final
consequences?
What are unintended
outcomes?
Is program worth
resources it costs?
Are activities
delivered as
intended? Fidelity of
implementation?
Are participants being
reached as intended?
What are participant
reactions?
Outcome Mapping
• Defines outcomes as changes in partner’s behavior
• Focuses on facilitating change, not causing it
• Recognizes complexities of developmental contexts
• Looks for logical links with outcomes, not attributed
• Requires program staff and partners in all stages
Logic Models – less utility with unequal distribution of power, complexities of the design and terminologies and self‐contained nature that assume causal links between program components are not as viable.
Applied HRH Planning & Evaluation
• Human Resources for Health Action Framework
• Handbook on Monitoring and Evaluation of HRH
• Rapid Assessment of the Effectiveness of the Country Coordination & Facilitation Process
• Evidence‐Informed Policy Network
WHO/GHWA/CCF
Rapid Assessment Lessons Learned
•
•
•
•
•
•
•
•
•
•
Review progress of stakeholder participation
Create Observatories, improve HRH information
Translate strategies into costed operational plans
Enhance professional and financial management
Incentives & working conditions crucial to recruitment
Rural shortages require task shifting /innovative roles
Create a national migration inventory Improve planning capacity of HRH committees Scale‐up stakeholder and government commitment
Sustain collaboration through annual progress meetings
$ $ $
Silcone Paint & Polish
Tire Pressure, Tread & Size
Air and Gas Filters
Gas Octane Level
Gearing
Acceleration
City Driving
Warm‐Ups/Start & Stop
Exhaust & Catalytic Converter
Spark Plugs
Engine Valves & Radiator
Synthetic Oil
Top & Windows Up
Air Conditioning & Power Assists
Altitude, Humidity & Termperature
UK
HRH Regional Goal #6 – Reduce the gap in Urban‐
Rural distribution in health personnel.
Linked Policy Areas –
•Recruitment and Retention Schemes
•Educational preparation
•Return‐in‐Service commitments
•Needs identification/Rural quotas
•Rural salary and benefits incentives
•Working conditions/health and safety
•Immigration/Temporary Workers
•Bursaries
•Task Shifting / Teams
Educational Bursaries (1)
CONTEXT
INVEST ( INPUT)
ACTIVITIES (OUTPUT)
Local HRH Priorities Budget Allocation Defined Targets
20 Goals/Targets Staff Assigned Strategy/Priorities
Health Plan Accountabilities Training
HRH Plan Equipment Communications
MoH Values Office Space Planning Groups Political Priorities Time Budget Plan/Targets
Planning Committees Admin Support Incentives
Selection Criteria
Recruiting Plan
Tracking/Data Support Educational Bursaries (2)
OUTPUT (WHO) OUTCOMES (LEARN) OUTCOMES (ACTION)
#’s
Partners
Awareness Uptake
Planning Groups Competencies Targeted/Approved
University Opinions Public/Private Mix Community Colleges Aspirations Responded/Accepted
Ministry of Health Acceptance Inter‐Professional Mix
Ministry of Education Interest Urban/Rural Ratios
Civil Service Budget Utilized
Finance Student Retention
Health Professions
Geographic Distribution
Students Professional Mix
Educational Bursaries (3)
IMPACT
•Dollars saved
•Health Outcomes improved
•Access to service enhanced
•Improved rural supply
•Improved workforce stability
•Reduced dependence on Migrant Workers/Temporary Staff
•Better working conditions
•Improved teams and staff support
QUALITATIVE AND QUANTIATIVE INDICATORS
Key questions…Next Steps
Are Regional HRH goals being met? If not, why not? How do we know?
Is the rate at which goals are being achieved reasonable and adequate?
Are current policies, practices, processes and partnerships adequate to support reaching the 20 Regional HRH goals?
What is the simplest and most affordable evaluation and monitoring mechanism that:
• Can be implemented in a timely and credible fashion?
•Produce credible and implementable results?
•Be acceptable to and supported by key partners, stakeholders, donors and government decision‐makers?
To monitor progress…
…but with clearly defined goals
To encourage Innovation…
…but with Accountability
To adopt key partnerships…
…but with clear roles and authority
To seek out the best evidence…
…and then Act on it!
HRH Policy Evaluation Process
HRH POLICY EVALUATION : One Option…
HERE’S ANOTHER…