Monitoring and Evaluation of
Service Delivery:
Strengthening the Base
“The whole Public Service will be organized around the needs of its customers,
directly accountable to them through guarantees of services which are of the highest
quality, accessible, convenient, easy to use, integrated, responsive, cost effective,
and which assure redress when things go wrong.”
Ministry Paper No. 56; Public Sector Modernization Vision and Strategy 2002-2012
Dwight Uylett,
Principal Director, Standards, Government of Jamaica
A sound Monitoring and Evaluation framework
consists of the following base factors:
– Current knowledge of where we are.
– Structure(s) that will facilitate leadership and requisite
changes.
– Citizen at the centre.
– Awareness of threats to targets and plans to mitigate.
– Measure to manage.
because a sound M&E framework has to be forwardlooking, outcome-oriented, facilitate change and
engender service quality!!
It is within this context, that we’ll present the Caribbean
perspective, looking at successes and learnings.
M&E Emphasis
Traditional emphasis
placed on base
factor
Emphasis as
dictated by
learnings
Pitfalls
Current
knowledge of
where we are.
High
High
M&E framework
focused on ‘what went
wrong’
Structure that
will facilitate
leadership and
requisite
changes.
Low to Medium
High
Leadership seen as a
personal thing; not
something that can
have a structure behind
it.
Citizen at the
centre.
Low
High
A nice to have, but a
structure to really make
it happen lacking.
Awareness of
threats to
targets and
plans to
mitigate.
Low to Medium
High
Measure to
manage.
Low to Medium
‘We hope for the best’
High
IS/IT acquisitions will
solve; ‘just set the
performance indicator’.
Base Factors
• What is the current situation?
•
•
•
•
Who will lead?
Are the citizens involved and how?
What are our threats?
Measure to manage.
Desired State: Service Delivery
Priority
Alignment
through
Strategic and
Corporate Plans
Service
Requirements
Definition
Social
Goal-Setting;
Performance
Assessments
National/
Organizational
Goal-Setting;
Performance
Assessments
Measure
&
Manage
..A Performance Monitoring and Evaluation Unit is being established
in the Cabinet Office to facilitate achievement of the desired state.
Actual State: Service Delivery
Priority
Alignment
through
Strategic and
Corporate Plans
Service
Requirements
Definition
Social GoalSetting;
Performance
Assessments
Traditional focus
National/
Organizational
Goal-Setting;
Performance
Assessments
Measure
&
Manage
Base Factors
• What is the current situation?
• Who will lead?
• Are the citizens involved and how?
• What are our threats?
• Measure and manage?
The Leader
• Authority- where does it reside,
championing
• Competence- recruitment, development,
retention, rotation, engagement
• Capacity- tools, opportunities, leveraging
• Change Agent- the way we work!
The Jamaican Model
Base Factors
• What is the current situation?
• Who will lead?
• Are the citizens involved and how?
• What are our threats?
• Measure and manage
Citizen Involvement: What we have
achieved
• Citizen’s Charters- penetration, integrity
• Impact Assessments- ongoing
• Social Policy Evaluation – engagement,
visibility, integrity
– Community Based Policy Monitoring Process
• Consultation policy and mechanism
- Consultation Feedback: Lessons to Learn
Base Factors
• What is the current situation?
• Who will lead?
• Are the citizens involved and how?
• What are our threats?
• Measure and manage
Risk Management
An Essential Requirement:
A general risk management framework inclusive of identifying current processes, interrelations with other processes and performance of these processes is an essential
requirement.
The depth, breadth and importance of a risk management framework usually results in it
being owned by a high-level Committee (Board sub-component) although a Risk Manager
may be assigned. The Committee approach is also preferable from a Governance
perspective as well.
Proper Governance practices dictate that you have in place a
plan for mitigating high risks!!
WE ARE ACTING NOW!
Sample ‘New Disease’ Risk Matrix
Risk Factor
NEW DISEASE
Sub-Risk
Component
HIGH RISK (7,5)
MEDIUM RISK (3)
Treatment
None available;
research low-keyed
or no immediate
results expected.
Scope
LOW RISK (1,0)
Mitigating Strategy
Risk
Score
Impact
Score
(7,5,3,1,
0)
Service
Risk
Research at
'favourable' testing
status; secondary
treatment
capabilities.
Treatment exists
International
collaboration; technical
working groups;
logistics plan for
treatment
dissemination.
5
5
25
Scope not known
Scope known only
in specified
conditions.
Scope known in all
conditions.
Technical working
groups and field
assessments.
7
5
35
Growth capacity
High; can be spread
through common
activities and
interaction.
Medium; can be
spread only through
localised activities
and interaction;
community and
upwards based.
Low. Only through
isolated activities
and interaction;
household based.
Demographic profiles,
public health
assessment and risk
profile; identification
and monitoring of
growth 'sites'.
5
7
35
Public perception
Seen as a non-issue
by public generally;
no ownership.
Focus/interest
groups concern and
ownership
Full national
concern and
ownership taken.
Participatory
methodologies;
surveys; public debate
and information
sharing.
1
5
5
Govt/Health Sector
Acknowlegement
None.
Technical based.
Acknowledged; lay
person terms.
Open Govt./Reporting
3
5
15
This matrix is a high-level summary but much work has to be
done in identifying and assessing all components of the matrix.
Do not underestimate!
Explaining the Results of the Matrix
-
Red: Scope and Growth Capacity to be controlled as a first step.
-
Yellow: Treatment and Govt. Acknowledgement are 2ndary
controls/steps esp. if first step implementation flawed.
-
Green: not an issue now.
Risk assessment is dynamic and continuous as Factors change!!
The less well managed the risk, the worse the impact. Almost
always in Post reviews, ‘obvious’ factors were not considered
that would have prevented or mitigated negative outcomes,
hence the risk management concept.
Base Factors
•
•
•
•
What is the current situation?
Who will lead?
Are the citizens involved and how?
What are our threats?
• Measure and manage.
Measure to Manage
Target setting and reporting requirements[very popular] necessary but not sufficient.
– How will we feed the reports? Who will do it?
When? Is the data feed in sync with the
reporting need?
– What are we managing against? Are we
managing by personal ‘expertise’? gut
feelings?
Measure to Manage
If we are not able to measure accurately and
appropriately then Service Quality will be an
elusive concept since we will never converge
service consistency and service sufficiency.
-The major challenge is ‘appropriately’ since for
service delivery the definition of satisfaction
changes frequently. This is quite unlike financial
performance measurements, for example, which
are fairly static.
Measure to Manage Hitpoints
Hitpoints
E
A
S
Y
H
A
R
D
Then
Now
Benefit
Data forms
enrichment
Capture bio-data
Capture
performance
data too
Reduces number of forms to complete, and
resources required to monitor each form and
bring data together. MOH looking at now to
improve real-time health service delivery
evaluations.
Increase
service
gateways
New physical
locations
E-service
Increase service gateways; reduce
duplication; reduce overheads.
Customs, Tax Offices, several EAs
Reduce access
points
Multiple stops
One stop
Straight data pull. Currently Tax compliance,
lost driver’s license replacement and food
import/export; strengthen control.
Framework
approaches
Working in Silos;
someone has to pull
data together;
Complex, expensive
ERP IT solns.
Integration;
requires a IT
soln. only in
the most
complex of
cases.
Increase in agility and cycle time
improvements; can get things going faster,
consistent information.
Customer Service Framework
Base Factors
•
•
•
•
•
What is the current situation?
Who will lead?
Are the citizens involved and how?
What are our threats?
Measure to manage.
..THANKS FOR ALLOWING ME THE
OPPORTUNITY TO SHARE..