Monitoring and Evaluation:
Frameworks
Learning Objectives
At the end of the session, participants will be able to:
• Identify and differentiate between conceptual
frameworks, results and logical frameworks,
and logic models
• Design goals and objectives for specific intervention
programs
• Design frameworks for specific intervention programs
• Discuss how frameworks are used for M&E planning
Module Outline
• Introduction to frameworks
– Conceptual Frameworks
– Goals and objectives
– Results and Logical Frameworks
– Logic models
• Exercises developing frameworks
• Interface between frameworks and M&E planning
• Work on group project
Why are Frameworks Useful?
Designing frameworks assist developing:
• Clearly understood program/project goals and
measurable, long-term, short-term, and intermediate
objectives
• Clearly defined relationships between
program/project inputs, processes, outputs, and
outcomes, and between program/project activities
and the external context (environmental factors)
• Sound implementation and M&E plans
Types of Frameworks
•
•
•
•
Conceptual frameworks
Results frameworks
Logical frameworks
Logic models
Conceptual Frameworks
(a.k.a., research or theoretical frameworks)
Diagram that identifies and illustrates the relationships between all relevant
systemic, organizational, individual, or other salient factors that may influence
program/project operation and the successful achievement of program or project
goals.
M&E Purpose:
•
•
•
•
•
To show where program fits into wider context
To clarify assumptions about causal relationships
To show how program components will operate to influence outcomes
To guide identification of indicators
To guide impact analysis (causal pathways)
Similar frameworks:
•
Proximate Determinants
Conceptual Frameworks
Individual
characteristics
Service
utilization
Program supply
Technical
inputs
Institutional
capacity
Healthy
practices
Health
status
Program
sustainability
Conceptual Frameworks
National Tuberculosis (TB) Program
External Factors
• Demographic:
TB infection
(age, gender)
• Socio-economic:
(education, occupation)
• Environmental:
(war, weather)
Health Systems
(DOTS)
• Availability
• Access
• Utilization
•Case Detection
Program Factors
• Political commitment
• Donor involvement
• National TB program
TB Knowledge
• Stigma
• Treatable
• Symptoms
TB Morbidity
• Prevalence
• Incidence
• HIV co-infection
• MDR-TB
TB
mortality
Co-morbidity
• HIV
• Malnutrition
• Alcoholism
•Diabetes
Source: MEASURE Evaluation 2004 (draft)
Conceptual Frameworks
Causes of malnutrition in society
Child malnutrition,
death and disability
Inadequate
dietary intake
Insufficient
access to food
Outcomes
Immediate
causes
Disease
Inadequate
maternal & child
care practices
Poor water/sanitation
& inadequate health
services
Quantity & quality of actual
resources - human, economic and
organizational – and the way they
are controlled
Underlying
causes at
household/
family level
Basic
causes at
societal
level
Potential Resources:
environment, technology, people
Source: UNICEF, State of the World’s Children, 1998
Activity
• Get into groups of 4 or 5
• Develop conceptual framework for
utilization of family planning/maternal
and child health services
– Identify factors that influence health service
utilization
– Arrange factors into a conceptual framework
• After 30 minutes, a member of each group
will share the framework with all
participants
Goals and Objectives
• Goal: a broad statement of a desired, long-term
outcome of the program
• Objectives: statements of desired, specific, realistic
and measurable program results
 SMART
• Specific: identifies concrete events or actions that will take place
• Measurable: quantifies the amount of resources, activity, or change
to be expended and achieved
• Appropriate: logically relates to the overall problem statement and
desired effects of the program
• Realistic: Provides a realistic dimension that can be achieved
with the available resources and plans for implementation
• Time-based: specifies a time within which the objective
will be achieved
Source: GAP 2003
Formulating Program Objectives
To
[action]
The
Among
[specify what knowledge, attitudes, skills,
behaviors]
[specific population or segment]
From – to
[from baseline to desired level]
By
or by
[ x percent ]
or to
[specific level]
[time frame]
Goals and Objectives
National AIDS Committee (NAC) 2005 Strategic Plan
• Goal: Prevent the spread of HIV epidemic and
minimise its impact on nation by 2009
• Objective 1: Reduction of HIV prevalence in nation
• Objective 2: Improved health & quality of life of
people infected & affected by HIV/AIDS
• Objective 3: Strengthened capacity of NAC &
stakeholders to respond to the HIV/AIDS epidemic
at all levels through improved research, M&E and
improved management & coordination
Goals and Objectives: Activity
Child malnutrition,
death and disability
Inadequate
dietary intake
Insufficient
access to food
Outcomes
Immediate
causes
Disease
Inadequate
maternal & child
care practices
Poor water/sanitation
& inadequate health
services
Quantity & quality of actual
resources - human, economic and
organizational – and the way they
are controlled
Underlying
causes at
household/
family level
Basic
causes at
societal
level
Potential Resources:
environment, technology, people
Source: UNICEF, State of the World’s Children, 1998
Results Frameworks
Diagrams that identify steps or levels of results and illustrate the
causal relationships linking all levels of a program’s objectives.
Purposes:
• Provides a clarified focus on the causal relationships that
connect incremental achievement of results to the
comprehensive program impact
• Clarifies project/program mechanics and factors’ relationships
that suggest ways and means of objectively measuring the
achievement of desired ends
Other terms used:
• Strategic frameworks
Results Frameworks
Goal: Improved Health Status
and/or Decreased Fertility
Objective: Improved Use of Health/FP Services
and/or Appropriate Practices
IR1: Access/Availability
IR2: Quality
IR3: Sustainability
IR4: Demand
IR1.1:Commodities/Facilities
IR2.2: Provider Performance
IR3.1: Policy
IR4.1: Attitude
IR1.2: Equity
IR2.3: Training/Supervision
IR3.2: Health Care Finance
IR4.2: Knowledge
IR2.4: Information System
IR3.3: Private Sector
IR4.3: Community Support
Source: David Marsh, 1999
Results Frameworks
Tuberculosis (TB) Control Programs
SO1: Increase tuberculosis case detection rate to 70%
IR1: Increased availability
of quality services
IR1.1: Services increased
IR1.2: Practitioners’ skills
and knowledge increased
IR1.3: Improved
program management
IR2: Increased demand
for quality services
IR2.1: Customer
knowledge of TB improved
IR2.2: Social support for
TB practices increased
Source: MEASURE Evaluation, 2004 (draft)
Results Frameworks
Donor/USAID Reproductive Health Program
SO1: Increased Utilization of Family Planning/Reproductive Health Services
IR1 Strengthened sustainability
of FP/RH Program
IR1.1 Improved policy
environment for the provision
of FP/RH services in the public
and private sectors
IR1.2 Strengthened NGO
advocacy for FP program
IR2 Expansion of high quality FP/RH
services in the public and private sectors
IR2.1 Increased availability of
postpartum and postabortion FP services
IR2.2 Increased accurate knowledge
of clients about modern methods
and FP services
IR2.3 Improved job performance of
health providers, trainers, and administrators
Source: USAID/Turkey Performance Monitoring Plan, 1998-2001
Results Frameworks
FIVE-YEAR GOAL: Foundation Established for
Reducing Famine Vulnerability, Hunger and Poverty
SO 14: Human capacity and social resiliency increased
IR 14.1: Use of high impact health,
family planning, and nutrition services,
products, and practices increased
IR 14.1.1: Community support
for high impact health
interventions increased
IR 14.2: HIV/AIDS prevalence
reduced and mitigation of the
impact of HIV/AIDS increased
IR 14.2.1: Reduced risk behavior
IR 14.2.2: Mother-to-child
transmission of HIV reduced
IR 14.3: Use of quality primary
education services enhanced
IR 14.3.1: Community
participation in the management
and delivery of primary education
services strengthened
IR 14.1.2: Availability of key
health services and products
improved
IR 14.2.3: Access to care and
treatment for people living with
HIV and AIDS increased
IR 14.3.2: Planning, mgmt and
monitoring and evaluation for
delivery of primary education
services strengthened
IR 14.1.3: Quality of key
health services improved
IR 14.2.4: Care and support for
orphans and vulnerable children
expanded
IR 14.3.3: Quality of primary
education improved
IR 14.1.4: Health sector
resources and systems
improved
IR 14.2.5: A more supportive
environment for responding to
HIV/AIDS
IR 14.3.4: Equitable primary
education services strengthened
Source: USAID/Ethiopia Integrated Strategic Plan, February, 2004 (revised draft)
Logical Frameworks
Present a standardized summary of the project and its logic.
Purposes:
• Summarizes what the project intends to do and how
• Summarizes key assumptions
• Summarizes outputs and outcomes that will be monitored
and evaluated
Other terms used:
• Logframe matrix
Logical Frameworks
Project Description
Performance Indicators
Means of
Verification
Assumptions
Goal: The broader development
impact to which the project
contributes - at a national and
sectoral level.
Measures of the extent to which a
sustainable contribution to the
goal has been made. Used during
evaluation.
Sources of information
and methods used to
collect and report it.
Purpose: The development
outcome expected at the end of
the project. All components will
contribute to this
Conditions at the end of the
project indicating that the Purpose
has been achieved and that
benefits are sustainable. Used for
project completion and
evaluation.
Sources of information
and methods used to
collect and report it.
Assumptions
concerning the
purpose/goal linkage.
Component Objectives: The
expected outcome of producing
each component's outputs.
Measures of the extent to which
component objectives have been
achieved and lead to sustainable
benefits. Used during review and
evaluation.
Sources of information
and methods used to
collect and report it.
Assumptions
concerning the
component
objective/purpose
linkage.
Outputs: The direct measurable
results (goods and services) of
the project which are largely
under project management's
control
Measures of the quantity and
quality of outputs and the timing
of their delivery. Used during
monitoring and review.
Sources of information
and methods used to
collect and report it.
Assumptions
concerning the
output/component
objective linkage.
Activities: The tasks carried out
to implement the project and
deliver the identified outputs.
Implementation/work program
targets. Used during monitoring.
Sources of information
and methods used to
collect and report it.
Assumptions
concerning the
activity/output
linkage.
Logical Frameworks
Taskforce on Communicable Disease Control
in the Region X: Tuberculosis
GOAL
A. Reduced
burden of TB
to reach
global
average levels
B. Further
development
of multi-drug
resistant TB
(MDR-TB)
prevented
PERFORMANCE
INDICATORS
A. Notification rate
MEANS OF
VERIFICATION
A. Annual notification
reports (surveillance)
B-1. Treatment
outcome
B-2. Prevalence of
Multi-drug
Resistance in “new”
and previously
treated TB patients
B-1. Annual reports on
outcome of treatment
(cohort analysis)
B-2. Periodic reports on
surveillance of antiTB drug resistance
ASSUMPTIONS
- A dual HIV/TB epidemic
causing increase in TB
incidence does not occur
- Control of private
practitioner and
pharmaceutical sectors to
prevent MDR
- Prevalence of resistance to
second line anti-TB drugs
low enough at the outset so
as not to seriously
compromise treatment
success ratio
Logical Frameworks
Taskforce on Communicable Disease Control
in the Barents and Baltic Sea Regions: Tuberculosis
PURPOSE
[Implementing]
cost-effective
measures for the
prevention and
control of TB
operating within
civil and
penitentiary
health services
in the Task
Force area
PERFORMANCE
INDICATORS
1. Coverage of TB programmes
in line with international
recommendations.
2. Proportion of patients
defaulting out of patients
treated.
3. Proportion of previously
treated cases among all
cases.
4. Proportion of patients on
ambulatory treatment out
of all patients treated.
MEANS OF
VERIFICATION
1.
Annual reports
2.
Annual reports
3.
National / local
annual
notification
reports
(surveillance)
4.
Annual record
reviews during
site visits
(consecutive
series of
patients)
ASSUMPTIONS
- Stable political situation,
sustained political commitment
and financing
- Sufficient numbers of
competent health care personnel
in the government sector
Logical Frameworks
Taskforce on Communicable Disease Control
in the Barents and Baltic Sea Regions: Tuberculosis
(only one output and related activities shown here)
OUTPUTS
8. Measures to increase
awareness of TB and its
treatment among all members
of the community developed
and tested
ACTIVITIES
8.1 Identify groups at risk for TB
8.2 Develop advocacy material
suitable for all target groups (not
only risk groups)
8.3 Organize health education
directed at all target groups
8.4 Involve the mass media
PERFORMANCE
INDICATORS
8.1. Number of
pamphlets / posters
printed and
distributed annually
8.2. Awareness of TB
among target groups
INPUTS
MEANS OF
VERIFICATION
8.1. Material produced /
distributed
8.2. KAP or other surveys
(before/after)
MEANS OF
VERIFICATION
Financial management
reports
ASSUMPTIONS
1. Relevant persons motivated to
participate
2. Professional interest, sufficient
financing
3. Target groups interested in (their)
health and able to participate
ASSUMPTIONS
Activity
• Return to small groups from previous activity
• Develop goals and objectives for a program
aimed at increasing utilization of FP/MCH
health services
• Develop a results or logical framework for
this program
• In 45 minutes, a member of each group will
share the goals, objectives and framework
Logic Models
Diagrams that identify and illustrate the linear relationships flowing
from program inputs, processes, outputs, and outcomes. Inputs or
resources affect Processes or activities which produce immediate
results or Outputs, ultimately leading to longer term or broader
results, or Outcomes.
Purposes:
• Provides a streamlined interpretation of planned use of
resources and desired ends
• Clarifies project/program assumptions about linear
relationships between key factors relevant to desired
ends
Other terms used:
• M&E Frameworks, Logical Frameworks
Logic Models: Training
INPUT
PROCESS
OUTPUT
Develop
clinical
training
curriculum
Conduct
training
events
Practitioners
trained in new
clinical
techniques
OUTCOME
IMPACT
Increase in
Declining
clients served
morbidity
by (newly) levels in target
trained
population
providers
Logic Models
Voluntary Counseling and Testing (VCT)
Problem Statement: HIV infection rates continue to rise, underscoring the importance for people to
know their serostatus, develop personalized risk-reduction strategies, and access care and treatment services.
INPUTS
Financial
Resources
Demand for
Services
PROCESSES
Supervision and
training for VCT
personnel
Provide pre-test
counseling
OUTPUTS
People
know their
HIV status
OUTCOMES
Clients (HIV+
and -) develop
& adhere to
personalized
HIV riskreduction
Infrastructure
Provide HIV testing
VCT MIS**
VCT protocols,
guidelines, and
training
documents**
Counseling
and Testing
Personnel*
HIV test kits
Referral
system for
prevention &
Tx services**
Provide post-test
counseling
Refer HIV+ clients
to appropriate
services (PMTCT,
care and support,
treatment)
Maintain patient
records
Complete
reporting
requirements
HIV+
people are
referred to
appropriat
e services
HIV+ Clients
develop &
adhere to
personalized
HIV care,
support, and
treatment plans
Client
records are
available
Continuity of
care is
available
Service
reports are
produced
Program and
services are
improved
IMPACTS
Risk
behaviors
decreased
Increase in
care,
prevention, and
treatment
services for
HIV+, HIV-,
and discordant
couples
Health
outcomes of
HIV +
improve
HIV
transmissi
on rates
decreased
HIV
incidence
decreased
HIV
morbidity
&
mortality
decreased
Results Framework
Tuberculosis (TB) Control Programs
SO1: Increase tuberculosis case detection rate to 70%
IR1: Increased availability
of quality services
IR1.1: Services increased
IR1.2: Practitioners’ skills
and knowledge increased
IR1.3: Improved
program management
IR2: Increased demand
for quality services
IR2.1: Customer
knowledge of TB improved
IR2.2: Social support for
TB practices increased
Source: MEASURE Evaluation, 2004 (draft)
Logic Model
Portion of model for tuberculosis control relating
to increasing demand for quality services
INPUT
PROCESS
OUTPUT
OUTCOME
IMPACT
•Human and
financial
resources to
develop and
print
educational
brochure
•Distribute
brochure to
health
facilities
•Brochure
distributed to
clients of
•Increased
customer
knowledge of
TB transmission
and treatment
•Decreased TB
infection,
morbidity and
mortality
•Meet with
physicians to
promote
distribution of
brochure
facilities
•Increased
demand for
quality TB
services
Activity
• Return to small groups from previous
activities
• Develop logic models for one of the activities
of your program to increase utilization of
FP/MCH health services
• After 20 minutes, a member of each group
will share one of the logical frameworks
Frameworks for M&E Planning
• Purposes:
– clarifying assumptions, goals, and interrelationships
between factors relevant to the project or program
– defining objectives
– selecting activities
– defining levels of performance and desired results in
terms of planned activities and realistic, objective
impacts
• Monitoring and evaluation plans incorporate:
– program managers’ assumptions and objectives, in a
given context
– a schematic design displaying the directional linkages
between key program elements and/or planned results,
and other relevant factors
Summary of Frameworks
Type of
Framework
Brief Description
Program
Management
Basis for Monitoring
and Evaluation
Conceptual
Interaction of various
factors
Determine which
factors the program
will influence
No. Can help to explain
results
Results
Logically linked
program objectives
Shows the causal
relationship between
program objectives
Yes – at the objective
level
Logical
Logically linked
program objectives,
outputs, and activities
Shows the causal
relationship between
activities and
objectives
Yes – at the output and
objective level
Logic model
Logically links inputs,
processes, outputs,
and outcomes,
Shows the causal
relationship between
inputs and the
objectives
Yes – at all stages of the
program from inputs to
process to outputs to
outcomes/ objectives
References
• AusGuide. The Logical Framework Approach.
http://www.ausaid.gov.au/ausguide/ausguidelines/1-1-1.cfm
• Bertrand, Jane T., Magnani, Robert J, and Rutenberg, Naomi, 1996.
Evaluating Family Planning Programs, with Adaptations for Reproductive
Health, Chapel Hill, N.C.: The EVALUATION Project.
• Global AIDS Program. (2003) Monitoring and Evaluation Capacity Building
for Program Improvement Field Guide, Version 1. U.S. Centers for Disease
Control and Prevention, Atlanta, GA.
• Marsh, David. 1999. Results Frameworks & Performance Monitoring. A
Refresher by David Marsh (ppt)
http://www.childsurvival.com/tools/Marsh/sld001.htm
• Tsui, Amy. 1998. Frameworks (ppt). Presented at the Summer Institute,
University of North Carolina, Chapel Hill.
• Tsui, Amy. 1999. Frameworks (ppt). Presented at the Summer Institute,
University of North Carolina, Chapel Hill.
• UNICEF. 1998. State of the World’s Children.
Group Projects
• Form groups for your group projects
• For your project, develop
– Goals and objectives
– Conceptual framework
– One other framework: results, logical or logic model