Frameworks
1 March 2011
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Session Outline
 Introduction to frameworks
 Conceptual frameworks
 Results Frameworks
 Logical Framework
 Hands on activity: developing frameworks
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
-- Why
frameworks
Designing M&E Frameworks assists in the development of
 Clearly understood program/project goals and measurable, long-term,
short-term, and intermediate objectives
 Clearly defined relationships between program/project inputs,
processes, outputs, and outcomes,
 Understanding between program/project activities and the external
context (environmental factors)
 Sound implementation of programs
 Design sound M&E plans
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Characteristics of frameworks
 All types of M&E frameworks:
 Inter-relate components, levels and directions of action
 Enable understanding of how
 programs influence health outcomes (conceptual
framework)
 programs should operate (logical framework)
 Programs achieve goals (results)
 Facilitate consensus building around a common paradigm
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Types of frameworks
 Different origins
 Many types:
 Conceptual
 Results
 Logical & Logic model
 Specific types
 Results Based Management framework
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Learning Objectives: Conceptual
Framework
 At the end of this session, participants will be able to:
 Understand why and how conceptual frameworks are useful for
understanding programs and planning M&E
 Describe the role of conceptual frameworks in program design, program
evaluation, and evaluation research.
 Describe the components of conceptual frameworks
 Design a conceptual framework to be most useful for M&E planning
 Design a conceptual framework for an intervention program
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
 Conceptual framework
 Research framework
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Group work- 1










Why did we have political revolt in Egypt
Why do we have poverty in province B
Why do we have crime in City Z
Why do we have famine in Country X
How come we have street abortion in South Africa
Why do we have teenage pregnancy in Community B
How come Spain won the FIFA soccer world cup
How come China has a booming economy
How come the people of Thailand are so hospitable
How did Uganda manage to control an epidemic
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
 What are concepts
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
What is happening?
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
What is happening
 World
 Society/community
 Workplace
 hospital
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
When we conceptualise
 Definitions
 What is the meaning of the word or term or idea
 What is the actual use of the word
 What are the boundaries within which a term
operates
 What are the contrary examples of the word
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Concepts
 Enable interpretation of a subject
 Analyse complex subject matter
 Synthesize separate pieces of a subject matter
into a unit
 Perceive similarities and differences
 Enable us to make value judgments
 Extend our knowledge
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
In program design/programming
 What do you need to know
 Why do have to know
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
M&E CONCEPTUAL FRAMEWORKS
Conceptual, or “research”, frameworks are diagrams that
identify and illustrate the relationships among all relevant
systemic, organizational, individual, or other salient
factors that may influence program/project operation and
the successful achievement of program or project goals.
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Purpose of conceptual frameworks
 Provides a perspective for understanding program objectives
within a complete context of relevant factors in a program’s
operating environment
 Clarifies analytical assumptions and their implications for
program possibilities or limitations on success, as well as
measuring and analyzing that degree of success
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Purpose of Conceptual Framework:cont.
 Makes explicit connections among relevant contextual
(environmental) factors and your program
 Helps to clarify the “why” and “how” questions of program operation
and design:
 Assumptions that underlie the chosen activities
 Factors the activities are expected to affect
 Causal linkages leading to achievement of program objectives
 Guides identification of appropriate indicators
 Guides impact analysis
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Conceptual Frameworks
Individual
characteristics
Service
utilization
Program supply
Institutional
capacity
Technical
Monitoring and Evaluation of HIV/AIDS Programs
inputs
Pretoria, South Africa 2011
Healthy
practices
Health
status
Program
sustainability
Proximate Determinants Model for HIV/STI
(Boerma and Weir)
Underlying
Context
Sociocultural
Socioeconomic
Programmatic
Interventions
VCT
STD control
Condom promote
IEC
Proximate
Biological
Partner acquis.
Mixing patterns
Concurrency
Exposure to
infected
Condom use
Concurrent STI
Risky sex
Treatment
Efficacy of
Transmission
Per contact
Treatment
Health
Outcome
HIV
incidence
STI
incidence
Duration of
activity
Socioeconomic Impacts:Socio-Economic Decline (household and national)
-Orphans and Vulnerable Children
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Demog.
Outcome
Health
Impacts:
Morbidity
Mortality
Research
 Concepts –connotations-defines the problem and
constructs for measurement
 Operationalise-denotations- variables
 Framework allows integration of the
program(research) with existing knowledge
( accepted thinking) of the problem.
Concepts are identified and linked to one another
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Research cont..
 Research starts from this conceptual or
theoretical framework
 Basis to Formulate our hypothesis
 Expresses Assumptions
 Testing/Falsification
 This is the basis of deductive approach in
research
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Trying it out!!
 Identify the problem ( concepts)
 Define drivers of the problem
 Define elements of your program( concepts)
 Identify factors(concepts) in your environment
 identify individual factors(concepts)
 Establish relationship between the factors
 Simplify and refine diagram
 Create and present your framework
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
summary
 To show where program fits into wider context
 To clarify assumptions about causal relationships
 To show how program components will operate to influence
outcomes
 To guide identification of indicators
 To guide impact analysis (causal pathways)
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Activity
 Get into your project/program groups
 Develop a conceptual framework for your
intervention
 After 45 minutes, a member of each group
will share the framework with all
participants
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Strategic Framework
 What are strategies
 What are results
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Revisit your conceptual framework
 What is your vision?
 What is your mission?
 Where are your strengths?
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
M&E STRATEGIC (RESULTS) FRAMEWORKS
Strategic/Results frameworks are diagrams that identify
steps, or levels, of results, and illustrate the causal
relationships linking all levels of a program’s objectives.
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Purposes:
 Provides a clarified focus on the causal relationships
that connect incremental achievement of results to the
comprehensive program impact
 Clarifies project/program mechanics and factors’
relationships that suggest ways and means of
objectively measuring the achievement of desired
ends
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Strategic framework
 Results framework
 Diagrams
 Summary of results
 They identify and illustrate the causal
relationships linking all levels of program
strategy.
 Objectives to impacts
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Results Frameworks
 Presents program strategy for achieving specific objective
 Usually presented as diagram,
 Includes objective and intermediate results
 Includes assumptions behind hypothesis (why the
program/strategy is expected to work)
 Both a planning and management tool
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Goals and Objectives
 Goal:
 A broad statement of a desired,
long-term outcome of the program
 Objectives:
 statements of desired, specific, realistic and and
measurable program results
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Source: GAP 2003
Terminology !
Higher
Level
Goal
Second
Level
Strategic
Objective
Third
Level
Intermediate
Result
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Results Framework Example –
PEPFAR funded ART Program
SO: Utilization of ART services
IR-1: Availability of quality
services
IR-1.1: Increase ART sites
IR-2: Demand for services
IR-2.1: Increase knowledge of
ART
IR-1.2: Supply sites with ARVs
IR-2.2: Increase referral from VCT
IR-1.3: Training for providers
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
EXAMPLE: STRATEGIC OR RESULTS FRAMEWORK
SO1: Increased use of VCT or
HIV/AIDS preventive measures
IR1: Availability of
quality services
IR1.1: Information and
services increased
IR1.2: Practitioners’ skills
and knowledge increased
IR1.3: Sustainable effective
management
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
IR2: Demand for
services
IR2.1: Clients or public
knowledge of HIV/AIDS
improved
Hierarchy of Program Dimensions
Goal
Objective
Availability
Access
Quality
Demand
Knowledge
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Attitudes
USAID labelling
Results framework
Strategic objective
SO1
IR1
Intermediate objective
IR1.1
IR1.2
IR1.2.1
IR2
intermediate objective
IR2.1
IR1.2.2
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
IR2.2
IR2.2.1
IR2.3
IR2.2.2
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Results Frameworks: Key Elements
 Goal
 Strategic Objective
 Should be appropriate & realistic
 Balance ambition and accountability
 Intermediate Results
 Smaller goals
 Integral to achieving SO
 Hypothesized cause-effect (based on conceptual theory) linkages
 Flow from one intermediate result or to many
 Move “up” in the results framework
 Critical assumptions
 Relative to the achievement of the SO
 General condition which holds true for SO to be achieved
 Be realistic!
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Results Frameworks: National TB Program
SO1: Increase tuberculosis case detection rate to 70%
IR1: Increased availability
of quality services
IR1.1: Services increased
IR1.2: Practitioners’ skills
and knowledge increased
IR1.3: Improved
Monitoring and Evaluation of HIV/AIDS Programs
program
management
Pretoria,
South Africa 2011
IR2: Increased demand
for quality services
IR2.1: Customer
knowledge of TB improved
IR2.2: Social support for
TB practices increased
Source: MEASURE Evaluation, 2004 (draft)
Results Frameworks
FIVE-YEAR GOAL: Foundation Established for
Reducing Famine Vulnerability, Hunger and Poverty
SO 14: Human capacity and social resiliency increased
IR 14.1: Use of high impact health,
family planning, and nutrition services,
products, and practices increased
IR 14.1.1: Community support
for high impact health
interventions increased
IR 14.1.2: Availability of key
health services and products
improved
IR 14.1.3: Quality of key
health services improved
IR 14.1.4: Health sector
resources and systems
improved
IR 14.2: HIV/AIDS prevalence
reduced and mitigation of the
impact of HIV/AIDS increased
IR 14.2.1: Reduced risk behavior
IR 14.2.2: Mother-to-child
transmission of HIV reduced
IR 14.2.3: Access to care and
treatment for people living with
HIV and AIDS increased
IR 14.2.4: Care and support for
orphans and vulnerable children
expanded
IR 14.2.5: A more supportive
environment for responding to
HIV/AIDS
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
IR 14.3: Use of quality primary
education services enhanced
IR 14.3.1: Community
participation in the management
and delivery of primary education
services strengthened
IR 14.3.2: Planning, mgmt and
monitoring and evaluation for
delivery of primary education
services strengthened
IR 14.3.3: Quality of primary
education improved
IR 14.3.4: Equitable primary
education services strengthened
Source: USAID/Ethiopia Integrated Strategic Plan, February, 2004 (revised draft)
Results Frameworks: basis for
programmatic process
 Agreement within operating unit and donor
expectations
 Selecting appropriate indicators for the M&E
system
 Use performance information for management
decisions
 Analyzing and reporting on program results
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Building a Results Framework
 Group work
 Using your conceptual framework as a basis
 Discuss and draw a strategic framework for your
program
 After 45 minutes, a member of each group
will share the framework with all participants
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Logical frameworks
M&E for HIV/AIDS Programs
LFA
 Stakeholder analysis
 Problem analysis
 Objective analysis
 selection of preferred implementation strategy
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Logical Frameworks-logic matrix
Means of
Verification
Project Description
Performance Indicators
Assumptions
Goal: The broader
development impact to which
the project contributes - at a
national and sectoral level.
Measures of the extent to
which a sustainable
contribution to the goal has
been made. Used during
evaluation.
Sources of
information and
methods used to
collect and report it.
Purpose: The development
outcome expected at the end
of the project. All
components will contribute to
this
Conditions at the end of the
project indicating that the
Purpose has been achieved
and that benefits are
sustainable. Used for project
completion and evaluation.
Sources of
information and
methods used to
collect and report it.
Assumptions
concerning the
purpose/goal
linkage.
Component Objectives:
The expected outcome of
producing each component's
outputs.
Measures of the extent to
which component objectives
have been achieved and lead
to sustainable benefits. Used
during review and evaluation.
Sources of
information and
methods used to
collect and report it.
Assumptions
concerning the
component
objective/purpose
linkage.
Outputs: The direct
measurable results (goods
and services) of the project
which are largely under
project management's control
Measures of the quantity and
quality of outputs and the
timing of their delivery. Used
during monitoring and review.
Sources of
information and
methods used to
collect and report it.
Assumptions
concerning the
output/component
objective linkage.
Activities: The tasks carried
Implementation/work program
out to implement the project
targets. Used during
and deliverand
the identified
monitoring.Programs
Monitoring
Evaluation of HIV/AIDS
outputs. Pretoria, South Africa 2011
Sources of
information and
methods used to
collect and report it.
Assumptions
concerning the
activity/output
linkage.
Logical Frameworks-Log frame
matrix
Present a standardized summary of the project and its logic.
Purposes:
 Summarizes what the project intends to do and how
 Summarizes key assumptions
 Summarizes outputs and outcomes that will be monitored
and evaluated
Other terms used:
 Logframe matrix
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
M&E LOGIC FRAMEWORKS or Logic model
Logical frameworks are diagrams that identify and
illustrate the linear relationships flowing from program
inputs, processes, outputs, and outcomes. Inputs or
resources affect Processes or activities which produce
immediate results or Outputs, ultimately leading to longer
term or broader results, or Outcomes.
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Purposes:
 Provides a streamlined interpretation of planned use of
resources and desired ends
 Clarifies project/program assumptions about linear
relationships between key factors relevant to desired
ends
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Logic Frameworks - Components
 Inputs
 Program resources like staff, curricula, money
 Activities
 What the program does like outreach, training, testing
 Outputs
 Deliverables of program like people trained or tested
 Outcomes
 Program results like change in knowledge, service use
 Impact
 Long term change in health status like decreased HIV incidence
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Source: CDC Evaluation Guidance Handbook at: http://www.cdc.gov/hiv/aboutdhap/perb/guidance/chapter4.htm
Log frame logic model
Inputs
Activities
Outputs
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Outcome
I
M
P
A
C
T
Logic Models: Training
INPUT
PROCESS
OUTPUT
Develop
clinical
training
curriculum
Conduct
training
events
Practitioners
trained in new
clinical
techniques
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
OUTCOME
IMPACT
Increase in
Declining
clients served
morbidity
by (newly) levels in target
trained
population
providers
Logic Model: Portion of model for tuberculosis control relating
to increasing demand for quality services
;
INPUT
PROCESS
OUTPUT
OUTCOME
IMPACT
•Human and
financial
resources to
develop and
print
educational
brochure
•Distribute
brochure to
health
facilities
•Brochure
distributed to
clients of
•Increased
customer
knowledge of
TB transmission
and treatment
•Decreased TB
infection,
morbidity and
mortality
facilities
•Meet with
physicians to
promote
distribution of
brochure
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
•Increased
demand for
quality TB
services
Logic Models
Voluntary Counseling and Testing (VCT)
Problem Statement: HIV infection rates continue to rise, underscoring the importance for people to
know their serostatus, develop personalized risk-reduction strategies, and access care and treatment services.
INPUTS
Financial
Resources
Demand for
Services
PROCESSES
Supervision and
training for VCT
personnel
Provide pre-test
counseling
OUTPUTS
People
know their
HIV status
OUTCOMES
Clients (HIV+
and -) develop
& adhere to
personalized
HIV riskreduction
Infrastructure
Provide HIV testing
VCT MIS**
VCT protocols,
guidelines, and
training
documents**
Counseling
and Testing
Personnel*
Provide post-test
counseling
Refer HIV+ clients
to appropriate
services (PMTCT,
care and support,
treatment)
HIV+
people are
referred to
appropriat
e services
HIV+ Clients
develop &
adhere to
personalized
HIV care,
support, and
treatment plans
Client
records are
available
Continuity of
care is
available
Service
Referral
reports are
Monitoring
of HIV/AIDS Programs
system for and EvaluationComplete
Pretoria,
South
Africa
2011
produced
prevention &
reporting
Tx services**
requirements
Program and
services are
improved
HIV test kits
Maintain patient
records
IMPACTS
Risk
behaviors
decreased
Increase in
care,
prevention, and
treatment
services for
HIV+, HIV-,
and discordant
couples
Health
outcomes of
HIV +
improve
HIV
transmissi
on rates
decreased
HIV
incidence
decreased
HIV
morbidity
&
mortality
decreased
Frameworks for M&E Planning
 Purposes:
 clarifying assumptions, goals, and interrelationships between
factors relevant to the project or program
 defining objectives
 selecting activities
 defining levels of performance and desired results in terms of
planned activities and realistic, objective impacts
 Monitoring and evaluation plans incorporate:
 program managers’ assumptions and objectives, in a given
context
 a schematic design displaying the directional linkages between
key program elements and/or planned results, and other
relevant factors
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
M&E FRAMEWORKS
Designing an M&E framework assists in determining:
Appropriate program elements to measure
Appropriate indicators and data
Appropriate methodology
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011
Summary of Frameworks
Type of
Framework
Brief Description
Program Management
Basis for Monitoring and
Evaluation
Conceptual
Interaction of various
factors
Determine which factors the
program will influence
No. Can help to explain results
Results
Logically linked program
objectives
Shows the causal
relationship between
program objectives
Yes – at the objective level
Logic model
Logically links inputs,
Shows the causal
processes, outputs, and
relationship between inputs
Monitoring
and Evaluation of HIV/AIDS Programs
outcomes,
and the objectives
Pretoria, South Africa 2011
Yes – at all stages of the
program from inputs to
process to outputs to
outcomes/ objectives
References








Measure : A trainers guide to the fundamentals of Monitoring and Evaluation
for population, health, and Nutrition Programs. 2002. Carolina Population
Centre. Chapel Hill. NC.
AusGuide. The Logical Framework Approach.
http://www.ausaid.gov.au/ausguide/ausguidelines/1-1-1.cfm
Bertrand, Jane T., Magnani, Robert J, and Rutenberg, Naomi, 1996.
Evaluating Family Planning Programs, with Adaptations for Reproductive
Health, Chapel Hill, N.C.: The EVALUATION Project.
Global AIDS Program. (2003) Monitoring and Evaluation Capacity Building for
Program Improvement Field Guide, Version 1. U.S. Centers for Disease
Control and Prevention, Atlanta, GA.
Marsh, David. 1999. Results Frameworks & Performance Monitoring. A
Refresher by David Marsh (ppt)
http://www.childsurvival.com/tools/Marsh/sld001.htm
Tsui Amy. 2004. Frameworks (ppt). Presented at the Bill & Melinda Gates
Summer Institute. John Hopkins, Townson. Maryland.
Tsui, Amy. 1999. Frameworks (ppt). Presented at the Summer Institute,
University of North Carolina, Chapel Hill.
Monitoring and Evaluation of HIV/AIDS Programs
Bloom S.Pretoria,
2007 South
Lecture
Africanotes
2011 . Asia HIV M&E Slides 2007
Activity
 Return to small groups from previous activities
 Develop Logical model for one of the activities
of your program.
 After 30 minutes, a member of each group will
share one of the logical frameworks
Monitoring and Evaluation of HIV/AIDS Programs
Pretoria, South Africa 2011