West Virginia University Demystifying the Logic Model Trace Gale, PhD Extension Specialist & Assistant Professor Community Resources & Economic Development May 2007 This is a really cute story… • Standing in the kitchen, he hears the rising whistle of the family tea kettle wooOOO... • Springing into action, he grabs a rolling pin and mercilessly bashes and smashes the once merry kettle into a useless, shapeless hunk of copper. • His doctor, hearing the fuss, rushes into the kitchen, sees what's happened and asks his friend, "Why'd you wreck our lovely tea kettle?" • The spaceman replies: “Hey, you gotta kill these things when they're small." Get it? Maybe a little context would help… The Train, the Spaceman, and the Tea Kettle… • A spaceman who'd lived all his life on Jupiter came to visit Earth. • Now he'd never laid eyes on a train or the iron rails on which they run. • Standing in the middle of the tracks one day, he heard a distant whistle... WOOOO--ooo---OOOOO! but didn't have a clue as to what it meant or his impending danger. • Predictably, the spaceman was hit -- fortunately it was just a side swipe. He was thrown, head over heels off the tracks, but got off with some minor internal injuries, a few broken bones, and some bruises. • After weeks in the hospital recovering, his doctor invited him to his house for dinner one evening. With the context in place… • Standing in the kitchen, he hears the rising whistle of the family tea kettle wooOOO... • Springing into action, he grabs a rolling pin and mercilessly bashes and smashes the once merry kettle into a useless, shapeless hunk of copper. • His doctor, hearing the fuss, rushes into the kitchen, sees what's happened and asks his friend, "Why'd you wreck our lovely tea kettle?" • The spaceman replies: “Hey, you gotta kill these things when they're small." …does it make more sense? Today’s Discussion The Context Logical Framework Approach The Logic Model Linking it All Together The Context • A logic model is a simple illustration of the logic behind a policy, program, or initiative. • It demonstrates the links between the theoretical assumptions/principles of a program, the program activities/processes, and the outcomes (both short and longterm). • It highlights how results will be monitored and evaluated. The Context • A Logic Model is the culminating document produced through the Logical Framework Approach (LFA). • LFA is a long established activity design methodology used by development agencies and organizations around the world. • LFA is based on a systematic analysis of the situation in place, particularly key problems that exist, and of the options for addressing those problems to produce positive change. Context • The Logical Framework Approach, and its culminating document, the Logic Model evolved into popular use based on the assumption that inadequate planning is a persistent fundamental problem in development. • Planning documents are often specific and clear as to the physical and financial inputs, personnel, activities and expected physical results, but… • Thorough assessment of the overall objectives, the target groups and the external factors which determine success or failure are often lacking… (NORAD, 1990:3) Context • Donor organizations repeatedly urge improved planning and project preparation with more emphasis on monitoring, analysis, and evaluation during the lifetime of projects. • There is ample evidence to show that a modest investment in improved planning usually pays off in terms of better projects and direct savings. (NORAD, 1990:3) The Context Ex/post Evaluation Impact Evaluation Identification Problem Analysis Termination Preparation Feasibility Study Transfer to Normal Administration The Project Program Cycle Monitoring and Reviews Design •LFA underlies the entire project / program cycle. •The Logic Model is a simple graphic representatio n of the master project logic. Appraisal Implementation Intervention Start Organization Negotiations and Agreement Source: Hvidt, 1987 Logical Framework Approach • The Logical Framework Approach is a set of methods for problem analysis which requires: – A minimum degree of consensus between the stakeholders over what the basic problem to be addressed is – The problem to be formulated in such a way that it can be addressed by specified interventions. Logical Framework Approach • The LFA methodology involves four guiding elements: – – – – Problem analysis Stakeholder analysis Objectives analysis Selection of a preferred implementation strategy Hints: • This is an iterative process, not a linear set of prescribed steps. • This analysis should be completed in a participatory process. • Consider a SSA workshop – processes and tools exist - WVU Extension can help! Logical Framework Approach • Problem Analysis: – Identify the existing problems, or negative states, that your development program will address. – Example: » There are no scenic byways in our area: WRONG » Tourists have no reason to come to our area: RIGHT – Establish the cause and effect relationships which result in, and flow from, these problems. Hint: Work hard to get to the root causes, not just the symptoms of the problems. Logical Framework Approach • A main tool for problem analysis is the problem tree. Source: AusAID, 2005 Logical Framework Approach • Stakeholder analysis gives further consideration to: – who is impacted most by these problems, and – the roles and interests of different stakeholders in addressing the problems and reaching solutions. Logical Framework Approach • The main steps in stakeholder analysis include: – Identifying the principal stakeholders (local, regional, national) – Investigating their roles, interests, relative power, and capacity to participate – Identifying the extent of cooperation or conflict in the relationship between stakeholders, and – Interpreting the findings of the analysis and defining how this should be incorporated into activity design. Logical Framework Approach • Analysis of objectives involves reassessing the problems and root causes you have identified and developing related objectives. Logical Framework Approach • Objective trees can help: – Take your problem tree and turn the problem statements into objective statements – While the problem tree shows you cause and effect, the objective tree will show you the means by which your desired ends will be achieved. Logical Framework Approach Example of an Objective Tree Source: AusAID, 2005 Logical Framework Approach • Once the Objective Tree is developed, you should check: – Clarity of the statements – Logic & Reasonability of the linkages – Whether the level of detail is sufficient – the likelihood that the actions of one level will lead to the desired objectives of the next Logical Framework Approach • An analysis of the alternative strategies should be conducted. • This should involve an assessment of the potential benefits and risks associated with the different interventions being identified. Logical Framework Approach • Evaluation criteria for alternative interventions might include: – Benefits to target groups – level of benefits, equity, participation – Sustainability of the benefits – Ability to repair and maintain assets postactivity – Total cost and recurrent cost implications – Financial and economic viability – Technical feasibility – Contribution to institutional strengthening and management capacity building – Environmental impact, and – Compatibility of activity with priorities of organization. The Logic Model Components of the Logic Model 9 2 Priorities Situation 1 Resources / Inputs 7 3 4 Activities Assumptions 5 Outputs 8 Your Planned Work 6 Outcomes Impact External Factors Your Intended Results Evaluation • A picture of how you believe your program is going to work. • Demonstrates the order of activities and how these activities connect to the expected program results. Sources: W.K. Kellogg Foundation, University of Wisconsin Extension Service The Logic Model Components of the Logic Model 9 2 Priorities Situation 1 Resources / Inputs 7 3 4 Activities Assumptions 5 Outputs 8 Your Planned Work 6 Outcomes Impact External Factors Your Intended Results Evaluation • A picture of how you believe your program is going to work. • Demonstrates the order of activities and how these activities connect to the expected program results. Sources: W.K. Kellogg Foundation, University of Wisconsin Extension Service The Logic Model Components of the Logic Model 2 Priorities Situation 1 9 1 Resources / Inputs 7 3 4 Activities Assumptions 5 Outputs 8 Your Planned Work 6 Outcomes Impact External Factors Your Intended Results Evaluation Situation / Priorities involves the context for your project. Includes needs and assets, problems, stakeholder engagement, and related priorities. Sources: W.K. Kellogg Foundation, University of Wisconsin Extension Service The Logic Model Components of the Logic Model 2 Priorities Situation 1 9 2 Resources / Inputs 7 3 4 Activities Assumptions 5 Outputs 8 Your Planned Work 6 Outcomes Impact External Factors Your Intended Results Evaluation Resources / Inputs include human, financial, organizational, and community resources a program has available to direct toward doing the work. Sources: W.K. Kellogg Foundation, University of Wisconsin Extension Service The Logic Model Components of the Logic Model 2 Priorities Situation 1 9 3 Resources / Inputs 7 3 4 Activities Assumptions 5 Outputs 8 Your Planned Work 6 Outcomes Impact External Factors Your Intended Results Evaluation Program Activities are what the program does with the resources. Activities are the processes, tools, events, technology, and actions that are an intentional part of the program implementation. These interventions are used to bring about the intended program changes or results. Sources: W.K. Kellogg Foundation, University of Wisconsin Extension Service The Logic Model Components of the Logic Model 2 Priorities Situation 1 9 4 Resources / Inputs 7 3 4 Activities Assumptions 5 Outputs 8 Your Planned Work 6 Outcomes Impact External Factors Your Intended Results Evaluation Outputs are the direct products of program activities and may include types, levels and targets of services to be delivered by the program. Sources: W.K. Kellogg Foundation, University of Wisconsin Extension Service The Logic Model Components of the Logic Model 2 Priorities Situation 1 9 5 Resources / Inputs 7 3 4 Activities Assumptions 5 Outputs 8 Your Planned Work 6 Outcomes Impact External Factors Your Intended Results Evaluation Outcomes are the specific changes in program participants’ behavior, knowledge, skills, status, and level of functioning. Short-term outcomes should be attainable within 1-3 years, while midtem outcomes should be achievable within 4-6 years. Sources: W.K. Kellogg Foundation, University of Wisconsin Extension Service The Logic Model Components of the Logic Model 2 Priorities Situation 1 9 6 Resources / Inputs 7 3 4 Activities Assumptions 5 Outputs 8 Your Planned Work 6 Outcomes Impact External Factors Your Intended Results Evaluation Impact (sometimes referred to as long-term outcomes) is the fundamental intended or unintended change occurring in organizations, communities, or systems as a result of program activities within 7 – 10 years. Impact often occurs after the conclusion of project funding. Sources: W.K. Kellogg Foundation, University of Wisconsin Extension Service The Logic Model Components of the Logic Model 2 Priorities Situation 1 9 7 Resources / Inputs 7 3 4 Activities Assumptions 5 Outputs 8 Your Planned Work 6 Outcomes Impact External Factors Your Intended Results Evaluation Assumptions are the principles, beliefs, theories, and ideas you hold about why and how the identified change strategies will work in your situation. Sources: W.K. Kellogg Foundation, University of Wisconsin Extension Service The Logic Model Components of the Logic Model 2 Priorities Situation 1 9 8 Resources / Inputs 7 3 4 Activities Assumptions 5 Outputs 8 Your Planned Work 6 Outcomes Impact External Factors Your Intended Results Evaluation External Factors involve the potential barriers and / or supports that are outside the control of your program and might impact the change your program intends to deliver. These often include aspects of power, policy, and environmental flux. Sources: W.K. Kellogg Foundation, University of Wisconsin Extension Service The Logic Model Components of the Logic Model 2 Priorities Situation 1 9 9 Resources / Inputs 7 3 4 Activities Assumptions 5 Outputs 8 Your Planned Work 6 Outcomes Impact External Factors Your Intended Results Evaluation Evaluation underlies the entire logic model by illustrating the audiences, priorities and objectives of your program for each of the key focus areas. These form the foundation for your evaluation questions and indicators. Sources: W.K. Kellogg Foundation, University of Wisconsin Extension Service The Logic Model 1 2 This is the situation. 3 If you have access to them, then you can use them to accomplish your planned activities. 4 5 If you accomplish your planned activities, then you will hopefully deliver the amount of product and/or service that you intended. 6 If these benefits to participants are achieved, then certain changes in organizations, communities, or systems might be expected to occur. 7 Certain assumptions have influenced the plan of work. 8 Certain external factors will influence the intended results. 9 Evaluation will help this program analyze and interpret results and enable informed and logical program evolution. Certain resources are needed to operate your program. How to Read a Logic Model If you accomplish your planned activities to the extent you intended, then your participants will benefit in certain ways. The Logic Model 9 2 Priorities Situation 1 3 Resources / Inputs 7 5 4 Activities Assumptions Outputs 8 Your Planned Work 6 Outcomes Impact External Factors Your Intended Results Evaluation Source: W.K. Kellogg Foundation The Logic Model Is a Logic Model an Action Plan? NO. • An action plan is a manager’s guide for running the project. • It shows the program objectives and a timeline or task outline of what must be done to implement the project • Having an action plan increases the probability that the project will be implemented in an effective manner. • A logic model shows the presumed effects of the project. • These hypotheses are described in the logic model, tested in a theory-based evaluation, and lead to lessons learned. • Having a Logic Model increases the probability that the project will lead to further learning. The Logic Model • Who should use a project’s Logic Model? All parties involved in the initiative: – Initiating organization’s board members & top administrators – Initiative leaders & staff – Participating organizations – Evaluators – Prospective & Existing Funders – Legislators – Stakeholders – Others seeking to understand work Linking it All Together Problem Analysis Stakeholder Analysis Situation Needs & Assets Cause & Effect Underlying Constraints Assumptions Priorities Inputs Activities Outputs Outcomes External Factors Resources / Inputs Opportunity Analysis Strategic Alternative Analysis Activities Outputs Outcomes Impact Assumptions External Factors Situation Priorities Linking it All Together Opportunity Analysis Impact Mid-term Outcomes Short-term Outcomes Resources, Activities & Outputs Source: AusAID, 2005 Linking it All Together Situation Priorities 1 One sentence summarizing the strategic situation addressed by your project. Should consider needs, assets, symptoms versus problems, and stakeholder engagement. Should reflect priorities and relate to the intended impact. 2 Resources / Inputs What is invested: Human, financial, Organizational, & community resources directed toward the project. 7 3 4 6 Activities Outputs Outcomes Impact Processes, tools, events, interventions, technology, and actions that are an intentional part of your program. Direct products of program activities: participation, clients, partners, satisfaction, reach, $ savings or earned. Short-term (1-3 yrs): Long-term Outcomes (7-10 yrs): Changes to Social, Economic, Civic, or Environmental Conditions Assumptions Key assumptions which underlie your planned work (may change over time). 9 5 8 learning, awareness, knowledge, attitudes, skills, opinions, aspirations, motivations Mid-term (4-6 yrs): Action, Behavior, Practice, Decisionmaking, Policies, Social Action External Factors Primary external factors (beyond your control) which may affect your expected results (Could be positive or negative effects). Evaluation One to two sentences which explain what the focus of evaluation will be, how, what, and when you will collect data, how and when you will analyze data, how and when you will interpret results (indicators), and how and when you will report. Linking it All Together • Writing Good Outcomes – – – – – Specific Measurable Attainable Results-Oriented Timed Examples: • Families participating in the Family Resource Center will increase their use of community resources and services within one year of joining. • Four school boards will adopt policies to improve student nutrition and physical activity by December 2005. • Information provided by the Health Report Card is used to make community health decisions. Linking it All Together – An Example Situation Priorities 1 Funder requires grantees to include a logic model in funding request; grantees have limited understanding of logic models and are unable to fulfill the funding requirement. 2 Resources / Inputs • • • • • Trainer Budget Equipment Research Base Training Curriculum 7 3 Activities • 3-hour training program • Interactive Activities • Group Work • Practice • Q&A Assumptions • Logic models improve planning, results, and evaluation. 9 4 5 6 Outputs Outcomes Impact • Participation by 25 potential grantees • Increased knowledge of logic models • Increased ability to create a meaningful logic model of a program • Increased confidence in using logic models • Use of logic models in project planning and evaluation • Improved planning – programs achieve more positive results • Improved evaluation – more credible and useful data 8 External Factors Weather, Participant Availability, Turnover, Capabilities Evaluation Evaluation will focus on measuring changes in logic model knowledge, changes in confidence levels, and the ability to create meaningful program logic models. Data will be collected prior to the training, in the form of a questionnaire and three months after the training session, in a follow-up questionnaire. Additionally, logic models submitted by grantees will be evaluated for evidence of improved planning skills and reporting will be required by grantees in terms of their use of logic models throughout the duration of projects and the affect this tool has had on project success. Linking it All Together • Strengths of the Logic Model Approach: – Provide a logical link between means and ends. – Place activity within broader development environment. – Encourage examination of risks. – Require an analysis of whether objectives are measurable – Help establish a monitoring and evaluation framework. – Link problem analysis to objective setting. – Visually accessible and relatively easy to understand. – Can be applied in a participatory way. Linking it All Together • Common Problems of the Logic Model Assessment and Approach – Getting consensus on objectives – Reducing objectives to a simple linear chain – Inappropriate level of detail (too much or too little) – Finding measurable indicators for higher level objectives and ‘social’ projects – Establishing unrealistic targets too early – Prepared too late and mechanistically – Problem analysis and objective setting not always linked – Risks marginalized. – High demands for training and judgment Linking it All Together • Possible dangers of the Logic Model Approach: – – – – – – – – – – – Oversimplification of objective. Objectives become too rigid (blueprint). Ignoring unintended effects. Hides disagreement. Downgrading of less quantified objectives Rigid targets Information overload The same fixed format applied in all cases Used for top-down control Can alienate staff Becomes a fetish rather than a help. Today’s Discussion – A Review The Context Logical Framework Approach The Logic Model Linking it All Together With the context in place… The Logic Model 2 Priorities Situation 1 9 3 Resources / Inputs 7 4 Activities Assumptions 5 Outputs 8 Your Planned Work 6 Outcomes Impact External Factors Your Intended Results Evaluation …does it make more sense? References AusAID. (2005). The Logical Framework Approach. Australia: Australian Agency for International Development. Baker, J.; Goeke, M.L.; Greenway, M.T.; Gregson, R.; Julian, D.; Lewis, K.; Peterson, E.; Pritchard, K.; Terry, B.L.; Von Brock, A.; Watanabe, D.; & Weinschrott, D. (1999). Achieving and Measuring Community Outcomes: Challenges, Issues, Some Approaches. Washington D.C.: United Way of America. Hinchcliff, G. (2004). Logic Model Development. Silver Spring, MD: Coastal Learning Services – NOAA Coastal Learning Center. Hvidt, M. (1987). Project Aid: The Project as a Medium for Transfer of Aid to Rural Areas of the Third World. University of Aarhus, 1987. IFAD (2002). Managing for Impact in Rural Development: A Guide for Project M&E – Annotated Example of a Project Logframe Matrix. Rome: International Fund for Agricultural Development. McCawley, P.F. (2001). The Logic Model for Program Planning and Evaluation. University of Idaho Extension. McNamara, C. (1997). Guidelines and Framework for Designing Basic Logic Model. Authenticity Consulting, LLC. Mikkelsen, B. (2005). Methods for Development Work and Research: A New Guide for Practicioners. 2nd Ed. New Delhi: SAGE Publications. Mosse, R.; Sontheimer, L.E.; & Rogers, P. (1996). Performance Monitoring Indicators: A handbook for task managers. World Bank: Operations Policy Department. Washington D.C. NORAD. (1999). The Logical Framework Approach (LFA). Handbook for Objectives-oriented Project Planning. Oslo. Parsons, B.A. (1999). Making Logic Models More Systemic: An Activity. Boulder, CO: InSites. Schmitz, C.C. & Parsons, B.A. (1999). Everything You Wanted to Know About Logic Models But Were Afraid to Ask. Minneapolis: MN: Professional Evaluation Services & Boulder, CO: InSites. Taylor-Powell, E. (2003). Logic models to enhance program performance. University of Wisconsin Extension. W.K. Kellogg Foundation. (2004). Using Logic Models to Bring Together Planning, Evaluation, and Action: Logic Model Development Guide. Battle Creek, MI. Zimmerman, B.; Lindberg, C.; & Plsek, P. (1998). Edgeware: Lessons from Complexity Science for Health Care Leaders. Dallas, TX: VHA Inc. Zimmerman, B.; Lindberg, C.; & Plsek, P. (2001). Alternatives to Logic Models: A brief introduction to Complex Adaptive Systems. Irvine, TX: VHA Inc. Questions can be directed to the presenter at: TEGale@mail.wvu.edu Appendix Contents: A Logic Model Template 2 Priorities Situation 1 Resources / Inputs 7 9 3 4 Activities Assumptions 5 Outputs 8 6 Outcomes External Factors Evaluation Impact