ASSIGNMENT 1 LOGIC MODEL GERO 400 PRESENTED TO DR. SUSAN CRAWFORD PRESENTED BY Gillian Farnsworth (301103354) Vikki Ram (301204983) Chester Sun (973013826) LOGIC MODEL SFU library http://www.lib.sfu.ca/help/subject-guides/gerontology/gero400 Psych Info + Ageline "logic model" 700+ results "logic model + program evaluation" 125 results "logic model + program planning" 36 results Grey Grey Lit lit Community Health Online Digital Archive Research Resource(CHODARR) " logic model" 0 results "logic model program planning" 0 results Google Scholar “Logic model for program planning” “Logic model for program evaluation” 53 results 13 results Google “Logic model for program planning” Logic model for program evaluation 3 million+ results 7 millon+ results Logic models to enhance program performance Ellen Taylor-Powell, Ph.D. Evaluation Specialist University of Wisconsin- Extension-Cooperative Extension “If you don’t know where you are going, how are you gonna’ know when you get there?” Yogi Berra Where are you going? How will you get there? What will show that you’ve arrived? Adapted from University of Wisconsin- Extension-Cooperative Extension 8 Many people say a logic model is a road map Adapted from University of Wisconsin- Extension-Cooperative Extension Logic model may also be called… Theory of change Program action Model of change Conceptual map Outcome map Program logic Logic model is a… Picture of your program or intervention Graphic representation of the “theory of action” – what is invested, what is done, and what results Core of planning and evaluation Provides a common framework for your work Definition LOGIC the principles of reasoning reasonable the relationship of elements to each other and a whole MODEL small object, representing another, often larger object (represents reality, isn’t reality) preliminary pattern serving as a plan tentative description of a system or theory that accounts for all of its known properties The American Heritage Dictionary, 2nd Ed The accountability era What gets measured gets done If you don’t measure results, you can’t tell success from failure If you can’t see success, you can’t reward it If you can’t reward success, you’re probably rewarding failure If you can’t see success, you can’t learn from it If you can’t recognize failure, you can’t correct it. If you can demonstrate results, you can win public support. Re-inventing government, Osborne and Gaebler, 1992 Source: University of Wisconsin- Extension-Cooperative Extension A bit of LM history Developed in 1970s by Carol Weiss and Joseph Wholey Many refinements and variations since added. Logic Model details how an intervention contributes to intended or observed results, distinguishing between short, medium and long term results and between direct and indirect results. The simplest form has four parts Come in as many sizes and shapes as the programs they represent Logic models can be applied to: a small program a process (i.e. a team working together) a large, multi-component program or even to an organization or business Logic model is in widespread use Private Sector Public Sector: GPRA Non-Profit Sector International Arena Evaluators Simplest form INPUTS OUTPUTS Adapted from University of Wisconsin- Extension-Cooperative Extension OUTCOMES Everyday example H U N G R Y Get food Eat food Adapted from University of Wisconsin- Extension-Cooperative Extension Feel better Everyday example H E A D A C H E Situation Get pills INPUTS Take pills Feel better OUTPUTS OUTCOMES Adapted from University of Wisconsin- Extension-Cooperative Extension Example: Every day logic model – Family Vacation Family Members Drive to state park Budget Set up camp Car Camping Equipment Cook, play, talk, laugh, hike Adapted from University of Wisconsin- Extension-Cooperative Extension Family members learn about each other; family bonds; family has a good time Logical chain of connections showing what the program is to accomplish INPUTS OUTPUTS Program investments Activities What we invest What we do Participation OUTCOMES Short Who we reach Adapted from University of Wisconsin- Extension-Cooperative Extension Medium What results Longterm INPUTS Program Investments What we invest OUTPUTS Activities What we do OUTCOMES Participation What we reach Short Medium What results Long Term Example: Smoke free worksites Situation: Secondhand smoke is responsible for lung cancer, respiratory symptoms, cardiovascular disease, and worsens asthma. Public policy change that creates smoke free environments is the best known way to reduce and prevent smoking. Inputs Coalition Time Outputs Assess worksite tobacco policies and practices Worksite owners, managers Outcomes Increased awareness of importance of SF worksites Demonstrations of public support for SF worksites Dollars Partners Including youth Develop community support for SF worksites Organize and implement strategy for targeted worksites Unions Workers; union members Public Increased knowledge of SF worksite benefits & options Increased commitment, support and demand for SF worksites Adapted from University of Wisconsin- Extension-Cooperative Extension SF worksites policies drafted SF worksite policies passed Adherence to smokefree policies SF worksites Connecting outputs to outcomes is a challenge “I think you should be more explicit here in Step Two.” Source: University of Wisconsin- Extension-Cooperative Extension Programs aren’t linear Feedback loops and multi-dimensions INPUTS Program investments What we invest OUTPUTS Activities What we do Participation OUTCOMES Short Who we reach Adapted from University of Wisconsin- Extension-Cooperative Extension Medium What results Longterm Defining the Situation: Critical first step in logic model development What problematic condition exists that demands a programmatic response? ◦ ◦ ◦ ◦ Why does it exist? For whom does it exist? Who has a stake in the problem? What can be changed? If incorrectly understood and diagnosed, everything that flows from it will be wrong. Factors affecting problems: protective factors; risk factors Review research, evidence, knowledge-base Source: University of Wisconsin- Extension-Cooperative Extension Defining the Situation: Critical first step in logic model development Traps: ◦ Assuming we know cause: symptoms vs. root causes. ◦ Framing a problem as a need where need is actually a program or service. “Communities need leadership training” Precludes discussion of nature of the problem: what is the problem? Whose problem? Leads one to value provision of the service as the result – is the service provided or not? Source: University of Wisconsin- Extension-Cooperative Extension Adapted from University of Wisconsin- Extension-Cooperative Extension OUTPUTS What we do ACTIVITIES Train, teach Deliver services Develop products Who we reach PARTICIPATION and resources Network with others Build partnerships Assess Facilitate Work with the media … Participants Clients Customers Agencies Decision makers Policy makers Satisfaction Adapted from University of Wisconsin- Extension-Cooperative Extension OUTCOMES What results for individuals, families, communities..… SHORT MEDIUM LONG-TERM Learning Action Conditions Changes in Changes in Changes in Awareness Knowledge Attitudes Skills Opinion Aspirations Motivation Behavioral intent Behavior Conditions Social (well-being) Health Economic Civic Environmental Decision-making Policies Social CHAIN action OF OUTCOMES Adapted from University of Wisconsin- Extension-Cooperative Extension Intended Results: Outputs (direct products of program) Outcomes (specific changes in behavior): knowledge, skill, status, functioning (short = 1 - 3 yr and long = 7 10 yrs) Impact (intended or not) change, occurring organizations, communities, systems, due to program in Most Logic Models are more complex: - Theory approach (conceptual) - Outcome approach (results) - Activities approach (applied) - No one logic model fits all needs Theory or Conceptual Logic Model Reason is to link ideas to explain underlying assumptions on which a program is based Emphasizes the assumptions or principles: "beginnings" Eg: Ontario Public Health Standards "This approach is intended to help move Ontario's public health system from a focus on processes to a focus on achieving outcomes." Some LM’s for specific public health issues http://www.health.gov.on.ca/en/pro/progra ms/publichealth/oph_standards/introlm.as px http://www.health.gov.on.ca/en/pro/progra ms/publichealth/oph_standards/docs/cdplv .pdf What logic model is not… A theory Reality An evaluation model or method It is a framework for describing the relationships between investments, activities, and results. It provides a common approach for integrating planning, implementation, evaluation and reporting. Source: University of Wisconsin- Extension-Cooperative Extension The Logic Model There are hundreds of examples of templates on Google, just click images and you can see. You can also download templates from websites such as adobe, docstoc and smart draw. Check your logic model 1. Is it meaningful? 2. Does it make sense? 3. Is it doable? 4. Can it be verified? Adapted from University of Wisconsin- Extension-Cooperative Extension Logic Model helps with Evaluation Provides the program description that guides our evaluation process Helps us match evaluation to the program Helps us know what and when to measure Are you interested in process and/or outcomes? Helps us focus on key, important information Prioritize: where will we spend our limited evaluation resources? What do we really need to know?? Logic model in evaluation What do you want to know? How will you know it? EVALUATION: check and verify Adapted from University of Wisconsin- Extension-Cooperative Extension Match evaluation questions to program INPUTS Program investments OUTPUTS Activities Participation OUTCOMES Short Medium Evaluation questions: What questions do you want to answer? e.g., accomplishments at each step; expected causal links; unintended consequences or chains of events set into motion Indicators: What evidence do you need to answer your questions? Adapted from University of Wisconsin- Extension-Cooperative Extension Longterm What do you want to know about your program? Evaluation: What to measure – when? What amount of $ and time were What did the program actually consist of? Who actually participated in what? Did this meet our target? To what extent did knowledge and skills increase? To what extent did practices change? invested? Adapted from University of Wisconsin- Extension-Cooperative Extension To what extent did phosphorus reduce? Savings accrue to farmers? Logic model with indicators for Outputs and Outcomes Outputs Program implemented Number of workshops held Quality of workshops Outcomes Targeted farmers Farmers learn Number and percent of farmers attending Number and percent who increase knowledge Farmers practice new techniques Number and percent who practice new techniques Adapted from University of Wisconsin- Extension-Cooperative Extension Farm profitability increases Number and percent reporting increased profits; amount of increase Typical activity indicators to track Amount of products, services delivered #/type of customers/clients served Timeliness of service provision Accessibility and convenience of service - Location; hours of operation; staff availability Accuracy, adequacy, relevance of assistance Courteousness Customer satisfaction Source: University of Wisconsin- Extension-Cooperative Extension For example: # of clients served # of consultations # of workshops held # of attendees # of referrals Quality of service Methods of data collection Sources Existing of Information Data Collection Methods Survey Interview Test Observation Group techniques Case study Photography Document review Expert or peer review data - Program records, attendance logs, etc - Pictures, charts, maps, pictorial records Program participants Others: key informants, nonparticipants, proponents, critics, staff, collaborators, funders, etc. Source: University of Wisconsin- Extension-Cooperative Extension BENEFITS: Provides a common language Helps us differentiate between “what we do” and “results” --- outcomes Increases understanding about program Guides and helps focus work Leads to improved planning and management Increases intentionality and purpose Provides coherence across complex tasks, diverse environments Enhances team work Guides prioritization and allocation of resources Motivates staff Helps to identify important variables to measure; use evaluation resources wisely Increases resources, opportunities, recognition Supports replication Often is required! Limitations Logic Model… Represents intention, is not reality Focuses on expected outcomes Challenge of causal attribution Many factors influence process and outcomes Doesn’t address: Are we doing the right thing? Cautions: Can become too time consuming – and just paperwork May become too focused on outcomes without adequate attention to inputs and outputs and the logical relationships that connect them to end results May end up perfecting the key to the wrong lock - Is the program focusing on the right thing? Mixing levels within one logic model Attending to context only at front end Thinking that logic model has to be “correct” - Map of Pyrennes vs Alps Becomes ‘fixed’ rather than flexible and dynamic Cautions Logic Model… Represents reality, is not reality Challenge of causal attribution Many factors influence process and outcomes Doesn’t address: Are we doing the right thing? Time Consuming – Paperwork Too much focus on outcomes Too little focus on testing the theory Summing up Demonstrates accountability with focus on outcomes Links activities to results: Prevents mismatches Integrates planning, implementation, evaluation and reporting Creates understanding Promotes learning A way of thinking – not just a pretty graphic Logic modeling is a way of thinking… not just a pretty graphic “We build the road and the road builds us.” -Sri Lankan saying Source: University of Wisconsin- Extension-Cooperative Extension REFERENCES AND RESOURCES Center for Disease Control and Prevention, Washington, DC www.cdc.gov/eval/resources/index.htm Demystifying the Logic Model by T. Gale of Western Virginia University Extension Services http://www.pawv.org/conferences/logicmodel.pdf Isaacs, B., Clark, C., Correia, S., & Flannery, J. (2009). Utility of logic models to plan quality of life outcome evaluations. Journal Of Policy And Practice In Intellectual Disabilities, 6(1), 52-61. doi:10.1111/j.1741- 1130.2008.00197.x Kellogg Institute http://www.wkkf.org/knowledge-center/resources/2006/02/wkkellogg-foundation-logic-model-development-guide.aspx Nathan Garber (*scroll down for list of resources) http://garberconsulting.com/Program_Logic_Model.htm Ontario Ministry of Health http://www.phac-aspc.gc.ca/phppsp/pdf/toolkit/logic_model_e.pdf Program evaluation models and related theories: AMEE Guide No. 67 http://informahealthcare.com/doi/abs/10.3109/0142159X.2012.66863 7 Region of Waterloo http://chd.region.waterloo.on.ca/en/partnersProfessionals/resourc es/PROGRAM_LOGIC_MODEL.pdf University of Idaho Extension http://www.uiweb.uidaho.edu/extension/LogicModel.pdf University of Toronto, The Health Promotion Centre toolkit http://www.thcu.ca/infoandresources/publications/logicmodel.wk bk.v6.1.full.aug27.pdf University of Wisconsin Extension Services http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.htm l LOGIC MODEL - Templates University of Wisconsin - Extension: http://www.uwex.edu/ces/pdande/evaluation/evallogicmodelworksheets.html Ontario Project for Inter Clinic Community Organizing: http://www.opicco.org/?q=Logic_model_templates Canadian International Development Agency: http://www.acdi-cida.gc.ca/acdi-cida/ACDI-CIDA.nsf/En/ANN-923155220-RB9