Community Need……..……Logical Approach………….. Assessment……..Sustainability……….Special Interest Areas urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ Lecture 05: Assessing Your Program Part I Community Need……..……Logical Approach………….. Assessment……..Sustainability……….Special Interest Areas urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom Upcoming- Imagination Library Phone Bank Mitchel Hall rm 16 http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ Meet here and walk over together Community Need……..……Logical Approach………….. Assessment……..Sustainability……….Special Interest Areas urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html Agenda Discuss https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ Welch and Comer ch 3 How we collect data Nominal, ordinal, continuous Validity and reliability DPIL Survey and the Assessment Planning Tool (lab 3) Super Size Me Community Need……..……Logical Approach………….. Assessment……..Sustainability……….Special Interest Areas urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ C.L.A.S.S. urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html Summative: Does it work? Yes or No https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm Does DARE reduce drug use? oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ http://www.sciencephoto.com/media/280069/enlarge http://www.fsg-impact.org/images/upload/From%20Insight%20to%20Action(3).pdf outcomes are specific changes in things like: •attitudes, •behaviors, •knowledge, •skills, status, •or level of functioning expected to result from program activities. (Kellogg p.18) urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ Why Might You Evaluate? urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html •Better Planning •Improved Implementation •Demonstrate Impact •Strategically Allocate Resources https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm http://lh3.ggpht.com/_yP6P-egnidU/TUOB6ySgZeI/AAAAAAAABlY/kqPx1v3rP-M/game_plan%5B2%5D.jpg oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ http://www.google.com/imgres?q=rube+goldberg+machine&um=1&hl=en&biw=1024&bih=537&tbm=isch&tbnid=Gg--Zk9MHVX3cM:&imgrefurl=http://predatorhaven.blogspot.com/2011/01/animated-music-rube-goldberg.html&docid=IIfIk6qyI1Jg9M&imgurl=http://1.bp.blogspot.com/_hpxaqo7FS9c/TUG5nscXapI/AAAAAAAADIE/p20ozd8ejE0/s1600/science%25252B%25252BRube%25252BGoldberg%25252Bmachine.png&w=500&h=280&ei=Wz_eTrGXFabC2wXUsd3sBA&zoom=1&iact=rc&dur=78&sig=104112844510945666901&page=2&tbnh=113&tbnw=201&start=8&ndsp=9&ved=1t:429,r:4,s:8&tx=107&ty=54 urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom We will explore both summative (why you are in business) and formative (how you stay in business), but first some history…. http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ History is important because it gives you a feel for the audiences for evaluation (beyond you running the program better). urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html Word Association… What comes to mind when you think of program evaluation? https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm Peace? Love? Happiness? Drugs? http://en.wikipedia.org/wiki/Hippie oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ urship/Arthur-C-Brooks/e/9780132330763 1960’s http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html Carman, Fredericks, & Introcaso (2008:10) http://www.maxwell.syr.edu/ The Birth of Evaluation During the The War on Poverty – L.B. Johnson – •The Economic Opportunity Act (1964) provided the basis for the Office of Economic Opportunity (OEO), •the Job Corps, •Volunteers in Service to America (VISTA), •Upward Bound, •Head Start, • Legal Services, • the Neighborhood Youth Corps, • the Community Action Program (CAP), •the college Work-Study program, •Neighborhood Development Centers, •small business loan programs, • rural programs, http://en.wikipedia.org/wiki/Lyndon_B._Johnson • migrant worker programs, •remedial education projects, • local health care centers •$11 billion tax cut (Revenue Act of 1964), •the Civil Rights Act (1964), •the Food Stamp Act (1964), •the Elementary and Secondary Education Act (1965), •the Higher Education Act (1965), the Social Security amendments creating Medicare/Medicaid (1965), •the creation of the Department of Housing and Urban Development (1965), •the Voting Rights Act (1965), •the Model Cities Act (1966), •the Fair Housing Act (1968), •several job-training programs, and various Urban Renewalrelated projects. http://www.faculty.virginia.edu/sixties/readings/War%20on%20Poverty%20entry%20Poverty%20Encyclopedia.pdf urship/Arthur-C-Brooks/e/9780132330763 1960’s http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom Trends in U.S. evaluation training programs LaVelle and Donaldson 2010 Academia (knowledge) Government (accountability) http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm http://en.wikipedia.org/wiki/File:Aerial_vie w_of_the_Capitol_Hill.jpg 2009 Social Innovation Fund oices/2008/12/say_yes_program_receives_1_mil.html Carman, Fredericks, & Introcaso (2008:10) http://www.maxwell.syr.edu/ http://www.nationalservice.gov/Default.asp http://www.faculty.virginia.edu/sixties/readings/War%20on%20Poverty%20entry%20Poverty%20Encyclopedia.pdf urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html Consultant Joke • • https://spears.okstate.edu/eee/classroom • • • http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm • • • • • • • • oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ from Erik van de Pol, Visible World: A shepherd was herding his flock in a remote pasture when suddenly a brand-new BMW advanced out of the dust cloud towards him. The driver, a young man in a Broni suit, Gucci shoes, Ray Ban sunglasses and YSL tie, leaned out the window and asked the shepherd,.......... "If I tell you exactly how many sheep you have in your flock, will you give me one?" The shepherd looked at the man, obviously a yuppie, then looked at his peacefully-grazing flock and calmly answered, "Sure." The yuppie parked his car, whipped out his IBM Thinkpad and connected it to a cell phone, then he surfed to a NASA page on the internet where he called up a GPS satellite navigation system, scanned the area, and then opened up a database and an Excel spreadsheet with complex formulas. He sent an email on his Blackberry and, after a few minutes, received a response. Finally, he prints out a 130 page report on his miniaturized printer then turns to the shepherd and says,......... "You have exactly 1586 sheep". "That is correct; take one of the sheep" said the shepherd. He watches the young man select one of the animals and bundle it into his car. Then the shepherd says: "If I can tell you exactly what your business is, will you give me back my animal?" "OK, why not" answered the young man. "Clearly, you are a consultant" said the shepherd. "That's correct" says the yuppie, "but how did you guess that?" "No guessing required" answers the shepherd. "You turned up here although nobody called you. You want to get paid for an answer I already knew, to a question I never asked, and you don't know crap about my business.... Now give me back my dog". http://www.wdolson.com/jokes.htm The role of outside experts and consultants- people see outsiders telling them the same things in a more receptive light urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html • • • • • • • • • https://spears.okstate.edu/eee/classroom • • • • • • • • http://student.maxwell.syr.edu/maxwellnetworking/ • http://www.waveland.com/Titles/Welch-Comer.htm • • • • • • • • • • • • • • oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ Copyright © 2014 NPR. For personal, noncommercial use only. See Terms of Use. For other uses, prior permission required. DAVID GREENE, HOST: There are times when I've thought about singing during the program or maybe telling a bad joke. I'm getting the feeling this morning that our producer and editor, Rachel Ward and Kenya Young, would shoot me down. You ever have this experience? Pitch what you think is a brilliantly creative idea and your boss or manager says nope. If so, it might be worth listening to Steve Inskeep's conversation with NPR's social science correspondent Shankar Vedantam. Shankar's found some research explaining why good ideas get rejected. STEVE INSKEEP, HOST: Hi, Shankar. SHANKAR VEDANTAM, BYLINE: Hi, Steve. INSKEEP: Okay. So what's the research say? VEDANTAM: Well, the research seems to suggest that part of the reason we miss seeing creative ideas that are right under our nose is because the ideas are right under our nose. There's this new research that looks at how people evaluate creativity. Jennifer Mueller at the University of San Diego and her colleagues, Cheryl Waxslack(ph) and Vishwin Athenkrisnan(ph), find that where the idea comes from appears to influence whether people think it's creative. So in this experiment they ran, they told volunteers about a new shoe that uses nano technology to reduce blisters. They told some volunteers that this idea was developed far away and they told other volunteers this idea was developed nearby. Here's Mueller. JENNIFER MUELLER: We found that when we told people the idea was generated far away, they rated the idea as significantly more creative than when the idea was generated nearby. VEDANTAM: You're kidding me. MUELLER: No. VEDANTAM: At a certain level, that doesn't make sense, though, does it? MUELLER: It really doesn't. INSKEEP: Okay. So same idea, it's all about the source. We're talking about how a manager, a boss, would evaluate an idea that's brought to them. VEDANTAM: Exactly. So it seems to happen, Steve, because our minds are prone to mixing these two things up. When things are nearby, they're concrete and you can see the details of the things. On the other hand, when things are far away, they're much more abstract. So thinking about things that are near and far puts us in different mental states. When you think about things nearby, you see the details, and so when a creative idea comes along, the first thing you ask is, can it work? Now, most creative ideas are risky and the risks are obvious when you look at the details, so when you think about it with this detail-oriented mindset, you're more likely to shoot the idea down. On the other hand, when you're thinking about things that are far away, you're in a more abstract frame of mind and so the first question you ask is not will this work; you're more open to seeing the creative possibilities. INSKEEP: So it's not just that as a manager, that the manager disrespects the employees. The manager is just familiar with the employees, he or she works with the employees every day, and they're thinking about the details of it. Whereas somebody comes from the outside, they can think big. VEDANTAM: Exactly. So obviously it has to be said that some ideas are not creative and the deserve to be shot down, but the reason managers often are shooting down ideas that might be creative that come from subordinates is not because they're necessarily bad managers, but they might be in this different mindset. Mueller reminded me of this famous rejection letter that the writer George Orwell got. Famous publisher said, look, Americans are just not interested in political stories about animals. "Animal Farm," of course, went on to become this classic book. Here's Mueller again. MUELLER: It's hard to get executives to say creativity is stupid, even though they may actually feel that way and very strongly. Very rarely will CEOs say renovation's horrible, why would we ever do that? INSKEEP: So wait a minute. I guess then, if I'm an employee trying to sell an idea to my manager, I should call somebody way outside the company, some stranger and tell them the idea, have them walk in the door with it, I gather. VEDANTAM: Well, that's right. So here's the challenge, Steve. So the people who make decisions in organizations and government, they tend to be the people with the most experience. In fact, that's why those people tend to be in charge. And having that experience is very valuable because it helps you spot these big mistakes. What this research is suggesting is that the experience also comes at a price. It makes it hard to recognize out of the box possibilities. So Mueller thinks that just before you pitch your boss on a big idea, it might be useful to get them in a more abstract frame of mind. I'm actually going to do what you just suggested. The next time I come up with a creative idea, I'm going to call it in on the phone and say, I'm calling from Antarctica with this fantastic idea. INSKEEP: So Shankar, when you told your editor about your idea to do this story, what did she say? VEDANTAM: I made sure to do it on the phone, Steve. INSKEEP: Shankar, thanks very much. VEDANTAM: Thank you, Steve. INSKEEP: That's NPR's Shankar Vedantam. You can follow him on Twitter @HiddenBrain. You can also follow this program @MorningEdition and @NPRInskeep. Copyright © 2014 NPR. All rights reserved. No quotes from the materials contained herein may be used in any media without attribution to NPR. This transcript is provided for personal, noncommercial use only, pursuant to our Terms of Use. Any other use requires NPR's prior permission. Visit our permissions page for further information. NPR transcripts are created on a rush deadline by a contractor for NPR, and accuracy and availability may vary. This text may not be in its final form and may be updated or revised in the future. Please be aware that the authoritative record of NPR's programming is the audio. http://www.npr.org/2014/02/26/282836487/why-we-miss-creative-ideas-that-are-right-under-ournoses urship/Arthur-C-Brooks/e/9780132330763 1960’s http://enitiative.syr.edu/eTeam.html Academia (knowledge) Trends in U.S. evaluation training programs LaVelle and Donaldson 2010 Government (accountability) http://en.wikipedia.org/wiki/File:Aerial_vie w_of_the_Capitol_Hill.jpg 2009 Social Innovation Fund https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.nationalservice.gov/Default.asp http://www.waveland.com/Titles/Welch-Comer.htm Philanthropy (impact) oices/2008/12/say_yes_program_receives_1_mil.html How many first names of your great grand parents can you name? •2- 28% •3- 15% (nationally that’s all that could name that number source: Byran Klontz) Carman, Fredericks, & Introcaso (2008:10) http://www.maxwell.syr.edu/ •Chances are in 2 generations you will be forgotten! Foundations are investments in immortality. Impact is important http://www.faculty.virginia.edu/sixties/readings/War%20on%20Poverty%20entry%20Poverty%20Encyclopedia.pdf to people. urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm http://morinorth.com/researchp ublications/researcharchive/213 /Over-Half-The-PopulationCant-Name-Any-Of-TheirGreatGrandparents.aspx oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ http://morinorth.com/resea ublications/researcharchive /Over-Half-The-Population Cant-Name-Any-Of-TheirGreatGrandparents.aspx urship/Arthur-C-Brooks/e/9780132330763 Carman, Fredericks, & Introcaso (2008:10) http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ urship/Arthur-C-Brooks/e/9780132330763 Carman, Fredericks, & Introcaso (2008:10) http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html Utilizing results includes for publicity and fund raising for non profits: http://www.9wsyr.com/news/local/story/More-parentsreading-to-their-kids/MCwPYPKY20SPNrUHf1p0Pw.cspx https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm http://www.socialresearchmethods.net/kb/pecycle.php oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom Assessment: Outputs and Outcomes http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ Its all about movement or gain. What changed? What gain was made from before to after? urship/Arthur-C-Brooks/e/9780132330763 Evaluation http://enitiative.syr.edu/eTeam.html INTERVENTION Programs Services Initiatives https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ How well did we do it? What did we do? http://www.waveland.com/Titles/Welch-Comer.htm What difference did it make? oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ Source: Apter O’Connor Associates Summative urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html What are you trying to measure? https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html Initially, focus on a few key goals that are critical to the success of the organization or business, and ensure they are SMART, i.e: Specific Measurable Achievable Relevant Timely http://www.businessballs.com/dtiresources/performance_measurement_management.pdf http://www.maxwell.syr.edu/ urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html Levels of Impact short term outcomes, those changes or benefits that are most closely associated with or caused by the programs outputs https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html Source: Ridzi 2012:103. Managing Evaluation Expectations When Measuring Philanthropic Impact. The Foundation Review http://www.maxwell.syr.edu/ Long term outcomes or program impacts, follow from the benefits accrued though the intermediate outcomes. urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html What Might You Evaluate? •Peter York (2011:6) on typical nonprofit goals such as “no longer homeless” or “eliminating a disease” “Such goals are inspirational, but they are almost always beyond the direct reach of a single intervention or program. It is more realistic, for example, to measure whether clients in a program to help the homeless actually follow through with a job referral or a doctor’s appointment. Such goals may sound less inspiring but actually represent a significant achievement. More importantly, they are within reach — and they are the building blocks for achieving broader societal change, one person at a time.” https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html Be Realistic… http://www.maxwell.syr.edu/ http://www.tccgrp.com/pdfs/7-21_TCC_Briefing_Paper_LR.pdf urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom Short vs. Long Term Outcomes http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ The Case of Supersize Me urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html short term intermediate Long term outcomes or outcomes, those outcomes, those program impacts, follow changes or changes that from the benefits benefits that are result from an accrued though the most closely application of intermediate outcomes. associated with the short term or caused by the outcomes. programs outputs https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm Tip: List out your outcomes them put them in chronological order. oices/2008/12/say_yes_program_receives_1_mil.html http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V7V-3WR4K4M-8&_user=735929&_handle=B-WA-A-W-AV-MsSAYZW-UUW-AAUDUAADDV-AAUVCEWCDV-YBACCWCBC-AVU&_fmt=full&_coverDate=06%2F30%2F1999&_rdoc=8&_orig=browse&_srch=%23toc%235852%231999%23999779998%23102970!&_cdi=5852&view=c&_acct=C000040778&_version=1&_urlVersion=0&_userid=735929&md5=6e6b6f92758532f01a88230d7ec293 http://www.maxwell.syr.edu/ 80 Fig. 1. Elements of the Logic Model. urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom Now let’s complete a logic model for a sample project using the project tool http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ Super Size Me! Health Onondaga County ranks 461 best out of 1877 counties in the nation (top 25%) in prevalence of obesity (at 28.2% of adults) and 32 out of 62 counties in New York (just worse than the top half). So there is room for improvement: Particularly with Diabetes where we are in the bottom 25% of the state- so we should focus on creating lifelong healthy habits early on. Centers for Disease Control and Prevention Interactive Atlas tool, which can be found at http://www.cdc.gov/diabetes/atlas/countydata/atlas.html urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html Obesity https://spears.okstate.edu/eee/classroom 70% 20% http://student.maxwell.syr.edu/maxwellnetworking/ Lack of Exercise http://www.waveland.com/Titles/Welch-Comer.htm Tri-Fit Scores go up Test Scores go up oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ PE 4 Life School kids Daily PE Vid Game Exerci urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ Complete the logic model for PE – 4 Life From Supersize Me urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html short term outcomes, exercise more, Eat healthier, more commitment to health https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm (these are the easiest to measure in a program evaluation) intermediate outcomes, able to exercise longer, run farther, lift more weight Long term outcomes healthier, less heart disease, better test scores? P.E. for life oices/2008/12/say_yes_program_receives_1_mil.html http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V7V-3WR4K4M-8&_user=735929&_handle=B-WA-A-W-AV-MsSAYZW-UUW-AAUDUAADDV-AAUVCEWCDV-YBACCWCBC-AVU&_fmt=full&_coverDate=06%2F30%2F1999&_rdoc=8&_orig=browse&_srch=%23toc%235852%231999%23999779998%23102970!&_cdi=5852&view=c&_acct=C000040778&_version=1&_urlVersion=0&_userid=735929&md5=6e6b6f92758532f01a88230d7ec293 http://www.maxwell.syr.edu/ 80 Fig. 1. Elements of the Logic Model. urship/Arthur-C-Brooks/e/9780132330763 Part 2 http://www.voki.com/php/viewmessage/?chsm=df087a23e838 8c7dcde5daa5d8bab2dd&mId=1727861 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ Versus Service Learning Project urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html Continuous Data Continuous Analysis https://spears.okstate.edu/eee/classroom Outcome Response http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm Categorical Categorical versus Continuous Data Analysis oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ http://www.lafollette.wisc.edu/Courses/PA819/Categorical%20Data%20Analysis_Logistic%2 0Regression.ppt urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ Change the tool to get data for each of the following… urship/Arthur-C-Brooks/e/9780132330763 Categorical http://enitiative.syr.edu/eTeam.html Nominal Ordinal or Continuous? Nominal Variables Variable: Kind of Beverage https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ OR http://www.waveland.com/Titles/Welch-Comer.htm 1 oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ 2 3 1 2 3 Order anyway you please! http://www.lafollette.wisc.edu/Courses/PA819/Categorical%20Data%20Analysis_Logistic%2 0Regression.ppt urship/Arthur-C-Brooks/e/9780132330763 Categorical http://enitiative.syr.edu/eTeam.html Nominal Ordinal or Continuous? Ordinal Variables Variable: Size of Beverage https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm Small Medium Large oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ http://www.lafollette.wisc.edu/Courses/PA819/Categorical%20Data%20Analysis_Logistic%2 0Regression.ppt GPA predict Continuous Nominal Ordinal or Continuous? urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html Continuous Variable s https://spears.okstate.edu/eee/classroom ? associated with in. http://student.maxwell.syr.edu/maxwellnetworking/ lb. http://www.waveland.com/Titles/Welch-Comer.htm Income Is weak 0 oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ Weight Purchase associated with STRONG Height Income ? 1 Purchase http://www.lafollette.wisc.edu/Courses/PA819/Categorical%20Data%20Analysis_Logistic%2 0Regression.ppt ? urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom Discuss DPIL Survey http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ Level of analysis Scale etc, Change the tool for DPIL urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm The utilization of the logic model as a system level planning and evaluation device David A. Julian , * oices/2008/12/say_yes_program_receives_1_mil.html United Way of Franklin County, U.S.A. Available online 10 June 1998. Evaluation and Program Planning Volume 20, Issue 3 , August 1997, Pages 251-257 http://www.maxwell.syr.edu/ http://0-www.sciencedirect.com.library.lemoyne.edu/science?_ob=ArticleURL&_udi=B6V7V-3SX0MC6-2&_user=735929&_handle=B-WA-A-W-AY-MsSAYZW-UUA-AAUDEZCWAA-AAUCCVZUAA-YVDVBV urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom Activity- What are realistic outputs and outcomes? http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm Print p 35 from the following link and match each with its example http://www.tccgrp.com/pdfs/per_art_evaluating.pdf oices/2008/12/say_yes_program_receives_1_mil.html Connolly, Paul and Peter York. (2002). “Evaluating Capacity-Building Efforts for Nonprofit Organizations.” Organizational Development (OD) Practitioner. VOL. 34 NO. 4 2002 http://www.tccgrp.com/pdfs/per_art_evaluating.pdf http://www.maxwell.syr.edu/ urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm We measure change using Crosstabs (aka pivot tables) Quickly translate this chart into a crosstab with percents and pretend it is data for your program. What did you find? (light blue is success) oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ http://www.tccgrp.com/pdfs/7-21_TCC_Briefing_Paper_LR.pdf urship/Arthur-C-Brooks/e/9780132330763 Outputs and Outcomes • To what extent and how did the participants, organization, or communities apply what was Outcome 100 presented during training sessions and advised during consulting engagements? What have they done differently?“ • To what extent and how have the attitudes and beliefs 200 of participants, staff members, or community Outcome members changed regarding the problem or issue being addressed?“ 300 Outcome • What did the participants learn as a result of the capacity-building activities, and how did they do so?“ 400 • How relevant were the services? Output • How satisfied were participants with the services? 500 Output What did they like and dislike about them? • To what extent do the services reflect best practices 600 Output and current knowledge? http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm A resource if you have trouble picking outcomes – because many people do in the professional world as well. Urban Institute… oices/2008/12/say_yes_program_receives_1_mil.html http://www.maxwell.syr.edu/ Community Need……..……Logical Approach………….. Assessment……..Sustainability……….Special Interest Areas urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.urban.org/center/cnp/projects/outcomeindicators.cfm http://www.maxwell.syr.edu/ Community Need……..……Logical Approach………….. Assessment……..Sustainability……….Special Interest Areas urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.urban.org/center/met/projects/upload/Adult_ http://www.maxwell.syr.edu/ Community Need……..……Logical Approach………….. Assessment……..Sustainability……….Special Interest Areas urship/Arthur-C-Brooks/e/9780132330763 http://enitiative.syr.edu/eTeam.html https://spears.okstate.edu/eee/classroom http://student.maxwell.syr.edu/maxwellnetworking/ http://www.waveland.com/Titles/Welch-Comer.htm oices/2008/12/say_yes_program_receives_1_mil.html http://www.urban.org/center/met/projects/upload/Adult_ http://www.maxwell.syr.edu/