Community Need……..……Logical Approach………….. Assessment……..Sustainability……….Special Interest Areas
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
Lecture 05: Assessing Your
Program
Part I
Community Need……..……Logical Approach………….. Assessment……..Sustainability……….Special Interest Areas
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
Upcoming- Imagination
Library Phone Bank Mitchel
Hall rm 16
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
Meet here and walk over together
Community Need……..……Logical Approach………….. Assessment……..Sustainability……….Special Interest Areas
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
Agenda
 Discuss
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
Welch and Comer ch 3
 How we collect data
 Nominal, ordinal, continuous
 Validity and reliability
 DPIL Survey and the Assessment
Planning Tool (lab 3)
 Super Size Me
Community Need……..……Logical Approach………….. Assessment……..Sustainability……….Special Interest Areas
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
C.L.A.S.S.
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
Summative:
Does it work?
Yes or No
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
Does DARE
reduce drug use?
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
http://www.sciencephoto.com/media/280069/enlarge
http://www.fsg-impact.org/images/upload/From%20Insight%20to%20Action(3).pdf
outcomes are specific changes in
things like:
•attitudes,
•behaviors,
•knowledge,
•skills, status,
•or level of functioning expected
to result from program
activities. (Kellogg p.18)
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
Why Might You
Evaluate?
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
•Better Planning
•Improved Implementation
•Demonstrate Impact
•Strategically Allocate Resources
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
http://lh3.ggpht.com/_yP6P-egnidU/TUOB6ySgZeI/AAAAAAAABlY/kqPx1v3rP-M/game_plan%5B2%5D.jpg
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
http://www.google.com/imgres?q=rube+goldberg+machine&um=1&hl=en&biw=1024&bih=537&tbm=isch&tbnid=Gg--Zk9MHVX3cM:&imgrefurl=http://predatorhaven.blogspot.com/2011/01/animated-music-rube-goldberg.html&docid=IIfIk6qyI1Jg9M&imgurl=http://1.bp.blogspot.com/_hpxaqo7FS9c/TUG5nscXapI/AAAAAAAADIE/p20ozd8ejE0/s1600/science%25252B%25252BRube%25252BGoldberg%25252Bmachine.png&w=500&h=280&ei=Wz_eTrGXFabC2wXUsd3sBA&zoom=1&iact=rc&dur=78&sig=104112844510945666901&page=2&tbnh=113&tbnw=201&start=8&ndsp=9&ved=1t:429,r:4,s:8&tx=107&ty=54
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
We will explore both summative
(why you are in business) and
formative (how you stay in
business), but first some history….
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
History is important because it gives
you a feel for the audiences for
evaluation (beyond you running the
program better).
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
Word Association…
What comes to mind when you think
of program evaluation?
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
Peace?
Love?
Happiness?
Drugs?
http://en.wikipedia.org/wiki/Hippie
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
urship/Arthur-C-Brooks/e/9780132330763
1960’s
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
Carman, Fredericks, & Introcaso (2008:10)
http://www.maxwell.syr.edu/
The Birth of Evaluation During the The War on Poverty –
L.B. Johnson –
•The Economic Opportunity Act (1964) provided the basis
for the Office of Economic Opportunity (OEO),
•the Job Corps,
•Volunteers in Service to America (VISTA),
•Upward Bound,
•Head Start,
• Legal Services,
• the Neighborhood Youth Corps,
• the Community Action Program (CAP),
•the college Work-Study program,
•Neighborhood Development Centers,
•small business loan programs,
• rural programs,
http://en.wikipedia.org/wiki/Lyndon_B._Johnson
• migrant worker programs,
•remedial education projects,
• local health care centers
•$11 billion tax cut (Revenue Act of 1964),
•the Civil Rights Act (1964),
•the Food Stamp Act (1964),
•the Elementary and Secondary Education Act (1965),
•the Higher Education Act (1965), the Social Security
amendments creating Medicare/Medicaid (1965),
•the creation of the Department of Housing and Urban
Development (1965),
•the Voting Rights Act (1965),
•the Model Cities Act (1966),
•the Fair Housing Act (1968),
•several job-training programs, and various Urban Renewalrelated projects.
http://www.faculty.virginia.edu/sixties/readings/War%20on%20Poverty%20entry%20Poverty%20Encyclopedia.pdf
urship/Arthur-C-Brooks/e/9780132330763
1960’s
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
Trends in U.S.
evaluation training
programs LaVelle
and Donaldson 2010
Academia
(knowledge)
Government
(accountability)
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
http://en.wikipedia.org/wiki/File:Aerial_vie
w_of_the_Capitol_Hill.jpg
2009 Social Innovation Fund
oices/2008/12/say_yes_program_receives_1_mil.html
Carman, Fredericks, & Introcaso (2008:10)
http://www.maxwell.syr.edu/
http://www.nationalservice.gov/Default.asp
http://www.faculty.virginia.edu/sixties/readings/War%20on%20Poverty%20entry%20Poverty%20Encyclopedia.pdf
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
Consultant Joke
•
•
https://spears.okstate.edu/eee/classroom
•
•
•
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
•
•
•
•
•
•
•
•
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
from Erik van de Pol, Visible World:
A shepherd was herding his flock in a remote pasture when suddenly a brand-new BMW advanced out of the
dust cloud towards him. The driver, a young man in a Broni suit, Gucci shoes, Ray Ban sunglasses and YSL tie,
leaned out the window and asked the shepherd,..........
"If I tell you exactly how many sheep you have in your flock, will you give me one?"
The shepherd looked at the man, obviously a yuppie, then looked at his peacefully-grazing flock and calmly
answered, "Sure."
The yuppie parked his car, whipped out his IBM Thinkpad and connected it to a cell phone, then he surfed to a
NASA page on the internet where he called up a GPS satellite navigation system, scanned the area, and then
opened up a database and an Excel spreadsheet with complex formulas. He sent an email on his Blackberry and,
after a few minutes, received a response. Finally, he prints out a 130 page report on his miniaturized printer
then turns to the shepherd and says,.........
"You have exactly 1586 sheep".
"That is correct; take one of the sheep" said the shepherd.
He watches the young man select one of the animals and bundle it into his car.
Then the shepherd says: "If I can tell you exactly what your business is, will you give me back my animal?"
"OK, why not" answered the young man.
"Clearly, you are a consultant" said the shepherd.
"That's correct" says the yuppie, "but how did you guess that?"
"No guessing required" answers the shepherd. "You turned up here although nobody called you. You want to
get paid for an answer I already knew, to a question I never asked, and you don't know crap about my
business.... Now give me back my dog".
http://www.wdolson.com/jokes.htm
The role of outside experts and consultants- people see
outsiders telling them the same things in a more receptive
light
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
•
•
•
•
•
•
•
•
•
https://spears.okstate.edu/eee/classroom
•
•
•
•
•
•
•
•
http://student.maxwell.syr.edu/maxwellnetworking/
•
http://www.waveland.com/Titles/Welch-Comer.htm
•
•
•
•
•
•
•
•
•
•
•
•
•
•
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
Copyright © 2014 NPR. For personal, noncommercial use only. See Terms of Use. For other uses, prior permission required.
DAVID GREENE, HOST:
There are times when I've thought about singing during the program or maybe telling a bad joke. I'm getting the feeling this morning that our producer and editor, Rachel Ward and Kenya Young, would shoot me down. You
ever have this experience? Pitch what you think is a brilliantly creative idea and your boss or manager says nope. If so, it might be worth listening to Steve Inskeep's conversation with NPR's social science correspondent
Shankar Vedantam. Shankar's found some research explaining why good ideas get rejected.
STEVE INSKEEP, HOST:
Hi, Shankar.
SHANKAR VEDANTAM, BYLINE: Hi, Steve.
INSKEEP: Okay. So what's the research say?
VEDANTAM: Well, the research seems to suggest that part of the reason we miss seeing creative ideas that are right under our nose is because the ideas are right under our nose. There's this new research that looks at how
people evaluate creativity. Jennifer Mueller at the University of San Diego and her colleagues, Cheryl Waxslack(ph) and Vishwin Athenkrisnan(ph), find that where the idea comes from appears to influence whether people
think it's creative.
So in this experiment they ran, they told volunteers about a new shoe that uses nano technology to reduce blisters. They told some volunteers that this idea was developed far away and they told other volunteers this idea was
developed nearby. Here's Mueller.
JENNIFER MUELLER: We found that when we told people the idea was generated far away, they rated the idea as significantly more creative than when the idea was generated nearby.
VEDANTAM: You're kidding me.
MUELLER: No.
VEDANTAM: At a certain level, that doesn't make sense, though, does it?
MUELLER: It really doesn't.
INSKEEP: Okay. So same idea, it's all about the source. We're talking about how a manager, a boss, would evaluate an idea that's brought to them.
VEDANTAM: Exactly. So it seems to happen, Steve, because our minds are prone to mixing these two things up. When things are nearby, they're concrete and you can see the details of the things. On the other hand, when
things are far away, they're much more abstract. So thinking about things that are near and far puts us in different mental states. When you think about things nearby, you see the details, and so when a creative idea comes
along, the first thing you ask is, can it work?
Now, most creative ideas are risky and the risks are obvious when you look at the details, so when you think about it with this detail-oriented mindset, you're more likely to shoot the idea down. On the other hand, when you're
thinking about things that are far away, you're in a more abstract frame of mind and so the first question you ask is not will this work; you're more open to seeing the creative possibilities.
INSKEEP: So it's not just that as a manager, that the manager disrespects the employees. The manager is just familiar with the employees, he or she works with the employees every day, and they're thinking about the details of
it. Whereas somebody comes from the outside, they can think big.
VEDANTAM: Exactly. So obviously it has to be said that some ideas are not creative and the deserve to be shot down, but the reason managers often are shooting down ideas that might be creative that come from subordinates
is not because they're necessarily bad managers, but they might be in this different mindset. Mueller reminded me of this famous rejection letter that the writer George Orwell got.
Famous publisher said, look, Americans are just not interested in political stories about animals. "Animal Farm," of course, went on to become this classic book. Here's Mueller again.
MUELLER: It's hard to get executives to say creativity is stupid, even though they may actually feel that way and very strongly. Very rarely will CEOs say renovation's horrible, why would we ever do that?
INSKEEP: So wait a minute. I guess then, if I'm an employee trying to sell an idea to my manager, I should call somebody way outside the company, some stranger and tell them the idea, have them walk in the door with it, I
gather.
VEDANTAM: Well, that's right. So here's the challenge, Steve. So the people who make decisions in organizations and government, they tend to be the people with the most experience. In fact, that's why those people tend to
be in charge. And having that experience is very valuable because it helps you spot these big mistakes.
What this research is suggesting is that the experience also comes at a price. It makes it hard to recognize out of the box possibilities. So Mueller thinks that just before you pitch your boss on a big idea, it might be useful to get
them in a more abstract frame of mind. I'm actually going to do what you just suggested.
The next time I come up with a creative idea, I'm going to call it in on the phone and say, I'm calling from Antarctica with this fantastic idea.
INSKEEP: So Shankar, when you told your editor about your idea to do this story, what did she say?
VEDANTAM: I made sure to do it on the phone, Steve.
INSKEEP: Shankar, thanks very much.
VEDANTAM: Thank you, Steve.
INSKEEP: That's NPR's Shankar Vedantam. You can follow him on Twitter @HiddenBrain. You can also follow this program @MorningEdition and @NPRInskeep.
Copyright © 2014 NPR. All rights reserved. No quotes from the materials contained herein may be used in any media without attribution to NPR. This transcript is provided for personal, noncommercial use only, pursuant to
our Terms of Use. Any other use requires NPR's prior permission. Visit our permissions page for further information.
NPR transcripts are created on a rush deadline by a contractor for NPR, and accuracy and availability may vary. This text may not be in its final form and may be updated or revised in the future. Please be aware that the
authoritative record of NPR's programming is the audio.
http://www.npr.org/2014/02/26/282836487/why-we-miss-creative-ideas-that-are-right-under-ournoses
urship/Arthur-C-Brooks/e/9780132330763
1960’s
http://enitiative.syr.edu/eTeam.html
Academia
(knowledge)
Trends in U.S. evaluation training programs LaVelle and Donaldson
2010
Government
(accountability)
http://en.wikipedia.org/wiki/File:Aerial_vie
w_of_the_Capitol_Hill.jpg
2009 Social Innovation Fund
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.nationalservice.gov/Default.asp
http://www.waveland.com/Titles/Welch-Comer.htm
Philanthropy
(impact)
oices/2008/12/say_yes_program_receives_1_mil.html
How many first names of your great
grand parents can you name?
•2- 28%
•3- 15%
(nationally that’s all that could name that number source: Byran Klontz)
Carman, Fredericks, & Introcaso (2008:10)
http://www.maxwell.syr.edu/
•Chances are in 2 generations you will be forgotten! Foundations are investments in
immortality. Impact is important http://www.faculty.virginia.edu/sixties/readings/War%20on%20Poverty%20entry%20Poverty%20Encyclopedia.pdf
to people.
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
http://morinorth.com/researchp
ublications/researcharchive/213
/Over-Half-The-PopulationCant-Name-Any-Of-TheirGreatGrandparents.aspx
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
http://morinorth.com/resea
ublications/researcharchive
/Over-Half-The-Population
Cant-Name-Any-Of-TheirGreatGrandparents.aspx
urship/Arthur-C-Brooks/e/9780132330763
Carman, Fredericks, & Introcaso (2008:10)
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
urship/Arthur-C-Brooks/e/9780132330763
Carman, Fredericks, & Introcaso (2008:10)
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
Utilizing results includes for publicity and fund
raising for non profits: http://www.9wsyr.com/news/local/story/More-parentsreading-to-their-kids/MCwPYPKY20SPNrUHf1p0Pw.cspx
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
http://www.socialresearchmethods.net/kb/pecycle.php
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
Assessment: Outputs and
Outcomes
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
Its all about movement or gain.
What changed? What gain was made
from before to after?
urship/Arthur-C-Brooks/e/9780132330763
Evaluation
http://enitiative.syr.edu/eTeam.html
INTERVENTION
Programs
Services
Initiatives
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
How well did we
do it?
What did we do?
http://www.waveland.com/Titles/Welch-Comer.htm
What difference did it
make?
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
Source: Apter O’Connor Associates
Summative
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
What are you trying to measure?
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
Initially, focus on a few key goals that are
critical to the success of the organization or
business, and ensure
they are SMART, i.e:
Specific
Measurable
Achievable
Relevant
Timely
http://www.businessballs.com/dtiresources/performance_measurement_management.pdf
http://www.maxwell.syr.edu/
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
Levels of
Impact
short term
outcomes,
those changes
or benefits
that are most
closely
associated
with or caused
by the
programs
outputs
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
Source: Ridzi 2012:103. Managing Evaluation Expectations When Measuring Philanthropic Impact. The Foundation Review
http://www.maxwell.syr.edu/
Long term outcomes or
program impacts, follow
from the benefits accrued
though the intermediate
outcomes.
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
What Might You
Evaluate?
•Peter York (2011:6) on typical
nonprofit goals such as “no longer
homeless” or “eliminating a disease”
“Such goals are inspirational, but they
are almost always beyond the direct
reach of a single intervention or
program. It is more realistic, for
example, to measure whether clients in a
program to help the homeless actually
follow through with a job referral or a
doctor’s appointment. Such goals may
sound less inspiring but actually
represent a significant achievement.
More importantly, they are within reach
— and they are the building blocks for
achieving broader societal change, one
person at a time.”
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
Be Realistic…
http://www.maxwell.syr.edu/
http://www.tccgrp.com/pdfs/7-21_TCC_Briefing_Paper_LR.pdf
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
Short vs. Long Term
Outcomes
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
The Case of Supersize Me
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
short term
intermediate
Long term outcomes or
outcomes, those outcomes, those program impacts, follow
changes or
changes that
from the benefits
benefits that are
result from an
accrued though the
most closely
application of
intermediate outcomes.
associated with
the short term
or caused by the
outcomes.
programs outputs
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
Tip: List out your outcomes them put them in
chronological order.
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V7V-3WR4K4M-8&_user=735929&_handle=B-WA-A-W-AV-MsSAYZW-UUW-AAUDUAADDV-AAUVCEWCDV-YBACCWCBC-AVU&_fmt=full&_coverDate=06%2F30%2F1999&_rdoc=8&_orig=browse&_srch=%23toc%235852%231999%23999779998%23102970!&_cdi=5852&view=c&_acct=C000040778&_version=1&_urlVersion=0&_userid=735929&md5=6e6b6f92758532f01a88230d7ec293
http://www.maxwell.syr.edu/
80
Fig. 1. Elements of the Logic Model.
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
Now let’s complete a logic
model for a sample project
using the project tool
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
Super Size Me!
Health
Onondaga County ranks 461 best out of 1877 counties in the nation
(top 25%) in prevalence of obesity (at 28.2% of adults) and 32 out
of 62 counties in New York (just worse than the top half).
So there is room for improvement: Particularly with Diabetes
where we are in the bottom 25% of the state- so we should focus on
creating lifelong healthy habits early on.
Centers for Disease Control and Prevention Interactive Atlas tool, which can be found at http://www.cdc.gov/diabetes/atlas/countydata/atlas.html
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
Obesity
https://spears.okstate.edu/eee/classroom
70%
20%
http://student.maxwell.syr.edu/maxwellnetworking/
Lack of Exercise
http://www.waveland.com/Titles/Welch-Comer.htm
Tri-Fit
Scores
go up
Test
Scores
go up
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
PE 4 Life
School kids
Daily PE Vid
Game Exerci
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
Complete the logic model for
PE – 4 Life
From Supersize Me
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
short term
outcomes,
exercise more,
Eat healthier,
more
commitment to
health
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
(these are the
easiest to measure
in a program
evaluation)
intermediate
outcomes, able
to exercise
longer, run
farther, lift
more weight
Long term outcomes
healthier, less heart
disease, better test
scores?
P.E. for life
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V7V-3WR4K4M-8&_user=735929&_handle=B-WA-A-W-AV-MsSAYZW-UUW-AAUDUAADDV-AAUVCEWCDV-YBACCWCBC-AVU&_fmt=full&_coverDate=06%2F30%2F1999&_rdoc=8&_orig=browse&_srch=%23toc%235852%231999%23999779998%23102970!&_cdi=5852&view=c&_acct=C000040778&_version=1&_urlVersion=0&_userid=735929&md5=6e6b6f92758532f01a88230d7ec293
http://www.maxwell.syr.edu/
80
Fig. 1. Elements of the Logic Model.
urship/Arthur-C-Brooks/e/9780132330763
Part 2
http://www.voki.com/php/viewmessage/?chsm=df087a23e838
8c7dcde5daa5d8bab2dd&mId=1727861
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
Versus Service Learning
Project
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
Continuous
Data
Continuous
Analysis
https://spears.okstate.edu/eee/classroom
Outcome
Response
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
Categorical
Categorical versus Continuous Data Analysis
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
http://www.lafollette.wisc.edu/Courses/PA819/Categorical%20Data%20Analysis_Logistic%2
0Regression.ppt
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
Change the tool to get data for
each of the following…
urship/Arthur-C-Brooks/e/9780132330763
Categorical
http://enitiative.syr.edu/eTeam.html
Nominal Ordinal or Continuous?
Nominal Variables
Variable: Kind of Beverage
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
OR
http://www.waveland.com/Titles/Welch-Comer.htm
1
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
2
3
1
2
3
Order anyway you please!
http://www.lafollette.wisc.edu/Courses/PA819/Categorical%20Data%20Analysis_Logistic%2
0Regression.ppt
urship/Arthur-C-Brooks/e/9780132330763
Categorical
http://enitiative.syr.edu/eTeam.html
Nominal Ordinal or Continuous?
Ordinal Variables
Variable: Size of Beverage
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
Small
Medium
Large
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
http://www.lafollette.wisc.edu/Courses/PA819/Categorical%20Data%20Analysis_Logistic%2
0Regression.ppt
GPA
predict
Continuous
Nominal Ordinal or Continuous?
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
Continuous Variable
s
https://spears.okstate.edu/eee/classroom
?
associated
with
in.
http://student.maxwell.syr.edu/maxwellnetworking/
lb.
http://www.waveland.com/Titles/Welch-Comer.htm
Income
Is
weak
0
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
Weight
Purchase
associated
with
STRONG
Height
Income
?
1 Purchase
http://www.lafollette.wisc.edu/Courses/PA819/Categorical%20Data%20Analysis_Logistic%2
0Regression.ppt
?
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
Discuss DPIL Survey
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
Level of analysis
Scale etc,
Change the tool for DPIL
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
The utilization of the logic model as a system level planning and evaluation device
David A. Julian , *
oices/2008/12/say_yes_program_receives_1_mil.html
United Way of Franklin County, U.S.A.
Available online 10 June 1998.
Evaluation and Program Planning
Volume 20, Issue 3 , August 1997, Pages 251-257
http://www.maxwell.syr.edu/
http://0-www.sciencedirect.com.library.lemoyne.edu/science?_ob=ArticleURL&_udi=B6V7V-3SX0MC6-2&_user=735929&_handle=B-WA-A-W-AY-MsSAYZW-UUA-AAUDEZCWAA-AAUCCVZUAA-YVDVBV
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
Activity- What are realistic
outputs and outcomes?
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
Print p 35 from the following link and
match each with its example
http://www.tccgrp.com/pdfs/per_art_evaluating.pdf
oices/2008/12/say_yes_program_receives_1_mil.html
Connolly, Paul and Peter York. (2002). “Evaluating Capacity-Building Efforts for Nonprofit Organizations.”
Organizational Development (OD) Practitioner. VOL. 34 NO. 4 2002
http://www.tccgrp.com/pdfs/per_art_evaluating.pdf
http://www.maxwell.syr.edu/
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
We measure change using
Crosstabs (aka pivot tables)
Quickly
translate this
chart into a
crosstab with
percents and
pretend it is
data for your
program. What
did you find?
(light blue is
success)
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
http://www.tccgrp.com/pdfs/7-21_TCC_Briefing_Paper_LR.pdf
urship/Arthur-C-Brooks/e/9780132330763
Outputs and Outcomes
• To what extent and how did the participants,
organization, or communities apply what was
Outcome
100
presented during training sessions and advised during
consulting engagements? What have they done
differently?“
• To what extent and how have the attitudes and beliefs
200
of participants, staff members, or community
Outcome
members changed regarding the problem or issue
being addressed?“
300
Outcome • What did the participants learn as a result of the
capacity-building activities, and how did they do so?“
400
• How relevant were the services?
Output
• How satisfied were participants with the services?
500
Output
What did they like and dislike about them?
• To what extent do the services reflect best practices
600
Output
and current knowledge?
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
A resource if you have trouble
picking outcomes – because
many people do in the
professional world as well.
Urban Institute…
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.maxwell.syr.edu/
Community Need……..……Logical Approach………….. Assessment……..Sustainability……….Special Interest Areas
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.urban.org/center/cnp/projects/outcomeindicators.cfm
http://www.maxwell.syr.edu/
Community Need……..……Logical Approach………….. Assessment……..Sustainability……….Special Interest Areas
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.urban.org/center/met/projects/upload/Adult_
http://www.maxwell.syr.edu/
Community Need……..……Logical Approach………….. Assessment……..Sustainability……….Special Interest Areas
urship/Arthur-C-Brooks/e/9780132330763
http://enitiative.syr.edu/eTeam.html
https://spears.okstate.edu/eee/classroom
http://student.maxwell.syr.edu/maxwellnetworking/
http://www.waveland.com/Titles/Welch-Comer.htm
oices/2008/12/say_yes_program_receives_1_mil.html
http://www.urban.org/center/met/projects/upload/Adult_
http://www.maxwell.syr.edu/