Unit 7:
Using your evaluation –
Communicating, reporting, improving
1.
2.
3.
4.
Who
What (Content)
How (Format)
When
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
“Think like a wise man, but communicate in
the language of the people.”
− William Butler Yeats
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Why communicate?
The proper function of evaluation
is to speed up the learning process
by communicating what might otherwise be
overlooked or wrongly perceived…
Success is to be judged by… success in
communication…
Payoff comes from the insight that the evaluator’s
work generates in others.
- L. J. Cronbach
Cronbach, L.J. (1982). Designing evaluations of educational and social programs. San Francisco: Jossey-Bass, p. 8.
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Plan with the end in mind.
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Types of communications - reports
• Written report long, short,
summary
• Success story
• Impact statement;
spotlight
• Elevator story
• Press release
• Media appearance
• Public meeting
• Memo, email, fax,
postcard
• Newsletter
• Personal discussion
• Bulletin, brochure
• Display/exhibit
• Audio/video
presentation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Communication plan
WHO?
WHAT?
HOW?
WHEN?
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
WHO - Potential Users
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
County board/City council (elected and appointed officials)
Funders
Program Committee
Participants
Team / Staff
Volunteers
Collaborating agencies; partners
Businesses; business groups
Police – law enforcement
Schools boards, parent-teacher organizations
Community
Church organizations
State legislators
Professional organizations
You
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
WHO is the Audience?
INTERNAL
1.
Primary audience or
secondary audience??
2.
3.
4.
EXTERNAL
1.
2.
3.
4.
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
WHAT –
will you say, report, communicate??
• What does the audience care about?
• What do you want the audience to have?
• What type of information resonates with the
audience: numbers, quotes, stories?
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
HOW will you communicate –
what formats will you use?
•
•
•
•
•
•
•
•
•
•
Report
Impact statement
Executive summary
Personal discussion
Oral presentation
Photo display
Press release
Newsletter, bulletin
Poster
??????
U n iv ers ity o f W is c o n sin-E xte n s io n pro g ra m s s e rv in g the
th
9 8 A s s e m b ly D is tric t
O ve r v ie w
C o o p e ra tiv e E x te n s io n
- 7 2 co u n ty o ffic e s
- F a m ily L iv in g P ro g ra m s
- 4 -H Y o u th D e ve lo p m e n t
- C o m m u n ity, N a tu ra l
R e s o u rc e s a n d
E c o n o m ic D e v e lo p m e n t
- A g ric u ltu re a n d N a tu ra l
R e s o u rc e s E d u c a tio n
- G e o lo g ic a n d N a tu ra l
H is to ry S u rv e y
- 1 .4 m illio n e d u c a tio n a l
c o n ta c ts p e r y e a r
C o n tin u in g E d u c a tio n
- P a rtn e rs h ip s w ith 2 6
U W S ys te m ca m p u s e s
- U W -L e a rn in g
In n o v a tio n s
- S c h o o l fo r W o rke rs
- 1 6 2 ,0 0 0 e n ro llm e n ts p e r
year
B ro a d c a s tin g & M e d ia
In n o v a tio n s
- W is c o n s in P u b lic
T e le v is io n
- W is c o n s in P u b lic R a d io
- In s tru c tio n a l
C o m m u n ica tio n s
S y s te m s
- N e a rly 1 m illio n v ie w e rs
a n d lis te n e rs p e r w e e k
R e p re s e n ta tive M a rc D u ff
W a u k esh a C o u n ty E x ten sio n — T h e W au k esh a C o u nty U W -E x tensio n
C o o p er ativ e E xten sion o ffice h as d ev elo p ed p ar tn er sh ip s w ith nu m er o u s
o r g an izatio n s to d eliv er ed u cation al p ro g r am s ad d r essin g lo cal n eed s.


F o r th e p ast tw o y ear s, b u dg etin g , n u tr itio n an d fo o d safety h av e b een tau gh t
th r o u g h W au k esh a C o un ty E x tensio n ’s N utr itio n E d u cation P r o gr am at th e
W au k esh a C o u n ty Jail. T h e p r o g r am also lin k s in m ates to r eso u r ces th at w ill
b e av ailab le to th em o n ce th ey ar e r eleased .
T h e N u tr itio n al E d u cation P r o g r am also w or k s w ith in div id u als w ith d r ug an d
alco h o l ab use p r o blem s w h o ar e liv in g at th e W au k esh a C ou n ty H u b er
F acility . T h e p r og r am fo cu ses o n h ealthy eating an d th e effects o f d r ug s an d
alco h o l o n th e b o d y , fo o d safety an d b u d g etin g .



S m a ll B u s in e s s
D e v e lo p m e n t C e n te rs
- 1 3 ca m p u s -b a s e d
c e n te rs
- W is c o n s in In n o va tio n
S e rv ic e C e n te r
- 1 5 ,0 0 0 h o u rs o f
c o u n s e lin g p e r ye a r

w w w .u w e x .e d u
in fo @ u w e x .e d u
(6 0 8 ) 2 6 2 -3 7 8 6
T h e W au k esh a C o u nty E x ten sio n o ffice d ev elo p ed a H u ng er in W au k esh a
C o u n ty N u tr ition S u r v ey fo r ar ea co m m u n ities. T h ese co m m u n ities also
r eceiv e n u tr itio n ed u catio n ser v ices th r ou g h th e W au k esh a C o u n ty N utr itio n
E d u catio n P r o g r am at sites lik e th e W IC clinics, eld er ly n utr itio n sites an d
fo o d p an tries.
T h e P a ren ting th e First Y ea r n ew sletter is distrib u ted th ro u g h C o m m u n ity
M em o r ial H o sp ital in M en o m o n ee F alls w ith fu nd in g fr o m th e lo cal Kiw an is
clu b . T h e P a ren tin g th e S eco n d a n d T h ird Y ea r n ew sletter is d istr ibu ted
th r o u g h C o m m u n ity M em o r ial u n d er a g r an t fr o m th e C hild A b u se
P r ev en tio n F un d (C A P ). T h e p r o ject cu r r en tly is b ein g ev alu ated in o r d er to
m easu r e th e v alu e to p ar en ts. A s a r esult o f th is p ar tn er ship , p ar entin g
ed u catio n p ro g r am s su ch as H o m e A lo n e, P ar en tin g Y ou r T een an d
D iscip lin e vs. P u n ish m en t ar e also o ffer ed .
T h e W au k esh a C o u nty E x ten sio n o ffice p r o vid ed lead er sh ip to a co llab o r atio n
o f 1 4 ag en cies/ o rg an izatio n s ser vin g child r en an d fam ilies to co n d u ct a tw o to -th r ee-y ear p r o g r am to d issem in ate r esear ch an d co n d u ct co m p r eh en siv e
tr ain in g ab o u t ear ly lear n in g an d b r ain d ev elo p m en t o f in fan ts an d ch ild r en .
W au k esh a C o u n ty E xten sio n o ffer s a G ra nt O p po rtun ities N ew sletter , w h ich
id en tifies g r an ts av ailable to th e co m m u n ity fr o m a v ar iety o f state an d fed er al
ag en cies as w ell as lo cal fo u nd atio ns. T h e list in clu d es fed er al ag en cies, su ch
as th e D ep artm en t o f H ealth an d H u m an S er v ices, D ep ar tm en t o f H o u sin g an d
U r b an D ev elo p m en t, D ep ar tm en t o f L ab o r an d E m p lo y m en t an d T r ainin g
A d m in istr atio n , an d state en tities in clud in g th e D ep ar tm en t o f W o r k fo r ce
D ev elo p m en t, D ep ar tm en t o f A d m in istr atio n - D ivisio n o f H o u sin g , T E A C H ,
D ep ar tm en t o f T r ansp o r tatio n , D ep ar tm en t o f N atu r al R eso u r ces an d th e
W isco n sin A d v an ced T eleco m m u n icatio n s F ou n d atio n .
T h e W au k esh a C o u nty E x ten sio n -U r b an In itiativ e P r o g r am in S ussex w ill
ad d r ess n eigh b o r h oo d r evitalizatio n , b uildin g a sen se o f co m m u n ity , cr im e
an d p u b lic safety . In 2 0 01 th e p r o gr am r eceiv ed a $ 1 0 , 0 00 C o m m u n ity
D ev elo p m en t Blo ck G r an t w h ich w ill allo w ex p an sio n o f th e p artn er sh ip an d
im p lem en tatio n o f an actio n p lan in at-r isk , tro u b led n eig h bo r h o o ds.
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
HOW – what format, style will you use??
• Format depends on purpose and
audience
• Written, oral
• Short, long
• How does the audience prefer receiving
information? How does the audience
best learn?
• What do you have the resources to do?
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
WHEN will you communicate?
•
•
•
•
Quarterly?
Opportunistically?
When requested?
???
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Reporting outline
follows
your
Components
of a Success
Story logic model
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Formal evaluation reports typically include…
• Abstract/executive summary
• Introduction
− Purpose of the evaluation; key questions
− Program background, description
• Methods/procedures
− Data sources
− Data collection procedures
− Sampling
− Limitations
• Results
• Discussion
• Conclusions/recommendations
• References
• Appendices
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Effective communications
•
•
•
•
•
•
•
•
•
•
•
Tailor message to issue and audience
Avoid jargon and technical terms
Be clear, concise
Use active voice
Eliminate wordiness
Check writing, grammar
Be accurate, balanced, impartial
Be timely
Use graphics, quotes, photos, real stories
Consult a communications specialist
Write-rewrite-rewrite
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Myths
• One report is enough.
• People read written reports.
• Complex analysis and big words impress people.
• Oral reports have the same effect as written
reports.
• Describing limitations weakens report.
• Everything should be reported.
• The audience knows why they are getting the
report.
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Ingredients of a Good Success Story
• Involves human interest: catches your attention
• Demonstrates behavior change; action taken
− change in practice that results in benefits to individual,
community, environment...
•
•
•
•
•
Uses words of/meaning for people involved
Indicates profit to the individual, if appropriate
Based on reliable, credible information
Includes numbers and narrative
Presents balanced, fair assessment
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Make sure the evaluation isn’t ignored
• Engage stakeholders in doing the
evaluation: in the design, data collection,
in analysis, interpretation of results…
• Get the information to the right people
• Address issues that people think are
important
• Keep it in front of people
• Present it in time to be useful
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Discuss limitations
Written reports:
• Be explicit about your limitations
Oral reports:
• Be prepared to discuss limitations
• Be honest about limitations
• Know the claims you cannot make
− Do not claim causation without a true experimental
design
− Do not generalize to the population without random
sample and quality administration (e.g., <60%
response rate on a survey)
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Reporting results to the media
All Media:
• Avoid using too many statistics.
• Focus on the key points.
• For quotes, speak more globally about
the issue.
• Always give the source and timeliness
of your stats. It’s the “news peg.”
Steve Busalacchi
Director, News & Information
Wisconsin Medical Society
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Reporting results to the media
Radio and TV:
• Do not offer exact statistics
– ear cannot track.
“73.6% of respondents”
vs.
“Nearly three quarters of those surveyed”
• Don’t go into great detail.
Have backup info ready.
Steve Busalacchi
Director, News & Information
Wisconsin Medical Society
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Using graphics
What do I want
my audience to
learn – remember
– from my graph
or table?
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Using graphics
•
•
•
•
Title
Clear units of measure
Date(s) data collected
Simple, straightforward design without
“clutter”
• Font size 10 point or larger
• Explicit data source(s)
• Sample size, if applicable for the
audience
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Graphs
• Bar graphs: show comparisons
− Stacked bar
− Multiple bars
− Vertical, horizontal bars
• Pie charts: show parts of a whole
• Line graphs: show progress over time
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Bar graph
• Compare groups of client outcomes
• Compare points in time
Each group you compare gets a “bar”
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Percentage of worksites
Percentage of worksites implementing
"Healthy Lifestyles" program
100
90
80
70
60
50
40
30
20
10
0
74%
60%
Williams County
Worksites
Wisconsin Worksites
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Horizontal Graph
Example Bar Graph
4
3
Series2
Series1
2
1
0
10
20
30
40
50
60
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Vertical Graph
Example Bar Graph
60
50
40
Series1
30
Series2
20
10
0
1
2
3
4
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Column Chart
Column Chart
100%
80%
60%
40%
20%
0%
1
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Pie charts: parts of a whole
• Useful for displaying proportions
− Percentage of clients achieving the outcome
− Percentage of participants who are boys
− Percentage of sites that are rural
• Ensure individual parts add to 100%
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Pie Chart
Example Pie Chart
1
2
3
4
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Line graphs: time lines
• X axis conveys the time
• Y axis is the variable of interest
• Multiple lines or multiple line graphs
show performance of different groups
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Line Graph
Example Line Graph
60
50
40
Series1
30
Series2
20
10
0
1
2
3
4
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Area Graph
Example Area Graph
70
60
50
40
Series2
30
Series1
20
10
0
1
2
3
4
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Pictogram
End of
November
Total!
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Graphing using Microsoft Excel
• Click on graph icon on toolbar, or select
“insert” and “chart”
• Proceed through chart wizard
− Select the type of graph you want
− Highlight the range of data to include
− Set up titles, axis labels, scales, legend
− Finish
• Edit the chart using drop down menu for
charts
− Click on chart to highlight it
− Use the top toolbar selection “chart” to edit
parts of the chart
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation