Focused Program Evaluation
Randolph R. “Randy”
Weigel
&
Susan James
EPIC – 2004
Lusk, WY
September 21
“Your tax dollars support this learning
experience either directly or indirectly.
Do you want your tax dollar to continue
supporting this type of program?”
1. Yes
2. No
Robert J. Fetsch, Extension Spec.
Colorado State University
Definition Of Evaluation
“To evaluate is to make an explicit judgment about
the worth of all or part of a program by collecting
evidence to determine if acceptable standards
have been met.”’
Andrews & Warner (1988)
Types of Evaluation
•Process (Formative)
•Impact
(Summative)
When should we conduct an evaluation?
Before?
During?
After?
When should we conduct an evaluation?
1. Project design
2. Start up
3. In-progress
4. End of program
5. Follow-up
When should we NOT conduct an evaluation?
• No one cares
• Not going to change the program
• One time program
• No time, $, resources or energy
• No goals or objectives
• To satisfy your “performance”
Evaluability Assessment
What are you going to evaluate?
What is the purpose of the evaluation?
Who will use the evaluation
What resources do you have and need?
What questions will the evaluation hope to answer?
Knowledge Item
How important is nitrogen fertilizer management for
maintaining our water quality?
*************
Farmers’ proper management of nitrogen fertilizer
applications is essential to maintain good water
quality.
TRUE
FALSE
DON’T KNOW
Attitude Item
How important is nitrogen fertilizer management for
maintaining our water quality?
*************
Farmer’s proper management of nitrogen fertilizer
applications is essential to maintain good water
quality.
Strongly
agree
Agree
Disagree
Strongly
disagree
Experience Item
How important is nitrogen fertilizer management for
maintaining our water quality?
*************
Do you follow recommendations for applying
nitrogen fertilizer when the recommendations are
based on deep soil sampling?
1. No
2. Yes—How much do you apply
1. About 20% less than the amount recommended
2. About 10% less than the amount recommended
3. The exact amount recommended
4. About 10% more than the amount recommended
5. About 20% more than the amount recommended
Feeling Item
How important is nitrogen fertilizer management for
maintaining our water quality?
*************
How do you feel when others imply that farmers
must better manage nitrogen fertilizer applications
to maintain good water quality?
Extremely
Extremely
angry ___ ___ ___ ___ ___ calm
Collecting the Data:
What sources will you use?
•Existing Information
•People
•Observations
•Pictorial Records
What data collection method(s) to use?
• Which method will give you the information you
need?
• Which method is appropriate for participants?
• Which method is least disruptive?
• Which method can you afford and manage?
Common Types of Data Collection
Surveys
Interviews
Observation
Case Study
Group Techniques
Tests
Photos
Common Types of Data Collection
(cont.)
Document Review
Testimonials
Expert Panel
Simulations
Journals
Unobtrusive
measure
“Triangulation”
Using the Information
How will the data be analyzed?
How will the information be interpreted?
Who will receive the evaluation?
How will the information be disseminated?
Avoid These Mistakes in
Your Program Evaluation
http://uwacadweb.uwyo.edu/Research/
Did today’s in-service provide you with information
to review and if necessary improve your program
evaluation skills?
1. Yes
2. No
Do you believe LISA research should or should
not be funded by CSREES/USDA?
1. Should
2. Should not
3. Undecided or unsure
Please interpret each of these words below as a
percentage. Example: Average = 50%
Frequently
Occasionally
Seldom
Sometimes
How much profit did your business earn in
2003?
_____________ Dollars
http://www.extension.psu.edu/evaluation/
http://www.uwex.edu/ces/pdande/evaluation/index.html
http://cyfernet.ces.ncsu.edu/cyfres/browse_2.php?search=Evaluation
http://cru23.cahe.wsu.edu/lifeskills/
http://www.four-h.purdue.edu/fourfold/main.htm
http://www.sagepub.com/
http://www.uwyo.edu/CES/LIFE/Program_Planning_Main.html