Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Impact Reporting:
Stepping Up to the Challenge
National Farm Business Management Conference
Fargo, ND
June 13 - 17, 2010
Bob Wells, Tim Eggers and Madeline Schultz
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Annie’s Project Mission
To empower farm women to be better business
partners through networks and by managing
and organizing critical information.
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Purposes of Evaluation
“An evaluation approach should be
rigorous in its efforts to determine the
worth of a program and to guide program
implementation and management, as well
as relevant and useful to program
practitioners.”
Anne C. Peterson
Senior Vice President for Programs, W.K. Kellogg Foundation
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Layers of ANNIES Evaluation
Outcomes:
Demographics:
1. Numbers
2. Characteristics
1.
2.
3.
Knowledge
Actions
Impacts
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Importance of Impacts
• Reports and requests
– Universities & organizations
– Government & private funders
• Impacts demonstrate true
success
– Profitability
– Rural lifestyle satisfaction
• ANNIES Learning & growth
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
ANNIES has Meaningful Data
• Class Evaluations
– Local, state and national
reports
• Journal of Extension
– The Effectiveness of
Women’s Agricultural
Education Programs: A
Survey from Annie’s Project
• ANNIES Fall 2009 Survey
• 2009 NIFA portfolio review
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Demographics Measures
21 Annie’s Project States
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Demographics Measures
408 Annie’s Project Classes
90
80
70
60
50
40
30
20
10
0
79
78
81
63
52
55
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Demographics Measures
7,237 Annie’s Project Participants
1800
1642
1600
1458
1309
1400
1200
1000
897
987
944
800
600
400
200
0
2003-05
2006
2007
2008
2009
2010*
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Demographics Measures
Sample Pre-Class
Survey Analysis
Business Information
Years Farming
Demographics
Average: 18
Age
Range: 2-33
Average: 49
Total: 217
Range: 29-57
Acres
Marital Status
Total Owned: 5646
11 (All) Married
Average Owned: 471
Children
Total Farmed*: 10,940
Average: 2
Average Farmed*:2817
Range: 1-5
Type of Business Organization
Counties
9 Sole Owners
McLeod -4
2 Partnerships
Wright - 4
Incomes
Sibley - 3
$50,001-$150,000: 6
Wright, Pope – 1
$15,001-$300,000: 2
Wright, Pope, Stearar – 1
Over $300,000: 2
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Outcome Measures of Knowledge
Journal of Extension Article
Table 1: Summary of 'Yes' Responses for Selected Questions
Question
Pre-test
Post-test
Difference
Percent Yes Percent Yes
Production records
Marketing plan
Balance Sheet
Income Statement
70.00%
10.00%
55.80%
43.20%
83.70%
22.80%
77.20%
68.50%
13.70%
12.80%
21.40%
25.30%
Comfort w/debt level
66.30%
84.80%
18.50%
Next generation plan
Life Insurance
Will
45.30%
90.50%
70.00%
64.10%
95.70%
76.10%
18.80%
5.20%
6.10%
Source: Hambleton-Heins Southern Illinois University, 2009
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Outcome Measures of Knowledge
Journal of Extension Article
Table 2: Pre-test/Post-test Comparison by Risk Category and by Total
# of
Pre-test % Post-test %
%
Risk Category
questions
yes
yes
Difference Change
Production
12
47.63%
52.81%
5.18% 10.80%
Marketing
6
26.58%
35.33%
8.75% 32.90%
Financial
15
56.74%
72.32%
15.58% 27.40%
6
50.96%
64.67%
13.71% 26.90%
Human Resource
10
49.21%
59.67%
10.46% 21.20%
All Categories
49
48.57%
59.49%
10.92% 22.50%
Legal
www.extension.iastate.edu/annie
Source: Hambleton-Heins Southern Illinois University, 2009
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Outcome Measures of Actions
ANNIES Fall 2009 Survey
• 61% of respondents
– developed a
network of local
peers and
professionals
– that motivate and
support them with
risk management
issues
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Outcome Measures of Actions
Percent of Participants Attending Additional
Educational Classes
60
54
46
40
42
30
% Yes
26
20
24
22
Farm
Leases
35
Lvstk
Mktg
50
10
Contr
acts
Crop
Ins
Grain
Mktg
Produ
ction
Cost
Estate
Plans
0
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Outcome Measures of Actions
Responses to the Question: What was the most
important thing you took away from Annie’s Project?
• “I did feel better informed; "empowered" to ask questions and apply
research; and overall encouraged to use the skills given in the class as
well as from other farm wives.”
• “I now know who I need to go talk to when I have a question on any
farming topic. Annie's Project brought in a great group of speakers who
know the industry well and are very helpful even after the classes.”
• “The support of other women, going thru the same challenges...it
changed my life forever, knowing I have valuable skills inside me that
are now emerging...to learn, grow and share with other people.
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Outcome Measures of Action
2009 NIFA Portfolio Review
– NIFA demonstrated ANNIES relevance to
•
•
•
•
•
Risk management education
FSA borrower training
NRCS collaborations
National network of educators working together
Intersection of farm and family finances
– Reported in NIFA Sustainable Agriculture
• Primary KA 902 and Secondary KAs 607, 801
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Outcome Measures of Impacts
Testimonials
• Stories of improved
conditions resulting from
new knowledge and the
actions women take after
participating in Annie’s
classes
• Qualitative data is useful,
but we still need
Quantitative measures
• http://www.extension.iastate.edu/annie/testimonials.html
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Stepping up to the Challenge
• ANNIES – more robust re-survey of participants
to collect impacts over time
– Qualitative stories
– Quantitative measures
• development
• ANNIES - implement
–
–
–
–
Collection of more complete and relevant data
Third party objectivity through an evaluation partner
Greater accuracy in analysis and reporting
Attention to creation of scholarly work
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Step One
Integrate Evaluation with Planning
•
•
•
•
•
•
•
Logic models
Evaluation teams
Stakeholders
Good questions
Eval. processes
Budgets
Analysis and Reporting
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Step Two
NIFA Logic Model
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Step Three
Evaluation Teams
• ANNIES has good
evaluation resources
• Access to third party
evaluator
– Adds credibility
– ISU Research Institute for
Studies in Agriculture
• Creates opportunity to
build evaluation capacity
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Step Four
Good Questions
• ANNIES is working to
– Develop new evaluation
instruments
– Pilot these in IA & IL
• We want input from the
network
• The goal is national
consistency
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Resulting In
Good Program Evaluation
Benefits
ANNIES and Women in Agriculture
• Gain new knowledge about women in agriculture and
their learning needs
• Strengthen effectiveness of courses
• Create new courses to meet identified needs
• Support further educator network development and
organization
• Build shared meaning and understanding about ANNIES
with other organizations
• Expand ANNIES to serve more women in more ways
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Thank You !
Your comments and ideas are welcome
Bob Wells
Iowa State University
Field Ag Economist
Co-Director ANNIES
641-673-5841
Tim Eggers
Iowa State University
Field Ag Economist
Co-Director ANNIES
641-673-5841
wellsjb@iastate.edu
teggers@iastate.edu
Madeline Schultz
Iowa State University
Extension Program Coordinator
ANNIES
515-294-0588
shultz@iastate.edu
www.extension.iastate.edu/annie