Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
ANNIES Impact Reporting:
Stepping Up to the Challenge
National Women in Agriculture
Educators Conference
March 25, 2010 in Baltimore, MD
Madeline Schultz and Bob Wells
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Annie’s Project Mission
To empower farm women to be better business
partners through networks and by managing
and organizing critical information.
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Purposes of Evaluation
“An evaluation approach should be
rigorous in its efforts to determine the
worth of a program and to guide program
implementation and management, as well
as relevant and useful to program
practitioners.”
Anne C. Peterson
Senior Vice President for Programs, W.K. Kellogg Foundation
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Layers of ANNIES Evaluation
Demographics:
Outcomes:
1. Numbers
2. Characteristics
1.
2.
3.
Knowledge
Actions
Impacts
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Importance of Impacts
• Reports and requests
– Universities & organizations
– Government & private funders
• Impacts demonstrate true
success
– Profitability
– Rural lifestyle satisfaction
• ANNIES Learning & growth
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
ANNIES has Meaningful Data
• Class Evaluations
– Local, state and national
reports
• Journal of Extension
– The Effectiveness of
Women’s Agricultural
Education Programs: A
Survey from Annie’s Project
• ANNIES Fall 2009 Survey
• 2009 NIFA portfolio review
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Demographics Measures
21 Annie’s Project States
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
State
Demographics Measures
CY 2009
Jan-Mar 2010
No. Level I
Classes
Number of
Participants
No. Level I
Classes
Number of
Participants
Arkansas
0
0
0
0
Delaware
0
0
2
35
Florida
0
0
5
67
Illinois
14
159
3
24
Indiana
8
73
0
0
Iowa
5
101
3
39
Kansas
1
15
0
0
Kentucky
8
159
4
65
Maryland
1
24
7
73
Michigan
2
33
7
148
Minnesota
2
30
3
35
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
State
Demographics Measures
CY 2009
No. Level I
Classes
Jan-Mar 2010
Number of
Participants
No. Level I
Classes
Number of
Participants
Mississippi
0
0
0
0
Missouri
7
78
5
69
Montana
0
0
9
69
Nebraska
1
12
2
30
North Dakota
14
150
7
77
Ohio
1
20
5
74
Oklahoma
2
16
0
0
Pennsylvania
0
0
0
0
South Dakota
2
23
1
28
Wisconsin
3
51
3
57
Totals
71
944
66
890
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Demographics Measures
Sample Pre-Class
Survey Analysis
Business Information
Years Farming
Demographics
Average: 18
Age
Range: 2-33
Average: 49
Total: 217
Range: 29-57
Acres
Marital Status
Total Owned: 5646
11 (All) Married
Average Owned: 471
Children
Total Farmed*: 10,940
Average: 2
Average Farmed*:2817
Range: 1-5
Type of Business Organization
Counties
9 Sole Owners
McLeod -4
2 Partnerships
Wright - 4
Incomes
Sibley - 3
$50,001-$150,000: 6
Wright, Pope – 1
$15,001-$300,000: 2
Wright, Pope, Stearar – 1
Over $300,000: 2
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Outcome Measures of Knowledge
Journal of Extension Article
Table 1: Summary of 'Yes' Responses for Selected Questions
Question
Pre-test
Post-test
Difference
Percent Yes Percent Yes
Production records
Marketing plan
Balance Sheet
Income Statement
70.00%
10.00%
55.80%
43.20%
83.70%
22.80%
77.20%
68.50%
13.70%
12.80%
21.40%
25.30%
Comfort w/debt level
66.30%
84.80%
18.50%
Next generation plan
Life Insurance
Will
45.30%
90.50%
70.00%
64.10%
95.70%
76.10%
18.80%
5.20%
6.10%
www.extension.iastate.edu/annie
Source: Hambleton-Heins Southern Illinois University, 2009
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Outcome Measures of Knowledge
Journal of Extension Article
Table 2: Pre-test/Post-test Comparison by Risk Category and by Total
# of
Pre-test % Post-test %
%
Risk Category
questions
yes
yes
Difference Change
Production
12
47.63%
52.81%
5.18% 10.80%
Marketing
6
26.58%
35.33%
8.75% 32.90%
Financial
15
56.74%
72.32%
15.58% 27.40%
6
50.96%
64.67%
13.71% 26.90%
Human Resource
10
49.21%
59.67%
10.46% 21.20%
All Categories
49
48.57%
59.49%
10.92% 22.50%
Legal
www.extension.iastate.edu/annie
Source: Hambleton-Heins Southern Illinois University, 2009
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Outcome Measures of Actions
ANNIES Fall 2009 Survey
• 61% of respondents
– developed a
network of local
peers and
professionals
– that motivate and
support them with
risk management
issues
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Outcome Measures of Actions
Percent of Participants Attending Additional
Educational Classes
60
54
46
40
42
30
% Yes
26
20
24
22
Farm
Leases
35
Lvstk
Mktg
50
10
Contr
acts
Crop
Ins
Grain
Mktg
Produ
ction
Cost
Estate
Plans
0
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Outcome Measures of Actions
Responses to the Question: What was the most
important thing you took away from Annie’s Project?
• “I did feel better informed; "empowered" to ask questions and apply
research; and overall encouraged to use the skills given in the class as
well as from other farm wives.”
• “I now know who I need to go talk to when I have a question on any
farming topic. Annie's Project brought in a great group of speakers who
know the industry well and are very helpful even after the classes.”
• “The support of other women, going thru the same challenges...it
changed my life forever, knowing I have valuable skills inside me that
are now emerging...to learn, grow and share with other people.
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Outcome Measures of Action
2009 NIFA Portfolio Review
– NIFA demonstrated ANNIES relevance to
•
•
•
•
•
Risk management education
FSA borrower training
NRCS collaborations
National network of educators working together
Intersection of farm and family finances
– Reported in NIFA Sustainable Agriculture
• Primary KA 902 and Secondary KAs 607, 801
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Outcome Measures of Impacts
Testimonials
• Stories of improved
conditions resulting from
new knowledge and the
actions women take after
participating in Annie’s
classes
• Qualitative data is useful,
but we still need quantitative
measures
• http://www.extension.iastate.edu/annie/testimonials.html
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Stepping up to the Challenge
• ANNIES would like to re-survey more
participants to collect impacts over time
– Qualitative stories
– Quantitative measures
• ANNIES would like to implement
–
–
–
–
Collection of more complete and relevant data
Third party objectivity through an evaluation partner
Greater accuracy in analysis and reporting
Attention to creation of scholarly work
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Integrate Evaluation with Planning
•
•
•
•
•
•
•
Logic models
Evaluation teams
Stakeholders
Good questions
Eval. processes
Budgets
Analysis and Reporting
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
NIFA Logic Model
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Evaluation Teams
• ANNIES has good
evaluation resources
• Access to third party
evaluator
– Adds credibility
– ISU Research Institute for
Studies in Agriculture
• Creates opportunity to
build evaluation capacity
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Good Questions
• ANNIES is working to
– Develop new evaluation
instruments
– Pilot these in IA & IL
• We want input from the
network
• The goal is national
consistency
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Good Program Evaluation Benefits
ANNIES and Women in Agriculture
• Gain new knowledge about women in agriculture and
their learning needs
• Strengthen effectiveness of courses
• Create new courses to meet identified needs
• Support further educator network development and
organization
• Build shared meaning and understanding about ANNIES
with other organizations
• Expand ANNIES to serve more women in more ways
www.extension.iastate.edu/annie
Annie’s National Network Initiative for Educational Success (ANNIES)
at Iowa State University Extension
Thank You for Your Interest!
Madeline Schultz
Iowa State University
Extension Program
Coordinator
515-294-0588
shultz@iastate.edu
Bob Wells
Iowa State University
Extension Farm
Management Specialist
641-673-5841
wellsjb@iastate.edu
www.extension.iastate.edu/annie