Your project was selected because it was determined to have the potential to make significant impact on sustainable strawberry production.
• NSSI – a results based program; the grant selection, reporting, and management
The Goal: To increase the probability that funded programs will improve the skills, practices and technology of the strawberry growers and industry that participate.
• Key components (Logic Model based):
• Outputs
• Outcomes
• Impacts
• Evaluation

Projects should Begin with the End in Mind
• Determine first what the outcome is that the participants will achieve – or you want them to achieve.
• Design the program (outputs) to lead the participants to those results/outcomes.
• “How will you get there?”
• Evaluation – how will you measure the change?

Logic Model: Beginning With the End in Mind
OUTCOMES
(Identify
Measurable
Technology/
Practice Results)
OUTPUTS
(Lead project participants to the desired results)
INPUTS
(Resources that will strengthen participant achievement of results)
INPUTS
Logic Model Implementing
Forward
OUTPUTS OUTCOMES
From: Western Risk Management Education Center, http://westrme.wsu.edu/award-management/

Logic Model as a Tool to Construct Realistic Outcomes
First:
Identify the situation(s) that your project will help to address (pest management, input reduction, food safety etc.) and consider if what you are proposing will be well received by the target audience you have identified
(Producer/Industry Demand).
Then:
1.
Outcomes – Identify the changes in knowledge, actions, or conditions/system you would like program participants to realize
2.
Outputs – Identify activities (workshops, on-farm trials, demonstrations, etc.) that will support and lead to participant outcomes
3.
Inputs – Resources that go into your program in the way of project team, partners, collaborators, who provide expertise, research, money etc.




*Useful Tool: Search Bloom’s Taxonomy of Action Verbs
Or See: http://www.clemson.edu/assessment/assessmentpractices/referencematerials/documents/Blooms%20Taxonomy%20Action%20Verbs.pdf


Learn or become aware of new fundamental or applied knowledge which could include:
1.
Practices and methods
2.
Technology
3.
Improved skills such as decision making, evaluating


Act upon what they have learned such as:
1.
Implementing new practices, methods or technology
2.
Changing or modifying a practice
3.
Adoption of improved skills
4.
Direct application of knowledge gained

Change in Conditions, System, Environment…
 A condition is changed because of industry/growers wide-spread actions
1.
Increased number of strawberry producers in a state/region leading to increased production and availability.
2.
Increased local/regional strawberry production or season extension to increase availability to consumers.
3.
Adoption of practices that leads to a reduction in greenhouse gas emissions.
Etc.

Will Depend Upon:
Knowledge of your target audience
Why industry/producers will choose to participate
Action and learning processes (project steps) that help industry/producers achieve long term outcomes
Program resources including the team, collaborators and expert presenters sufficient to lead participants to the desired outcomes.

Producer Results or Outcomes an example of progressive steps…
↓ Understand or have improved understanding of the issue, the practice/technology/methods or the need to take action.
↓ Decide to take an action.
- Producer makes a commitment to write a business or marketing plan, set up a record keeping system, or initiate other steps that will lead to adoption
↓ Develop –customize a system or plan
↓ Implement or take an action that improves farm or business i.e. – implementing a food safety plan, using a new technology, etc.

Outcomes
• Measure of success for the participant
• Ability to change/incorporate/implement/adopt practices, technology or skill, etc. as a result of participation in your project.
• Achievable and measureable within a reasonable timeframe
• Sequential
• Identifies changes in condition or practice you seek for the participant
• Specifies the degree of change required for participant success
• Stated in a measurable way knowing methods you will use to verify changes in practices, condition, etc.

“ List your outputs in relation to the objectives of the proposal. Include activities such as presentations
(professional, academic, outreach, stakeholders, consumers, etc.), publications (professional, academic, outreach, etc.), workshops, websites, blogs, etc. Include target audience and number of people.
”
What activities did you conduct to move your participants towards project outcomes?

“List the outcomes of your project, i.e. what change in knowledge, attitude or actions did your outputs accomplish in the short to intermediate time frame.
Include who was reached, how many were reached and how outcomes were measured.”
• What did your participants/target audience do with the project outputs?
• Increase knowledge? Initiate change in thought processes? Change behaviors/practices?
• How did you measure this?

“List the actual or anticipated impact of your project, i.e. the big-picture effect of your project on the industry/growers/region etc. (changes in condition, environment, society) over the longer-term. Include how impact were or will be measured.”
True impacts are harder to measure and may not be evident until late in the project or after the project has been completed.

• Summary of your project
• Objectives of your project
• Media contacts
• Internet media
• Problems delays or changes in plans
• Next accomplishments
• Attachments/Uploads

• Performance will be evaluated based upon your timely and thorough reporting
• Underperforming projects risk termination.
• Performance metrics include;
• Timely activities as proposed
• Accomplishments towards your project goal
• Timely and appropriate use of funds
• Termination
• Unsatisfactory project performance will receive notice
• Notice will include corrective actions and reporting
• Project have one reporting period to correct
• Reminder: Project Deadline is June 30, 2014

• Project Progress Reports should be in alignment with your Project Financial
Reports.
• The progress, outputs, and outcomes of your project will be compared to what funding has been used and its purpose.

How will you measure the effect of your outputs to ensure they are creating suitable/appropriate participant change or any change at all?
• Protocols
• Methodologies
• Data collection

• Begin with the end in mind
• Begin with determining:
• The results that will be accomplished
• How they will be evaluated
• Design evaluation when developing project
• Key tip
• For each proposed outcome you should be able to write an evaluation question
• Do this when you develop your funding proposal
• Validates that proposed results are actual results
• Helps design the evaluation from the beginning

Test your methods
 Will your evaluation tool(s) clearly demonstrate progress participants are making towards achieving the outcomes?
Changes in Knowledge?
Changes in Attitude?
Changes in Behavior/Approach?
Changes in Condition or the System?

1.
Identify measurable expected outcomes.
2.
Define criteria for success.
3.
Develop evaluation tools to measure results.
– End-of-training evaluations
– Post-training evaluations
4.
Evaluation distribution.

Consider
• What will you actually be measuring?
• What is the standard for success?
• How many participants need to achieve ‘success’ in order to be successful?
• When should participants achieve results?

Suitable Question Formats
–
Multiple Choice – Yes/No
–
Rating scales
–
Open ended
 Simple for respondent to understand
 Easy to analyze for project teams with less evaluation experience
Variety of Methods:
• Pre/Post Tests
• Audience Response Systems (interactive polling Clickers or apps)
• Follow-up Evaluations (3 Months – Year)
• Mail or Internet Survey
• Telephone Interview
• Face-to-face interviews (in-person survey)
• Dot survey
• Informal Interview

 Benchmark info on knowledge levels
 Information on participant background, past experience, attitudes and needs for program managers and instructors
 Measures
 Knowledge Gain
 Attitude Change
 Actions participants expected to take as a result of participating in workshop/demonstration/project.

• “Please rate your knowledge PRIOR to attending today’s workshop”
• “Please rate your knowledge AFTER attending this workshop”
• “After attending today’s workshop, answer the following questions:
• Please rate to what extent you have the knowledge, tools and resources needed to …
• I better understand…”
 Likert Scale or Similar Method

 To see what actions participants have taken as a result of participating in your project and how these actions have led to benefits for their farm/business.

• Choose methods that balance best practices with available resources
 Combination of direct and indirect measures
• Direct measures (assignments, action plans etc.) yield stronger evidence of change in knowledge and beneficial practices, but are more time intensive and may take more expertise to evaluate.
• Indirect measures can give instant feedback but may not provide real evidence of knowledge gained.
• Qualitative measures (like phone interviews or focus groups) that may be more resource intensive could be accomplished through a representative sample.

Measuring results/outcomes successfully:
• Direct Interactions with Producers can increase the chance that participants will implement beneficial practices/technology etc.
• Good follow-up can lead to collegial relationships that may bring results beyond the initial results/outcomes that were initially projected (Unexpected results).
• Good program design allows for learning and application to take place in linked stages within a timeframe that encourages the implementation/adoption of longer term results/outcomes.
When writing the evaluation questions, think about:
• How the information gathered will be used.
• The way you ask a question will influence how participants respond.
• When you ask a question will influence how participants respond.

• The Rensselaerville Institute: http://whatworks.org/displaycommon.cfm?an=1&subarticlenbr=266
“Outcome Funding: A New Approach to Targeted Grant making”
Based on the premise that grant makers are really investors in human gain, Outcome Funding addresses the key questions that the funder as investor should ask: What am I buying? What is the probability I will get it? Outcome Funding begins with an abrupt and timely challenge to the standard wisdom of the Request for Proposal process. The authors make the point that very little of the traditional proposal content addresses results, favoring explanations of process instead. This book is not only persuasive on the need for change; it is clear and practical on how to achieve it.
• Developing a Basic Logic Model For Your Program: http://www.publichealth.arizona.edu/chwtoolkit/pdfs/logicmod/chapter2.pdf
I have become convinced that it makes a considerable difference if you do the outcomes before planning the activities. I definitely advocate doing the outcomes first! I find that people come up with much more effective activities when they do. Use the motto, “plan b ackward, implement forward.” Beverly Anderson Parsons, WKKF Cluster Evaluator http://www.uwex.edu/ces/pdande/evaluation/evallogicmodelworksheets.html
• Ties to the Logic Model. http://extension.oregonstate.edu/sites/default/files/for-employees/employee-resources/soarsmodules/logic_model/index.htm\
• Measuring Impacts for Sustainable Agriculture Programming http://agsci.oregonstate.edu/sites/default/files/understanding_program_development_and_evaluation.pdf

• Information in these slides was borrowed and adapted from
Western Risk Management Education Center
• http://westrme.wsu.edu/award-management/
• How Will Your Verify? – Measuring Producer Results
• Extension Risk Management Education
• http://www.extensionrme.org/
• Oregon State University, Writing Powerful Impacts
• http://agsci.oregonstate.edu/research/writingimpacts

sberries@uark.edu