Simple Survey Resources:
Templates, Tabulation &
Impact
Jeff Buckley, Jenna Daniel,
& Casey Mull
Overview
•
•
•
•
•
Situation
Need Being Met
Audience (Agents & Associates)
Basic Overview of the Training
How Can This Be Utilized in Your State?
Situation
• As a result of shrinking state budgets, the need to
effectively document program impact is more critical than
ever.
• Many states do not have a full time Evaluation Specialist.
• County Agents and State Specialists must develop their
own evaluation instruments.
• Many County Agents and State Specialists may not feel
qualified or confident enough to develop their own
instruments and/or to tabulate and report data.
Need Being Met
• A 4-H program evaluation training was
created
• Implemented by Georgia 4-H State
Specialists & CAES Faculty
• Piloted at the 2012 Winter School
– 4-H Agents & Associates
Simple Survey Training
• Training Format:
– 90 minute session
– Interactive
– 4-H Agents and Associates
– Participants created their own program
evaluation
Simple Survey Training
• Why is it Important to Gather Data?
– Who is your audience?
– Process Evaluation – Internal Use
– Outcome Evaluation – External Audience
• How will you share the results?
– Impact Statement
– News Article
– Journal Article
– Other
Simple Survey Training
• What Can a Survey Measure?
– Knowledge
– Attitude
– Behavior
•
Designing Measurable Objectives
– Refer to the handout “Verbs for Writing Measurable
Objectives.”
– Measurable objectives will translate into more
effective constructs and questions.
Simple Survey Training
• Consider…
– What you want your participants to know.
– How you want their attitude to change.
– What sort of behavior changes you want to see.
• Creating A Construct
– Pick two outcomes/constructs and develop
three “questions” each, OR
– Pick three outcomes/constructs and develop
two “questions” each.
– (The template has space for six “questions”.)
Simple Survey Training
• Creating A Construct
–
–
–
–
–
–
The food in the cafeteria was hot.
The dining area was clean and comfortable.
There were a variety of fruits and vegetables.
Beverage selections included healthy options.
Vegetables were not over-cooked.
The servers were on task and pleasant.
Simple Survey Training
• Likert Style Questions
– These templates are designed for use with
Likert style questions.
– Pick the rating scale that works best for your
objectives/program.
– This is the most challenging part of developing
your survey. Allow time. Ask for feedback.
Simple Survey Training
• Likert Style Questions
– Consist of a statement and a rating scale.
– Need to conform to one rating scale.
– Measurable verbs that fit the appropriate cognitive
domain.
– One data point per question.
– Is what you’re measuring important to the intended
audience of the evaluation?
Simple Survey Training
• Instrument Templates
–
–
–
–
–
Traditional
Retrospective Postthenpre
6 quantitative items
1 qualitative item
Feedback on the program
Simple Survey Training
• Tabulation
– Excel Spreadsheet
– Directions
– Worksheet for both instrument templates
– “Locked” format
Now You’re Ready to Gather Data!
Now You’re Ready to Gather Data!
Now You’re Ready to Gather Data!
Results
• Presentations Conducted
– Galaxy
– Georgia 4-H Winter Conference
– Georgia Association of 4-H Agents Conference
– Southern Region Volunteer Conference
• Impact
Results
– The percentage of program participants who stated that they
could…
– develop measurable objectives increased from 60% before
to 100% after the program.
– design evaluation instruments for their programs increased
from 69% before to 91% after.
– are likely to develop their own evaluation tools in the future
increased from 46% to 89/%.
– A domain analysis of the qualitative revealed the following
themes…
– Participants were very glad to have the templates.
– Participants plan to use the tools and knowledge they gained
to gather data.
– Participants plan to use the data they collect to develop
higher quality impact statements.
Results
• Extension Agent Quotes & Impact Statements
– With a mean score of 3.1 (SD=.994), overall
respondents from Question 5 indicated they
"agreed" they are more likely to get involved in
their community as a result of the GPK Leadership
Adventure weekend.
– Retrospective post then pre: [In Question 2], 84.6%
of respondents indicated an increase in their
confidence concerning their ability to “design or
construct a map of the distribution of major world
languages”.
Simple Surveys in Florida 4-H
• Tabulation of Data
– Dropping the data in the sheets is so simple and
saves a ton of time. It is much easier than manually
tabulating learning gains
– Data is easy to interpret from the sheets
• How Data Has Been Used
– Data has been used with stake holders to secure
program support and has been used in annual
reports of accomplishment.
– Data will be used in presentations at professional
conferences to show outcomes and projected
impacts of programming.
• 5.
Training Evaluation
• Training Program Evaluation Exercise
– Confidence in ability
– Knowledge gain
– Training evaluation
How Can You Utilize this Program?
• Resources:
– www.georgia4h.org/evaluationresources
• State Meetings
• Webinars
• New Agent Training
Questions?
For more information contact:
University of Georgia, State 4-H Office
Jeff Buckley, jbuckley@uga.edu
Jenna Daniel, jbrown10@uga.edu
Casey Mull, mullcd2@uga.edu
State 4-H Office – 706-542-4444