Cooperative Extension 4-H Youth Development Programs University of Wisconsin-Extension 431 Lowell Hall 610 Langdon Street Madison, WI 53703-1195 608-262-1221 608-265-6407 (fax) 800-947-3529 (TTY) December 1, 2005 Dear County-based Extension Faculty and Staff, Many 4-H Afterschool programs are up and running and making a difference in your communities. In fact, in 2004 36 people reported on 4-H Afterschool programs and we expect even more in 2005! As you enter your Success Stories, Impact Data and Individual Statistics into the Planning and Results System to meet the Dec. 31, 2005 deadline, we wanted to make you aware of two new instruments to help you collect even more information for your fall 2005 and/or spring 2006 Afterschool programs. We will also discuss these evaluation instruments at the March 10, 2006 4-H Youth Development Wisline, but we wanted to bring them to your attention now. A big thank you goes to Kenosha, Fond du Lac, Brown and Waukesha Counties who participated in our spring 4-H Afterschool 2005 pilot study. From that experience, we learned to hone down the evaluation instrument options to these two: Evaluation Instrument Evaluation Instrument A Questionnaire: Youth Life Skills [pick one or more life skills] Making Decisions Communications Healthy Lifestyle Choices Accepting Differences Evaluation Instrument B Interview Questions for Case Study Questions for Youth Questions for Parents Information Source Your on-site Afterschool person who worked directly with youth in grades 3-5 as well as the youth. Purpose of Evaluation Document how 4-H Afterschool programs helped improve youth’s life skills of: Making Decisions Communications Healthy Lifestyle Choices Accepting Differences. Page # Page 2-7 One youth and one parent/guardian pair/per Afterschool site Gain additional insights and stories from youth and their parents and/or guardians Page 810 If you have any questions, please do not hesitate to call us. Sincerely, Dr. Kathleen E. (Kathi) Vos Experiential Learning Specialist kathi.vos@uwex.edu (608) 262-2496 Paula Huff Oconto County 4-H Youth Development Agent paula.huff@ces.uwex.edu (920) 834-6845 Denise Retzleff Fond du Lac County 4-H Youth Development Agent denise.retzleff@ces.uwex.edu (920) 929-3172 Heather Boyd Evaluation Specialist Heather.boyd@uwex.edu (608) 263-0857 D:\99026384.doc Page 1 D:\99026384.doc Page 2 Life Skill Development in 4-H Afterschool Evaluation Packet Please send completed evaluation instruments by June 1, 2006 to: Heather Boyd, Evaluation Specialist Preferably via E-mail: heather.boyd@uwex.edu Or Surface Mail to: 617 Extension building, 432 North Lake Street Madison, WI 53706 Office phone: 608-263-0857 Evaluation Packet Checklist Please send completed evaluation instruments by June 1, 2006 to: Heather Boyd, Evaluation Specialist Preferably via E-mail: heather.boyd@uwex.edu Or Surface Mail to: 617 Extension building, 432 North Lake Street Madison, WI 53706 Office phone: 608-263-0857 4-H faculty and staff; Please place a check mark next to each set of evaluation results you are returning in this packet. It will help us know what constitutes a completed set for your Afterschool Evaluation. Thanks! Evaluation Results Returned Check items returned Evaluation Instrument A Questionnaire: Youth Life Skills Demographics sheet for each group at each site Evaluation Instrument A Questionnaire: Youth Life Skills [pick one or more life skills] Making Decisions [Head] Communications [Heart] Healthy Lifestyle Choices [Health] Accepting Differences [Heart] Evaluation Instrument B Interview Questions for Case Study Please Indicate: Your County: Name of staff submitting Report: D:\99026384.doc 1 Evaluation Instrument A: Questionnaire: Youth Life Skills Protocal Information source: On-site Afterschool person, who worked directly with youth in grades 3-5 Interview Procedures for Youth Life Skill Surveys 1. These evaluation tools are intended to be used with third to fifth graders. Youth should be surveyed at the end of the series of sessions focused on the development of one of these life skills. 2. Don’t change the wording of any of the survey items, but remove any items on the instrument that are not relevant to your program. You can also add more specific indicators to the tools if you like. 3. Use only the life skill tools that were taught as part of your 4-H Afterschool program. You may use more than one tool if you emphasized more than one life skill. This evaluation packet has tools for four different life skills: Making Decisions Communications Healthy Lifestyle Choices; and/or Accepting Differences. Background Information on Life Skills 4-H Afterschool activities usually identify both a topic that youth want to learn more about, such as foods, aerospace, arts or the environment, as well as an intended life skill, such as decision making. For example, if you were facilitating an activity on how to make healthy snacks and you were just showing kids how to prepare the snacks, then this would NOT be considered a “decision-making” activity. However, if there was a choice of snacks that contain nutritional labels and you provided youth with a decision-making process on how to select the healthiest snack, then this would be considered an activity that is intended to teach youth the life skill of decision making. Usually, life skills are discussed during the “Talk It Over” questions found at the end of most 4-H activities. However, Helpers can also observe youth practicing life skills while doing and/or showing others how to do the activity. 4. Before you distribute the surveys, evaluators should review the 4-H Afterschool sessions with participants that were part of this evaluation period. If you have examples of things that youth made or photos taken during the 4-H sessions, be sure to share/review them with the children at this time. This is done to refresh children’s memories in the event that the sessions were carried over several weeks, or in the event that children are attending other Afterschool programs that are not 4-H Afterschool efforts. 5. Slowly read the paragraph on Human Subjects Review at the top of each survey out loud to all youth being surveyed. The name of this study is Life Skill Development in 4-H Afterschool. Kathi Vos, Ph.D., is the faculty member providing leadership to this study and can be reached at (608) 262-2496, if you have questions. The purpose of the study is to determine if youth who participate in Afterschool activities based on 4-H curriculum improve life skills such as communication, decision making, accepting differences and healthy lifestyle choices. We will not connect your name to your answers. It is up to you whether you want to participate or not. A copy of our Human Subject Protection Statement is on file at the UW-Extension Provost and Vice Chancellor’s Office at 432 N. Lake St., Madison, WI 53706. Your participation in this study implies your consent to participate. Thank you for helping us out. 6. For younger kids, evaluators should distribute the life skill evaluation worksheets and read each statement out loud to the group. Encourage youth to think about their own answer. Tell them that this is not a test, so there is not a right or wrong answer. If youth do not understand the questions, ask them to raise their hand for help. 7. If youth have good reading skills, evaluators can have youth read the questions on their own and fill the survey out. 8. Allow enough time at the end of your session so that youth can hand in the surveys before they have to leave. 9. Each youth should fill out one questionnaire. Please do not have youth answer questionnaires in pairs or as a group. 10. Be sure to thank youth for their participation and if you can, pass out 4-H promotional materials as a thank you. D:\99026384.doc 2 Evaluation Instrument A Questionnaire: Youth Life Skills Demographic Profile Information source: On-site 4-H Afterschool person, who worked directly with youth in grades 3-5 and the youth who participated in the program DEMOGRAPHIC DATA (Local evaluator must fill this out and return it with each set of youth surveys. Complete this form to describe each group/site that is participating in this evaluation.) County: Person Submitting Evaluation Forms: 1. Total number of youth that participated in the 4-H Afterschool session(s) that focused on the development of life skills. (Do not count children more than once if they came to more than one session within a program): 2. Total number of children completing this survey: 3. What life skills were taught at this site (please check all): Making Decisions [Head] Communications [Heart] Healthy Lifestyle Choices [Health] Accepting Differences [Heart] 4. Of the children completing the survey, indicate: CURRENT GRADE LEVEL Number in third grade: Number in fourth grade: Number in fifth grade: SEX Number of males: Number of females: ETHNIC BACKGROUND Number of Black (non-Hispanic): Number of White (non-Hispanic): Number of Hispanic/Latino: Number of American Indian or Alaskan Native: Number of Asian or Pacific Islander: Number of Multi-racial: Number Unknown Please staple your completed Youth Life Skill Evaluation Forms to this Demographic Data Sheet then send completed evaluation instruments to: Heather Boyd, Evaluation Specialist by June 1, 2006 Preferably via E-mail: heather.boyd@uwex.edu Or Surface Mail to: 617 Extension building, 432 North Lake Street Madison, WI 53706 Office phone: 608-263-0857 D:\99026384.doc 3 WHAT I HAVE LEARNED FROM 4-H AFTERSCHOOL ABOUT MAKING DECISIONS Today’s Date: Please read each statement and think about your answer. We want your truthful answer. This is not a test, so there is no right or wrong answer. If you do not understand a question, raise your hand for help. Because of 4-H Afterschool Activities…. (Check one box for each statement.) YES NO NOT SURE a. I think about my choices before making a decision. b. I think about what might happen because of my choice. c. I am happy with the choices that I make. d. When I need to, I ask an adult for help in making choices. e. [Sites can add other indicators specific to its Afterschool program] [Note to Evaluators: Don’t change the wording of any of the survey items, but remove any items on the instrument that are not relevant to your program. You can also add more specific indicators to the tools if you like. Delete this statement BEFORE you administer the survey to youth.] Tell us how you learned about making decisions in 4-H Afterschool: Thanks for completing this form. Please return it to the person who handed it to you today. The evaluator will send completed evaluation instruments to: Heather Boyd, Evaluation Specialist by June 1, 2006 Preferably via E-mail: heather.boyd@uwex.edu Or Surface Mail to: 617 Extension building, 432 North Lake Street Madison, WI 53706 Office phone: 608-263-0857 D:\99026384.doc 4 WHAT I HAVE LEARNED FROM 4-H AFTERSCHOOL ABOUT COMMUNICATIONS Today’s Date: Please read each statement and think about your answer. We want your truthful answer. This is not a test, so there is no right or wrong answer. If you do not understand a question, raise your hand for help. Because of 4-H Afterschool Activities …. (Check one box for each statement.) YES NO NOT SURE a. I do not talk when others are talking. b. I listen when someone is talking to me. c. I tell people how I feel when they hurt my feelings. d. I apologize when I am wrong. e. I get along with people. f. I ask for help if I don’t understand something. g. [Sites can add other indicators specific to its Afterschool program] [Note to Evaluators: Don’t change the wording of any of the survey items, but remove any items on the instrument that are not relevant to your program. You can also add more specific indicators to the tools if you like. Delete this statement BEFORE you administer the survey to youth.] Tell us how you learned these communication skills in 4-H Afterschool: Thanks for completing this form. Please return it to the person who handed it to you today. The evaluator will send completed evaluation instruments to: Heather Boyd, Evaluation Specialist by June 1, 2006 Preferably via E-mail: heather.boyd@uwex.edu Or Surface Mail to: 617 Extension building, 432 North Lake Street Madison, WI 53706 Office phone: 608-263-0857 D:\99026384.doc 5 WHAT I HAVE LEARNED FROM 4-H AFTERSCHOOL ABOUT ACCEPTING DIFFERENCES Today’s Date: Please read each statement and think about your answer. We want your truthful answer. This is not a test, so there is no right or wrong answer. If you do not understand a question, raise your hand for help. Because of 4-H Afterschool Activities…. (Check one box for each statement. Evaluators may delete questions that don’t apply.) YES NO NOT SURE a. I play with kids who have a different skin color than I do. b. I have friends who have a different skin color than I do. c. I would invite a kid who has a different skin color than I do over to my house to play d. I play with kids with disabilities. e. I have friends with disabilities. f. I would invite a kid with a disability to my house to play. g. I do not pick on kids who are different from me. h. I let others play with me even if they aren’t very good at the game. i. [Sites can add other indicators specific to its Afterschool program] [Note to Evaluators: Don’t change the wording of any of the survey items, but remove any items on the instrument that are not relevant to your program. You can also add more specific indicators to the tools if you like. Delete this statement BEFORE you administer the survey to youth.] Tell us how you learned about accepting differences in 4-H Afterschool. Thanks for completing this form. Please return it to the person who handed it to you today. The evaluator will send completed evaluation instruments to: Heather Boyd, Evaluation Specialist by June 1, 2006 Preferably via E-mail: heather.boyd@uwex.edu Or Surface Mail to: 617 Extension building, 432 North Lake Street Madison, WI 53706 Office phone: 608-263-0857 D:\99026384.doc 6 WHAT I HAVE LEARNED FROM 4-H AFTERSCHOOL ABOUT HEALTHY LIFESTYLE CHOICES Today’s Date: Please read each statement and think about your answer. We want your truthful answer. This is not a test, so there is no right or wrong answer. If you do not understand a question, raise your hand for help. As a result of my participation in 4-H Afterschool Activities …. (Check one box for each statement.) YES NO NOT SURE a. I choose healthy snacks like vegetables and fruits. b. I wear a safety helmet when riding a bike. c. I talk to someone when I am upset or scared. d. I say “no” to others if things look dangerous to me. e. I play outside whenever I can. f. [Sites can add other indicators specific to its Afterschool program] [Note to Evaluators: Don’t change the wording of any of the survey items, but remove any items on the instrument that are not relevant to your program. You can also add more specific indicators to the tools if you like. Delete this statement BEFORE you administer the survey to youth.] Tell us about something healthy you learned in 4-H Afterschool. Thanks for completing this form. Please return it to the person who handed it to you today. The evaluator will send completed evaluation instruments to: Heather Boyd, Evaluation Specialist by June 1, 2006 Preferably via E-mail: heather.boyd@uwex.edu Or Surface Mail to: 617 Extension building, 432 North Lake Street Madison, WI 53706 Office phone: 608-263-0857 D:\99026384.doc 7 Evaluation Instrument B Interview Questions for Case Study Protocol Information sources: One youth and parent/guardian per site CASE STUDY EVALUATION Case Study Protocol Plan 30 minutes for each interview; it will probably take less time than this. We have left space for you to take notes right on this instrument. Remember, these questions are just a guide, so feel free to restate them in a way that best suits your audience. You may also need to ask additional probing questions such as: Could you tell me more about …? What do you mean …? Could you expand on that …? What else do you know about …? Right after the interview, or within a day of the interview, type the responses into this Word document and send them to Heather Boyd. It is best to interview the child and his or her parent/guardian (from the same family) at different times; at the very least, they should not be present in the same room during their respective interviews. However, keep the door open if you are alone in the room with the child. Our goal is to get one child/parent-guardian pair per site. Read this statement to participants before conducting the interview. Human Subjects Statement The name of this study is Life Skill Development in 4-H Afterschool. Kathi Vos, Ph.D., is the faculty person providing leadership to this study and can be reached at (608) 262-2496 if you have questions. The purpose of the study is to determine if youth who participate in Afterschool activities based on 4-H curriculum improve life skills such as communication, decision making, healthy lifestyle choices and accepting differences. We will not connect your name to your answers. It is up to you whether you want to participate or not. A copy of our Human Subject Protection Statement is on file at the UW-Extension Provost and Vice Chancellor’s Office at 432 N. Lake St., Madison, WI 53706. Your participation in this study implies your consent to participate. Thank you for helping us out. Information source: The Program Evaluator or person conducting the interview may need to ask the on-site coordinator to answer this question if they do not know the answer. Case Study Questions about Afterschool Site Please describe the 4-H Afterschool setting, including: Location, meeting space and how often 4-H Afterschool meets? D:\99026384.doc 8 Suggested Questions for Interviewing Youth Participant(s): Today’s Date: How old are you? What grade are you currently in? What did you do (this week, this session, this year) 1at 4-H Afterschool? What did you like the most of all the things that you did (this week, this session, this year)? What do you know now that you didn’t know before you came to 4-H Afterschool? What things do you do differently because of 4-H Afterschool? What do you tell your friends about 4-H Afterschool? What do you tell your parents about 4-H Afterschool? What do you think you will always remember about 4-H Afterschool? What else you would like to tell us about your involvement in 4-H Afterschool? The evaluator will send completed evaluation instruments to: Heather Boyd, Evaluation Specialist by June 1, 2006 Preferably via E-mail: heather.boyd@uwex.edu Or Surface Mail to: 617 Extension building, 432 North Lake Street Madison, WI 53706 Office phone: 608-263-0857 1 It is probably most valid to assess the current week, especially with younger children. D:\99026384.doc 9 Suggested Questions for Parent/Guardian of Youth Participant: Today’s Date: How old in your child that is involved in 4-H Afterschool? What grade are they currently in? How long has your child been involved in 4-H Afterschool? What has your child shared with you regarding 4-H Afterschool? What growth, if any, have you noticed in your child as a result of his/her participation in 4-H Afterschool? What, if any, examples can you share regarding improvements in your child’s -Communication skills? -Decision-making skills? -Ability to accept differences? -Ability to make healthy lifestyle choices? -Any other life skills? What differences have you noticed in your child’s activities as a result of his/her participation in 4-H Afterschool (for example, does he have a new hobby, does he read more, is he interested in insects or drama, etc. 2)? What else would like to tell us about your child’s involvement in 4-H Afterschool? The evaluator will send completed evaluation instruments to: Heather Boyd, Evaluation Specialist by June 1, 2006 Preferably via E-mail: heather.boyd@uwex.edu Or Surface Mail to: 617 Extension building, 432 North Lake Street Madison, WI 53706 Office phone: 608-263-0857 2 You may alter these to reflect offerings in your program. D:\99026384.doc 10