Running head: RIR STEM EDUCATION DAY PROGRAM EVALUATION RICHMOND INTERNATIONAL RACEWAY STEM EDUCATION DAY PROGRAM EVALUATION Submitted to the 2014 RIR STEM Education Day Capstone Committee: Dr. Barbara Driver Dr. Hollee Freeman Dr. Rosalyn Hobson-Hargraves By Ingrid Granberry Grant Seth Robert Hickerson Timothy Carter Lampe William Thomas Noel, Sr. Virginia Commonwealth University School of Education Monroe Campus Richmond, Virginia April 9, 2014 1 RIR STEM EDUCATION DAY PROGRAM EVALUATION 2 Acknowledgements The authors cannot express enough appreciation to Dr. Barbara Driver, VCU School of Education and Committee Chair, for her support and encouragement. We also offer our sincere gratitude to Dr. Hollee Freeman, Executive Director – MathScience Innovation Center, and Dr. Rosalyn Hobson-Hargraves, VCU School of Engineering. We are extremely grateful for your time, commitment, and expertise on this project. Special thanks and love go to: --------------------My daughters Sekani and Allhana Grant and special friends Milton Rawles and Shanika Robinson. I appreciate your encouraging words and support, I love you. Dr. Ingrid Granberry Grant --------------------My wife Alison Noel, daughter Micah Moss Jackson, and son William Noel, Jr. I cannot thank you enough for your patience and support. I love you all. Dr. William T. Noel, Sr. --------------------Brenna Lampe, Nichole, Carter, and Brayden Smithson I love you more than words can express…thank you for being there for me! Dr. Timothy C. Lampe --------------------To my wonderful and supportive wife Dr. Lindsay Hickerson and our amazing son Hudson. Thank you for your confidence in me throughout this journey. Dr. Seth R. Hickerson RIR STEM EDUCATION DAY PROGRAM EVALUATION 3 Table of Contents Section One - Introduction ............................................................................................................8 Context .......................................................................................................................................11 Historical Considerations .......................................................................................................12 Policy Considerations .............................................................................................................13 Logic Model ...........................................................................................................................15 Conclusion ..................................................................................................................................17 Section Two – Literature Review ...............................................................................................18 Improving the Appeal of STEM.................................................................................................19 Student Perceptions ................................................................................................................20 Student Motivation .................................................................................................................21 Student Engagement ...............................................................................................................23 Student Exposure to STEM ....................................................................................................25 Female and Minority Students in STEM....................................................................................27 Supporting Female and Minority Students in STEM .............................................................28 An Identifiable Role Model ....................................................................................................29 A Supportive Role Model .......................................................................................................29 Providing a Positive STEM Classroom Experience ...............................................................30 Collaboration Needed .................................................................................................................31 The Importance of University Partnerships ............................................................................33 Business and Community Partnerships ..................................................................................35 Collaborative Efforts ..............................................................................................................36 Conclusion ..................................................................................................................................36 Section Three - Methods ..............................................................................................................38 Evaluation Focus ........................................................................................................................38 Evaluation Design ......................................................................................................................39 Participant Characteristics ..........................................................................................................40 Data Collection ...........................................................................................................................41 Quantitative ............................................................................................................................41 Survey ................................................................................................................................42 Participants .........................................................................................................................42 RIR STEM EDUCATION DAY PROGRAM EVALUATION 4 Instrument ..........................................................................................................................42 Data Analysis .....................................................................................................................43 Qualitative ..............................................................................................................................44 Focus Groups .....................................................................................................................45 Teacher Focus Groups .......................................................................................................46 Teacher Participants ...........................................................................................................46 Teacher Focus Group Questions ........................................................................................47 Teacher Data Analysis .......................................................................................................47 Student Focus Groups ........................................................................................................48 Student Participants ...........................................................................................................48 Student Focus Group Questions.........................................................................................49 Student Data Analysis ........................................................................................................50 Student Survey Comments .....................................................................................................50 Data Analysis .....................................................................................................................50 CTE Teacher Feedback ..........................................................................................................50 Data Analysis .....................................................................................................................51 Strategies for Enhancing Credibility and Dependability............................................................51 Potential Limitations ..................................................................................................................52 Time Lapse .............................................................................................................................52 Teacher Retention ...................................................................................................................52 Student Survey........................................................................................................................53 Section Four - Findings ...............................................................................................................54 Themes ..................................................................................................................................55 Theme 1: Awareness ..........................................................................................................56 Theme 2: Making Connections ..........................................................................................57 Theme 3: Role Models .......................................................................................................58 Theme 4: Time ...................................................................................................................59 Summary of Themes .............................................................................................................60 Answering the Evaluation Questions .............................................................................................60 Evaluation Question One .......................................................................................................61 RIR STEM Education Day Field Trip Survey ...................................................................61 RIR STEM EDUCATION DAY PROGRAM EVALUATION 5 CTE Teacher Feedback ......................................................................................................62 Student Focus Groups ........................................................................................................62 Teacher Focus Groups .......................................................................................................63 Evaluation Question Two .......................................................................................................64 RIR STEM Education Day Field Trip Survey ...................................................................64 CTE Teacher Feedback ......................................................................................................65 Student Focus Groups ........................................................................................................66 Teacher Focus Groups .......................................................................................................66 Evaluation Question Three .....................................................................................................68 RIR STEM Education Day Field Trip Survey ...................................................................68 Student Focus Groups ........................................................................................................69 Teacher Focus Groups .......................................................................................................69 Evaluation Question Four.......................................................................................................70 RIR STEM Education Day Field Trip Survey ...................................................................71 Student Focus Groups ........................................................................................................72 Teacher Focus Groups .......................................................................................................72 Evaluation Question Five .......................................................................................................73 Student Focus Groups ........................................................................................................74 Teacher Focus Groups .......................................................................................................74 Summary ........................................................................................................................................76 Section Five - Recommendations ................................................................................................78 Evaluation Question 1 ............................................................................................................78 Recommendation 1a. Increase Hands-on Activities ..........................................................79 Recommendation 1b. Invite Non-CTE Students ...............................................................79 Evaluation Question 2 ............................................................................................................80 Recommendation 2a. Provide Teachers with Pre and Post Event Materials .....................80 RIR STEM EDUCATION DAY PROGRAM EVALUATION 6 Recommendation 2b. Collaborate with Local Education Organization ............................81 Recommendation 2c. Allocate More Time at Hands-on Stations......................................82 Evaluation Question 3 ............................................................................................................82 Recommendation 3. Invite 6th or 7th Graders .....................................................................82 Evaluation Question 4 ............................................................................................................83 Recommendation 4a. Collaborate with Local Businesses .................................................83 Recommendation 4b. Expose Students to Career Options ................................................84 Evaluation Question 5 ............................................................................................................84 Recommendation 5. Continue to Highlight Females and Minorities ................................85 Summary of Recommendations .................................................................................................86 Recommendations for Future Evaluations .............................................................................87 Conclusion .................................................................................................................................88 References .....................................................................................................................................91 Appendices Appendix A CTE 8403 Block Demographics ...............................................................................98 Appendix B 8403 Course Description and Competency-Based Task List ..................................100 Appendix C RIR STEM Education Day Field Trip Survey ........................................................105 Appendix D Student Survey Response Percentage Conversion ..................................................125 Appendix E Student Survey Questions ........................................................................................127 Appendix F RIR STEM Education Day Program Floor Diagram ...............................................128 Appendix G Coded Student Survey Comments...........................................................................129 Appendix H Coded CTE Teacher Feedback ................................................................................134 Appendix I Evaluation Questions ................................................................................................135 Appendix J Teacher Focus Group Interview Questions ..............................................................136 Appendix K Student Focus Group Interview Questions..............................................................137 RIR STEM EDUCATION DAY PROGRAM EVALUATION 7 Appendix L Focus Group Transcriptions ....................................................................................138 Appendix M Focus Group Key Words and Phrases ....................................................................188 Appendix N How Data Relates to Evaluation Questions ............................................................203 Appendix O Focus Group Protocol..............................................................................................208 Appendix P Teacher Focus Group Script ....................................................................................211 Appendix Q Program Evaluation Subject Consent Form ............................................................213 Appendix R Program Evaluation Parental Permission Form ......................................................214 Appendix S Program Evaluation Student Assent Form ...............................................................215 List of Tables Table 1. RIR STEM Education Day - Logic Model ......................................................................16 Table 2. Focus Group Themes and Associates Evaluation Questions Components......................60 Table 3. Data Sources: Evaluation Question One ..........................................................................61 Table 4. Data Sources: Evaluation Question Two .........................................................................64 Table 5. Data Sources: Evaluation Question Three .......................................................................68 Table 6. Data Sources: Evaluation Question Four .........................................................................71 Table 7. Data Sources: Evaluation Question Five .........................................................................73 List of Figures Figure 1. Sequential Mixed Methods Design ................................................................................39 Figure 2. Sequential Steps for Collecting Data ..............................................................................40 RIR STEM EDUCATION DAY PROGRAM EVALUATION 8 SECTION ONE Introduction In an effort to increase students’ interest and literacy in science, technology, engineering, and math (STEM), the idea of creating an Educational Day program at Richmond International Raceway (RIR) was born. The primary focus was to create a program that would provide a mechanism to connect classroom theory with real-world relevance through motorsports. The goal of the RIR STEM Education Day program was to provide a program that would build interest in STEM, as well as inspire and motivate middle school students to learn and understand STEM through motorsports. The focus of this evaluation was to determine to what degree the RIR STEM Education Day program increased interest in STEM, improved understanding of STEM, increased interest in STEM educational and career opportunities, and if there are differences in STEM interest and understanding based on gender and ethnicity. In researching STEM education there are varying definitions. For the purpose of this program evaluation, the working definition of STEM is an education that provides students with science, technology, engineering, and math in sequences that build upon each other and used with real-world applications (Eberle, 2010). Furthermore, “STEM should be taught in an integrated and cross-curricular manner, not just in ‘silos’ where the individual subject areas dominate and the other subjects are only afterthoughts” (Dugger, 2011, p. 2). RIR’s inaugural STEM Education Day program took place on April 25, 2013. Collaborating with a local suburban school district, local university, and Arena Racing USA, RIR worked with eighth grade students from all twelve middle schools where they participated in a twenty-station program that introduced students to numerous STEM aspects through the National Association for Stock Car Auto Racing (NASCAR) and Arena Racing. The RIR STEM EDUCATION DAY PROGRAM EVALUATION 9 evaluation of this program was the first of its kind and included post-program survey responses, student write-in comments on the post-program survey, teacher feedback immediately following the event, and focus group interviews with students and teachers who participated in the RIR STEM Education Day program. The evaluation team employed Stake’s (1975) Responsive evaluation model for this evaluation. Responsive evaluation supports observing how the program works and requires getting to know stakeholders at various levels of the program. The program was observed on April 25, 2013 and numerous conversations and informal meetings were held with key RIR administrators to determine what questions needed to be answered by this evaluation. This mutual process of determining evaluation questions occurred primarily to gain a better understanding of what RIR and the program stakeholders wanted to know. For a responsive evaluation to be effective, the evaluation process needs to be responsive to the concerns of the stakeholders for the main purpose of understanding the program (Mertens & Wilson, 2012). The focus of this evaluation was to determine to what degree the RIR STEM Education Day program increased interest and improved understanding of STEM, STEM educational and career opportunities, and if there were differences in STEM interest and understanding based on gender and ethnicity regarding the RIR STEM Education Day program. The following questions emerged from extended discussions with key administrators and a thorough review of literature: 1. To what degree did the RIR STEM Education Day program increase middle school students’ interest in STEM education? 2. To what degree did the RIR STEM Education Day program improve students’ understanding of STEM education? 3. To what degree did the RIR STEM Education Day program increase students’ interest in STEM educational opportunities? RIR STEM EDUCATION DAY PROGRAM EVALUATION 4. To what degree did the RIR STEM Education Day program increase students’ interest in STEM-related careers? 5. Are there differences in STEM interest and understanding based on gender and ethnicity regarding the RIR STEM Education Day program? 10 After-school and community-based events similar to the RIR STEM Education Day program have been effective in helping students make the connection between STEM-related classroom content and the real-world application of STEM education. “These programs present innovative ways to engage students who may be detached and disinterested during the traditional school day” (Afterschool Alliance, n.d.). By making this connection through early exposure to STEM and through exciting hands-on activities, it is anticipated that middle school students’ interest and engagement in STEM-related education will persist throughout their high school and college years. This program evaluation allowed for the hermeneutic process to transpire which is needed in order to generate discourse between program participants. From an ontological perspective, this is an important design element because multiple realities from the RIR STEM Education Day participants, within various levels of the program, needed to be uncovered to appropriately evaluate the program from diverse viewpoints. Using the evaluation questions as a base-point, actual focus group questions were formulated that will bring these different viewpoints forward. The sequential mixed methods design was utilized for this evaluation. This design allowed for the analysis of the quantitative data portion of the evaluation and then using the discoveries from this analysis to inform the qualitative design portion of the evaluation (Mertens & Wilson, 2012). The rationale behind the use of sequential mixed methods design for this evaluation was to integrate the qualitative data from the focus group interviews, student RIR STEM EDUCATION DAY PROGRAM EVALUATION 11 comments, and teacher feedback with the quantitative data collected from student surveys. Context The idea of developing the RIR STEM Education Day program was initiated during the summer of 2012. With the help of Arena Racing, USA, a local school district, the Center for Sport Leadership at Virginia Commonwealth University, and the Richmond International Raceway organization, the first RIR STEM Education Day program was created and then implemented on April 25, 2013. What started out as a quest for ideas to promote the sport of auto racing, specifically NASCAR, original RIR marketing concepts turned to educational concepts primarily through the vision of Arena Racing, USA’s Ricky Dennis. RIR was looking for ideas to get a younger and more diverse fan base interested in motorsports and Arena Racing was looking for a way to tie the science, technology, engineering, and math of the sport in a way that would appeal to youth groups through the sport of arena racing. These ideas brought the two organizations together in a unified effort with help from the Center for Sport Leadership at VCU. RIR is a well-established organization with deep community ties because of events such as Richmond International Raceway Cares, Marine Toys for Tots, and other notable programs. Due to their long-standing community-based mindset, RIR quickly saw the value and benefit of an educational-based program such as the Education Day idea, and how it could help them get in front of a younger and more diverse audience. Through a pre-existing relationship, RIR solicited and received the help and support from the superintendent of a local school district, establishing and solidifying a robust collaborative group that implemented the first RIR STEM Education Day program. RIR STEM EDUCATION DAY PROGRAM EVALUATION 12 To garner a more broad ranging appeal to middle-school aged students and to give the program maximum visibility, Education Day moved under the auspices of RIR with Arena Racing, the Center for Sport Leadership at VCU, and a local school district as collaborative partners to help bring the program from the idea stage to the implementation stage. The location would be the Richmond International Raceway Complex and the date would be during raceweek when the community’s attention on racing would be at its peak. Historical Considerations. Even before the acronym STEM was coined by Dr. Judith Ramaley, former Assistant Director of the Education and Human Resources Directorate at the National Science Foundation (Chute, 2009), the topic of STEM had been a hot-button topic among educators and government officials. Previously known as SMET, the former acronym was not as vocally appealing to say as STEM, nor did it carry the underlying important message that the acronym STEM supports. The former term implies that science and math came first or were more important; however, the term STEM suggests a meaningful connection among the components and demonstrates how the math and science disciplines support the other two disciplines of technology and engineering (Chute, 2009). Regardless of which term is used, the issue in the U.S. education system remains the same; a better method of educating America’s youth has to be identified if the United States is to compete in the global economy in the future. Awareness of this issue became prevalent during the Clinton administration and gained momentum throughout the Bush and Obama administrations as it became more and more obvious that the educational system in the United States was not adequately preparing our youth for STEM-related education, degrees, and careers. As the United States searches for innovative ways to educate its youth through creative and RIR STEM EDUCATION DAY PROGRAM EVALUATION 13 collaborative teaching methods, RIR decided to bring its rich background of science, technology, engineering, and math disciplines to the classroom. Richmond, Virginia was already a part of the racing scene as early as 1946 when the Atlantic Rural Exposition Fairgrounds, currently known as the Richmond International Raceway Complex, opened in Henrico County, Virginia. Richmond hosted its first NASCAR race in 1953 (Repsys, 2009). Since that first NASCAR race, RIR has grown into one of the premier NASCAR race tracks in the United States. RIR’s notoriety, combined with the organization’s commitment to its local community, makes RIR a perfect platform from which to engage middle school students and to get them interested in STEM education and careers. With the evolution and the worldwide popularity of the sport, the potential learning opportunities motorsports can provide are virtually limitless. Policy Considerations. In November of 2009, the Obama administration announced the launch of the “Educate to Innovate” campaign to increase students’ interest and improve their understanding of STEM subjects. The goal of the campaign was to help improve American students’ test scores and to be among the best in the world in science and math achievement by 2020. To help achieve this goal, President Obama announced numerous partnerships designed to unite major companies, universities, foundations, non-profit organizations and government agencies on a mission that would attract, develop, reward, and retain outstanding educators in science, technology, engineering, and mathematics. In 2013, the United States Senate Committee on Health, Education, Labor, and Pension’s (HELP) supported the Strengthening America’s Schools Act, a bill that supports the Elementary and Secondary Education Act. United States Senate support effectively demonstrated that the nation was trying to move forward with legislation to improve our K-12 schools. “The RIR STEM EDUCATION DAY PROGRAM EVALUATION 14 Strengthening America’s Schools Act clearly recognizes that STEM education is closely linked with our nation’s economic prosperity in the modern global economy and that strong STEM skills are a central element of a well-rounded education” (Brown, 2013, p.1). Elevating STEM education as a national priority through education reform policies creates a much needed urgency toward innovation and federal spending. In the same year, the Senate also passed STEM legislation as a part of the Border Security, Economic Opportunity, and Immigration Modernization Act (S.744), better known as the Immigration Bill, further validating the overwhelming importance STEM plays in today’s education policies. The STEM Education Coalition, an organization that works to support STEM programs for teachers and students at the U.S. Department of Education, the National Science Foundation, and other agencies that offer STEM related programs, acknowledged that the immigration reform bill contained requirements that supported systematic improvements in STEM education. “Our nation’s future depends in great measure on our ability to prepare American students to succeed in the best, high-paying jobs of the future, which are increasingly in STEM fields” (Brown, 2013, p. 1). The bill supported improvements in U.S. STEM education to prepare students for the best jobs in order to improve American global competitiveness. In addition to the two examples of Senate support, the STEM Education Coalition lists all legislative updates being sponsored by each state on its website. In May 2013, there were 17 states sponsoring legislation in support of STEM education. A total of 64 STEM-related legislative bills were being introduced in state legislatures nationwide, with only New York (8) and Texas (10) sponsoring more STEM-related bills than the Commonwealth of Virginia (7). Logic Model. To clearly identify the diverse resources, various activities, and the desired outputs of the program, it was necessary to establish a logical framework, or logic model. RIR STEM EDUCATION DAY PROGRAM EVALUATION 15 The design of the logic model for the RIR STEM Education Day program exhibits how several different components work together to accomplish desired outcomes (Mertens & Wilson, 2012). The “Inputs” in this logic model show key resources such as personnel, organizations, funding, and data. Items appearing as inputs are those that were required to make the program work and come from each of the various organizations involved in the implementation of the RIR STEM Education Day program. The “Activities” reflect the actual day, as well as any lead-up activities that took place in order to create and implement the program. The activities listed are comprehensive in nature and consider many of the ancillary activities such as sponsorship package presentations, the collection of items for gift-bags, and the selection and planning of the program location, which are in addition to the actual program day activities. The combined inputs and activities establish the “Outputs” in the logic model. These items include the participants from middle schools, various staff and administrators from each of the participating organizations, as well as survey data and teacher feedback received from middle school teachers and administrators. “Outcomes” in the logic model display the expected changes resulting from the RIR STEM Education Day program. These expected changes included increased awareness and interest in STEM, motivating middle school students to pursue STEM education, and survey data results. There are immediate outcomes from the program as well as the anticipated outcomes after repeated exposure to STEM-related programs like the RIR STEM Education Day. The “Impact” in this logic model lists the expected long-term influence of the RIR STEM Education Day program. Long-term impact of the program includes, but is not limited to, a sound understanding of STEM, an improved performance in STEM related courses, an increased RIR STEM EDUCATION DAY PROGRAM EVALUATION 16 enrollment in STEM electives, an increased pursuit of STEM-related degrees and/or careers, and a possible increase in earning potential and therefore an improved standard of living. Logic Model Table 1 RIR STEM Education Day - Logic Model Inputs RIR Arena Racing, USA Local School District Central Office, 12 middle schools (CTE 8th grade classes) Activities Outputs Activity stations 500 student Interest in participants from STEM 12 middle schools Live track time Motivation to 50 local school pursue STEM district education Lead-up staff/admin activities (Arena Survey data Racing/CSL) RIR staff/admin results Sponsor package presentation Thirty-five CSL student/program coordinators Collection of gifts for “gift bags” Twenty activity stations CSL @ VCU Virginia 529 (financial) Outcomes Program/space planning for stations Lunch/food donations Transportation coordination Yes/No data from student survey Student survey comments Teacher feedback Improve the appeal and perception of STEM Increased awareness of STEM in everyday life Impact Understanding of STEM Improved performance in STEM related courses Increased enrollment in STEM electives Personal increased earning potential Improved standard of living STEM-related degrees and/or careers RIR STEM EDUCATION DAY PROGRAM EVALUATION 17 Conclusion The goal of the RIR STEM Education Day program was to provide real-world relevance to classroom curriculum by creating hands-on experiences through motorsports. The focus of this evaluation was to determine to what degree the April 25, 2013 RIR STEM Education Day program increased interest and understanding of STEM, STEM educational and career opportunities, and if there were differences in STEM interest and understanding based on gender and ethnicity regarding the RIR STEM Education Day. Taking into account the context, historical background, and goals of the client, the evaluation team employed a responsive evaluation consisting of the collection and analysis of quantitative and qualitative data in a sequential mixed method design that helped answer the evaluation questions. RIR STEM EDUCATION DAY PROGRAM EVALUATION 18 SECTION TWO Literature Review In a letter to President Barack Obama from the National Science Board, Chairman Steven C. Beering (2009) wrote, “Our national economic prosperity and security require that we remain a world leader in science and technology. Pre-college STEM education is the foundation of that leadership and must be one of our highest priorities as a Nation.” The definition of STEM education is an education that provides students with science, technology, engineering, and math in sequences that build upon each other and are used in real-world applications (Eberle, 2010). Chairman Beering went on to write that “it is essential that we act now to ensure all of our children and American society as a whole can continue to prosper in the 21st century technologybased economy” (Beering, 2009, p. 1). Beering’s comments underscore the immense and long-term importance of STEM education for all “pre-college” (high school) students; however, if the United States is to continue to be a world leader in technology innovation then Beering’s use of the word ‘precollege’ may not be specific enough. For example, a Metlife Foundation Afterschool Alert (September 2010), stated that “middle school is a ‘make or break’ time for young adolescents academically” and that “it is during middle school that many youth…turn away from math and science” (Metlife Afterschool Alert, 2010, p. 2). The report went on to emphasize the importance of exposure to STEM education during middle school years (Metlife Afterschool Alert, 2010). Strengthening this view, a study by Whitehead (2011) noted that positive attitudes regarding STEM education decreased for eighth grade students because of the disconnection between classroom practices and real-world applications. Yet another pitfall of postponing RIR STEM EDUCATION DAY PROGRAM EVALUATION 19 STEM education until high school is that students do not have adequate time to acquire the prerequisite skills necessary to be successful in college courses that lead to STEM careers (Whitehead, 2011). The RIR STEM Education Day experience provided middle school students with an introduction to STEM-related activities through exposure to the field of motorsports. This literature review examined literature on STEM education and programs that effectively promote interest and understanding of middle school students related to STEM. Key words and phrases (collaboration in STEM, what defines STEM, STEM careers, STEM and motivation, STEM middle school interest) were used for researching in the following databases; Educational Resources Information Center (ERIC), Elton B. Stephens Company (EBSCO) Host using SPORTDiscus, Education Research Complete, and the American Psychological Association (Psych NET), as well as peer-reviewed articles presented through Google Scholar. A graphic organizer was used to list the research articles with key words for use in the literature review and to analyze and derive themes from the literature. Three overarching themes emerged from this process: 1. Improving the Appeal of STEM 2. Female and Minority Students in STEM 3. Collaboration Needed Improving the Appeal of STEM Why is STEM education so important for 21st Century education? What is the reasoning behind the “push” for increasing STEM awareness? How can we get our students to become more enthusiastic about STEM? The answers to these questions may lie within the research and perhaps provide a guide to the development and implementation of STEM intervention programs for the future. RIR STEM EDUCATION DAY PROGRAM EVALUATION 20 Student Perceptions Students may begin developing less than positive perceptions of science, technology, engineering, and math as early as middle school and perhaps this disinterest could ultimately lead to a shortage in STEM workers for the United States. Many researchers feel the need to move from less content driven and lecture style teaching to more of an inquiry based approach and group projects if a stronger connection to STEM is to be achieved. To illustrate this point, a study by McNally (2012) examined innovative approaches to augmenting science lessons taught in middle school by delving deeper into STEM initiatives and programming with the intention of changing the perception of STEM to make it more understandable, relatable, and appealing to young students in our school systems. McNally stated, “U.S. Science education begins primarily during middle school. Studies show that this is typically where we start to see test scores begin to wane, where the emphasis shifts from the elementary schools focus on animals, weather, and the local landscape, to lab experiments and difficult terminology, that is perceived to be distant from everyday life” (McNally, 2012, p. 50). The traditional lecture style of teaching may be partly to blame because of the lack of creativity and student engagement, that may diminish student perceptions that they are capable of understanding STEM. Reaffirming this notion for a shift in teaching styles, a study by the National Educational Longitudinal Study (NELS) cited attitudes formed in early adolescence, primarily from out of school experiences, as the single most important influence on the choice of a science career (Falk & Dierking, 2010). Student Motivation Motivation to learn is crucial to influencing both student engagement and achievement (Jang, Reeve, & Deci, 2010). Accordingly, course content should connect directly to students’ RIR STEM EDUCATION DAY PROGRAM EVALUATION 21 interest and to what motivates them to learn. If attitudes regarding STEM are to change then the natural conclusion is that teachers must work toward making curriculum and lessons relevant to students. “Through collaboration between STEM content areas and activities that are relevant, students are able to observe the natural correlations between STEM areas and the everyday applications” (Whitehead, 2010, p. 27). To demonstrate this point, a Faculty Learning Community in Western Massachusetts took the initiative to make a stronger connection with their students by integrating active-learning methods into course design by revamping their professional development-training program. By developing a learning community and teacher-preparation model, schools can create substantial change in how their faculty teaches, and as a result, improve learning for all involved. To evaluate this, O’Meara (2007) assessed the effects that the Faculty Learning Community had on improving student learning and concluded that the program had significant positive effects on teachers’ familiarity and use of active-learning methods, and that student learning substantially improved by making the changes to their courses that incorporated relevant projects. The result of the assessment demonstrated that student motivation, as it relates to STEM, can be enhanced if effective active-learning methods and relevancy were added to course curriculum and faculty are accordingly trained. Gannod, Burge, and Helmick (2008) further validated this premise through what they referred to as an “inverted classroom” where learning activities, such as reviewing new content typically done in class, are done outside of class thus freeing up time for in-class discussion and hands-on, active engagement. Conversely, passive activities such as listening to lectures are performed outside of the classroom through platforms such as online voiceover PowerPoint RIR STEM EDUCATION DAY PROGRAM EVALUATION 22 presentations. The result is valuable and engaging teacher ‘face’ time with students when they are involved in in-depth learning activities. This hands-on style of teaching invariably takes the emphasis off the teacher and places it appropriately on the student. Because of this, the focus is no longer on lectures and notes but on active-learning activities and hands-on applications that result in improved student-teacher interaction and stronger reinforcement of STEM. Active-learning activities, hands-on applications, and increased student-teacher interactions are essential ingredients that help lead to a more thorough understanding of STEM and increase students’ motivation in learning about STEM-related careers. Additional methods to increase motivation lie within the context of social frameworks and cognitive theories. For instance, a study by Bryan, Glynn, and Kittleson (2011) set out to see what motivates girls and boys ages 14-16 to learn science in their introductory science courses in high school. The researchers worked from the framework of social cognitive theory to assess intrinsic motivation, self-efficacy, and self-determination, as these are all attributes that should foster a positive perception of STEM if addressed accordingly. To better understand the Bryan Study, social cognitive theory, intrinsic motivation, selfefficacy, and self-determination are defined as follows: social cognitive theory supports that learning can be done through observing others behaviors; intrinsic motivation refers to one’s internal drive to succeed or improve; self-efficacy refers to one’s ability to successfully perform a given task; and self-determination refers to one’s ability to make decisions without external influence. The Bryan study, conducted in a suburban public high school located in the Southeastern United States, included 288 first and second year students consisting of 146 females and 142 RIR STEM EDUCATION DAY PROGRAM EVALUATION 23 males. The participants completed a three part online survey with each part of the survey asking questions that addressed the areas of intrinsic motivation, self-efficacy, and selfdetermination. Researchers found that of the three components, self-efficacy was the one most related to achievement and motivation. Additionally, they found that many of the students had aspirations of becoming science teachers or pursuing a career in STEM but were lacking sufficient learning opportunities to enhance their interest. In order to change student’s perception of STEM, the findings suggested that science teachers use social modeling, which provides observable behaviors where students are more likely to replicate the actions of the classroom teachers. Collaborative learning activities and interactive engagement on common activities also help enhance students’ interest in STEM education and careers. The understanding is that the perceptions of middle school students concerning STEM can improve if educators address the three cognitive areas of intrinsic motivation, self-efficacy, and self-determination. Student Engagement An additional effective strategy for increasing positive perceptions of young students, and to change negative perception that science is boring or too difficult, is to make learning experiences engaging. If students are enjoying a task or looking forward to working on an assignment then the likelihood of them gaining a better understanding of the subject at hand increases (Angle, 2011). Teachers should utilize pedagogical strategies that focus on projectbased learning that concentrate on games and manipulatives that are creative, while blending a real-world element into the curriculum or instructional activity. Teachers can promote student engagement and make it fun in numerous ways. In particular, the following two examples illustrate principles and techniques of effective RIR STEM EDUCATION DAY PROGRAM EVALUATION 24 pedagogical strategies related to project-based learning and real-world application. One unique classroom teaching activity described by Angle (2011) had teachers and students build stock cars out of pasta and hot glue. The goal for the students was to learn about physics concepts that deal with motion by building a car that could travel down a ramp. By planning and constructing their stock cars using only the pasta at hand, they not only needed to think about how it would look, but also how it would move (Angle, 2011). Activities such as this might better stimulate the interest of young learners and, as a result, get them to inquire more about STEM and ultimately facilitate a change in attitudes. In a second example described by Wilcox, Roberts, and Wilcox (2010), a group of educators used a similar technique to capitalize on students’ natural curiosity about the world around them by exploring the concept of motion. Through implementing a weeklong series of STEM activities using a 21st century instructional approach, they encouraged the students to explore, explain, engage in, and elaborate on the project. Students received multiple objects that had different moving parts and were challenged to build a “gravity racer” that would successfully travel down a ramp. The goal was to have the students realize that every movement is caused by a push or pull and to develop a basic understanding of the concepts of inertia and friction. Breaking down the science of this experiment into related terminology and practices may be an effective way to engage students and to motivate them to pursue STEM education. For example, Angle (2011) stated, that student learning is enhanced when they are given ownership to test their own ideas during inquiry-based instruction. RIR STEM EDUCATION DAY PROGRAM EVALUATION 25 Student Exposure to STEM In order to increase the number of young people entering the job market with the appropriate technological literacy and skills, exposure to STEM education may motivate students to be interested in STEM as it relates to real life experiences. Getting students excited is a key to getting students engaged. In his article Technology Education, Lacey (2010) discussed the importance of getting students excited about STEM. According to Lacey, just 4% of American college graduates major in engineering, compared with 13% of students in Europe, and 20% of students in Asian countries. According to Lacey, educators must inspire and convince students in a fun and meaningful way of the value and benefits of working with STEM before they leave the K-12 environment. The lecture-based style of teaching may not work for many students; therefore, educators and programmers need to be creative with the development of their plans for teaching STEM. Exposing students at an early age to STEM and providing them with varying teaching and learning environments could prove beneficial. “Schools have the power to create the conditions under which students can achieve highly; become motivated for learning” (ASCD, 2009; Yassie-Mintz, 2010, p. 55). Providing exposure through non-traditional teaching can be done in numerous ways. In one example described by Hylton (2010), a teacher at Indiana University-Purdue University in Indianapolis (IUPUI), Indiana taught STEM-related concepts by connecting them with motorsports through the development of a program involving both motorsports engineering and motorsports technology. RIR STEM EDUCATION DAY PROGRAM EVALUATION 26 Hylton (2010) demonstrated the significance that topics such as friction, traction, speed, force, torque, and energy-absorption can have if implemented into the curriculum by using reallife motorsports examples in conjunction with STEM course material. For instance, by demonstrating the use of a mechanical advantage to lift a car, via a mechanical jack, demonstrated that the smallest of students in the class could lift an object that was too heavy for the strongest students. Examples such as this can positively affect student awareness and interest in STEM topics and can be an excellent way to connect middle school students to various STEM concepts and possible careers (Hylton, 2010). A second example is of a similar program with a focus on robotics, conducted in Pennsylvania, that had the goal of improving middle school students’ attitudes toward three of the four STEM areas, technology, engineering, and mathematics (TEM) through inquiry based learning. Instead of introducing STEM to students through motorsports, this program introduced TEM by implementing robots in the classroom to enhance curriculum understanding. The idea was to improve the educational experience while learning information related to technology, engineering, and mathematics with the hope of also increasing the classroom inquiry level. To determine whether this program was effective, a study by Whitehead (2011) focused on robotics and student interest in which the goal was to demonstrate how robotic activities designed around math content could influence middle school students’ interest in TEM. Whitehead’s study investigated whether or not the use of robotics, along with collaboration between mathematics and technology education teachers, could aid in curtailing the documented decline in students’ interest toward TEM from middle school forward. Nine schools participated in the study comprised of 107 students (71 males and 36 females). Data was collected over one semester from classrooms of 18 teachers who also agreed RIR STEM EDUCATION DAY PROGRAM EVALUATION 27 to participate in the study. Data analysis of 107 completed pre and post surveys indicated that middle school students developed a positive interest in technology, engineering, and math. The findings revealed that robotic centered activities along with classroom collaboration can aid to reverse middle school students’ negative beliefs and interest towards TEM. Female and Minority Students in STEM The United States has experienced a steady decline in students pursuing an education or career with an emphasis in science, technology, engineering, and math (STEM). Of those students who did pursue STEM-related education and careers, relatively few are female or minority students. This underrepresentation of females and minorities in American STEM education is a contributing factor to our nation’s decline as a global leader; therefore, the task for educators in the United States is to increase female and minority students’ interest to pursue STEM education by providing support, and to making it a positive experience. America’s overall participation in STEM education has been on a decline, and female and minority students are particularly scarce in STEM-related education and careers. For instance, the number of African-American students in the areas of engineering and technology has declined when compared to other ethnicities (STEM Connector, 2012). In addition, female students are three times less likely to pursue STEM-related majors and careers when compared to male students. This information serves as a call to action for educators and policy makers to promote the importance of STEM education and careers from K-12 education to the university level. To promote STEM-related subjects to a wider audience, organizations like the Virginia Advanced Study Strategies (VASS) exist in order to increase the number of female and minority students who participate in Advanced Placement (AP) classes (Nichols, 2008). Participation in RIR STEM EDUCATION DAY PROGRAM EVALUATION 28 advanced classes in science, technology, and math help prepare tomorrow's workforce by shifting the focus of K-12 education from meeting minimum competencies to a culture of advanced academics. This culture shift promotes rigorous career and college academia and skill development in K-12, especially in the areas of STEM education (VASS Website). Supporting Female and Minority Students in STEM Research indicates that female and minority students have been discouraged at some point from pursuing STEM careers (Bayer Corporation, 2012). A Bayer Corporation online survey of 413 STEM department chairs found that, “40% of the country’s female and underrepresented minority (URM) chemist and chemical engineers working in their field today faced discouragement from pursuing their STEM career at some point in their lives” (Bayer Corporation, 2012, p. 323). Rather than discouraging female and minority students from participating in STEM education, educators would be wise to recruit these two missing demographics. In an article by Kramer (2011), First Lady Michelle Obama stated, “If we’re going to outinnovate and out-educate the rest of the world, we’ve got to open doors for everyone. We need all hands on deck, and that means clearing hurdles for women and girls as they navigate careers in science, technology, engineering, and math.” To increase female and minority representation in STEM education, the Obama Administration introduced a number of initiatives such as Race to the Top, Educate to Innovate, and Change the Equation. If successful, these initiatives will have a favorable impact on America’s STEM future; however, these initiatives alone may not be enough to ensure female and minority students’ success in STEM. Females and minorities benefit from seeing and having supportive teachers who act as role models. A role model, in this context, is not only someone who female and RIR STEM EDUCATION DAY PROGRAM EVALUATION 29 minority students can easily identify with, but also as someone who possesses high expectations for student success. An identifiable role model. Female and minority students are scarce among STEM majors, and female and minority teachers in STEM are practically non-existent (Beasley & Fischer, 2012). If female and minority students do not have role models in which to make a connection, they stand a chance of not receiving the necessary support for success. According to Stephanie Hill (2013), president and general manager of Lockheed Martin's Information Systems, it is important for students in STEM to have a stimulating curriculum, outstanding teachers, and real-world experiences; however, another important piece of the puzzle for female and minority students interested in STEM is to have role models who "look like them". Lockheed Martin supports K-12 STEM education across the nation, but a specific example is in Baltimore County, Maryland, where the company committed to send “STEM ambassadors” into middle schools to collaborate with teachers, visit classrooms, and provide that career role model that many young students are seeking (Hill, 2013). According to Reeve and Jang (2006), teachers who serve as role models and who create positive interpersonal relationships with their students, produce academic and developmental benefits for their students. A supportive role model. To demonstrate the significance of a supportive role model, a study by Garibaldi (1992) of the New Orleans Public School system found that AfricanAmerican male and female students scored in the lower quartile on the math section of the California Test of Basic Skills. One possible explanation for these disappointing results might be the level of student expectations held by the teachers. For example, female and minority students benefit from having a supportive teacher who has high academic expectations. When RIR STEM EDUCATION DAY PROGRAM EVALUATION 30 teachers have high expectations of their students, the students are generally more successful. Conversely, if low expectations are the norm, students will have a difficult time succeeding (Garibaldi, 1992). Garibaldi stated that teachers often have lower expectations for AfricanAmerican students, particularly African-American boys; these low expectations help explain why many minority students are underachieving (Garibaldi, 1992). Supportive teachers can have a positive impact on student academic achievement, as demonstrated by Reeve and Jang (2006) in a study about how teachers’ instructional behaviors can create a self-directed learning environment. The researchers found that social support in the form of a facilitative teaching style resulted in improved academic achievement among middle school students and, in the process, provided supportive role models who promoted and encouraged students to continue in STEM education. Providing a Positive STEM Classroom Experience It is essential that female and minority students feel welcomed and a valued part of STEM classrooms. Traditionally STEM classrooms have often not been a positive experience for females and minorities, and as a result, they shy away from STEM majors (Beasley & Fischer, 2012). For instance, a study by Torres and Charles (2004), found that white students viewed African-American students as academically unqualified for STEM at the college level, and attend only because of affirmative action quotas or athletics. When asked about black students, white students reported that they are more musically inclined and/or athletic, but less equipped for more cerebral pursuits (Torres & Charles, 2004). K-12 organizations such as VASS reject such stereotypes and encourage female and minority students to take the academic challenge of becoming involved in STEM education. RIR STEM EDUCATION DAY PROGRAM EVALUATION 31 In a similarly discouraging example, Rinehart and Watson (1998) concluded that female students in STEM classrooms sense discriminatory behaviors from their professors’ more than male students. These concerns inevitably affect the performance of female students in STEM disciplines (Steele, James & Barnett, 2002). For that and other reasons, it is vital for educational leaders to create and promote an academic environment that welcomes all students to STEM education and not just the academically elite. In an effort to gauge student views on perceptions, feelings, and beliefs about their K-12 experiences, Yazzie-Mintz (2010) administered the High School Survey of Student Engagement (HSSSE) to high school students. The results from the 2009 survey revealed that a disconnection exists between what students actually experience in class and what the instructors thought they were experiencing. Students reported that they felt bored and not respected by their instructors, which indicated a need for a stronger student-school connection. Furthermore, “creating a stronger student-school connection involves listening to students without prejudice and altering the school environment to meet their needs” (Yazzie-Mintz, 2010, p. 54). Results from the survey suggested that listening to students about their needs and making changes to the classroom created a more engaging learning environment. “Bridging the gulf between students and adults is a great challenge for school leaders, but it is key to strengthening both student engagement and the connection between students and school” (Yazzie-Mintz, 2010, p. 56). Collaboration Needed Collaboration in STEM Education is a means to maximize problem solving to improve and enhance the future success of students in STEM-related education regardless of gender and ethnicity. A strong connection between schools, universities, business, and community partners RIR STEM EDUCATION DAY PROGRAM EVALUATION 32 could enhance students’ educational experiences (Epstein, 1995). The importance of teachers collaborating with local universities, businesses, and communities can provide optimal learning experiences for students. In addition, a strong partnership between schools and universities can provide teachers with professional development programs to produce strong outcomes for students in STEM education. This is particularly true with STEM-related curriculum giving opportunities to all stakeholders to strengthen and inspire learning within their own communities (Clark & Button, 2011). With schools being challenged to increase the number of students graduating with science, technology, engineering and math-related degrees, connections with colleges and universities, as well as community partners and local businesses, are important. The need for collaboration among school districts, colleges, and corporate partners within communities is essential to increase students’ interest in STEM education. Through collaborative partnerships with other schools, districts, and states, STEM education can enhance their instructional programs. This is accomplished by providing relevant classroom experiences derived from within the collaborative community. Collaborative relationships between school districts, local businesses, and universities are necessary if the United States is to keep the focus on math and science and compete in a global economy. Finding collaborative ways to integrate real hands-on experiences within the community may increase students’ interest and understanding of STEM. Collaboration with community partners also ensures students receive better preparation for a future in STEM, which will create economic growth and alleviate outsourcing of jobs to other countries (Office of the Press Secretary, 2009). Focused collaboration among community stakeholders can increase opportunities for students to gain relevance and real-world application of STEM curriculum. The 1983 report, A Nation at Risk, brought to the public’s attention what RIR STEM EDUCATION DAY PROGRAM EVALUATION 33 the authors described as complacency of the educational system in the United States. Relying too heavily on past successes, America became complacent and failed to keep pace with technological innovations and, as a result, a decline in science and math was evident (Ornstein, 2010). President Obama announced in 2010 his plans for the Educate to Innovate program to increase the nation’s focus on STEM (White House, 2010). The White House introduced private and public partnerships to mobilize resources for STEM throughout the U.S. In addition, Obama called for 200,000 federal workers in the field of science and technology to volunteer to work with educators to improve STEM education (White House, 2010) through community collaboration. The Importance of University Partnerships The role of local universities is paramount in increasing interest, participation, and targeting more women and minorities in STEM. Schools might collaborate by partnering with universities to improve the quality of content and experiences. For example, Hofstra University’s Center for Technology Literacy (CTL) goal is to improve middle school students’ achievement through rigorous researched based strategies in math and science through developing partnerships with public schools to promote and support STEM learning. In addition, Hofstra initiated collaboration with International Technology and Engineering Educators Association (ITEEA) to provide professional development giving teachers opportunities to integrate real-world simulations into the curriculum (McKay, Klein-Gardner, Yoder, Moskal, & Hacker, 2011). Partnerships with various stakeholders can play an important role addressing issues in increasing the interest, participation and awareness of STEM for women and minorities. To help address the issues facing STEM teachers, Virginia Polytechnic Institute and State University (VT) has developed a collaborative relationship by coordinating and bridging a RIR STEM EDUCATION DAY PROGRAM EVALUATION 34 strategic partnership through VT’s Science, Technology, Engineering, and Mathematics K-12 Education Outreach Initiative (VT-STEM). The program serves as a connection between the state university and Virginia’s PK- 12 schools to create meaningful learning experiences involving students throughout the state. This collaboration also facilitates professional development for teachers with intentions of increasing student achievement and student interest in STEM-related fields. As an example, two school districts in Virginia, Arlington and Loudoun County Public Schools, enrolled teachers and administrators in VT’s online Integrative STEM Education Certificate Program to assist them in changing the culture of STEM education within their district (Sanders, 2009). Sanders (2009) asserts, “professional development is most effective when site-based and sustained over an extended period of time” (p. 22). The intention of the program is to introduce STEM education across disciplines and not in silos. The opportunity for VT to provide high-quality professional development, research based best practices for teaching and learning, and support for teachers will ultimately influence student achievement and add relevance to the curriculum. Sanders (2009) concurs that, “technology education’s future in American education will depend upon its ability to demonstrate relevance to the school curriculum” (p. 25). An example closer to home is the university, business, and public schools collaborative partnership that made the RIR STEM Education Day program possible. Richmond International Raceway and Arena Racing, USA were brought together by the Center for Sport Leadership (CSL) at Virginia Commonwealth University through existing relationships. RIR brought the local public schools to the table through an existing relationship to help make the program a reality; however, it seems certain that this collaborate project would not have gotten off the RIR STEM EDUCATION DAY PROGRAM EVALUATION 35 ground if it were not for the thirty graduate students that CSL provided. The other members of the team simply did not have the staff or the time to take on such a program. Business & Community Partnerships “The issue of attracting more young people to choose careers related to STEM has become critical for the United States” (Merrill, & Daugherty, 2010). Business and community partnerships can foster excellence in K-12 STEM education (President's Council of Advisors on Science and Technology (US), 2010). By collaborating with schools, these partnerships can give students increased opportunities to “learn by doing,” or through hands-on experiences. In addition, providing more input and feedback to schools to improve STEM curriculum increases students’ awareness of business and community opportunities. By engaging in strategic partnerships and aligning resources to enhance STEM education programs in schools, collaboration is needed to increase students’ knowledge through real-world STEM opportunities (President's Council of Advisors on Science and Technology (US), 2010). These experiences help inspire students to pursue STEM-related curriculum during their secondary years in school (Packard, 2012). Supportive business/community partnerships are important because they provide STEM opportunities by providing students with role models and possible internship possibilities that might steer them into STEM related careers. Koch, D., Johnson, M. E., & Marshall, B. H. (2013) suggest that, “a student’s choice to pursue STEM careers is also impacted by an awareness of career opportunities” (p.55). Business/community partnerships can provide resources for laboratories by giving students opportunities to engage in innovative solutions to solve real-world challenges (Carraway, Rectanus, & Ezzell, 2012). Financial resources can give schools opportunities to RIR STEM EDUCATION DAY PROGRAM EVALUATION 36 actively engage students in inquiry based research practices to enhance teaching and learning. Business/community partnerships can ignite the next generation of STEM-workforce to pursue interest in STEM education. Collaborative Efforts The process of bringing hands-on activities into the classroom needs to be coordinated with STEM curriculum designers. Accordingly, STEM instructors need to participate in professional development that is “aligned and designed to train the teachers to use integrated curriculum” (Rockland, Bloom, Carpinelli, Burr-Alexander, Hirsch, & Kimmel, 2010, p. 1). If done properly, learning can be simplified so that students are motivated to probe deeper into STEM topics that interest them on an individual level. Active hands-on learning activities can be designed to create more student engagement that will subsequently deepen student understanding, build interest in STEM, and motivate students to want to pursue a degree or career in a STEM related field; therefore, allowing for increased opportunities for student learning beyond the classroom. Collaborative efforts between school districts, colleges, universities, and businesses can ensure that students are exposed to STEM-related curriculum and relevant hands-on experiences designed within the community. Conclusion As technology brings our world closer together, we find ourselves moving toward a more global economy. Policy makers worldwide understand the importance of a country’s position in this global economy and each is working to improve their economic position. Leaders in the United States have taken notice of the rapid decline in our country’s position and are launching initiatives and programs to address this challenge. RIR STEM EDUCATION DAY PROGRAM EVALUATION 37 Several factors may have contributed to this decline. The lack of focus on STEM education as it relates to the United States being contributors to the global economy, as well as antiquated, inefficient, and discriminatory methods educators have used to teach STEM curriculum seem to propagate this decline. An additional factor identified by the literature review was the need to change the appeal of STEM-related education and careers. For instance, one perception is that STEM classes are boring, lack engagement, and are only for the “smart kids”. STEM opportunities are not readily available to female and minority students. Creating a mindset that female and minority students are fully capable of succeeding in STEM settings is profoundly important if the United States is to meet the task before them. To accomplish this task, leaders in education need to continue to develop and implement non-traditional programs. Collaborating with community partners to present STEM curriculum in a setting outside of the traditional classroom environment has proven to enhance student learning. If the United States is serious about tackling this issue and moving our country forward then a paradigm shift is paramount. Leaders and educators can improve the appeal of STEM by building bridges with community partners that creates a strategic collaborative relationship between students, schools, and communities, as well as encouraging and inspiring women and minorities to participate in STEM-related programming. RIR STEM EDUCATION DAY PROGRAM EVALUATION 38 SECTION THREE Methods This section describes the evaluation design and methods, the participants and how they were selected, the data collection methods, the data analysis used, and strategies that enhanced credibility and dependability of the evaluation. In this section, program evaluators also addressed potential limitations. Evaluation Focus The focus of this evaluation was to determine to what degree the April 25, 2013 RIR STEM Education Day program increased interest and understanding of STEM, STEM educational and career opportunities, and if there were differences in STEM interest and understanding among female and minority students. For the purpose of this evaluation, the working definition of STEM is an education that provides students with science, technology, engineering, and math in sequences that build upon each other and used with real-world applications (Eberle, 2010). This working definition accepted that the components of “STEM should be taught in an integrated and cross-curricular manner, not just in ‘silos’ where the individual subject areas dominate and the other subjects are only afterthoughts” (Dugger, 2011, p. 2). The following evaluation questions emerged from extended discussions with key program administrators and information gleaned from the review of literature: 1. To what degree did the RIR STEM Education Day program increase middle school students’ interest in STEM education? 2. To what degree did the RIR STEM Education Day program improve students’ understanding of STEM education? RIR STEM EDUCATION DAY PROGRAM EVALUATION 3. 39 To what degree did the RIR STEM Education Day program increase students’ interest in STEM educational opportunities? 4. To what degree did the RIR STEM Education Day program increase students’ interest in STEM-related careers? 5. Are there differences in STEM interest and understanding based on gender and ethnicity regarding the RIR STEM Education Day program? Evaluation Design The evaluation team employed Stake’s (1975) Responsive evaluation model for this program evaluation. Responsive evaluation supported observing how the program worked and required getting to know stakeholders at various levels of the program. The design for this evaluation was mixed-methods. Creswell (2009) described mixed-methods designs as those that include both quantitative and qualitative design elements. Mixed method designs combine both quantitative and qualitative strengths into a single research method. Each method has unique strengths but by bringing them both together, more benefits are realized. This point was highlighted by Connelly (2009) who wrote, “the goal of mixed methods research is to draw on the strengths to minimize the weaknesses of both types of research” (p. 31). Accordingly, the mixed-methods design specifically selected for this program was sequential mixed-methods. Figure 1 illustrates how the quantitative data provided information to inform the collection of the qualitative data in a sequential manner. Quantitative Results Figure 1. Sequential mixed methods design. Qualitative Results RIR STEM EDUCATION DAY PROGRAM EVALUATION 40 The rationale behind using the sequential mixed methods design for the RIR STEM Education Day program evaluation is further detailed in Figure 2. Figure 2 demonstrated how quantitative data collected from the student survey was first supplied to the evaluation team. This data was used to help develop the focus group questions from which qualitative data was obtained. The qualitative data obtained through student and teacher focus groups, along with student and teacher comments made on the day of the program, was then used to help answer the evaluation questions. Student Survey Comments Student Surveys Quantitative Results Student Focus Groups Qualitative Teacher Feedback Results Teacher Focus Group Figure 2. Sequential steps for collecting data for the RIR STEM Education Day program. Participant Characteristics Two groups of participants were involved in this evaluation. They included the teachers and students who participated in the RIR STEM Education Day program. There were 12 middle school teachers involved with the RIR STEM Education Day program. Of the 12 participating teachers, only seven were still teaching in their respective schools; the other five were unavailable. The role of the teachers during the program was to make each station students’ experienced at education day applicable to lessons taught in the classroom. As teachers took students from station to station, they helped bridge the gap between classroom theory and real-world application. In initial meetings, the Career and Technical Education (CTE) Educational Specialist agreed to invite the seven remaining middle school RIR STEM EDUCATION DAY PROGRAM EVALUATION 41 teachers to participate in the focus group interview process in order to provide valuable information regarding the inaugural program. Additionally, there were approximately 500 students who attended the RIR STEM Education Day program. CTE administrators supplied the evaluation team with demographic information pertaining to the students who participated in the RIR STEM Education Day program (see Appendix A). The information revealed that the 87.5% of the students were male, 12.5% of the students were female, and of these male and female students, 43% were minorities. Data Collection To answer the evaluation questions, a sequential mixed methodology using both quantitative and qualitative data sources were used for this evaluation. Quantitative data consisted of information gleaned from the RIR STEM Education Day student survey responses. Qualitative data collected for this program evaluation consisted of transcribed interview responses from student and teacher focus groups, student comments from the RIR STEM Education Day survey, as well as CTE teacher feedback received at the conclusion of the program. Additionally, an analysis of a public document, Competency List for CTE 8403 Technology Foundations (see Appendix B), was performed to affirm the CTE curriculum was relevant to STEM education and the RIR STEM Education Day program. Quantitative Data Quantitative data consisted of information collected from the RIR STEM Education Day field trip survey completed by the students who participated in the program. Survey information was provided to the evaluators by the CTE administrators who collected the completed surveys from participating teachers. RIR STEM EDUCATION DAY PROGRAM EVALUATION 42 Survey. The Center for Sport Leadership and CTE administrators developed a survey to capture information from the students about their experience at the RIR STEM Education Day program. Teachers were responsible for administering and facilitating the survey immediately upon completion of the program. Completed surveys were provided to program evaluators. Information from the survey was received in a spreadsheet that listed “yes” or “no” responses from students regarding the five survey questions as well as student comments (see Appendix C). Responses did not identify students by name but did identify the school attended and included the students’ written feedback. Evaluators de-identified the school attended by assigning each school an alphabetic letter (see Appendix C). Written feedback provided by students was addressed in the qualitative portion of this document. This survey, which was a secondary data source, allowed for analysis of data already collected. Secondary data sources are commonly collected by educational institutions and are geared toward improving student experiences (King, Hernandez, & Lott, 2012). According to Royse (2008), secondary data analysis involves analysis of an existing data set that results in additional knowledge, interpretations, and conclusions” (p. 201), and allows an unobtrusive means of gathering data. Participants. There were approximately 500 students who participated in the program, of which 249 sufficiently completed the survey with usable information, producing a yield of 49%. Students from all 12 middle schools who attended the program were represented on the survey. Instrument. A paper survey consisted of five questions, to which students responded “yes” or “no”, was developed by the Center for Sport Leadership and CTE administrators. The survey addressed student’s awareness, interest, and understanding of STEM as well as RIR STEM EDUCATION DAY PROGRAM EVALUATION 43 educational and career opportunities related to STEM. Following the program, students were divided into groups by school and their teachers administered the survey. Teachers then collected the completed surveys and returned them to the CTE administrators. Data analysis. Evaluators received a spreadsheet from CTE administrators, via email, that represented the raw data collected from students following the RIR STEM Education Day survey. This raw data was in the form of an unorganized spreadsheet with checked boxes and student comments (see Appendix C). Evaluators organized the data by grouping student responses by school and tabulated “yes” and “no” responses, as well as converting responses into percentages (see Appendix D). This allowed evaluators to analyze and interpret data among individual schools as well as a collective group to see whether the program increased interest in career opportunities, improved understanding of STEM, and increased interest in educational opportunities. After reviewing the survey questions, evaluators determined that only three of the five survey questions were applicable to help answer the evaluation questions (see Appendix E). The three survey questions (two, three, and four) specifically pertained to measuring student awareness and interest in career opportunities within STEM, student understanding of STEM because of the program, and interest in educational programs that are specific to STEM. Therefore, the evaluators ensured that adequate focus group questions were developed to provide further information to answer the evaluation questions. The evaluation team cleaned up the survey data by first removing responses from question one and five from the survey because they did not pertain to the evaluation questions (see Appendix D). They then organized, analyzed, and interpreted the quantitative data from questions two, three, and four from the survey to determine to what degree the RIR STEM RIR STEM EDUCATION DAY PROGRAM EVALUATION 44 Education Day program addressed student’s awareness, interest, and understanding of STEM, and educational and career opportunities related to STEM. To do this, results were formulated into percentages to demonstrate how the students answered survey questions. Results were reported as a simple means of the frequency of how the yes or no answers were distributed (see Appendix D). Responses were then broken out by each school so that the results could be converted into bar charts, which allowed for a comparison between the participating schools (see Appendix D). This data may reveal differences and/or trends between each school as it pertained to gender and ethnicity. The data also informed evaluators of the direction needed to develop focus group questions to obtain qualitative data that would delve deeper into student responses as they pertain to the evaluation questions. Qualitative Data Qualitative data consisted of written feedback obtained from the student survey, teacher feedback obtained from CTE administration, as well as data collected through student and teacher focus groups interviews. The dynamic nature of focus groups provided insightful information through questions asked by the evaluators and produced a level of awareness rarely derived from unidirectional information collection devices such as observations and surveys (New York State Teacher Centers, 2008). Methods of recording and analyzing information gathered during focus groups, and strategies for collecting unbiased information, helped focus group researchers gain credibility as an accurate and useful source of information (Creswell, 2009). Due to the nature of this particular evaluation, and the varying perspectives that the participants had about the program, program evaluators determined that focus groups were the optimal choice of qualitative data RIR STEM EDUCATION DAY PROGRAM EVALUATION 45 collection because of “what they can add to other qualitative and quantitative data collection strategies” (Morgan & Spanish, 1984). Focus groups. The evaluators’ goal was to conduct only three focus group interviews. One group consisted of the seven remaining teachers and the other two consisted of students. The ideal situation was to form one student group that consisted of male students that included minority students and the other that consisted of female and minority students; however, program evaluators conducted more student and teacher focus group interviews because of the inability to get all of the students, and teachers, together at the same time. The number and location of the student focus group interviews depended on which school the students attended at the time of the evaluation. The hope was that the proper number of students who reflected pre-selected ethnic and gender criteria, based on purposeful sampling, became available at one or two schools, rather than spread out across all nine high schools. Evaluators ultimately conducted student focus group interviews at several different schools during the students’ lunch period. Evaluators provided food for the students to eat while conducting the interview. The focus group interviews lasted no more than 30 minutes and were facilitated by two evaluators. To begin each focus group interview, evaluators made introductions, explained the purpose of the focus group, and provided a historical perspective of the RIR STEM Education Day program through the use of pictures taken the day of the program, as well as diagrams of the program layout (see Appendix F). This reminded and re-familiarized participants of the program and activities that occurred at the Richmond International Raceway on April 25, 2013. Following the introductions and historical perspective of the program, the primary evaluator facilitated each of the focus group interviews by asking open-ended questions and RIR STEM EDUCATION DAY PROGRAM EVALUATION 46 audio-recording participant responses. The secondary evaluator took notes and documented each participant’s location in the room, their gender and ethnicity (based on information provided by the school district), and any nonverbal communications. This information was coded to accurately document who said what and how it was said without revealing the identity of the participants. Prior to conducting the focus group interviews, focused-coding was established to ensure that the essence of the evaluation questions was addressed. This was done by establishing codes that pertained to each of the five evaluation questions in order to determine that each one was addressed through the open-ended questions being asked by the primary facilitator. The secondary facilitator assigned codes to responses, based on gender, ethnicity, and key words or phrases related to the evaluation questions. As the interviews progressed, the secondary facilitator prompted the primary facilitator if an area was not addressed. Teacher focus group. Teachers were provided informed consent forms that fully explained the purpose of the evaluation, a description of the evaluation, as well as possible risks and benefits of the evaluation. Focus group questions were clearly written and open ended; answers were audio recorded and transcribed for analysis by program evaluators. Information recording strategies were utilized, such as a combination of note taking and audio recording, to accurately know who said what regarding gender, ethnicity, or any other factors that surfaced. Evaluators ensured that note taking did not interfere with or distract from the dynamic of the focus group process. Program evaluators conducted the teacher focus group interviews in various middle schools within the local school district. Teacher participants. Seven teachers from the participating schools were asked to participate in the focus group. The Career and Technical Education (CTE) Educational RIR STEM EDUCATION DAY PROGRAM EVALUATION 47 Specialist agreed to solicit the 12 teachers who participated in the RIR STEM Education Day program; however, only seven remained in their respective schools at the time of the evaluation. These seven teachers participated in the focus group interview process and provided valuable information regarding the inaugural program. Program evaluators had to conduct individual teacher interviews because they were not able to assemble as one group. While the number of teachers was small, the seven teachers represented 100% of the teachers who were available. Teacher focus group questions. Teacher focus group interview questions were designed to solicit detailed answers to the evaluation questions. The formation of the focus group questions was based on information obtained from the quantitative results of the RIR STEM Education Day survey and from the information from the literature review. The focus group questions were carefully considered by the evaluators and the responses served as an adjunct to the quantitative data, as well as provided information on perceptions, insights, attitudes, and beliefs regarding experiences at the RIR STEM Education Day program. Evaluators developed coding schemes that categorized teacher responses that related to the five evaluation questions as well as topics that were of particular interest. Illustrative statements and detailed codes denoting specific in-depth information were used along with notes on non-verbal communication. Teacher data analysis. Upon completion of the focus group interviews, evaluators conducted a post-session wrap up to debrief among themselves. Evaluators transcribed audio recordings and reviewed their notes using key words, codes, or themed comments. Evaluators analyzed the data and noted their observations as they pertained to the evaluation questions. The intention of the qualitative data collection was to seek supporting information from the focus groups that addressed each of the evaluation questions. RIR STEM EDUCATION DAY PROGRAM EVALUATION 48 Student focus group. Permission from parents of each student was secured prior to conducting focus group interviews. Students were provided with informed assent forms to ensure they fully understood the purpose of the evaluation, description of the evaluation, as well as possible risks and benefits of the evaluation. Focus group questions were clearly written and open ended. Answers were audio recorded and transcribed for analysis by the program evaluators. Information recording strategies were utilized such as a combination of note taking and audio recording to accurately know who said what regarding gender, ethnicity, or any other factors that surfaced. Evaluators ensured that the note taking did not interfere with or distract from the dynamic of the focus group process. Program evaluators conducted the student focus group interviews in the schools while students were at lunch. Student participants. Students who participated in the RIR STEM Education Day program were asked to participate in the focus group. For this evaluation, parental consent forms were sent home with every student who attended the April 2013 RIR STEM Education Day program as eighth graders, but who attended high school as ninth graders at the time of this evaluation. This process enabled the evaluation team to cast a wide net with the intent of capturing as many RIR STEM Education Day participants who were willing to contribute in the focus group process. The local school district assisted the program evaluators in selecting students who met the demographic needs for the student focus groups. This provided the evaluation team with the largest possible pool of potential focus group participants. Program evaluators utilized purposeful sampling to select focus group participants from this pool according to preselected criteria, also known as quota sampling. RIR STEM EDUCATION DAY PROGRAM EVALUATION 49 Purposeful sampling was used for the development of the student focus groups. Specifically, the use of quota sampling was based upon the demographic information that most closely represented the number of males, females, and diverse ethnicities who attended the RIR STEM Education Day program (see Appendix A). Quota sampling is a method of gathering representative data from a group. This method required that representative individuals be chosen out of specific sub-groups, such as gender and ethnicity (Woolley, 2009). Quota sampling allowed evaluators to collect meaningful information and an accurate representation of the total number of participants without having to acquire a large sample size. CTE administrators wanted to keep the numbers small as it was extremely difficult to get students in one place because of school policies, parental permission, and transportation concerns. Student focus group questions. The development of the questions for the focus groups was designed to provide detailed answers to the evaluation questions. Focus group questions were formed based on information obtained from the quantitative results of the RIR STEM Education Day survey and from information from the literature review. Focus group questions were carefully considered by the evaluators, and the responses served as an adjunct to the quantitative data, as well as provided information on perceptions, insights, attitudes, and beliefs regarding experiences at the RIR STEM Education Day program. Evaluators developed coding schemes that categorized student responses that related to the five evaluation questions as well as topics that may be of particular interest. Illustrative statements and detailed codes denoting specific in-depth information were used along with notes on non-verbal communication. RIR STEM EDUCATION DAY PROGRAM EVALUATION 50 Student data analysis. Upon completion of the focus group interview, evaluators conducted a post-session wrap up to debrief among themselves. Evaluators transcribed audio recordings and reviewed their notes using key words, codes, or themed comments. The evaluators analyzed the data and noted their observations as they pertained to the evaluation questions. The intention of the qualitative data collection was to seek supporting information from the focus group that addressed the evaluation questions. Student Survey Comments In addition to the five “yes or no” questions that were asked of the students on the RIR STEM Education Day field trip Survey there was a section that allowed for students to provide comments about the program. The program evaluation team analyzed these comments to seek information that might further inform the direction and development of the focus group questions and to answer evaluation questions. Data analysis. Student comments were first isolated by removing the “yes or no” responses from the survey. Evaluators identified common key words or phrases from the comments that allowed them to recognize patterns in the responses that were specific to the evaluation questions. Coded responses were then tabulated (see Appendix G). CTE Teacher Feedback CTE administrators sent an email to the 12 participating teachers upon conclusion of the program inviting teachers to comment on the RIR STEM Education Day program. The email asked teachers if they had any comments or feedback they would like to share regarding the program. The program evaluation team analyzed these comments to seek information that might further inform the direction and development of the focus group questions. RIR STEM EDUCATION DAY PROGRAM EVALUATION 51 Data analysis. Comments from teachers were copied and pasted to a word document and forwarded to evaluators by CTE administrators. Evaluators received comments from 7 of the 12 teachers who participated in the program (see Appendix H). Evaluators analyzed these comments to identify common key words or phrases that allowed them to recognize patterns in the responses that were specific to the evaluation questions. Coded responses were then tabulated (see Appendix H). Strategies for Enhancing Credibility and Dependability Multiple steps were taken to enhance credibility and dependability of the program evaluation. Peer debriefing and member checks were used to enhance credibility. Thorough documentation of the entire evaluation process, as well as periodic audits of the documentation, were done to enhance dependability. Peer debriefing occurred among the four evaluators throughout the course of the evaluation. This allowed the evaluators to challenge viewpoints and share ideas with each other throughout the process. As an additional step, member checks were performed which allowed the team to share data and information among the four evaluators as well as with key stakeholders such as RIR, Arena Racing, and the CTE administrators. Member checks conducted with the capstone review team further enhanced credibility of the evaluation. These member checks were conducted throughout the evaluation process so valuable feedback could be shared among stakeholders along the way. Moreover, member checks provided the evaluation team with insight and feedback from individuals who were knowledgeable on the subject matter but who were not directly involved with the program evaluation. Evaluators enhanced dependability by establishing a system for documenting each step of the evaluation process that included any changes that occurred. Evaluators stored all relevant RIR STEM EDUCATION DAY PROGRAM EVALUATION 52 documentation on WordPress, an online management site, established specifically for this purpose. The evaluation team met numerous times each week for the duration of the evaluation. Audits of the program evaluation documentation were performed at each meeting in order to keep track of any updates and/or procedural changes made during the evaluation process. Any procedural changes were discussed with the team prior to implementation. Potential Limitations Potential limitations of this evaluation included the amount time between the date of the program and the evaluation, the number of teachers who participated in the program who were available to participate in the focus group, and the student survey that was conducted at the conclusion of the program. Time Lapse This evaluation took place eight months after the date of the program. This lapse in time resulted in some participants not being able to retrospectively recall all of the specifics of the program; however, recall was aided by displaying poster-sized pictures of the program in the interview room, as well as diagrams that displayed the activity stations and program layout. Additionally, the evaluators began each focus group session with a general prompt that asked participants to recall the RIR STEM Education Day activity stations topics, content, and speakers. Responses from multiple students and teachers were obtained that allowed individual responses to be comparatively cross-checked. Teacher Retention Another possible limitation in this program evaluation was that only seven of the 12 original teachers remained available for the focus group interview; however, these seven teachers RIR STEM EDUCATION DAY PROGRAM EVALUATION 53 represented 100% of the teachers who were available. The teachers were a valuable component of this program evaluation and offered diverse viewpoints and opinions. Student Survey The student survey could potentially be another limitation to this program evaluation. While it did offer valuable information that helped answer the evaluation questions, the survey was developed prior to the formulation of this program evaluation and the related evaluation questions; therefore, the survey was a secondary data source. Secondary data sources are data that were developed for a purpose other than helping to evaluate the RIR STEM Education Day program. The survey was developed and administered by individuals other than the evaluators; therefore, additional data that helped answer the evaluation questions was not developed by the evaluators specifically for this program evaluation. The evaluators determined that only three of the five survey questions were applicable to help answer the evaluation questions (see Appendix E). Evaluators ensured that adequate focus group questions were developed to provide further information to answer the evaluation questions. RIR STEM EDUCATION DAY PROGRAM EVALUATION 54 SECTION FOUR Findings This section presents the findings for the evaluation of the Richmond International Raceway (RIR) Science, Technology, Engineering, and Math (STEM) Education Day program. These findings highlight information obtained from quantitative and qualitative data sources to determine the degree to which the RIR STEM Education Day program increased interest in STEM, improved understanding of STEM, increased STEM educational and career opportunities, and to determine if there were differences in STEM interest and understanding based on gender and ethnicity. These findings, which form the basis for recommendations, are provided in the sections that follow. Both quantitative and qualitative data were collected which provided information that informed the answers to each evaluation question. Quantitative data consisted of information collected from the RIR STEM Education Day field trip survey which was completed by students who participated in the event. The survey was a secondary data source; however, it was determined that the results from three of the five survey questions were valuable in answering the evaluation questions. Qualitative data consisted of student comments from the RIR STEM Education Day field trip survey, Career and Technical Education (CTE) teacher feedback, and information received from student and teacher focus group interviews. The collective analysis of these qualitative data sources were used in establishing overall themes and were also used in addressing the evaluation questions. These themes, derived from focus group interview data, will be discussed first. Following this discussion, each evaluation question will be addressed individually and will be based on both quantitative and qualitative data. RIR STEM EDUCATION DAY PROGRAM EVALUATION 55 Themes To answer the five evaluation questions related to the RIR STEM Education Day program, evaluators analyzed student comments from the RIR STEM Education Day field trip survey, CTE teacher feedback, and transcriptions from the student and teacher focus group interviews. Prior to considering all data sources collectively, evaluators thoroughly examined the focus group transcriptions to identify key words or phrases that appeared with regularity or stood out as interesting or significant. Each of the evaluators did this examination process independently in order to obtain four unique perspectives. Once this process was completed, commonly identified key words and phrases from all four evaluators were combined into one document. That document, which was a list of key words and phrases by school, was then used to identify initial themes. These themes were then examined and similar ideas were combined. The following list represents an amalgamation of those themes: “Awareness” of educational and career opportunities “Making connections” through real-world hands-on activities Exposure to “role models” More “time” at stations to learn Evaluators further refined the list to the following themes: Awareness Making Connections Role Models Time RIR STEM EDUCATION DAY PROGRAM EVALUATION 56 Theme 1: Awareness. The word “Awareness” appeared often throughout the focus group transcriptions. Teachers especially used this word when referring to how the program was effective at making students more aware of how STEM is applicable to the real world. One teacher specifically talked about awareness of STEM in motorsports and the connection it provided back to classroom content by stating, “I absolutely think their awareness increased. I’m glad we got a chance to see the show car because it was the most direct relationship between what we learned in class…it was very, very technical”. When asked about how the event helped raise awareness of career opportunities, one teacher talked about how students became more aware of career pathways and job opportunities surrounding the motorsports industry. He said, “I was able to connect it in the form of employability skills in business orientation and awareness…meaning, we talked about it, they experienced it…” These comments were significant for two reasons. First, they provided evidence that suggested the program was effective at creating an awareness of STEM, which was the main purpose of the event. Secondly, comments like these helped evaluators recognize that the success of the program extended beyond the organizers goal of creating awareness; the program was also effective at increasing students’ awareness of STEM career opportunities. Events such as the RIR STEM Education Day program that offer opportunities for students to be more aware of STEM can positively affect student interest (Hylton, 2010). Students felt like the experience introduced “…all the details and helped me understand what is really going on and basically get to know more about [STEM].” One student who “really did like it”, also added that it “showed me a lot more that I did not know. I wasn’t aware of [everything] they had.” RIR STEM EDUCATION DAY PROGRAM EVALUATION 57 Theme 2: Making Connections. “Making Connections” also surfaced as a theme primarily because of the importance of connecting the science, technology, engineering, and math experienced at the event back to classroom content. One particular teacher’s response was illustrative of this by saying, “They were able to see all of those things that I talked about because they came back and said, ‘… we now understand more about what you are talking about because that’s what we saw, and that’s what we heard.’” This was especially true when it came to the importance of “hands-on activities”, which was described by both teachers and students in the focus group interviews. Teachers spoke about how they appreciated the hands-on nature of the event and how hands-on activities help students better understand what was being taught in the classroom. One teacher commented, “Sometimes kids need to see a hands-on…they really need to see a visual. They need to have a hands-on experience, and they got that.” While addressing a question regarding the relationship between the event activities and classroom content, one teacher specifically talked about how “making connections through hands-on activities,” brings classroom content to life in a real-world setting and can make all the difference in the world when it comes to students retaining key STEM concepts. Students also shared their appreciation of the hands-on aspect of the event. One particular student mentioned, “I feel like people learn better when it’s hands-on because they can actually see and feel what they are doing and as they are doing it. So they can learn easier and better from doing hands-on and actually doing it rather than having somebody telling them how to do it.” Evaluators found numerous student comments in the transcriptions that addressed the fun and excitement of the hands-on activities. Students talked specifically about hands-on RIR STEM EDUCATION DAY PROGRAM EVALUATION 58 activities that increased their interest, calling those activities “fun” and “cool”. For example, one student commented, “The simulation was really cool,” and “It was fun because it was hands-on”. Theme 3: Role Models. The RIR STEM Education Day program provided additional opportunities for students to make connections by seeing individuals in nontraditional roles. Specifically, an African-American female was the facilitator of one station. Students took notice and commented that this particular station helped erase perceived limitations regarding gender and ethnicity related to STEM. Students made comments indicating that this individual “overcame barriers” and was “an inspiration” to them. As one student put it, “I really liked [her] station, ‘cause actually she was like, she was like an inspiration for real. She showed you that girls could do anything a man could do clearly.” A teacher even stated that, “the kids need to see that” when referring to exposing students to stations with role models such as the African-American female racer. A different teacher reinforced the impact of this individual station by saying, “They saw that girl . . . we need more of that . . . the kids need to see that they can do these things.” A teacher’s response that highlighted the impact of the African-American female driver’s station as a role model for students was, “[She] herself broke down so many double barriers, again by being a female and being an African American and there were so many African-American students in the group and I think they were very impressed.” Another teacher’s comment similarly spoke to the importance of having a role model: “[She] had a big impact on kids, especially the females, because females will look at this as if you see like all guys there, okay this is not for me. By having her in there and she had a lot of energy, a whole lot of enthusiasm, so I think RIR STEM EDUCATION DAY PROGRAM EVALUATION 59 having her there really brought it home for our kids.” Theme 4: Time. “Time” was also identified as a theme, especially from students. As students talked about some of the “cool” and “fun” activities, they spoke about how they would have liked to have had more time to spend at these activities. For starters, one student was critical of the opening comments by stating during a focus group interview, “I would probably leave out, you know when you are in the center stage, the guest speakers, because that took a lot of time, [we wanted] more time at the stations.” Some students were more specific about where they would have liked to spend more time. For instance, one student said, “more time on the simulator”, and another student indicated that, “my class didn’t get to like maybe five or six”, when referring to the number of stations they were able to visit. Data gleaned from the focus group interviews and student comments from the RIR STEM Education Day field trip survey provided additional insight that was related to time. A number of the students made comments about how they did not have enough time at the stations. For example, one student reported, “[the program] could be longer . . . we didn’t learn enough at the stations,” and another student added, “We could learn a little more about the pit crews and stuff like that. Make it a little longer so we have more time to learn.” These comments suggest that student learning did take place, but a deeper more meaningful understanding of STEM is possible if students had more “time” to spend at each station. Allowing more time for hands-on learning is important. Hands-on activities actively engages learning and leads to better understanding than more passive-learning activities such as listening to lectures and note taking (Gannod, Burge, and Helmick, 2008). The RIR STEM Education Day program presents an opportunity for more time on in-depth learning activities that can help students better understand real-world application of STEM. RIR STEM EDUCATION DAY PROGRAM EVALUATION 60 Summary of Themes Four themes emerged from the focus group interview process: Awareness, Making Connections, Role Models, and Time. They provided evaluators with insight into what students and teachers perceived to be valuable concerning the RIR STEM Education Day program. The themes spoke to the effectiveness of the overall program through the eyes of the participants by making authentic connections to the real world and by helping students increase interest and gain a better understanding of STEM. Obtaining student and teacher perspectives was an important step in achieving a comprehensive evaluation of the program and to provide answers to the evaluation questions, as well as to offer meaningful and attainable recommendations for the future. Table 2 displays the four themes in the left column and their association with the key component of each evaluation question in the right column. Table 2 Focus Group Themes and Associated Evaluation Question Components Themes Evaluation Question Key Components Awareness Interest Making Connections Educational and Career Opportunities Understanding Role Models Educational and Career Opportunities Gender and Ethnicity Time Understanding Answering the Evaluation Questions Program evaluators analyzed each data source to answer the five evaluation questions. A detailed analysis of the data is presented which will answer each evaluation question. RIR STEM EDUCATION DAY PROGRAM EVALUATION 61 Evaluation Question One: To what degree did the RIR STEM Education Day program increase middle school students’ interest in STEM education? To answer this evaluation question, data was obtained from the RIR STEM Education Day field trip survey and CTE teacher feedback solicited by the local school district’s CTE administrators. Additional data was obtained when students and teachers were asked during focus group interviews how effective they thought the RIR STEM Education Day program was at increasing interest in STEM education. Collectively, information gathered from each of these data sources helped determine that the RIR STEM Education Day program was effective at increasing interest in STEM education. Table 3 consists of the questions used to most accurately answer evaluation question one. Table 3. Data Sources: Evaluation Question One Survey Question Four Do you have an interest in learning more about the technical education programs offered within the local public school district? Student Focus Group Question One Think back to the RIR Education Day. What did you find interesting about that day? Teacher Focus Group Question Four What did students say or do following the RIR STEM Education Day program that may have indicated an increased interest in STEM educational opportunities? RIR STEM Education Day field trip survey. To arrive at this determination, evaluators first analyzed data from the RIR STEM Education Day field trip survey (See Appendix C) which indicated that 73 percent of the student respondents answered “Yes” to survey question four: Do you have an interest in learning more about technical education programs offered through your school? The responses to this question were particularly important because they immediately came from the students on the day of the event, contrasting RIR STEM EDUCATION DAY PROGRAM EVALUATION 62 with the eight-month time lapse between the date of the event and the student focus group interviews. The immediacy of the information collected from this survey was important as it allowed students to demonstrate how they felt about the program directly upon its conclusion. Furthermore, student comments from the RIR STEM Education Day field trip survey indicated a high interest in STEM education. For instance, a student shared these thoughts from the comments section of the survey, “The stations had some very interesting concepts that were interesting to learn about.” Other students echoed the same sentiment, such as, “It was really cool, I’ve learned a lot since I’ve been there”, “It was fun and a great experience”, and “The event was fun and informative.” When responding, a majority of the student participants frequently mentioned the word “fun” when describing their experience; however, one specific written comment that spoke directly to the program’s effectiveness came from a student who emphasized, “The field trip was excellent. I was already thinking about something like mechanical engineering and the mechanical engineer I talked to during a station helped to solidify my viewpoint.” CTE teacher feedback. Reinforcing the value of the RIR STEM Education Day, one CTE teacher’s feedback stated, “To sum it up, I will quote what one challenging student said to another on the bus on the way home, ‘THIS WAS THE BEST FIELD TRIP I HAVE EVER BEEN ON.’” These glowing remarks, and the frequency at which they were made, demonstrated an increased interest in STEM education among students because of this program. Student focus groups. Similarly, information collected from the student focus group interviews offered comments which led evaluators to conclude the RIR STEM Education Day program was effective at increasing students’ interest in STEM education. A major indication of the program’s success was the number of student comments regarding the hands-on stations that RIR STEM EDUCATION DAY PROGRAM EVALUATION 63 promoted interest and awareness such as, “The simulation was really cool,” and “It was fun because it was hands-on” support this point. Another student shared the same enthusiasm by stating, “It was interesting how we got to actually do the hands-on stuff and we got to try everything.” That same student went on to suggest that, “Because some people don’t necessarily like things, and once you try new things you get to know it and we got to do the hands-on things.” The majority of student responses to focus group question one indicated an interest in STEM education. Teacher focus group. Likewise, teacher focus group comments suggested that teachers were able to precisely recall the educational value of the event and its ability to spark student interest. Through separate interviews, teachers consistently displayed enthusiasm in describing how the event affected their students. For example, one highlight from the transcriptions came from a teacher who mentioned how a student asked a question that demonstrated an increase in interest regarding STEM in which the student said, “How can I get into this [racing], if I wanted to become a racecar driver? I want to do this, how can I get this?” and how, “seeing people in the industry got them much more excited” about STEM education. Another teacher responded by saying, “All of them came back more aware of what opportunities and possibilities they have.” Speaking more directly to how the program increased student interest in STEM education, one teacher reflected how the program was influential in shaping a student’s decision for their future: There was one student in the class, to augment the power of the event, who was very negative, he was the most negative student I have ever taught in my 30 years of teaching and I found out later on that he went over there [to the Engineering Center]. RIR STEM EDUCATION DAY PROGRAM EVALUATION 64 In essence, the majority of the comments from teacher interviews indicated that the students had a positive attitude toward the event and that interest levels in STEM had risen. Evaluation Question Two: To what degree did the RIR STEM Education Day program improve student understanding of STEM education? Students and teachers were asked to reflect on how effective they thought the RIR STEM Education Day program was at improving students’ understanding of STEM education. Table 4 consists of the questions used to most accurately answer evaluation question two. Similar to the first evaluation question, evaluators used information provided by CTE teacher feedback and focus group transcriptions to help answer this evaluation question. The combination of these data sources helped determine that the RIR STEM Education Day program was effective at improving students’ understanding of STEM education. Table 4. Data Sources: Evaluation Question Two Survey Question Three Do you better understand the science, technology, engineering, and math components of racing because of this program? Student Focus Group Question One Think back to RIR Education Day. What did you find interesting about that day? Teacher Focus Group Questions Two and Three TFGQ2: In what ways were you able to connect your class content to what students experienced that day? TFGQ3: What changes did you see in how students talked about STEM following the RIR STEM Education Day? RIR STEM Education Day field trip survey. The results from the survey (See Appendix C), which were immediately collected from students at the completion of the event, indicated that 84 percent of the student respondents answered, “Yes” to the question, “Did you RIR STEM EDUCATION DAY PROGRAM EVALUATION 65 better understand the science, technology, engineering, and math components of racing because of this program?” This percentage indicated the majority of students thought the program helped them better understand STEM education as a result of their experience at the program. When students were presented with an opportunity to provide written comments on the student survey regarding their experience at the RIR STEM Education Day program, a number of their comments specifically addressed ways to improve learning. As an example, one student said, “[the program] could be longer . . . we didn’t learn enough at the stations,” while another student made a similar suggestion by saying, “We could learn a little more about the pit crews and stuff like that. Make it a little longer so we have more time to learn.” The fact that the students said, “we did not learn enough” suggested that student learning did take place, but that a deeper understanding of STEM might be achieved if students had more time to spend at each station. While some comments provided program improvement suggestions, other comments were complimentary, “[The event] was really cool, I’ve learned a lot since I’ve been there, and stuff” and, “The stations had some very interesting concepts that were interesting to learn about.” Additional comments from the survey that addressed the program’s effectiveness at improving understanding of STEM were, “We got to see how all the engineering concepts we learned applied to the sport in an exciting way,” and “I love what you guys did, I learned so much while I was there,”. Another student commented, “I am so happy I had the opportunity to be there and learn things I never knew.” Comments such as these demonstrate the program’s effectiveness at improving students’ understanding of STEM education. CTE teacher feedback. Teacher feedback obtained by CTE administrators was also supportive of improved student understanding as a result of the field trip experience. One RIR STEM EDUCATION DAY PROGRAM EVALUATION 66 teacher stated, “I believe that these types of experiences are life changing for many of our students and help them critically view and understand the functioning world outside the classroom.” A second teacher supported this notion by sharing, “Kids really enjoyed it and learned a lot.” Teachers communicated that the program was beneficial in making connections to classroom content. Results from the student survey administered at the conclusion of the event also showed a high percentage of students believed the program increased their understanding of STEM education. Student focus group. During focus group interviews, students were asked to think back to the RIR STEM Education Day program and share what they found interesting about that day. Students who reflected back to the event indicated that the program may have helped to make a connection by sharing comments like, “seeing all the tires and how they work . . . that was really cool.” Likewise, other students stated, “The speakers were talking about the application of math and science with racing and stuff and how the fuels, and the car, and tires work”, and “I learned a bit more of how STEM applies to racing.” Teacher focus group. Focus group interview transcriptions revealed that teachers spoke favorably about the program’s effectiveness regarding the improvement of students’ understanding of STEM. Teachers made comments that suggested the RIR STEM Education Day program improved students’ understanding of STEM education because students began to make connections to classroom content and real-world experiences. For instance, one middle school teacher summarized this point by saying, “[The students] came and said, ‘We now understand more about what you are talking about because that’s what we saw”’ and “what we learned at the raceway, we were able to tie into the classroom.” RIR STEM EDUCATION DAY PROGRAM EVALUATION 67 Two additional teachers supported these statements by saying, “When you go to an event like this [students] start answering some more questions by themselves and you can start challenging their minds” and this program “gives the kids an opportunity to see real-world applications of what we are learning.” These comments reiterate the event’s relevance and importance to increasing students’ understanding of STEM curriculum. When asked what changes teachers saw in how students talked about STEM following the RIR STEM Education Day, one teacher commented, “You could see that they understood why we do things a certain way; why we go through a certain process and problem solving.” This level of understanding can help students see the bigger picture by making the connection between what is being taught in the classroom and how it relates to real-world application. When asked the same question regarding what changes teachers saw in students, another teacher expressed a deep appreciation of the program and the impact it had on students by saying, “This is where the real learning takes place. This [real-world application] was the heart of the matter as far as learning and STEM.” Another teacher reinforced this comment: “[Students] see how math, science, and technology…all those things kind of interrelate, not only in racing but in the different facets of their lives.” Correspondingly, another teacher stated, “[students] were able to see that you had physics involved. There was math involved, more than just banging metal together and nuts and bolts. There is a lot more that goes into it.” Overall, the analysis of the student survey comments, CTE teacher feedback, and student and teacher focus group interview data indicated that the program did increase students’ understanding of STEM. RIR STEM EDUCATION DAY PROGRAM EVALUATION 68 Evaluation Question Three: To what degree did the RIR STEM Education Day program increase students’ interest in STEM educational opportunities? It was determined that the questions shown in Table 5, combined with information obtained through student comments from the field trip survey would answer evaluation question three. Table 5. Data Sources: Evaluation Question Three Survey Question Four Student Focus Group Question Three Teacher Focus Group Questions One and Four Do you have an interest in learning more about the technical education programs offered within the local public school district? Were there stations where you would have liked to spend more time? Was there anything you did that made you think that you would like to learn more about STEM? TFGQ1: What did the students think about RIR STEM Education Day? What did they say about it afterwards? TFGQ4: What did students say or do following the RIR STEM Education Day program that may have indicated an increased interest in STEM educational opportunities? RIR STEM Education Day field trip survey. When students were asked on the survey whether they had an interest in learning more about the technical education programs immediately following the RIR STEM Education day event, 73% percent responded “Yes” (See Appendix C). This percentage speaks to the initial effectiveness of the event at increasing students’ interest in educational opportunities. RIR STEM EDUCATION DAY PROGRAM EVALUATION 69 Student comments from the RIR STEM Education Day field trip survey were also analyzed and a couple of students shared comments regarding educational opportunities. One student stated, “I can improve by going to [one of the specialty centers] to learn more about motors and other stuff”, and another student added, “I was already thinking about something like mechanical engineering, and the mechanical engineer I talked to during a station helped to solidify my viewpoint.” Student focus group. Evaluators also asked students about their experience at the event. Specifically, students were asked if there were stations where they would have liked to have spent more time and whether they experienced anything at the event that sparked an interest to learn more about STEM. While students did not provide in-depth comments they did reveal some encouraging points concerning the program’s effectiveness. One student spoke about the impact the program had on their level of interest in STEM when they shared, “[the program] did kind of help me think about getting a STEM degree. They talked about how smart the drivers are and how they have all these degrees in technology and engineering.” An even more telling comment about the program’s capability to increase interest in STEM educational opportunities came from a student who indicated, “[the program] gave me an interest in different types of fields, and this was one of the more mechanical fields of engineering and it gave me a good idea in what field I want to go in.” Teacher focus group. When evaluators asked teachers about what students thought regarding the event and what they said about it afterwards, they agreed that students’ interest in educational opportunities increased. One teacher’s comment addressed the impact the event had on introducing educational opportunities by stating, “He is really interested in getting into, not necessarily the racing itself, but the industry” which indicated the program did present new RIR STEM EDUCATION DAY PROGRAM EVALUATION 70 educational opportunities that may have been unfamiliar to this particular student prior to the event. Teachers were also asked in the focus group interviews what students said or did following the event that may have indicated an increased interest in STEM educational opportunities. In response to this question, one teacher revealed that following the RIR STEM Education Day program, “One student was ready to get into the Technology Student Association (TSA) or technical event…anything that had to do with motors, speed, etc.” A second teacher commented, “In my class, they wanted to know what jobs can they get into, what path would they need to go into in order to get that particular job.” Another teacher spoke directly to the effectiveness of the program at increasing students’ interest in STEM educational opportunities provided by their school district by stating: I know after the program, I know two students who were in my program who were walking the fence about whether to go that had been accepted at other specialty centers, said hey, ‘I’m going to [one of the specialty centers].’ I think I can say that was the icing on the cake after they went to that program. Based on student and teacher comments, evaluators determined the program was effective at generating students’ interest in STEM educational opportunities. Evaluation Question Four: To what degree did the RIR STEM Education Day program increase students’ interest in STEM-related careers? In order to gauge student’s interest in STEM-related careers, evaluators analyzed multiple data sources they anticipated would reveal student interest levels regarding career opportunities in STEM. Table 6 consists of the questions that were posed to students and teachers asking them RIR STEM EDUCATION DAY PROGRAM EVALUATION 71 to reflect on how effective they viewed the RIR STEM Education Day program when it came to increasing students’ interest in STEM-related careers. Table 6. Data Sources: Evaluation Question Four Survey Question Two From these experiences, have you developed an interest/awareness of career opportunities within the motorsports industry?” Student Focus Group Question Three Were there stations where you would have liked to spend more time? Was there anything you did that made you think that you would like to learn more about STEM? Teacher Focus Group Questions Five and Six TFGQ5: What did students say or do following the RIR STEM Education Day program that may have indicated an increased interest in STEM career opportunities? TFGQ6: In what ways did the RIR STEM Education Day program promote female and minority students’ interest in STEMrelated education and careers?” RIR STEM Education Day field trip survey. The results from the RIR STEM Education Day field trip survey (See Appendix C) indicated that 63 percent of the respondents answered “Yes” to the question that asked if the program helped them develop an increased awareness of career opportunities within the motor sports industry. This percentage was a strong indicator that the program was effective; however, qualitative data such as student comments from the survey and data from the focus group transcriptions helped evaluators come to this conclusion. A written comment from the student survey spoke directly to the impact the event had on STEM-related careers by saying, “I thought the field trip was excellent, it really opened my eyes to careers involving [STEM].” RIR STEM EDUCATION DAY PROGRAM EVALUATION 72 Student focus groups. The aforementioned comment was consistent with a student focus group interview remark that stated, “It gave me an interest in different types of fields, and this was one of the more mechanical fields of engineering and it gave me a good idea in what field I want to go in.” Comments such as these provided evidence that the RIR STEM Education Day program did have an influence and was effective at increasing students’ interest in STEMrelated careers. Teacher focus group. Upon completion of the teacher focus group interviews, evaluators identified comments that spoke to the program’s effectiveness at increasing students’ interest in STEM-related careers. The following teachers’ comments suggested the program did introduce different career pathways to students who attended the event. One teacher said, “[The event] opened their eyes that they can do this.” Likewise, another teacher stated, “If you expose them to…greater career choices, it just makes you more excited about your future.” Comments from two additional teachers helped evaluators determine the program was effective at increasing students’ interest in STEM-related careers. In particular, one teacher noted, “We talked about careers more from an educational standpoint and used this event in my discussions about career opportunities, how many of these jobs deal with STEM, deal with technology.” A second teacher displayed similar enthusiasm by saying, “They want to go to mechanics school . . . they’re on it but they just need to find out what to do to prepare themselves . . . all of our talk is about careers.” These comments demonstrated to the evaluators that teachers discussed careers with students and that the RIR STEM Education Day program helped to stimulate career discussions, thereby increasing students’ interest in STEM-related careers. RIR STEM EDUCATION DAY PROGRAM EVALUATION 73 Evaluation Question Five: Are there differences in STEM interest and understanding based on gender and ethnicity regarding the RIR STEM Education Day program? There was initial concern that answering evaluation question five would be challenging as there was only limited information from the five data sources; however, the analysis of the information from the focus group transcriptions exposed extremely rich and stimulating insight, from both teachers and students, which revealed there was a difference in “interest” based on gender and ethnicity. There was not, however, evidence found among the data based on gender and ethnicity which would suggest that the program had any effect on “understanding” of STEM. Although survey question three from the RIR STEM Education field trip survey asked students “Do you better understand the science, technology, engineering and math components of racing as a result of this program”, the survey did not provide demographic information. Therefore, the evaluators were not able to compare and contrast a particular level of understanding among students based on race and gender for this particular data set. Table 7 displays questions in which evaluators asked teachers and students during focus group interviews to reflect on whether there were differences in STEM interest and understanding based on gender and ethnicity. Table 7. Data Sources: Evaluation Question Five Student Focus Group Question Five What station made the biggest impact on you? Teacher Focus Group Question Six In what ways did the RIR STEM Education Day program promote female and minority students’ interest in STEM-related education and careers? RIR STEM EDUCATION DAY PROGRAM EVALUATION 74 Student focus groups. When evaluators asked students which station made the biggest impact on them, they made several comments about a variety of stations. Specifically, AfricanAmerican females from all focus groups universally mentioned the same driver who facilitated a particular station. African-American female students consistently mentioned how that driver’s gender impressed upon them that women can participate in male-dominated careers. For example, one female student commented that “. . . the fact that she was a women and she knew how to do all of that stuff just like the men,” influenced her by shattering the misperceptions that STEM careers are less suitable for females. Another female student shared her enthusiasm by stating, “[She]... was an inspiration for young ladies all around the world!” Teacher focus group. When asked in what ways the program promoted female and minority students’ interest in STEM-related education and careers, teachers overwhelmingly responded with comments associated with the importance of exposure to “role models”, and were emphatic in their support concerning this point. One very animated teacher shared his enthusiasm by referring to his students’ reaction to a station facilitated by the young AfricanAmerican female driver. He said, “Once my girls saw [her] . . . you guys can forget it!” This response implied how the attention of his students (all African-American females) was extremely heightened and positive because of their experience at this particular station. When answering a question pertaining to evaluation question one regarding increasing interest in STEM education, one of the teachers had this to say: The young lady really helped. Seeing that she was young, she was African American, and she was a female, the girls were like, ahhhhh, I don’t know if it’s something I want to do, I said, “just go and let’s see” and they saw her and they got excited. RIR STEM EDUCATION DAY PROGRAM EVALUATION 75 A couple of my little girls went up to her and asked ‘how can we get into it?’ In a separate interview, a different teacher reinforced the impact of this individual station by saying, “They saw that girl . . . we need more of that . . . the kids need to see that they can do these things.” The most noteworthy and demonstrative teacher response that highlighted the impact of the African-American female driver’s station was, “[She] herself broke down so many double barriers, again by being a female and being an African-American and there were so many African-American students in the group and I think they were very impressed.” Interestingly, when responding to a focus group question that was intended to address improving students’ understanding of STEM, a veteran teacher responded with this comment: We talked in class about gender misperceptions in Tech. Ed. and CTE. There‘s still that 250 year old Horace Mann philosophy that girls need to sew and guys need to cut wood and do things like that. We’re bringing down those barriers and the best thing for both ethnically and gender wise was when they saw that young girl. Responses such as these from the African-American female students, as well as from teachers, led the evaluators to determine that there were differences in interest based on gender as it uniquely related to this single station. In contrast, none of the male students made any mention of the station facilitated by the female driver. This specific station provided an opportunity for students who participated in the event, regardless of gender or ethnicity, to see an African-American female in a nontraditional career. Evaluators also noticed general differences in interest based on ethnicity. This was made evident during the student focus group interviews and the subsequent data analysis in which the RIR STEM EDUCATION DAY PROGRAM EVALUATION 76 evaluators noticed that minority students were much more enthusiastic and appreciative of the opportunity to attend the event. Minority students recalled specific detailed information about stations, speakers, and activities. For example, one student said, “When I was there, I liked when they were teaching you about the insides of the car. I wanted to learn more about the engines and stuff. It seemed cool.” While non-minority students were excited about the event, some did not seem as passionate as all of the minority students. Some non-minority students even appeared disinterested in the event. An example is when one non-minority female student stated, “trying to figure out. I really don’t remember this day. I just can’t.” Similarly, a non-minority male student stated, “I really don’t remember” when asked what he really liked about the event. The culmination of these data sources and the information obtained led to the determination that there were differences in interest based on ethnicity. Summary The main purpose of this evaluation was to determine the effectiveness of the RIR STEM Education Day program at increasing interest in STEM, improving understanding of STEM, increasing interest in STEM educational and career opportunities, and to determine if there were differences in STEM interest and understanding based on gender and ethnicity. Upon completion of the data analysis, the program was not only found to be effective at meeting the client’s original goal of creating “awareness” of STEM, but was also found to be effective at increasing interest, improving understanding in STEM, and increasing interest in STEM educational and career opportunities. Additionally, evaluators found that there was a difference in “interest” based on gender and ethnicity; however, no evidence was found that would suggest that the program had any effect on “understanding” of STEM. RIR STEM EDUCATION DAY PROGRAM EVALUATION 77 The findings revealed that students held a favorable opinion of the program based on the quantitative data acquired from the RIR STEM Education Day field trip survey. The survey revealed that 73% of the students indicated an increased interest in STEM and STEM-related educational opportunities, while 84% had an improved understanding of STEM, and 63% had an increased interest in STEM-related career opportunities. The findings, particularly qualitative data obtained from the student and teacher focus groups, also revealed that there are opportunities to further enhance the program and build on its current success. The themes that emerged from the focus group transcriptions: Awareness, Making Connections, Role Models and Time, provided insight into what students and teachers perceived to be valuable concerning the RIR STEM Education Day program. These themes spoke to the effectiveness of the overall program through the eyes of the participants and were paramount to obtaining a comprehensive evaluation of the program. Although the focus group data was collected eight months after the program, most students and teachers were able to recall the specifics of the day and could discuss stations that were meaningful and impactful. This type of recollection demonstrates the significance of the program and the lasting impact it made on many of the students and teachers. The program proved that a coordinated and collaborative event such as the RIR STEM Education Day program has the ability to accomplish the goals of the client and meet the needs of the stakeholders. Evaluators concluded that participation in the program was successful at increasing student awareness by making authentic connections to the real world and helping students better understand STEM. RIR STEM EDUCATION DAY PROGRAM EVALUATION 78 SECTION FIVE Recommendations The following recommendations evolved from the “Findings” section of the Richmond International Raceway (RIR) Science, Technology, Engineering, and Math (STEM) Education Day program evaluation. These recommendations address each evaluation question for the purpose of improving the program’s overall effectiveness. Recommendations are ordered by evaluation question, and address how the RIR STEM Education Day program might improve in the areas of, Interest, Understanding, Educational and Career Opportunities, and Gender/Ethnicity for participating students. Evaluation Question One: To what degree did the RIR STEM Education Day program increase middle school students’ interest in STEM education? After-school and community-based programs similar to the RIR STEM Education Day program have been effective in helping students make the connection between STEM-related classroom content and the real-world application of STEM education. “These programs present innovative ways to engage students who may be detached and disinterested during the traditional school day” (Afterschool Alliance, n.d.). By making this connection through early exposure to STEM and through exciting hands-on activities, it is anticipated that middle school students’ interest and engagement in STEM-related education will persist throughout their high school and college years. The following recommendations are proposed to increase students’ interest in STEM education: (a) Increase the number of stations with hands-on activities and (b) invite non-Career and Technical Engineering (CTE) students to participate at the event. RIR STEM EDUCATION DAY PROGRAM EVALUATION 79 Recommendation 1.a: Increase hands-on activities. The inaugural RIR STEM Education Day program presented 20 stations that included interesting topics that captured students’ attention. This was made evident by the many positive comments received during focus group interviews that spoke to increased interest. While this information suggested the event was effective at increasing middle school students’ interest in STEM education, many students advocated for more hands-on activities at the next event. This would allow students to not only see, but to also touch, feel, and observe the key STEM components that station presenters are conveying. When asked about ways to improve the next event, one student enthusiastically commented, “more touchy-touchy and less talky-talky”. That student’s comment resonated with the program evaluators throughout the “Findings” section of this evaluation and perfectly summed up the point for this particular recommendation. Providing students with engaging hands-on activities and staying away from lecture-style stations establishes an environment where the focus is on the student and not the presenter. Student’s comments made it clear that stations that provided hands-on activities were more memorable and impactful than stations that did not offer hands-on activities. Recommendation 1.b: Invite non-CTE students. The second recommendation to RIR involves inviting different students to the event. Under the current arrangement, the event is limited in its catchment and may not be introducing enough students from diverse subject areas to STEM education. In general, middle school students who take CTE classes already have some awareness of STEM and know whether they are interested in this type of education; however, students in non-CTE classes may not be aware that STEM education exists as a viable RIR STEM EDUCATION DAY PROGRAM EVALUATION 80 option. Exposing non-CTE middle school students to events like the RIR STEM Education Day program increases the likelihood that the experience could spark an interest in STEM education. Evaluation Question Two: To what degree did the RIR STEM Education Day program improve student understanding of STEM education? Literature on improving understanding of STEM shows that if students are enjoying a task or looking forward to working on an assignment then the likelihood of them gaining a better understanding of the subject at hand increases (Angle, 2011). A strong partnership between schools and universities can provide teachers with professional development programs to produce strong outcomes for students in STEM education. This is particularly true with STEMrelated curriculum giving opportunities to all stakeholders to strengthen and inspire learning within their own communities (Clark & Button, 2011). The following recommendations offer guidance to improve students’ understanding of STEM by creating and maintaining a firm connection to classroom activities and real-world experiences. Evaluators suggest: (a) provide teachers with pre and post event materials; (b) continue collaborating with key community partners; and (c) allocate more time at each station. Recommendation 2.a: Provide teachers with pre and post event materials. Teacher responses to focus group questions suggested that the program could improve students’ understanding of STEM education if RIR provided pre and post event materials to the schools. Providing middle school teachers with pre-event materials would give them an opportunity to connect classroom content to the stations’ topics before the event takes place. Teachers mentioned that having information pertaining to program activities ahead of time will help them plan classroom content accordingly. RIR STEM EDUCATION DAY PROGRAM EVALUATION 81 Acquiring post-event materials was also mentioned by teachers. A follow-up packet that describes the program content and how the different areas of STEM related to each station topic would enhance the program significantly. The follow-up packet would help make a stronger connection between what students experienced at the event and what they learn in the classroom, as well as how these STEM components relate to real-world application. Pre and post event materials could also act as a two-way communication system between schools and program administrators. Through this type of collaboration, middle school teachers could inform program administrators about what topics to present at the event. These topics could help drive home important elements of classroom curriculum specific to STEM. This type of collaboration would also assist administrators with getting appropriate STEM-related resources into the hands of middle school teachers before and after the event. Recommendation 2.b: Collaborate with local educational organizations. Evaluators also recommend that RIR continue to collaborate with local educational organizations such as the local school district, Communities in Schools, and the MathScience Innovation Center. This recommendation will not only enhance the academic quality and validity of the program, but will also provide a higher level of educational expertise and rigor to the development of station content to enrich student understanding. Educational support and guidance from these groups would help bring community members to the table who can help sustain events such as the RIR STEM Education Day program. A strong connection between schools and community partners could enhance students’ educational experiences (Epstein, 1995). Specifically, the importance of teachers collaborating with local schools and educational organizations can provide optimal learning experiences for students. RIR STEM EDUCATION DAY PROGRAM EVALUATION 82 Recommendation 2.c: Allocate more time at hands-on stations. It is recommended that more time be allocated to each hands-on station to improve students’ understanding of STEM. Time can be approached from different perspectives. More time for student engagement at the stations can be accomplished in several ways. First, reduce wait time at the beginning of the event and the time taken by guest speakers to open the event. This will allow more time for students to be directly engaged in the station experiences. Secondly, increase event time and time participating at hands-on stations. Lastly, reducing the amount of transition time between stations would decrease the time students are idle and unengaged, and allow for recoupment of time to be allotted to each station. Evaluation Question Three: To what degree did the RIR STEM Education Day program increase students’ interest in STEM educational opportunities? Research indicates “middle school is a ‘make or break’ time for young adolescents academically” and “it is during middle school that many youth…turn away from math and science” (Metlife Afterschool Alert, 2010, p. 2). Postponing STEM education until high school does not provide students adequate time to acquire the prerequisite skills necessary to be successful in college courses that lead to STEM careers (Whitehead, 2011). By the time eighth graders attend the RIR event they are only months away from entering high school which might indicate that the most formative years are already behind them. With the goal of increasing students’ interest in STEM educational opportunities as the guiding force, it is recommended that the program be extended to a younger audience. Recommendation 3: Invite 6th or 7th graders. Evaluators recommend extending an invitation to 6th or 7th graders. Evaluators recognized that the RIR STEM Education Day program increased interest in STEM educational opportunities among eighth grade participants; RIR STEM EDUCATION DAY PROGRAM EVALUATION 83 however, eighth graders have already selected their high school academic path prior to the date the event. Teacher comments supported the idea that more students might take advantage of the school district’s STEM-related specialty centers if they had the opportunity to experience the RIR event prior to their eighth grade year. This recommendation would provide more awareness of educational opportunities as students begin to think about life beyond middle school. Evaluation Question Four: To what degree did the RIR STEM Education Day program increase students’ interest in STEM-related careers? Although increasing interest in STEM-related careers was not the focus of the event, with minor program modifications there is an opportunity to be more effective at increasing interest in this area. Through the review of the literature it was found that collaboration with community partners ensures students receive better preparation for a future in STEM, which will create economic growth and alleviate outsourcing of jobs to other countries (Office of the Press Secretary, 2009). Focused collaboration among community stakeholders can increase opportunities for students to gain relevance and real-world application of STEM curriculum. In order to increase interest in STEM-related careers, evaluators offer the following two recommendations. First, bring together local businesses that are associated with the motorsports industry to discuss ways to create interest in STEM-related careers. Secondly, provide graphic displays to provide exposure to career options at the event that list various STEM-related careers that are available throughout the motorsports industry. Recommendation 4.a: Collaborate with local businesses. The primary recommendation associated with increasing interest in STEM-related careers is for RIR to consider initiating community roundtable events that would bring local business partners RIR STEM EDUCATION DAY PROGRAM EVALUATION 84 together in support of improving interest in STEM-related careers. The focus would be on creating innovative ways to encourage more students to study and pursue careers in STEMrelated fields, which could also serve as a logical pathway for developing a more skilled workforce in STEM. Research suggests that business and community collaboration can provide resources and real-world experiences that give students more opportunities to engage in innovative solutions to solve real-world challenges (Carraway, Rectanus, & Ezzell, 2012). Recommendation 4.b: Expose students to career options. Evaluators recommend that RIR create graphic displays that list possible STEM-related career opportunities within the motorsports industry. The displays would include eye-catching images of the numerous jobs related to the motorsports industry and associated salary ranges. The displays would be posted in prominent locations throughout the event venue where students congregate between station rotations. Providing this information would expand students’ awareness of STEM-related career opportunities by expanding their knowledge of each career path. Evaluation Question Five: Are there differences in STEM interest and understanding based on gender and ethnicity regarding the RIR STEM Education Day program? Female and minority students are scarce among STEM majors, and female and minority teachers in STEM are practically non-existent (Beasley & Fischer, 2012). America’s overall participation in STEM education has been on a decline, but female and minority students are particularly uncommon in STEM-related education and careers. For instance, the number of African-American students in the areas of engineering and technology has declined when compared to other ethnicities (STEM Connector, 2012). In addition, female students are three times less likely to pursue STEM-related majors and careers when compared to male RIR STEM EDUCATION DAY PROGRAM EVALUATION 85 students. This information serves as a call to action for educators and policy makers to promote the importance of STEM education and careers from K-12 education to the university level. During the analysis of the collected data, the evaluation revealed a difference in “interest” based on gender and ethnicity, especially among the African-American female participants. Because if this, evaluators recommend that RIR continue to include stations that highlight the accomplishments of females and minorities and to invite diverse groups to this program. Recommendation 5: Continue to highlight females and minorities. Evaluators recognized the importance of having females and minorities represented at the program and recommend that RIR continue to highlight the importance of females and minorities in the motorsports industry, particularly females and minorities in STEM-related education and careers. By having more females and minorities as station facilitators, RIR will continue to make a connection between female and minority students as well as STEM-related opportunities within the motorsports industry. This recommendation is supported by Lockheed Martin’s Stephanie Hill (2013), who pointed out the importance for students in STEM classes to have a stimulating curriculum, outstanding teachers, and real-world experiences; however, another important piece of the puzzle for female and minority students interested in STEM is to have role models who "look like them.” While emphasizing the accomplishments of females and minorities is important to increasing interest in STEM, it is recommended that RIR take an additional step by purposefully inviting diverse participants. In doing so, more female and minority students will gain additional exposure to STEM education, which might inspire students to learn and understand more about STEM-related educational and career opportunities. This is an additional important step because, “40% of the country’s female and underrepresented minority…engineers working in RIR STEM EDUCATION DAY PROGRAM EVALUATION 86 their field today faced discouragement from pursuing their STEM career at some point in their lives” (Bayer Corporation, 2012, p. 323). Summary of Recommendations The program was found to be effective at meeting the client’s goals of increasing students’ interest in STEM education, improving students’ understanding of STEM, and increasing students’ interest in STEM educational and career opportunities. Evaluators also found that there were differences in students’ “interest” in STEM related to gender and ethnicity; however, no differences were found regarding students’ “understanding” of STEM as it relates to gender and ethnicity. Evaluators propose these recommendations to provide the client with the opportunity to enhance the delivery and subsequent outcome of the RIR STEM Education Day program. In order to deliver the most effective and comprehensive program, evaluators made the following nine recommendations: 1. Increase hands-on activities; 2. Invite non-CTE students; 3. Provide teachers with pre and post event materials; 4. Collaborate with local educational organizations; 5. Allocate more time at hands-on stations; 6. Invite 6th or 7th graders; 7. Collaborate with local businesses; 8. Expose students to career options; 9. Continue to highlight females and minorities. RIR STEM EDUCATION DAY PROGRAM EVALUATION 87 Recommendations for Future Evaluation The following recommendations for future evaluation suggest ideas that RIR may want to consider when planning the second STEM Education Day program. Recommendations include obtaining permission from the participating school district to conduct the evaluation, making arrangements to conduct student and teacher focus group interviews within 30 days after program, and creating a field trip survey specific to this evaluation. Due to challenges evaluators encountered during the collection of qualitative data for this evaluation, it is recommended that RIR obtain permission from the school district’s Department of Research and Planning prior to the date of the next program. In order to meet school district policy guidelines, the process for obtaining permission to conduct the program evaluation must be started early. This will allow for future evaluators to begin the evaluation process immediately following the program. By obtaining permission early, future evaluators will be able to schedule student and teacher focus group interviews in advance of the program. This will provide students and teachers an opportunity to share their thoughts and experiences with the program still fresh in their mind. This step is essential due to the limited amount of time between the date of the program and the end of the academic school year. If evaluators are unable to conduct focus group interviews within the April-June timeframe, qualitative data collection will be prolonged until the next academic school year. It is recommended that a survey be created specifically for the next program evaluation. This should be done to adequately address the client’s goals for future programs and should be administered to students immediately upon the conclusion of the event. This will allow future evaluators to capture valuable data specific to future evaluation questions. Lastly, it is suggested RIR STEM EDUCATION DAY PROGRAM EVALUATION 88 that future program evaluators carefully examine the recommendations proposed in this evaluation in order to determine if the recommendations were successful at increasing interest in STEM education, improving students’ understanding of STEM, increasing students’ interest in STEM-related educational and career opportunities, and if the program continued to be successful at increasing interest among females and minorities. Conclusion The goal of this program evaluation was to determine the degree in which the RIR STEM Education Day program increased students’ interest in STEM, improved students’ understanding of STEM, increased students’ interest in STEM educational and career opportunities, and to determine if there were differences in STEM interest and understanding based on gender and ethnicity. The first step toward accomplishing this goal was a comprehensive review of literature. Emerging from the review of literature were three overarching themes; Improving the Appeal of STEM, Female and Minority Students in STEM, and the Need for Collaboration. To improve the appeal of STEM, literature indicated that course content should connect directly to students’ interest and to what motivates them to learn. If attitudes regarding STEM are to change, teachers must work toward making curriculum and lessons relevant to students. Teachers should utilize pedagogical strategies that focus on hands-on experiences and project-based learning that concentrate on creative games and manipulatives, while blending a real-world element into the curriculum or instructional activity. The second literature review theme addressed females and minorities in STEM. Research indicated that female and minority students are underrepresented in STEM education and was a contributing factor to America’s decline as a global leader; therefore, the task for educators in the United States is to increase female and minority students’ participation in STEM RIR STEM EDUCATION DAY PROGRAM EVALUATION 89 education by providing support and making it a positive experience. It is vital for educational leaders to create and promote an academic environment that welcomes all students to STEM education. The need for collaboration is essential to effectively develop a program that would accomplish the goals of RIR and program stakeholders. A strategic collaboration between schools, universities, business and community partners could enhance students’ educational experiences and foster excellence in K-12 STEM education. Schools can collaborate with universities to improve the quality of content and experiences. By collaborating with schools, these partnerships give students increased opportunities to “learn by doing” through hands-on activities. Furthermore, by collaborating with business and community partners, schools can provide students with real-world hands-on experiences that far exceed the traditional classroom environment. As a result, learning can be simplified so that students are motivated to probe deeper into STEM topics that interest them on an individual level. If the United States is serious about addressing the decline of the country’s position as a global leader, a paradigm shift is paramount. Educators and community leaders can improve the appeal of STEM by making it more understandable, relatable, and appealing to young students. Leaders should encourage and inspire female and minority students to participate in STEMrelated programming, as well as to build bridges with community partners to establish a strategic collaborative relationship between students, schools, and communities. Upon completion of the literature review and the subsequent analysis of the data collected, the findings revealed that participating students held a favorable opinion of the RIR STEM Education Day program based on quantitative data acquired from the RIR STEM Education Day field trip survey. Qualitative data obtained through student and teacher focus RIR STEM EDUCATION DAY PROGRAM EVALUATION 90 group interviews also revealed that the program was effective at increasing student interest and that there are opportunities to further enhance the program and build on its current success. The themes that emerged from the focus group interview transcriptions: Awareness, Making Connections, Role Models and Time, provided additional insight into what students and teachers perceived to be valuable concerning the RIR STEM Education Day program. These themes spoke to the effectiveness of the overall program through the eyes of the participants and were paramount to obtaining a comprehensive evaluation of the program. The evaluation found that a coordinated and collaborative event such as the RIR STEM Education Day program was effective at accomplishing the goals of RIR, as well as meeting the needs of the program stakeholders. RIR STEM EDUCATION DAY PROGRAM EVALUATION 91 References Afterschool Alliance, (n.d.). Afterschool and summer programs: Committed partners in stem education. Retrieved from http://www.afterschoolalliance.org/STEM_JointPositionPaper.pdf. Angle, J. (2011). Drivers, start your glue guns. Using model stock cars to explore motion and force concepts. Science Scope, 35(4), 45-51. Bayer Corporation (April 2012). Bayer facts of science education XV: A view from the gatekeepers-STEM department chairs at America’s top 200 research universities on female and underrepresented minority undergraduate STEM students. (2012). Journal of Science Education & Technology, 21(3), 317-324. doi:10.1007/s10956-012-9364-1. Beering, S.C., (2009). Actions to improve science, technology, engineering, and mathematics (STEM) Education for all American students. National Science Board; National Science Foundation. January 11, 2009. Retrieved from http://www.nsf.gov/nsb/publications/2009/01_10_stem_rec_obama.pdf. Beasley, M. A., & Fischer, M. J. (2012). Why they leave: The impact of stereotype threat on the attrition of women and minorities from science, math and engineering majors. Social Psychology of Education: An International Journal, 15(4), 427-448. Bryan, R. R., Glynn, S. M., & Kittleson, J. M. (2011). Motivation, achievement, and advanced placement intent of high school students learning science. Science Education, 95(6), 1049-1065. Brown, J. (2013). STEM ed. coalition statement on strengthening America’s schools-act-esealegislation. Retrieved from http://www.stemedcoalition.org/2013/06/11/stem-edcoalition-statement-on-strengthening-americas-schools-act-esea-legislation/. RIR STEM EDUCATION DAY PROGRAM EVALUATION 92 Carraway, A., Rectanus, K., & Ezzell, M. (2012). The do-it-yourself guide to STEM community engagement. Retrieved from http://www.ncpublicschools.org/docs/stem/resources/diy-guide.pdf. Clark, B., & Button, C. (2011). Sustainability transdisciplinary education model: interface of arts, science, and community (STEM). International Journal of Sustainability in Higher Education, 12(1), 41-54. Connelly, L.M (2009) Mixed methods studies. MEDSURG Nursing. Vol. 18, No. 1. Creswell J.W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches. Sage. Chute, E. (2009). Lack of diversity part of equation in STEM fields. Colleges try to increase numbers of women, minorities in science and engineering. Pittsburgh Post-Gazette. Retrieved from http://www.post-gazette.com/pg/09041/947952-298.stm#ixzz0g1ceZiGK. Dugger, W. E. (2011). STEM: Some basic definitions – a compilation. Reston, VA, ITEEA. Retrieved from www.iteaconnect.org/Resources/PressRoom/STEMDefinition.pdf . Eberle, F. (2010). Why STEM education is important. InTech, 57(5). Epstein, J. (1995). School/Family. Community partnerships: Caring for the children we share. Phi Delta Kappan May, 701-712. Falk, J., & Dierking, L. (2010). The 95% solution. American Scientist, 98, 486-493. Gannod, G. C., Burge, J. E., & Helmick, M. T. (2008, May). Using the inverted classroom to teach software engineering. In Proceedings of the 30th International Conference on Software Engineering (pp. 777-786). ACM. Garibaldi A. M. (1992). Educating and motivating African American males to succeed. Journal of Negro Education, 61, 4–11. RIR STEM EDUCATION DAY PROGRAM EVALUATION 93 Hylton, P. (2010). Using motorsports design concepts to further STEM education. Journal of Technology Studies, 36(1), 12-15. Retrieved from http://search.proquest.com/docview/822505610?accountid=14780. Hill, S. C. (2013). Making STEM cool. The Baltimore Sun. Retrieved from http://search.proquest.com/docview/1426249605?accountid=458. King, J. P., Hernandez, J. M., & Lott, J. L. (2012). Using existing data sources/programs and multilevel modeling techniques for questions in institutional research. New Directions for Institutional Research, 2012(154), 41-55. Koch, D., Johnson, M. E., & Marshall, B. H. (2013). Connecting k-12 teachers to STEM careers through industry collaboration. Conferences for Industry and Education Collaboration. Retrieved from http://www.indiana.edu/~ciec/Proceedings_2013/ETD/ETD455_KochJohnson-Marshall.pdf. Kramer, D., (2011). Solar decathlon has wetlands and wool but little sunshine. Physics Today 64(11), 29 (2011); doi: 10.1063/PT.3.1329. Jang, H., Reeve, J., & Deci, E. L. (2010). Engaging students in learning activities: It is not autonomy support or structure but autonomy support and structure. Journal of Educational Psychology, 102(3), 588. Lacey, G. (2010). 3D printing brings designs to life. Tech Directions. Retrieved from http://www.researchgate.net/publication/234638208_Get_Students_Excited-3D_Printing_Brings_Designs_to_Life RIR STEM EDUCATION DAY PROGRAM EVALUATION 94 Mckay, M., Klein-Gardner, S. S., Zook, K. A., Yoder, M., Moskal, B. M., Hacker, M., & Houchens, B. C. (2011). Best practices in K-12 and university partnerships panel winners ASEE k-12 and pre-college engineering division. In American Society for Engineering Education. Merrill, C., & Daugherty, J. (2010). STEM education and leadership: A mathematics and science partnership approach. Retrieved from http://scholar.lib.vt.edu/ejournals/JTE/v21n2/merrill.html. Mertens, D. M., & Wilson, A. T. (2012). Program evaluation theory and practice: A comprehensive guide. Guilford Press. McNally, T. (2012). Innovative teaching and technology in the service of science: Recruiting the next generation of STEM students. Journal of the Scholarship of Teaching and Learning, 12(1), 49-58. Retrieved from http://search.proquest.com/docview/1037909443?accountid=14780. Metlife Afterschool Alert. (September 2010). Afterschool: Middle school and science, technology, engineering, and math (STEM). Issue Brief No. 44. Retrieved from http://www.afterschoolalliance.org. Morgan, D. L., & Spanish, M. T. (1984). Focus groups: A new tool for qualitative research. Qualitative sociology, 7(3), 253-270. New York Teacher Centers, (2008). Focus groups. Retrieved from http://www.sjsu.edu/people/fred.prochaska/courses/ScWk242Spring2013/s2/New%20Y ork%20State%20Teachers%20Focus%20Groups.pdf. Nichols, P. (2008, September). The need for advanced studies options in Virginia. Retrieved from http://www.virginiaadvancedstudystrategies.org/newsletter.pdf. RIR STEM EDUCATION DAY PROGRAM EVALUATION 95 Office of the Press Secretary. (2009). President Obama launches - Educate to innovate campaign for excellence in science, technology, engineering & math (STEM) education. http://www.whitehouse.gov/the-press-office/president-obama-launches-educate-innovatecampaign-excellence-science-technology-en. O'Meara, K. (2007). Stepping up: How one faculty learning community influenced faculty members' understanding and use of active-learning methods and course design. Journal on Excellence in College Teaching, 18(2), 97-118. Retrieved from http://search.proquest.com/docview/815956214?accountid=14780. Ornstein, A. C. (2010). Achievement gaps in education. Society, 47(5), 424-429. Packard, B. W. L. (2012). Effective outreach, recruitment, and mentoring into STEM pathways: Strengthening partnerships with community colleges. Community Colleges in the Evolving STEM Education Landscape: Summary of a Summit, 57. Report to the President. (September, 2010). Prepare and inspire: K-12 education in science, technology, engineering, and math (STEM) for America’s future. President’s Council of Advisors on Science and Technology: Executive Report. Repsys, S. (2009). NASCAR’s history at Richmond international raceway. Retrieved from http://voices.yahoo.com/nascars-history-richmond-international-raceway3216158.html?cat=37. Reeve J., & Jang H. (2006). What teachers say and do to support students_ autonomy during a learning activity. Journal of Educational Psychology, 98(1), 209–218. Rinehart, J., & Watson, K. (1998). A campus climate survey at Texas A&M University. Proceedings of the 1998 women in engineering conference: Creating a global engineering community through partnerships, (93-100), West Lafayette, IN. RIR STEM EDUCATION DAY PROGRAM EVALUATION 96 Rockland, R., Bloom, D. S., Carpinelli, J., Burr-Alexander, L., Hirsch, L. S., & Kimmel, H. (2010). Advancing the “E” in k-12 STEM education. Retrieved from http://scholar.lib.vt.edu/ejournals/JOTS/v36/v36n1/rockland.html. Royse, D. (2008) Research Methods in Social Work. Nelson-Hall, Chicago. Sanders, M. (2009). Stem, stem education, stemmania. The Technology Teacher, 68(4), 20-26. Stake, R. E. (1975). Evaluating the arts in education: A responsive approach. Merrill Publishing Company. Steele, J., James, J. B., & Barnett, R. C. (2002). Learning in a man’s world: Examining the perceptions of undergraduate women in male-dominated academic areas. Psychology of Women Quarterly, 26(1), 46–50. STEMconnector. (2012). Research on gender in science and engineering (NSF). Retrieved from http://stemconnector.org/genderresearchNSF. Torres, K. C., & Charles, C. Z. (2004). Metastereotypes and the black-white divide: A qualitative view of race on an elite college campus. Du Bois Review: Social Science Research on Race, 1(1), 115-149. Yassie-Mintz, E. (2010). Leading for engagement. Principal Leadership. 54-58. Retrieved from http://www.principals.org. Whitehead, S. H. (2011). Relationship of robotic implementation on changes in middle school students' beliefs and interest toward science, technology, engineering and mathematics. Proquest , 93-93. Retrieved from http://search.proquest.com/docview/964173108?accountid=14780. RIR STEM EDUCATION DAY PROGRAM EVALUATION Whitehouse, (2010). Educate to innovate. Retrieved from http://www.whitehouse.gov/issues/education/k-12/educate-innovate. Wilcox, D. R., Roberts, S., & Wilcox, D. (2010). Gravity Racers. Science and Children, 47(7), 19-23. Woolley, C. M. (2009). Meeting the mixed methods challenge of integration in a sociological study of structure and agency. Journal of Mixed Methods Research, 3(1), 7-25. 97 RIR STEM EDUCATION DAY PROGRAM EVALUATION 98 Appendix A – CTE 8403 Block Demographics School Block Total # Students 2nd 14 14:0 5th 20 18:2 2nd 17 13:4 2nd 17 12:5 1st 22 17:5 Male: Female Racial Demographics School A 72% 14% 7% 7% 45% 35% 15% 5% African American (10) Hispanic (2) Caucasian (1) Asian (1) African American (9) Caucasian (7) Asian (3) Hispanic (1) 88% 12% 88% 6% 6% Caucasian (15) Asian (2) Caucasian (15) Asian (1) Hispanic (1) School B School C 86% Caucasian (19) 14% African American (3) School D No data School E 3rd 12 10:2 59% 25% 8% 8% 5th 13 12:1 70% Caucasian (9) 15% Hispanic (2) 15% Asian (2) 6th 21 19:2 62% 19% 9.5% 9.5% Caucasian (13) African American (4) Hispanic (2) Asian (2) 4th 29 22:7 5th 28 25:3 59% 31% 7% 3% 61% 21% 14% 4% Caucasian (17) African American (9) Asian (2) Hispanic (1) Caucasian (17) Asian (6) African American (4) Hispanic (1) Caucasian (7) Asian (3) African American (1) Hispanic (1) School F School G School H RIR STEM EDUCATION DAY PROGRAM EVALUATION 99 2nd 22 21:1 91% Caucasian (20) 4.5% African American (1) 4.5% Hispanic (1) 2nd 25 19:6 4th 15 15:0 6th 19 16:3 96% 4% 80% 20% 95% 5% 1st 24 22:2 4th 24 23:1 2nd 21 19:2 5th 24 21:3 6th 25 25:0 School I African American (24) Caucasian (1) African American (12) Caucasian (3) African American (18) Hispanic (1) School J 67% 12.5% 12.5% 4% 4% 75% 13% 8% 4% Caucasian (16) Am. Indian/Alaska Native (3) Hispanic (3) African American (1) Asian (1) Caucasian (18) Am. Indian/Alaska Native (3) Asian (2) African American (1) 71% 19% 5% 5% 63% 17% 12% 8% 56% 20% 16% 8% Caucasian (15) African American (4) Hispanic (1) Asian (1) Caucasian (15) Hispanic (4) African American (3) Asian (2) Caucasian (14) African American (5) Hispanic (4) Asian (2) School K School L No data Summary A total of 392 eighth grade students took the Technology Foundations course across 10 (should be 12, but no data for Fairfield or Wilder) schools in Henrico County. More than half of these students were Caucasian, representing 57 percent. An overwhelming majority of the students were male, with only 49 females out of the total 392 students. o o o o o o o 28% African American (109) 1.5% American Indian/Alaska Native (6) 8% Asian (30) 57% Caucasian (222) 5.5% Hispanic (25) 87.5% males (343) 12.5% females (49) RIR STEM EDUCATION DAY PROGRAM EVALUATION 100 Appendix B - 2013/2014 CTE 8403 Course Description and Competency-Based Task/Competency List All tasks are considered essential statewide and are required of all students. Course Description Course Code: 8403 Duration: 36 weeks Suggested Grade Levels: 9–11 (http://teachers.henrico.k12.va.us/CTE/home.htm) In this beginning high school course, students acquire a foundation in technological resources including material, energy, and information and apply processes associated with the technological thinker. Challenged by laboratory activities, students create new ideas and innovations, build systems, and analyze technological products to learn further how and why technology works. They work in groups to build and control systems using engineering design in the development of a technology. Career Clusters and Pathways For federal reporting (choose one) Manufacturing o Health, Safety, and Environmental Assurance o Logistics and Inventory Control o Manufacturing Production Process Development o Production o Quality Assurance Science, Technology, Engineering and Mathematics o Engineering and Technology o Science and Mathematics RIR STEM EDUCATION DAY PROGRAM EVALUATION 101 Demonstrating Workplace Readiness Skills: Personal Qualities and People Skills 1. Demonstrate positive work ethic. 2. Demonstrate integrity. 3. Demonstrate teamwork skills. 4. Demonstrate self-representation skills. 5. Demonstrate diversity awareness. 6. Demonstrate conflict-resolution skills. 7. Demonstrate creativity and resourcefulness. Demonstrating Workplace Readiness Skills: Professional Knowledge and Skills 8. Demonstrate effective speaking and listening skills. 9. Demonstrate effective reading and writing skills. 10. Demonstrate critical-thinking and problem-solving skills. 11. Demonstrate healthy behaviors and safety skills. 12. Demonstrate an understanding of workplace organizations, systems, and climates. 13. Demonstrate lifelong-learning skills. 14. Demonstrate job-acquisition and advancement skills. 15. Demonstrate time-, task-, and resource-management skills. 16. Demonstrate job-specific mathematics skills. 17. Demonstrate customer-service skills. Demonstrating Workplace Readiness Skills: Technology Knowledge and Skills 18. Demonstrate proficiency with technologies common to a specific occupation. 19. Demonstrate information technology skills. 20. Demonstrate an understanding of Internet use and security issues. 21. Demonstrate telecommunications skills. RIR STEM EDUCATION DAY PROGRAM EVALUATION 102 Examining All Aspects of an Industry 22. Examine aspects of planning within an industry/organization. 23. Examine aspects of management within an industry/organization. 24. Examine aspects of financial responsibility within an industry/organization. 25. Examine technical and production skills required of workers within an industry/organization. 26. Examine principles of technology that underlie an industry/organization. 27. Examine labor issues related to an industry/organization. 28. Examine community issues related to an industry/organization. 29. Examine health, safety, and environmental issues related to an industry/organization. Addressing Elements of Student Life 30. Identify the purposes and goals of the student organization. 31. Explain the benefits and responsibilities of membership in the student organization as a student and in professional/civic organizations as an adult. 32. Demonstrate leadership skills through participation in student organization activities, such as meetings, programs, and projects. 33. Identify Internet safety issues and procedures for complying with acceptable use standards. Using Information as a Technological Resource 34. Obtain information through interviews, computer databases, and media resource centers. 35. Convey three-dimensional objects on a two-dimensional surface, using conventional and computer technology. 36. Use various technologies to prepare and present information regarding technological solutions. 37. Identify career opportunities in a variety of technological systems. RIR STEM EDUCATION DAY PROGRAM EVALUATION 103 Using Materials as a Technological Resource 38. Identify materials by the major family groups. 39. Trace materials used in common products to their source. 40. Use tools, machines, and processes to change materials. 41. Select materials for specific applications according to their properties. 42. Recycle materials for useful means. 43. Apply safety procedures and practices. Using Energy as a Technological Resource 44. Identify the sources of energy used in technological devices. 45. Describe major forms of energy. 46. Control the use of energy with mechanical, electrical, fluidic, and thermal systems. 47. Conserve energy through technological modification. Analyzing Consumer Products 48. Identify ways that consumer products have impacted people, society, and the environment. 49. Select a consumer product to analyze. 50. Gather product information. 51. Record information about the product, using computer technology. 52. Present information about the product, using computer technology. 53. Analyze how the product works, using mathematical and scientific concepts. 54. Identify an innovation that would improve the product. 55. Draw illustrations or construct models of the improved product. 56. Construct a display to explain the improved product, emphasizing mathematical and scientific concepts. 57. Report results of data collection, product analysis, and product improvement. Understanding Technological Systems 58. Explain the concept of a technological system. 59. Distinguish between a system and a subsystem. 60. Describe the impacts of technological systems on people, society, and the environment. RIR STEM EDUCATION DAY PROGRAM EVALUATION 104 Designing and Building a System Controlled by Computers or Electronics 61. Analyze a problem whose solution uses computer or electronic controls. 62. Obtain information on the computer- or electronically-controlled options for solving the problem. 63. Design the computer- or electronics-controlled application to be used for solving the problem. 64. Construct a model of the application. 65. Control the application with computers or electronics. 66. Present information about the application, demonstrating how the controls work. Designing a New or Improved Product to Solve a Problem 67. Identify the needs and wants of people in school, home, community, or world that could be solved through technological change. 68. Write a statement of the technological problem to be solved. 69. Gather information about the technological problem to be solved. 70. Create ideas, sketches, notes, and graphics of alternative solutions. 71. Select the best solution by applying knowledge of mathematics, science, technology, and other subjects. 72. Build a prototype of the best solution. 73. Evaluate the solution by comparing it with the problem statement. 74. Present the new or improved product to others. RIR STEM EDUCATION DAY PROGRAM EVALUATION 105 Appendix C – RIR STEM Education Day Field Trip Survey 1. Has your experienc e through 3. Do you classroo better m understan discussio d the ns and science, multiple technolog racing 2. From these y, programs experiences, engineeri increased have you ng, and your developed an math interest interest/awaren compone or ess of career nts of awarenes opportunities racing as s in the within the a result of Timesta sport of motor sports this mp racing? industry? program? 4. Do you have an interest in learning more about the technic al educati 5. Do on you progra plan ms on offered contin within uing the your Henrico educat County ion Public after 6. The School high middle District school school I ? ? attend is: 4/29/201 3 8:18:38 Yes Yes 4/29/201 3 8:24:10 4/29/201 3 8:24:13 4/29/201 3 8:25:43 4/29/201 3 8:25:45 4/29/201 3 8:27:23 4/29/201 3 8:28:21 4/29/201 3 8:35:57 4/29/201 3 8:36:47 4/29/201 3 8:40:54 Yes Yes Yes Yes No Yes Yes Yes Yes Yes No Yes Yes No No Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes No Yes 7. Additional Comments about the fieldtrip and how it could be improved: School C School C School C There could be more Nascar racers to attend and say what it is like. School C I liked getting to sit in the car. School C N/A School C School C less talking School C Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes School C Yes Yes Yes Yes Yes School J School J No Yes Yes Yes Yes Yes Yes Yes 4/29/201 3 8:41:07 Yes 4/29/201 3 8:41:09 Yes School J more food it should be planned out to the second Make the stations a little bit more interesting. RIR STEM EDUCATION DAY PROGRAM EVALUATION 4/29/201 3 8:41:22 Yes 4/29/201 3 8:41:22 No 106 School J Yes Yes Yes Yes No Yes No Yes School J School J 4/29/201 3 8:41:36 Yes 4/29/201 3 8:41:39 Yes Yes Yes Yes Yes Yes No No Yes School J School J 4/29/201 3 8:41:40 Yes Yes Yes No Yes School J 4/29/201 3 8:41:42 Yes No Yes Yes Yes School J 4/29/201 3 8:41:42 Yes Yes Yes Yes Yes School J 4/29/201 3 8:41:51 Yes No Yes No Yes School J 4/29/201 3 8:41:57 Yes No Yes Yes Yes School J 4/29/201 3 8:42:00 Yes Yes Yes Yes Yes School J 4/29/201 3 8:42:09 Yes Less blatant sponsorship Make the students actually met one of the people on the track and go on the track. Yes Yes Yes Yes I have grown a great interest in Nascar Racing after this feild trip. Better communication, I could not hear when to change stations and when to go to the race track. This caused a lot of confusion between the students. Try to make time for kids to see how an actual pit stop works and perform one in front of them. If there was someway that they could make it warmer while we watch the cars race that would make the field trip a lot better. Because we couldn't enjoy watching the cars go by because it was too cold. Make the time for the different stattions shorter. That way we would have more time to see different stations. The groups could have been more orgainized which would have given me a better oppertunity to learn more. Very fun and informative. Provide earplugs/headphone RIR STEM EDUCATION DAY PROGRAM EVALUATION 107 s for observing race. 4/29/201 3 8:42:13 Yes School J No Yes Yes Yes School J 4/29/201 3 8:42:20 Yes Yes No No Yes School J 4/29/201 3 8:42:41 Yes Yes Yes Yes Yes School J 4/29/201 3 8:43:03 4/29/201 3 8:44:07 4/29/201 3 9:18:50 4/29/201 3 9:19:54 4/29/201 3 9:20:17 4/29/201 3 9:21:42 4/29/201 3 9:21:45 Yes Yes Yes Yes Yes Yes No No Yes Yes Yes Yes Yes Yes Yes School J Yes Yes School A School A Yes Yes Yes No Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes It was really fun. i think it would better if we had the chance during the field trip to watch an arena race or if we had other food options other than pizza and water There didnt need to be any improvements because it was a great trip. maybe last a little longer, or we could get to meet a driver. also we could get more than one peice of pizza because i was really hungery It was great School A School A School A School A 4/29/201 3 9:21:51 No 4/29/201 3 9:23:04 Yes 4/29/201 3 9:23:04 Yes No No Yes Yes No Yes No Yes Yes Yes Yes Yes School A School A School A 4/29/201 3 9:23:06 Yes 4/29/201 3 9:24:22 Yes No Yes No Yes Yes Yes Yes Yes It was really cool, i've learned alot since i've been there & the stuff i learned has been a great time and it is making me realize how great national racing is. School A School A 4/29/201 3 9:24:27 Yes One of the administraters where stalking me and my group Yes Yes Yes Yes - I think their should be more hands on activities other than RIR STEM EDUCATION DAY PROGRAM EVALUATION 108 that it was a great fieldtrip 4/29/201 3 9:24:27 4/29/201 3 9:24:39 4/29/201 3 9:25:18 4/29/201 3 9:25:45 4/29/201 3 9:26:04 Yes No Yes Yes No School A it was fun Yes No Yes Yes Yes School B School B couldn't go No No Yes Yes Yes School B No Yes Yes Yes Yes Yes Yes No No No the pizza was a good to eat MORE TIME AND TALK TO RACERS It was pretty much all good, I wouldn't change anything. School B School B 4/29/201 3 9:26:28 Yes 4/29/201 3 9:26:38 Yes 4/29/201 3 9:26:45 Yes No Yes Yes Yes No Yes Yes Yes No Yes No Yes School B School B School B 4/29/201 3 9:26:56 Yes 4/29/201 3 9:27:20 Yes No Yes Yes Yes Yes Yes No Yes School B School B 4/29/201 3 9:27:25 No 4/29/201 3 9:27:29 Yes 4/29/201 3 9:27:33 Yes 4/29/201 3 9:27:45 Yes 4/29/201 3 9:28:10 Yes 4/29/201 3 9:28:14 Yes No Yes No No No No No Yes Yes Yes No Yes Yes Yes No Yes No Yes Yes No Yes Yes Yes Yes School B School B School B School B School B School B 4/29/201 3 9:28:16 No 4/29/201 3 9:28:57 Yes 4/29/201 3 9:29:09 No 4/29/201 No You should have tissues at the stations because my nose was running could it be more hands on? It was super dooper fun learning about racing. Vroom Vroom A closer look at how these cars go. Make it more handson for the students. have an actual race to experience what its like We could have could gone down on the track No No No No Yes No Yes Yes No No No No No Yes No Yes The field trip is longer If you let us stay the whole day at the race track and gave us more pizza, soda, and water thanks =3 School A School B School B It was awful, it made me want to stab my ear drums and shake. Thanks for the hurtful trip. love brett Less stations more RIR STEM EDUCATION DAY PROGRAM EVALUATION 3 9:29:15 4/29/201 3 9:29:39 4/29/201 3 9:30:21 4/29/201 3 9:30:45 4/29/201 3 9:30:49 racing Yes Yes Yes Yes Yes Yes Yes School B No No No No No School E Yes Yes Yes Yes Yes School B 4/29/201 3 9:32:44 Yes 4/29/201 3 9:32:45 Yes 4/29/201 3 9:32:54 No 4/29/201 3 9:33:08 Yes 4/29/201 3 9:33:20 4/29/201 3 9:33:27 4/29/201 3 9:33:27 4/29/201 3 9:33:28 4/29/201 3 9:33:39 109 School J I think it made me gay It was good but we needed some earplugs I think it was fun and a great experience. Even though I might not want that in my future i know it could help me in some way or another Yes Yes Yes Yes School H No Yes No Yes School B No No No No School H School H more food selection No Yes No Yes School H I would recommend to have more time in the stands and less time walking around the stations. Also, it would have been nice to have an opportunity to buy earplugs. Yes Yes Yes Yes Yes Yes Yes No No Yes No Yes Yes Yes Yes School H School H School H Yes Yes Yes Yes Yes Yes No Yes Yes Yes School H School H 4/29/201 3 9:33:51 Yes 4/29/201 3 9:34:01 Yes 4/29/201 3 9:34:29 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes School H School H The field trip should be form the very begining of school to the very end and get to into a real race car. I would have had less stations because they all looked good and we only got to go through 3 of them before lunch. the gift bags were sweet! Try and let kids go into the pits next time RIR STEM EDUCATION DAY PROGRAM EVALUATION 4/29/201 3 9:34:38 Yes 110 School H No Yes Yes Yes School H 4/29/201 3 9:34:43 No No No No Yes School H 4/29/201 3 9:34:45 Yes 4/29/201 3 9:34:47 Yes 4/29/201 3 9:34:58 No Yes Yes Yes Yes No No No Yes School H I liked the goodybag School H No Yes Yes Yes School H 4/29/201 3 9:34:59 Yes 4/29/201 3 9:36:06 Yes 4/29/201 3 9:36:35 No 4/29/201 3 9:37:54 No I would add more interactive things could be longer, we didn't learn enough at the stations The wait time for other schools to get there was about 36 minutes. There should have been some activity to do while we waited. No Yes No Yes Yes Yes Yes No School H Yes Yes Yes Yes School B No Yes Yes Yes School H More food We could learn a little more about the pit crews and stuff like that. Make it a little longer to so we have more time to learn. The wait time before it started was 45 minutes long. You could have had us do something while we were waiting. i think it is already good for now. I was alright because I don't have much interest of car racing. When the field trip has started, we exit the bus and recieved different colored wristbands. I had the red one, so I had to stay with the others that has red wristbands. After every red colored station, I went with my technology class and walked to the racing stadium. It was very loud and cold and I covered my ears and try to cover myself to warm up. I waited five minutes for my two slices pizza and I recieved a bottle of RIR STEM EDUCATION DAY PROGRAM EVALUATION 111 water. When we left the stadium, it was finally warming up and we recieved goodie bags and a Tshirt. I would have more fun if it was more quiet and warm. 4/29/201 3 9:41:09 No Yes No Yes Yes 4/29/201 3 9:41:42 No No No No Yes 4/29/201 3 9:53:43 Yes Yes Yes Yes Yes School A The feild trip was very unorganised and the racing trials were not enjoyable ue to weather. I only got to go to four stations and one i didn't even get to enjoy(Racing simulation) due to and overwhelming amount of rude, and bellergerant high school kids. I feel as though it would be more enjoyable to go to a newpaper printing facility and take a tour. Watching the cars go in circles was not enjoyable either. It was too cold and winding and smelled of gas and oil. Maybe some ear plugs would of helped for the expirence of watchign thirty cars drive in circles. I feel even worse about the "sport". Thanks for the expirence and i wish you all a great School G day. I love Nascar. Its the best sport eva. I wanna go fast and loud won day like Dale Earnhardt Jr.. I got to meet a car driver. It was sooo cool. That was the best pizza evur. School I Great job on RIR STEM EDUCATION DAY PROGRAM EVALUATION 112 organizing the car timing. I love the sound of v8 in da mornin. Thanks for tha trip and i can't wait to go back for a race. 4/29/201 3 11:14:54 Yes 4/29/201 3 11:15:04 Yes 4/29/201 3 11:15:26 No 4/29/201 3 11:16:03 4/29/201 3 11:16:10 4/29/201 3 11:16:20 4/29/201 3 11:16:28 Yes No No Yes School G Yes Yes Yes Yes School G No No No No School J Yes Yes Yes No Yes School J Yes No Yes Yes Yes School G Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes School J School J School J 4/29/201 3 11:16:37 No 4/29/201 3 11:16:50 Yes 4/29/201 3 11:16:51 Yes No Yes No Yes Maybe more stations about the actual science and technolgy of the cars and less about the history. Go to the actual race instead of the practice. Maybe we could of been inside when we eated and not outside in the freezing weather. And maybe we could get earplugs for those who have sensitive ears and you should choose if you want to see them practice. School J Yes No Yes Yes No Yes Yes Yes School J I thought that the field trip was very interesting, but it could be improved. First of all, I think that we should be allowed to go to more than just four or five stations. Second, I RIR STEM EDUCATION DAY PROGRAM EVALUATION 113 think that we should be allowed to stay at the stations for a longer amount of time. Otherwise, I think the field trip was overall great. 4/29/201 3 11:16:52 Yes 4/29/201 3 11:16:55 Yes School J Yes Yes Yes Yes School J Yes Yes Yes Yes School J 4/29/201 3 11:16:55 4/29/201 3 11:17:07 4/29/201 3 11:17:12 4/29/201 3 11:17:18 4/29/201 3 11:17:25 4/29/201 3 11:17:27 4/29/201 3 11:17:40 4/29/201 3 11:17:44 It was a very intresating expierence and it improved my awerness and expierence with profensional racing Yes Yes Yes Yes Yes Yes No No No Yes More Good Looking School G RaceCar Drivers. Yes Yes Yes Yes Yes School J Yes No No No Yes School G School J No No Yes No Yes give everyone more than 2 slices of pizza School J No No Yes No Yes School J Yes No Yes Yes Yes give everyone more than 2 slices of pizza I wish their was somewhere where I can buy earplugs School J No No Yes Yes Yes School J 4/29/201 3 11:18:09 No 4/29/201 3 11:18:23 Yes 4/29/201 3 11:18:28 Yes No Yes No Yes I thought it was a fun day at the race way, but it was very windy and cold. School J No Yes Yes Yes No No Yes Yes School J I thought the program was very fun and informative. I thing that we should be able to select our own food or pay for it. RIR STEM EDUCATION DAY PROGRAM EVALUATION 114 School J 4/29/201 3 11:18:44 Yes Yes Yes No Yes School J 4/29/201 3 11:18:56 Yes 4/29/201 3 11:19:12 Yes Yes Yes Yes Yes School J Yes Yes Yes Yes School J 4/29/201 3 11:19:26 Yes 4/29/201 3 11:19:32 Yes 4/29/201 3 11:21:12 Yes 4/29/201 3 11:22:00 Yes I think it could be improve by letting us learn more about the actual race cars Have people go and talk to the actual drivers and pit crew and learn about how the program is run. also have more onhands stuff and better lectures for each station Yes Yes Yes Yes I thought that the field trip was excellent. It really opened my eyes to careers involving racing. I was already thinking about something like mechanical engineering, and the mechanical engineer I talked to during a station helped to solidify my viewpoint. The only thing that I thought could be improved was to have more food for us during the lunch portion of the program. School J Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes it was a little boring at the begining i think we should have gone done in to School G the inter circle The field trip was very organized and it was a pleasant experience to attend. The stations had some very interesting concepts that were interesting to learn about. One thing that could be improved about the stations School G was how we were RIR STEM EDUCATION DAY PROGRAM EVALUATION 115 organized around it. Sometimes, it was hard to see over taller people or other things like that were distrtacting occasionally. However the overall experience was fun. The racing componet of the field trip was also fun as we got to see how all the enginnering concepts we learned applied to the sport in an exciting way. 4/29/201 3 11:22:57 Yes 4/29/201 3 11:24:40 Yes 4/29/201 3 11:24:57 Yes 4/29/201 3 11:25:46 Yes 4/29/201 3 11:26:20 4/29/201 3 11:48:28 4/29/201 3 11:49:59 4/29/201 3 11:52:09 4/29/201 3 11:54:14 4/29/201 3 11:54:22 No No No Yes Yes Yes Yes Yes School G School G School G No Yes No Yes Yes Yes Yes Yes No No No Yes Yes Yes Yes Yes No Yes Yes Yes No Yes School G The field trip was great. i am so happy that i had the oppurtunity to be there and to learn things that i never knew. thank you for allowing me to be there School G The trip was very intresting and fun, it was cool that we got to meet some of the K&N Racers Pro. You need to give more food to us. And School A do more stuf. School A More pizza. School A Yes No Yes Yes Yes School A Yes No No Yes more hands on experience Yes School A Yes Yes Yes Yes Yes it was kind of confusing RIR STEM EDUCATION DAY PROGRAM EVALUATION 116 School A 4/29/201 3 11:54:42 Yes No Yes No Yes School A 4/29/201 3 12:13:07 Yes Yes Yes Yes Yes School A 4/29/201 3 12:13:25 Yes Yes Yes Yes Yes School A 4/29/201 3 12:13:25 Yes 4/29/201 3 12:21:09 Yes Yes Yes Yes Yes School A No Yes No Yes School A 4/29/201 3 12:24:19 Yes 4/29/201 3 12:27:49 Yes 4/29/201 3 12:28:38 No 4/29/201 3 12:55:51 Yes 4/29/201 3 Yes My trip to the IRS was an once in a lifetime oppourtunity. I really enjoyed exploring the tract and speaking to the racers. Hopefully next time we could have a chance to ride the cars. the staff disrespected me and judged me on my religon and beliefs. i am highly disatisfide. the staff disrespected me and judged me on my religon and beliefs. i am highly disatisfide. the staff disrespected me and judged me on my religon and beliefs. i am highly disatisfide. Yes Yes Yes Yes School A Yes Yes Yes Yes No No No No School L Yes No No Yes School G No No Yes Yes School G I think that fieldtrip could been better. But I loved how some how the race car driver came and told us how it is when you get in a car that goes 200 or 191 mph. I think it would of been better if some of the racers pulled a little piece of the car and signed the part. This lady was on stalker mode, so give people more freedom Change everything it was terrible and the seats were too high which made it cold and yall were stingy as fuck RIR STEM EDUCATION DAY PROGRAM EVALUATION 117 13:01:14 4/29/201 3 13:07:31 4/29/201 3 14:26:08 4/29/201 3 14:26:18 4/29/201 3 14:27:05 4/29/201 3 14:27:18 4/29/201 3 14:28:37 4/29/201 3 14:28:53 4/29/201 3 14:30:30 4/29/201 3 14:30:50 No Yes Yes Yes Yes Yes No Yes Yes Yes Yes No Yes No Yes Well, I know that the cars are meant to be loud, but after I left my head hurt, it also doesn't help that I can hear the raceway School G from my bedroom. School C School C It was good. School C No No Yes No Yes School C Yes No Yes Yes Yes School C Yes Yes No Yes Yes None School C No No Yes Yes Try going on a warmer day. Yes School C Yes Yes No No Yes School C Yes Yes Yes Yes Yes School C 4/29/201 3 14:31:05 Yes 4/30/201 3 7:36:39 Yes 4/30/201 3 8:43:54 Yes 4/30/201 3 10:17:28 Yes 4/30/201 3 11:02:17 Yes Yes Yes Yes Yes No No Yes Yes Yes Yes Yes Yes Yes Yes Yes School I Yes Yes Yes Yes School K School F let us get in a car next time I really did like it, showed me a lot more that i didnt know. i wasn't aware of the real car they had, if they would have let us talk to a real nascar driver it would have been a lot cooler.... but i had a really good time! More hands on activities / compatitions. School L it was awesome The time and efficiency food was delivered. this could improve by having a RIR STEM EDUCATION DAY PROGRAM EVALUATION 4/30/201 3 11:06:54 Yes 4/30/201 3 11:14:49 Yes 4/30/201 3 11:18:16 Yes 4/30/201 3 11:23:21 4/30/201 3 11:46:44 4/30/201 3 11:46:47 4/30/201 3 11:46:58 4/30/201 3 13:12:53 4/30/201 3 13:13:27 4/30/201 3 13:13:38 4/30/201 3 13:13:54 4/30/201 3 13:14:03 4/30/201 3 13:14:12 118 Yes Yes Yes Yes School K Yes Yes Yes Yes School K Yes Yes Yes Yes School K delivery plan and giving ii section by section row by row, instead of giving it out randomly Provide ear buds so our ear drums don't deteriorate next time. Yes Yes Yes Yes Yes School K It was very interesting but I suggest to get lots more food (ie. junk food) and also let the students participate by actually racing the cars. Yes Yes Yes Yes Yes School F I give the students more food No No No Yes Yes School F make the trip longer No No Yes Yes Yes School F Make the trip all day Yes Yes Yes Yes Yes School E School E Yes No Yes No Yes School E Yes Yes Yes Yes Yes School E Yes Yes Yes No Yes School E Yes Yes No Yes No i liked racing School E Yes Yes Yes Yes Yes School E 4/30/201 3 13:15:24 Yes 4/30/201 3 Yes Yes Yes Yes Yes No Yes Yes No School E Some people wanted earplugs and we did not know where could get them. It was so cool I got to meet Ryan Grifford. RIR STEM EDUCATION DAY PROGRAM EVALUATION 119 13:15:30 4/30/201 3 13:16:30 4/30/201 3 13:18:20 4/30/201 3 13:56:14 4/30/201 3 13:56:15 4/30/201 3 13:56:16 4/30/201 3 13:56:17 4/30/201 3 13:56:17 4/30/201 3 13:56:20 4/30/201 3 13:56:22 4/30/201 3 13:56:22 4/30/201 3 13:56:38 4/30/201 3 13:57:19 4/30/201 3 13:57:34 4/30/201 3 13:57:48 4/30/201 3 13:58:04 4/30/201 3 14:00:48 4/30/201 3 14:03:43 4/30/201 Can you get some Sprint Cup drivers next time? Yes No Yes No Yes School E Yes Yes No Yes Yes School E Yes Yes Yes Yes Yes School D School D Yes Yes Yes Yes No More time at stations School D Yes Yes Yes Yes Yes School D Yes Yes Yes Yes Yes School D Yes Yes Yes Yes Yes School D Yes Yes Yes Yes I WILL LIKE TO BE A DRIVER Yes School D Yes Yes Yes Yes Yes School D Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes School D School D School D Yes Yes Yes Yes Yes School D Yes Yes Yes Yes Yes School D Yes Yes Yes Yes Yes School D Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No No School F School F I hated the stations! I hated the stations! RIR STEM EDUCATION DAY PROGRAM EVALUATION 3 14:03:54 4/30/201 3 14:04:53 4/30/201 3 14:05:18 4/30/201 3 14:05:38 4/30/201 3 14:06:05 4/30/201 3 14:07:38 4/30/201 3 14:19:32 120 Yes Yes Yes Yes No School F Yes No Yes Yes Yes School F School F Yes Yes Yes No Yes #funtimes School F Yes No Yes Yes Yes School F Yes Yes Yes Yes Yes School F It was Yes Yes Yes Yes Yes School F 4/30/201 3 14:24:50 Yes No Yes Yes Yes School F 4/30/201 3 14:24:57 Yes 4/30/201 3 14:27:29 Yes 4/30/201 3 14:30:08 4/30/201 3 14:58:15 4/30/201 3 14:58:50 4/30/201 3 14:59:33 4/30/201 3 15:01:40 4/30/201 3 15:04:01 4/30/201 TIM WHITING HATES BLACK PEOPLE No Yes Yes Yes School F aweeeesommmmme eeee i think that when they are teaching you about the nascar that they should be faster in changing people to different parts i think that when they are teaching you about the nascar that they should be faster in changing people to different parts It was really educational and fun. More Pizza i think i could be improved by having more time in each of the stations. Yes Yes Yes Yes Yes No Yes Yes Yes School F Yes Yes Yes Yes Yes School I Yes Yes Yes Yes Yes School I Yes Yes Yes Yes Yes School I .................. Yes Yes No No Yes School I If we cou drive the cars No Yes No No Yes Yes No Yes Yes Yes School I School I passing out food RIR STEM EDUCATION DAY PROGRAM EVALUATION 121 3 15:05:35 4/30/201 3 15:23:03 Yes 4/30/201 3 15:26:31 Yes 5/1/2013 8:35:06 Yes 5/1/2013 9:24:21 Yes 5/1/2013 9:25:49 Yes 5/1/2013 9:26:36 Yes 5/1/2013 9:35:43 Yes 5/1/2013 11:36:37 Yes 5/1/2013 11:36:55 Yes 5/1/2013 11:37:07 Yes better Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No No Yes Yes Yes Yes School E School E School E Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes School L School L 5/1/2013 11:41:03 No Thank you for allowing us to attend the Raceway program. School L School L Yes Yes Yes Yes Yes Yes Yes Yes School I School I School I Yes Yes Yes Yes School I 5/1/2013 11:37:26 Yes 5/1/2013 11:39:29 Yes I think that the groups shoiuld be elimintated. I wished that anyone could walk to any station they wanted to. I really liked the field trip but I think that this field trip could be improved if we could get inside of a racecar and or have someone explain to us all of the components and inner workings of a racecar. Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes I found the field trip to be very educational, but it could be improved by putting the classes with classes instead of by wrist band color. more hands on stuff no i don't have any comments this field trip can be improved by actullay having people teach you the different componets of the engine and the cars School I School F more hands on stuff We should have had more time to do more stations. RIR STEM EDUCATION DAY PROGRAM EVALUATION 5/1/2013 11:41:59 5/1/2013 11:42:01 5/1/2013 11:42:30 5/1/2013 11:42:54 Yes Yes No Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No No Yes 122 School F School F School F Free race car rides School F School F 5/1/2013 11:43:00 Yes 5/1/2013 11:43:11 Yes 5/1/2013 11:43:23 Yes Yes Yes Yes Yes Yes Yes Yes Yes School F Yes Yes Yes Yes No No Yes Yes Yes No Yes Yes No Yes No Yes Yes No Yes No No Yes No Yes No Yes Yes No Yes free racecar rides Should have been warmer. Don't make the students eat in the cold bleachers during lunch School F it was badass School F it was cool School F School F School F 5/1/2013 11:44:48 5/1/2013 11:44:52 5/1/2013 11:46:16 5/1/2013 11:46:47 5/1/2013 11:47:18 5/1/2013 11:48:55 5/1/2013 11:54:34 soda instead of water I thought that the field trip was cool, I wish that we had more time to see more displays, and that we could chose our own groups. School F School F 5/1/2013 11:43:33 5/1/2013 11:43:44 5/1/2013 11:44:03 5/1/2013 11:44:07 5/1/2013 11:44:17 The stations could be longer and all the class could stay together Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes No No Yes Yes Yes Yes Yes Yes No No Yes Yes Yes Yes No Yes Yes Yes It was cool. Well instead of just sitting and watching the cars go in circles they should've really raced so we could watch it. Because sometimes we can always make it to the real race. School F the field trip was fun School F Riding in the cars School F School F School F School D less blacks, more food RIR STEM EDUCATION DAY PROGRAM EVALUATION 5/1/2013 11:55:57 Yes 5/1/2013 11:58:03 Yes 5/1/2013 12:05:35 Yes 5/1/2013 12:12:06 Yes 5/1/2013 12:15:33 Yes 5/1/2013 12:17:08 Yes 5/1/2013 12:18:34 Yes 123 No Yes Yes Yes School D Yes Yes Yes Yes School F No Yes Yes Yes School D Yes Yes Yes No School I School I Yes Yes Yes Yes No Yes Yes Yes ... School I No Yes Yes Yes Yes Yes Yes Yes School I 5/1/2013 12:26:22 Yes 5/1/2013 12:28:41 Yes 5/1/2013 12:29:48 Yes 5/1/2013 12:37:55 Yes I really like the fieldtrip because like to know about race cars because i want to built one and work on cars. Let everybody see all the stations Have a better variety of lunches to choose from, bring more racers to meet/race cars to see. Last Chances for students to ride in race car as driver drives around track once or twice. School I School I 5/1/2013 12:22:06 Yes I love what you guys did. I learned so much while i was there. It was so much fun and thanks for everything. Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes I can improve by going to Highland Springs to learn more about motors and other stuff. It can be improved by letting use go see the car on the race track School I School I Yes Yes Yes Yes School I we could have tried some of the different equipment on. some of the speakers did not talk loud enough. allow us to buy our own pizza. besides that i had alot of fun and it was interesting. i certainly have a different feel towards nascar. they could have had more racers. i think nascar RIR STEM EDUCATION DAY PROGRAM EVALUATION 5/1/2013 12:48:19 5/1/2013 12:52:45 5/1/2013 13:26:15 5/1/2013 13:27:54 5/1/2013 13:36:14 5/1/2013 13:37:00 5/1/2013 13:38:11 Yes Yes No No Yes Yes No Yes Yes Yes 124 School D School D is much different when your watching it live then on t.v. i would like to thank everyone who made that trip possible. on a scale of 1 to ten with ten being the highest and one being the lowest i would give it an eight I think the field tour was great being there. I was amazed that i learn things about race track. I want to learn more while we was there for a limited time. Im hoping i have that opportunity again. nothing really School D Yes Yes No Yes Yes Yes Yes Yes Yes Yes No No No No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes School D School D School D School D School D 5/1/2013 13:39:41 Yes 5/1/2013 13:54:16 Yes Yes Yes Yes No Yes Yes Yes School I It was fun i thought they could let us wear the gear do the stuff they experienced. RIR STEM EDUCATION DAY PROGRAM EVALUATION 125 Appendix D – Student Survey Responses Percentage Conversion Respondents by School K 2% N/A 0% L 2% A 9% J 17% B 10% C 7% I 9% D 10% H 8% G 6% E 6% F 14% RIR STEM EDUCATION DAY PROGRAM EVALUATION 126 Question 2: From these experiences, have you developed an interest/awareness of career opportunities within the motor sports industry? YES 100% 80% 55% 46% 56% 74% 71% 65% 44% 83% 67% 53% 0% A B C D E F G H I J K (5)L (6) N/A (22) (24) (18) (25) (14) (34) (16) (19) (23) (42) (1) Question 3: Do you better understand the science, technology, engineering, and math components of racing as a result of this program? YES 82% 79% 83% 88% 79% 85% 79% 100% 96% 90% 100% 83% 44% A B C D E F G H I J K (5)L (6) N/A (22) (24) (18) (25) (14) (34) (16) (19) (23) (42) (1) Question 4: Do you have an interest in learning more about the technical education programs offered within the Henrico County Public School District? YES 92% 73% 58% 72% 64% 100% 91% 76% 56% 68% 100% 83% 60% RIR STEM EDUCATION DAY PROGRAM EVALUATION 127 Appendix E – Student Survey Questions Question 1: Has your experience through classroom discussions and multiple racing programs increased your interest or awareness in the sport of racing? Question 2: From these experiences, have you developed an interest/awareness of career opportunities within the motor sports industry? Question 3: Do you better understand the science, technology, engineering, and math components of racing as a result of this program? Question 4: Do you have an interest in learning more about the technical education programs offered within the Henrico County Public School District? Question 5: Do you plan on continuing your education after high school? RIR STEM EDUCATION DAY PROGRAM EVALUATION 128 Appendix F – RIR STEM Education Day Program Floor Diagram Restrooms Arena Racing Weather Channel REV Driver (Women) Business of NASCAR Safer Wall Goodyear REV Racing (Hispanics) Arena Racing (Dystany) Stage FORD Showcar (3M) Arena Racing REV Racing Showcar Doors Doors Rouch ZEST Showcar Impact Racing REV Racing Simulator Dr. Diandra REV Racing (AfricanAmerican) Chevy Showcar Arena Racing Doors Safety Kleen REV Racing (Fuel) RIR STEM EDUCATION DAY PROGRAM EVALUATION 129 Appendix G – Coded Student Survey Comments COMMENT I liked getting to sit in the car Make stations a bit more interesting I have grown a great interest in NASCAR racing after this trip Make the students actually meet one of the people on the track and go on the track Better communication, I could not hear when to change stations and when to go to the race track. This caused a lot of confusion between the students. There could have been more NASCAR racers to attend and say what it is like Less talking More Food It should be planned out to the second Less blatant sponsorship Try to make time for the kids to see how an actual pit stop works and perform one in front of them If there was some way that they could make it warmer while we watch the cars race that would make the field trip a lot better Because we could not enjoy watching the cars go by because it was too cold Make the time for the different stations shorter. That way we would have more time to see different stations. The groups could have been more organized which would have given me a better opportunity to learn more. Very fun and informative. Provide earplugs/headphones for observing the race It was really fun I think it would be better if we had the chance during the field trip to watch an arena race or if we had other food options other than pizza and water There didnt need to be any improvements because it was a great trip. maybe last a little longer, or we could get to meet a driver. also we could get more than one peice of pizza because i was really hungery It was great One of the administraters where stalking me and my group It was really cool, i've learned alot since i've been there & the stuff i learned has been a great time and it is making me realize how great national racing is. it was fun couldn't go the pizza was a good to eat MORE TIME AND TALK TO RACERS It was pretty much all good, I wouldn't change anything. You should have tissues at the stations because my nose was running could it be more hands on? It was super dooper fun learning about racing. Vroom Vroom A closer look at how these cars go. Make it more hands-on for the students. have an actual race to experience what its like CODE I I I I N I I N I N I, U I N I I, E I I I I I I N I, U, E I N N I I N I I, U, E I I I RIR STEM EDUCATION DAY PROGRAM EVALUATION We could have could gone down on the track The field trip is longer If you let us stay the whole day at the race track and gave us more pizza, soda, and water thanks 3 It was awful, it made me want to stab my ear drums and shake. Thanks for the hurtful trip. love brett Less stations more racing It was good but we needed some earplugs I think it was fun and a great experience. Even though I might not want that in my future i know it could help me in some way or another more food selection I would recommend to have more time in the stands and less time walking around the stations. Also, it would have been nice to have an opportunity to buy earplugs. The field trip should be form the very begining of school to the very end and get to into a real race car. I would have had less stations because they all looked good and we only got to go through 3 of them before lunch. the gift bags were sweet! Try and let kids go into the pits next time I would add more interactive things could be longer, we didn't learn enough at the stations The wait time for other schools to get there was about 36 minutes. There should have been some activity to do while we waited. I liked the goodybag More food We could learn a little more about the pit crews and stuff like that. Make it a little longer to so we have more time to learn. The wait time before it started was 45 minutes long. You could have had us do something while we were waiting. i think it is already good for now. I was alright because I don't have much interest of car racing. When the field trip has started, we exit the bus and recieved different colored wristbands. I had the red one, so I had to stay with the others that has red wristbands. After every red colored station, I went with my technology class and walked to the racing stadium. It was very loud and cold and I covered my ears and try to cover myself to warm up. I waited five minutes for my two slices pizza and I recieved a bottle of water. When we left the stadium, it was finally warming up and we recieved goodie bags and a Tshirt. I would have more fun if it was more quiet and warm. The feild trip was very unorganised and the racing trials were not enjoyable ue to weather. I only got to go to four stations and one i didn't even get to enjoy(Racing simulation) due to and overwhelming amount of rude, and bellergerant high school kids. I feel as though it would be more enjoyable to go to a newpaper printing facility and take a tour. Watching the cars go in circles was not enjoyable either. It was too cold and winding and smelled of gas and oil. Maybe some ear plugs would of helped for the expirence of watchign thirty cars drive in circles. I feel even worse about the "sport". Thanks for the expirence and i wish you all a great day. I love Nascar. Its the best sport eva. I wanna go fast and loud won day like Dale Earnhardt Jr.. I got to meet a car driver. It was sooo cool. That was the best pizza evur. Great job on organizing the car timing. I love the sound of v8 in da mornin. Thanks for tha trip and i can't wait to go back for a race. 130 I I I N I I I,E N I I I I I I,U N N N I,U,E N I I N I RIR STEM EDUCATION DAY PROGRAM EVALUATION 131 Maybe more stations about the actual science and technolgy of the cars and less about the history. Go to the actual race instead of the practice. Maybe we could of been inside when we eated and not outside in the freezing weather. And maybe we could get earplugs for those who have sensitive ears and you should choose if you want to see them practice. I thought that the field trip was very interesting, but it could be improved. First of all, I think that we should be allowed to go to more than just four or five stations. Second, I think that we should be allowed to stay at the stations for a longer amount of time. Otherwise, I think the field trip was overall great. It was a very intresating expierence and it improved my awerness and expierence with profensional racing More Good Looking RaceCar Drivers. give everyone more than 2 slices of pizza give everyone more than 2 slices of pizza I wish their was somewhere where I can buy earplugs I thought it was a fun day at the race way, but it was very windy and cold. I thought the program was very fun and informative. I thing that we should be able to select our own food or pay for it. I think it could be improve by letting us learn more about the actual race cars Have people go and talk to the actual drivers and pit crew and learn about how the program is run. also have more on-hands stuff and better lectures for each station I thought that the field trip was excellent. It really opened my eyes to careers involving racing. I was already thinking about something like mechanical engineering, and the mechanical engineer I talked to during a station helped to solidify my viewpoint. The only thing that I thought could be improved was to have more food for us during the lunch portion of the program. it was a little boring at the begining i think we should have gone done in to the inter circle The field trip was very organized and it was a pleasant experience to attend. The stations had some very interesting concepts that were interesting to learn about. One thing that could be improved about the stations was how we were organized around it. Sometimes, it was hard to see over taller people or other things like that were distrtacting occasionally. However the overall experience was fun. The racing componet of the field trip was also fun as we got to see how all the enginnering concepts we learned applied to the sport in an exciting way. The field trip was great. i am so happy that i had the oppurtunity to be there and to learn things that i never knew. thank you for allowing me to be there The trip was very intresting and fun, it was cool that we got to meet some of the K&N Racers Pro. You need to give more food to us. And do more stuf. More pizza. more hands on experience it was kind of confusing My trip to the IRS was an once in a lifetime oppourtunity. I really enjoyed exploring the tract and speaking to the racers. Hopefully next time we could have a chance to ride the cars. the staff disrespected me and judged me on my religon and beliefs. i am highly disatisfide. I I N I I N N N N I I I,E I,U,E I,E,C N I I,U I,U I I N I N I N RIR STEM EDUCATION DAY PROGRAM EVALUATION 132 I think that fieldtrip could been better. But I loved how some how the race car driver came and told us how it is when you get in a car that goes 200 or 191 mph. I think it would of been better if some of the racers pulled a little piece of the car and signed the part. This lady was on stalker mode, so give people more freedom Well, I know that the cars are meant to be loud, but after I left my head hurt, it also doesn't help that I can hear the raceway from my bedroom. It was good. None Try going on a warmer day. let us get in a car next time I really did like it, showed me a lot more that i didnt know. i wasn't aware of the real car they had, if they would have let us talk to a real nascar driver it would have been a lot cooler.... but i had a really good time! More hands on activities / compatitions. it was awesome The time and efficiency food was delivered. this could improve by having a delivery plan and giving ii section by section row by row, instead of giving it out randomly Provide ear buds so our ear drums don't deteriorate next time. It was very interesting but I suggest to get lots more food (ie. junk food) and also let the students participate by actually racing the cars. I give the students more food make the trip longer Make the trip all day i liked racing Some people wanted earplugs and we did not know where could get them. It was so cool I got to meet Ryan Grifford. Can you get some Sprint Cup drivers next time? More time at stations I WILL LIKE TO BE A DRIVER I hated the stations! #funtimes It was aweeeesommmmmeeeee i think that when they are teaching you about the nascar that they should be faster in changing people to different parts It was really educational and fun. More Pizza i think i could be improved by having more time in each of the stations. If we cou drive the cars passing out food better I think that the groups shoiuld be elimintated. I wished that anyone could walk to any station they wanted to. I really liked the field trip but I think that this field trip could be improved if we could get inside of a racecar and or have someone explain to us all of the components and inner workings of a racecar. Thank you for allowing us to attend the Raceway program. I found the field trip to be very educational, but it could be improved by putting the classes with classes instead of by wrist band color. I N N I N N I I,E I I N N I N I I I N I I I N I I I,E I,E I I N I I I I,E RIR STEM EDUCATION DAY PROGRAM EVALUATION 133 more hands on stuff I no i don't have any comments N this field trip can be improved by actullay having people teach you the different componets of I the engine and the cars I more hands on stuff We should have had more time to do more stations. I The stations could be longer and all the class could stay together I I Free race car rides soda instead of water N I thought that the field trip was cool, I wish that we had more time to see more displays, and that I we could chose our own groups. I free racecar rides Should have been warmer. N Don't make the students eat in the cold bleachers during lunch N I it was badass I it was cool I It was cool. Well instead of just sitting and watching the cars go in circles they should've really raced so we N could watch it. Because sometimes we can always make it to the real race. I the field trip was fun I Riding in the cars I love what you guys did. I learned so much while i was there. It was so much fun and thanks I,U,E for everything. I really like the fieldtrip because like to know about race cars because i want to built one and I,U,E,C work on cars. I Let everybody see all the stations Have a better veriaty of lunches to choose from, bring more racers to meet/race cars to see. Last I Chances for students to ride in race car as driver drives around track once or twice. I can improve by going to Highland Springs to learn more about motors and other stuff. I,U,E,C It can be improved by letting use go see the car on the race track I we could have tried some of the different equipment on. some of the speakers did not talk loud I enough. allow us to buy our own pizza. besides that i had alot of fun and it was interesting. i certainly have a different feel towards nascar. they could have had more racers. i think nascar is much different when your watching it live then on t.v. i would like to thank everyone who made that trip possible. on a scale of 1 to ten with ten being the highest and one being the lowest i would give it an eight I think the field tour was great being there. I was amazed that i learn things about race track. I I,U,E want to learn more while we was there for a limited time. Im hoping i have that opportunity again. nothing really Legend I Interest U Understanding E Education C Career R Race G Gender N None Total Responses Interest Understanding Education Careers Gender Race None 142 101 10 16 3 0 0 39 RIR STEM EDUCATION DAY PROGRAM EVALUATION 134 Appendix H – Coded CTE Teacher Feedback COMMENT I thought things went smoothly. Bus drop off in the a.m. could have been better for other schools. We were the first ones there and I guess the wait time was getting to me. Program started about 20 minutes late – but that’s OK. Program speakers were good but the kids were ready to see some action at the stations. Stations were good…some better than others. Pizza (free) was nice. Sitting in the covered overhang was cold, but hey, weather is weather…I should have worn a jacket!!! Kids really enjoyed it and learned a lot. We had a class discussion the next time we met and filled out the survey. This was truly a worthy initiative and it was executed in a very organized and professional manner. I appreciate the drive and support that is displayed by the leadership in this County. I believe that these types of experiences are life changing for many of our students and help them to critically view and understand the functioning world outside the classroom. I am a believer because I remember how these types of lessons impacted me when I was a CTE student many years ago. The students were extremely positive after debriefing. To sum it up, I will quote what one challenging student said to another on the bus on the way home. “THIS WAS THE BEST FIELD TRIP I HAVE EVER BEEN ON “. Great work Gentlemen. Overall the program was very good & the kids had a good time. Maybe some kind of a prep activity for the bus ride to the track. Have some kind of a program or agenda for the kids to look at, at the beginning of the day. Also, fewer people on the stage at the beginning of the program. This was good, but perhaps geared to a more adult audience Make the round-robins more interactive for the kids. Have some presenters provide more visual aids. I’m not sure a sales presentation on air filters was what the kids were looking for. My kids were a little bummed that they didn’t have a chance to try the race simulator (Either a better plan or more crowd control was needed there). Combine some round-robin activities, such as the African- and Latin-American driver signing. The kids were really interested in the Toyota Monster Truck parked outside, but didn’t get a chance to check it out. There were also lots of other cool stuff around the parking lot that the kids didn’t get a chance to see. Get the kids out of the barn sooner & keep the small groups moving. This will have to be done if additional school districts participate. Have some kind of a note card for the kids to use while doing the round-robins. If kids complete the card, they could turn it in for a chance for a prize to be awarded at the end of the day. Also, allow the kids to provide feedback, what they liked best (other than pizza) … etc… During the practice runs, have someone explain to the kids what they are seeing. Provide a map wit fill in the blanks as to what is going on at different points inside the track. Talk about what to expect in an actual race. Add some facts & figures about the cars, drivers Legend I Interest U Understanding E Education C Career R Race G Gender N None Total Responses Interest Understanding Education Careers Gender Race None 11 4 2 0 0 0 1 6 CODE I, U U, I I N N N N R I N N RIR STEM EDUCATION DAY PROGRAM EVALUATION 135 Appendix I - Evaluation Questions E1. To what degree did the RIR STEM Education Day program increase middle school students’ interest in STEM education? E2. To what degree did the RIR STEM Education Day program improve students’ understanding of STEM education? E3. To what degree did the RIR STEM Education Day program increase students’ interest in STEM educational opportunities? E4. To what degree did the RIR STEM Education Day program increase students’ interest in STEM-related careers? E5. Are there differences in STEM interest and understanding based on gender and ethnicity regarding the RIR STEM Education Day program? RIR STEM EDUCATION DAY PROGRAM EVALUATION 136 Appendix J – Teacher Focus Group Interview Questions 1. What did the students think about RIR STEM Education Day? What did they say about it afterwards? 2. In what ways were you able to connect your class content to what students experienced that day? Possible follow up: How have you adapted lesson plans that tied back to STEM components such as force, speed, etc. 3. What changes did you see in how students talked about STEM following the RIR STEM Education Day? Possible follow up: Did their grades improve? Were students engaged and/or interested in STEM? Was there increased participation? 4. What did students say or do following the RIR STEM Education Day program that may have indicated an increased interest in STEM educational opportunities? 5. What did students say or do following the RIR STEM Education Day program that may have indicated an increased interest in STEM career opportunities? 6. In what ways did the RIR STEM Education Day program promote female and minority students’ interest in STEM-related education and careers? RIR STEM EDUCATION DAY PROGRAM EVALUATION 137 Appendix K – Student Focus Group Interview Questions 1. Think back to RIR Education Day. What did you find interesting about that day? Possible follow up: It was fun how? What made it boring? In what ways did it help you understand STEM? 2. What did you do that you really liked? Possible follow up: What did you like about the hands-on activities? Did that make you better understand STEM? Did any activities relate back to classroom material? 3. Were there stations where you would have liked to have spent more time? Was there anything you did that made you think that you would like to learn more about STEM? Possible follow up: Would you like to be a scientist/engineer, etc. one day? 4. What do you think would have made it better? Possible follow up: Was there anything you would leave out? 5. What station made the biggest impact on you? Possible follow up: Why do you remember this station the most? RIR STEM EDUCATION DAY PROGRAM EVALUATION 138 Appendix L – Focus Group Transcriptions Teachers School B Interviewee 1: I thought it worked out pretty good. The kids were excited when we brought them back. They all had a lot of fun. I’m sure all of them are much more aware what goes on in racing. Interviewer 1: You actually answered the first question. Interviewee1: Perfect Interviewee 2: Raised the awareness Interviewee 1: Yeah it did, the middle school kids, the way I thought of it; they had no clue what they were going to see. All of them came back more aware of what opportunities and possibilities they have. Interviewer 2: What did they say that would lead you to believe that? Interviewee 1: They saw the girl from Highland Springs and they said that they could do something like that. Interviewer 2: Okay Interviewee 1: And there was one girl that we had, and her dad has something to do with sports. He does the medical helicopter. She wanted to do something on those lines. Interviewer: ok Interviewee1: So when we took her done there, she loved it. She had a good time. Interviewee 2: The way the program set up, like when we went down to the Arena racing the kids realized that there is more to it. Like what you was saying, there is more to it than just driving the car and there is a business side of it that they can get into. The found that they their interest in NASCAR just as watching the race, that it could be more to it. Interviewer2: And that was the point of the program. Interviewer 1: In what ways were you able to connect what you teach in class to what the kids saw that day? From the classroom to actually the hands on side of it when they got to RIR that day. RIR STEM EDUCATION DAY PROGRAM EVALUATION 139 Interviewee 2: We have those mega dragster cars that our kids are making. We race them. It’s a contest of technology contest. It’s a National contest and they start with a 2 x 4, they have to make these cars, a 70 foot track and they travel the lanes in about 1 second. Interviewer 1: Wow Interviewee 1: We’re in the process of racing these right now. So I tell the kids that all the cars have the same engines. All of the cars have to travel the same distance. So the lighter the cars the faster they go. Interviewer 1: Aerodynamic Interviewee 1: Exactly Interviewer 1: The power is the same, as well as the car, but the design is what’s going to make it go faster. Interviewee 1: I have Highland Springs and some of the middle school that will cut these. brings that Technology aspect into them. Our kids use all hand tools. That Interviewer 1: Is that the 3d imaging? Interviewee 1: yes, they use the 3d imaging machine, they draw it, the c & c machine cuts it out and all these are done by hand. We use drill press and the band saw. It’s mostly just handsaws. Interviewer 1: Do you have a wind tunnel here. Interviewee1: Yes we do, we actually just got the wind tunnel. It’s in the closet. It’s hooked up on the computer so you can record on the computer. Interviewer2: What changes did you see in how students talked about STEM following that program. The day seemed to be more understanding of the entire aspects of STEM or the different components of STEM. Interviewee 1: I think they were able to see that you had physics involved. There was Math involved, more than just banging metal together and nuts and bolts. There is a lot more that goes into it. Same with these cars, they figure out that the lighter cars run faster than the heavier cars. Looking at the cars, there’s nothing to the cars. Just the frame and the body, and it’s the weight of the car. People try to make it lighter and lighter. Interviewer 1: Did you find yourself adapting any of your curriculum based on the program and go back to the classroom to make a connection to maybe what you were teaching. Do you think it hit home a little bit or for some of the kids was it just a fun thing? RIR STEM EDUCATION DAY PROGRAM EVALUATION 140 Interviewee1: I tried to emphasize more the weight of the cars more so this year than I have any other year. In years past, the kids would just design a car, just make a car. I started telling them, the lighter the car the faster it’s going to go. That’s what I really emphasize to the kids this year. In the races that we’ve see, we noticed that they have been a lot closer. Before there was a wide margin. Interviewer 2: I would like to see that. Interviewee1: What we learned here at the raceway, we were able to tie in the classroom. Interviewer 2: From the program, did you notice the students grades improve at all or were they more engaged in the classroom following the program? Interviewee 1: I didn’t notice a big change in my class. The kids that went were already the kids that are engaged in class. It was our 9th grade credit class. So they are a little more advanced than most of the kids we teach. I didn’t see much difference in my class. Interviewee 2: I didn’t notice much difference either. Last year was the first year the county offered to take us on a field trip. I think it’s big that the county is stepping up and doing something different and I appreciate that and I’ll talk that up. Interviewee 2: I love it; I thought it was a great program. The pizza got people talking about that pizza. They just thought it was great that they got free pizza. Nobody had to pay anything for it. The got to see the racing and the cars, it was a little cold that day. Interviewer 2: What did you students say or do following the program that may have indicated an increase interest in STEM educational opportunities? Did they say anything about going to any of the specialty centers following middle school? Interviewee1: We had some kids that actually went out to those races that we gave tickets to. So they liked what they saw, and they used their time to go to see the races. They were more involved in the sport than going to the specialty centers. We’re down here, we don’t hear much of that. At least I don’t. Interviewer 2: Did they talk about any career opportunities in STEM. If they went to the race and they may of want to go back then they may be interested in other things. Interviewer 1: To be recognized, there are hundreds of career opportunities, do you think that came out in this program. Was there any talk from the students? Interviewee1: I think they were more aware that there is more involved. Like how athletic the pit crews have to be. The fact that it’s just not driving the car. It’s stuff in making the track. Just so many aspects, just about anything you can think of. There’s a spot in racing Interviwer2; Design RIR STEM EDUCATION DAY PROGRAM EVALUATION 141 Interviewee 1: Design, there’s the design architecture of the track, the stadium, the business side of it, someone has to be the business manager Interviewer 2: It’s not how NASCAR looked in the 70’s and how it look today. You don’t know how they going to look in another 20 years. Interviewer 2: In what ways did the RIR STEM Education Day promote female and minority students interest in STEM, or STEM education or STEM careers? Interviewee 1: Generally in the past we haven’t had a lot of females in our programs. Last year and this year, I noticed, we have been getting more. I think it’s because of engineering. How technology is associated with engineering. I had a girl in here yesterday, she wanted to build this windmill for a Science Fair. She wants to win a contest. The girls are a lot more driven at this age. So we are seeing more females. Interviewer 1: Anything as far as ethnicity Interviewee 1: I know that one girl her dad flies a helicopter for the rescue squad over there and she told me she is looking for something to do. Interviewer 2: I asked about minority. Interviewee 1: No, No, she wants to be in racing or something. She wants to get involved. Interviewee 2; I’ve heard that, not from our students but friends of mines kids that want to get into a program where they can do more than just turn a wrench. They want to make a career out of NASCAR. With the way of this and the more kids we can get involved, the sport can grow and what the program is designed to do and be successful. Interviewee 2: I think it’s going to help the tech centers also. Our guidance department takes a field trip to the Hermitage Technical Center and I think when they did that, they had these cars there. So all the kids came back and said they saw that. I had some 6th and 7th grade kids come up to me and said are you the guy that takes the kids to the race track. It’s sparking interest in the lower grades. Inerviewer2: Research says we should start early Interviewee 2: Good, then I can say it. School C TL - I have six questions for you. 1. What did the students think about RIR STEM Education Day? What did they say about it afterwards? RIR STEM EDUCATION DAY PROGRAM EVALUATION 142 I know they all had a good time. Not just for days but for weeks. They were all talking about the field trip the kids that didn’t go on the field trip were upset that they didn't go. We drummed it up but the kids coming back talking about it got everyone else involved in it...the ones who went really liked the arena car because one of their peers was there and they got to see him in his fire suit and all. The kids really liked the FORD show car (cutaway car) that was a big hit. What I picked up on was that it was more technical. When we are in here in tech class we focus on the whys'...how does this piece of equipment work and what is it designed to do...and that more than everything else seemed to be geared that way...we are trying to generate down force...this is how we how we do it at this portion and how we do it at this portion....it was more of a direct engineering relationship than any of the other things we’ve got here ...I think that that direct technical application really got the kids thinking about...I understand how this works now...I see how they designed this part this way because this is what they’re trying to accomplish. 2. In what ways were you able to connect your class content to what students experienced that day? We build race cars...I’ve got a couple over there on the end of the table....Co2 dragsters...before we even go on the trip the kids are familiar with their dynamics with reducing friction...a lot of the key elements of building a successful race car...and going to something like this the education day at RIR gives the kids an opportunity to see real world applications of what we are learning and not just building a 50, 60, 70 gram race car they see a real race car here where something small like the arena car where they see what we are learning are applies outside of these walls. When I was in school I would always ask when am I going to need to know or use this stuff again...this is right in their face how they can use these skills beyond the classroom. - How have you adapted lesson plans that tied back to STEM components such as force, speed, etc.? We are already doing it and I will probably come back to doing a car...come up with a transportation thing to have this fresh in their mind before education day. 3. What changes did you see in how students talked about STEM following the RIR STEM Education Day? STEM is just an acronym but you could see how they're appreciating for what we were doing in the classroom...we always try tie in what they’re learning in their core classes to what they are doing in here and to the outside world. This program gave them the opportunity to feel and touch ...how this effect these skills we are learning...these concepts and how they are used in the real world...it was a big deal...the kids came back and you could see that they understand what we do things a certain way...why do we go through this process and problem solving...if you are car engineer and you are trying to generate more down force...you do something like this and put the spoiler up on the back of the car ...that makes sense to me. RIR STEM EDUCATION DAY PROGRAM EVALUATION 143 - Did their grades improve? I don't. But I do think that it was advantageous. - Were students engaged and/or interested in STEM? Was there increased participation? I absolutely think their awareness increased. I'm glad we got a chance to see the show car because it was the most direct relationship between what we learn in class...it was very, very technical. 4. What did students say or do following the RIR STEM Education Day program that may have indicated an increased interest in STEM educational opportunities? I don't recall anybody saying anything about educational opportunities...we were working on rockets at the time... it had nothing to do with racing...we had finished our cars...having just been to this and seeing people in the industry got them much more exciting...when we got back they even more engaged with their projects. 5. What did students say or do following the RIR STEM Education Day program that may have indicated an increased interest in STEM career opportunities? They all wanted to be race car drivers after that! They were really excited about going to this and there was a lot of really neat stuff they got to see that I would not have been able to show them...you can show them picture and videos but it's not the same as standing up against this car to see in there...there is the carburetor...there is the radiator...this is how all this stuff is tied in....we got to go out there and see the cars run...they were really excited...there is a lot to be said for being able to put your hands on these things...having something right in front of you instead of pics and video and me describing it. 6. In what ways did the RIR STEM Education Day Program promote female and minority students' interest in STEM-related education and careers? The girls that I took were surprised to see the Highland Springs girl with the motorcycle...she rides a motorcycle??...they thought it was a guy thing...it opened their eyes that they can do this too. Ethnicity? No. But a bunch of them were surprised that there was a girl there. She was not a vender, not support...she was a rider...the star of the show. School D Interviewer: What did the student think about the RIR Stem Education Day and what did they say afterwards. RIR STEM EDUCATION DAY PROGRAM EVALUATION 144 Interviewee 1: My students we talked about it when we came back from visiting, They were very excited, they saw that they had greater possibilities outside of basketball, you know sports, and that they were able to go into another area, something that would be more interesting to them. Its fast paced, you know a lot of them, you know very quick ethics, it has to be quick done. They loved that. The students I talked to wanted to come back they wanted more, and wanted to know when they could do this again. They were telling other students that won’t in the classroom about it. They really loved it. Interviewee 2: I think a lot of my kids, they really did enjoy it. We had a conversation, in one class, the class afterwards, A lot of them said they didn’t know that the industry was as big as it was. A lot of times when they look at images on TV and they only see the cars racing they didn’t know like the transportation of the trucks, all those things, they didn’t know about the technology. I think one kid, one kid talked about the fire suit and how much technology went into creating the fire suit itself and how hot it could withstand heat and things of that nature. But overall a lot of them liked it and a lot of them didn’t know about the racing industry in general and they want to know more information about it. Interviewee 1: There was another thing. They didn’t know that blacks were actually into racing. They said ‘Oh, that’s not our sport, we don’t do that” until they actually went in and saw. The young lady really helped. She, seeing that she was young, she was African American , and she was a female, the girls were like, ahhhhh, I don’t know if it’s something I want to do, I said, “just go and let’s see” and they saw her and they got excited. A couple of my little girls went up to her and asked “how can we get into it?” Interviewer 2: Very good Interviewee 2: One kid asked me about, I forgot what is was, He is really interested into getting into, not necessarily racing itself but getting into the industry itself So it’s all different types of things. You look at the design on the side of the trucks and things. The design of the cars, that’s something you can get into. There are so many different graphics and the kids don’t realize that it’s a part of racing. The small detail things. Interviewer 2: You talked about the fire proof suits, and I don’t quite understand, I know the fire proof underwear was a big hit with the students. Interviewee 2: Oh Yeah, Oh Yeah Interviewer: Can’t be too safe! In what ways were you able to connect your class content with what students experienced today? Interviewee1: I teach Business. For them to learn, to see the real world, this is an aspect of what I’ve been teaching them in the classroom. We talked about Finance in the classroom, and they saw how much money it cost to get into it. They learned about, excuse me, I’m trying to get RIR STEM EDUCATION DAY PROGRAM EVALUATION 145 it all back together. Everything we touched on besides finances beside learning how to run a business , because by that point in time we were talking about entrepreneurship, so this really tied in for them on how can they get involved, what does it take to get started. They already put together a business plan by this point and time in the classroom and they were just asking a lot of questions geared to what we had learned in the classroom financially are mainly what they were excited about. Interviewee 2: I had like one or two kids from my class who actually went, I do remember having this conversation with this one particular kid, I said that everything we do relates to the racing industry. In terms of, my kids do like a lot of things working with their hands and I said, “when you working with cars, you developing those skills that will help you to, you know, work with your hands and things like that. That’s how I can relate it to it. Interviewer 2: Do you teach Technology Interviewee 2: yes Interviewer 2: And you teach Business? Interviewee 1: yes Interviewee 1: International Bachelor is Design. Interviewee 2: We didn’t take the design kids; we only took our 8th graders Interviewee 1: Yeah, 8th graders Interviewer: As a follow up to that did you create any lesson plans that could tie back to the STEM components? Interviewee2: Honestly, I have not Interviewee 1: No, I haven’t either Interviewer: What changes did you see in how your students talked about STEM, following the RIR Stem Education Day. Interviewee 1: They were more interested in those careers because we were doing Careers Exploration towards the end. They wanted to know more about Science and technology, more in depth about Technology and Science, just the STEM program itself. They wanted to know what careers were related, more to that. Interviewee 2: Not only that, a lot of them now started to take Math seriously too. Where at one time they just kind of coasted and didn’t really take it seriously. A lot of them started to take it seriously, not only that, see how Math, Science, and Technology all those things, kind of interrelate, not only racing but different fascist of their lives. And we have conversations to that, RIR STEM EDUCATION DAY PROGRAM EVALUATION 146 yes we have. So, in terms of the racing industry itself? No. In terms of how technology or how my class relates to this, not necessarily but in terms of technology in general and how it relates. Interviewee 1: Yeah, and also It helped them with them in their process, when they try to get into those specialty centers, Math is a huge component, they want to see that you are getting the high levels of math. Like Algebra and so forth. That was another push for them. It’s really an excellent program as far as I am concerned. Interviewer: Do you think this experience improved their grades? Interviewee 1: Grades, No I don’t think so. It was just too short of a period. To of gaged that. Interviewee 2: I think more than anything, It brought about awareness, and sometimes that’s what the kids need, especially over here on the east end, you know a lot of our kids don’t really make it outside of where they live, and they are right up the street from the Richmond international Raceway and never been there before. So this brought awareness more than anything else. Interviewee 1: And they wanted to know when they can get more tickets. After that first race, I had students come back, saying, “Can we get more tickets?” I was like, I can try or you can go down to the coliseum. Interviewer: Do you think there was overall increase in participation by our students dealing with STEM in any of your classes to increase the awareness. Interviewee2: Not, honestly I don’t think so, and reason being is because, you have to look at when everything happened, I think like April, we had went through our curriculum, we were in the testing cycle., so Interviewee 1: We don’t see them as much in electives because they pull them a lot during elective time so we don’t get a chance to see them, now if it was earlier on in the year, we could probably incorporate more or we could think about it for the following school year. A lot of what I’m teaching seem to coincide, with what we learned hear. The only thing that I don’t have is the science peace, you know like the cars speeding around and how fast they going, but a lot of that was already incorporated through my distant courses anyway, but I really loved it. Interviewer: Okay thanks. What did student say or do following the RIR STEM Education Day Program, that may have indicated an increase interest in STEM Education on opportunities? Interviewee 2: What did they do? Interviewer 1: Yeah, what did they do or say or any behaviors in your classes that increased interest in Stem Education RIR STEM EDUCATION DAY PROGRAM EVALUATION 147 Interviewee 1: In my classes they wanted to know, what jobs can they get into, what path would they need to go into in order to get that particular job and by us looking at careers they were able to explore, use the computer to explore. I had them do a project, what are your career interest is, some of it changed for some of them. Some of them were like I want to get into the NBA and NFL, but when they saw the cars and stuff, they were like, “how can I get into, if I wanted to become a race car driver?” “I want to do this, how can I get this” I had to kind of guide them to the area and let them do a project. Interviewee 2: Same thing, I think a lot of them, many of the kids we brought were part of or in our class, but I think it changed some conversations in terms of , one, a lot of kids were l like Mr. Lucy, if it happens again, can we go the next time. So I think it kind of spread the word, in terms of kind of putting the information out there and one thing we do in my Technology classroom, we do a Career related project towards the end of the school year and a lot of kids were, choosing like Ms. Carter said, a lot of the kids want to go to the NFL, NBA, in a sports related thing, they start of thinking, not necessarily about racing itself, but they started thinking about other careers they could actually get into. Interviewer 2: So I’m trying to ask mesh these question, 4 and 5, and we just getting started here so its interesting hearing the question was there any indication or increased in interest in STEM Education opportunities and both of you went right to careers. Interviewee 1 & 2: [both laugh] Interviewer 2: No, No, Good point, that’s interesting, as 8th graders, did it spur any talk or actions as to what maybe they are going to study in high school or what direction they might want to go, like college or technic al school, or interest in the technical centers. Interviewee 1: I had two young men that wanted to go to the tech, in the 9th grade you can’t go to tech, you have to be in the 11th or 12th grade, who wanted to go to tech. One was for automotive, he was interested in cars. Every project I gave him, somehow geared to cars, or his project was going to be about some type of motor oil or something and the other one if I’m not mistaken, he was going building homes and that’s also Tech. As we were talking, I’m thinking that maybe we should open up to 7th or 6th grades that way we can kind of gage how they are from one year to the next, instead of just 8th graders. Well maybe 6th grade may be a little too immature, but it just depends on if they are IB. 7th and 8th graders so we can see how they are next year. We could look at them the following year and say, what did you do, based on what we learned, cause some of them we see in our classes from year to year.. Interviewee 2: And they can have more meaningful conversation because again, this thing was kind of short. It was kind of short lived. Interviewers 1 & 2: Right RIR STEM EDUCATION DAY PROGRAM EVALUATION 148 Interviewee 2: We were in our testing cycle last year when all the stuff transpired. We really didn’t have a chance to get into it. One thing with my kids we are kind of geared toward Technical centers and stuff like that. Then I tell them to go to the high school, these are some of the courses you should you should take anyway. Some of them were trying to steer my class into technical fields, where there are like a small number, you know, shortages in. Interviewer 1: Right Interviewee 2: I don’t know if I answered your question or not. Interviewee 1: And those kids would fall through the cracks Interviewer 2: We are not looking for a scripted answer. There are some kids, and I always tell my kinds, I ask them who want to go to college and everybody raise their hands. Not everybody is college material, or can actually get there. I let them know that there are other options. There is a two year college, technical college, and apprenticeship. So I let them know that there are other options. You don’t have to feel that college is your only option because everybody is not going to go there. Interviewer 2: We have one more question, you mentioned something about blacks and racing , What ways did RIR STEM Education day promote female and minorities students interest in STEM Education related careers It opened it up to them. They didn’t realize that there was something they could do. They always just saw Caucasians, even on TV that’s all they saw. They never saw a Latino or African American or anyone on TV in racing. If they were in it, they were behind the scenes. So they didn’t see them. So this opened it up to them, that it’s not just a career, I don’t know why they thought they couldn’t get into it., maybe because when I was young, I was taught that it was very expensive. If it’s an expensive sport to get into it, you have to have sponsors, and if you don’t know, I used to always like to go fast. Race up and down Chamberlayne Ave, I‘m ashamed of it, my family use to like racing, but to do this, I didn’t think that was an option even for me as a young person, because of what I was told, it was very expensive. Blacks don’t do that. Interviewee 2:I think that having the young lady who graduated, who was an alumni from Fairfield Middle school and Highland Springs High school, I think she spend a lot of time with the kids talking to them about what her interest are, what her goals are, what she is trying to accomplish within the industry. I think that had a real big impact on kids, especially the females, because females will look at this sort of thing as if you see like all guys there, okay this is not for me. By having her in there and she had a lot of energy, a whole lot of enthusiasm, so I think having her there really, kind of brought it home for our kids. Interviewer 1: It goes back to what you said about awareness. RIR STEM EDUCATION DAY PROGRAM EVALUATION 149 Interviewee 1: Definitely, and I think this needs to continue to expose more kids. Just not east end, west end, but just kids in general, with all you hear about being doctors, and lawyers, and teachers, they think that’s all you can do. But if you expose to the, you know, greater career choices, it just make you more excited about your future.. Interviewer 2: This program had women in racing, African American racing, and Hispanic in racing. I think it was Destiny that had the biggest impact. Interviewee 1: Yes Interviewee 2: She had so much energy and enthusiasm. She was just the total package. I’m glad she is in it. I think she said she modeled at one time. She just has that drive and that’s what kids need to see. Kids feed off other kids. Like we said earlier, when they see somebody who is enthusiastic, who really has goals set, and try to obtain those goals, the kids are like, “if she can do It, I can do it too, it’s no difference. Interviewer 1: It helps to see a role model. Interviewee 2: Not only that, they can use her as a resource. As a matter of fact, she came back and talked to our kids during career day. Interviewee 1: Yeah, she did, pass out her car, and said, you can call me, talk to me. She made herself accessible to the kids and that made them just glow. She was a big hit. I say every time I went around there, that table, you couldn’t get to her because it was such a huge big crowd. So for us I think that was very good. School E Interviewer: What did the student think about the RIR Stem Education Day and what did they say about it afterwards. Interviewee 1: The kids were really excited. In that particular group I had last year I only had 3 or 4 females, but they were definitely interested and they learned some things. I would say it peaked their curiosity, brought home some things we had talked about in class, and the best thing it did, I think it gave them a different focus for the public perception about NASCAR and how things, different systems integrate one another. There’s more to just a driver driving a car around the track, there is lots of behind the scene things that have to happen, people have to collaborate and work together to accomplish a goal. So I would say it was definitely more than just another day out of school. Interviewer: It what ways were you able to connect your class content to what the students experienced that day. Interviewee1: Well, we talk about the systems approach is a big part of what I try to do in my classroom. Teamwork and personality styles, how people bring different strengths to a team. RIR STEM EDUCATION DAY PROGRAM EVALUATION 150 They were able to see all of those things that I talked about because they came back and said, “Mr. Turner we now understand more about what you talking about because that’s what we saw, and that’s what we heard.” We talked about a lot of all discussions in class was about gender misperceptions in Tec h Ed and CTE. There’s still that 250 year old Horace Mann philosophy, girls need to sew and guys need to cut wood and do things like that. We’re bringing down those barriers and the best thing for that both ethnically and gender wise was when they saw, the young girl. Interviewer: Destiny Interviewee 1: Destiny Spurlock, as a matter of fact, I talked to Destiny personally and I was supposed to get her to come over and talk to the kids, and I need to follow up on that. Just seeing her there, it just knocked down so many walls. So many barriers. Interviewer: You actually answered my last question, as far as content, did you have to adapt your lesson plans at all to tie back to any of the programs or any of the stations that you saw or was in pretty much in line. Interviewee 1: It was definitely in line with the, it just so happens, I’m a STEM based instructor, and we have our student organization TSA, so a lot of our 32 challenges that we have all collaborate around the same things, same ideas. So I was easily able to bring it back to the kids, not just in organization in systems and stuff like that, but technical details as well. Perfect example, one of my first lessons in class after safety is measurement. We talk about the English system, the metric system, the decimal system. We talk about careers, where I tell them that 90% of the careers that most people have, that just measuring to the nearest center-meter, millimeter, or the nearest 16th of an inch is going to take care of it, and we talked about how some people are going to have high-tech jobs, in robotics, computer chip manufacturing, NASA and the space program, and I always bring in NASCAR. One reason I bring in NASCAR because in a typical engine, as an example, The cylinder piston clearance is about 12 ten thousandths of an inch, so the metaphor I use is, when you go to those races there are three types of drivers. The drivers that dominate the race, and about 2/3 of the way through the engine blows. Then there are drivers, their cars won’t keep up, their lapped 30 or 40 laps around the track during the course of the race, and there are cars that run consistently through the race, all the way. I write 12 then thousandths on the board, I say “tell me what happens when you go up 1 ten thousandths and down 1 ten thousandths”. You got 11 ten thousandths, 12 ten thousandths, and 13 ten thousandths. So I said, the guys that run strong, like the Jimmy Johnson’s and those guys, I said their builders are hitting it right there on the mark at 12 ten thousandths. The guys that blow the engines, their engines are a little bit tighter, but parts expand when they get hotter, but that one ten thousandths of an inch, may be giving it 4 or 5 more horsepower and at the end of the race, it’s just not going to stay together, but of course the guys at 13 ten thousandths, they not making as much power so they the ones that can’t keep up. So that’s my second lesson in class and they got a chance to see that. They say, “Now Mr. Turner, I see what you talking about.” RIR STEM EDUCATION DAY PROGRAM EVALUATION 151 Interviewer 1: That is a perfect tie in Interviewee 1: And this is the difference in having a billion dollar race team and a team that can’t find enough sponsors to run a whole season. Think on those lines? Maybe just to run 2 or 3 races. Interviewer: What changes did you see in how students talked about STEM following that day, or did their grades improve, anything along those lines. Interviewee1: It was more of, middle school students and students, even we ourselves, like we hear our parents talk about I did this and I did that, and maybe talk about Bill Russell scoring 100 points, some fantastic achievement, sounds like a cartoon character, fantasy character. Then all of a sudden you have the opportunity to see that Bill did score 100 points, or Mr. Turner’s right, he does know what he talking about. I guess what I’m saying, it more or less, for the kids you can see the trip and the experience validated all the points that we been talking about in class. Sometimes kids need to see a hands on, they really need to see a visual. They need to have an hands on experience, and they got that. Interviewer: Did the students say or do anything following this program that indicated they have an interest in STEM education opportunities? Like perhaps going to one of the specialty centers, anything? Interviewee 1: I would have to say yes. There were some students that were leaning towards going to the engineer center, and I do know that three of the students that were leaning on the fence, they did go over there. Because of perceptions, I know that the Director of the specialty center, I know her personally; we had many discussions about how, for years, they not had a strong support cast for picking up students from the west end. It’s mainly the distance, they don’t want to, misperceptions of the East End. Secondary but primarily they don’t want to make that 30 minute trip. I know after that program, I know two students that were in my program, that was walking the fence, about whether to go, that had been accepted at other specialty centers, said, hey, I’m going to Highland Springs. Definitely that took it; I think I can say, that was the icing on the cake, after they went to that program. As a matter of fact, there was one student in the class, to augment the power of the program. He was very negative; he is the most negative student I ever taught in my 30 years of teaching. I found out later on that he went over there. Interviewer 1: To the specialty center Interviewee 1: To the Engineering center and he was negative about a bowl of chocolate ice cream. I think this program Interviewer 1: Did he attend the program? RIR STEM EDUCATION DAY PROGRAM EVALUATION 152 Interviewee 1: He did attend the program. He said he didn’t like my class. He was very vocal about it, but I was shocked he went over to the Engineering center. Interviewer 1: They need to see, feel, and smell. The hands on activities are amazing. The next question is very similar. As far as education opportunities, did they say anything about career opportunities, maybe going into something involving STEM because of this program? So I guess you can see that it’s a whole lot more than just going fast in a car around the track. Interviewee 1: That’s correct, again, just thinking about it 6 months later, I know those two students. The two students that wanted to go were excellent students anyway. I heard a lot of chatter back and forth between them about it. Maybe it’s another side point. One of my parents with one of the companies with the Arena Racing, they must of given me 800 or 900 tickets. Those tickets disappeared like popcorn. These kids were taking 4,5, or 6 tickets at a time. I didn’t know, it seems like it’s already a fanfare following of the Arena Racing component out here in the West End. I didn’t really. I’m supposed to know about these thing, I didn’t know anything about Arena Racing until we had the first Interviewer 1: I didn’t either. I went not too long ago, and Destiny raced there. She got 2nd in that race. She’s popular in that whole program. She actually like 8th in the nation in motorcycle racing right now. Interviewee 1: That’s good Interviewer 1: Yeah, real good Interviewer 1: Last question, you kind of touched on it a little bit. In what ways did the STEM Education program promote female and minority students into STEM related education and careers. Interviewee 1: In that particular group last year, we always fighting at battle. I know one of the students; one of the four girls, her mother is an engineer. She is already leaning in that direction anyway. Right now her focus is swimming. She was with me for 3 years and again her work was exceptional. She showed more interest in it, and like I said, Destiny herself, just broke down so many double barriers again by being female and by being an African American, and there was so many African American students in the group and I think they were impressed. Interviewer 1: We were at Highland Springs this morning, and some of the students gushed over Destiny. Role models and our research indicated that females and minorities need to see role models in STEM and know that they have support. School H TL - I have six questions for you. RIR STEM EDUCATION DAY PROGRAM EVALUATION 153 1. What did the students think about RIR STEM Education Day? What did they say about it afterwards? The kids really liked it...I only took my 8th grade class...the reason being I really didn’t know what the program was going to be like. I want to continue the tradition because to me I used that as a springboard the whole 9 yards too as it was leading up the program. The kids really thought it was an honor to go to the program because they earned it...so when they got there....we didn’t know what to anticipate and once we got into the large building...the kids were able to walk around at first which was good ...we were one of the first buses so I think our kids had more time...I also think the program got off a little bit later because the other buses...so from a logistic standpoint...my kids were excited...if you were on a later bus the kids had to sit and listen to Russo and the others...getting there early was great for the kids...the later kids got sent to the wolves. The kids loved the stuff in the building and the activities on the track...the pizza...all in all the feedback from the kids when they got back was that they were excited...and it has carried over a little bit this year as the kids are asking me about race day and I haven't said a word yet....this year will be better in that kids will know what they are going to and the teachers will be able to better prepare the students for the trip....I can push to do more activities to lead up to it. 2. In what ways were you able to connect your class content to what students experienced that day? basically through the co2 cars which are the metric 500...those schools throughout the country...so that unit along with the new unit on crash dummy...a paper car activities with Pringle’s potato chips...we have a ramp coming down and the paper vehicle at the end...the chip is the passenger...so with those two classroom activities done beforehand again that leads up to watching the real thing...so we talk about roll cages, we talk about forces, impact, aerodynamics...we can bring in these terms that were used with these units and now when the kids get here they are kind of clued into what’s going on...the kids are used to NASCAR here so the classroom activities tie into prepping the kids before they go to it. - How have you adapted lesson plans that tied back to STEM components such as force, speed, etc.? At one time I thought that there would be a follow-up packet that had terminology like a lesson or two that might deal with g forces, impact, speed, distance-time...all that stuff you know how things are figured...kind of like a crayon book type of thing that kids could have done that were simple enough to get the concepts across but at the middle school/high school level....pre lessons and post lessons to follow up...it could be the rules of NASCAR or something like that...something that the kids can have when they get back that the teacher can follow-up with...the next couple of days we will be figuring out...metrics, rpm, engines, horsepower...tie in the math curriculum...you could also do advertisements/marketing/business...you need the revue RIR STEM EDUCATION DAY PROGRAM EVALUATION 154 coming in to run businesses and to promote your product...go through how NASCAR markets its cars...the teachers have a lot of creativity...some type of follow-up packet would be good...pre or post with some math concepts. 3. What changes did you see in how students talked about STEM following the RIR STEM Education Day? a lot of it would have if we would have had that follow up activity as a follow up...we have things we have to accomplish and since this was the first time I was asked for feedback right away and the kids gave their feedback right there...I couldn’t really tell you...I substituted that document for a classroom discussion from a time standpoint...would I elaborate on it or go further, yes...didn't know if this was a one-time deal...in hind sight if I had known this program was annual I would have asked for more feedback. - Did their grades improve? Had we had a pre and a post-test...we probably would have seen a correlation. - Were students engaged and/or interested in STEM? Was there increased participation? I would say yes. It was a good opportunity to expand what we do in the classroom with the racecars and crash exercises...they are able to see the real thing...and see these things beyond racing...they can correlate the design process 4. What did students say or do following the RIR STEM Education Day program that may have indicated an increased interest in STEM educational opportunities? By the time we had this program the students were already signing up for high school programs...we talked about careers more than from an educational standpoint and used this program in my discussions about career opportunities and how many jobs were needed to even put this program on...and how many of these jobs deal with STEM...deal with technology...and that the jobs out there...weren’t it comes to technology you are looking at 80% of available jobs deal with technology...I push a lot of our technology and education classes which is a part of CTE 5. What did students say or do following the RIR STEM Education Day program that may have indicated an increased interest in STEM career opportunities? Covered above. 6. In what ways did the RIR STEM Education Day Program promote female and minority students' interest in STEM-related education and careers? Did you see what was coming off our busses? 90% boys. So that’s one of the biggest problems we have as CTE when it gets into the technology aspect...home economics, business, consumer RIR STEM EDUCATION DAY PROGRAM EVALUATION 155 science are all fine...technology education I have 60 kids, 1 girl, 8th grade...I’ve had girls before but most of them are from other countries...they took my class because they wanted to be an engineer. I have 1 girl this year and she is just one of the guys...she doesn't want to be an engineer but just likes working with her hands. That is par for the course nation-wide...girls are not jumping into these classes...so as much as you emphasized the minorities...the gender...NASCAR is probably 75%-25% male-female so you have a male activity pretty much...it’s a STEM activity but its male...how do you get more females to your program when they are not in our classes....start bringing the family and consumer science kids too and then you will get the girls to the program...start introducing the kitchens and all the other aspects of a race and how it all works together...you bring the business class in which is more females...so you are catering to kids who the mechanics, etc....right now, the way things are now you are going to have mostly boys...what we need to do is to offer a class for all girls...they are intimidated by a traditional all boys class...it would be a whole different ballgame...I think we would see immediate results where we would see more girls going into technology curriculum...you know STEM fields... School K TL - I have six questions for you. 1. What did the students think about RIR STEM Education Day? What did they say about it afterwards? We did not have one negative comment and we had close to 50 kids. They all had a good time...riding back on the bus I heard one student in particular talk to another student and said that this was the best trip that had ever been on. All in all it was extremely positive because they even mentioned it to their teacher because when we got back and school was still in session...they dispersed to their classrooms and then I heard from other teachers the kids really had a good time because they were talking about it with other students....they all had a real nice positive experience....the program definitely had a cool factor to it. 2. In what ways were you able to connect your class content to what students experienced that day? Last year...being that the time of year that it was...and being it was our first time around ...I was only able to connect it to the form of employability skills an business orientation and awareness...meaning, we talked about it...they experienced it....it was getting close to the end of the year...we didn’t do the project which is a co2 dragster in conjunction with because it wasn’t timed in such a manner...but we will try to do this year. So, I was not able to incorporate the dragster and talk more about racing but we typically would have done. We talked about opportunities in racing...not just the driver but we mentioned business and mentioned a lot of money in the business, job opportunities outside of the actual and physical race... I wanted them to have more of an awareness of that...and so we talked about those things and I would have RIR STEM EDUCATION DAY PROGRAM EVALUATION 156 liked to have done our car so that we could link it together more so and them dive more into the STEM of it...the STEM skills and knowledge of it. - How have you adapted lesson plans that tied back to STEM components such as force, speed, etc.? We are going to be doing more of that this year. 3. What changes did you see in how students talked about STEM following the RIR STEM Education Day? I didn’t witness a lot of conversation amongst them about it because when we got back it toward the end of the year so we are jumping into other things to prepare them for their final exams...we were trying to wrap up the end of the year...it was late April so there was not a lot of time left to finish up what we were doing and finish up the year to get ready for final exams...and they are dealing with their other coursework and SOL's, etc. - Did their grades improve? The stations were very educational...they obviously closer to one on one...they don't just view cars going round and round...this is where the real learning takes place...the was the heart of the matter as far as learning and STEM...but I don't think there was time for grades to actually improve but students had better attitudes toward everything which might lead to their grades improving...they might say hey this is cool and this was what school is all about....this is how it connects to racing and other things...if their attitudes changed just a little bit it might lead to better grades if there was more time. - Were students engaged and/or interested in STEM? Was there increased participation? 4. What did students say or do following the RIR STEM Education Day program that may have indicated an increased interest in STEM educational opportunities? We haven’t discussed this with the kids...I think by that point the kids already knew which specialty center...it was already established. 5. What did students say or do following the RIR STEM Education Day program that may have indicated an increased interest in STEM career opportunities? Some of the kids took us up on the race tickets and went that same night and I had kids and their families went and they really enjoyed it...I really don't know. 6. In what ways did the RIR STEM Education Day Program promote female and minority students' interest in STEM-related education and careers? RIR STEM EDUCATION DAY PROGRAM EVALUATION 157 There were quite a few females there and I thought that they had a good experience...I don’t think we necessarily had a conversation about females and minorities in particular... it was just they were there and in it and partook like everybody else...it wasn’t like there were a group set aside or set apart I any way...we didn’t have a conversation because we look as everybody as equals...we feel like hey you can do this and they know they can do anything....they saw that girl there...we ned more of that...the kids need to see that they can do these things. School L TL - I have six questions for you. 1. What did the students think about RIR STEM Education Day? What did they say about it afterwards? My kids thought it was great. they had an opportunity to see the young kid from Elko who races...it gave them the feeling that they can do this...we were building the co2 cars at the time and that really made it more exciting for them...it made them wonder just how fast their car could go...it motivated my kids in so many ways...first it showed my kids that they could do this...and second I really want to race a nice car...and the design process was awesome it really jump started a lot of things we were doing here...we race c02 cars and speed...how fast can something go?...they wanted to get in and race the cars...the small cars had the most impact...they thought it was adults in those cars and saw that kids can do this too...it also showed that they can have fun learning this stuff. My kids wanted to race the cars and to see how fast these cars could go...how long the tires will last...how hot do these tires get...they were amazed at how loud these cars were...it was awesome...they really enjoyed it...they could see that what they were doing in the classroom could lead to what was going on at the program. The questions were about safety, helmets, fire suits...what’s the fastest car ever raced...were there any black racers?...will that kid race in the big race>...it brought up so many questions and it opened up their minds to the sport...how much money can a racer make...can we go back?...they wanted to start getting into mechanics...specialty centers...it was great...the specialty centers are the next big step because our kids wanted to know if they still teach welding...what centers have what they wanted to learn... 2. In what ways were you able to connect your class content to what students experienced that day? I got to elaborate on how important design is...how it important it is to test run...how important it is to test a model...how important it is to take your time on what you’re doing and safety became more important because everyone was thinking man that was fast...speed grabbed their attention and brought how important safety was...how important it is to create a good model to test speed, aerodynamics...how long it can take to get from here to there and at what speed...lots of math and engineering, since comes in play...sometimes kids don't want to talk about it...they don’t teach it like this in our classes and this program made it fun...the kids were able to see how what they RIR STEM EDUCATION DAY PROGRAM EVALUATION 158 were doing related to the science, technology related to ... It was amazing because they started asking more questions themselves...they were more aware and more engaged - How have you adapted lesson plans that tied back to STEM components such as force, speed, etc.? It allowed me to make it more challenging for the kids...when you go to an program like this they start answering some more questions by themselves you can start challenging their minds with more challenging questions...they’ve taken a part of it and i may not have to spend a whole day on because the program did it for me...they love hands on a lot better than what’s in that book. 3. What changes did you see in how students talked about STEM following the RIR STEM Education Day? It was easier...it made sense...the kids started saying we talked about that Mr. Handsome’s class...they tried to say ...and didn't I tell you that in class...they seemed to be on top of it. - Did their grades improve? For some it did...for the majority it did...the reason I say this is because it kept their attention....where some kids were tended to fall away...you know what? I really do want to do that...you hear a lot of kids say that electives are not important...then they realize you are doing these are the types of things and this is related to that...they see that it is fun and this is what they want to do...things are easier to do if there is some fun...that’s the reaction that I enjoy. - Were students engaged and/or interested in STEM? Was there increased participation? 4. What did students say or do following the RIR STEM Education Day program that may have indicated an increased interest in STEM educational opportunities? some of the kids really start ...this kid right here...he was ready...he was ready to get into TSA or tech program...anything that had to do with motors, speed...he was geared up...he was ready at that point and some of those kids the light comes on and that’s what you want to see...one of the main questions that they ask is what kind of money do these people make...I say to look it up...but they think everyone is a millionaire and don’t understand that people start at the bottom 5. What did students say or do following the RIR STEM Education Day program that may have indicated an increased interest in STEM career opportunities? It sparked their mind and their attention...when they say college they say this is what I want to do...these kids, some want to go to IT Tech...they want to know how to get to that point...when you say it’s a sport we look at anything is possible and the right avenue to that sport...they see the commercial...they want to go to mechanics school...they're on it but they just need to find out what to do to prepare themselves....all of our talk is about careers...these kids want to invent RIR STEM EDUCATION DAY PROGRAM EVALUATION 159 something...they want to be a designer of a particular car....and I tell them they can do that...to design that...to build that 6. In what ways did the RIR STEM Education Day Program promote female and minority students' interest in STEM-related education and careers? Once my girls saw Destiny you guys can forget it...we will take over...I get so many girls in my class and they are on top of it...they love it. At the beginning they don't think they can do it but soon they are taking over the class...I think more are headed in that direction and into that field. They have been asking me to get Destiny over here to talk with them...we talk about ethnicity...we try to figure out how to get them involved but the majority wants to know the salaries...they asked if blacks were involved in racing...I know the kids thought that the program belonged to them and that it was what they wanted to do...especially the girls. They said we're in! The guys said that they wanted to build this or build that but the girls wanted to get in the car or on the bike...I want to race! The guys were saying we are going to get a bike any way but I want to make this thing faster. Students High School 1 Interviewer 1: I’ll just start with question #1. Think back to the RIR Education Day which is the program you attended, what did you find interesting about that day? Interviewee1: I found like the speakers were talking about the application of Math and Science with the racing and stuff and how the fuels, and car, and tires work. Interviewer 2: So the speakers not at the stage but at each of the stations. Interviewee1: Un huh. Interviewer1: okay, how about you? Interviewee2: I liked listening to the speakers at the stations as well. I think my favorite part of it was looking at the structure of the walls where the cars were racing. Interviewer 1: Was there anything specific about that, that you liked, that helped you understand any components of STEM or anything? Interviewee2: I learned a bit more of how STEM applies to racing and they really Interviewer 1: Was there anything that helped you understand the concepts of STEM that you saw at particular station. Interviewee1: Business station explained a lot about how they make money off their system and stuff. RIR STEM EDUCATION DAY PROGRAM EVALUATION 160 Interviewer 1: The sponsors and all Interviewee1: Yeah, I really didn’t know they made money. Interviewer 2 : Think back guys as to how you was in middle school did you work on any of the co2 dragsters, like you did in middle school, now how might that program relate to the STEM Day? Interviewee2: things. They both kind of relate to how it applies to cars and stuff you know, everyday Interviewer 1: Ok, so what was it you did, what did you do there that you really liked? Interviewee2: really cool. Kind of like looking at the different structures of the cars, the tires, that was Interviewer 1: What about any hands on activities, did you get a chance to do any hands on activities, and if so, what was it? Did you like that or did you like more of the information? Interviewee 2: I didn’t really get a chance to do any hands on stuff, there was a lot of people there. Interviewer 1: Did you get a chance to do anything hands on or interactive? Interviewee3: I don’t really think I did either Interviewer 1: Do you think that might have been something you would have been interesting in doing? Interviewee 1: yes Interviewer 1: Did any of you get a chance to do the simulator? Interviewee1 & 2: No Interviewer 1: That’s been a popular one. Interviewer 1: Like what Will was talking about and relating things back to classroom, you did the co2 dragster. If you guys did that, was there anything specifically you could say, was the station you saw or participated in, you could say, Oh that’s kind of like what we did with the dragster. Interviewee2: There was one station I talked about, talked about the different art design. How they design the outside of the car on paper and stuff. Interviewer 1: ok, anything else RIR STEM EDUCATION DAY PROGRAM EVALUATION 161 Interviewee1: There was one of the stations that was talking about the wheel s of the car. I thought about how when we had made our c02 dragsters, how different people had made where their axles go. I had made mine like a thicker area than some people had made it thinner and theirs ran faster, and it gave me ideas if I ever got to make another c02 car. Interviewer 1: That’s great. Were there stations where you would of liked to spend more time? Any stations that you can think of that you maybe would like to spend a little extra time at. Interviewee 2: Probably the tire station Interviewer 1: Did you find anything specifically that you found out that was interesting Interviewee2: How thick those tires are Interviewer 1: A lot different than the ones that go on your car Interviewee 1& 2: yeah Interviewer 1: What about you. Interviewee3: Like I said before, the axles and where they connected and I hadn’t really seen much of that because when you watching cars drive, you don’t really see much of the mechanics, you just see the car moving and like the driver. Interviewer 1: So it made you think a lot more about the Science and Technology behind it after you experienced that station. Interviewer 1: Was there anything as the result of this program that made you think you might want to be an Engineer, Mechanical Engineer, Scientist, or anything like that. Is it something you thought of or think you might be interested in doing? Did this experience help you in that way? Interviewee2: Yeah, it did kind of help me think about getting a STEM degree. They talked about how smart the drivers are actually and how they have all these degrees in Technology and Engineering. Interviewer 1: Sure, it probably helped to understand all that Geometry and mechanics behind it if you had to be behind the wheel. So you found that interesting. Interviewee 1: I’ve always wanted to be an engineer from when I started middle school. When I first took my technology class, I thought it was interesting. I joined TSA in 6th grade and I’m currently a chapter officer here at Deep Run. Interviewer 1: Great, so you have some background knowledge in technology and engineering, did you feel this program was effective in increasing interest in it. RIR STEM EDUCATION DAY PROGRAM EVALUATION 162 Interviewee 1: It gave me an interest in different types of fields, and this was one of the more mechanical fields of engineering and gave me a good idea in what field I want to go in to Interviewer 1: Do you have an interest in a career in engineering? Interviewee 2: Not really me, but it helped Interviewer 1: Provide some interest in, exposure Interviewer 2: What kind of engineer do you want to be. Interviewee 1: I’m not really sure yet, but I’m interested in learning more. Interviewer1: Do you know what you want to do? Interviewee 3: No, but probably not engineering. Interviewer 1: So what do you think maybe could have made it better? Or something we could of added, to leave out just to make the experience better? Interviewee 2: I would probably leave out, you know when you are in the center stage, the guest speakers, because that took a lot of time, and more time on the stations Interviewee 3: Definitely more time at different stations, it’s like you said before, the simulator would of definitely Interviewer 2: Last question, what station made the biggest impact on you, which one jumps out of your head first when you think back to this day, which station, and why? Interviewee 2: probably the station where you got to see the inside of the car, and see the cut outs Interviewee1: I think the cut outs interested me too and whenever I think of this field trip, I think of the station that explains how the walls were made and I always think that it would be interesting to see how we could use those types of engineering properties and other things, like bridges and such Interviewer 1: If you had the opportunity to go back to this program, would you go? Interviewee 1: Yeah Interviewee 2: Definitely Interviewer1: Just to get a chance to get on the simulator Interviewer 1: yeah RIR STEM EDUCATION DAY PROGRAM EVALUATION 163 Interviewer 1: Did you get to watch the cars take a lap around the track? Interviewee 2: Yeah Interviewer 1: Did you see it with a different perspective after you been in and seen all of this? Intervieweer2: yeah Interviewer 2: Are you guys NASCAR fans Interviewee 2: No Interviewee 1: Not really, I never been to an actual race, but when I was there, it gave me a good idea what it look like High School 2 (Males) Interviewer 1: First question is, think back to RIR Education Day over at the raceway. What did you find interesting about that day, what kind of things jumped out at you, you found interesting. Interviewee1: The cars, seeing all of them and how they work, and the simulation was also really cool. Interviewer 1: What about the simulation did you like? Interviewee1: like playing it I guess Interviewer 1: Okay, very good. So for the others who went, was it a fun experience or a boring experience. Interviewee1: some was bored, some was having fun Interviewer 1: Which ones did you fine boring? Interviewee1: ummmm Interviewer 1: Better yet, which ones did you find exciting besides the simulator. Interviewee1: seeing all the tires, and how they work, that was really cool, ahhh and Interviewer 2: did you meet any of the drivers Interviewee1: uhh No Interviewer 2: Do you recall anything that you saw, that was fun Interviewee2: yeah I thought the simulator was fun too Interviewer 2: do you guys study STEM? Science, Technology, Engineering, and Math RIR STEM EDUCATION DAY PROGRAM EVALUATION Interviewee1: 164 we don’t really study it, but we learn about it Interviewer 1: We touched on the second question about what you really liked, As far as STEM, what sort of hands on activities did you like that was at RIR education day, where you was actually had to, you were able to actually apply and look at what you studied in class Interviewee 2: Was there anything that popped out Interviewee 2: I don’t really remember Interviewer 1: Yeah we understand it was a while ago. Anything from that day that you remember that made you understand STEM better. Understand Science, Technology, the Engineering, or the Math Interviewee1: Looking at the cut out car and seeing like the motor Interviewer 1: Do you remember the simulator? Interviewee1: Yeah, it was a long line Interviewer 1: Yeah that was one of the more popular areas. I guess the simulator will be the answer here, but were there stations, that you would like to spend more time? Interviewee1: My class didn’t get to like maybe 5 or6. Interviewer 1: You got like half way. Of the stations that you did get to, which ones stood out for you Interviewee 2: The simulator Interviewee: The three things I remember were the simulator, and the guy was talking to us about market, like promoting NASCAR or something like that and the tire one. Interviewer 1: Ok Interviewer 2: Do you recall any other ones that you like Interviewee1: There was one, where they demonstrated how the oil filtration. How it works Interviewer 2 : You guys remember the air filter kind of thing with the ping pong ball Interviewee1: Oh Yeah Interviewer 1: did you find any of that stuff interesting, was it stuff you already knew, understood? Interviewee1: sort of RIR STEM EDUCATION DAY PROGRAM EVALUATION 165 Interviewee2: I kind of understood it before. Interviewer 1: What about the tires, you mentioned something about the tires you found interesting. What about that. Interviewee1: Like how they have the thickness that programed like the rubber from getting all over the track. I think that was interesting Interviewer 1: What do you think would have made that experience better? Anything you think that would of made that trip out to RIR more interesting for you? Interviewee1: Maybe like have a NASCAR driver come over and talk to us Interviewer 1: ok Interviewee1: Like Jimmy Johnson Interviewer 2: Like a celebrity driver Interviewee 2: Yeah Interviewer 1: What would you ask him if he was there? Interviewee1: How does it feel on the track? Like did it take a lot of preparation to do that, like to drive Interviewer 2: Ok, Are you a NASCAR fan? I use to be, are you guys race fans at all Interviewee1: No Interviewer 1: Even if you are not racing, would you like to get out on the track and drive a car Interviewee 1: yeah Interviewee 2: yeah Interviewer 2: Do any of you guys want to grow up to be an Engineer or Scientist? Interviewee 1: I want to seems like a good job like this, like rising in, Interviewer 1: You are absolutely right about that. Interviewer 2: So the two of you guys that was there or I don’t remember the two of you seeing this, do you think the program make you more aware of Science and Technology, Engineering, and Math is in racing. Like maybe you didn’t think about before. Interviewee 2: Like on the car, like geometry and stuff make it strong and not break while you’re driving RIR STEM EDUCATION DAY PROGRAM EVALUATION 166 Interviewer 2: What do you guys want to be when you grow up? Interviewee1: Construction worker Interviewer 2: A lot of STEM Interviewee2: Military Interviewee 1: Maybe something in alternative energy Interviewer 1: Definitely a lot of STEM Interviewee 1: That’s what my dad does Interviewer 1: What station made the biggest impact on you? Interviewee 1: Probably seeing like the cut out car, that again showed me like how everything works together to for STEM. Interviewee 1: Anything from that day, made you more interested in pursuing STEM Education . When you get into your high school years, you are already in your high school years, would you want to pursue STEM as you move forward Interviewee 1: Probably a lot of it Interviewer 1: ok High School 2 (Females) Interviewer 1: First question is, think back to RIR Education Day the program we were just showing you with the pictures. What did you find interesting about that day. Anything that was interesting about it? Interviewee: Ummm Interviewer 1: There’s no wrong answers. You can just… Interviewee: Trying to figure out. I really don’t remember this day. Ummm I just can’t. Interviewer 1: Any of the cars, the cut outs, the learning about how… Interviewee: I remember going to VCU, and watching, no not VCU, I lied. Interviewer 1: RIR Interviewee: The Coliseum and watching… Interviewer 1: The Arena Racing, Ok RIR STEM EDUCATION DAY PROGRAM EVALUATION Interviewee: 167 The Arena Racing, that’s all I remember Interviewer 1: Ok, anything fun about that experience, did you think it was interesting at all? Interviewee: Yeah Interviewer 1: Yeah, ok. Would you think it was boring or was more fun? Interviewee: Umm, It really wasn’t my thing Interviewer 1: Wasn’t really your thing, yeah, that’s ok. Umm, Did it help you , was there anything that you saw with that helped you understand any science or technology stuff that you been doing in a classroom that was maybe specific with that kind of program? Interviewee: No Interviewer 1: No, ok. So what was it, what did you do, did you do anything, you remember, that you really like? Interviewee: Uh, we walked around and looked at the cars that were there. Interviewer 1: Ok, was that pretty cool to see the different cars? Interviewee: [nods] Yeah Interviewer 1: Yeah, have you ever seen anything like that before? Interviewee: Ugh-un Interviewer 1: Ok, Did you do any hands on activities? Do you remember getting to like touch stuff, play with stuff, or you know like interact Interviewee: No Interviewer 1: Would you like to of had have done any hands on activities stuff Interviewee: Yeah Interviewer 1: Ok, do you think the hands on activity would help you understand like the science and technology as it relates to racing if you were doing hands on activities? Interviewee: Yeah Interviewer 1: If you got to, do you. Ok. Interviewee: Yeah RIR STEM EDUCATION DAY PROGRAM EVALUATION 168 Interviewer 1: Can you think of any of the stations, they had stations set up, you know different stations showing different things. Were there any of them that you could think of that you might of liked to of spent some more time at or looked at a little longer. Interviewee: Umm, They had like the cars and we got in the cars Interviewer 1: Was that the simulator Interviewee: Yeah Interviewer 1: The simulator Interviewee: Yeah Interviewer 1: You like that; yeah the simulator is pretty cool. Interviewer 1: Umm, let’s see…Was there anything that you did like the simulator that you think made you think you might want to learn more about how this stuff works, how the cars actually work or any of the science or technology, and engineering that’s involved with it. Interviewee: When we did the cars, when we got in the cars that made me think how it works Interviewer 1: When you got in them. That made you more interested in how it works, ok and do you have anything that you want feel free to follow up on. Interviewer 1: Did you do anything there or see anything there that maybe peaked your interest in maybe being like an engineer or driver or working with Interviewee: ugh un [laughs] Interviewer 1: [laughs] No nothing. That’s ok. Nothing wrong with that. Interviewer 2: [laughs] Interviewer 1: What do you think could have been done to maybe make that program better? Interviewee: If we had longer time. Interviewer 1: Longer time, longer time like at each station or longer time just for the whole day? Interviewee: Longer time for the whole day. Interviewer 1: For the whole day, ok, that’s a good suggestion. Was there anything that you think could be left out, maybe or was there anything that you didn’t think wasn’t really necessary? RIR STEM EDUCATION DAY PROGRAM EVALUATION Interviewee: 169 No Interviewer 1: What station do you think made the biggest impact on you, like was there one station that really stood out that you kind of thought about maybe after the program was over. Any station that made some more impact on you. Interviewer 1: Just the stimulator, yeah the stimulator, you like that, you liked being able to get in and drive it and do that. Okay. Interviewer 1: What made you remember that station the most, the stimulator station? Interviewee: Because I got to get inside it. Interviewer 1: Because you actually got to get inside of it, yeah, and that about as hands on. Yeah, ok. Did you have anything that you want to follow up on? Interviewer 2: Are you very interested in learning more about Science and technology? Interviewee: No Interviewer 2: Why is that? Interviewee: I don’t really like Science and technology and engineering, it’s not. Interviewer 2: Do you think of your teacher s would of made it more hands on, creative or had made Science fun, you would be interested in Science and technology? Interviewee: Ummm, I don’t know Interviewer 2: Are you good in Math? Interviewee: Ummm,Not really Interviewer 2: [laughs] Interviewer 1: Well that’s it. Do you have any questions or anything, do you want to ask anything, elaborate on anything else? Interviewer 2: No Interviewee: No Interviewer 1: Alright, what? Interviewer 2: You said you didn’t participate? Interviewee: No RIR STEM EDUCATION DAY PROGRAM EVALUATION 170 Interviewer 1: That is it. High School 3 (Males) Interviewer 1: I’ll just start with question #1. Feel free to answer and provide specific details and if you want to elaborate, feel free to do so. So think back to the RIR Education Day which is the program you attended, what did you find interesting about that day? Interviewee1: I found all the physics and all that kind of stuff interesting. Like how all that played into, how they shift the cars, why they don’t have doors on the cars, and stuff like that. Interviewer 1: So learning that physics, was there one particular station. Interviewee1: Just when I asked questions, you know like why they don’t have doors, they were talking about wind can get into all the crevices and stuff. Interviewer1: okay Interviewee2: was cool. How many people it takes for this development and how much fuel it takes, that Interviewer 1: A lot more than you may expected. Interviewee2: They kind of said the behind the scenes things that go into these races and when you watch on TV and stuff, like you see them practicing, some of the people that participated and , kind of like how everything happens. Interviewer 1: Okay, and you kind of mentioned, what ways did it help you understand STEM and if it did in any way. Science, Technology, Engineering, and Math, any of those components. Was there something you did at this program that helped you understand something Interviewee1: Again, I guess a lot of the physics and also for the technology part, all the technology that they use and how it’s developed over the years, I guess. Interviewee2: Pretty much the same thing, Technology, the Math it takes Interviewer 1: Were there any specifics thing you saw or with regards to Math that may you kind of be like “oh that’s interesting”? Interviewee1: Like how long it really takes them to prep up for these races and stuff and it is like Science behind the cars and the engine and stuff. Makes you kind of want to be an engineer. Interviewer 1: Ok, that’s great. What did you do while at the program that you really liked. RIR STEM EDUCATION DAY PROGRAM EVALUATION 171 Interviewee1: Other than eating pizza, I kind of just liked walking around the room looking at all the different stations and checking out all that goes into racing and all the Technology, Engineering, and Math that it takes to actually have a race. Interviewer 1: Ok, anybody else? Interviewer 2: Was there anything specific that you really liked? Interviewee2: Listening to the drivers stories. There was one of them, we asked what happens in VIP when you are in there, he sort of told of some funny stories, also one of the stations we had to like yell as loud as you can and if it didn’t match the sound that the car’s engine makes. That was really cool, to like comparison. Interviewer 2: Were there like any activities like hands on activities that you go to participate in that you liked? Interviewee 1: The one with the guy in the oil, he actually showed that they have to use this specific cloth. To sop up all the oil and he kind of let us try it out and he said it is basically like a ten times a million Interviewee 2: I got to touch a tire Interviewer 1: A what? Interviewee2: A tire. Interviewer 1: Ok, that’s pretty cool Interviewer 2: What did you like about the tire? Did you learn anything about the tire? Interviewee2: I forgot, I think it was bigger, and it looked smoother. The inside, I think the insulation, I kind of liked the insulation. Interviewee3: The part I liked was the Redskins car. Because I’m a Redskins fan, I just like seeing that. I was the last student so I kept the Redskin picture. So that’s a popular car. Interviewer 1: Was there any activities that you did at the program that you can relate back to classroom activities? Activities you’ve worked on in class you can say, oh that makes me kind of understand why we were doing that in one of your classes. Interviewee3: The one at the bottom left corner did like these experiments kind of, I don’t quite remember Interviewee1: Was it the potato chip car? RIR STEM EDUCATION DAY PROGRAM EVALUATION 172 Interviewee3: I don’t remember, it’s like the raceway and how it like, like the floor of the raceway, they had a sample. I can’t quite remember but it was like kind of the floor, and you could like feel Interviewer 1: Was it a woman that was doing that? Interviewee3: yeah Interviewer 1: Was there anything in class you had done that made you understand, we talked about that in class, yeah you know Interviewee 1: We did make C02 cars and we kind of tried to shake them to give them speed and they talked about that with the race cars Interviewer 1: What do you mean by shaking them for speed? Interviewee1: Like how to make the aerodynamic, sending them down and stuff like that. Interviewer 2: What about the weight to the car, anything? Interviewee 1: We didn’t really weigh it, it was a really lite wood, so it really didn’t matter Interviewer 1: Okay Interviewee1: And they were all pretty much the same size, so it was just shaking it. Interviewer 2: Okay, where there station you would of like to spend more time? Interviewee2: We didn’t go to that simulator, so it would have been cool to of went to that, honestly. Interviewee1: Yeah Interviewer 1: That seemed to be a popular one Interviewee1: Yeah, I would of like to go to that Interviewer 1: So you would of like at least had a chance to go to the simulator and spend some time on that. Interviewer 2: We have to see if there are more simulators. Interviewer 1: Was there anything that you did that made you think you would like to learn more about, Science, Technology, Engineering, and Math? After you went to this program, was there anything that kind of prompted you that you really want to learn more about STEM? RIR STEM EDUCATION DAY PROGRAM EVALUATION 173 Interviewee 1: It was kind of the things that got me thinking again back to the physics with why they don’t have doors and stuff, kind of thinking how air would get trapped in there and stuff like that, the things that kind of make me think about it for a little bit. Interviewer 1: You want to learn a little more about it perhaps Interviewee 1: yeah Interviewee2: It’s like so big, the car is so heavy and stuff so they really fast Interviewee3: The engines, like fixing them, preparing them and stuff, that’s interesting if you want that kind of job Interviewer 1: That leads us, do you think you want to be a Scientist, Engineer, or Mechanic, or anything like that in the future? Interviewee 2: I want to be a Mathematician Interviewer 1: Mathematician, that’s great. Interviewee 2: I got to learn how much it pays but is interesting. It’s a fun thing to do. Interviewee 1: I said definitely, because like I told you guys before, I kind of want to be a Physical Therapist and that field is in Science and yeah. Interviewer13: Yeah ok. What do you think was some things you think that might have made the experience better? Interviewee 1: Not really, other than again, more time on the simulator Interviewer 1: You didn’t get to the simulator, making sure everyone has time to get around to all the stations perhaps. Interviewee 1 & 2: yeah Interviewee 2: Meeting more racers, that would be cool. To like learn their stories and stuff. Interviewer 1: Did you get to meet any at all. I think you sat down and talked to one. Interviewee 2: yeah one guy, that was pretty cool Interviewee3: There were boring stations there. Interviewer 2: There were boing stations there, what made them boring? Interviewee 3: Just the tire one. Interviewer1: I thought you said you loved the tires RIR STEM EDUCATION DAY PROGRAM EVALUATION 174 Interviewer 1: You was joking? Interviewee 3: it was like touching the tires Interviewer 1: You said you got to meet the female motorcycle rider? Interviewee 3: Yeah, did she have like an injury or something in the past? Interviewer 1: I’m not sure Interviewer 2: She had a pink, pink bike. Do you remember meeting her, Destiny was her name. Interviewee 1: I don’t think I met her Interviewer 1: You didn’t get a chance to meet her. Interviewee 2: I like seeing the cars practice and also they gave us like free tickets to the race afterwards which I thought was cool. I wasn’t able to go but I had a friend that I was able to give free tickets. Interviewer 1: Did you get the bar of soap that was shaped like a car. Interviewee 1, 2, 3: Yeah Interviewer 1: Were there anything you would leave out? Any part of the experience you think you would leave out? Interviewee1: Probably not. Interviewee2: Some stations maybe like finding more interactive stations for the kids, I know that one thing that makes me interested. Interviewee 2: Hands on, interactive Interviewee2: yeah, instead of someone telling you about it, you get to hands on Interviewer 1: What station made the biggest impact on you? What station really stands out the most for you during the experience? Interviewee 3: I think I went to one where there was like half a car. Interviewer 1: The cut out? What impact, what do you remember about it? Interviewee 3: You can see what is behind it. Like how it cover, it’s like an overlapping like what is under the car, what it takes to make it go. Interviewer 2: What is under the body? RIR STEM EDUCATION DAY PROGRAM EVALUATION 175 Interviewee 3: yeah Interviewer 1: I was going to say, it used to be called the body, now it’s called the skin. Interviewer 2: Any other stations, besides the cut out, any other stations, as you thinking back, you can, oh yeah I remember that one. Interviewee 2: I was going to yell, and even if you yelled, it didn’t match up to the sound of an engine. Interviewer 1: Did you give it your best shot? Interviewee2: Yeah, but not close, and just watching the cars, some people didn’t think it was cool. I just thought it was cool just watching the cars. Interviewer 1: Did you see them in a different perspective, now after you’ve been in and seen some of the stations Interviewee 1: Definitely. Interviewer 1: It wasn’t just kind of the cars going in a circle anymore, you kind of saw some of the physics and understand some of the concepts Interviewee 3: Watching NASCAR was like watching the future, like what we did and stuff. Interviewer 2: If you did have the opportunity to go back to this program, would you go back Interviewee 1: yeah Interviewee 2: yeah like different stations that you been to. I would go back to the simulator. High School 4 (Males) WN - Ok, first question, think back to the RIR Education Day - what did you find interesting about that day? S1 - The cars, and how they are made. TL – The cars?? That was my favorite right there (pointing), you guys remember that? S2 – Yeah, I like that one, too. TL – It’s called a cut-away car. So you can see what it looks like on the outside but then you get to look at everything on the inside of it too. WN - Do you think the program itself was fun? All Students - Yes, it was fun. RIR STEM EDUCATION DAY PROGRAM EVALUATION 176 WN - How so? S2 - Because it was hands-on. WN - Okay, hands-on . . . that is key. WN - What made it boring, if anything? S2 – Nothing was boring. S1 – The long wait. WN - Where? TL - You mean you from station-to-station? So to move the program quicker would be better? S2 - Yes. WN - In what ways did that experience help you understand STEM: science, technology, engineering, and math? S1 – How they ummm . . . the weather thing . . . how they check the weather. WN - What did you do that day that you really liked . . . which stations jumped out at you the most? Besides the cut-away. S1 - The safer wall . . . and how they made the walls safer. TL - You guys do any activities at all? S2 - We did that (pointing to picture). WN - What did you think? S2 - It was cool. TL - Do you know what that is? S2 – The simulator. TL - We need to get more simulators. All students – Yeah! WN - You mentioned the hands-on. What do you like about the hands-on activities? S2 - I like how they explained everything and showed us and passed stuff around. WN - So they talked to you about it and then you had a chance to actually handle it. Very good. RIR STEM EDUCATION DAY PROGRAM EVALUATION 177 S2 - Yes. WN - So did that hands-on activity make you better understand some things you may have heard in the classroom? S3 – Uhhh, yes. WN - How so? S3 - Well I can't really remember exactly but I just remember it helped me understand some stuff. It helped me with my CO2 car a little bit when I was designing it. WN - Did anything you experience that day relate back to your classroom materials? Students – No. TL - Anything more about the CO2 cars? Did you talk about speed or . . . ? S3 - Oh yeah we talked about speed and drag and stuff. TL - You guys talk about aerodynamics or even force? You talk about the safer wall? WN - Were their stations where you would've liked to have spent more time? S2 - The simulator. TL - Anything else? S2 - The safer wall was pretty cool, too. WN - What if anything there me you want to learn more about STEM based on the experience on that day? S1 - Arena racing and Destiny. TL - Have you been to and Arena race? S1 – (shook his head to indicate, no). WN - Anybody considering going into science, technology, engineering, or math once you get to the 11th or 12th grade or one of the specialty centers? Students – (no answers.) TL - Do you guys know what you want to be when you grow up? S2 – Yes, I'm trying to be a baseball player. S3 - I'm going to be a computer engineer. RIR STEM EDUCATION DAY PROGRAM EVALUATION 178 WN - That is right up the technology alley. S1 – Football. TL - Okay we have the sports guys over here. WN - What do you think would make that day better? S3 - If this stations move faster. I wanted to see all of the stations. I think the last one I saw was Arena racing. TL - You guys went out to the track and watch the cars go around the track. . . was that okay? S3 – Yeah. WN - Were there any stations that you would cut out? S3 - No, just move faster. WN - What station made the biggest impact on you? S2 - The one with the cutaway car. S3 - Yeah that was kind of the same for me. TL - What was it about the cutaway car that you liked? S2 & S3 - How it was made and the designed. WN – Anything specific about the design? S1 – How there was only one driver’s seat and the rest was the mechanics inside. TL - Did your instructor talk about any geometry or angles in the cars? S2 - We talked about how the wheel turns. The angle of the wheels and axle. WN & TL - Thank you guys very much for your time and have been a tremendous help. High School 4 (Females) Interviewer: First question is, think back to RIR Education Day the program that you attended. What did you find interesting about that day? Interviewee: Meeting the race car drivers and the younger race drivers in the smaller cars and the simulations. Interviewer: You like that? RIR STEM EDUCATION DAY PROGRAM EVALUATION Interviewee: 179 I really liked the simulations. Interviewer: Were you able to get into it. Interviewee: Uh huh, I went on it like 3 or 4 times. Interviewer: Okay that’s great. Have you ever done anything like that? Interviewee: When I went to the race. Like before the race starts. There always those carts and stuff. They have other simulations just like that. Interviewer: So did you think it was a fun program? Interviewee: Yes. The long speech we had at the beginning. That was kind of boring and there was some boring stations. Interviewer: Do you remember any of them specifically? Interviewee: No Interviewer: Did it help you understand STEM or Science, Technology, Engineering, and Math Interviewee: uh huh Interviewer: Is there any ways or specific examples you can think of, with the station you went to Interviewee: No Interviewer: But you understood that it was Science, Technology, and Engineering related. Interviewee: yeah Interviewer: So what did you do that you really liked while you was there. Interviewee: What I really liked was meeting Destiny. Interviewer: Meeting Destiny, what did you like about her? Interviewee: was young. That she was a female race car driver and that she rode motorcycles and that she Interviewee: Did you get a chance to talk to her? Interviewee: Yeah, and she let me get on the motorcycle Interviewer: Would that be something you would be interested in doing one day? Interviewee: Yeah RIR STEM EDUCATION DAY PROGRAM EVALUATION 180 Interviewer: That’s great. Did you get to do any of the hands on activities? You mentioned the simulator. Interviewee: Yeah Interviewer: Did you like being able to touch and feel stuff as oppose to just hearing about it. Interviewee: Yeah Interviewer: Was there anything of the stations that you went to, can you remember any of them where you can relate back to something you learned in class. Like a lesson or something you can relate to before you went. Did you do a c02 dragster or anything like that. Interviewee: No Interviewer: So wasn’t anything that you could relate to a classroom assignment. Interviewee: No, well in class year, at the time we were making cars. All we were really learning was how to build them. We didn’t learn anything about real cars. Interviewer: Yeah, did he talk about force Interviewee: yeah, force Interviewer: Did you see anything that RIR Ed day where they talked about force, where you could think back to “oh that’s what we were talking about” Interviewee: No Interviewer: You just heard the terms and stuff? Interviewee: Yeah Interviewer: Were there stations where you would have liked to spend more time? Interviewee: Yeah, the simulator Interviewer: The simulator Interviewee: The ones with Destiny, and the ones where we were looking at the cars Interviewer: Like the ones with the cut out cars. Interviewee: and they had the cut out cars and they had the full cars Interviewer: What did you like about looking at the cars? RIR STEM EDUCATION DAY PROGRAM EVALUATION 181 Interviewee: I just want to get in them. All the times when they have races, they have the car set ups, like other cars I always get in them. Interviewer: Yeah, so you interested in that Interviewee: Yeah Interviewer: So, I think you were answering this, but you can elaborate on it if you want, was there anything that you did that you think would like to learn more about Science or Technology based on looking at one of the stations, like looking at the card did it make you Interviewee: Yeah, looking at the cars made me want to learn how they are built. I always want to take a car apart and put it back together. Interviewer: Do you think that’s something you might be able to do. Interviewee: If I put in the work. Interviewer: Can you see yourself being like an engineer or a mechanic or something like that that builds or takes apart cars. Interviewee: I want to be the driver Interviewer: Ok, but you want to know how the car works though right? Interviewee: Yeah, I want to be able to fix it if something was to happen. Interviewer: That’s great. So what do you think would of made the whole experience better? Interviewee: Instead of when we had the long speech in the beginning, they showed us of how if something little was to happen on the side of the road, like how to fix it or important parts of the car. Interviewer: Maybe learning about things you can do on your car. Applications Interviewee1: And how the race car drivers will have to shift when they going on Interviewer1: Shifting gears Interviewer 1: I think you mentioned it, was there anything that you would leave out totally of the program? Interviewee1: The long speech and some of the boring stations. Interviewer 1: Do you remember specifically any of the boring stations? Interviewee1: Like the ones when the man just talked and showed pictures. RIR STEM EDUCATION DAY PROGRAM EVALUATION 182 Interviewer1: The stations where you didn’t get to interact. Interviewee1: There was some where you got to see them do things, those were okay, and there was ones were you actually was able to touch stuff. I just like looking at the pictures. Interviewer: What station do you think made the biggest impact on you on that day. Interviewee: The Destiny Interviewer: Yeah, the Destiny. What do you think was most impactful for you? Interviewee: Just the fact that she was a woman and she knew how to do all that stuff just like the men Interviewer: Just like the men, that’s great. Do you have some suggestion? Interviewee: Something added Interviewer: Something added, okay. Interviewee: Some cars that you can get in and look at closer. Interviewer: some cars that you can get in and look at closer. You want to see the gears and the engine and stuff like that. You think it will help you understand the Science and Technology and help you understand it more. High School 5 (Females) Interviewer 1: First question ladies, think back to RIR Day. What did you find interesting about that day? Interviewee1: I guess because like, you know how sometimes you are doing stuff, you just stop for minute and you just sitting there like what is the next thing we going to do, you know like that day, we just kept on moving and kept on moving, you know like we never got really bored. Interviewer 1: Anything about the stations that was interesting? Interviewee1: It was interesting how we got to actually do the hands on stuff and we got to try everything because some people don’t necessarily like things and once you try new things, you get to know it and we got to do the hands on things. Interviewer1: Alright, in what ways did it help you understand STEM? Interviewee1: It helped me by introducing me into all the details and helping me understand what is really going on and basically get to know more about this association. Interviewer 1: Okay, anything else? RIR STEM EDUCATION DAY PROGRAM EVALUATION Interviewee1: 183 ummmm, that’s it Interviewer 1: What did you really like about the day? What stations dropped out at you that you found interesting? Interviewer1: Don’t say Destiny’s Interviewee1: Yeah Destiny’s! I feel Destiny’s because she pretty much opened up my eyes to make me realize that it is not really just men that can do stuff but women can also do drag racing and regular racing and really awesome Interviewer 1: Ok, very good. What did you like about, you mentioned hands on, what did you like about the hands on activities? Interviewee1: Basically everything about them. Okay, I feel like people learn better when its hands on because they can actually see and feel what they doing and as they doing it. So they can learn easier and better from doing hands on and actually doing it rather than having somebody telling them how to do it. And I learned when we went, I learned things like that didn’t really have to do with the racing. I learned things like that have to do with it but it went more into other things that I didn’t know. I know like at his station he was showing us the different clothes that pick up the oil and I never knew that before he told me. I was that very good, that’s cool. Interviewer 1: Very good, my next question, did any of the activities you experienced that day relate back to thing that you covered in class. Interviewee1 & 2: uh huh Interviewer 1: How so? Interviewee1: I asked one of the guys like, is there a certain speed that you have to go to go around the corners and all of that. He was just like, he knows the speed that he should do when he goes around the corner and how you should just time it and all of that went into like Math and Science. Interviewee 2: Yeah, Basically it relates to Geometry and Math and all that good stuff Interviewer 2: Can you be specific as to how it relates to Math? Interviewee 1: Like Angles Interviewee 2: wrong. Yeah, make sure you have your calculations right cause something might go Interviewee 1: He was like he got into a car accident and he told us different stuff and I was like ooh, I learned that in Science. RIR STEM EDUCATION DAY PROGRAM EVALUATION Interviewee1: 184 Yeah Interviewer 1: Did it seem interesting to see how Science can relate with something you could relate it with. Interviewee1: Yeah Interviewer 1: Were there stations, where you would of like to spend more time? Interviewee1: I really liked Destiny’s station, ‘cause actually she was like, she was like an inspiration for real. She showed you that girls could do anything a man could do clearly. Interviewer 1: Were there anything at the program that made you think that you would like to learn more about STEM? Interviewee 2: When I was there, I like when they were teaching you about the insides of the car. I wanted to learn more about the engines and stuff. It seemed cool. Interviewer 1 & 2: Yeah Interviewer 1: What was cool about it? Interviewee1: How the engine worked, it is not usually how a regular car engine, it had more things to do with the engine. Interviewer 2: Have you seen an engine before. Are you familiar at all with seeing how they work or operate? Interviewee1: Oh Yeah Interviewer 1: Were there anything from the RIR day that made you want to make you learn more or becoming a Scientist or Engineer or something like that. Mechanic or anything. Interviewer 1: Or anything from that day that want you to pursue Specialty Centers that this county has to offer? Interviewee1: Yessss, yeah! Interviewer 3: Like what? Interviewee 1: I already know I kind of want to go to a Tech Center for Nurse Aid Interviewee2: I already know what Tech Center I want to go to. Interviewer 3: What Tech center Interviewee1: Nurse Aid RIR STEM EDUCATION DAY PROGRAM EVALUATION 185 Interviewer 3: What about you Interviewee 3: Hotel and Catering Interviewer 1: Definitely Science there. Alright. Interviewee1: He said something about animals when I went to the statin they used the cloths when they had the oil spill in the Gulf of Mexico and they used the cloth to get the oil off the animals and how they had to have a certain cloth that wouldn’t hurt the animals when they were trying to help them. That was cool. Interviewer 1: Thanks, you guys are great. Interviewer 1: What do you think would make that program better? Interviewee1: It was like, when I was there, because we were in middle school and so all the high school boys were like they in middle school and some of the boys that were here, they were trying to talk to us. I mean but, it was cool. They were like wandering off and I was like what are you doing and some of the kids in our group got mixed with other middle school kids and they paid attention if they wanted to and if they didn’t, they just kind of talked. We were just like. Interviewer 1: What about the stations we had there? What do you think will make them better or do you think if we cut some out. Interviewee1: Some of them could have had less talking. Interviewee 2: It was like this older guy, and he was just talking, talking and I didn’t understand what he was talking about, I was just like oh, ok. I was like give me a hug. Interviewer 1: More hands on, more to the point Interviewee 1: More to the point, with those charts and stuff, we were just like, oh! Interviewee 2: It was one station I like because she had cookies. Interviewee 3: One station had candy Interviewee 1: We can add more candy and less talking at the beginning thing because I was about to go to sleep. Interviewer 3: So less talking at the beginning. Interviewee 2: Less talking all together. Interviewer 3: So do you want to do more hands on, more demonstrations RIR STEM EDUCATION DAY PROGRAM EVALUATION 186 Interviewee 1: yes Interviewer 3: Ok Interviewee1: More touch touchy, less talky-talky Interviewer 3: More touch touchy, less talky-talky Interviewer1: That’s perfect, that’s going to be the title of our program Interviewee 1: Last question, what stations made the biggest impact on you? Interviewee 1: Destiny’s Interviewer 1: Destiny’s, why? Interviewee 2: I already said why. She was an inspiration for young ladies all around the world Interviewee 1: Anybody else Interviewee 1: I didn’t get to go to her station Interviewer 3: But you wished you had the opportunity? Interviewee 2: I really wanted to go to her station. We met her afterwards. She came to Fairfield a couple of times. She came to Fairfield for the job fair and then she came here for the pep rally. Interviewee 1 & 2: yeah she came here for the pep rally Interviewer 3: What did you like about going to RIR day? Interviewee 3: You mean the racing track? Interviewer 3: yeah Interviewee3: Seeing the racing cars, I guess. Interviewer 3: Did it inspire you to want to participate in Math and Science more? Interviewee 3: A little Interviewer3: A little Interviewer 1: Did anyone try the simulator Interviewee 1: I tried to get on it but all the boys were hogging it. Interviewee 2: They should have had a sign that said Girls only. They was just hogging. RIR STEM EDUCATION DAY PROGRAM EVALUATION 187 Interviewee 1: I think there was any girl that got to be on it. Interviewer 3: So you think the boys were taking over? Interviewee 1: it was like we would go to another station and the people was staring at us, like you not going anywhere but the boys were just like spread out. Interviewee 1: I was they gone why can’t we go. Interviewee 2: And we also got in trouble because they were like why you can’t keep your group together. Interviewee 1: My teacher she was like all over the place, I think she worked for the place or she was volunteering. I was like, where are you. Interviewer 3: So you think it should be more structure. So you have an opportunity to visit more stations? Interviewee 1, 2, 3: Yeah Interviewee1: Not more structure, but if you wasn’t interested in one place, you can like go to another. Interviewer 3: So if you think they had fewer stations, and you were able to go to all of the stations and the stations were interactive, less talky-talky, and more touch, touchy Interviewee 2: Some places we were at long and some stations we were at for 2 minutes. I was like , we at this station for 5 minutes and this one for 30 seconds, what happened to the other 4 ½ minutes. Interviewer 1: Yeah I understand Interviewee1: I walked away from the old man talking because somebody told me the lady had cookies. RIR STEM EDUCATION DAY PROGRAM EVALUATION 188 Appendix M - Focus Group Key Words and Phrases Evaluation Question 1 - To what degree did the RIR STEM Education Day program increase middle school students’ interest in STEM education? Teacher Focus Group Question 4 Middle School L Get into TSA/tech program Light goes on Middle School C Seeing people in the industry got them much more excited. Even more engaged with their projects Middle School H “Discussions” about career opportunities that deal with STEM “Discussions” about career opportunities that deal with Technology Available jobs deal with technology Middle School K N/A Middle School D What jobs can they get into What path would they need to go to get those jobs “How can I get into, if I wanted to become a race car driver? I want to do this, how can I get this? Middle School E That of the students that were leaning on the fence, they did go over there (engineering center) I know after the program, I know two students that were in my program that was walking the fence about whether to go that had been accepted at other specialty centers said hey I’m going to highland springs. I think I can say that was the icing the cake after they went to that program. There was one student in the class, to augment the power of the program. He was very negative, he is the most negative student I have ever taught in my 30 years of teaching and I found out later on that he went over there (to the engineering center). Middle School B They were able to see that you had physics involved. There was math involved, more than just banging metal together and nuts and bolts. There is a lot more that goes into it RIR STEM EDUCATION DAY PROGRAM EVALUATION 189 Student Focus Group Question 1 High School 2 (Males) The simulation was really cool Seeing all the tires and how they work, that was really cool Simulator was fun High School 2 (Females) It really wasn’t my thing High School 4 (Females) Meeting the young race car drivers The smaller cars I really liked the simulation Really like meeting destiny She was a female race car driver High School 4 (Males) The cars and how they are made It was fun because it was hands on High School 1 (Males) The speakers were talking about the application of math and science with the racing and stuff and how the fuels, and car, and tires work I like listening to the speakers as well. I think my favorite part of it was looking at the structure of the walls where the cars were racing I learned a bit more of how STEM applies to racing High School 3 (Males) I found all they physics and all that kind of stuff interesting High School 5 (Females) We never got really bored It was interesting how we got to actually do the hands on stuff and we got to try everything because some people don’t necessarily like things and once you try new things you get to know it and we got to do the hands on things. Evaluation Question 2 – To what degree did the RIR STEM Education Day program improve students’ understanding of STEM Education? Teacher Focus Group Question 2 Middle School L How important it is to create a good model Lots of math and engineering RIR STEM EDUCATION DAY PROGRAM EVALUATION 190 They don’t teach it like this in our classes and this program made it fun It was amazing because they started asking more questions themselves; they were more aware and more engaged It allowed me to make it more challenging for the kids When you go to an program like this they start answering some more questions by themselves and you can start challenging their minds They love hands-on Middle School C Gives the kids an opportunity to see real world applications of what we are learning This is right in their face how they can use these skills beyond the classroom Middle School H Two classroom activities done beforehand that leads up to watching the real thing…so we talk about roll cages, we talk about force, impact, aerodynamics Simple enough to get the concepts across but at the middle school/high school level Middle School K Employability skills Job opportunities Middle School D To see the real world Everything we do relates to the racing industry Working with their hands Middle School E They came and said, “we now understand more about what you are talking about because that’s what we saw.” We talked in class about gender misperceptions in tech. ed. and CTE. There‘s still that 250 year old Horace Mann philosophy that girls need to sew and guys need to cut wood and do things like that. We’re bringing down those barriers and the best thing for that both ethnically and gender wise was when they saw that young girl Just seeing her there, it just knocked down so many walls A lot of our 32 challenges that we have all collaborate around the same things, same ideas We talked about how some people are going to have high-tech jobs in robotics, computer chip manufacturing, NASA and the space program, and I always bring in NASCAR. That’s my second lesson in class and they got a chance to see that. They say…”I see what you are talking about.” Middle School B N/A RIR STEM EDUCATION DAY PROGRAM EVALUATION 191 Teacher Focus Group Question 3 Middle School L It was easier…it made sense…the kids started saying, “we learned that in class” It kept their attention They see that as fun Things are easier to do if there is some fun Middle School C Opportunity to feel and touch…how this affects the skills we are learning, these concepts, and how they are used in the real world. It was a big deal. You could see that they understood why we do things a certain way; why we go through a certain process and problem solving. I do think that it was advantageous I absolutely think their awareness increased. It was the most direct relationship between what we learn in class; it was very, very technical Middle School H Follow up activity as a follow up Had we had a pre-test and a post-test we probably would have seen a correlation It was a good opportunity to expand what we do in the classroom They are able to see the real thing and see these things beyond racing. Middle School K The stations were very educational This is where the real learning takes place. This was the heart of the matter as far as learning and STEM This is what school is all about…this is how it connects to racing and other things. If their attitudes changes just a little bit it might lead to better grades if there was more time. Middle School D Were more interested in those careers They wanted to know more about science and technology, more in depth about technology and science…they wanted to know what careers were related A lot of them now started to take math seriously See how math, science, and technology…all those things kind od interrelate, not only in racing but in the different facets of their lives Math is a huge component It’s really an excellent program as far as I’m concerned. More than anything it brought about awareness, and that’s what the kids need especially over here on the East end This brought awareness more than anything else. A lot of that was already incorporated I really loved it RIR STEM EDUCATION DAY PROGRAM EVALUATION 192 Middle School E The teacher is right…he does know what he is talking about. For the kids to see the trip and the experience validated all the points that we have been talking about in class. Kids need to see a hands-on; they really need to see a visual. They need to have a handson experience and they got that. Middle School B They were able to see that you had physics involved. There was math involved, more than just banging metal together and nuts and bolts. There is a lot more that goes into it. What we learned at the raceway, we were able to tie into the classroom. Student Focus Group Question 1 High School 2 (Males) The simulation was really cool Seeing all the tires and how they work, that was really cool Simulator was fun High School 2 (Females) It really wasn’t my thing High School 4 (Females) Meeting the young race car drivers The smaller cars I really liked the simulation Really like meeting destiny She was a female race car driver High School 4 (Males) The cars and how they are made It was fun because it was hands on High School 1 (Males) The speakers were talking about the application of math and science with the racing and stuff and how the fuels, and car, and tires work I like listening to the speakers as well. I think my favorite part of it was looking at the structure of the walls where the cars were racing I learned a bit more of how STEM applies to racing High School 3 (Males) I found all they physics and all that kind of stuff interesting High School 5 (Females) We never got really bored RIR STEM EDUCATION DAY PROGRAM EVALUATION 193 It was interesting how we got to actually do the hands on stuff and we got to try everything because some people don’t necessarily like things and once you try new things you get to know it and we got to do the hands on things. Evaluation Question 3 – To what degree did the RIR STEM Education Day program increase students’ interest in STEM educational opportunities? Teacher Focus Group Question 1 Middle School L My kids thought it was great It gave them a feeling that they can do this Made it more exciting for them Motivated my kids in so many ways Showed my kids they can do this The design process was awesome Jump started a lot of things The small cars had the most impact They can have fun learning this stuff They could see that what they were doing in the classroom could lead to what was going on at the program. Opened their minds Middle School C They all had a good time They all talked about it for weeks It was more of a direct engineering relationship Direct technical application really got the kids thinking about…I understand how this works now Middle School H The kids really liked it Was great for the kids They were excited The teachers will be able to better prepare the students for the trip…I can push more activities that lead up to it. Middle School K We did not have one negative comment and we had close to 50 kids Was the best trip they had ever been on It was extremely positive I heard from other teachers that the kids really had a good time because they were talking about it with other students Definitely had a cool factor to it RIR STEM EDUCATION DAY PROGRAM EVALUATION 194 Middle School D They were very excited They saw that they had greater possibilities outside of basketball They were able to get into another area, something that would be more interesting to them The students I talked to wanted to come back; they wanted more Wanted to know when they could do this again They were telling other students They really loved it They really enjoyed it They didn’t know the industry was as big as it was They didn’t know about the technology One kid talked about the fire suit and how much technology went into creating the fire suit Didn’t know about the racing industry in general and they wanted to know more information They didn’t know that blacks were actually into racing Until they actually went in and saw; the young lady really helped Seeing that she was young, she was African-American, and she was a female; the girls were like Ahhhh He is really interested in getting into, not necessarily the racing itself but the industry Middle School E The kids were really excited I only had 3-4 females but they were definitely interested and they learned some things It piqued their curiosity; brought home some things we had talked about in class Gave them a different focus Middle School B The kids were excited All of them are much more aware about what goes on in racing Raised the awareness All of them came back more aware of what opportunities and possibilities they have They saw the girl from Highland Springs and they said that they could do something like that Teacher Focus Group Question 4 Middle School L Get into TSA/tech program Light goes on Middle School C Seeing people in the industry got them much more excited. Even more engaged with their projects RIR STEM EDUCATION DAY PROGRAM EVALUATION 195 Middle School H “Discussions” about career opportunities that deal with STEM “Discussions” about career opportunities that deal with Technology Available jobs deal with technology Middle School K N/A Middle School D What jobs can they get into What path would they need to go to get those jobs “How can I get into, if I wanted to become a race car driver? I want to do this, how can I get this? Middle School E That of the students that were leaning on the fence, they did go over there (engineering center) I know after the program, I know two students that were in my program that was walking the fence about whether to go that had been accepted at other specialty centers said hey I’m going to highland springs. I think I can say that was the icing the cake after they went to that program. There was one student in the class, to augment the power of the program. He was very negative, he is the most negative student I have ever taught in my 30 years of teaching and I found out later on that he went over there (to the engineering center). Middle School B They were able to see that you had physics involved. There was math involved, more than just banging metal together and nuts and bolts. There is a lot more that goes into it Student Focus Group Question 3 High School 2 (Males) The simulator The three things I remember were the simulator, the guy talking about marketing or promoting NASCAR or something like that, and the tire one Demonstrated how the oil filtration works How they have the thickness that prprogramed like the rubber from getting all over the track. I think that was interesting. High School 2 (Females) We got in the cars The simulator We got in the cars and that made we think about how it works High School 4 (Females) RIR STEM EDUCATION DAY PROGRAM EVALUATION 196 The simulator The ones with Destiny and the ones where we were looking at the cars They had the cut-out cars and they had the full cars Looking at the cars made me want to learn how they are built. I always want to take a car apart and put it back together I want to be able to fix it if something happens to it. High School 4 (Males) The simulator The safer-wall was pretty cool, too. Arena racing and destiny I’m going to be a computer engineer High School 1 (Males) It did kind of help me think about getting a STEM degree. They talked about how smart the drivers are and how they have all these degrees in technology and engineering I’ve always wanted to be an engineer I thought it was interesting. I joined the TSA in 6th grade and I’m currently a chapter officer here at my school. It gave me an interest in different types of fields, and this was one of the more mechanical fields of engineering and it gave me a good idea in what field I want to go in Provide some interest and exposure High School 3 (Males) That simulator…it would have cool to have gone to that It was kind of the things that got me thinking again back to the physics with why they don’t have doors and stuff, kind of thinking how air would get trapped in there and stuff like that, the things that kind of make me think about it for a little bit. The engines, like fixing them, preparing them and stuff, that’s interesting if you want that kind of job I want to be a mathematician It is interesting. It is a fun thing to do. High School 5 (Females) I really liked Destiney’s station because actually she was like an inspiration for real. They were teaching you about the insides of the car. I wanted to learn more about engines and stuff. It seemed cool. I already know that I kind of want to go to a tech. center. He said something about animals when I went to the station they used cloths when they had the oil spill in the Gulf of Mexico and they used the cloth to get the oil off of the animals and how they had to have a certain cloth that wouldn’t hurt the animals when they were trying to help them. That was cool. RIR STEM EDUCATION DAY PROGRAM EVALUATION 197 Evaluation Question 4 - To what degree did the RIR STEM Education Day program increase students’ interest in STEM-related careers? Teacher Focus Group Question 5 Middle School L It sparked their mind and their attention They want to go to mechanics school…they’re on it but they just need to find out what to do to prepare themselves…all of our talk is about careers Middle School C They were really excited A lot of really neat stuff they got to see They were really excited. There is a lot to be said for being able to put your hands on these things…having something right in front of you instead of pictures and videos and me describing it. Middle School H We talked about careers more from an educational standpoint and used this program in my discussions about career opportunities…how many of these jobs deal with STEM, deal with technology 80% of available jobs deal with technology Middle School K They really enjoyed it Middle School D I had two young men that wanted to go to the tech. Maybe we should open to 6th or 7th graders that way we can kind of gage how they are from one year to the next, instead of just 8th graders Middle School E I heard a lot of chatter back and forth between the students about it Middle School B More aware that there is more involved The fact that it’s not just driving the car Its stuff in making the track There’s the design architecture of the track, the stadium, the business side of it Teacher Focus Group Question 6 Middle School L Once my girls saw destiny you guys can forget it! They are top of it…they love it. RIR STEM EDUCATION DAY PROGRAM EVALUATION 198 We talk about ethnicity. We try to figure out how to get them involved but the majority wants to know the salaries…they ask if blacks are involved with racing. They guys said they want to build this or build that but the girls wanted to get in the car or on the bike…I want to race! Middle School C The girls that I took were surprised to see the girl from Highland Springs with the motorcycle It opened their eyes that they can do this too Middle School H 90% boys…that’s one of the biggest problems we have as CTE when it gets into the technology aspect. I have 60 boys and 1 girl I’ve had girls in my class before but most of them were from other countries. They took my class because they wanted to be an engineer The 1 girl I have this year likes working with her hands. She does not want to be an engineer. It’s a STEM activity but it’s male…how do you get girls to your program if they are not in our classes? What we need to do is offer a class for all girls…they are intimidated by a traditional allboys class…it would be a whole new ballgame…I think we would see immediate results where we would see more girls going into technology curriculum, you know, the STEM fields. Middle School K There were quite a few females there and I thought they had a good experience They saw that girl there. We need more of that…the kids need to see that they can do these things. Middle School D It opened it up to them They always just saw Caucasians So this opened it up to them, that it’s not just a career, I don’t know why they thought they couldn’t get into it. Blacks don’t do that Having the young lady who graduated, who was an alumni from Fairfield Middle School and Highland Springs High School That had a real big impact on the kids, especially the females If you expose them to the, you, greater career choices, it just makes you more excited about your future. She just had that drive and that’s what kids need to see. Kids feed off of other kids. So for us I think that it was very good RIR STEM EDUCATION DAY PROGRAM EVALUATION 199 Middle School E Destiny herself broke down so many double barriers again by being a female and being and African-American and there were so many African-Americans students in the group and I thing they were impressed Middle School B We haven’t had a lot of females in our programs We have been getting more. I think it’s because of engineering. How technology is associated with engineering Girls are more driven at this age Student Focus Group Question 3 High School 2 (Males) The simulator The three things I remember were the simulator, the guy talking about marketing or promoting NASCAR or something like that, and the tire one Demonstrated how the oil filtration works How they have the thickness that programed like the rubber from getting all over the track. I think that was interesting. High School 2 (Females) We got in the cars The simulator We got in the cars and that made we think about how it works High School 4 (Females) The simulator The ones with Destiny and the ones where we were looking at the cars They had the cut-out cars and they had the full cars Looking at the cars made me want to learn how they are built. I always want to take a car apart and put it back together I want to be able to fix it if something happens to it. High School 4 (Males) The simulator The safer-wall was pretty cool, too. Arena racing and destiny I’m going to be a computer engineer High School 1 (Males) It did kind of help me think about getting a STEM degree. They talked about how smart the drivers are and how they have all these degrees in technology and engineering I’ve always wanted to be an engineer RIR STEM EDUCATION DAY PROGRAM EVALUATION 200 I thought it was interesting. I joined the TSA in 6th grade and I’m currently a chapter officer here at my school. It gave me an interest in different types of fields, and this was one of the more mechanical fields of engineering and it gave me a good idea in what field I want to go in Provide some interest and exposure High School 3 (Males) That simulator…it would have cool to have gone to that It was kind of the things that got me thinking again back to the physics with why they don’t have doors and stuff, kind of thinking how air would get trapped in there and stuff like that, the things that kind of make me think about it for a little bit. The engines, like fixing them, preparing them and stuff, that’s interesting if you want that kind of job I want to be a mathematician It is interesting. It is a fun thing to do. High School 5 (Females) I really liked Destiney’s station because actually she was like an inspiration for real. They were teaching you about the insides of the car. I wanted to learn more about engines and stuff. It seemed cool. I already know that I kind of want to go to a tech. center. He said something about animals when I went to the station they used cloths when they had the oil spill in the Gulf of Mexico and they used the cloth to get the oil off of the animals and how they had to have a certain cloth that wouldn’t hurt the animals when they were trying to help them. That was cool. Evaluation Question 5 – Are there differences in STEM interest and understanding based on gender and ethnicity regarding the RIR STEM Education Day program? Teacher Focus Group Question 6 Middle School L Once my girls saw destiny you guys can forget it! They are top of it…they love it. We talk about ethnicity. We try to figure out how to get them involved but the majority wants to know the salaries…they ask if blacks are involved with racing. They guys said they want to build this or build that but the girls wanted to get in the car or on the bike…I want to race! Middle School C The girls that I took were surprised to see the girl from Highland Springs with the motorcycle It opened their eyes that they can do this too RIR STEM EDUCATION DAY PROGRAM EVALUATION 201 Middle School H 90% boys…that’s one of the biggest problems we have as CTE when it gets into the technology aspect. I have 60 boys and 1 girl I’ve had girls in my class before but most of them were from other countries. They took my class because they wanted to be an engineer The 1 girl I have this year likes working with her hands. She does not want to be an engineer. It’s a STEM activity but it’s male…how do you get girls to your program if they are not in our classes? What we need to do is offer a class for all girls…they are intimidated by a traditional allboys class…it would be a whole new ballgame…I think we would see immediate results where we would see more girls going into technology curriculum, you know, the STEM fields. Middle School K There were quite a few females there and I thought they had a good experience They saw that girl there. We need more of that…the kids need to see that they can do these things. Middle School D It opened it up to them They always just saw Caucasians So this opened it up to them, that it’s not just a career, I don’t know why they thought they couldn’t get into it. Blacks don’t do that Having the young lady who graduated, who was an alumni from Fairfield Middle School and Highland Springs High School That had a real big impact on the kids, especially the females If you expose them to the, you, greater career choices, it just makes you more excited about your future. She just had that drive and that’s what kids need to see. Kids feed off of other kids. So for us I think that it was very good Middle School E Destiny herself broke down so many double barriers again by being a female and being and African-American and there were so many African-Americans students in the group and I thing they were impressed Middle School B We haven’t had a lot of females in our programs We have been getting more. I think it’s because of engineering. How technology is associated with engineering Girls are more driven at this age RIR STEM EDUCATION DAY PROGRAM EVALUATION 202 Student Focus Group Question 5 High School 2 (Males) Probably seeing like the cut-out car, that again showed me how everything works together for STEM. High School 2 (Females) Because I got to get inside it. I don’t really like science, technology, engineering. High School 4 (Males) The one with the cut-away car. How it was made and the design. High School 4 (Females) Destiny. The fact that she was a woman and she knew how to do all that stuff just like the men. Cars that you can get in and look at closer High School 1(Males) The station where you got to see inside the car and see the cut-outs The cut-outs interested me The station that explains how the walls were made. We could use those types of engineering properties on other things, like bridges and such When I was there it gave me good ideas, what it looks like. High School 3 (Males) It wasn’t just kind of the cars going around in circles any more, you kind of saw some of the physics and understood some of the concepts. Watching NASCAR was like watching the future. High School 5 (Females) Destiny. She was an inspiration for young ladies all around the world. If you wasn’t interested in one place, you could like go to another Less talky, talky and more touchy, touchy RIR STEM EDUCATION DAY PROGRAM EVALUATION 203 Appendix N - How Data Relates to Evaluation Questions Evaluation Questions 1. To what degree did the RIR STEM Education Day program increase middle school students’ interest in STEM education Data Collected to Answer Evaluation Questions Survey Question 4 Do you have an interest in learning more about the technical education programs offered within the local public school district? Teacher Focus Group Question 4 What did students say or do following the RIR STEM Education Day program that indicated an increased interest in STEM educational opportunities? Student Focus Group Question 1 Think back to RIR STEM Education Day. What did you think about that day? How Data will be Analyzed - Analyze and interpret student survey response percentages to find answers, or patterns; - Analyze focus group interview transcriptions to identify common key words and phrases that address the theme of interest; Student Comments Teacher Feedback - Analyze student comments and teacher feedback to identify information relative to the evaluation question RIR STEM EDUCATION DAY PROGRAM EVALUATION Evaluation Questions 2. To what degree did the RIR STEM Education Day program improve students’ understanding of STEM education? 204 Data Collected to Answer Evaluation Questions Survey Question 3 Do you better understand the science, technology, engineering and math components of racing as a result of this program? Teacher Focus Group Question 2 In what ways were you able to connect your class content to what students experienced that day? Teacher Focus Group Question 3 What changes did you see in how they talked about STEM following the RIR STEM Education Day program? Did their grades improve? How Data will be Analyzed - Analyze and interpret student survey response percentages to find answers, or patterns; - Analyze focus group interview transcriptions to identify key words and phrases that address the theme of understanding; Student Focus Group Question 1 Think back to RIR STEM Education Day. What did you think about that day? Student Comments Teacher Feedback - Analyze student comments and teacher feedback to identify information relative to the evaluation question RIR STEM EDUCATION DAY PROGRAM EVALUATION Evaluation Questions 3. To what degree did the RIR STEM Education Day program increase students’ interest in STEM educational opportunities? 205 Data Collected to Answer Evaluation Questions Survey Question 4 Do you have an interest in learning more about the technical education programs offered within the local public school district? Teacher Focus Group Question 1 What did the students think about the RIR STEM Education Day? What did they say afterwards? Teacher Focus Group Question 4 What did students say or do following the RIR STEM Education Day program that indicated an increased interest in STEM educational opportunities? How Data will be Analyzed - Analyze and interpret student survey response percentages to find answers, or patterns; - Analyze focus group interview transcriptions to identify key words and phrases that address the theme of educational opportunities; Teacher Focus Group Question 6 In what ways did the RIR STEM Education Day program promote female and minority students’ interest in STEM-related education and careers? Student Focus Group Question 3 Were there stations where you would have liked to have spent more time? Was there anything you did that made think that you would like to learn more about STEM? Student Comments Teacher Feedback - Analyze student comments and teacher feedback to identify information relative to the evaluation question RIR STEM EDUCATION DAY PROGRAM EVALUATION Evaluation Questions 4. To what degree did the RIR STEM Education Day program increase students’ interest in STEMrelated careers? 206 Data Collected to Answer Evaluation Questions Survey Question 2 From these experiences, have you developed an interest/awareness of career opportunities within the motorsports industry? Teacher Focus Group Question 5 What did students say or do following the RIR STEM Education Day program that indicated an increased interest in STEM career opportunities? Teacher Focus Group Question 6 In what ways did the RIR STEM Education Day program promote female and minority students’ interest in STEM-related education and careers? How Data will be Analyzed - Analyze and interpret student survey response percentages to find answers, or patterns; - Analyze focus group interview transcriptions to identify key words and phrases that address the theme of career opportunities; Student Focus Group Question 3 Were there stations where you would have liked to have spent more time? Was there anything you did that made you think that you would like to learn more about STEM? Student Comments Teacher Feedback - Analyze student comments and teacher feedback to identify information relative to the evaluation question RIR STEM EDUCATION DAY PROGRAM EVALUATION Evaluation Questions 5. Are there differences in STEM interest and understanding based on gender and ethnicity regarding the RIR STEM Education Day program? 207 Data Collected to Answer Evaluation Questions Survey Question 3 Do you better understand the science, technology, engineering and math components of racing as a result of this program? Teacher Focus Group Question 6 In what ways did the RIR STEM Education Day program promote female and minority students’ interest in STEM-related education and careers? Student Focus Group Question 5 What station made the biggest impact on you? How Data will be Analyzed - Analyze and interpret student survey response percentages to find answers, or patterns; - Analyze focus group interview transcriptions to identify key words and phrases that address the theme of gender and ethnicity; RIR STEM EDUCATION DAY PROGRAM EVALUATION 208 Appendix O - RIR STEM Education Day Program Evaluation PROTOCOL Below is the protocol for student and teacher focus groups. The teacher focus group consists of seven teachers who participated in the RIR STEM Education day program at Richmond International Raceway on Thursday, April 25, 2013. The student focus group consists of 20 Career and Technical Education students who participated in the RIR STEM Education day program. These 20 students will be separated into two separate groups; one male group to include minority students, and one female group to include minority students. The focus of this evaluation is to determine to what degree the April 25, 2013 RIR STEM Education Day program increased interest and understanding of STEM, STEM educational and career opportunities, and if there are differences in STEM interest and understanding based on gender and ethnicity regarding the RIR STEM Education Day program. These focus groups will be audio-recorded and conducted by a primary facilitator and a secondary facilitator. Each focus group will last approximately 30 minutes. STUDENT FOCUS GROUP (Permission and consent forms are obtained before focus group interviews begin) Primary Facilitator Good afternoon/evening. Thank you for taking the time to join our discussion and evaluation of the RIR STEM Education Day program. My name is ____Prime Facilitator____, and this is __Secondary Facilitator__. We are conducting a program evaluation on the RIR STEM Education Day program that you attended on April 25, 2013 at the Richmond International Raceway. In particular, we hope to learn if the RIR STEM Education Day program increased interest and understanding of STEM, STEM educational and career opportunities, and if there are differences in STEM interest and understanding based on gender and ethnicity regarding the RIR STEM Education Day program. We want to talk with you about your experiences at the RIR STEM Education Day program. We will be asking you about what activities you thought were interesting and if you felt that any of the activities helped you better understand things that you may have gone over in your classrooms. We will also be asking you whether the RIR STEM Education Day program increased your interest in STEM-related education and/or careers. RIR STEM EDUCATION DAY PROGRAM EVALUATION 209 Before we begin, let me suggest some things to make our discussion more productive. Because we’ll be recording for an accurate record, it is important that you speak up and that you only speak one at a time. We don’t want to miss any of your comments. We’ll only use first names here and no reports will link what you say to your name or school. In this way, we will maintain your confidentiality. In addition, we ask that you also respect the confidentiality of everyone here. Please don’t repeat who said what when you leave this room. During the next thirty minutes that we’ll be here, I will ask you questions, and I will listen to what you have to say. I will not participate in the discussion. So please, feel free to respond to each other and to speak directly to others in the group. We want to hear from all of you. We’re interested in everyone’s viewpoints. I may sometimes act as a traffic cop by encouraging someone who has been quiet to talk, or by asking someone to hold off for a few minutes. If it is OK with you, we will turn on the recorder and start now. This student focus group is being conducted for the RIR STEM Education Day on [DATE] by __Primary Facilitator__ and __Secondary Facilitator__. The tape ID number is _____________. START TIME - _____________. I. Let’s begin with introductions. A. II. the Please tell us what school you attended in the 8th grade. Now that we know a little about you, I’d like you to think back to when you attended the RIR STEM Education Day program. Please look around the room at pictures taken on day of the program back on April 25, 2013. B. Focus Group Questions: 1. Think back to RIR Education Day. What did you find interesting about that day? Possible follow up: It was fun how? What made it boring? In what ways did it help you understand STEM? 2. What did you do that you really liked? Possible follow up: What did you like about the hands-on activities? Did that make you better understand STEM? Did any activities relate back to classroom material? 3. Were there stations where you would have liked to have spent more time? Was there anything you did that made you think that you would like to learn more about STEM? Possible follow up: Would you like to be a scientist/engineer, etc. one day? 4. What do you think would have made it better? Possible follow up: Was there anything you would leave out? 5. What station made the biggest impact on you? Possible follow up: Why do you remember this station the most? RIR STEM EDUCATION DAY PROGRAM EVALUATION 210 POSSIBLE SUMMARIZATION: It looks like there were (quite a few/some) positive/negative features of the RIR STEM Education Day program that interested you. PROMPTS Thank you again for taking the time to participate in this discussion. END TIME - _____________. RIR STEM EDUCATION DAY PROGRAM EVALUATION 211 Appendix P – Teacher Focus Group Script (Consent forms are obtained before the focus group interview begins) Primary Facilitator Good afternoon/evening. Thank you for taking the time to join our discussion and evaluation of the RIR STEM Education Day program. My name is ____Prime Facilitator____, and this is __Secondary Facilitator__. We are conducting a program evaluation on the RIR STEM Education Day program that you attended on April 25, 2013 at the Richmond International Raceway. In particular, we hope to learn if the RIR STEM Education Day program increased interest and understanding of STEM, STEM educational and career opportunities, and if there are differences in STEM interest and understanding based on gender and ethnicity regarding the RIR STEM Education Day program among your students. We want to talk with you about your students’ experiences at the RIR STEM Education Day program. We will be asking you about what activities you thought were interesting and if you felt that any of the activities helped your students better understand things that you may have gone over in your classrooms. We will also be asking you whether the RIR STEM Education Day program increased students’ interest in STEM-related education and/or careers. Before we begin, let me suggest some things to make our discussion more productive. Because we’ll be recording for an accurate record, it is important that you speak up and that you only speak one at a time. We don’t want to miss any of your comments. We’ll only use first names here and no reports will link what you say to your name or school. In this way, we will maintain your confidentiality. In addition, we ask that you also respect the confidentiality of everyone here. Please don’t repeat who said what when you leave this room. During the next thirty minutes that we’ll be here, I will ask you questions, and I will listen to what you have to say. I will not participate in the discussion. So please, feel free to respond to each other and to speak directly to others in the group. We want to hear from all of you. We’re interested in everyone’s viewpoints. I may sometimes act as a traffic cop by encouraging someone who has been quiet to talk, or by asking someone to hold off for a few minutes. If it is OK with you, we will turn on the recorder and start now. This teacher focus group is being conducted for the RIR STEM Education Day on [DATE] by __Primary Facilitator__ and __Secondary Facilitator__. The tape ID number is _____________. START TIME - _____________. I. Let’s begin with introductions. A. Please tell us your first name and what school you represent. RIR STEM EDUCATION DAY PROGRAM EVALUATION 212 II. Now that we know a little about you, I’d like you to think back to when you attended the RIR STEM Education Day program. Please look around the room at pictures taken on the day of the program back on April 25, 2013. B. Focus Group Questions: 1. What did the students think about RIR STEM Education Day? What did they say about it afterwards? 2. In what ways were you able to connect your class content to what students experienced that day? 3. What changes did you see in how students talked about STEM following the RIR STEM Education Day? Did their grades improve? 4. What did students say or do following the RIR STEM Education Day program that indicated an increased interest in STEM educational opportunities? 5. What did students say or do following the RIR STEM Education Day program that indicated an increased interest in STEM career opportunities? 6. In what ways did the RIR STEM Education Day program promote female and minority students’ interest in STEM-related education and careers? POSSIBLE SUMMARIZATION: It looks like there were (quite a few/some) positive features of the RIR STEM Education Day program that interested you. PROMPTS Thank you again for taking the time to participate in this discussion. END TIME - _____________. RIR STEM EDUCATION DAY PROGRAM EVALUATION 213 Appendix Q – Program Evaluation Subject and Consent Form TITLE: RIR STEM Education Day Program Evaluation PURPOSE OF THIS EVALUATION The focus of this program evaluation is to determine to what degree the RIR STEM Education Day program increased interest and understanding of STEM, STEM educational and career opportunities, and if there are differences in STEM interest and understanding based on gender and ethnicity regarding the RIR STEM Education Day program. You are being asked to participate in this program evaluation because you were either a student or teacher in the Henrico County Public Schools Career and Technical Education (CTE) 8403 class who participated in the RIR STEM Education Day program as part of an educational field trip at the Richmond International Raceway Complex on April 25, 2013. DESCRIPTION OF THE EVALUATION AND YOUR INVOLVEMENT If you decide to participate in this program evaluation, you will be asked to sign this letter of consent form after you have had all your questions answered and understood what will happen to you. Student participants in this evaluation will participate in focus groups of approximately eight to ten people. The duration of the focus group will be approximately 30 minutes in time. Teacher participants in this evaluation will participate in a focus group of seven people. The duration of this focus group will be approximately 30 minutes in time. The focus group questions are very open ended and will try to assess to what degree the RIR STEM Education Day program was effective in increasing interest in STEM, improving understanding of STEM, increasing interest in STEM-related education and careers, and if there were any differences in STEM interest and understanding based on gender and ethnicity. The focus groups will be audio-recorded but will not reveal the identity of any focus group participants. Findings will be available to you at the conclusion of the program evaluation. RISKS AND DISCOMFORTS The program evaluation poses minimal risk to you. At the beginning of the focus group, participants are assured that the focus group facilitators will not identify you. You will not have to talk about subjects you do not want to talk about and you may leave the group at any time without penalty. BENEFITS TO YOU AND OTHERS You may not get any direct benefit from this program evaluation but the information learned from participants in this program evaluation may help design improved STEM education programs for future students. RIR STEM EDUCATION DAY PROGRAM EVALUATION 214 Appendix R – Program Evaluation Parental Permission Form TITLE: RIR STEM Education Day Program Evaluation PURPOSE OF THIS EVALUATION A team of doctoral students from Virginia Commonwealth University is interested in finding to what degree the Richmond International Raceway (RIR) Science, Technology, Engineering, and Mathematics (STEM) Education Day program increased interest and understanding of STEM, STEM educational and career opportunities, and if there are differences in STEM interest and understanding based on gender and ethnicity regarding the RIR STEM Education Day program. We are asking for your child to participate in this program evaluation because he or she was an eighth grade student in the Career and Technical Education (CTE) 8403 class and who participated in the RIR STEM Education Day program as part of an educational field trip at the Richmond International Raceway Complex on April 25, 2013. Student participants in this evaluation will be asked to participate in focus groups with approximately eight to ten other students. The duration of the focus group will be approximately 30 minutes in time. The group of students will be asked five open ended questions that will address their experience at the RIR STEM Education Day event. Their answers will help the program evaluators assess to what degree the RIR STEM Education Day program was effective in increasing students’ interest in STEM, improving understanding of STEM, increasing interest in STEM-related education and careers, and if there were any differences in STEM interest and understanding based on gender and ethnicity. The focus groups will be audio-recorded but will not reveal the identity of any focus group participants. We are working closely with the school division’s CTE administrators on this program evaluation and will identify a time that will minimize the loss of instructional time from the classroom. The intent is to conduct the student focus group interviews in the school while students are at lunch. The evaluators will bring food for the students to eat while the interview is conducted. RISKS AND DISCOMFORTS The program evaluation poses no risk to your child. At the beginning of the focus group, participants are assured by the focus group facilitators that they will not identify them. Students will not have to talk about subjects they do not want to talk about and they may leave the group at any time without penalty. Participation is completely voluntary. BENEFITS TO YOU AND OTHERS Students may not get any direct benefit from this program evaluation but the information learned from student participants in this program evaluation may help design improved STEM education programs for future students. QUESTIONS OR CONCERNS If you have any question or concerns regarding this evaluation, please contact Ms. Ingrid Grant at (804) 652-3676, or grantig@vcu.edu. Please return the signed portion of this form in the self-addressed stamped envelope by Friday, December 13, 2013. --------------------------------------------------------------------------------------------------------------------------------------------------- Please check this box if you agree to let your child take part in the focus group interview. ___________________________ Student’s Name _________________________ Parents Signature _____________ Date RIR STEM EDUCATION DAY PROGRAM EVALUATION 215 Appendix S – Program Evaluation Student Assent Form TITLE: RIR STEM Education Day Program Evaluation PURPOSE OF THIS EVALUATION We are doing an evaluation about the field trip that you took to Richmond International Raceway (RIR) last April with your school. We want to learn what students thought went well and what could be improved. You are being asked to participate in this program evaluation focus group because you were a student who participated in the RIR Science, Technology, Engineering, and Mathematics (STEM) Education Day program as part of a field trip at the Richmond International Raceway on April 25, 2013. DESCRIPTION OF THE EVALUATION AND YOUR INVOLVEMENT If you decide to participate in this program evaluation focus group, you will be asked to sign and date this form after you have had all your questions answered. You will be in a group with approximately eight to ten other students and will talk with us for about 30 minutes. The questions we will ask you are only about what you think. There are no right or wrong answers. This is not a test. We will be recording you so that we can remember exactly what you said, but we will not identify you in any way. If you sign this form, it means that you want to participate in this evaluation. If you don’t want to participate, don’t sign the form. If you decide at any time that you no longer want to participate or if there is a question you do not want to answer, let us know. No one will be upset if you change your mind or if you don’t want to answer. Your answers may help make the “RIR Day” even better for other students. Please check this box if you agree to participate in this focus group interview. __________________________ ___ Signature ________________________ Print ___________ Date