Development, Dissemination, and Preimplementation Evaluation of Food Safety Educational Materials for Secondary Education Adrienne E.H. Shearer, O. Sue Snider, and Kalmia E. Kniel Abstract: With the persistence of microbiological foodborne illness and anticipated future shortage of scientists with agricultural and food science expertise in the United States, it is imperative to educate youth on microbiological food safety and enhance their awareness of opportunities to become engaged in finding solutions to food safety challenges. To help integrate food science education across the high school curriculum, new educational materials on microbiological food safety were developed and then disseminated to and evaluated by educators of secondary basic and applied sciences. The materials present food safety concepts in the context of foodborne illness outbreaks to introduce basic concepts of food microbiology, epidemiology, and food safety strategies as well as their broad impact on economics, communication, and regulations. The ready-to-implement educational materials support educational content standards and various learning styles and encourage critical thinking skills. The materials include a presentation on food microbiology and foodborne illness surveillance, case studies on foodborne illness outbreaks, a video on the laboratory investigation of foodborne illness, interactive web-based activities, and supporting materials for teachers and classroom display. Exposure to the materials in a 1-d workshop positively impacted educator familiarity with general microbiology, food safety strategies, regulatory requirements, and associated terminology as measured by a test administered prior to and after use of the materials. Teachers of biology, chemistry, family and consumer sciences, and related sciences rated the materials favorably on applicability, anticipated ease of implementation, and anticipated reception by students. Introduction United States compartmentally address life sciences (Natl. Science Education Standards, Natl. Research Council 1996), health and disease prevention (Natl. Health Education Content Standards; CDC 2007), and food safety (Agriscience Education – Food Science Technology; Delaware Dept. of Education 2007). However, not all high school students are exposed to microbiological food safety as life science concepts can be appropriately taught with a multitude of different systems and applications, and food science may be an elective course of study. Outside of the formal classroom, families of high school students may not serve as alternative sources of accurate food safety information as knowledge and practices among adults also generally appear to be inadequate or at least inconsistent (Harris and others 2006). High school students are also defining future personal education and career goals and should be aware of opportunities to utilize skills and interest in science to keep the food supply safe. An anticipated shortage of U.S. students with STEM (science, technology, engineering, and mathematics) expertise, including agricultural science, is a growing concern among many educators, industry experts, and policy makers. It is important to make high school students aware of MS 20121062 Submitted 8/2/2012, Accepted 12/19/2012. Authors are with the necessary aptitudes to pursue careers in food safety because Dept. of Animal and Food Sciences, Univ. of Delaware, 044 Townsend Hall, 531 South College Ave., Newark, DE 19716-2150, U.S.A. Direct inquiries to author the educational paths elected in high school affect opportunities for successful entry into baccalaureate programs and other careers Kniel (E-mail: kniel@udel.edu). postgraduation. Further, whether or not students eventually work The need to increase food safety knowledge among high school students is critical to improve safe food handling skills, cultivate the next generation of food safety professionals, and to enhance scientific civic literacy. All high school students are consumers of food, and consumers have an important role in protecting themselves from illness by using proper food handling techniques. Responsibilities increase post high school graduation as youth live with greater independence. Many youth are also involved in food preparation for others both in the home and when employed in food service, retail, or manufacturing; young adults aged 16 to 19 y represent 20% of the food service workforce in the United States (Bureau of Labor Statistics 2011). Studies have also concluded that young adults are generally more likely to practice risky food handling behaviors than other age groups (Patil and others 2005; McArthur and others 2007); and therefore, it is important that students learn the basis for safe food practices from legitimate sources. Written content standards for high school education in the 28 Journal of Food Science Education r Vol. 12, 2013 c 2013 Institute of Food Technologists® doi: 10.1111/1541-4329.12004 Food safety materials for secondary education . . . in a segment of the food industry, they can affect change through political and communication channels and should understand the issues before them to do so appropriately. Less than a quarter of the general populous of the United States has been characterized as having scientific civic literacy (Natl. Research Council 2011), and previous studies have illustrated disconnects between attitudes and practices of young adults as well as appreciation of the impact of individual behavior in relation to agriculture and other societal issues (Harmon and Maretzki 2006). Microbiological food safety is a topic of popular press during foodborne illness outbreaks and food product recalls. Some food safety education may occur as a result of outbreak events, as perceived risk of foodborne illness affects food purchasing and preparation behaviors (Ralston and others 2002; Harris and others 2006). The familiar news stories of foodborne illness outbreaks provide a platform for more complete and enduring food safety education in the academic setting. The utility of storytelling based on illness outbreaks for conveying food safety information to food handlers has previously been reported (Chapman and others 2011) and supports the importance of teaching rationale and consequence in an effort to engage and affect behavioral change. Compelling stories that provide a personal connection are memorable and can build a sense of community among learners and instructors (Abrahamson 1998). The use of storytelling (Abrahamson 1998; Lordley 2007) and case studies (Herreid 2005; DeSchryver and others 2007) to enhance science education in the academic setting has also received positive analysis in support of comprehension of facts, development of critical thinking and analytical skills, and enhancement of cognitive and emotional connections. The objectives of this study were to develop and disseminate educational materials that feature basic and applied microbial food safety concepts, utilization of mathematics in science, food safety impact on global societal issues including public health, communication, regulations and the economy, and the requisite aptitudes to pursue related career opportunities. This paper presents the insight of secondary educators for the development of food safety educational materials, the impact of these materials on educator food safety knowledge, and initial educator assessment of the utility of the materials for science instruction across the curriculum. Materials and Methods Development of educational materials Careful consideration was given to the necessary features of food safety educational materials intended for use across the high school science curriculum. We endeavored to develop materials with the following attributes: (1) engaging but with sensitivity to the seriousness of the topic, (2) utilization of techniques that provide for individuals with varied learning styles, (3) flexibility for application in courses of various subjects, grade levels, skill levels, and time available for dedication to the subject, (4) cost effective for development and implementation, (5) incorporation of a safe laboratory component without requirement for additional materials or facilities, (6) recognition of education content standards, (7) includes material that is both relevant and sustainable for years, (8) widely applicable throughout the nation and possibly beyond (9) factually based, while encouraging consideration of unresolved issues without undue sensation, and (10) ready to implement. The educational materials developed consisted of a presentation to provide a foundation of knowledge in food microbiology, case studies to engage students in problem solving, a video on laboratory detection of foodborne pathogens, interactive web-based activities, Available on-line through ift.org a classroom poster, and supporting materials including discussion questions and a glossary. The developed materials are described further in the “Results and Discussion” section. Insight of intended audience sought during materials development The input of regional secondary science educators and youth was sought throughout the development of the food safety educational materials. Four teachers were invited to serve on an advisory board based on previous professional relationships for outreach and curriculum development activities. Subject matters taught by board members included chemistry, biology, family and consumer sciences, and agriscience. Communication with board members was conducted in-person, via email, and by telephone approximately 3 times during a 9-mo period of materials development. Board members were presented with the overall concept and drafts of materials and were asked to provide input on quality, level of challenge, adaptability for varied student abilities and learning styles, compatibility with current curriculum, and classroom time investment. Advisory board members were under no obligation to participate in the workshop offered to disseminate materials or to use the materials in their classrooms. Teachers were provided a modest honorarium for their time and expertise at the conclusion of the development stage. College students (recent high school graduates, predominantly freshman and sophomores) of the Univ. of Delaware 200-level Foodborne Diseases course (approximately 60 students total over 3 classes) were given 1 of the case studies in partial and draft form to complete in groups. The students’ insight on novelty, difficulty, format, and interest was sought. In addition, high school students (personal contacts of the authors) were informally asked for insight on the appeal and level of challenge of various components of the educational materials during development. Dissemination of educational materials in workshop for secondary educators The educational materials were disseminated to the regional community of secondary science educators in a 1-d workshop held at the Univ. of Delaware College of Agriculture and Natural Resources. The interactive program was designed to introduce teachers to the materials, facilitate exchange among participants and with investigators, and to provide field insight on foodborne illness and their investigations through a keynote address by a representative of the US Public Health Service with the United States Food and Drug Administration, Center for Food Safety and Applied Nutrition, Office of Food Defense Communication and Emergency Response. The workshop format mirrored that of how students would ultimately use each of the materials whether as a listener, viewer, or in-group discussion. For case studies, participants worked through 2 case studies each in 4 groups of teachers of similar subjects (family and consumer sciences, biology, microbiology, and other physical sciences). The groups provided specific feedback on the cases at the conclusion of the group sessions. Although time was originally allotted for independent experimentation with the web games, the games had to be introduced to the participants as a demonstration due to time constraints because the active discussion surrounding the presentation ran longer than anticipated. Workshop participants were provided with all newly developed educational materials in hard copy in a binder and on a portable drive to allow flexibility for implementation. The web-based games were made available through a website for the workshop Vol. 12, 2013 r Journal of Food Science Education 29 Food safety materials for secondary education . . . Table 1–Methods to announce workshop. Media Direct Mail—approximately 1000 registration brochures Listserve announcementsa Website Full page advertisement Press releases (before/after workshop) Target audience Natl. Science Teachers Assoc. (NSTA) members biology/chemistry grades 11 and 12, within approximate radius of 2-h drive Family and Consumer Science teachers—all Del. high schools Agriscience teachers—all Del. high schools Agriscience teachers in Del. Science teachers in N.J. via content specialist Linked to the UD College of Agriculture and Natural Resources website Del. Teacher Center Workshop Brochure Offered to 32 print media sources in region including those with audience with agriculture interests a Responsiveness to requests to announce the workshop to constituent educators varied among state education content specialists. while instructions and answers were provided in hard copy and on the portable drive. Participants also received a laminated classroom poster of common foodborne disease agents, video on DVD, and a copy of the Intl. Assoc. for Food Protection Publication, Procedures to Investigate Foodborne Illness – 5th Edition. Certificates of participation were provided with Delaware Teacher Center recognition of 8 credit hours toward recertification. The event was promoted to the community via print and electronic communication resources beginning 4 mo in advance of the workshop as outlined in Table 1. Immediate impact of materials on secondary educators The immediate impact of the materials on familiarity of microbial food safety topics to the workshop participants was measured by administration of 27 written questions at the beginning and conclusion of the workshop. Questions addressed general food microbiology, foodborne illness outbreaks, regulatory issues, and food safety behaviors. A 25-question survey was also administered at the conclusion of the workshop to gather participant demographic information and seek evaluation of the workshop and the educational materials. Participants were introduced to an opportunity to participate in a research study to evaluate the educational materials after implementation in secondary science classes and were invited to submit their potential interest in this opportunity. Results and Discussion Educator and student insight A key recommendation of educators throughout development of the materials was to assure flexibility in design for application across the curriculum, classroom time constraints, and students’ prior familiarity with the topic. The structure of high school classes varies considerably with some courses offered in longer time blocks per session but for only portions of the school year, while others have shorter class periods year round. Facilities vary considerably as well in both laboratory and technology resources. A further recommendation was for each of the components to be complementary but also appropriate for stand-alone use so that teachers could integrate selected materials based on time, audience, and complement to previously used resources. Teachers and youth reacted positively to the use of a familiar scenario in case studies and the investigative nature as an approach to engage students. Student recommendations for the video during development were to focus 30 Journal of Food Science Education r Vol. 12, 2013 on simply “delivering the message” in an interesting scenario with good quality technology and to not make overt attempts to “relate” to the age group. Educational materials developed Presentation. A Power Point® slide presentation (see Supporting Information at: http://ag.udel.edu/foodinvestigation/) was prepared to provide a foundation of knowledge in food microbiology. The presentation consists of 76 slides and covers basics of foodborne microorganisms, factors affecting disease, illness outbreaks, surveillance, epidemiology investigations, laboratory detection, and process control measures. The presentation was provided in a flexible format such that the teachers could select the slides that best complement the learning concepts for their courses, age groups, and prior exposure, as well as to complement existing curricula. During the educator workshop, audio of the presentation was captured and made available via the web to provide a supplementary resource for educator reference to support future implementation in the classroom. Case studies. Four foodborne illness outbreak case studies were developed in which the student assumes the role of investigator. The etiologies of the outbreaks are provided along with a brief scenario of information gathered to-date. A series of questions directs the student to solve some element of the outbreaks such as the most likely food vehicle based on epidemiology data or knowledge of control factors for food safety. Other questions focus on good manufacturing or agricultural practices. Example questions from the case studies are presented in Table 2 to illustrate the major science education concepts that are addressed for secondary education as well as specific examples as they apply to food safety and foodborne illness investigations. The cases are intended to enhance skills in solving problems, designing experiments, communicating verbally and in writing, and collecting, compiling, calculating, presenting, and interpreting data. The cases also address societal issues including regulatory response and communication of public health issues in mass media. The 4 cases are based, to some degree, on actual foodborne illness outbreaks that occurred in the United States. Each case introduces the student to a different type of microorganism that has been implicated in illness outbreaks including vegetative bacteria, spore-forming bacteria, viruses, and protozoan parasites. The variety of cases is intended to help the student appreciate the biological similarities and differences among these types of organisms, as well as the complexities of food production, processing, distribution, and handling. The real cases on which the case studies are based had a major impact on food safety scientific knowledge, food regulations, international trade, and/or economics, and as such, were intentionally modeled to enhance awareness of these connections. For easy implementation in the classroom, both student and teacher versions of the case studies were prepared. The teacher version has proposed answers for factual-based questions, concepts for discussion for opinion-based questions, and brief historical sketches of the actual outbreaks on which the case studies are based. Two cases, hepatitis A virus and Escherichia coli O157:H7, are similar in length (18 questions each) and approach with focus on epidemiology and data handling. The other 2 cases have fewer questions (9 questions each) and different emphases. The Clostridium botulinum case places more emphasis on an understanding of food composition and processing for control. The Cyclospora cayetanensis case addresses traceback and good agricultural practices. The cases may be worked individually or in groups. One Available on-line through ift.org Food safety materials for secondary education . . . Table 2–Select example questions from case studies and the knowledge or skill addressed.a Knowledge and/or skills addressed General Scientific method Epidemiology Specific Data collection Epidemiology Epidemiology Data analysis and interpretation Epidemiology Laboratory detection of pathogens Pathogen characteristics Incubation period Communication networks, Distribution Environmental influences Distribution, Point of contamination Food preservation strategies Global distribution Mathematics Critical thinking and problem solving Epidemiology, Mathematical applications Economics, Mathematical applications Connection between epidemiological and laboratory investigations Resource allocation and response time in emergency situation Handling, Spread of disease Public health, Roles Regulatory roles, Food processing controls Food processing Good agricultural practices Good agricultural practices Laboratory detection of pathogens Societal, economic Accountability/roles Clinical intervention, accountability Economic impact of illness Prevention, regulations Communication Communication, public health professional roles Epidemiology Media and public health response Organizational, written and oral communication Example question excerpt Aside from talking with patrons who contracted . . . , who else should you interview to help determine . . . what menu items or other exposures did not cause illness? Based on the information you already have . . . prepare a list of interview questions . . . Using the questions you developed, conduct interviews with the latest cases . . . Using data . . . , prepare a bar graph of the number of patients who developed symptoms for the dates of symptom onset . . . Laboratory data for various foods consumed by patrons are presented . . . Do the laboratory results support your hypothesis on which foods were associated with illness? Review the symptoms and circumstances . . . and determine which individuals may or may not be cases associated with this outbreak. Provide an explanation for your assessment on each individual. Could the ill employee have been the original source of contamination in this outbreak? Do the data suggest any similarities between the outbreaks? A year-long study of infection rates . . . What, if any, correlations can you make regarding the seasonality of . . . illnesses? . . . given the distribution of cases, propose a hypothesis as to whether the contamination was more likely to have occurred on the farm, by a food service handler, or either scenario with equal likelihood. . . . indicate whether or not the foods found in this patient’s home have the potential to support C. botulinum growth and toxin production Review the data . . . Indicate whether the implicated product was produced domestically, was imported, or if the data are insufficient to draw conclusions. . . . calculate the odds ratio to determine what foods/events are positively associated with illness. Use the USDA/ERS calculator . . . what is the estimated total cost associated with this outbreak using ERS assumptions? What do you want to have tested for contamination? Give a reason why you would not necessarily want to have all foods tested. When multiple foods are associated with illness, what could be the reasons for several foods being implicated in any outbreak? One of the individuals diagnosed with . . . in this outbreak is a food handler in another . . . What additional measures should be taken to protect the public? Suspect source of food is traced to . . . and review of operational records ensues. Why do the inspectors need the following records? Record review reveals critical problems with a production line currently running. What actions do you think should be taken? Why are inspectors interested in the following agricultural practices as they relate to this outbreak? The following GAPs were not fully implemented . . . in consideration of known reservoirs, . . . which may be more critical to prevent contamination of fields . . . ? C. cayetanensis cannot be grown in a laboratory, model animals or cell culture. ..The following approaches are used to study . . . Indicate potential limitations for . . . If it is concluded . . . provide an opinion statement on who should be held financially accountable . . . Vaccinations are available . . . state your opinion on whether vaccination should be required and who should be financially responsible. Use the USDA/ERS calculator . . . what is the estimated total cost associate with this outbreak using ERS assumptions? After this outbreak, various groups with stake in this issue met to discuss appropriate actions for consumer protection . . . Which of these proposed alternatives do you favor? State the reason . . . A search of networked databases for reportable illnesses yields no other cases in the same . . . . . . conduct interviews with those individuals identified . . . Prepare a 60-s television news piece that informs the public of the critical information related to the case. Prepare a 15-min presentation for the class giving an overview of the case study. a These are not all inclusive but representative examples from the 4 case studies. All cases addressed various aspects of the scientific method as it applies to foodborne illness investigations and food safety strategies. Available on-line through ift.org Vol. 12, 2013 r Journal of Food Science Education 31 Food safety materials for secondary education . . . Figure 1–Example screenshots of Foodborne Illness Outbreak Investigation – Behind the Scenes video to illustrate variety of visual formats to support education on foodborne pathogens, disease transmission, detection technologies, and biochemical principles of pathogen detection. case may be used for the whole class, or different groups could work through different cases with similar responsibility levels. The teacher version also lists suggestions for modifying the length of the case studies where time is constrained. Video. A 14-min video (Figure 1) entitled Foodborne Illness Outbreak Investigation: Behind the Scenes was developed to introduce students to the principles of detection and identification of foodborne pathogens in the context of a foodborne illness outbreak investigation. The video was developed as a means to illustrate laboratory practices that typically cannot be conducted in high school laboratories due to safety, cost, facility resources, and time constraints. The audiovisuals vary throughout the video with laboratory footage mixed with animations and still photography overlaid with narration and music. A scenario is presented in which common gastrointestinal illness symptoms could be attributed to different microorganisms, and determination of the etiology is based on fundamental concepts of microbial biochemistry. With rapid advancements in technologies, the focus remains on principles though related technologies are visualized and explained. Principles introduced include metabolism, antigenic properties, genetic material, and infectivity. Technologies illustrated include culturing bacteria, immunoassay, centrifugation, cell culture, polymerase chain reaction (PCR), and pulsed-field gel electrophoresis (PFGE). The video concludes with the importance of global surveillance and communication for outbreak resolution. At the suggestion of the teacher advisory board, a list of discussion ques32 Journal of Food Science Education r Vol. 12, 2013 tions (30 questions) related to the video was developed to expand upon issues brought out in the video. Also at the suggestion of the advisory board, the video duration is less than 20 min for maintenance of student interest. Web-based activities. Three web-based activities were developed (Figure 2) and made available to secondary educators. Two activities were newly developed using software licensed for use by the Univ. of Delaware including StudyMateTM and Tile Sorter. The 3rd web-based game, Outbreak Concentration, was developed for use in the Univ. of Delaware Foodborne Diseases course and was made available to secondary educators in conjunction with the other educational materials. The content for these activities was built and made available through the Univ. of Delaware College of Agriculture and Natural Resources website. Paper and electronic copies of the text content of the activities were provided to educators. The web content cannot be manipulated by the user due to license agreements and copyright restrictions. StudyMateTM web activities provide for 10 different activities based on the same database of 50 questions (5 topic categories with 10 questions each) related to the educational materials. The topic categories are food microbiology, outbreak investigations, laboratory detection, safety strategies, and critical numbers and roles. Some activities prompt for a response, but offer hints; others provide multiple-choice options. Several have gaming features including scoring of successful responses; 1 game permits 2 players. Available on-line through ift.org Food safety materials for secondary education . . . Figure 2–Sample of images from web-based activities to reinforce concepts from other educational components: (A) StudyMateTM Challenge Game; (B) StudyMateTM Crossword Puzzle; (C) StudyMateTM Flash Card; (D) Tile Sorter Activity before and (E) after shuffling; (F) Outbreak Concentration Game Case Notes, and (G) Case Book. Tile sorter web activities reinforce concepts of processes or events that happen in a specific sequence of up to 8 steps. The player is given the opportunity to review a procedure in its proper sequence, shuffle the cards, and then put them back in correct sequence. One sequence is on an outbreak investigation and 3 sequences are based on laboratory methods for pathogen detection: culture-based methods, PCR, and PFGE. The objective of the web-based Outbreak Concentration Game is to win clues to a foodborne illness outbreak puzzle to help solve an outbreak through surveillance, epidemiological, and environmental phases of the investigation. Text and pictorial clues are won by finding the appropriate matches between questions and answers. Success at finding matches depends on knowledge of the material, memory of card location, and ability to work quickly. Questions fade with time, and opportunities to earn clues are lost to emphasize the importance of gathering the appropriate information quickly in a real outbreak before it may become unavailable. Available on-line through ift.org Supporting documents. Supporting documents for the presentation, case studies, video and games include a glossary and classroom poster. The16-page glossary includes terms commonly used in food microbiology, basic epidemiology, and foodborne illness surveillance and outbreak investigations. The laminated classroom poster (18 × 26 inches) lists the primary foodborne microbial pathogens, illness symptoms, incubation period, unique features, and previously associated foods. Collectively, the materials support (1) conceptual learning of food microbiology and food safety strategies, (2) application of this foundational knowledge to solving a foodborne problem using the scientific method and analytical skills, (3) identification with the numerous roles of professionals in food safety, (4) development of verbal and written communication skills, and (5) deliberation on unresolved societal issues. The student role in learning varies in nature from passive (video, listening to presentation) to active (case studies, web games) to leadership (presentation to classmates) with varied visual, auditory, and interactive stimuli. National and/or Vol. 12, 2013 r Journal of Food Science Education 33 Food safety materials for secondary education . . . Table 3–Education content standards supported by food safety educational materials. Education Content Area Grades 9–12 Natl. Science Education Food Science, Delaware Natl. Health Education Family and Consumer Sciences, Delaware (with crosswalks to English Language Arts) Mathematics, Delaware Content standard (A) Science as Inquiry; (C) Life Science; (E) Science and Technology; (F) Science in Personal and Social Perspectives; (G) History and Nature of Science (1) Safety; (2) Economics; (4) Food Processing Technology; and (6) Careers in Food Science Technology (1) health promotion and disease prevention; (2) societal and technology influences on health behaviors; (4) use of interpersonal communication skills to avoid or reduce health risks; (5) decision-making skills to enhance health Consumer Rights and Responsibilities (CRM) 3.1, Identifying consumer rights and responsibilities; CRM 3.4 policies and agencies to support consumer rights Interpersonal Relationships (IPR) 1.5, demonstrate teamwork and leadership skills; IPR 1.6, explore community-based support services that assist individuals in management of crisis; IPR, explore career opportunities that help develop interpersonal skills Food, Nutrition, and Wellness (FNW) 1.4, factors affecting food safety; FNW 1.5 impact of science and technology on food composition; FNW 2.1 explore career opportunities in food and wellness; FNW 2.6.2, identify causes of foodborne illness and ways to avoid them Standard 2, Algebraic Reasoning Standard 4, Quantitative Reasoning Standard 7, Communication Standard 8, Connections state education content standards for science, food science, health, mathematics, and family and consumer sciences are supported for grades 9 through 12 (Table 3). With the exception of the webbased activities and video, all materials were provided in a format permissive of modification to accommodate varying educators’ needs and time constraints. Workshop to disseminate materials to educators Eighteen educators from 3 states (6 from Del.; 11 from N.J.; and 1 from Pa.) participated in the workshop. Participants taught various grade levels including junior high (2), 9th (11), 10th (15), 11th (13), 12th (12), college (2) and subject matters including biology (9), chemistry (5), consumer sciences (7), and other sciences (9) including emerging diseases, forensics, environmental chemistry, anatomy and physiology, and physics. None of the workshop participants were primarily involved in agriscience education though the workshop was promoted by both direct postal mail and electronic mail within this community of educators in the state. No time conflicts were known for this professional group, though an individual conflict precluded attendance by the agriscience advisory board member. A subsequent suggestion from an extension professional was to host future workshops in closer proximity to the majority of agriculture educators to increase their participation. Participants indicated they learned of the workshop by direct mailed registration brochure (8), workshop announcement in teacher center brochure (3), and/or from a colleague (10). Interestingly, none of the participants reported learning of the workshop through websites or listserves. 34 Journal of Food Science Education r Vol. 12, 2013 The educators were enthusiastically involved in all activities of the workshop. The morning activities were dedicated to the presentation, video, and web-based activities, followed by a lunchtime keynote address and an afternoon session on the case studies. Lengthy discussion surrounded the presentation to introduce food microbiology and disease surveillance. The workshop participants were particularly interested in and asked questions related to foodborne illness estimates, microorganism features, emerging diseases, commercial food safety control strategies, and various food safety topics in popular press. Due to the time devoted to the discussion, the web games were introduced as a demonstration rather than hands-on activity during the workshop. Teachers were very engaged in the keynote address that provided an overview of a foodborne illness outbreak by a field investigator, and this was followed by group discussion on the outbreak case studies. The teachers offered considerable insight on how they might integrate the case studies in their classes. Some participants deemed the case studies reminiscent of popular fictional investigative television programs and thought this would make the cases somewhat fun for the students to solve. The teachers also cited that the cases bring concepts to life, including concepts taught in ServSafe® . One group envisioned further personalizing the cases by associating individual students with an individual mock victim in the case studies. The teachers reacted positively to the development of student skills in graphing, including its application for geographical concepts in food production, and the incorporation of mathematics in making correlations. The teachers liked the incorporation of communication skills including persuasive and technical writing and interviewing mock victims, though the teachers also predicted mixed reaction among students to various exercises with developing news announcements anticipated to be received positively by students while providing oral presentations to classmates to be received negatively. The crossover to historical concepts was also well received by educators, and the opportunity to link the individual experience to outcome of the case was noted as a potential class discussion point. The teachers noted considerable variation in student backgrounds and skill levels and indicated some students would need more support with background information, graphing exercises, and terminology, with the latter possibly supported by use of a word wall in the classroom to introduce unfamiliar glossary terms. The potential length of time for students to work through cases was noted as a concern that may be addressed by discussion of some components during class with other sections completed as homework. Impact of educational materials The design of the workshop was to expose the teachers to the same content that would be used in the high school classroom. This included use of the same assessment test questions (Table 4) on learning as would be administered to high school students as part of the subsequent implementation study to evaluate impact of the educational materials on student familiarity with food microbiology concepts. The assessment questions were formatted as multiple choice or true/false and addressed food microbiology facts as well as demographic and self-reported food handling behaviors. Responses to the latter questions would not be expected to change during the course of a 1-d workshop, but are captured in Table 4 to illustrate self-characterization of educators with a vested interest in the subject matter. At the beginning of the workshop, participating educators were most familiar with food contamination routes and safety intervention strategies. The questions most often missed involved Available on-line through ift.org Food safety materials for secondary education . . . Table 4–Change in educator familiarity with food microbiology concepts after introduction of food safety educational materials during workshop. Question 1. Match the categories of foodborne microbial agents to the best description (provided): Bacteria Virus Protozoan parasite Mold Yeast Prion 2. Match the microbe genus name to its classification (as yeast, bacterium, virus, mold, prion, protozoan parasite): Hepatitis A Salmonella Cyclospora Escherichia Saccharomyes Clostridium 3. Incubation period for a disease-causing microbe is defined as: 4. In optimal growth conditions in a food, a single bacterium of a rapidly growing species can multiply to over 1 million bacteria within 12 h. 5. In optimal growth conditions in a food, a single virus particle of a rapidly growing species can multiply to over 1 million virus particles in 12 h. 6. Circle each of the following that can be points of food contamination: Farm Food manufacturing facility Restaurant Grocery store Home kitchen 7. The odor and appearance of food are fail-safe indicators of whether or not a food is contaminated with disease-causing microbes. 8. Circle each of the following processes that are used to enhance food safety: Heat Reduced water activity Irradiation Acidification High hydrostatic pressure 9. Which of the following affects the growth of bacteria in foods: pH Oxygen availability Temperature All of the above None of the above 10. Principles used for detecting and identifying microbes in foods include: Metabolic characteristics Genetic material Antigenic structures All of the above None of the above 11. Foodborne illnesses in the United States are most often attributed to: Microbiological agents Pesticides Hormones Preservative overdose 12. A foodborne illness is usually classified as an outbreak when: 13. The estimated number of individuals to be sickened by contaminated food in the United States every year 14. Food products implicated in a foodborne illness outbreak will be recalled only if the contaminant causing the illness has been recovered from a previously unopened container of the implicated food. 15. A contaminated food will always cause the same symptoms and severity of illness in all individuals who consume the product. 16. Human vaccines are not available in the United States for protection against any foodborne disease agents. 17. Imported and domestically grown foods have the same regulatory safety standards for legal distribution in the United States 18. The U.S. food regulatory agencies require that all foods sold in the United States carry warning labels if the product has the potential for carrying disease-causing microorganisms. 19. Which of the following groups (producers/manufacturers, USDA, FDA, CDC, Nat’l. Rest Assoc.) have primary accountability for the following responsibilities: Initiating recall Safety of food prepared for sale in restaurants Safety of food prepared for sale in grocery stores Determining food source responsible for foodborne illness outbreak Pretest correct (%) Posttest correct (%) Change in correct responses (%) 52.6 84.2 57.9 63.2 89.5 63.2 73.7 89.5 63.2 89.5 100 78.9 21.1 5.3 5.3 26.3 10.5 15.7 Multiple choice True/false 73.7 84.2 21.1 42.1 36.8 36.8 73.7 94.7 94.7 89.5 47.4 57.9 63.2 57.9 94.7 94.7 21.0 5.3 26.3 15.8 26.4 21.1 21.0 0 True/false 42.1 73.7 31.6 94.7 100 94.7 94.7 100 100 100 100 94.7 94.7 100 100 5.3 0 0 0 0 0 94.7 78.9 89.5 89.5 63.2 100 100 94.7 100 100 5.3 21.1 5.2 10.5 36.8 94.7 94.7 0 73.7 94.7 21.0 94.7 100 5.3 Multiple choice Multiple choice 10.5 47.4 94.7 84.2 84.2 36.8 True/false 78.9 68.4 (10.5) True/false 100 100 0 True/false 68.4 78.9 10.5 True/false 5.3 31.6 26.3 True/false 68.4 78.9 10.5 10.5 21.1 31.6 68.4 31.6 15.8 36.8 52.6 21.1 (5.3) 5.2 (15.8) Format Matching Matching Selection True/false Selection Multiple choice Multiple choice Multiple choice Matching Continued. Available on-line through ift.org Vol. 12, 2013 r Journal of Food Science Education 35 Food safety materials for secondary education . . . Table 4–Continued. Question 20. The recommended temperature for a household refrigerator is (40 ◦ F, 45 ◦ F, 50 ◦ F) Format Multiple choice Self-reported behavior questionsa Pretest correct (%) 73.7% Posttest correct (%) 89.5% Change in correct responses (%) 15.8 Ideal behavior (%) 21. I wash my hands with soap and water after using the toilet: (never, sometimes, always) 22. I wash my hands with soap and water just before cooking: (never, sometimes, always) 23. In my household, a thermometer is used to determine when meat is done cooking: (never, sometimes, always, do not know) 24. After a meal is cooked in my household, leftover perishable food is always put in the refrigerator: (within 2, 5, and 8 h, do not know) Demographic questionsa Multiple choice 100% always Multiple choice 94.7% always Multiple choice 15.8% always Multiple choice 94.7% within 2h 25. I would rate my knowledge of safe food handling practices as: (poor, good, excellent, cannot rate) Multiple choice 26. Education level: (9th grade, 10th grade, 11th grade, 12th grade, college) 27. Prior exposure to microbiology topics: (none, some, a lot) Multiple choice 52.6% good; 42.1% excellent 100% college Multiple choice 63.2% good; 36.8% excellent 15.8% none; 57.9% some; 26.3% a lot a Pretest responses. scale and to provide additional written comments as desired. The results are presented in Figure 3. Each of the components of the Presentation educational materials was given an average rating of approximately Video 3.5 to 4.5 on a 5-point scale with 5 being the best rating. Similar Case Studies ratings were given for how well the educational materials were anGames ticipated to engage students and to complement existing curricula, Supporting Material comply with time constraints and content standards. Verbal feedCompliment to Current Curriculum back from some FCS educators during the workshop suggested Grade Level Appropriateness Learning Styles Addressed the scientific technical content may be too detailed or challenging Education Content Standards for some of their students; however, there were no significant difAnticipated Student Interest ferences in ratings among teachers based on subject matter taught. Anticipated Comfort in Teaching Biology teachers reported responsibility for teaching varied subLikelihood of Implementation ject matters, grade levels, and challenge levels for their courses. Likelihood of Recommendation All respondents rated the materials positively on their anticipated comfort to teach, likelihood of implementation, and likelihood of Figure 3–Evaluation of educational materials by educators at conclusion recommending the resources to a colleague. The workshop format of workshop with 1 = poor, 3 = good, 5 = excellent. was given an average rating of 4.6 (5 best score) for organization. We also sought insight on the valuation of the workshop for parmicrobe classification, viral replication, outbreak classification, and ticipant investment; the nominal workshop registration fee was accountability for safety within the food system. After the pretest, considered between “a bargain” and “reasonable.” the educational materials were introduced and used to the extent possible in a 1-d workshop. Only a few of the assessment questions were directly addressed in the educational materials in which re- Interest in implementation Workshop participants and those who requested the materials iteration of facts was ample; more questions addressed knowledge gained through problem-solving exercises. The extent of expo- at the recommendation of colleagues were invited to participate sure during the workshop was sufficient to enhance the awareness in a research study to assess the materials after implementation of food microbiological safety concepts among the educators with in the classroom. Teachers were asked to complete and submit a positive change recorded in correct responses to the same questions form indicating their interest, tentative plans to utilize the maat the workshop conclusion. The pre- and posttest results were not terials, and the potential number of students to be exposed to viewed until after the conclusion of the workshop, and there was the materials in their classes. At the conclusion of the workshop, no discussion of test questions or responses with workshop partici- 7 teachers of varying subjects, including agriscience (1), micropants. Among self-reported safe food handling behaviors, the least biology (3), emerging diseases (1), chemistry (1), and food scipracticed was consistent use of thermometers to determine when ence/biology (1) expressed interest in participating in the study. meat is adequately cooked. Approximately 16% of participants The interested agricultural science teacher noted was the individual who served on the advisory board and made direct contact to reported having no prior exposure to microbiology topics. receive the materials for classroom use. The teachers anticipated Educator evaluation of materials intended use of the presentation (4), case studies (6), video (5), and At the conclusion of the workshop, participants were asked to games (4) with potential exposure to approximately a total of 400 anonymously evaluate the program and materials on a 5-point students. 1 2 3 36 Journal of Food Science Education r Vol. 12, 2013 4 5 Available on-line through ift.org Food safety materials for secondary education . . . Conclusions Food safety educational materials as presented in a foodborne illness outbreak scenario were well received by secondary educators of biology, chemistry, family and consumer sciences, and other applied sciences. The educators had particularly favorable opinions on potential flexible application of the materials, crossover of concepts (communication, history, mathematics, graphing, and biology), complement to current curriculum, and support for tiered teaching. Exposure of the materials to secondary educators in a 1-d workshop positively impacted their familiarity with general food microbiology, food safety strategies, regulatory requirements, and associated terminology. There was interest among workshop participants to implement the materials in their science courses. Acknowledgments The authors gratefully acknowledge the following individuals whose contributions were vital to development of the video, Foodborne Illness Outbreak Investigation: Behind the Scenes: Raymond Lewis, Paul Hyde, Paul Rickards, Kirsten Hirneisen, Andrew Shearer, Joe Woodall, Jie Wei, and Dallas Hoover. The authors thank Christy Mannering, Becky Kinney, and Carrie Finnie for assistance with website and games development. The authors thank advisory board members, students, and Cheryl Bush for helpful discussions and insight. The authors also gratefully recognize Allison Wilson, Casey Johnson, and especially Elizabeth Appleby for organizational and creative assistance in preparation of the workshop and Thomas Hill for sharing field experience and expertise during the workshop. The materials presented in this publication were prepared by the Univ. of Delaware based upon work supported by the Cooperative State Research, Education, and Extension Service, U.S. Dept. of Agriculture, under Award No. 2009–38414-19698. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the authors and do not necessarily reflect the view of the U.S. Dept. of Agriculture. References Abrahamson C. 1998. Storytelling as a pedagogical tool in higher education. Education 118:440–52. Bureau of Labor Statistics. Career guide to industries, 2010–2011 Edition. Available from: http://www.bls.gov/oco/cg/cgs023.htm#emply. Accessed Mar 27, 2012. CDC (Centers for Disease Control and Prevention), Joint Committee on Natl. Health Education Standards. 2007. Available from: http://www.cdc.gov/healthyyouth/sher/standards/index.htm. Accessed May 10, 2012. Available on-line through ift.org Chapman B, MacLaurin T, Powell D. 2011. Food safety infosheets: design and refinement of a narrative-based training intervention. Br Food J 113(2):160–86. Delaware Dept. of Education. 2007. Delaware Content Standards and Recommended Curriculum AgriScience Education Food Science Pathway Standards. Available from: http://www.doe.k12.de.us/infosuites/staff/ci/ content_areas/agriscience.shtml. Accessed May 10, 2012. DeSchryver M, Dirkin K, Herreid CF, Lundeberg M, Maier K, Schiller NA. 2007. Teaching science with case studies: a national survey of faculty perceptions of the benefits and challenges of using cases. J College Sci Teaching 37(1):34. Harmon AH, Maretzki AN. 2006. Assessing food system attitudes among youth: development and evaluation of attitude measures. J Nutr Educ Behav 38:91–5. Harris C, Knight A, Worosz MR. 2006. Shopping for food safety and the public trust: What supply chain stakeholders need to know about consumer attitudes. Food Safety Mag. 12(3):52–59. Herried CF. 2005. Using case studies to teach science. An ActionBioscience. org article. Available from: http://www.actionbioscience.org/education/ herreid.html#primer. Accessed December 10, 2008. Lordly D. 2007. Once upon a time . . . Storytelling to enhance teaching and learning. Can J Diet Pract Res 68(1):30–5. McArthur L, Holbert D, Forsythe W. 2007. College students and awareness of food safety. J Food Sci 99:60–8. Natl. Research Council, Committee on Highly Successful Science Programs for K-12 Science Education. Board on Science Education and Board on Testing and Assessment, Div. of Behavioral and Social Sciences and Education. 2011. Successful K-12 STEM Education: Identifying Effective Approaches in Science, Technology, Engineering, and Mathematics. Washington, D.C.: The Natl. Academies Press. Natl. Research Council, Committee on Science Learning: Computer Games, Simulations, and Education. 2011. Learning science through computer games and simulations. Honey MA, Hilton M, editors. Washington, D.C.: Natl. Academies Press. Natl. Research Council, Natl. Committee on Science Education Standards and Assessment. 1996. Natl. Science Education Standard. Washington, D.C.: The Natl. Academies Press. Available from: http://www.nap.edu/ catalog/4962.html. Accessed May 10, 2012. Patil SR, Cates S, Morales R. 2005. Consumer food safety knowledge, practices, and demographic differences: findings from a meta-analysis. J Food Prot 68:1884–94. Ralston K, Brent CP, Starke Y, Riggins T, Jordan Lin CT. 2002. Consumer food safety behavior: a case study in hamburger cooking and ordering. Washington, D.C.: Economic Research Service/USDA. AER-804. Supporting Information Additional Supporting Information may be found in the online version of this article at the publisher’s web site: Website: Foodborne Illness Outbreak Investigation, http://ag. udel.edu/foodinvestigation/ Vol. 12, 2013 r Journal of Food Science Education 37