ELECTRONIC SUBMISSION FOR CONSIDERATION IN THE UNIVERSITY OF TORONTO MEDICAL JOURNAL PedsCases Content Production Handbook: A Glimpse into the Future of Medical Education Debraj Das1, Michelle Bischoff1, Peter MacPherson1, Chris Gerdung1 Lauren Kitney2, Peter J Gill1, Melanie Lewis1 1 University of Alberta Faculty of Medicine & Dentistry, Department of Pediatrics, 8213 Aberhart Centre 1, Edmonton, Alberta. 2 University of Toronto, Department of Paediatrics. 555 University Avenue, Toronto, Ontario. Corresponding Author: Debraj Das 780-907-6769 Email: debrajd@gmail.com UTMJ ORIGINAL RESEARCH SUBMISSION Page 1 of 16 DEBRAJ DAS ABSTRACT INTRODUCTION: PedsCases (http://www.pedscases.com) is a pediatric educational website for undergraduate medical students developed through student-faculty collaboration. PedsCases consists of pediatric specific clinical multiple-choice questions (MCQs), podcasts on key learning objectives, and interactive cases that teach students clinical decision-making skills. Our objective was to create a handbook for medical students that outlined how to create high quality online learning resources. METHODS: The PedsCases contributors worked together on this project and drew upon prior experiences and medical education theory to develop a handbook that carefully summarizes the core elements of writing MCQs, podcasts, and clinical cases. Individual members drafted a version of the handbook, which was then peer reviewed and verified by members of the PedsCases team, including the director of pediatric undergraduate education. DESCRIPTION OF HANDBOOK: Each section consists of: 1) specific writing mechanics; 2) how to test knowledge application; and 3) tips specific for PedsCases content. The clinical case section explores how to develop and organize a clinical case scenario while ensuring that the content is applicable to medical students. The MCQ portion discusses how to properly write a question stem, challenging distracters, and how to integrate MCQs into clinical situations. The section on podcasts explains characteristics that make a podcast different from a spoken lecture, giving an outline for podcasts to follow, and discussing the advantages and disadvantages of podcasts as a learning venue. In addition, we review important considerations to take into account UTMJ ORIGINAL RESEARCH SUBMISSION Page 2 of 16 DEBRAJ DAS while creating medical education content specifically for undergraduate medical education. CONCLUSIONS: This handbook highlights how students can effectively transform textbook knowledge into an online resource that complements day-to-day medical education. Putting students in a virtual setting where they are able to make decisions, follow their curiosity, and arrive at the right conclusions, helps to develop well-rounded medical students and future medical educators. Keywords: Medical education, pediatrics, peer teaching, undergraduate, e-learning UTMJ ORIGINAL RESEARCH SUBMISSION Page 3 of 16 DEBRAJ DAS INTRODUCTION Canadian medical schools are replacing traditional didactic lectures with interactive problem-based learning. This shift reflects consensus among educators that active inquisition and discovery is a more effective learning method than passive absorption. This signals a shift toward student-centered-learning, which puts the onus on the student to identify useful educational tools. There is a deficiency of pediatric medical education in the general undergraduate curriculum. Pediatrics is often taught longitudinally, with lectures sparsely distributed throughout the preclinical and clinical curriculum. The resources available often overwhelm students and this has compelled the creation of a comprehensive educational tool to focus on the core objectives of undergraduate pediatric education tailored to medical students. PedsCases (www.pedscases.com) is a collaborative resource created for and by medical students, which provides an opportunity for active self-directed learning. The site engages students by inviting participation, either as authors or by contributing to the discussion surrounding content. The premise of the website is that students create the interactive cases, multiple choice questions, and podcasts, directed towards educational deficiencies and the content is peer-reviewed by pediatricians to ensure validity, accuracy and usefulness. UTMJ ORIGINAL RESEARCH SUBMISSION Page 4 of 16 DEBRAJ DAS Figure 1: Student and faculty partnership - PedsCases PedsCases has been integrated into the third year undergraduate pediatric medical education curriculum at the University of Alberta in Edmonton, Alberta, Canada. It is one of the main sources recommended to students to cover the core objectives of the clinical pediatric rotation and to assist in preparing for the final examinations. Since the focus of medical education has shifted towards independent learning, PedsCases has become an essential tool. Currently, PedsCases has 11 publications on the Association of American Medical Colleges (AAMC) peer-reviewed publication service MedEdPORTAL (www.aamc.org/mededportal) and 42,509 downloads of 22 podcast episodes. The usage of PedsCases around the world signifies the widespread and increasing use of online learning tools in undergraduate medical population. Recently, the PedsCases team at the University of Alberta developed a handbook that outlines the steps behind the creation of the core elements of the PedsCases: UTMJ ORIGINAL RESEARCH SUBMISSION Page 5 of 16 DEBRAJ DAS interactive clinical cases, multiple-choice questions, and podcasts. The objective of the handbook is to provide a thorough overview of the creative and technical process involved in order to create high-quality learning tools. The overall aim is to create a comprehensive and practical handbook that will empower students to take initiative in creating online learning resources for the benefit of themselves and their peers. METHODS The PedsCases contributors worked together on this project and drew upon prior experiences and medical education theory to develop a handbook that carefully summarizes the core elements of writing MCQs, podcasts, and clinical cases. Individual members drafted a version of the handbook, which was then peer reviewed and verified by members of the PedsCases Team, including the pediatric undergraduate education director. Figure 2: Collaborative student teaching and mentorship - PedsCases UTMJ ORIGINAL RESEARCH SUBMISSION Page 6 of 16 DEBRAJ DAS DESCRIPTION OF HANDBOOK Clinical Cases The purpose of a case is broader than simply testing facts and concepts. You need to feel like you are in the situation itself – making decisions, following your curiosity, and arriving at the right conclusions. A well-written case will be able to integrate that practical experience with all of the important knowledge that a student should know for the topic being presented. 1. Start Basic! Complete Your Research Cases require background reading of textbooks, journals, and review articles. 2. Organize Your Case Organize your case to cover points such as patient history, physical exam, and diagnosis. Each case is composed of 7-10 steps that cover initial visit to discharge. 3. Write Your Case Objectives Objectives cover exactly what the reader should be able to understand, answer, or explain at the end of the case. Make sure objectives are clear and concise. 4. Develop a Scenario Think of a ‘real-life’ situation where your case may exist (e.g. family clinic, pediatrician’s office etc.). Scenarios allow you to actively engage the reader so they feel like they are making clinical decisions. UTMJ ORIGINAL RESEARCH SUBMISSION Page 7 of 16 DEBRAJ DAS 5. Avoid Excess One of your tasks is to filter the research and content you have to read from your background research. Overloading the student with facts, figures, and details will be overwhelming and counterproductive. Each step of a case presents relevant information that will shed more light on the patient you are working with. The steps should be presented in a logical order similar to how you would deal with a patient in an actual clinic setting. A logical flow will allow the case to unfold in an appropriate manner and therefore, ensure that the reader comes to a conclusion that makes clinical sense. Scattered throughout the case there should multiple-choice questions that challenge the reader’s understanding of core concepts. Creating a successful multiple-choice question demands many important considerations from the writer. Not only does the question need to test the application of a concept but the distracters need to be effectively written so that the reader is challenged with every option. A well-written multiple-choice question resounds with the reader well after the case and guarantees that they fully appreciate the idea being tested. (Refer to MCQ section of handbook) In conjunction with the multiple-choice questions there needs to be thorough explanations of not only the correct answer, but why the distracters were incorrect. Feel free to be detailed, and spend time explaining concepts and ideas to their fullest. This portion of the case is of the most value to your reader – it needs to be thorough and comprehensive. UTMJ ORIGINAL RESEARCH SUBMISSION Page 8 of 16 DEBRAJ DAS Multiple Choice Questions There are certain fundamental ways to make multiple-choice questions successful indicators of knowledge. Wording, format, and question style noticeably differentiate good questions from ineffectual ones. This section provides a brief overview of how to construct effective multiple-choice questions including question mechanics and how to test reasoning and application. Keep Details in the Question o The question stem should provide all necessary information. This allows for the question to be read easily by the student1, 2, 3. Utilize Consistency in Language o Question stem, answer and distracters should have consistent tense, units, pleurisy, subject-verb agreement, and length1, 2, 3. Write Effective Distracters o Distracters should be, Plausible but incorrect Correct but irrelevant to the question stated Incorrect but relevant to the question stated3 Like in all circumstances, with multiple-choice questions we always aim to assess a student’s comprehension of concepts, not rote memorization of facts. Testing broader concepts will ensure that the material and content will remain with the student well after the question is completed. In addition, testing concept allows for greater applicability of the information compared to a detailed fact-based multiple-choice question. Questions UTMJ ORIGINAL RESEARCH SUBMISSION Page 9 of 16 DEBRAJ DAS that promote analysis and application better examine for understanding, versus straightforward questions that require identification of recalled specifics. Also, in similar fashion to clinical cases, incorporating the question into a clinical vignette is a helpful method of testing application of knowledge. For instance, instead of asking a precise question about disease X, provide a description of clinical manifestations and ask the student to identify the most likely diagnosis. An example of a knowledge-based question is as follows: Which of the following drugs is the first line treatment for a school-aged child in status epilepticus? In contrast, here is a question that tests knowledge application via a clinical scenario: You are working in the Emergency Department when EMS brings in a 9-year-old boy presenting with a generalized tonic-clonic seizure that began 20 minutes ago. He is receiving oxygen via mask and an IV was initiated in the ambulance en route. Which of the following medications should be administered first? Podcasts As a learning venue, podcasts provide learners with the ability to access content ‘anytime, any-where’. This is their principal advantage as a learning tool. They are also shorter and more focused than traditional lectures and provide another way to learn to match individual learners’ preferences. In the end, the goal of a podcast is to provide a concise overview of a topic. 1. Selecting a Topic UTMJ ORIGINAL RESEARCH SUBMISSION Page 10 of 16 DEBRAJ DAS The first step is to choose an appropriate topic and consider your target audience. Currently, PedsCases has created podcasts that reflect the undergraduate pediatric curriculum. 2. Background Reading Consult several different textbooks, journal articles, and lectures. Pediatrics In Review and American Family Physician have comprehensive review articles that provide an approach to various clinical presentations. 3. Writing a Script The core elements of your script should include the following: Introduction History Physical Examination Specific Diagnoses Take-home points 4. Review If you are a junior learner, you will need a staff physician to review your script to ensure its validity, accuracy and usefulness as well as its appropriateness for the target audience. You should find a physician to help you before you do the research and write the script. CONSIDERATIONS There are several important considerations to keep in mind when creating medical education content for medical students: UTMJ ORIGINAL RESEARCH SUBMISSION Page 11 of 16 DEBRAJ DAS 1. AUDIENCE: Always keep your audience in mind when writing your case. PedsCases is a tool for every medical student both pre-clinical and those in clerkship. Questions should focus on key teaching principles at the medical student level, and not, for instance, trivial details in the footnotes of a pediatric subspecialty textbook. 2. CLINICAL APPLICABILITY: Formulate each case based on diseases that are relevant to the average medical student. Focusing your case on common circumstances, presentations, and illnesses will ensure broad applicability. Using teaching points can also be helpful since they focus on concepts that are important during clerkship years. Write your case always keeping clinical relevance in mind. Although integrating unique facts is fun, remember that you are writing the case to challenge your reader’s clinical understanding of a problem. 3. EXPLANATIONS: What distinguishes an excellent question from a good question on PedsCases, in large part, is the presence of a thorough explanation of why answers are correct or incorrect. Construct a comprehensive description, as the true learning lies in the rationalization. The more detailed your explanations are, the more knowledge can be obtained by students using your cases, questions, and podcasts. CONCLUSIONS & FUTURE DIRECTIONS PedsCases is continually growing and expanding to address the needs identified by medical students. Currently, wide ranges of podcasts are being recorded to comprehensively address the key identified learning outcomes in undergraduate UTMJ ORIGINAL RESEARCH SUBMISSION Page 12 of 16 DEBRAJ DAS pediatric medical education. Furthermore, videos are being filmed to outline the core procedural skills for medical students, including lumbar puncture, otoscopy, etc. In addition, the volume of cases and questions is continually growing, as is the large number of pediatricians participating in the site, both at the University of Alberta and other institutions. In regards to the handbook, we aim to verify its usefulness in the near future by distributing the resource to past students and faculty contributors and surveying its usefulness to these individuals. We also hope to open the content and information of the handbook to other faculties that may benefit from high-quality online learning resources. Since medicine is an inter-disciplinary profession producing relevant interactive educational tools for nurses, physiotherapists, psychologists etc. will be highly valuable in the overall evolution of teaching styles. Current drawbacks to PedsCases include the large time commitment required to create online learning resources. Conducting the background research, formulating objectives, contacting physician editors etc. can be a long and difficult process. In addition, finding adequate funding to support the continued growth of PedsCases can at times be challenging. On the other hand, one of the major strengths of PedsCases is that we are able to tackle and address issues that are specific to the undergraduate medical student population because PedsCases is a student driven initiative. As well, there are multiple UTMJ ORIGINAL RESEARCH SUBMISSION Page 13 of 16 DEBRAJ DAS learning modalities present on the website, which is open and accessible to students across Canada and the globe. Lastly, as an online project the website is in a constant state of growth, development and improvement meaning that the future of PedsCases and pediatric medical education is bright. Figure 3: Layout of PedsCases Website (www.pedscases.com) UTMJ ORIGINAL RESEARCH SUBMISSION Page 14 of 16 DEBRAJ DAS ACKNOWLEDGMENTS The authors would like to acknowledge Dr. Dan Kozan for his technical support in creating and supporting PedsCases. We would also like to thank the Office of Undergraduate Medical Education and the Department of Pediatrics at the University of Alberta for their continued personal and financial support in all of the endeavors undertaken by the PedsCases team. CONFLICTS OF INTEREST There are no conflicts of interest to report. SUPPORTING INFORMATION The additional document provided with the manuscript is our most updated and complete version of the PedsCases Content Production Handbook. The handbook covers all of the topics discussed in the manuscript. UTMJ ORIGINAL RESEARCH SUBMISSION Page 15 of 16 DEBRAJ DAS REFERENCES 1. Boland, RJ, Lester, NA, Williams, E. Writing multiple-choice questions. Acad Psychiatry. 2010 Jul-Aug;34:4. 2. Braddom, CL. A brief guide to writing better rest questions. Am J Phys Med Rehabil. 1997 Nov-Dec;76:514-516. 3. Collins J. Writing multiple choice questions for continuing medical education activities and self-assessment modules. RadioGraphics 2006 Mar;26:543-551. 4. NBME: Constructing written test questions for the basic and clinical sciences [Internet]. Philadelphia: National Board of Medical Examiners; c2002 [cited 2010 Nov 24]. Available from: http://www.nbme.org UTMJ ORIGINAL RESEARCH SUBMISSION Page 16 of 16