Standard Operating Procedure for the Preparation and Execution of the Examinations for Specialist Certification in “Clinical Pharmacology and Toxicology” Version 2.2, approved 24 MAY 2013 This document has been approved by the Executive Committee of the Swiss Society for Clinical Pharmacology and Toxicology (SSCPT). It is mainly intended for the Committee members in charge of preparing the specialist certification examination for candidates to the FMH title in ‘Clinical Pharmacology and Toxicology’. It may be freely distributed, and it is available on the website of the SSCPT (www.clinpharm.ch). Please check that you have the latest version. General overview The examination procedures for the specialist certification examination for candidates for the title “Specialist for Clinical Pharmacology and Toxicology FMH” compose of a written part and an oral part. The regulation and modalities of the examinations are laid down in chapter 4 of the postgraduate training programme “Specialist for Clinical Pharmacology and Toxicology” of January 1, 2001, last revision September 6, 2007, which can be obtained in German and French from the website of the Society or of the FMH. Introduction and delegation of responsibilities The Swiss Society for Clinical Pharmacology and Toxicology (SSCPT) has the responsibility to organise once per year a Specialist Certification exam session for MD candidates to the specialist title in “Clinical Pharmacology and Toxicology” issued by the Swiss Medical Association (FMH). The results of the exam are transmitted to the FMH, which is responsible for the validation of the entire training programme and the attribution of the specialist title. The SSCPT organises the exam according to its postgraduate training programme. The Executive Committee of the SSCPT delegates the exam-related tasks to several persons; the list is updated each year (appendix 1). Persons to be named are: a responsible for the specialist certification examination (member of the executive committee) a person in charge of maintaining the pool of questions for the written exam at least 2 independent assessors for the written exam (must have the FMH specialist title) 2 to 4 examiners for the oral exam (must have the FMH specialist title and hold responsibilities in the post-graduate training of MDs) 1 or 2 assessors for the oral exam (must have the FMH specialist title) a responsible for secretarial duties (SSCPT secretary) a responsible for financial aspects (SSCPT treasurer) SOP Specialist Exam, © SSCPT 2013 page 1 / 8 The responsible person for the specialist certification exam, the assessors for the written exam and the examiners for the oral exam compose the “Exam Commission” of the SSCPT. At least one member of this commission must represent category B training sites. The language regions of Switzerland should be represented appropriately. Announcement and registration The annual exam session is announced at least 6 month before the date. The announcement contains the following information: the date and place of the written examination session the date and place of the oral examination session (may also be announced directly to the candidates not later than one month before the oral examination session) the address accepting registrations and the deadline for registration (usually at least 3 months before the written exam session) the fee requested for registration and how the fee has to be paid The announcement is sent to the FMH (Secretariat SIWF, Elfenstrasse 18, POB 170, CH3000 Bern 15, Tel 031 359 11 11, Fax 031 359 11 12, E-mail siwf@fmh.ch). The FMH publishes the announcement in the Swiss Medical Journal and on its website. The SSCPT publishes the announcement on its website. The responsible for secretarial duties receives the registrations of candidates on the appropriate registration form (appendix 2) prepares a list with names, addresses, phone numbers, e-mail addresses sends a postage mail to the candidates after closure of the registration deadline, to acknowledge their participation, invoice their exam fees and inform them on the schedule and practical details of the exam. The written exam Composition of the written exam The written exam is carried out as a web-based examination and consists of 120 questions. The questions can be Multiple Choice (MCQ) or Short Answer Questions (SAQ). 1 Two types of MCQ are in use: type A (one-best-answer of five options) and type Kprim (four options, each can be true or false). The so-called blueprint of the exam (appendix 3) identifies the content areas covered by the exam and specifies the proportion of questions requested to cover each area. The collection of written exam questions is stored in the Webpool, a web-based tool for the storage, collection and revision of the pool of exam questions maintained by the Institute of Medical Education (Institut für Medizinische Lehre, IML) of the University of Bern. 1 Specialist in Clinical Pharmacology and Toxicology, Postgraduate Training Programme of 1. January 2001, last revision 6. September 2007, art 4.4.1 SOP Specialist Exam, © SSCPT 2013 page 2 / 8 For the exam, 120 questions will be selected and transferred into the Online Assessment Tool, through which the candidates will perform the exam online using a Web Browser. Maintenance of the written exam questions pool The person in charge of maintaining the pool of questions for the written exam maintains with high confidentiality a pool of written multiple choice (MCQ) and short answer questions (SAQ) that have undergone validation by a panel of experts in clinical pharmacology and toxicology. The questions expert panel (the “proof reading committee”) is composed of one delegate of each postgraduate training institution in Switzerland and the person in charge of maintaining the pool of questions (all must be members of the SSCPT). The person in charge of the pool makes sure that the question pool covers all chapters of the theoretical content of postgraduate training in sufficient excess (ideally more than 3-fold) compared with the exam “blueprint” and that the questions are up to date. Each year, a subset of questions is to be renewed. Therefore, the person in charge invites the members of the questions expert panel to provide a series of new questions in the various chapters, according to the needs to check a selection of questions out of the pool on correctness of content and answer. For SAQ clear correction criteria must be provided. The questions are centralized by the person in charge of the pool. The person in charge organizes a validation session. The validated questions are included in the pool. Selection of the set of questions for the written exam The IML adds an exam folder in the Webpool. The person in charge of the pool selects 120 questions out of the actualised pool according to the distribution given in the “blueprint” (appendix 3) and adds them to the exam folder within the time frame prespecified by the IML. A final control of the questions is made by the person in charge of the question pool. Once the definitive selection is made, the person in charge of the pool authorises the IML to transfer the questions from the Webpool to the Online Assessment Tool. Preparation of the written exam session The person responsible for the specialist certification examination organizes a sufficiently large room with a sufficient number of computers with internet access for the conduct of the written part of the exam. checks the payment of exam fees by each candidate in cooperation with the treasurer and the secretary of the SSCPT. sends a date confirmation to each candidate approximately 4 weeks before the written exam date, with reminders about the examination procedures (e.g. to bring an identification card to the exam, to read this document on the SSCPT website). prepares sheets of reference mathematical formulae which will be distributed to the candidates; such a formulary is shown in appendix 4. prepares sufficient reserve of paper and pens, and pocket calculators for the exam. sends exam data (date, time) to IML. sends candidate data (first name, last name) to the IML to prepare their access to the Online Assessment Tool. SOP Specialist Exam, © SSCPT 2013 page 3 / 8 sends assessor data (first name, last name) to the IML to prepare their access to the Online Assessment Tool. The IML transfers the exam questions from the Webpool to the Online Assessment Tool and provides a test access to the person responsible for the exam session. The person responsible for the exam session verifies that the technical installations and software products work properly and that the online assessment tool can be properly accessed. As soon as the exam is tested, the IML deactivates the test access and prepares the exam access. The IML sends the access data for the candidates to the person responsible for the exam session. The latter stores them in a secure place until the start of the exam. Additionally, IML provides a manual for the online assessment tool for the candidates. Conduct of the written exam session The online assessment is made available to the candidates during the pre-specified examination time through a secure web access at the examination site. The questions appear in a random order, so that all candidates have a different order of questions. All answers are to be entered in the Online Assessment Tool during the duration of the exam in the examination room. The Online Assessment Tool also provides space for comments to the assessors. The individual time for each candidate is exactly 4 h. The online assessment tool automatically stops accepting entries after this time. The person responsible for the exam session welcomes the examinees and checks the identity of the candidates. writes a time protocol about the conduct of the exam (e.g. documentation of toilet times, system interrupts, etc.). distributes the manuals for the online assessment tool, the mathematical formulae sheets, papers, pencils, pocket calculators and the access codes to the candidates. supervises the examination (verifies the absence of any forbidden communication of the examinees between one another or with external sources of information, e.g. books, phones, etc.). checks the examination time (which is set to exactly 4 hours by the online assessment tool); announces the time 2 hours and 3:30 hours after the start of the exam. contacts immediately the IML emergency support if any change of the duration of the exam must take place (e.g. because of interruption of the exam). collects carefully all exam material after the end, so that questions cannot be disseminated outside. The IML activates the access for the candidates according to the time frame agreed with the person responsible for the exam session. provides an IT-specialist on stand-by for emergency support during the exam. SOP Specialist Exam, © SSCPT 2013 page 4 / 8 Analysis of the exam data and publications of the results The IML sends access data for the two assessors to the person responsible for the specialist certification examination Analysis of MCQ The IML extracts the results from the Online Assessment Tool. provides access for the 2 assessors to the comments given by the candidates. The 2 assessors read the comments of the candidates. decide whether questions, which have been criticized by one or several candidates during the online exam, are invalid and must be excluded from the evaluation. Analysis of SAQ The IML grants access for the 2 assessors to the correction mode of SAQ. The 2 assessors rate the SAQs independently from one another in the online assessment tool after having received the invitation for rating by IML. A sample SAQ with answers and rating scheme is given in appendix 6. confront their results between one another, and converge to a common evaluation in case of disagreement. read the comments of the candidates. decide whether questions, which have been criticized by one or several candidates during the online exam, are invalid and must be excluded from the evaluation. Completion of the analysis The 2 assessors inform IML that the evaluation process has been completed. inform IML, which questions have been eliminated (if any). inform the person in charge of the pool, which questions have been eliminated and why (if any). keep strictly confidential their copies of answer forms up to the end of the exam procedure, and destroy them thereafter, if any. The IML includes, after validation by the two assessors, the results of the SAQ in the results of the written exam. excludes, after the two assessors have convened to do so, invalid questions from the evaluation. divides the total number of correct answers of valid questions by the number of valid questions: the candidates having answered correctly at least 57% of valid questions (i.e. >56%) will be considered as having passed the written exam. SOP Specialist Exam, © SSCPT 2013 page 5 / 8 provides a result form of the written exam for each candidate, which shows in detail the percentage results according to the blueprint topics along with the overall outcome (passed/not passed) of the written exam. This form is to be transferred to the person responsible for the specialist exam and to the secretary (see template in appendix 5). provides an audit track in lieu of a protocol for the evaluation procedures of the exam. Feedback to the candidates The person responsible for the specialist exam informs the candidates about the overall outcome of the written part of the exam by email within one week after having received the results from IML. In this email, the date, time and location of the oral part of the examination are again communicated to the candidates who have successfully passed the written part. The candidates who have not passed the written part are not allowed to participate in the oral part of the exam. The secretary keeps the results protocols and joins them to the protocols of the oral examination. Organisation of the oral exam session The responsible for the specialist certification exam prepares in high confidentiality a series of examination sets, according to the number of candidates registered (at least 3 sets); the set of material for one candidate is put in an envelope and consists of (see example in appendix 7): one paper case (clinical history, laboratory values, etc.) aimed to receive a written assessment from the candidate (specialised consultation in Clinical Pharmacology and Toxicology), one scientific paper to be read by the candidate, along with some suggestions for the discussion with examiners, an instruction for the conduct of the oral exam session. prepares a couple of short questions about regulatory aspects in clinical pharmacology for the examiners. prepares evaluation grids corresponding to the questions as shown in appendix 8. has the questions validated and approved by the Commission. appoints pairs of examiners from the Commission, according to an appropriate schedule, so that none of them has to examine trainees from his own center. appoints the assessor(s) in charge of writing the exam protocol for each candidate. sends the scientific papers aimed to serve for examination to the examiners (alternatively, the examiners may be proposed to select papers themselves). organises the audio recording of each session (digital audio recording is allowed). organises the session with the assessor(s) and the examiners, so that the candidates have access to a personal computer connected to the internet, SOP Specialist Exam, © SSCPT 2013 page 6 / 8 the following books, or online access to the following data collections: Martindale, the Swiss collection of drug labels (Summary of Product Characteristics; www.swissmedicinfo.ch) and Meyler’s Side Effect of Drugs, sufficient amounts of papers and pens, a pocket calculator. The language for questions and written information is English, answers are accepted in German, French, Italian or English, depending on the candidates and examiners. The assessor writes the examination protocol, but does not otherwise play an active role during the interview. The interview is entirely recorded on an audio tape or electronically (the record is intended to be used only in case of a controversy). The two oral examiners and the assessor check the exam material and prepare the questions on their own side. welcome the candidate(s) and check their identity. have the candidate draw at random one envelope among all available sets of material (recycling of the material is possible between different candidates). give each candidate 1:30 hour to work out the questions. examine them individually during 30 minutes each, thus leaving 15 more minutes for the examiners to discuss their evaluation in the absence of the candidate. each examiner fills an individual evaluation grid with percentages of realisation of training objectives in each tested item. after completion of the interview with the candidate, both examiners compare their respective evaluations and fill a common evaluation grid. based on the percentage degree of realisation of training objectives, and on the global evaluation of the examiners, a decision on exam success or failure is taken in the end of the session. the common evaluation grid is then signed by both examiners and by the assessor. The assessor sends the evaluation grid to the SSCPT secretary, along with the corresponding examination set identified with the name of the candidate, with the protocol and with the audio tape. The secretary collects the protocols of the written and oral examinations for each candidate, along with all related materials. prepares the summary reports for the candidates within three weeks after the date of the oral examination. These summary reports should contain the marks for the written and the oral examinations; the exam is passed if both the written and the oral exam have been successfully passed, according to the rules set above; see example in appendix 9. gives feed-back to the responsible for the specialist certification exam, and asks him and the president of the SSCPT to review and validate the procedure and to countersign the summary reports. sends the summary reports to the successful candidates, along with a copy of the result form of the written part and the joint evaluation grid of the oral part. SOP Specialist Exam, © SSCPT 2013 page 7 / 8 sends a copy of the result form of the written part of the exam to the non-successful candidates. joins to the letters an invitation to apply for membership in the SSCPT. sends copies of the summary reports, result forms and evaluation grids to the Exam Commission. sends a list of exam results to the FMH. ensures the proper archiving of all exam materials in individual records; the audio tape is erase-protect and sealed in an envelope. The candidate sends a copy of the examination certificate to the FMH along with other documents required for the submission of a title request. Inspection of examination records The candidate has has 60 days after reception of the exam results to express any opposition in a letter written to the FMH Commission of Post-graduate Titles. In case of (planned) objections of a candidate who has not passed the written exam, or who, after having successfully passed the written exam, has failed in the oral exam, this candidate may wish to see the documentation of the objected part of the exam for the preparation of the objection. Those parts of the written or oral exam which are in question will be made available to the candidate upon written request to the Exam Commission of the SSCPT according to the regulations of the SIWF / ISFM (Schweizerisches Institut für ärztliche Weiter- und Fortbildung / Institut suisse pour la formation médicale postgraduée et continue), which has issued guidelines for this case (published on the SIWF /ISFM homepage). The process of the handling of the objection recommended by the SIWF / ISFM will be followed without modification. SOP Specialist Exam, © SSCPT 2013 page 8 / 8