organization of the examinations

advertisement
Standard Operating Procedure
for the Preparation and Execution of the Examinations for
Specialist Certification in
“Clinical Pharmacology and Toxicology”
Version 2.2, approved 24 MAY 2013
This document has been approved by the Executive Committee of the Swiss Society for Clinical
Pharmacology and Toxicology (SSCPT). It is mainly intended for the Committee members in charge of
preparing the specialist certification examination for candidates to the FMH title in ‘Clinical
Pharmacology and Toxicology’. It may be freely distributed, and it is available on the website of the
SSCPT (www.clinpharm.ch). Please check that you have the latest version.
General overview
The examination procedures for the specialist certification examination for candidates for the
title “Specialist for Clinical Pharmacology and Toxicology FMH” compose of a written part
and an oral part. The regulation and modalities of the examinations are laid down in chapter
4 of the postgraduate training programme “Specialist for Clinical Pharmacology and
Toxicology” of January 1, 2001, last revision September 6, 2007, which can be obtained in
German and French from the website of the Society or of the FMH.
Introduction and delegation of responsibilities
The Swiss Society for Clinical Pharmacology and Toxicology (SSCPT) has the responsibility
to organise once per year a Specialist Certification exam session for MD candidates to the
specialist title in “Clinical Pharmacology and Toxicology” issued by the Swiss Medical
Association (FMH). The results of the exam are transmitted to the FMH, which is responsible
for the validation of the entire training programme and the attribution of the specialist title.
The SSCPT organises the exam according to its postgraduate training programme.
The Executive Committee of the SSCPT delegates the exam-related tasks to several
persons; the list is updated each year (appendix 1). Persons to be named are:







a responsible for the specialist certification examination (member of the executive
committee)
a person in charge of maintaining the pool of questions for the written exam
at least 2 independent assessors for the written exam (must have the FMH specialist
title)
2 to 4 examiners for the oral exam (must have the FMH specialist title and hold
responsibilities in the post-graduate training of MDs)
1 or 2 assessors for the oral exam (must have the FMH specialist title)
a responsible for secretarial duties (SSCPT secretary)
a responsible for financial aspects (SSCPT treasurer)
SOP Specialist Exam, © SSCPT 2013
page 1 / 8
The responsible person for the specialist certification exam, the assessors for the written
exam and the examiners for the oral exam compose the “Exam Commission” of the SSCPT.
At least one member of this commission must represent category B training sites. The
language regions of Switzerland should be represented appropriately.
Announcement and registration
The annual exam session is announced at least 6 month before the date. The announcement
contains the following information:




the date and place of the written examination session
the date and place of the oral examination session (may also be announced directly
to the candidates not later than one month before the oral examination session)
the address accepting registrations and the deadline for registration (usually at least 3
months before the written exam session)
the fee requested for registration and how the fee has to be paid
The announcement is sent to the FMH (Secretariat SIWF, Elfenstrasse 18, POB 170, CH3000 Bern 15, Tel 031 359 11 11, Fax 031 359 11 12, E-mail siwf@fmh.ch).
The FMH publishes the announcement in the Swiss Medical Journal and on its website. The
SSCPT publishes the announcement on its website.
The responsible for secretarial duties



receives the registrations of candidates on the appropriate registration form
(appendix 2)
prepares a list with names, addresses, phone numbers, e-mail addresses
sends a postage mail to the candidates after closure of the registration deadline, to
acknowledge their participation, invoice their exam fees and inform them on the
schedule and practical details of the exam.
The written exam
Composition of the written exam
The written exam is carried out as a web-based examination and consists of 120 questions.
The questions can be Multiple Choice (MCQ) or Short Answer Questions (SAQ). 1 Two types
of MCQ are in use: type A (one-best-answer of five options) and type Kprim (four options,
each can be true or false). The so-called blueprint of the exam (appendix 3) identifies the
content areas covered by the exam and specifies the proportion of questions requested to
cover each area.
The collection of written exam questions is stored in the Webpool, a web-based tool for the
storage, collection and revision of the pool of exam questions maintained by the Institute of
Medical Education (Institut für Medizinische Lehre, IML) of the University of Bern.
1
Specialist in Clinical Pharmacology and Toxicology, Postgraduate Training Programme of 1. January
2001, last revision 6. September 2007, art 4.4.1
SOP Specialist Exam, © SSCPT 2013
page 2 / 8
For the exam, 120 questions will be selected and transferred into the Online Assessment
Tool, through which the candidates will perform the exam online using a Web Browser.
Maintenance of the written exam questions pool
The person in charge of maintaining the pool of questions for the written exam maintains with
high confidentiality a pool of written multiple choice (MCQ) and short answer questions
(SAQ) that have undergone validation by a panel of experts in clinical pharmacology and
toxicology. The questions expert panel (the “proof reading committee”) is composed of one
delegate of each postgraduate training institution in Switzerland and the person in charge of
maintaining the pool of questions (all must be members of the SSCPT). The person in
charge of the pool makes sure that the question pool covers all chapters of the theoretical
content of postgraduate training in sufficient excess (ideally more than 3-fold) compared with
the exam “blueprint” and that the questions are up to date.
Each year, a subset of questions is to be renewed. Therefore, the person in charge invites
the members of the questions expert panel


to provide a series of new questions in the various chapters, according to the needs
to check a selection of questions out of the pool on correctness of content and
answer. For SAQ clear correction criteria must be provided.
The questions are centralized by the person in charge of the pool. The person in charge
organizes a validation session. The validated questions are included in the pool.
Selection of the set of questions for the written exam
The IML adds an exam folder in the Webpool. The person in charge of the pool selects 120
questions out of the actualised pool according to the distribution given in the “blueprint”
(appendix 3) and adds them to the exam folder within the time frame prespecified by the IML.
A final control of the questions is made by the person in charge of the question pool. Once
the definitive selection is made, the person in charge of the pool authorises the IML to
transfer the questions from the Webpool to the Online Assessment Tool.
Preparation of the written exam session
The person responsible for the specialist certification examination







organizes a sufficiently large room with a sufficient number of computers with internet
access for the conduct of the written part of the exam.
checks the payment of exam fees by each candidate in cooperation with the treasurer
and the secretary of the SSCPT.
sends a date confirmation to each candidate approximately 4 weeks before the
written exam date, with reminders about the examination procedures (e.g. to bring an
identification card to the exam, to read this document on the SSCPT website).
prepares sheets of reference mathematical formulae which will be distributed to the
candidates; such a formulary is shown in appendix 4.
prepares sufficient reserve of paper and pens, and pocket calculators for the exam.
sends exam data (date, time) to IML.
sends candidate data (first name, last name) to the IML to prepare their access to the
Online Assessment Tool.
SOP Specialist Exam, © SSCPT 2013
page 3 / 8

sends assessor data (first name, last name) to the IML to prepare their access to the
Online Assessment Tool.
The IML transfers the exam questions from the Webpool to the Online Assessment Tool and
provides a test access to the person responsible for the exam session.
The person responsible for the exam session verifies that the technical installations and
software products work properly and that the online assessment tool can be properly
accessed.
As soon as the exam is tested, the IML deactivates the test access and prepares the exam
access. The IML sends the access data for the candidates to the person responsible for the
exam session. The latter stores them in a secure place until the start of the exam.
Additionally, IML provides a manual for the online assessment tool for the candidates.
Conduct of the written exam session
The online assessment is made available to the candidates during the pre-specified
examination time through a secure web access at the examination site. The questions
appear in a random order, so that all candidates have a different order of questions. All
answers are to be entered in the Online Assessment Tool during the duration of the exam in
the examination room. The Online Assessment Tool also provides space for comments to
the assessors. The individual time for each candidate is exactly 4 h. The online assessment
tool automatically stops accepting entries after this time.
The person responsible for the exam session







welcomes the examinees and checks the identity of the candidates.
writes a time protocol about the conduct of the exam (e.g. documentation of toilet
times, system interrupts, etc.).
distributes the manuals for the online assessment tool, the mathematical formulae
sheets, papers, pencils, pocket calculators and the access codes to the candidates.
supervises the examination (verifies the absence of any forbidden communication of
the examinees between one another or with external sources of information, e.g.
books, phones, etc.).
checks the examination time (which is set to exactly 4 hours by the online
assessment tool); announces the time 2 hours and 3:30 hours after the start of the
exam.
contacts immediately the IML emergency support if any change of the duration of the
exam must take place (e.g. because of interruption of the exam).
collects carefully all exam material after the end, so that questions cannot be
disseminated outside.
The IML


activates the access for the candidates according to the time frame agreed with the
person responsible for the exam session.
provides an IT-specialist on stand-by for emergency support during the exam.
SOP Specialist Exam, © SSCPT 2013
page 4 / 8
Analysis of the exam data and publications of the results
The IML

sends access data for the two assessors to the person responsible for the specialist
certification examination
Analysis of MCQ
The IML


extracts the results from the Online Assessment Tool.
provides access for the 2 assessors to the comments given by the candidates.
The 2 assessors


read the comments of the candidates.
decide whether questions, which have been criticized by one or several candidates
during the online exam, are invalid and must be excluded from the evaluation.
Analysis of SAQ
The IML

grants access for the 2 assessors to the correction mode of SAQ.
The 2 assessors




rate the SAQs independently from one another in the online assessment tool after
having received the invitation for rating by IML. A sample SAQ with answers and
rating scheme is given in appendix 6.
confront their results between one another, and converge to a common evaluation in
case of disagreement.
read the comments of the candidates.
decide whether questions, which have been criticized by one or several candidates
during the online exam, are invalid and must be excluded from the evaluation.
Completion of the analysis
The 2 assessors




inform IML that the evaluation process has been completed.
inform IML, which questions have been eliminated (if any).
inform the person in charge of the pool, which questions have been eliminated and
why (if any).
keep strictly confidential their copies of answer forms up to the end of the exam
procedure, and destroy them thereafter, if any.
The IML



includes, after validation by the two assessors, the results of the SAQ in the results of
the written exam.
excludes, after the two assessors have convened to do so, invalid questions from the
evaluation.
divides the total number of correct answers of valid questions by the number of valid
questions: the candidates having answered correctly at least 57% of valid questions
(i.e. >56%) will be considered as having passed the written exam.
SOP Specialist Exam, © SSCPT 2013
page 5 / 8


provides a result form of the written exam for each candidate, which shows in detail
the percentage results according to the blueprint topics along with the overall
outcome (passed/not passed) of the written exam. This form is to be transferred to
the person responsible for the specialist exam and to the secretary (see template in
appendix 5).
provides an audit track in lieu of a protocol for the evaluation procedures of the exam.
Feedback to the candidates
The person responsible for the specialist exam informs the candidates about the overall
outcome of the written part of the exam by email within one week after having received the
results from IML. In this email, the date, time and location of the oral part of the examination
are again communicated to the candidates who have successfully passed the written part.
The candidates who have not passed the written part are not allowed to participate in the oral
part of the exam.
The secretary

keeps the results protocols and joins them to the protocols of the oral examination.
Organisation of the oral exam session
The responsible for the specialist certification exam









prepares in high confidentiality a series of examination sets, according to the number
of candidates registered (at least 3 sets); the set of material for one candidate is put
in an envelope and consists of (see example in appendix 7):
 one paper case (clinical history, laboratory values, etc.) aimed to receive a
written assessment from the candidate (specialised consultation in Clinical
Pharmacology and Toxicology),
 one scientific paper to be read by the candidate, along with some suggestions
for the discussion with examiners,
 an instruction for the conduct of the oral exam session.
prepares a couple of short questions about regulatory aspects in clinical
pharmacology for the examiners.
prepares evaluation grids corresponding to the questions as shown in appendix 8.
has the questions validated and approved by the Commission.
appoints pairs of examiners from the Commission, according to an appropriate
schedule, so that none of them has to examine trainees from his own center.
appoints the assessor(s) in charge of writing the exam protocol for each candidate.
sends the scientific papers aimed to serve for examination to the examiners
(alternatively, the examiners may be proposed to select papers themselves).
organises the audio recording of each session (digital audio recording is allowed).
organises the session with the assessor(s) and the examiners, so that the candidates
have access to
 a personal computer connected to the internet,
SOP Specialist Exam, © SSCPT 2013
page 6 / 8
 the following books, or online access to the following data collections:
Martindale, the Swiss collection of drug labels (Summary of Product
Characteristics; www.swissmedicinfo.ch) and Meyler’s Side Effect of Drugs,
 sufficient amounts of papers and pens,
 a pocket calculator.
The language for questions and written information is English, answers are accepted in
German, French, Italian or English, depending on the candidates and examiners. The
assessor writes the examination protocol, but does not otherwise play an active role during
the interview. The interview is entirely recorded on an audio tape or electronically (the record
is intended to be used only in case of a controversy).
The two oral examiners and the assessor









check the exam material and prepare the questions on their own side.
welcome the candidate(s) and check their identity.
have the candidate draw at random one envelope among all available sets of material
(recycling of the material is possible between different candidates).
give each candidate 1:30 hour to work out the questions.
examine them individually during 30 minutes each, thus leaving 15 more minutes for
the examiners to discuss their evaluation in the absence of the candidate.
each examiner fills an individual evaluation grid with percentages of realisation of
training objectives in each tested item.
after completion of the interview with the candidate, both examiners compare their
respective evaluations and fill a common evaluation grid.
based on the percentage degree of realisation of training objectives, and on the
global evaluation of the examiners, a decision on exam success or failure is taken in
the end of the session.
the common evaluation grid is then signed by both examiners and by the assessor.
The assessor sends the evaluation grid to the SSCPT secretary, along with the
corresponding examination set identified with the name of the candidate, with the
protocol and with the audio tape.
The secretary




collects the protocols of the written and oral examinations for each candidate, along
with all related materials.
prepares the summary reports for the candidates within three weeks after the date of
the oral examination. These summary reports should contain the marks for the written
and the oral examinations; the exam is passed if both the written and the oral exam
have been successfully passed, according to the rules set above; see example in
appendix 9.
gives feed-back to the responsible for the specialist certification exam, and asks him
and the president of the SSCPT to review and validate the procedure and to
countersign the summary reports.
sends the summary reports to the successful candidates, along with a copy of the
result form of the written part and the joint evaluation grid of the oral part.
SOP Specialist Exam, © SSCPT 2013
page 7 / 8





sends a copy of the result form of the written part of the exam to the non-successful
candidates.
joins to the letters an invitation to apply for membership in the SSCPT.
sends copies of the summary reports, result forms and evaluation grids to the Exam
Commission.
sends a list of exam results to the FMH.
ensures the proper archiving of all exam materials in individual records; the audio
tape is erase-protect and sealed in an envelope.
The candidate

sends a copy of the examination certificate to the FMH along with other documents
required for the submission of a title request.
Inspection of examination records
The candidate has has 60 days after reception of the exam results to express any opposition
in a letter written to the FMH Commission of Post-graduate Titles.
In case of (planned) objections of a candidate who has not passed the written exam, or who,
after having successfully passed the written exam, has failed in the oral exam, this candidate
may wish to see the documentation of the objected part of the exam for the preparation of
the objection. Those parts of the written or oral exam which are in question will be made
available to the candidate upon written request to the Exam Commission of the SSCPT
according to the regulations of the SIWF / ISFM (Schweizerisches Institut für ärztliche
Weiter- und Fortbildung / Institut suisse pour la formation médicale postgraduée et continue),
which has issued guidelines for this case (published on the SIWF /ISFM homepage). The
process of the handling of the objection recommended by the SIWF / ISFM will be followed
without modification.
SOP Specialist Exam, © SSCPT 2013
page 8 / 8
Download