Writing MCQs
Susan Chamberlain
Dept of OBS/GYN
Queen’s University
Writing MCQs
At the end of this session, the participant will
be able to
 Identify different types of MCQ items
 Describe the structure of an MCQ
 List the technical flaws common to MCQ’s
 write one well-structured MCQ testing
appropriate content
Writing MCQs
At the end of this session, the participant will
be able to
 Identify different types of MCQ items
 Describe the structure of an MCQ
 List the technical flaws common to MCQ’s
 write one well-structured MCQ testing
appropriate content
 All of the above
MCQ Check-list
 Why
we write them
 What type is best
 What flaws are to be avoided
 What content to test
 How to construct it
Why do we test?
Communicate to the students what material
is important (formative vs summative)
 Motivate students to study
 Identify areas of deficiency in need of
remediation or further learning
 Determine final grades or make promotion
decisions
 Identify areas where the course/curriculum
is weak

What Should Be Tested?
Exam content should match course
objectives
 Important topics should be weighted more
heavily than less important topics
 The testing time devoted to each topic
should reflect the relative importance of the
topics
 The sample of items should be
representative of the instructional goals

What Should Be Tested?

Sampling
“Performance on the sample provides a basis
for estimating achievement in the broader
domain that is actually of interest.”

Questions sample:
– TOPICS
– SKILLS ie determining diagnosis, next step in
management, etc.
MCQ ‘Items’
Should test important information – either
common or life-threatening
 Should be well-structured

– Flaws benefit test-wise examinees
– Irrelevant difficulty does not test examinees’
specific knowledge
Anatomy of an MCQ Item
Stem
 Lead-in
 Options

– One correct answer
– Distractors
Types of MCQ’s
True/False family
 Assess recall of isolated facts
– Stems must be clear and unambiguous
– Options must be absolutely true or false or
examinee must decide how true options are

Generally not recommended – usually have
subtle flaws
Types of MCQ’s: True/False
Which of the following statements is true regarding
contraception?
A) oral contraceptives should not be initiated until six weeks
after therapeutic abortion to avoid the risk of
hypercoagulability
B) an intrauterine device can provide good contraceptive
protection 5 years after insertion
C) oral contraceptives should not be prescribed to individuals
under the age of 15 without parental consent
D) lesbian women do not require counseling about contraception
or sexually transmitted diseases
E) oral contraceptives should not be prescribed for women over
the age of 35
Types of MCQ’s
One-Best-Answer family (A type)
 Options may be diagrammed:
D C A
Least correct


E
B
Most correct
Correct answer is the most likely correct
Better application of knowledge, integration,
synthesis and judgment
Types of MCQ’s:
One-Best-Answer family
A 24 year old university student came to see you because she is
missing several days of school each month because of severe
dysmenorrhea unresponsive to the OCP and NSAIDS. She also
complains of deep dyspareunia and has a tender uterosacral
nodule on the pelvic examination. Which one of the following
conditions is most likely responsible for her symptoms?
A) Chronic pelvic inflammatory disease
B) Endometriosis
C) Adenomyosis
D) Fibroids
MCQ Check-list
Why we write them
What type is best
 What flaws are to be avoided
 What content to test
 How to construct it
MCQ Technical Flaws
Issues Related to Test-Wiseness
 Issues Related to Irrelevant Difficulty

MCQ Technical Flaws

Issues Related to Test-Wiseness
– provide clues to the right answer
– examinee can eliminate options based on flaws
and not on knowledge

Issues Related to Irrelevant Difficulty
Item Flaws: Test-wiseness
Grammatical cues
 Logical cues
 Absolute terms
 Long correct answer
 Word repeats
 Convergence strategy

Item Flaws: Test-wiseness
Grammatical cues
 One or more distractors don’t follow
grammatically from the stem
Item Flaws: Test-wiseness
Grammatical cues
A 24 year old patient presents with a one year
history of dysmenorrhea, mid-cycle pain,
dyspareunia, and painful bowel movements with
her periods. The most likely diagnosis is:
A)
B)
C)
D)
Fibroid uterus
Adenomyosis
Endometriosis
Psychosomatic condition
Item Flaws: Test-wiseness
Logical cues
 A subset of the options are collectively
exhaustive
Item Flaws: Test-wiseness
Logical cues
Arterial blood gases in a normal pregnant
woman, compared to her non-pregnant state may
show which of the following?
A)
B)
C)
D)
E)
Decreased pO2
Increased pCO2
Decreased pCO2
Decreased pH
No difference in pCO2
Item Flaws: Test-wiseness
Logical cues
 One or more distractors don’t follow
logically from the stem
Item Flaws: Test-wiseness
Logical cues
An important tool for assessing fetal well-being is
the Non Stress test. A well baby demonstrates
the following fetal heart rate pattern:
A)
B)
C)
D)
E)
A baseline heart rate above 120 bpm
Evidence of fetal movement
Baseline variability
Two fetal heart rate accelerations in 20 minutes
Fetal breathing movements
Item Flaws: Test-wiseness
Absolute terms
 Always, never
 Not a problem if stem is long and lead-in
asks a precise question
Item Flaws: Test-wiseness
Absolute terms
In patients with a previous Cesarean section:
A) Vaginal birth is NEVER an option
B) Repeat Cesarean section is less dangerous to the
mother
C) The uterine scar may rupture during labour
D) Care should ALWAYS be supervised by an
obstetrician
E) No further pregnancies should EVER be considered
Item Flaws: Test-wiseness
Long correct answer
 Correct answer is longer, more specific, or
more complete than other options
Item Flaws: Test-wiseness
Long correct answer
Which of the following problems in vaginal
breech delivery is most likely to cause serious
long-term complications?
A)
B)
C)
D)
Failure to descend
Cord prolapse
Nuchal arms
Entrapment of the after coming head with an
incompletely dilated cervix
E) 1 minute Apgar score of 4
Item Flaws: Test-wiseness
Word repeats
 A word or phase is included in the stem and
in the correct answer
Item Flaws: Test-wiseness
Word repeats
A 24 year old with a previous history of chronic
PID presents with primary infertility. Select the
procedure that would be most appropriate to
evaluate the status of her tubes and pelvic
anatomy
A) Post-coital test
B) Endometrial biopsy
C) Laparoscopy with dye insufflation of the fallopian
tubes
D) Hysterosalpingogram
Item Flaws: Test-wiseness
Convergence strategy
 The correct answer includes the most
elements in common with the other options
Item Flaws: Test-wiseness
Convergence strategy
Normal bladder neck mobility is measured by:
A) A Q-tip angle measurement under 40 degrees
B) A Q-tip angle measurement under 30 degrees
C) A bladder neck ultrasound angle less than 30
degrees
D) A Q-tip angle measurement under 20 degrees
MCQ Technical Flaws

Issues Related to Test-Wiseness

Issues Related to Irrelevant Difficulty
– do not assess knowledge of intended topic
Irrelevant Difficulty







Options are long, complicated or double
Extraneous details in stem
Numeric data not stated consistently
Vague frequency terms
Non-parallel language in options
Stems are tricky or unnecessarily complicated
‘None of the above’
Irrelevant Difficulty
Options are long, complicated or double
Extraneous details in stem
 Shifts difficulty from content knowledge to
speed reading
Irrelevant Difficulty
Options are long, complicated or double
Mrs. Brown is a 70 year old woman who had a stroke 6 months
ago and is still impaired. Her daughter tells you that her
mother’s recent urinary incontinence makes her care
unmanageable. She is seeking permanent placement. Mrs.
Brown complains of urinary frequency, nocturia, and urge
incontinence. Which of the following statements is true?
A)
B)
C)
D)
E)
Neurological conditions are involved in the majority of women with
symptoms like those presented by Mrs. Brown
Diabetes has a greater impact on quality of life than Mrs. Brown’s
condition.
The pharmacological blockage of muscarinic receptors in the detrusor
muscle controls symptoms compatible with Mrs. Brown’s condition
Overactive bladder is defined as urinary incontinence accompanied by
or immediately preceded by urgency
Alpha stimulants are effective in treating this condition
Irrelevant Difficulty
Extraneous details in stem
You see in your office a 24 year old woman for her routine postpartum
visit, at 6 weeks. Her baby is irritable and cries “all the time’ (including
during her visit.) As she is changing the diaper during her visit, you
notice that the child has a very red bum but upon questioning, she tells
you she did not notice anything abnormal. Her partner is away in
Afghanistan and she does not have a lot of friends, as they just recently
were posted in Kingston. She states that she has a hard time getting
dressed in the morning, cries easily and feels like she does not love her
child. She denies any medical or psychiatric history. Her exam is
essentially normal. She weighs 2 kg less than at her first antenatal visit
at 10 weeks. You most likely clinical diagnosis is:
A) Postpartum hypothyroidism
B) Hyperthyroidism
C) Postpartum blues
D) Postpartum depression
E) Psychotic depression
Irrelevant Difficulty
Numeric Data
 Not stated consistently
 Mixed format
 Illogical order
Irrelevant Difficulty
Numeric Data not stated consistently
Following a second episode of salpingitis, what
is the likelihood that a woman is infertile?
A)
B)
C)
D)
E)
Less than 20%
20 – 30 %
Greater than 50 %
90%
75%
Irrelevant Difficulty
Vague frequency terms
 Often, usually, rarely, frequently
 Not consistently defined
*may be used in stem if otherwise carefully
constructed unambiguous question
Irrelevant Difficulty
Vague frequency terms
Gonorrhea is
A) sometimes asymptomatic
B)
C)
D)
E)
treated with cefixime 450 mg orally
caused by a gram positive intracellular diplococcus
diagnosed by a DNA test
usually associated with vulvar ulcers
Irrelevant Difficulty
Non-parallel language in options
 Non-logical order

Irrelevant Difficulty
Non-parallel language in options
In a vaccine trial, 200 2-year-old boys were given a vaccine
against a certain disease and then monitored for five
years for occurrence of the disease. Of this group, 85%
never contracted the disease. Which of the following
statements concerning these results is correct?
A) No conclusion can be drawn, since no follow-up was
made of non-vaccinated children
B) The number of cases (ie 30 cases over five year) is
too small for statistically meaningful conclusions
C) No conclusions can be drawn because the trial
involved only boys
D) Vaccine efficacy (%) is calculated as 85-15/100
Irrelevant Difficulty
“None of the above”
 Problematic where judgment is involved or
if options not absolutely true or false
 Turns question into true/false question
 Confuses very knowledgeable students
Irrelevant Difficulty
“None of the above”
The main reason that fertility is reduced in a
woman in her early 40’s is:
A) Irregular cycles make it hard to time intercourse
B) Chromosomal damage adversely affects oocytes’
genetic complement
C) Male erectile difficulties increase
D) Hormone imbalance leads to poor endometrial
development
E) None of the above
Irrelevant Difficulty
Stems are tricky or unnecessarily complicated
Arrange the parents of the following children with Down’s syndrome
in order of the highest to lowest risk of recurrence. Assume that the
maternal age in all cases is 22 years and that a subsequent pregnancy
occurs within 5 years. The karyotypes of the daughters are:
I: 46, XX, –14,+T (14q21q) pat
II: 46, XX, –14,+T (14q21q) de novo
III: 46, XX, –14,+T (14q21q) mat
IV: 46, XX, –21,+T (14q21q) pat
V: 46, XX, –21,+T (21q21q) (parents not typed)
A) III, IV, I, V, II
B) IV, III, V, 1, II
C) III, I, IV, V, II
D) I, V, III, I, V, II
E) III, IV, I, II, V
MCQ Check-list
Why we write them
What type is best
What flaws are to be avoided
 What content to test
 How to construct it
What to test?
Items classified by cognitive processes required to
answer the question:
 Recall (Memory)– tests knowledge of isolated facts
 Interpretation (Comprehension)– review information
to reach conclusion, eg diagnosis
 Problem solving (Reasoning) – situation requires
action, eg next step in management
– Process depends on background of trainee
What to test?
Simpler classification based on task:
 Application of knowledge item **
– Reach conclusions
– Make predictions
– Select course of action

Recall item
– Test knowledge of isolated facts
What to test?

Test application of knowledge using clinical
vignettes
– Adds ‘face-validity’
– Usually selects more important, less trivial
– Identifies student who’ve memorized but do not
understand
BUT experience in patient care should not
be necessary
What to test - summary




Focus item on key concepts and principles that are
essential information (without access to
references) for all examinees to understand
Test material that is relevant to learning in clinical
clerkships and beyond
Avoid items that only require recall of isolated
facts
Avoid esoteric or interesting topics that are not
essential
MCQ Check-list
Why we write them
What type is best
What flaws are to be avoided
What content to test
 How to construct it
Writing the Questions
Construct stem –include all necessary
information to arrive at the right answer
 Choose lead-in – pose a clear question
 Choose distractors

Stem*
Usually a clinical scenario
 Clear, unambiguous
 Should be long relative to options
 Includes all pertinent information

–
–
–
–
Patient’s age/gender
Clinical setting
Complaint
Other important info – history, physical
findings, test results
Lead-in*
Follows stem
 In the form of a question
 Should relate to the stem
 Should be answerable without looking at
options

– “Cover the Options test”
Distractors
Each should be selected by some, therefore
all plausible, none obviously incorrect
 3 or 4 choices

– Common misconceptions
– Faulty reasoning
Distractors






Homogeneous in content to the correct answer
Should be in same category as correct answer–
diagnosis, treatment, diagnostic test
Clearly incorrect or inferior to the correct answer
Plausible and attractive to the uninformed
Similar to the correct answer in construction and
length
Grammatically consistent and logically compatible
with the stem
General Guidelines for Item
Construction



Include as much of the item as possible in the
stems: Long stems, short options
Avoid negatively phrased items (except or not in
the lead-in). If you must, use only with short
options
Avoid writing question of the form:
– Which of the following statements is correct?
– Each of the following statements is correct EXCEPT
» Unfocussed
» Heterogeneous options
“Top-down” versus “Bottom-up”
Top-Down
 Knowledge or recall question
 Cite a disease and then ask what patient
findings are expected
 Structured like textbooks
Clinically backward
“Top-down” versus “Bottom-up”
Bottom-up
 Application of knowledge
 Gives findings and asks examinee to
indicate the disease
 Examinees need to be able to synthesize
information from several pages of a
textbook (or notes) to answer these
questions
Clinically realistic
Guidelines for Clinical Sciences
ie Clerkship and beyond…
Test application of knowledge using clinical
vignettes to pose medical decisions in
patient care situations
 Focus items on common or potentially
catastrophic problems
 Pose clinical decisions that would be
expected of a successful examinee
 Avoid clinical situations that would be
handled by a (sub)specialist

MCQ Check-list
Why we write them
 What type is best
 What flaws are to be avoided
 What content to test
 How to construct it

Basic Rules for MCQ’s (A type)*
Focus item on important concept
 Item should assess application of
knowledge, not recall of an isolated fact
 Stem must pose clear question – ‘Cover the
options’ test
 All distractors should be homogeneous
 Avoid technical item flaws

Reference
Constructing Written Test Questions For the
Basic and Clinical Sciences
– Case & Swanson, NBME 1996