PREPARING EFFECTIVE MULTIPLE
CHOICE EXAMINATION QUESTIONS
PATRICK DUFF, M.D.
LEARNING OBJECTIVES
 Describe
the levels of complexity of
an examination question
 Describe the types of examination
questions
 Describe desirable and undesirable
features of examination questions
KEY REQUIREMENTS FOR AN
EXAMINATION
REQUIREMENT
DEFINITION
Validity
Measures what it is
supposed to measure
Reliability
Provides reproducible
and consistent results
Discrimination
Distinguishes between
the excellent, very
good, and fair student
COMPLEXITY OF EXAMINATION
QUESTIONS
CLASSIFICATION
DESCRIPTION
Recall
Recall of a specific fact
Interpretation
Identification and
interpretation of
illustrative material
Recall, interpretation,
and practical
application
Problem solving
TYPES OF EXAMINATION
QUESTIONS
TYPE
REMARKS
C – either, or, both,
neither
Unacceptable
K – select among
several alternatives
Unacceptable
True – false
Unacceptable
TYPES OF EXAMINATION
QUESTIONS
TYPE
REMARKS
A – select the one
best answer
Ideal question –
appropriate for recall,
interpretation, problem
solving
Appropriate for recall,
interpretation, and
simple management
problems
B – matching
EXAMPLE
RECALL
 Which
of the following is the most
important risk factor for squamous
cell carcinoma of the lung?
A.
B.
C.
D.
Age > 50
Diabetes
Smoking
Obesity
EXAMPLE
INTERPRETATION

A 24 year old nulliparous woman at 16
weeks gestation had the following values
on her serum analyte screening test:
AFP …below the mean
HCG…above the mean
Estriol…below the mean
Inhibin…above the mean
EXAMPLE
INTERPRETATION

Which of the following chromosome
abnormalities is her baby most likely to
have?
A.
B.
C.
D.
Triploidy
Trisomy 21
45, X
Trisomy 18
EXAMPLE
INTERPRETATION

A term neonate has
hepatosplenomegaly,
thrombocytopenia,
and the diffuse rash
noted in the
photograph.
EXAMPLE
INTERPRETATION
 Which
likely?
 A.
congenital infection is most
Parvovirus
 B. CMV
 C. HIV
 D. Mumps
EXAMPLE
PROBLEM SOLVING

A 22 year old male has a 48 hour history
of malaise, fever (39 deg C), shaking
chills, and cough productive of rustcolored sputum. On physical examination,
there is dullness to percussion in the lower
left lung field, and tubular breath sounds
can be heard in this area.
EXAMPLE
PROBLEM SOLVING
 Which
of the following is most
appropriate for treatment of this
patient?
A.
B.
C.
D.
Isoniazid
Fluconazole
Ceftriaxone
Acyclovir
APPROPRIATE MIX OF
EXAMINATION QUESTIONS
Recall
Interp
Problem
GUIDELINES FOR AN EFFECTIVE
TYPE-A QUESTION
 Determine
a single clear objective
prior to writing the question
 Have only one objective per question
 Make sure the question is at an
appropriate level of difficulty for the
target audience
GUIDELINES FOR AN EFFECTIVE
TYPE-A QUESTION
 Correct
response should be obvious
to the well-informed examinee before
the answers are read
 If the examinee has to read all of the
answers before deciding, the question
is probably a K-type “disguised” as an
A-type
TYPE A QUESTION
GOOD EXAMPLE
 Which
of the following is the most
common complication of multiple
pregnancy?
A. Gestational diabetes
B. Placenta previa
C. Postpartum hemorrhage
D. Preterm delivery
BEWARE THE PSEUDO-A QUESTION
 All
of the following statements about
preterm delivery are true except
 The
frequency is 10 %.
 It can be prevented by use of
progesterone.
 It is more common in whites than in
African-Americans.
 It may result in neonatal RDS.
GUIDELINES FOR AN EFFECTIVE
TYPE-A QUESTION
 Include
sufficient information in
the stem, but avoid unnecessary
verbiage
 Be certain the question has a
clear, unambiguous answer
GUIDELINES FOR AN EFFECTIVE
TYPE-A QUESTION
Include
4 to 5 possible
answers
Be certain the incorrect
answers are reasonably
plausible
TYPE A QUESTION
BAD EXAMPLE
 What
is the risk of pregnancy loss
after a second trimester
amniocentesis?
A. 0.5%
B. 10 %
C. 25 %
D. 50 %
GUIDELINES FOR AN EFFECTIVE
TYPE-A QUESTION
Do
not use “none of the
above” or “all of the above”
as distractors
Confusing
Rewards
guessing
GUIDELINES FOR AN EFFECTIVE
TYPE-A QUESTION
Avoid
clues in the responses
such as answers of unequal
length or repetition of key
words
GUIDELINES FOR AN EFFECTIVE
TYPE-A QUESTION
Avoid
compound
answers, ie, answers
that include multiple
actions
TYPE A QUESTION
BAD EXAMPLE

Which of the following is the most
appropriate treatment for communityacquired pnemonia?
 A.
Clindamycin
 B. Vancomycin
 C. Valacyclovir
 D. Administer oxygen, check blood gases, and
administer levofloxacin
GUIDELINES FOR AN EFFECTIVE
TYPE-A QUESTION
Be
wary of “except” type
questions, especially when a
“double negative” is present
GUIDELINES FOR AN EFFECTIVE
TYPE-A QUESTION
 In
certain instances, an “except”
type question may be appropriate
and may accurately assess the
breadth of an examinee’s
knowledge
TYPE A QUESTION
GOOD EXAMPLE

All of the following are risk
factors for gestational diabetes
except
A. Obesity
B. Positive family history
C. Age < 25 y
D. History of a macrosomic child
GUIDELINES FOR AN EFFECTIVE
TYPE-A QUESTION
 Be
certain that illustrative
materials are clear and that they
reproduce well
 Vary the correct response among
the letters a - d
GUIDELINES FOR AN EFFECTIVE
TYPE-B QUESTION
 Clearly
identify
the nature of the
items in column
A and column B
 Include at least 1
or 2 unused
distractors
GUIDELINES FOR AN EFFECTIVE
TYPE-B QUESTION
 Indicate
if answers in one of the
columns may be used only once,
more than once, or not at all
 Be certain the matching items are
reasonably related to one another
GUIDELINES FOR AN EFFECTIVE
TYPE-B QUESTION
 Matching
items are particularly useful
for testing interpretation skills
 X-ray
 ECG
 Sonogram
 Histology
TYPE B QUESTION
GOOD EXAMPLE
COLUMN A
Antibiotics
1. Clindamycin
COLUMN B
Target Organisms
a. Bacteroides species
2. Penicillin
b. Staphylococci
3. Gentamicin
c. E. coli
d. Group B
streptococci
TYPE B QUESTION
BAD EXAMPLE
DRUG
PRINCIPAL SIDE
EFFECT
1. Nifedipine
a. Marrow suppression
2. Ampicillin
b. Decreased libido
3. Fluoxetine
c. Rash
d. Hypotension
CONCLUSIONS
 Use
only type-A and type-B
questions
 Minimize recall questions and
maximize questions that test
interpretive and problem-solving
skills
CONCLUSIONS
 Be
certain that each question has
a single and specific objective
 Be certain the questions can be
answered in the time allotted