Writing Better
MCQs
Why Multiple Choice Questions?
•
Most commonly used format in the College
•
Most commonly used format on licensing examinations
•
This session will NOT cover essays, fill-in-the-blank items, true/false,
or any other question types
Strengths of MCQ
•
Can assessment achievement of learning from simple to complex
•
More reliable than True/False items
•
Scoring is easy, reliable, and objective
•
Can cover a lot of material
Zimmaro (2004)
Limitations of MCQs
•
Time consuming to write good questions
•
Acceptable distractors are sometimes hard to find
•
Is not the best way to evaluate problem-solving
•
Sometimes there is more than one correct answer
•
May encourage guessing
Zimmaro (2004)
Why Good Questions Matter
•
Strengths of MCQs are only valid if constructed well
•
Use in national licensure and credentialing examinations
 Example: BPS examinations
•
Well written items can assess more than recall of knowledge
•
MCQs become “tricky” when rewritten from the original best format
Parts of a MCQ
•
Stem – the text of the question
•
Options – the choices provided after the stem
•
Key – the CORRECT answer in the list of options
•
Distracters – the INCORRECT answers in the list of options
Start with a Good Stem
•
The stem should have a single, definitive statement or question
•
It should be clear from the stem what is expected
•
The majority of the information should be in the stem, not in the key
•
Avoid unnecessary and irrelevant information
•
Use clear, straightforward language
•
Avoid use of negatives, and if you use them capitalize, underscore, or
bold to make it clear what you are asking (i.e. EXCEPT, not)
What’s wrong with this stem?
A nurse is assessing a client who has
pneumonia. Which of these
assessment findings indicates that
the client does NOT need to be
suctioned?
Which of these assessment findings,
if identified in a client who has
pneumonia, indicates that the client
needs suctioned?
a.
Absence of adventitious breath
sounds
a.
Diminished breath sounds
b.
*Absence of adventitious breath
sounds
b.
Respiratory rate of 18
breaths/minute
c.
Inability to cough up sputum
c.
*Inability to cough up sputum
d.
Wheezing following
bronchodilator therapy
d.
Wheezing prior to bronchodilator
therapy
What’s wrong with this stem?
Mitochondria evolved from freeliving bacteria that could carry our
oxidative phosphorylation. For this
reason, they have circular genomes
that reproduce independently of the
nuclear genome. What characteristic
is relatively constant in
mitochondrial genomes across
species?
a.
Content (i.e., types of genes)
b.
Organization
c.
Size
What characteristic is relatively
constant in mitochondrial genomes
across species?
a.
Content (i.e., types of genes)
b.
Organization
c.
Size
What’s wrong with this stem?
Polysaccarides
a.
Are made up of thousands of
smaller units called
monosaccharides
b.
Are NOT found in the aloe vera
leaf
c.
Are created during
photosynthesis
d.
Can be described by the chmical
formula CHHOH
Polysaccharides of the plant cell wall
are synthesized mainly in the
a.
Endoplasmic reticulum
b.
Cytosol
c.
Plasma membrane
d.
Golgi complex
Writing Keys and Distractors
•
There should be only one best answer.
Ideally there is only one right answer.
•
Maintain formatting and avoid
overlapping answers.
•
Avoid K-type and A-type items that
include multiple answers (i.e. All of
the above or Both A and C)
•
Don’t repeat words from the stem.
•
Avoid grammar clues.
•
Wrong answers should be plausible.
•
•
Avoid funny or silly distractors.
Avoid absolute statements (e.g.
Always, Never, All)
•
Keep the length of options the same.
•
Vary the location of the correct
answer.
What’s wrong with the
key/distractors?
During what age period is thumbsucking likely to produce the
greatest psychological trauma?
During what age period is thumbsucking likely to produce the
greatest psychological trauma?
a.
Infancy
a.
From birth to 2 years old
b.
Preschool period
b.
From 2 years to 5 years old
c.
Before adolescence
c.
From 5 years to 12 years old
d.
During adolescence
d.
From 12 years to 20 years old
e.
After adolescence
What’s wrong with the key/
distractors?
Albert Einstein was a:
Who was Albert Einstein?
a.
Anthropologist
a.
An anthropologist
b.
Astronomer
b.
An astronomer
c.
Chemist
c.
A chemist
d.
Mathematician
d.
A mathematician
What’s wrong with the key/
distractors?
Which of the following artists is
known for painting the ceiling of the
Sistine Chapel?
Which of the following artists is
known for painting the ceiling of the
Sistine Chapel?
a.
Warhol
a.
Botticelli
b.
Flinstone
b.
Da Vinci
c.
Michelangelo
c.
Michelangelo
d.
Santa Claus
d.
Raphael
What’s wrong with the key/
distractors?
The term operant conditioning refers
to the learning situation in which:
The term operant conditioning refers
to the learning situation in which:
a.
A familiar response is associated
with a new stimulus
a.
A familiar response is associated
with a new stimulus
b.
Individual associations are
linked together in sequence
b.
Individual associations are
linked together in sequence
c.
*A response of the learner is
instrumental in leading to a
subsequent reinforcing event
c.
*The learner’s response leads to
reinforcement
d.
Verbal responses are made to
verbal stimuli
d.
Verbal responses are made to
verbal stimuli
What’s wrong with the key/
distractors?
Who received a Noble Prize for
discovering the structure of DNA?
Who received a Noble Prize for
discovering the structure of DNA?
a.
Francis Crick
a.
Paul D. Boyer
b.
James Watson
b.
Francis H. C. Crick
c.
Rosalind Franklin
c.
Robert F. Curl Jr.
d.
A and B
d.
Rosalind E. Franklin
e.
B and C
e.
Dorothy C. Hodgkins
f.
A and C
f.
Sir Robert Robinson
Tips and Pearls
•
Good stem
•
One-best answer
•
Test application vs. recall of isolated facts
•
Cover the options rule
•
Options of similar length and structure
•
Good distractors
•
Avoiding
 Always, never, rarely, usually, none of the above, all of the above
Time to Review
Your Own Test
Resources
•
Constructing Written Test Questions for the Basic and Clinical
Sciences (NBME)
•
Dos and Don’ts of Writing MCQs
•
Good, Better, best: Multiple Choice Exam Construction
•
Writing Good Multiple Choice Test Questions
•
Writing Multiple Choice Questions: An Introductory Tutorial
(NBME)