Guidelines for Writing Effective Multiple-Choice Exam Items
1. Defining a Question Objective:
A. Select a fact, idea or concept to be tested.
Due to time and structural limitations, we have to prioritize learning content for testing.
Exam writers should focus on key or representative ideas or concepts. Item relevance
depends on the degree to which questions relate back to course and, ultimately, program
objectives.
Question Objectives ◄ Course Objectives ◄ Program Objectives
B. Decide the level of learning to be tested.
Exam writers need to determine the desired level of mastery for the selected concept.
According to a widely-used learning taxonomy by Bloom, the abilities to recall and
comprehend facts and ideas are low-order cognitive skills while the abilities to apply,
synthesize and evaluate concepts and theories involve higher-order learning. Learning
levels must realistically match instruction.
Since our students are very good at ascertaining what they need to do to get a desired
grade, testing for higher-order learning of key ideas will encourage them to spend more
of their finite resources (time and effort) mastering the skills and concepts that will
ultimately make them good physicians.
2. Crafting a Question:
A. Writing the stem:
The most effective stems will set up situations or problems that require students to think
like an osteopathic physician.
Well-written stems should:
•
•
•
•
•
•
•
•
•
•
clearly anticipate an answer;
set up the targeted cognitive task;
be straightforward, well-focused and concise;
be expressed grammatically in the simplest possible way;
avoid unnecessary jargon;
provide no more or less information than is required to answer;
be expressed as a full sentence;
avoid negatives whenever possible and never use double negatives;
not provide unintended cues; and
be completely factually, conceptually, grammatically and stylistically correct.
B. Creating Distractors:
To discriminate between different degrees of learning, it is necessary to list a set of
effective distractors along with a clearly stated correct answer. Effective distractors
1
reflect common misconceptions and/or partial understandings of the targeted concept.
All answer choices should be:
•
•
•
•
•
•
•
plausible;
conceptually related;
independent;
equally varied;
grammatically consistent;
similar in length and appearance; and
listed in alphabetical or logical order.
In conformance with NBOME standards, answer choices such as “All of the above,”
“None of the above” and “Both A and B” are strongly discouraged.
3. Reviewing Questions:
All question items need to be carefully reviewed before being placed on an exam.
A. Individual Review:
Exam writers should reexamine their own items several days or more after writing them.
Start by comparing them against the guidelines listed above. It is very useful to get
feedback from colleagues both within and outside of course groups.
B. Group Review:
Course groups need to collectively review items within the context of a broader exam.
They should especially look for:
•
•
•
•
reoccurring patterns among questions
stems or answer choices that provide clues for other items
undue weighing toward particular content areas
proportion of items testing lower verses higher-order learning
2