Jack B. Monpas-Huber, Ph.D.
Director of Assessment & Student Information

What are the poles of a magnet called?
a. Anode and cathode b. North and south c. Strong and weak d. Attract and repel
Stem
Responses

Easy for students to take
Easy to score
Can quickly assess a broad range of content
Distractors can have diagnostic (formative) value
Efficient
*Stiggins, R. J., Arter, J. A., Chappuis, J., & Chappuis, S. (2006). Classroom assessment for learning: Doing it right-using it well . Educational Testing Service.

Limitations of multiple-choice items
Guessing can skew score
“With three options, the probability of randomly guessing the correct answer is
0.33. However, the probability of getting two of these three-option test items right by random guessing alone is 0.11, three correct is 0.04, four correct is
0.01 and five correct is 0.004”
(Downing, 2006, p. 291-2)
Can be hard and time-consuming to craft plausible distractors
Not as useful for measuring other kinds of proficiencies we value
Downing, S. M. (2006). Selected-response item formats in test development. Pp. 287-301 in Handbook of Test Development , edited by Downing, S.M. and Haladyna, T. M. (Eds.). Lawrence Erlbaum.
Stiggins, R. J., Arter, J. A., Chappuis, J., & Chappuis, S. (2006). Classroom assessment for learning: Doing it right-using it well .
Educational Testing Service.

To assess mastery of knowledge
Recall of factual knowledge
Some procedural knowledge
To assess some reasoning skills i.e., compare/contrast, inference, cause and effect
*Stiggins, R. J., Arter, J. A., Chappuis, J., & Chappuis, S. (2006). Classroom assessment for learning: Doing it right-using it well . Educational Testing Service.

Some general guidelines for multiple-choice items*
State the whole question in the item stem
Eliminate repetition of material in response options
Be sure there is only one correct or best answer
Keep response options brief and parallel
Make all response options the same length
Limit use of “all of the above” or “none of the above”
Use “always” and “never” with caution
*Stiggins, R. J., Arter, J. A., Chappuis, J., & Chappuis, S. (2006). Classroom assessment for learning: Doing it right-using it well . Educational Testing Service.

Multiple-choice items on the Reading MSP*
*Office of the Superintendent of Public Instruction. (2010). Test and item specifications for grades 3-high school Washington state reading assessment . Retrieved on October 31, 2010 from http://k12.wa.us/Reading/Assessment/default.aspx.

Multiple-choice items on the Math MSP*
*Office of the Superintendent of Public Instruction. (2010). Measurements of student progress test and item specifications: Grades 6-8 mathematics . Retrieved on October 31, 2010 from http://k12.wa.us/Mathematics/TestItemSpec.aspx

1. Start with a learning objective
A single point or problem that can be stated as a proposition (a statement of the most important facts, concepts, or understanding we think every student should have at the end of instruction):
Knowledge propositions (statements of learned fact)
Manifest Destiny was first used as a justification for annexing Texas to the United States.
Manifest Destiny represents a belief that was used to justify the taking of Native American lands.
Reasoning propositions (states the result of reasoning applied to info at hand)
Identify knowledge to be applied, apply the pattern of reasoning, and state result as a declarative sentence:
Example: “Compare/Contrast” reasoning about “Reasons settlers went west”
The Mormons went west to practice their religion without persecution, whereas the settlers in Texas went west because land was cheap or free and they wanted a place to start over
Both Mormons and settlers in Texas were searching for a better life
In order to assess reasoning, must provide with a context different than that in which the students practiced. Otherwise we are assessing memory rather than transfer .
Kehoe, J. (1995). Writing multiple-choice test items. Practical Assessment, Research & Evaluation, 4(9) . Retrieved October 30,
2010 from http://PAREonline.net/getvn.asp?v=4&n=9 .
Stiggins, R. J., Arter, J. A., Chappuis, J., & Chappuis, S. (2006). Classroom assessment for learning: Doing it right-using it well .
Educational Testing Service.

2. Write a good stem
Identify one point or one problem
Try to make it an incomplete statement or a direct question, and avoid familiar phraseology
Try not to use negatives
Try to avoid giving clues to the correct answer
Don’t offer superfluous information as an introduction to a question.
Kehoe, J. (1995). Writing multiple-choice test items. Practical Assessment, Research & Evaluation, 4(9) . Retrieved October 30,
2010 from http://PAREonline.net/getvn.asp?v=4&n=9 .

3. Write some response options
Three good ones is sufficient. Based on a meta analysis of 80 years of research, “MC items should consist of three options, one correct option and two distractors. Using more options does little to improve item and test score statistics and typically results in implausible distractors” (Rodriguez, 2005, p.11, quoted in Downing, 2006, 292)
The correct answer should be unequivocal to students who know the material and do well on the test overall
Construct distractors that are similar in length, complexity, and grammatical form to the correct answer.
Don’t use “all of the above.” Recognition of one wrong option eliminates “all of the above” and recognition of two right options identifies it as the answer.
Downing, S. M. (2006). Selected-response item formats in test development. Pp. 287-301 in Handbook of Test Development , edited by Downing, S.M. and Haladyna, T. M. (Eds.). Lawrence Erlbaum.
Kehoe, J. (1995). Writing multiple-choice test items. Practical Assessment, Research & Evaluation, 4(9) . Retrieved October 30,
2010 from http://PAREonline.net/getvn.asp?v=4&n=9 .

Evaluating the quality of multiple-choice items:
Judgment
Content considerations
What exactly is the purpose of this item? Does it accurately represent the content? Is it essential to measuring the learning target?
Wording / Bias
Is there anything in the wording (apart from the learning objective) that could affect how the students responds? How can we word the item more clearly and succinctly?

Evaluating the quality of multiple-choice items:
Item analysis data
Item difficulty (or easiness)
Percent of students scoring item correctly (or “p-value”). What is your expectation for an item’s easiness? What if 90% of students score an item correctly? 10%? Depending on purpose , might want to include mostly items of moderate difficulty (40-60% correct), with just a few easy and difficult items included.
Item discrimination
Items should discriminate those who understand from those who don’t. Itemtotal correlation is one tool for seeing item discrimination. A high correlation would mean high achieving students (on total test score) are getting item correct, low achieving students are missing the item. Want items to correlate with total score at least .30. Definitely don’t want really low or negative correlation (means kids are mostly guessing).
Possibly Scantron machine provides these or similar statistics; if not, they are easy enough to do in Excel.

Downing, S. M. (2006). Selected-response item formats in test development. Pp. 287-301 in Handbook of Test Development , edited by Downing, S.M. and Haladyna, T. M. (Eds.).
Lawrence Erlbaum.
Frary, R. B. (1995). More multiplechoice item writing do’s and don’ts. Practical
Assessment, Research & Evaluation, 4(11) . Retrieved October 30, 2010 from http://PAREonline.net/getvn.asp?v=4&n=11 .
Kehoe, J. (1995). Writing multiple-choice test items. Practical Assessment, Research &
Evaluation, 4(9) . Retrieved October 30, 2010 from http://PAREonline.net/getvn.asp?v=4&n=9 .
Office of the Superintendent of Public Instruction. (2010). Measurements of student progress test and item specifications: Grades 6-8 mathematics . Retrieved on October 31,
2010 from http://k12.wa.us/Mathematics/TestItemSpec.aspx
Office of the Superintendent of Public Instruction. (2010). Test and item specifications for grades 3-high school Washington state reading assessment . Retrieved on October 31,
2010 from http://k12.wa.us/Reading/Assessment/default.aspx.
Stiggins, R. J., Arter, J. A., Chappuis, J., & Chappuis, S. (2006). Classroom assessment for learning: Doing it right-using it well . Educational Testing Service.