Writing Objective Test Questions Xiufeng Liu, PhD Director, Office of Educational Innovation & Assessment Professor, Department of Learning & Instruction What is objective assessment? A test consisting of questions or items that are highly structured and have only one right or best answer determined in advance. Examples: Multiple-choice True-false Matching Hierarchy of Learning Outcomes Objective Assessment is Effective for Measuring Lower Levels of Cognitive Learning Outcomes Knowing Recall a definition, a term or a fact Recognize a symbol Understanding Explain how things work Interpret a diagraph Compare and contrasts two entities Applying Use a procedure to solve a problem Make a prediction GUIDELINE FOR WRITING MULTIPLE-CHOICE QUESTIONS Guideline 1: The stem of an item should be meaningful by itself and should present a definite problem South America… a. Is a flat, arid country. b. Imports coffee from the United States. c. Has a larger population than the United States. d. Was settled mainly by colonists from Spain*. Most of South America was settled by colonists from? a. England b. France c. Holland d. Spain* Guideline 2: Make all choices plausible to uninformed students What are electrons? a. Negative particles* b. Positive particles c. Neutral particles d. Mechanical tools What are electrons? a. Negative particles* b. Positive particles c. Neutral particles Guideline 3: The stem should be concise, free of irrelevant materials, and include as much information as possible Most of South America was settled by colonists from Spain. How would you account for the large number of Spanish colonists settling there? a. They were adventurous. b. They were in search of wealth.* c. They wanted lower taxes. d. They were seeking religious freedom. Why did Spanish colonists settle most of South America? a. They were adventurous. b. They were in search of wealth.* c. They wanted lower taxes. d. They were seeking religious freedom. Guideline 4: Avoid extraneous clues to the correct and incorrect choices An electric transformer can be used a. for storing electricity. b. to increase the voltage of alternating current.* c. it converts electrical energy into mechanical energy. d. alternating current is changed to direct current. An electric transformer can be used to a. store electricity. b. increase the voltage of alternating current.* c. convert electrical energy into mechanical energy. d. change alternative current to direct current. Guideline 5: Avoid using the “none of the above”, “all of the above” and “I don’t know” alternatives Which of the following agencies should you contact to find out about a tornado warning in your locality? a. Farm bureau b. Post office c. Weather Bureau d. None of the above* Which of the following agencies should you contact to find out about a tornado warning in your locality? a. Farm bureau b. Post office c. Radio station* c. Weather Bureau Practice Write a multiple-choice question on any topic you are familiar with. Exchange with the person sitting next to you to critique each other’s question. Guidelines for Writing Matching Questions 1. Use only homogeneous materials in both the premises and responses 2. Include an unequal number of responses and premises, and instruct students that responses may be used once, more than once, or not at all 3. Arrange the list of responses in a logical order when appropriate A Sample Matching Question On the line to the left of each achievement listed in column A, write the letter of the person’s name in Column B who is noted for that achievement. Each name in Column B may be used once, more than once, or not at all. Column A _A_ 1. Invented the telephone _B_ 2. Discovered America _C_ 3. First United States astronaut to orbit the earth _F_ 4. First president of the United States Column B A. Alexander Graham Bell B. Christopher Columbus C. John Glenn D. Abraham Lincoln E. Ferdinard Magellan F. George Washington G. Eli Whitney Guidelines for Writing T-F Questions 1. Avoid broad general statements if they are to be judged true or false. Poor: The president of the United States is usually elected to that office. T F Better: The president of the United States is elected to that office. T F 2. Avoid double negative statements. Poor: None of the steps in the experiment was unnecessary. T F Better: All of the steps in the experiment were necessary. T F 3. If opinion is stated, attribute it to some source. Poor: Adequate medical care can be best provided through socialized medicine. T F Better: The American Medical Association favors socialized medicine as the best means of providing adequate medical care. T F Guidelines for Writing T-F Questions 4. Avoid complex sentences Poor: Whales are mammals because they are large. T Better: Whales are mammals. T F F 5. Avoid long sentences Poor: Despite the theoretical and experimental difficulties of determining the exact pH value of a solution, it is possible to determine whether a solution is acid by the red color formed on litmus paper when it is inserted into the solution. T F Practice Write a true-false and a matching question on a topic you are familiar with. Exchange with the person sitting next to you to critique each other’s questions. Useful Resources 1. Anderson, L. W.; Krathwohl, D. R. (eds.) (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. New York: Longman 2. Bloom, B. S., et al., (eds.). (1956). Taxonomy of educational objectives: Handbook I: Cognitive domain. New York: David McKay Company. 3. Haladyna, T. M., & Downing, S. M. (1989a). A taxonomy of multiple-choice itemwriting rules. Applied Measurement in Education, 2(1), 37-50. 4. Haladyna, T. M., & Downing, S. M. (1989b). Validity of a taxonomy of multiple-choice item-writing rules. Applied Measurement in Education, 2(1), 51-78. 5. Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiplechoice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309-334.