How to Construct Good Multiple-Choice Test Questions Step One: Determine what content you want to test. The questions should be based on the objectives for your session, which are based on the course’s objectives. Each question should test the application of important concepts, not recall of trivial facts. Step Two: Write the question so that it neither favors the test-wise students nor unnecessarily confuses others. One-best-answer multiple-choice questions test student knowledge much more effectively than other formats, and leave the least amount of room for confusion. True/false questions (i.e., indicate all responses that are appropriate or which of the following options is NOT correct), on the other hand, leave room for interpretation and should be avoided. Refer to “General Guidelines” section below. Step Three: Recruit a few of your colleagues to review your questions. Do not provide your colleagues with answers to the questions. This will help you determine whether the questions are confusing, ambiguous, poorly worded, focus on minutia or are too difficult. You may also find it helpful to have a colleague "score" the importance of the points being tested (5=core, important concept; 1=minutia, irrelevant, above the level of the students). General Guidelines for Multiple-Choice Test Question Construction Write the stem (the question) so the question can be answered without looking at the options. Include as much of the information in the stem as possible; stems should be long and the options short. Avoid superfluous information. Avoid "tricky" and overly complex items. Write options that are grammatically and logically compatible with the stem and each other; list them in logical or alphabetical order. Write options that are plausible and the same relative length as the answer. Avoid using absolutes such as always, never and all in the options; also avoid using vague terms such as usually and frequently. Avoid negatively phrased items (e.g., those with except or not in the lead-in). If you must use a negative stem, use only short (preferably single word) options. Common Technical Flaws Issues Related to “Test-wiseness”: Providing grammatical cues (one or more of the options do not follow grammatically from the stem). Providing logical cues (a subset of the options is collectively exhaustive). Using absolute terms in some of the options (i.e., "always" or "never"). Making the correct answer longer or more complete than the other options. Repeating a word or phrase in the answer that is included in the stem question. Allowing for use of the convergence strategy (i.e., the correct answer includes the most elements in common with the other options). Issues Related to Irrelevant Difficulty: Making options long, complicated or duplicative Not stating numeric data consistently Stating the terms in options vaguely (i.e., "rarely" and "usually") Not making the language in the options parallel Not listing the options in logical order Using "None of the above" as an option Writing tricky or unnecessarily complicated stems Hinging the answer of one item to the answer of a related item Sample Test Questions: A 65-year-old man has difficulty rising from a seated position and straightening his trunk, but he has no difficulty flexing his leg. Which of the following muscles is most likely to have been injured? A. B. C. D. E. Gluteus maximus* Gluteus minimus Hamstrings Iliopsoas Obturator internus An otherwise healthy 33-year-old man has mild weakness and occasional episodes of steady, severe abdominal pain with some cramping but no diarrhea. One aunt and one cousin have had similar episodes. During an episode, his abdomen is distended, and bowel sounds are decreased. Neurological examination shows mild weakness in the upper arms. These finding suggest a defect in the biosynthetic pathway for: A. B. C. D. E. F. Collagen Cortiocosteroid Fatty acid Glucose Heme* Thyroxine An asymptomatic 50-year-old woman has hypertension. Urinary excretion of catecholamines is increased. A CT scan shows a suprarenal mass. Which of the following findings is most likely on microscopic examination of the resected mass? A. B. C. D. E. F. Benign neoplasm of the adrenal cortex Benign neoplasm of the adrenal medulla* Malignant neoplasm of the adrenal cortex Malignant neoplasm of the adrenal medulla Diffuse hyperplasia of the adrenal cortex Diffuse hypoplastia of the adrenal medulla (Insert patient vignette describing a patient with a problem.) Which of the following is the most likely gastrointestinal lesion? A. Candidal esophagitis F. Pyloric stenosis B. Diverticulitis G. Regional enteritis C. Hiatal hernia H. Subphrenic abscess D. Peptic ulcer I. E. Pseudomembranous enterocolitis * Denotes correct answer Ulcerative colitis