TEST DEVELOPMENT
Ginny Price
CETL
WRITING MULTIPLE CHOICE ITEMS
•
Write only a few items at a time
• Immediately after preparing class lesson or after class discussions
• Focus on important points
• Spread out work time increasing depth of questions
• Build a bank of questions on Angel
• Include a few new items each time test is given
• Use item analysis to determine worth of test items
ANATOMY OF ITEM
•
Stem
•
Responses
• Correct response
• Three or four plausible distractors
STEM
•
Students can read and understand stem
•
Students can formulate an answer without reading responses
•
Can be written as an incomplete sentence or a question
•
Focus on important concept or principle
•
Trivial details or obscure facts should not be used
DISTRACTORS
•
Use four responses unless doing so would require using distractors that are implausible
•
Use student misconceptions and common errors to write plausible distractors
• Give a short answer quiz to obtain sample of students’ incorrect responses to use as
distractors
GUIDELINES
•
Stem presents problem clearly and concisely
• Include a verb in the statement
• Contains all information necessary to understand problem
•
Responses should not contain repetitive words or phrases.
• Include these items in the stem
•
Write correct response first then distractors
• Best answer (reduces disagreements)
• Agreed upon answer by authorities in field
• Correct answer
• Facts: anatomy, terminology
GUIDELINES 2
•
Avoid giving clues to answers
• Measure test taking skills not learning
• Reduces test’s validity
•
Types of clues
• Length of responses: correct answer is either longer or shorter than distractors
• Verbal association: using similar words in stem and correct response
• Grammatical clue: correct answer is only grammatically correct one
• Specific determiners: always, never, all, none
• Instead use: usually, sometimes, typically, maybe
GUIDELINES 3
•
Use positive statements as negative statements confuse students
• If negative statements are used the negative word should be bold and in caps
•
Refrain from using all of the above
• too easy
• often the correct response
•
Refrain from using none of the above options
• Do not use with negative stem; creates confusing double negative
• Does not reinforce correct answer as correct answer is not present
• Use only with correct answer format instead of best answer (absolutes)
• Can be used to increase item difficulty
GUIDELINES 4
•
Each item is independent
• No clues in one item for other items
• No item dependent on previous item’s response
•
For responses: use a logical order or randomize
• Avoids patterns
• Response “A” rarely correct answer; “C” most often correct answer
•
Cite authority in stem if item controversial
•
Avoid quoting source verbatim in item
• Encourages memorization instead of understanding
•
Arrange responses in vertical column
CONCENTRATIONS
•
Purpose of test
• Reading material
• Understanding material
• Synthesizing material
• Applying material
• Analyzing material
•
Can adjust weights of items based on purpose
•
Important concepts and principles in course
•
Develop novel or thought provoking situations in items
•
Use students’ misconceptions as distractors
TECHNIQUES
•
Premise → consequences
•
Premise + premise → consequence
•
Analogy
•
Classification
•
Proposition 1 & proposition 2 → 1 correct but not 2, 2 correct but not 1, both correct, both
incorrect
•
Context dependent item set
PREMISE → CONSEQUENCES
•
p→c
• A premise (stem)
• Followed by consequences (responses)
• Evaluate
•
Example:
• What could happen if a liquid is instilled into an animal’s external ear canal without
determining the patency of the animal’s tympanic membrane?
• Damage to the animal’s hearing
• What must be examined before instilling substances into an animal’s external ear
canal?
• Patency of the tympanic membrane
PREMISE + PREMISE → CONSEQUENCE
•
•
p+p→c
•
Premise + premise (stem)
•
Followed by a consequence (responses)
•
Evaluate
Example:
•
If an animal presented with a body temperature of 104.0°F and they walked to the hospital
with their owner on a 90°F day it can be concluded the hyperthermia was caused by
• Environmental factors
• Stress
• Infection
• Shock
ANALOGY
•
a:b=c:d
• a is to b as c is to d
• Precise relationships between two sets of words
• Analysis
•
Example:
• Operant conditioning is to voluntary as classical conditioning is to
• Involuntary
• Association
• Habituation
• Salivating
CLASSIFICATION
•
Classifies terms, names and statements
• Comprehension
•
Example
• Pavlov would be classified as a
• Behaviorist
• Physiologist
• Biologist
• Sociologist
P1 P2 → 1 CORRECT BUT NOT 2, 2 CORRECT
BUT NOT 1, BOTH CORRECT, BOTH INCORRECT
•
Two propositions are presented in the stem. It must be decided whether both are true,
neither are true, A but not B is true or B but not A is true.
•
Example:
• A fearful dog with passive defense reflexes comes into the hospital for an exam. This
dog will exhibit (A) appeasement body language (B) agonistic body language.
• A. A but not B
• B. B but not A
• C. neither A nor B
• D. both A and B
CONTEXT DEPENDENT ITEM SET
•
A stimulus is used to generate a set of questions
• Written scenario, graph, chart, map, picture, table, article
• Measures high level cognitive skills: comprehension, analysis and application
• Use a novel stimulus to gain an idea of student’s quality of learning
•
Example on next slide
CONTENT DEPENDENT ITEM
The purpose of the porous tape in this
picture is to
•
prevent the catheter from sliding out of
the bladder
•
allow sutures to be placed attaching
the tape to the skin
•
prevent the animal from chewing out
the catheter
•
allow the technician to better monitor
catheter length when inserting it
RESOURCES
•
Jacobs. L.C. & Chase, C.I. (1992). Developing and using tests effectively: A guide for
faculty.