Completion, Short-Answer, and True-False Items Educational Assessment of Students Chapter 7 Anthony J. Nitko (Brenda Stewart) Three Fundamental Principles for Crafting Assessments Focus each assessment task entirely on important learning targets Craft each assessment task to elicit from students only the knowledge and performance that are relevant to the learning targets you are assessing Craft each assessment task to neither prevent nor inhibit a student’s ability to demonstrate attainment of the leaning targets you are assessing Short-Answer Items Require a word, short phrase, number, or symbol response. Three types of short-answer items: Question – What is the capital of Texas? Completion – The capital of Texas is _________. Association – On the blank next to the state name, write the abbreviation for that state. TexasTX California CA Usefulness of Short-Answer Items Abilities Assessed Lower-order thinking skills Recall Comprehension of information Higher-level abilities Make simple interpretations of data and applications of rules Ability to solve numerical problems in science and mathematics Ability to manipulate mathematical symbols and balance mathematical and chemical equations Usefulness of Short-Answer Items Strengths Relatively easy to construct Can be scored objectively Lowers the probability of getting the answer correct by random guessing Shortcomings Not free of subjectivity Cannot anticipate all possible responses Tends to lower the reliability of the obtained score Crafting Short-Answer Items Importance of what is assessed How does it match the test blueprint Question format is the preferred format for a short-answer item. Focus the item on the specific knowledge sought Word the items specifically and clearly Put the blank near the end of the sentence If possible do not copy statements verbatim Crafting Short-Answer Items A completion item should omit important words and not trivial words Limit blanks to one or two Keep all blanks the same length Specify the precision you expect in your answer Avoid irrelevant clues True-False Items Consists of a statement or proposition the student may judge as true or false Six varieties: True-False Yes-No Right-Wrong Correction Multiple True-False Yes-No with explanation Usefulness of True-False Items Advantages Certain aspects of the subject matter readily lend themselves to verbal propositions that can be judged true or false Relatively easy to write Scored easily and objectively Can cover a wide range of content within a relatively short period Criticisms Poorly constructed assess only specific, frequently trivial facts Ambiguously worded Answered by random guessing Encourage students to study and accept only oversimplified statements of truth and factual details Usefulness of True-False Items Generalizations in a subject area Comparisons among concepts Casual or conditional propositions Relationships between two events, concepts, facts, or principles Explanations for why events or phenomena occurred Instances or examples of a concept or principle Usefulness of True-False Items Evidential statements Predictions about phenomena or events Steps in a procedure of process Computations (or other kinds of results obtained from applying a procedure) Evaluations of events of phenomena Validity of True-False Item Format Essence of educational achievement is the command of useful verbal knowledge All verbal knowledge can be expressed in propositions Proposition is any sentence that can be said to be true or false Particular area of knowledge is indicated by success in judging truth or falsity of propositions Guessing on True-False Items: An Ill Wind? Guessing – getting 50% correct Blind (complete random guessing) quite unlike informed guessing (based on partial knowledge) Well-motivated guess blindly only on a small percentage of questions on a test Difficult to obtain a good score on a test by blind guessing only A given true-false test has a high reliability coefficient – evidence blind guessing does not seriously affect the scores Guessing on True-False Items: An Ill Wind? Random guessing – sometimes called blind guessing Partial knowledge – make an informed guess Higher than 50-50 chance of success Measures of partial knowledge Full knowledge – 100% chance of answering correctly Percentages of correct answers go down with the number of blind guesses made Suggestions for Getting Started Properly Create pairs of items, one true and one false Rephrase evaluative judgments into a comparative statement Write false statements that reflect the actual misconceptions held by students Convert a multiple-choice item into two or more true-false items Suggestions for Improving True-False Items Importance of what is assessed How does it match the test blueprint Assess important ideas, rather than trivia, general knowledge, or common sense Make sure it is either definitely true or definitely false Use short statements when possible Use exact language Use positive statements and avoid double negatives Suggestions for Improving True-False Items Avoid copying sentence verbatim True and false statements should have approximately the same number of words Don’t present in a repetitive or easily learned pattern Do not use verbal clues that give away the answer Attribute the opinion in a statement to an appropriate source Focus on one idea Crafting Multiple True-False Items Looks like a multiple-choice item – followed by several alternatives No single correct or best answer Response to each alternative is true or false Each alternative is scored correct or incorrect Crafting Multiple True-False Items Advantages Two or three multiple true-false responses in the time it takes to make one multiple-choice response Multiple true-false tests created from multiple-choice items have higher reliability than original multiplechoice Can assess same abilities as straight multiple-choice items that are crafted to assess parallel content Crafting Multiple True-False Items Advantages Students believe multiple true-false items do a better job of assessing knowledge than multiple-choice items Students perceive them to be harder May be easier to write – not limited to one correct answer Limitations Shares some of same limitations as multiple-choice Works Cited Nitko, A. J., (2004). Educational Assessment of Students, 4th ed. Upper Saddle River, NJ: Pearson Education, Inc.