English Language Arts and Mathematics Selected Response, Constructed Response, Technology-Enhanced Item Design Adapted from SBAC Discussion With a partner, talk about all you know about the Smarter Balanced Assessment Consortium. Learning Outcomes • Understand the purpose of selected response, constructed response, and technology-enhanced items • Describe the components of selected response, constructed response, and technology-enhanced items • Understand the essential requirements of quality items • Describe the expectations of what students will be able to do • Understand the districts initiative to begin with writing Claims ELA Claim 1 Claim 2 Claim 3 Claim 4 MATH Claim 1 Claim 2 Claim 3 Claim 4 Six Assessment Item Types 1. Selected Response 2. Constructed Response 3. Extended Response 4. Performance Task 5. Technology Enhanced 6. Technology Enabled SELECTED RESPONSE Benefits and Limitations of Selected Response Items • Benefits • Answered quickly • Assess a large range of content on one test • Inexpensive to score • Results collected quickly • Limitations • Limited ability to reveal a student’s reasoning process • Difficult to assess higher-order thinking skills Formats and Components of Selected Response Items ELA Lizards are fascinating creatures. There are over 3,000 known species, including monitors, skinks, geckos, andcreatures. iguanas, and theyare vary greatly appearance. largestmonitors, lizard, theskinks, Lizardschameleons, are fascinating There over 3,000inknown species,The including Komodo can grow ten feet and the smallest, the Jaragua lizard, canlizard, fit on the a dime. geckos,dragon, chameleons, andover iguanas, and long, they vary greatly in appearance. The largest Skinks usually have smooth scales like snakes, iguanas have mohawk-like crests running down Komodo dragon, can grow over ten feet long, and the smallest, the Jaragua lizard, can fit on their a dime. backs, and the moloch is covered with spikes from head to tail.mohawk-like Lizards vary crests in colorrunning from shades of Skinks usually have smooth scales like snakes, iguanas have down their gray and and brown bright is red or green, spotted striped. fourvary legsinbut some areshades leglessof backs, thetomoloch covered with spikes or from head Most to tail.have Lizards color from and easily confused with snakes (Hint: if it has external eardrums and eyelids it’s a lizard). Geckos gray and brown to bright red or green, spotted or striped. Most have four legs but some are legless can walk up walls. Chameleons not(Hint: only change but eardrums also have prehensile those and easily confused with snakes if it hascolor external and eyelidstails, it’s asimilar lizard).toGeckos of can monkeys, that wrap around branches and their eyes in different directions. walk up walls. Chameleons not only change colorcan butmove also have prehensile tails, similar to those of monkeys, that wrap around branches and their eyes can move in different directions. What is the best way to revise the highlighted sentence to match the language and style of the What is the best way to revise the highlighted sentence to match the language and style of the paragraph? paragraph? A. Geckos are able to adhere to flush surfaces because setae on their footpads facilitate van der Waals A. forces Geckosbetween are ablethe to adhere to flush surfaces setae structures and thebecause surface.setae on their footpads facilitate van der Waals forces betweenbecause the setae structures andtoes the surface. B. Geckos are awesome they have sticky that allow them to climb windows like B. Spiderman. Geckos are awesome because they have sticky toes that allow them to climb windows like Spiderman. C. Geckos have the remarkable ability to walk up walls thanks to tiny hair-like structures on their toes C. that Geckos the remarkable clinghave to smooth surfaces.ability to walk up walls thanks to tiny hair-like structures on their toes that clingup to walls smooth D. Geckos scurry likesurfaces. tiny dancers gliding effortlessly across a stage, their movements as D. natural Geckosas scurry up walls like dancers gliding effortlessly across a stage, their movements as a well-rehearsed tiny ballet. natural as a well-rehearsed ballet. STIMULUS STEM OPTIONS Formats and Components of Selected Response Items Read the passage and then answer the question. The playback rate can be sped up or slowed down. When NASA's Juno The National Aeronautics and Space Administration (NASA) leads research spacecraft launched on Aug. 5, 2011, users could look ahead to see the in space exploration and aeronautics. mission's five-year journey to Jupiter in a matter of seconds. NASA Gives Public New Internet Tool To Explore The Solar System Point of view can be switched from faraway to close-up to right "on board" spacecraft. Dozens of controls on a series of pop-up menus allow users to PASADENA, Calif. – NASA is giving the public the power to journey through fully customize what they see, and video and audio tutorials explain how to the solar system using a new interactive Web-based tool. use the tool's many options. Users may choose from 2-D or 3-D modes, with the latter simply requiring a pair of red-cyan glasses to see. The "Eyes on the Solar System" interface combines video game technology and NASA data to create an environment for users to ride along with agency "By basing our visualization primarily on mission data, this tool will help spacecraft and explore the cosmos. Screen graphics and information such as both NASA and the public better understand complex space science planet locations and spacecraft maneuvers use actual space mission data. missions," said Kevin Hussey, manager of Visualization Technology Applications and Development at JPL, whose team developed "Eyes on the "This is the first time the public has been able to see the entire solar system Solar System." and our missions moving together in real-time," said Jim Green, director of NASA's Planetary Science Division at the agency's Headquarters in "Eyes on the Solar System" is in beta release. It has been demonstrated at Washington. "It demonstrates NASA's continued commitment to share science conferences, in classrooms and at the 2011 South by Southwest our science with everyone." Interactive Conference in Austin, Texas. The virtual environment uses the Unity game engine to display models of planets, moons, asteroids, comets and spacecraft as they move through our solar system. With keyboard and mouse controls, users cruise through space to explore anything that catches their interest. A free browser plugin, available at the site, is required to run the Web application. Designers are updating "Eyes on the Solar System" to include NASA science missions launching during the coming months, including GRAIL to the moon and the Mars Science Laboratory Curiosity rover. Select the two highlighted sentences that show that NASA makes predictions about the data space missions will generate before the missions occur. "You are now free to move about the solar system," said Blaine Baggett, executive manager in the Office of Communication and Education at NASA's Jet Propulsion Laboratory (JPL) in Pasadena, Calif. "See what NASA's seeis–populated and where they right now all without leaving your and projected to 2050.” AND “When NASA's Juno spacecraft KEY:spacecraft “The tool withareNASA data– dating back to 1950 computer." launched on Aug. 5, 2011, users could look ahead to see the mission's five-year journey to Jupiter in a matter of seconds.” In order for Users NASAmay to include projected data to 2050 five-year mission of a spacecraft that just launched the agency must predict data experience missions in real-time, andand "Eyesthe on the Solar System" from missions before the missions occur. also allows them to travel through time. The tool is populated with NASA data datingThe backother to 1950 and projected to important 2050. Distractors: options provide information about the new internet tool but none of them support the inference that NASA makes predictions about the data space missions will generate before the missions occur. Formats and Components of Selected Response Items MATH • Traditional Selected Response Item Which number is both a factor of 100 and a multiple of 5? A. 4 A. 4 B. 40 B. 40 C. 50 C. 50 D. 500 D. 500 STEM Statement of the question DISTRACTOR OPTIONS: Possible answers the students must select from Non-Traditional Selected Response STIMULUS Item A multiplication problem is shown below. STEM 17 × 12 Which model(s) below could represent the solution to this problem? Select all that apply. A. B. C. (1×1)+(1×7)+(1×2)+(2×7) D. E. F. (17×2)+(17×1) Non-Traditional Selected Response Item Key and Distractor Analysis: A. Does not understand how to model multiplication of two two-digit numbers using area models. B. Correct C. Did not account for the values of the digits in the tens places. D. Correct E. Did not understand that the 1 represents 10 in the multiplication problem F. Showed multiplication of 17 and (1 + 2) instead of 17 and 12 Responses to this item will receive 0–2 points, based on the following: 2 points: B, D 1 point: Either B or D 0 points: Any other combination of selections. Non-Traditional Selected Response Item For numbers 1a – 1d, state whether or not each figure has ⅖ of its whole shaded. STEM 1a. 1b. MULTIPLE PARTS OPTIONS 1c. 1d. Non-Traditional Selected Response Item Scoring Rubric: Responses to this item will receive 0–2 points, based upon the following: • 2 points: YNYN The student has a solid understanding of ⅖ as well as the equivalent form of ⅖. • 1 point: YNNN, YYNN, YYYN The student has only a basic understanding of ⅖. Either the student doesn‘t recognize an equivalent fraction for ⅖ or doesn‘t understand that all 5 parts must be equal-sized in figure 1b. • 0 points: YYYY, YNNY, NNNN, NNYY, NYYN, NYNN, NYYY, NYNN, NNNN, NYNY, NNYN, NNNY. The student demonstrates inconsistent understanding of ⅖ or answers “Y” to figure 1d, clearly showing a misunderstanding of what ⅖ means. Figure 1d is considered a “disqualifier “and an answer of “Y” to this part of the item would cancel out any other correct responses as “guesses” on the part of the student. Non-Traditional Selected Response Item Match each shape below to its name. Scoring Rule: Students who properly match the four shapes to their name will receive two points. Students who make two or three correct matches will receive partial credit of one point All other connections will receive a score of 0. Non-Traditional Selected Response Item CONSTRUCTED RESPONSE Purpose of Constructed Response Items • Constructed Response Items • Address assessment targets and claims that are of greater complexity • Require more analytical thinking and reasoning Administration of Constructed Response Items • Administered during the computer-adaptive component • Scored using artificial intelligence • Most constructed response items take between 1 and 5 minutes to complete • Some more complex items may take up to 10 minutes to complete Components of a Constructed Response Item The Shepherd’s Boy and the Wolf The Shepherd’s Boy and the Wolf STIMULUS A Shepherd's Boy was tending his flock near a village, and thought it would be A Shepherd's Boy was tending his flock near a village, and thought it would be great fun to trick the villagers by pretending that a Wolf was attacking the great fun to trick the villagers by pretending that a Wolf was attacking the sheep: so he shouted out, "Wolf! Wolf!" and when the people came running sheep: so he shouted out, "Wolf! Wolf!" and when the people came running up he laughed at them because they believed him. He did this more than up he laughed at them because they believed him. He did this more than once, and every time the villagers found they had been tricked, for there was once, and every time the villagers found they had been tricked, for there was no Wolf at all. At last a Wolf really did come, and the Boy cried, "Wolf! Wolf!" no Wolf at all. At last a Wolf really did come, and the Boy cried, "Wolf! Wolf!" as loud as he could: but the people were so used to hearing him call that they as loud as he could: but the people were so used to hearing him call that they took no notice of his cries for help. And so no one came to help the boy, and took no notice of his cries for help. And so no one came to help the boy, and the Wolf attacked the sheep. the Wolf attacked the sheep. InInaafew fewsentences, sentences,explain explainwhat whatlesson lessonthe thereader readercan canlearn learnfrom fromthe the shepherd’s boy. Use details from the story to support your response. shepherd’s boy. Use details from the story to support your response. STEM Components of a Constructed Response Item 2-point Scoring Rubric 2 The response: • gives evidence of the ability to explain inferences about theme • includes specific inferences that make reference to the text • supports the inferences with relevant details from the text 1 The response: • gives limited evidence of the ability to explain inferences about theme • includes inferences but they are not explicit or make only vague references to the text • supports the inference with at least one detail but the relevance of that detail to the text must be inferred 0 A response gets no credit if it provides no evidence of the ability to explain inferences about theme and includes no relevant information from the text. Scoring Notes Response may include but is not limited to: The shepherd’s boy learned that he shouldn’t call wolf unless there is really a wolf. The shepherd’s boy learned that he should only ask for help if he needs it or else he wouldn’t get help when he really needs it. “The people were so used to hearing him call that they took no notice of his cries.” The shepherd’s boy learned not to have fun by tricking people because the people learn not to trust you. Score Point 2 Sample: The lesson learned from this story is do not cry for help when nothing is wrong. The shepherd’s boy pretends that a big wolf is attacking his sheep and yells, “Wolf! Wolf!” The people in the village run out to help him because they believe he needs help. After he tricks the villagers more than once, they realize he is just pretending. Score Point 1 Sample: The lesson learned from this story is do not cry for help when nothing is wrong. The shepherd’s boy cries wolf when there is no wolf and the people come to help him. Score Point 0 Sample: Readers learn a good lesson about how to cry wolf. Purpose of Constructed Response Items • Constructed Response items • Address assessment targets and claims that are of greater complexity • Require more analytical thinking and reasoning Administration of Constructed Response Items • Administered during the computer-adaptive component • Scored using artificial intelligence • Most constructed response items take between 1 and 5 minutes to complete • Some more complex items may take up to 10 minutes to complete Components of a Constructed Response Item A teacher asked her students to use estimation to decide if the sum of the problem below is closer to 4,000 or 5,000. 496 + 1,404 + 2,605 + 489 = STIMULI One student replied that she thinks the sum is closer to 4,000. She used the estimation shown below to support her reasoning. Is the student’s reasoning correct? In the space below, use numbers and words to explain why or why not. If the student’s reasoning is not correct, explain how she should have estimated. STEM Components of a Constructed Response Item Sample Top-Score Response: The student’s reasoning is incorrect. She was rounding to the thousands place. She had 2 numbers that were less than 500, and she decided to round these numbers to 0. This is like saying these numbers were not in the problem at all. She needs to account for these two numbers. Together, they have a sum that is very close to 1,000. I think adding 1,000 + 1,000 + 3,000 is a better strategy. This means the sum is closer to 5,000 than to 4,000. Scoring Rubric: Responses to this item will receive 0–2 points, based on the following: 2 points: Student has thorough understanding of how to estimate and how improper estimation can lead to flawed reasoning. Student states that the student in the scenario used reasoning that is incorrect and provides reasoning that shows a better estimation strategy. 1 point: Student has partial understanding of how to estimate and how improper estimation can lead to flawed reasoning. Student states that the student in the scenario used reasoning that is incorrect, but alternate estimation strategy is also flawed. 0 points: Student has little or no understanding of how to estimate and how improper estimation can lead to flawed reasoning. Student states that the student in the scenario used reasoning that is correct. TOP-SCORE SCORING RUBRIC Qualities of a Rubric • Focus on the essence of the primary claim and sometimes secondary claim • Address the requirements of the specific assessment targets • Distinguish between different levels of understanding and/or performance • Contain relevant information, details, and numbers that support different levels of competency related to the item or task Essential Requirements of Selected Response and Constructed Response Items • Alignment to claims and assessment targets • Appropriate content and contexts • Proper presentation: clarity, readability, and accessibility • Stimulus text is at or below the assessed grade level for assessed claim • Free of bias issues • Claim 1 and 4 constructed responses require support • Plausible distractors • Distractors represent common mistakes Capitalize on Technology • Technology-Enabled • Technology-Enhanced Technology-Enabled Items • Digital Media • Video • Animation • Sound • Response Types • Selected Response • Constructed Response Example: Listen to President Kennedy’s 1961 inaugural address and then write an essay analyzing metaphors used regarding foreign policy. Example: View video and write a summary explaining steps in a process. Example of Technology-Enabled Item Gregory is installing tile on a rectangular floor. • He is using congruent square tiles that each have a side length of ½ foot • The area of the floor is 22 square feet. • The width of the floor is 4 feet. Use the grid and the tile below to model the floor. What is the length, in feet, of the floor? Technology-Enhanced Items • Specialized interaction • May have digital media for stimulus • Same requirements as selected and constructed response items • Students manipulate information • Defined responses Technology-Enhanced Example Item Below is a poem, a sonnet, in which the speaker discusses her feelings about a relationship. Read the poem and answer the question that follows. Remember When you can no more hold me by the hand, Nor I half turn to go yet turning stay. Remember me when no more day by day You tell me of our future that you plann’d: Only remember me; you understand It will be late to counsel then or pray. Yet if you should forget me for a while And afterwards remember, do not grieve. For if the darkness and corruption leave A vestige* of the thoughts that once I had Better by far you should forget and smile Than that you should remember and be sad. *vestige: a mark, trace, or visible evidence of something that is no longer present or evident. In the sonnet “Remember,” which two lines reveal a change in the speaker’s message to her subject? Key Components of a TechnologyEnhanced Item Remember When you can no more hold me by the hand, Nor I half turn to go yet turning stay. Remember me when no more day by day You tell me of our future that you plann’d: Only remember me; you understand It will be late to counsel then or pray. Yet if you should forget me for a while And afterwards remember, do not grieve. For if the darkness and corruption leave A vestige* of the thoughts that once I had Better by far you should forget and smile Than that you should remember and be sad. The notes for a summary need to be arranged correctly in the order in which the events occurred in the passage. Click on each sentence and move it to arrange the sentence into correct chronological order. Summary of Events: Maria laughs with the old women. The guest and family eat dinner. Maria’s mother asks the guests for a story. Maria’s guests arrive. Maria becomes sad. The guests take turn telling stories. INTERACTION SPACE Key Components of a TechnologyEnhanced Item Below is a poem, a sonnet, in which the speaker discusses her feelings about a relationship. Read the poem and answer the question that follows. Remember When you can no more hold me by the hand, Nor I half turn to go yet turning stay. Remember me when no more day by day You tell me of our future that you plann’d: Only remember me; you understand It will be late to counsel then or pray. Yet if you should forget me for a while And afterwards remember, do not grieve. For if the darkness and corruption leave A vestige* of the thoughts that once I had Better by far you should forget and smile Than that you should remember and be sad. *vestige: a mark, trace, or visible evidence of something that is no longer present or evident. In the sonnet “Remember,” which two lines reveals a change in the speaker’s message to her subject? TYPE OF INTERACTION SPACE Key Components of a Technology-Enhanced Item Parameter: A variable provides input to a computer program Below is a poem, a sonnet, in which the speaker discusses her feelings about a relationship. Read the poem and answer the question that follows. Remember When you can no more hold me by the hand, Nor I half turn to go yet turning stay. Remember me when no more day by day You tell me of our future that you plann’d: Only remember me; you understand It will be late to counsel then or pray. Yet if you should forget me for a while And afterwards remember, do not grieve. For if the darkness and corruption leave A vestige* of the thoughts that once I had Better by far you should forget and smile Than that you should remember and be sad. Interaction Space Parameters Block Size: Custom blocks Valid Blocks: All Limit Selection: True Max Selections: 1 *vestige: a mark, trace, or visible evidence of something that is no longer present or evident. In the sonnet “Remember,” which two lines reveals a change in the speaker’s message to her subject? Below is a poem, a sonnet, in which the speaker discusses her feelings about a relationship. Read the poem and answer the question that follows. Remember When you can no more hold me by the hand, Nor I half turn to go yet turning stay. Remember me when no ore day by day You tell me of our future that you plann’d: Only remember me; you understand It will be late to counsel then or pray. Yet if you should forget me for a while And afterwards remember, do not grieve. For if the darkness and corruption leave A vestige* of the thoughts that once I had Better by far you should forget and smile Than that you should remember and be sad. *vestige: a mark, trace, or visible evidence of something that is no longer present or evident. Select all verbs that suggest the author is no longer alive. Interaction Space Parameters Block Size: Whole word Valid Blocks: All Limit Selection: False Max Selections: NA Scoring Rule Scoring Rule: Logic used to score student response Who is the protagonist in the story? A. Joe Scoring Rule: If student response = C, then correct B. Sue Otherwise, incorrect C. Maria D. The father Classify each word below based on whether it is a verb or a noun. Verbs Nouns Scoring Rule: If object 1 = B, 2 = A, 3 = B, 4 = A, 5 = A, then correct Otherwise, incorrect Doll Run Dog Swim Eat Technology-Enhanced Item Types • Common English Language Arts Technology-Enhanced item types – Dropdowns – Classification – Reorder text – Select and order – Select text Below is a poem, a sonnet, in which the speaker discusses her feelings about a relationship. Read the poem and answer the question that follows. The notes for a summary need to be arranged correctly in the order in which the events occurred in Remember Classify eachWhen word below based on and move it to the passage. Click on sentence you can noeach more hold me by the hand, Nor I half turn toitgois whether aturning verbstay.or ainto noun. arrange theyetnosentence correct chronological order. Remember me when more day by day You tell me of our future that you plann’d: Verbs Nouns Only remember me; you understand ItSummary will be late to counsel then or pray. of Events: Yet if you should forget me for a while Maria laughsAndwith theremember, old women. afterwards do not grieve. For if the darkness and corruption leave The guest and family dinner. A vestige* of theeat thoughts that once I had Better by far you should forget and smile Maria’s mother asks the guests for a story. Than that you should remember and be sad. Doll Run Dog Swim Eat Maria’s guests arrive. *vestige: a mark, trace, or visible evidence of something that is no longer Maria becomes sad. present or evident. The guests take turn telling stories. In the sonnet “Remember,” which two lines reveals a change in the speaker’s message to her subject? Technology-Enhanced Items Draw a line of symmetry through the figure below. The graph on the right shows a triangle. Draw the triangle after it is reflected over the y-axis. Classify each shape below based whether it contains at least one pair of parallel sides. Reorder the fractions below so that they are ordered from smallest to largest. 3/5 3/4 2/6 1/2 2/3 Key Components of a TechnologyEnhanced Item Draw a line of symmetry through the figure below. Classify each shape below based on whether it contains at least one pair of parallel sides. INTERACTION SPACE Key Components of a TechnologyEnhanced Item Draw a line of symmetry through the figure below. Key Components of a Technology-Enhanced Item Parameter: A variable provides input to a computer program Draw a line of symmetry through the figure below. Interaction Space Parameters Default Coordinates: False Coordinates for corners: (1,1), (13,13) Grid Visible: True Labels: False Snap: True Object Displayed: True Object Coordinates: (4,10), (7,11), (10,10), (9,8), (10,4), (6,5), (4,4), (3,7), (4,10) Points Connected: True Draw a line of symmetry through the figure below. Interaction Space Parameters Default Coordinates: False Coordinates for corners: (1,1), (13,13) Grid Visible: False Labels: False Snap: True Object Displayed: True Object Coordinates: (4,10), (10,10), (4,20), (10,20) Points Connected: True Scoring Rule Scoring Rule: Logic used to score student response Which number is both a factor of 100 and a multiple of 5? Scoring Rule: A. 4 If student response = C, then correct B. 40 Otherwise, incorrect C. 50 D. 500 Classify each shape below based on whether it contains at least one pair of parallel sides. Scoring Rule: If object 1 = B, 2 = A, 3 = B, 4 = A, 5 = A, then correct Otherwise, incorrect Scoring Rule Draw a line of symmetry through the figure below. Scoring Rule: Start Point: Do not consider End Point: Do not consider x-Intercept: Consider, 0, .5 y-Intercept: Consider, 0, .5 Slope: Consider, -1, .25 Alternate Scoring Rule: Start Point: Do not consider End Point: Do not consider x-Intercept: Consider, 0, 0 y-Intercept: Consider, 0, 0 Slope: Consider, -1, 0 Comparing Technology-Enabled and Technology-Enhanced Items Gregory is installing tile on a rectangular floor. • He is using congruent square tiles that each have a side length of ½ foot. • The area of the floor is 22 square feet. • The width of the floor is 4 feet. Use the grid and the tile below to model the floor. What is the length, in feet, of the floor? 5.5 feet Draw a line of symmetry through the figure below. DEPTH OF KNOWLEDGE PERFORMANCE TASKS WRITING