The Standards-based Instructional Planning Process: Backwards Mapping From Standards To Instruction Module 4 Page 1 Module IV Using Student Work to Set Performance Standards and Develop a Scoring Guide Module Outcome Participants will determine what is “good enough” with a set of student work, use the student work to set a performance standard, and analyze the student work to identify gradations of quality in order to develop a scoring guide. The process of setting performance standards is one of the most crucial components of a standards-based system. Standards-based practice requires that teachers participate in this process together to build a collective understanding and agreement about what quality of work is good enough to meet standard(s). Teachers can no longer make these vital decisions alone in their own classrooms. As a result of the intense professional discussion that takes place while setting performance standards, teachers gain clarity about the quality, level, and specific characteristics of work students must demonstrate to meet the standard(s). This conversation also provides an opportunity to inform, change, and improve instructional practice to assure that all students meet the content standards (what students should know) and the performance standards (the quality of work every student is expected to do). The process of setting performance standards involves teachers analyzing a set of student work samples up against the standard and the criteria for success to determine if the work is “to standard” or “not yet to standard.” Teachers sort the work into the two separate stacks as they look for evidence to support their view that the quality of student work is good enough or not good enough to meet the content standard(s). Teachers struggle with the question, “What quality of work is necessary to meet the standard?” As the conversation progresses, teachers begin to identify the specific characteristics of work that are good enough to meet the standards. Through the process teachers develop a commonly agreed upon understanding of what quality work looks like. © 2002 WestEd. All Rights Reserved. Western Assessment Collaborative at The Standards-based Instructional Planning Process: Backwards Mapping From Standards To Instruction Module 4 Page 2 Content Overview A. Setting Performance Standards by Deciding the Characteristics of Work that are Good Enough to Meet Standards B. Steps for Designing a Scoring Guide Materials Handout #22: Sorting Worksheet and/or Wall Chart Handout #23: Scoring Guide/Performance Standard Criteria Handout #24: Steps for Designing a Scoring Guide Handout #25A: Bowling Task Scoring Guide Handout #25B : Bowling Task Student Work Packet (15 pieces of student work) © 2002 WestEd. All Rights Reserved. Western Assessment Collaborative at The Standards-based Instructional Planning Process: Backwards Mapping From Standards To Instruction Module 4 Page 3 IV.A. Setting Performance Standards by Deciding the Characteristics of Work that are Good Enough to Meet Standards 1. Read and Analyze Student Work from the Bowling Task Work in pairs Review the content standard(s), assessment, and criteria for success in preparation for reading the student work Read all student work samples from the assessment looking for evidence of the content standard(s) and criteria for success. Discuss and sort the student work samples into two separate piles/stacks: HO #22 “To Standard” or “Not Yet To Standard” Use the “Sorting Worksheet” to record decisions and to record evidence that supports each decision. 2. Record Results of Group and Identify Agreement and Disagreement Each pair records decisions on large easel sorting chart compiles decisions for the whole group Summarize the decisions that the group has made referring to papers where everyone agrees, slightly disagrees and those with the widest range of disagreement Circle the letter on the chart identifying the pieces of student work where teachers disagreed 3. Discuss Disagreements Using Evidence to Identify Characteristics of Work That Are Good Enough to Meet Standards Select the paper that has the most disagreement or controversy to discuss first Give directions: © 2002 WestEd. All Rights Reserved. “Let’s all look at Paper B” “Some of you thought it was to standard and some of you thought it was “not yet to standard.” Western Assessment Collaborative at The Standards-based Instructional Planning Process Backwards Mapping From Standards To Instruction Sorting Wall Chart Student Work Samples To Standard Not Yet To Standard Evidence A B C D E F G H I J K L M N O © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #22 The Standards-based Instructional Planning Process: Backwards Mapping From Standards To Instruction Module 4 Page 4 “What evidence convinces you it is to standard or not to standard?” Listen for and help teachers to identify and name the characteristics of work that make the student work good enough to meet standards as revealed during the whole group discussion Record the characteristics on large easel chart titled “Characteristics of Work That Are Good Enough to Meet Standards” If necessary, push participants’ thinking by paraphrasing, asking a clarifying question, or asking a question that causes the participant to clearly identify the characteristic of work and/or the quality of work that is needed to meet the standard. For example: © 2002 WestEd. All Rights Reserved. “So you think that students need to include the actual math term or use mathematical vocabulary in order to meet the standard” “You think that the student needs to be more explicit with his/her position, state it more clearly” “You are saying that students need to demonstrate at least three statistical procedures and this student only provided two” “So you think this paper meets the standard because the student clearly addresses the readers counter arguments by ________” “What specifically do you think this paper is lacking?” “What specifically do you think this paper needs to meet the standard?” “Can you read to us the part that supports your position?” “How does that make Paper B good enough to meet the standard?” “Now that we’ve heard some of your thinking behind why its to standard or not, what do you now think about Paper B? Has anyone changed their thinking?” Summarize the characteristics of work that have been listed Western Assessment Collaborative at The Standards-based Instructional Planning Process: Backwards Mapping From Standards To Instruction Module 4 Page 5 IV.B. Steps for Designing a Scoring Guide 1. Present the “Scoring Guide Criteria” commenting on each criterion HO #23 A. Is consistent with the task requirements of the assessment B. Clearly aligns to the content standard(s) and the criteria for success C. Describes the specific characteristics of work at each performance level D. Uses language to describe performance levels that refer to meeting or not yet meeting the standard rather than using language that focuses on the degree of proficiency or passing or failing E. Provides an accurate measurement of progress towards achieving the standard, levels build on each other F. Written in accessible language that describes what must be present in the student work, not merely what is missing G Is useful in giving students feedback on their work . H. Is easy to use to score student work The I. p The performance standard is set at an appropriate level of rigor HO #24 2. Using the handout, “Steps for Designing a Scoring Guide”, briefly walk participants through each of the steps as the introduction to their actually following the steps with Bowling Task student work. © 2002 WestEd. All Rights Reserved. Each pair records decisions on large easel sorting chart compiles decisions for the whole group Western Assessment Collaborative at The Standards-based Instructional Planning Process: Backwards Mapping From Standards To Instruction Module 4 Page 6 Summarize the decisions that the group has made referring to papers where everyone agrees, slightly disagrees and those with the widest range of disagreement Circle the letter on the chart identifying the pieces of student work where teachers disagreed © 2002 WestEd. All Rights Reserved. Western Assessment Collaborative at The Standards-based Instructional Planning Process Backwards Mapping From Standards To Instruction HANDOUT #23 Scoring Guide/Performance Standard Criteria A. Is consistent with the task requirements of the assessment B. Clearly aligns to the content standard(s) C. Specifically describes each performance level D. Written in language that is accessible E. Provides an accurate measurement of progress towards achieving the standard, levels build on each other F. Written in language that describes what must be present in the student work, not merely what is missing G. Is useful in giving students feedback on their work H. Is easy to use to score student work I. The performance standard is set at an appropriate level of rigor © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at The Standards-based Instructional Planning Process Backwards Mapping From Standards To Instruction HANDOUT #24 Steps for Designing a Scoring Guide 1) Use the criteria for success to further sort student assessments into additional levels of Performance Quality 2) Write distinct characteristics for each performance level 3) Write performance level descriptions and the design scoring guide 4) Indicate the performance standard levels and clearly mark which levels are to standard and not yet to standard © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at The Standards-based Instructional Planning Process: Backwards Mapping From Standards To Instruction Module 4 Page 7 3. Guide participants through each of the steps providing time for them to complete each step before moving on to your next set of directions. Step 1- Further sort papers into two additional levels of quality Instruct pairs to return to their two original piles and sort each pile into two additional levels of quality, a “high” and a “low”. They should end up with four piles, a high quality “to standard”, a low quality “to standard”, a high quality “not yet to standard”, and a low quality “not yet to standard.” NOTE: There can be many levels of quality in the student work. For the purposes of this exercise, four levels are sufficient to give participants an understanding of gradations of quality and feel for the process. Step 2-Determining the characteristics of each level After sorting, have each pair list characteristics of each pile on large post-it notes or notepaper. Each list should link to the criteria for success and answer the questions: What distinct characteristics do these papers have that the others do not? At this point, they do not have to write detailed descriptions, just simple, distinct, clear characteristics. Examples of characteristics of work from the Bowling Task: Included at least two tables or graphs Supported choice with mathematical average and mode Step 3-Write performance level descriptions and design the scoring guide HO #25A Refer to handout “Bowling Task Scoring Guide.” Instruct participants to convert list of characteristics from step 3 into more detailed descriptions for each performance level. The descriptions become the basis for the scoring guide. The difference between the characteristic and the scoring guide language is that rather than just 2-3 words; you have a phrase that is clear and explicit, yet still concise. The objective © 2002 WestEd. All Rights Reserved. Western Assessment Collaborative at The Standards-based Instructional Planning Process: Backwards Mapping From Standards To Instruction Module 4 Page 8 is to have a precise statement that avoids misunderstanding or misinterpretation. Examples of Scoring Guide Descriptors for Bowling Task Displays accurate and clear charts that strongly support their choice of position for selecting Bill or Dave. Student used 2 or more statistical measures and probability. The key is to describe what is there, versus what is missing from work. This is more difficult with lower levels of quality since it’s easier to list what is not there. For example, with lower quality you might have only “showed computation of averages of scores.” The scoring guide should be able to demonstrate clearly to the student what they need to do for improvement. Pointing out what is missing will not necessarily inform the student what steps they need to take to improve. After writing the individual level descriptions, participants will need to look at the guide as a whole and to check for such things as flow from level to level. Refer them to the “Criteria for Scoring Guides” to guide their work. Step 4-Indicating the performance standard Instruct participants to draw a bold line on the scoring guide to make clear which performance levels are “to standard” and which are “not yet to standard”. This line shows where you have set your performance standard. 4. Discussion in Pairs Have two or three pairs come together to share and discuss their “to standard” performance level descriptions noting the similarities and differences in content and language. Ask them to evaluate their work against the scoring guide criteria. 5. Whole Group Discussion Debrief the group focusing on their questions and any misunderstandings about the process as well as on issues related to their replicating this with their own student work. © 2002 WestEd. All Rights Reserved. Western Assessment Collaborative at The Standards-based Instructional Planning Process Backwards Mapping From Standards To Instruction HANDOUT #25A Bowling Task Scoring Guide - Mathematics Level 4 Meets the standard and includes advanced work Level 3 Meets the standard Level 2 Progressing toward meeting the standard Level 1 Does not meet the standard Uses two or more of these statistical measures: mean, mode, median, range Uses probability to identify outcomes that are more likely, less likely, or equally likely to occur Shows strong evidence of considering a variety of statistical processes to solve the problem Selects the statistical or mathematical procedure or procedures most appropriate to the question at hand Makes valid inferences from the analysis Makes reasonable predictions from the data Displays accurate and clear charts and graphs appropriate to the data in the question and effectively uses them in the letter as evidence Clearly describes the logic of the student’s use of data in decision-making Uses at least two of these statistical measures: mean, mode, median, range May use probability to identify outcomes that are more likely, less likely, or equally likely to occur Shows evidence of considering only the two statistical measures used in solving the problem Selects the statistical or mathematical procedure or procedures most appropriate to the question at hand Makes valid inferences from the analysis Makes reasonable predictions from the data Displays accurate mathematical computations or displays at least one accurate and clear chart or graph appropriate to the data in the question, but may not use as evidence in the letter Clearly describes the logic of the student’s use of data in decision-making Uses one or two of these statistical measures: mean, mode, median, range Shows some evidence of considering only the one (or two) statistical measures to solve the problem May select the statistical or mathematical procedure or procedures most appropriate to the question at hand Makes reasonable, but weak predictions from the data May display a chart or graph, but does not include in the letter as evidence Attempts to describe the logic of the student’s use of data in decision making, but is unclear and/or weak Inaccurately uses one or two statistical measures: mean, mode, median, range Attempts to make valid inferences from the analysis, but often lacks mathematical vocabulary to explain thinking May display a chart or graph, but tend to be unclear or unlabeled and are not used in the letter as evidence © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at The Standards-based Instructional Planning Process Backwards Mapping From Standards To Instruction HANDOUT #25A Bowling Task Scoring Guide —Writing Level 4 Meets the standard and includes advanced work Level 3 Meets the standard Level 2 Progressing toward meeting the standard Level 1 Does not meet the standard States an assumption and supports it with strong evidence and specific examples Shows evidence of planning the structure of the piece to skillfully move the reader purposefully through the text Shows evidence of considering a variety of statistical processes to make selection Anticipates the reader’s concerns or viewpoint for not being selected Addresses the reader’s anticipated concerns using appropriate facts, details, or examples and provides positive statements about skill or talent or future opportunities Selects and justifies the criteria to be used in decision to select either Bill or Dave Includes illustrations or graphics as needed to enhance the reader’s understanding of the mathematical evidence for the decision Makes precise word choices in order to communicate ideas clearly to the reader States an assumption and supports it with ample evidence and specific examples Shows evidence of planning the structure of the piece to move the reader purposefully through the text Anticipates the reader’s concerns or viewpoint for not being selected Addresses the reader’s anticipated concerns using appropriate facts, details, or examples and provides positive statements about skill, talent or future opportunities Selects and justifies the criteria to be used in decision to select either Bill or Dave Illustrations or graphics may or may not used to enhance the reader’s understanding Makes precise word choices in order to communicate ideas clearly to the reader Selects Bill or Dave and supports choice with weak evidence and/or examples Anticipates the reader’s concern or viewpoint for not being selected by including apologies, recognizing feelings of disappointment, and/or words of encouragement Uses weak, unclear, or no criteria for justifying selection Illustrations or graphics are not used to enhance the reader’s understanding Tends to explain a particular word or concept rather than using precise words to communicate ideas to reader Selects Bill or Dave without giving reasonable and/or accurate evidence for selection Attempts to justify choice, but either makes up the explanation, simply describes a mathematical computational process, or uses inaccurate mathematical data May focus only on the reader’s concerns and/or feelings without using any mathematical data or rationale Letter tends to be brief (2-5 sentences) using simple vocabulary rather than precise words to communicate to reader © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction HANDOUT #25B Bowling Task Assessment Student Work © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B The Standards-based Instructional Planning Process Backwards Mapping From Standards to Instruction © 2002 WestEd. All Rights Reserved Western Assessment Collaborative at HANDOUT #25B