Washington State Teacher and Principal Evaluation Project Combining Multiple Measures Into a Summative Rating 1 Updated March 2014 Entry Task As you enter, please take a moment to place a sticky note under the appropriate place on the chart paper. Make sure to put your district name on the note, along with which instructional framework you are using. Four options include: 1. We know the types of evidence that we will use for criterion scoring and how to derive a criterion score using that evidence. 2. We know the types of evidence that we will use for criterion scoring, but are still working out how to derive a criterion score using that evidence. 3. We know some of the types of evidence that we will use for criterion scoring. 4. We do not know the types of evidence that we will collect for criterion scoring. 2 2 Welcome! Introductions Logistics Agenda Agenda 3 Connecting Learning I Learning II Implementing Reflecting Wrap-Up Modules Introduction to Educator Evaluation in Washington Using Instructional and Leadership Frameworks in Educator Evaluation Preparing and Applying Formative Multiple Measures of Performance: An Introduction to Self-Assessment, Goal Setting, and Criterion Scoring Including Student Growth in Educator Evaluation Conducting High-Quality Observations and Maximizing Rater Agreement Providing High-Quality Feedback for Continuous Professional Growth and Development Combining Multiple Measures Into a Summative Rating 4 Overview of Intended Outcomes for Module Examine the most appropriate types of evidence needed to assign each criterion score. Translate multiple forms of evidence to the eight criterion scores. Understand how the instructional framework rubric and student growth rubrics are used within the summative scoring methodology for the focused and comprehensive evaluations. Assign a summative score to EXAMPLE teachers or principals using the instructional AND leadership framework rubrics and student growth rubrics. 5 Guidance Icon Key RCW 28A.405.100 6 A capital “G!” indicates that the guidance represents Washington state law. RCW 28A.405.100 A lower-case “g” indicates that the guidance represents research-based best practice but is not mandated by law. Session Norms Pausing Paraphrasing Posing Questions Putting Ideas on the Table Providing Data Paying Attention to Self and Others Presuming Positive Intentions What Else? 7 Connecting Builds community, prepares the team for learning, and links to prior knowledge, other modules, and current work 8 Where Are We? Let’s look at the chart papers from the entry task Where are most districts at in using multiple forms of evidence to create criterion scores? 9 What is the distribution of responses? What is this distribution telling us? The Year-Long Evaluation Cycle 8 Criteria Frameworks + Student Growth Rubrics • • • Observation Student Growth Evidence Step 1: • • • • Criteria aligned to instructional /leadership and student growth rubrics Professional goals ( Instructional/leadership goals ( Student growth goals ( Step 2 & 3: Select and collect evidence • 2 observations ( • Student growth ( • Other evidence ( Districtdetermined process Statedetermined process Distinguished Proficient Basic Unsatisfactory Distinguished Proficient Basic Unsatisfactory Step 4: Step 5: Determine 8 Criteria Scores ( 10 Summative Score ( From Plan to Action Review the goal sheet for Tom Wilson On your handout (Handout 3), write down two pieces of evidence that could be collected to show progress toward that goal. Think about how the evidence aligns to the criteria. 11 Learning I: Transforming Evidence Review the type of evidence needed to assign multiple criterion scores Review how to translate multiple forms of evidence to all eight criterion scores 12 OSPI’s Guiding Principles for Criterion Scoring for Teacher Evaluation The primary goal of any system of teacher evaluation is to promote teacher and student learning. Accurate teacher evaluation requires trained observers using a researchbased instructional framework. Trained observers make accurate assessments of practice based on evidence. The value of accurate assessments of practice is to shape the conversations that lead to improved practice. Embedded in each instructional framework is a system for growth in teaching practice. Reliability and validity of the instructional framework relies on implementation of the full framework rather than individual components/indicators. It is imperative to remain in the formative mindset until the final summative rating is determined. 13 Sources of Evidence for Summative Scoring RCW 28A.405.100 Three sources of information 1. Observations based on your chosen instructional framework 2. Student growth data as measured by student growth rubrics 3. Other evidence relevant to the frameworks 14 The Evidence Cycle – Roles and Responsibilities RCW 28A.405.100 The Evidence Cycle 3. Interpret and clarify 1. Collect 2. Sort and align 15 4. Draw conclusions Step of the Evidence Cycle Teacher Role 1. Collect X 2. Sort and align X Evaluator Role X 3. Interpret and clarify X 4. Draw conclusions X Strategies for Collecting Pieces of Evidence Identify pieces of evidence that the majority of educators will need to collect (e.g., lesson plan, parent communications) Staff share examples of high-quality pieces of evidence Discuss how they provide evidence of a criterion score OR how they can cut across multiple criteria Think about how the pieces of evidence align to the OSPI Guiding Principles Remember, it’s about collecting quality and a variety of evidence that you already use in your classroom 16 Artifacts From Tom Wilson Included are five pieces of evidence Evidence cover pages are missing 17 Alignment to criteria Evidence statements Set Evidence Location A A two-day lesson plan Handout 3 B Unit assessment data Team meeting minutes Handouts 4 and 5 C Parent communication log E-mail exchange Handouts 6 and 7 Identifying Evidence Within your school team, divide into pairs Each pair will do the following: Review one set of evidence Complete the evidence cover page for the missing components Consider these questions: 18 After reviewing these pieces of evidence, do you have enough evidence to make an accurate assessment of practice? Do these pieces of evidence provide sufficient evidence to help shape a conversation that will improve practice? How can these pieces of evidence be used for teacher growth in practice? Learning II: Summative Scoring Understand how the instructional framework rubric and student growth rubrics are used within the summative scoring methodology for the formative and comprehensive evaluations Assign a summative score to teachers or principals using the instructional framework rubrics and student growth rubrics for the formative and comprehensive evaluations 19 Summative Performance Rating Things to remember: 20 It is a process, not a final rating! Balance between professional judgment and transparent rating process Uniformity and transparency in developing the summative rating Comprehensive Evaluation: Teachers RCW 28A.405.100 Assesses all eight evaluation criteria All criteria contribute to the comprehensive summative evaluation rating Student growth rubrics embedded in criteria (3, 6, and 8) All provisional classroom teachers and classroom teachers not on level 3 or level 4 receive comprehensive evaluation All classroom teachers shall receive a comprehensive summative evaluation at least once every four years 21 Comprehensive Evaluation: Principals RCW 28A.405.100 Assesses all eight evaluation criteria All criteria contribute to the comprehensive summative evaluation rating Student growth rubrics embedded in criteria (3, 5, and 8) “Due to the importance of instructional leadership and assuring rater agreement among evaluators, particularly those evaluating teacher performance, school districts are encouraged to conduct comprehensive summative evaluations of principal performance on an annual basis.” — Section 1, (12 c(v)) 22 Comprehensive Evaluation Three Steps to Assign Comprehensive Evaluation System 23 Step 1: Assign Preliminary Summative Score Step 2: Determine Impact on Student Learning Step 3: Use the Summative Rating and Impact on Student Learning Matrix to determine summative score Step 1: Comprehensive Evaluation Step 1: Assign Preliminary Summative Score 24 1a: Transfer criterion scores to summative scoring sheet 1b: Add the eight criterion scores to create a sum 1c: Compare the sum score to the scoring band 1d: Assign a preliminary summative score The RAW Score Model: Preliminary Summative Score Teaching Criteria * Indicates criterion embedded with student growth rubrics Criterion 1: Centering instruction on high expectations for student achievement Criterion 2: Demonstrating effective teaching practices *Criterion 3: Recognizing individual student learning needs and developing strategies to address those needs Criterion 4: Providing clear and intentional focus on subject matter content and curriculum Criterion 5: Fostering and managing a safe, positive learning environment *Criterion 6: Using multiple student data elements to modify instruction and improve student learning Overall Criterion Score 3 4 3 2 3 2 Criterion 7: Communicating and collaborating with parents and school community 3 *Criterion 8: Exhibiting collaborative and collegial practices focused on improving instructional practice and student learning 2 Total Summative Score Evaluators place teachers into preliminary summative rating categories based on score bands. As shown, this teacher would receive a preliminary overall summative rating of proficient. 22 OSPI-Approved Summative Scoring Band 8–14 1 Unsatisfactory 25 15–21 2 Basic 22–28 3 Proficient 29–32 4 Distinguished 25 Step 2: Comprehensive Evaluation Step 2: Determine Impact on Student Learning 26 2a: Transfer student growth rubric scores to student growth summative scoring sheet 2b: Add the five student growth rubric scores 2c: Compare the sum to the student growth scoring band 2d: Assign impact on student learning score Student Growth Rubric and Rating (Teachers Only) Student Growth Criterion 3 Criterion 6 Criterion 8 Student Growth Score Goal-Setting Score Based on Rubric Student Growth* Score Based on Rubric 3 2 2 2** 2** N/A Overall Student Growth Criterion Score 5 4 2 7 4 11 Evaluators place teachers into summative rating categories based on score bands. As shown here, this teacher would receive a low student growth rating. OSPI-Approved Student Growth Impact Rating Scoring Band 5-12 Low 13-17 Average 18-20 High *Must include a minimum of two student growth measures (i.e., state-, district-, school-, and classroom-based measures). ** A student growth score of “1” in any of the student growth rubrics will result in a low growth rating. 27 27 Summative Rating & Impact on Student Learning Matrix Distinguished Proficient Rating Distinguished Rating Summative Rating Student Growth Inquiry Proficient Proficient Rating Student Growth Inquiry Basic Basic Rating Student Growth Inquiry Consequences as a result of intersection between summative rating and impact on student learning rating Basic Rating Unsatisfactory Rating Unsatisfactory 28 Proficient Rating Plan of improvement Low Average High Impact on student learning 28 Step 3: Comprehensive Evaluation RCW 28A.405.100 Step 3: Use the Summative Rating and Impact on Student Learning Matrix to determine summative score Educators with preliminary rating of distinguished with average or high student growth rating: These educators will receive an overall distinguished rating and will be formally recognized and/or rewarded (per regulations). Educators with preliminary rating of unsatisfactory and high student growth rating: These evaluations will be reviewed by the evaluator’s supervisor when an educator is rated unsatisfactory and receives a high student growth rating. The supervisor will take these discrepancies into account in the evaluator’s evaluation. Educators who receive a score of 1 on the achievement of student growth goals will automatically receive a low student growth rating. 29 Learning Activity: Putting the Pieces Together Step 1: Determine framework dimension/component score Step 2: Determine criterion score Use summative scoring sheet (Handout 8) Step 4: Determine student growth impact rating Use criterion scoring sheet (instructional framework-specific; Handout 12, and Handout 13a, b, or c) Step 3: Determine preliminary summative score Use evidence collection alignment form (instructional frameworkspecific; Handout 14a, b or c) Use student growth rubric and rating form (Handout 9) Step 5: Determine comprehensive evaluation score 30 Use summative rating and impact on student learning matrix (Handout 10) Putting the Pieces Together Debrief Find an individual from another district using the same framework Discuss three main questions Did you come up with the same summative score? How do/how will the five steps fit in with your district's current process? 31 Why or why not? Will you have to make big changes or adjust a few things? How can/will the summative scoring process be used as a way to promote professional growth? Focused Evaluation Certificated Classroom Teachers RCW 28A.405.100 Includes an assessment of one of the eight criteria Student growth rubrics from one of the three criteria If a teacher chooses criterion 3, 6, or 8, their accompanying student growth rubrics will be used. If a teacher chooses criterion 1, 2, 4, 5, or 7, the accompanying student growth rubrics from criterion 3 or 6 will be used. Approved by the teacher’s evaluator A focused evaluation must be performed in any year that a comprehensive evaluation is not scheduled 32 Focused Evaluation Principals and Assistant Principals RCW 28A.405.100 Includes an assessment of one of the eight criteria Student growth rubrics from one of the three criteria The focused evaluation will include the student growth rubric row selected by the principal or assistant principal. Criterion and student growth rubric rows must be approved by the principal’s evaluator A focused evaluation must be performed in any year that a comprehensive evaluation is not scheduled 33 Focused Evaluation: Selected Criterion Includes Student Growth Score Similar process to criterion scoring Criterion 3: Marzano Formative evaluation score based on two Marzano components aligned to criterion 3 and two student growth rubric scores aligned with criterion 3 Indicator Unsatisfactory 1 Basic 2 3.1 Effective scaffolding of information Distinguished Summative Score 4 X 3.2 Planning and preparing for needs of all students X Criterion 3: Goal-setting student rubric score X Criterion 3: Student growth rubric score X 34 Proficient 3 2 Focused Evaluation: Selected Criterion Does Not Include Student Growth Score To create formative score, the following are needed: Component scores for the criterion (similar to criterion score) Growth rubric scores from criterion 3 or 6 (from teachers); from criterion 3 or 5 (from principals) Criterion 4: Marzano Formative evaluation score based on two Marzano components aligned to criterion 4 and two student growth rubric scores aligned with criterion 6 Indicator Unsatisfactory 1 Basic 2 Proficient 3 4.1 Attention to est. content standards X 4.2 Use available resources and technology X Criterion 6: Goal-setting student rubric score X Criterion 6: Student growth rubric score 35 Distinguished Summative Score 4 ? X Learning Activity IIB: Reteach Split into pairs.You will reteach each other the process of the focused evaluation process. When you reteach, provide an example from the teacher or principal framework your district uses. Partner 1: Reteach the focused evaluation process when educator selects a criterion that includes student growth. Partner 2: Reteach the focused evaluation process when educator selects a criterion that does NOT include student growth. 36 Implementing Supports teams in problem solving and planning next steps for schools and districts 37 Identifying Tools and Processes for Gathering and Organizing Evidence Create a plan for implementing the data collection process and how that will be used for summative evaluation. Use Handout 11 (Implementation Planning – Three Steps) from your packet to structure your conversation with your school team. This packet is similar what you did at the end of the Preparing and Applying Formative Multiple Measures of Performance module 38 If you have completed part of it, go back through your decisions and refine them based on this module, and any other modules you have completed since then. Implementing Activities Debrief Each team shares one decision that was made today to increase the clarity and feasibility of the teacher evaluation process. 39 Reflecting Engages participants in providing feedback, reflecting on learning, and closing the session 40 Debrief: Hand Plant Plant your hand on a piece of blank paper and trace it. On each finger, write the five most important facts to remember and teach others about combining multiple measures into a summative rating. Share at your tabletops and be prepared to share one with the large group. 41 What’s Next? Homework Options 42 District: Continue to work on a district teacher evaluation guidebook that includes all of the nuts and bolts of the teacher evaluation process. Use the information you recorded on the “Implementation Planning” handout as a starting place. School or Teams: Identify the processes and procedures at your school for how the evidence teachers collect will be organized and stored for effective implementation of the summative scoring process. Thank you! Presenter’s Name XXX-XXX-XXXX xxxxxxxxxxx@xxx.xxx 1234 Street Address City, State 12345-1234 800-123-1234 43