Leading DataDriven Schools Kids First … New Mexico Wins Expected Outcomes Examine the rationale for and four keys of datadriven instruction. Complete test-in-hand analysis of interim assessment data and write a detailed action plan. Participate in role plays of an analysis meeting and a results meeting. Plan steps to launch, strengthen, or extend your school’s data-driven instructional process. Kids First … New Mexico Wins Acknowledgements This training is based on Paul Bambrick-Santoyo’s Driven by Data: A Practical Guide to Improve Instruction Kids First … New Mexico Wins Acknowledgements Through our collaboration with the University of Virginia, School Turnaround Specialist Program (STSP), we have had the opportunity to learn from the STSP team. Mike Kight has graciously shared his resources and work on leading data-driven schools that set the foundation for our work. Kids First … New Mexico Wins NY State Public School ELA 4th Performance vs. Free-Reduced Rates 100% Percent Proficient 90% 80% 70% 60% 50% 40% 30% 20% 10% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Percent Free-Reduced Lunch Kids First … New Mexico Wins NY State Public School ELA 4th Performance vs. Free-Reduced Rates 100% Percent Proficient 90% 80% 70% 60% 50% 40% 30% 20% 10% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Percent Free-Reduced Lunch Kids First … New Mexico Wins Norms Start/end on time Engage as a leader Be present Be respectful Challenge one another All technology is on task Kids First … New Mexico Wins GETTING POOLSIDE: Examining Student Data Kids First … New Mexico Wins Man on Fire What were the key moments in Creasy’s attempt to help Pita? What made Creasy’s analysis effective? Kids First … New Mexico Wins Analysis Core Idea: If you are not at the pool, you cannot do the analysis. Kids First … New Mexico Wins ANALYSIS Kids First … New Mexico Wins GETTING STARTED Kids First … New Mexico Wins Part 1 – Global Impressions How well did the class do as a whole? What are the strengths and weaknesses in the standards? Where do we need the most work? How did the class do on old versus new standards? Are they improving on or forgetting old material? How were the results in different question types (multiple choice, open-ended, written response)? Who are the strong and weak students? Kids First … New Mexico Wins Part 2 – Deep Analysis Squint: Bombed questions – did all students choose the same wrong answer? Why or why not? Break down each standard: Did students do similarly on every question or were some questions harder? Why? Compare similar standards: Do results in one influence the other? Sort data by student scores: Are there questions that separate proficient from non-proficient students? Look horizontally by student: Are there any anomalies? Kids First … New Mexico Wins Interim Assessment Teacher Reflection – Question-level Analysis Kids First … New Mexico Wins Interim Assessment Teacher Reflection – Standards-level Analysis Kids First … New Mexico Wins Interim Assessment Teacher Reflection – Action Plan Kids First … New Mexico Wins Analysis CORE IDEA: Test-in-hand analysis is not one possible way to analyze student error – it is the only means by which to do effective analysis. Kids First … New Mexico Wins Four Steps for Data-Driven Analysis Praise Probe Kids First … New Mexico Wins Role Play Cheat Sheet Kids First … New Mexico Wins ROLE PLAY Kids First … New Mexico Wins Role Play Reflection What did you learn about the teachers? How was this interim assessment analysis meeting different from a post-observation conference? Kids First … New Mexico Wins BREAK Kids First … New Mexico Wins IMPACT OF DATA-DRIVEN INSTRUCTION Kids First … New Mexico Wins North Star Middle Schools 2003 vs. 2008 Kids First … New Mexico Wins SAT Best Combined Scores – 2005 to 2012 1150 1100 1050 National 1000 North Star 950 900 850 2005 2006 2007 2008 2009 2010 2011 2012 Kids First … New Mexico Wins Percent of Juniors and Seniors Taking AP Exams Over Time 70% 65% 60% 49% 50% 43% 38% 40% 30% 20% 17% 19% 10% 0% 2007 2008 2009 2010 2011 2012 Kids First … New Mexico Wins AP Results – Six Year Score Summary 90% 81% 75% 73% 62% 60% 60% 75% 74% 73% 72% 61% 60% 60%60% 75% 60% 47% NJ Statewide 45% Global 30% 30% 15% North Star 33% 11% 0% 2007 2008 2009 2010 2011 2012 Kids First … New Mexico Wins Dodge Academy: Turnaround through Transparency 90.0% 79.4% 80.0% 69.7% 70.0% 58.1% 60.0% 50.0% 2005 2008 40.0% 32.5% 30.0% 21.6% 20.5% 20.0% 10.0% 0.0% Reading Math Science Kids First … New Mexico Wins Baltimore Bombshell Kids First … New Mexico Wins Principals Pursuing Excellence – One Year of Growth, 2013-14 Pojoaque Valley Intermediate increased reading proficiency by 11.1% and math proficiency by 18%. Jornada Elementary in Las Cruces increased reading proficiency by 7.7% and math proficiency by 15.8%. White Mountain Elementary in Ruidoso increased reading proficiency by 4.9% and math proficiency by 20%. Kids First … New Mexico Wins Quick-Write Reflection From what you know right now, what are the most important things you would need to launch a datadriven instructional model in your school? Kids First … New Mexico Wins The Four Keys to Data-Driven Instruction: ASSESSMENTS ANALYSIS ACTION In a Data-Driven CULTURE Kids First … New Mexico Wins The Power of the Question – Math 1. Read the six assessment questions. 2. Discuss the prompts with a partner. Kids First … New Mexico Wins Assessment CORE IDEAS: Standards (and objectives) are meaningless until you define how to assess them. Because of this, assessments are not the end of the teaching and learning process; they’re the beginning. Kids First … New Mexico Wins The Power of the Question – Reading 1. Read the four assessment questions. 2. Discuss the prompts with a partner. Kids First … New Mexico Wins Assessment CORE IDEAS: In an open-ended question, the rubric defines the rigor. In a multiple choice question, the options and text difficulty define the rigor. Kids First … New Mexico Wins Assessment Common Interim Assessments Transparent Starting Point Aligned to State Tests, Instructional Sequence, and College Ready Expectations System of Teaching, Assessment, Reteaching, Reassessment Kids First … New Mexico Wins The Four Keys to Data-Driven Instruction: ASSESSMENTS (Interim, Transparent, Aligned, Re-Assess) ANALYSIS ACTION In a Data-Driven CULTURE Kids First … New Mexico Wins “Man on Fire” – A Video Case Study After a solid analysis, what made Creasy’s action plan effective? Kids First … New Mexico Wins Assessment & Analysis CORE IDEA: How do we find more time to teach our students? Spend less time teaching what they already know and more on what they need. Kids First … New Mexico Wins Analysis Immediate turnaround of results and reports provide user-friendly, succinct item-level analysis, standards-level analysis, and bottom-line results. Teacher-owned analysis Test-in-hand analysis Deep analysis moves beyond what students got wrong to answer why they got it wrong. Kids First … New Mexico Wins The Four Keys to Data-Driven Instruction: ASSESSMENTS (Interim, Transparent, Aligned, Re-Assess) ANALYSIS (Quick, User-Friendly, Teacher-owned, Test-in-hand, Deep) ACTION In a Data-Driven CULTURE Kids First … New Mexico Wins Looking at a Model of Deep Analysis Review one of the following teacher analysis/action plans: – 2nd grade Math (p. 23) – 7th grade English (p. 26) – 7th grade Math (p. 31) – 10th grade English (p. 35) What makes these teacher analysis/action plans effective? Kids First … New Mexico Wins One-and-a-half-pager: Pre-Work for Effective Analysis Meetings Kids First … New Mexico Wins Deep Analysis in Practice Revisit your analysis from the first role play – where could your analysis have been deeper? Pick at least 1 standard from the assessment – analyze deeply in preparation for both a) playing the teacher and b) giving feedback to a teacher in the role play after lunch. – – – – Start from the end goal – generate an exemplar Identify the gap in student work Identify the gap in instruction Make explicit action steps for each standard Kids First … New Mexico Wins Lunch Kids First … New Mexico Wins Opening Praise – Julie What makes Julie’s praise effective? Kids First … New Mexico Wins Praise – Narrate the Positive Genuine: heart-felt, authentic Reinforce Positive Actions: particularly those that are connected to the teacher’s goals for development Teacher-Driven: teacher does the reflection and narrates why they were successful Kids First … New Mexico Wins Probe on Analysis – Beth What prompts does Beth use to guide Steve to a deeper analysis of his results? What makes these prompts effective? Kids First … New Mexico Wins Probe on Analysis Narrow the Focus: look at the standards that merit deeper analysis or better action plans Begin with the End Goal: start with what you want students doing to answer the question correctly Look at the Gap: what is the gap between what students should do and what they actually did? Analyze the Error: analyze what students wrote/did that led to the error Kids First … New Mexico Wins Action Plan & Follow-Up – Beth What prompts does Beth use to guide Steve to a deeper action plan? What makes Beth’s prompts so effective? Kids First … New Mexico Wins Action Plan & Follow-Up Detailed, specific actions: “What would that [lesson, worksheet, exit ticket] look like?” Time-saving: teacher leaves with more accomplished, rather than more action steps Kids First … New Mexico Wins The Four Keys to Data-Driven Instruction: ASSESSMENTS (Interim, Transparent, Aligned, Re-Assess) ANALYSIS (Quick, User-Friendly, Teacher-owned, Test-in-hand, Deep) ACTION (Action Plan, Ongoing, Accountability, Engaged Students) In a Data-Driven CULTURE Kids First … New Mexico Wins Preparing to Lead an Effective Analysis Meeting Read through the onepager on leading effective analysis meetings Kids First … New Mexico Wins Preparing to Lead an Effective Analysis Meeting Script out your questions and the prompts you will use to guide the teacher through each step of the analysis meeting Kids First … New Mexico Wins Analysis Simulation, Round 1 – Probe IDENTIFY ROLES: Teacher, Leader ROLE PLAY ANALYSIS MEETING (3 min): – 1 min – Praise – 2 min – Probing Analysis • Get teacher to a deep analysis DEBRIEF THE ROLE PLAY (2 min): – 1 min – Review the Feedback Cheat Sheet – ID the top feedback to give the leader – 1 min - Teacher gives feedback • What did the leader do most effectively? Where can s/he improve? MINI RE-DO (3 min): – 2 min – Select the hardest part of the feedback meeting and re-do that section – 1 min – Teacher gives quick feedback Kids First … New Mexico Wins Analysis Simulation, Round 2 – Probe SWITCH ROLES ROLE PLAY ANALYSIS MEETING (3 min): – 1 min – Praise – 2 min – Probing Analysis • Get teacher to a deep analysis DEBRIEF THE ROLE PLAY (2 min): – 1 min – Review the Feedback Cheat Sheet – ID the top feedback to give the leader – 1 min - Teacher gives feedback • What did the leader do most effectively? Where can s/he improve? MINI RE-DO (3 min): – 2 min – Select the hardest part of the feedback meeting and re-do that section – 1 min – Teacher gives quick feedback Kids First … New Mexico Wins Analysis Simulation, Round 3 – Action and Follow-Up IDENTIFY ROLES: Teacher, Leader ROLE PLAY ANALYSIS MEETING (2 min): – Assume you have completed the deep analysis – 2 min – Begin from the Action Step • Get teacher to make explicit action plan for the classroom • Schedule the follow-up DEBRIEF THE ROLE PLAY (2 min): – 1 min – Review the Feedback Cheat Sheet – ID the top feedback to give the leader – 1 min – Teacher gives feedback • What did the leader do most effectively? Where can s/he improve? MINI RE-DO (2 min): – Select the hardest part of the feedback meeting and re-do that section (1 min) – Teacher gives quick feedback (1 min) Kids First … New Mexico Wins Analysis Simulation, Round 4 – Action and Follow-Up SWITCH ROLES ROLE PLAY ANALYSIS MEETING (2 min): – Assume you have completed the deep analysis – 2 min – Begin from the Action Step • Get teacher to make explicit action plan for the classroom • Schedule the follow-up DEBRIEF THE ROLE PLAY (2 min): – 1 min – Review the Feedback Cheat Sheet – ID the top feedback to give the leader – 1 min – Teacher gives feedback • What did the leader do most effectively? Where can s/he improve? MINI RE-DO (2 min): – Select the hardest part of the feedback meeting and re-do that section (1 min) – Teacher gives quick feedback (1 min) Kids First … New Mexico Wins Role Play Reflection: What are your biggest takeaways for leading effective analysis meetings? What are the actions we need to take to improve the depth of our teacher action plans and the quality of the analysis meetings? Kids First … New Mexico Wins ACTION, REVISITED Kids First … New Mexico Wins Mr. Holland’s Opus What made the difference? How did Lou Russ finally learn to play the drum? What changed Mr. Holland’s attitude and actions? Kids First … New Mexico Wins Teachers plan lessons collaboratively based on data analysis Action Teachers write action plans to apply insights from analysis meetings Specific timeline for implementation Accountability – leaders check for alignment Engaged Students Kids First … New Mexico Wins BREAK Kids First … New Mexico Wins Case Study: Douglass Street School Kids First … New Mexico Wins Case Study: Douglass Street School 1. Did Krista win the bet? What percentage of teachers’ classes do you think achieved gains of 15 percentage points on the state test? 2. Based on your answers, name the biggest stumbling blocks to the school’s success. 3. Based on your answers, name the most important drivers of school improvement. Kids First … New Mexico Wins Poll the Crowd What percentage of teachers’ classes do you think achieved gains of 15 percentage points on the state test? – 0-25% – 26-50% – 51-75% – 76-99% – 100% Kids First … New Mexico Wins The Four Keys to Data-Driven Instruction: ASSESSMENTS (Interim, Transparent, Aligned, Re-Assess) ANALYSIS (Quick, User-Friendly, Teacher-owned, Test-in-hand, Deep) ACTION (Action Plan, Ongoing, Accountability, Engaged Students) In a Data-Driven CULTURE (Leadership, PD, Calendar, Build by Borrowing) Kids First … New Mexico Wins Data-Driven Culture CORE IDEA: Any initiative that requires buy-in from the beginning will fail. When done well, data-driven instruction doesn’t require buy-in, it creates it. Kids First … New Mexico Wins Data & Culture Kids First … New Mexico Wins Introductory Professional Development: what (assessments) and how (analysis and action) Implementation Calendar: with time for assessments, analysis and action Data-Driven Culture Highly Active Leadership Team: teacher-leader data analysis meetings, maintain focus Ongoing Professional Development: aligned with calendar, flexible to adapt to student learning needs Build By Borrowing: identify and implement best practices from high-achieving teachers and schools Kids First … New Mexico Wins The Four Keys to Data-Driven Instruction: ASSESSMENTS (Interim, Transparent, Aligned, Re-Assess) ANALYSIS (Quick, User-Friendly, Teacher-owned, Test-in-hand, Deep) ACTION (Action Plan, Ongoing, Accountability, Engaged Students) In a Data-Driven CULTURE (Leadership, PD, Calendar, Build by Borrowing) Kids First … New Mexico Wins Data-Driven Instruction CORE IDEA: Data-driven instruction shifts the focus from the teaching to the learning. Kids First … New Mexico Wins Roadside Distractions The pursuit of total buy-in Reliance on poorly implemented professional learning communities (PLCs) Year-end assessment analysis Kids First … New Mexico Wins Eight Mistakes that Matter Inferior Interim Assessments Secretive Interim Assessments Delayed Results Separation of Teaching and Analysis Infrequent Assessments Ineffective Follow-Up Curriculum-Assessment Disconnect Not Making Time for Data Kids First … New Mexico Wins Pitfalls Kids First … New Mexico Wins Pitfalls Giving teachers too much data (or too many forms) that they are not sure where they should focus A cycle is seen as an “event” instead of a process Not engaging teachers who teach content areas for which data is not available Kids First … New Mexico Wins Pitfalls Action Plans developed from one assessment Assumption that plans are being implemented Analysis alone will not help teachers translate ideas into classroom practice Disproportionate attention for students on the “bubble” Kids First … New Mexico Wins Finding the Time Meaningful data analysis, identifying student learning problems by triangulating numerous data sources, analyzing student work samples, finding root causes for emerging problems, and creating a plan to tackle these problems takes time. We suggest 90 minutes of uninterrupted time. Kids First … New Mexico Wins But… Kids First … New Mexico Wins Finding the Time Using time better – Using time that is already built in for other less important tasks or those that could be handled through electronic communication. Restructured or rescheduled time – Altering the school day, school year, or teaching schedules. Stagger teacher and student schedules to have one group report early or later in the day. Kids First … New Mexico Wins Finding the Time Reallocating funds to provide teachers with stipends for staying after school or giving up other time. Hiring subs or finding volunteers to free teachers from regular instructional time for interim assessment analysis. Kids First … New Mexico Wins DDI Implementation Rubric INDIVIDUAL: – Score your school on the rubric – Where are the weakest areas that could be addressed first? SCHOOL TEAM (OR SMALL GROUP): – Identify highest-leverage area of focus Kids First … New Mexico Wins Visualizing Success Kids First … New Mexico Wins Final Reflection What is my personal commitment to making datadriven instruction work more effectively in my school/district? Kids First … New Mexico Wins Relentlessness Through a relentless commitment to observation, feedback, courageous conversations, and customized support, we are seeing LEAs within New Mexico improve outcomes for our children. Kids First … New Mexico Wins Kids First … New Mexico Wins Contact information Hannah Peria School Improvement Specialist Priority Schools Bureau hannah.peria@state.nm.us (505) 827-8005 Howard Oechsner School Improvement Specialist Priority Schools Bureau howard.oechsner@state.nm.us (505) 827-6580 Kids First … New Mexico Wins