How Can Teachers Use Data Effectively to Make Instructional Changes? Alicia Pérez-Katz The Cahn Fellows Program My Individual Challenge Develop systems in the school to support teachers in effectively looking at, disseminating and using data to support struggling students and align teachers in a common purpose/curriculum. o This will enable teachers to articulate standards for the high school and analyze trends and patterns, while also beginning to set individual learning goals. o Additionally, there will be more cohesion across grades, in terms of spiraling skills and expectations, and in the long-term, across departments as well. Ultimately, my long-term goal for the school is for there to be school-wide systems that support the teaching and development of students in writing, through teachers becoming reflective practitioners The long-term goals are: o Teachers will use data to make instructional programs. o Students will have action plans that teachers will refer to when making instructional decisions. o The school community will be aware of the needs of each student. o There will be alignment of skills across grades and departments. Short-term goals for this year were: o All at-risk students will have been tested in the extended day program. Teachers will have analyzed data and created an action plan for the extended day program. o Departments will have created a rubric/assessment for interim assessments and will give their first assessment by February. Teachers will analyze the data in department teams twice in the school year. o In June, departments will look at all data gathered and come to conclusions, planning for structural changes for the following school year. o A datafolio will be assembled and kept in a central location for staff to utilize. o Advisors will meet with each student quarterly and keep running records of student progress and goals. Process o Create a system of data analysis to more thoughtfully craft the interventions and structures in the school rather than depending on mostly anecdotal forms of data on student progress. o Gain teacher ownership of the project by having teachers develop their own interim assessments as a team, and then analyze and track the data that would come from it. o Teachers used their weekly department meeting time to develop the assessment, by beginning with an inventory (Murnane, City, Boudett, ed: Data Wise A Step-by-Step Guide to Using Assessment Results to Improve Teaching and Learning) of what data they used already and looking at State Standards, and other professional resources. o Provide teachers models of best practices found in Millennium High School’s “Do Your Own Assessments” o I shared articles throughout the year during whole staff professional development sessions around ongoing assessment, formative assessment and data analysis o Hired a data specialist (secured through additional funding provided by the new reallocation of funds through the Department of Education) to analyze the scores that teachers collected and create graphs, allowing teachers to see patterns, outliers and the like. o All staff development was dedicated to this initiative, providing teachers time to execute. o Team leaders were assigned for all departments and met with me regularly to receive updates on team progress o Throughout the year, teachers had opportunities to share their progress with each other. I also implemented an external assessment measure called Scantron in the extended day program, as a way to provide teachers with data they could use to monitor student progress in the after school programs and refine their instruction. However, an obstacle I faced was that it was too much too soon. Because teachers were already spending a lot of time on the internal assessments they were creating, they did not feel ownership or buy-in over the Scantron assessments, nor did they understand what the test measured and they needed more professional development support to utilize this program efficiently. Therefore, after reflection, I did not continue with the Scantron, but rather, shifted the focus back to the departments work. Lessons Learned o Departments were able to work cooperatively toward a goal when given a focus and a common project. o It was critical to establish buy-in so that teachers took ownership over the project and were eager to look at patterns in data. Outcomes o English and Math departments are able to develop and assess students and look at data patterns, while other departments are able to create a set of assessment standards and begin using them informally in their classrooms. o The English department created a rubric around measuring student writing and understanding of tasks. o The math department united around teaching problem solving skills across grades and gave teachers a benchmark to measure student progress and how to adapt their instruction to meet student needs. o The hope is that next year, all departments will be able to analyze trends in student performance data and in five years, we will be able to develop school-wide standards and assessments, rather than just by department. o I also hope that now that teachers had a year to develop and analyze trends, they will be able to do so without the charts, but rather will notice trends in student performance through their own analysis. Leadership Reflection o It is critical to develop leadership within the school o The principal cannot hold the vision alone, or it will not truly become part of the school culture. o By developing a team of teacher leaders, and giving other teachers voice in this project, there was more universal buy-in and teachers felt that they could then take on this challenge themselves as well. o By tying my challenge to concerns staff had voiced from years prior, staff was more likely to buy-in and try this challenge along with me. o In order for this challenge project to work, I had to identify for myself what the non-negotiables were and the priorities, and communicate that to staff o I had to meet with resistant teachers throughout the year and try to get them to buy-in o I found that by being more clear in my goals, teachers were more responsive, as there was one focus, rather than many, or an unknown focus. Appendices: A: English Rubric that was created for the Interim assessment. B: Letter to parents re. English assessment C: Letter to parents re. Math assessment D: Science skill break-down (created this year in department meetings) E: Data analysis of English assessment, broken down by subgroups. [ Appendix B April 10, 2008 Dear 10th grade parent, On February 11th, the English Department administered an interim assessment to the entire 10th grade. The purpose of this assessment was to estimate your child’s strengths and challenges in writing analytical essays. Furthermore, this data helps us to develop individualized learning goals for your child. We will be giving these assessments periodically throughout the year, approximately one per year, to reassess your child’s needs. The assessment consists of six components: 1. Comprehension of Task: Students’ writing demonstrates a clear understanding of the assignment. The task is achieved through effective use of literary devices and rhetorical structures. 2. Development of Written Ideas: Students have a clear, focused topic using relevant and accurate supporting details; students present a thorough analysis with accurate and insightful references. 3. Organization: Students’ writing has a clear introduction, body and conclusion using thoughtful transitions. 4. Language Use: Students use a formal tone along with original and expressive language that furthers their writing purpose. 5. Sentence Fluency: Sentences are structured in a way that contributes to purpose and demonstrates sophistication. 6. Conventions: Students utilize correct grammar and mechanics; their use of conventions enhances their ideas and style. We would like you to be aware of your child’s strengths and challenges. As part of this letter you are receiving your child’s score on this first interim assessment as well as a breakdown of their skills. We encourage you to support your son/daughter as s/he works towards certain goals in reading and writing. Each time we administer this assessment, you will receive a breakdown of his/her results. You will be able to see your child’s progress over time as a result. If you have any questions or concerns, please do not hesitate to contact your child’s English teacher. The next interim assessment is scheduled for Tuesday, April 30th. Sincerely, The English Department Diana Di Rico, Josie Burgos, Carolyn Castagna, Scot Nourok, Rita Ross and Leah Witman Appendix B.1 Component Comprehension of Task Development of Written Ideas Organization Language Use (Voice and Word Choice) Sentence Fluency Conventions (Grammar, Usage and Spelling) Score (out of 5) Appendix C Dear 10th Grade Parent, February 2008 On January 22 the mathematics department gave your child an interim assessment. The purpose of this assessment was to determine your child’s strengths and weaknesses and to develop learning objectives for your child. We will be giving these assessments periodically throughout the year, approximately one per quarter, to reassess your child’s goals. The assessment consisted of five components. 1) Multiple Choice Questions: In this section we asked students to answer fifteen Regents multiple choice questions and to explain their answers. If they guessed, they must identify where they had difficulty with the problem (ex. vocabulary, language, creation of equations, etc.). 2) Methods of Representation: We asked students to solve a problem and represent the solution using several different methods. In this case it was a systems of equations problem in which students solved graphically, algebraically, and used tables. 3) Problem Solving: Students completed two problems that tested your child’s problem solving skills and their knowledge of problem solving strategies. 4) Questioning: We tested your child’s ability to ask high level questions based upon a problem and its solution. 5) Written Component: This component determined how well your child could describe the mathematics in their everyday lives. We would like you to be aware of your child’s strengths and weaknesses. As part of this letter, you are receiving your child’s score on this interim assessment as well as a breakdown of their skills. We encourage you to use this when working with your child either directly with their mathematics or to help them formulate their own goals. Each time we administer this assessment you will receive this breakdown of their score in addition to their scores on all the interim assessments administrated earlier in the year. You will be able to see your child’s progress. If you have any questions, please do not hesitate to call any mathematics teacher at (646) 660-6400. Thank you. Sincerely, The Mathematics Department Angela Esquibel Pagee Cheung Ruth Cogan Mojaven (9th Grade) (9th &10th Grades) (10th &11th Grades) Grade) This is the score report for _________: Elisabeth Jaffe Paul (11th Grade) (12th Appendix C.1 Component Regent’s Multiple Choice Methods of Representation Problem Solving High Level Questioning Written Component Total (This was a scaled score) Score (1/22/08) / 30 / 35 / 14 / 10 / 10 / 100 Appendix D Skill Area 1: Using Models and Graphs 9th Grade Identify independent and dependent variables Label X and Y axes Give a title to graph Create appropriate scale Correctly plot points and connecting points Interpret data from graph (e.g., at which temp is enzyme activity highest) Identify trends, extrapolate, and predict (i.e. if the temp stays at 35 what you would you expect to happen 10th Grade Creating and Interpreting Graphs Identify independent and dependent variables Label X and Y axes Give a title to graph Create appropriate scale Correctly plot points and connecting points Interpret data from graph (e.g., identify direct and inverse relationships) Identify trends, extrapolate, and predict 11th Grade Identify independent and dependent variables Label X and Y axes Give a title to graph Create appropriate scale Correctly plot points and connecting points Interpret data from graph Identify trends, extrapolate, and predict Creating best fit lines and curves Determining slope of a best fit line or qualitatively analyzing a curve. (e.g., is it quadratic? Exponential? Inverse Proportional? Etc…) Recognizing what a model represents Label and describe parts Describe the function/how it works Explain phenomena based on a model Choosing appropriate perspective from which to draw diagram Draw or construct physical models to explain observations Interpreting Models Recognizing what a model represents Explain phenomena based on a model Recognizing what a model represents Explain phenomena based on a model Creating Models Choosing appropriate perspective from which to draw diagram Draw or construct physical models to explain observations Choosing appropriate perspective from which to draw diagram Draw or construct physical models to explain observations Using physical phenomena as models for Building models to test engineering related phenomena or general concepts principles (launcher project) Using physical phenomena as models for related phenomena or general concepts (e.g., ballistics car represents horizontal/vertical relationship) Building models of technology (e.g., light bulb) Labeling trajectories Labeling appropriate measurements Identify relationships within variables from data tables Using Data in Calculations Identify relationships within variables from data tables Choosing appropriate equations; using reference tables Solving equations for the desired variables Show mathematical work stating formula and steps for solution Estimate answers Use appropriate equations, units and significant digits Working from graph information to create an equation to express the relationship Choosing appropriate equations; using reference tables Solving equations for the desired variables Show mathematical work stating formula and steps for solution Estimate answers Use appropriate equations and units Combining equations to solve problems and cancel variables (e.g., PEg = KE cancels mass) 9th Grade Skill Area 2: Designing Experiments 10th Grade 11th Grade Formulates a scientific question Generates falsifiable hypothesis that can be tested via appropriate scientific methodology Identifies controls Identifies variables Materials are listed Plan is numbered step by step Detailed, clear, specific Uses diagrams where appropriate Describes appropriate use of materials and relevant techniques Q’s/Hypotheses/Predictions Formulates a scientific question Formulates a scientific question Generates falsifiable hypothesis that can Generates falsifiable hypothesis that can be tested via appropriate scientific be tested via appropriate scientific methodology methodology “If X is true, and a particular test Y is done, then prediction Z will occur.” Creating and/or conducting a plan/procedure Designs controlled experiment Includes safety procedures Acquires and builds apparatus Gathering Data/Evidence Includes data table/chart where Includes data table/chart where appropriate to record observations appropriate to record observations and/or measurements: and/or measurements: Heading Heading quantity measured quantity measured appropriate units appropriate units significant digits Drawing Conclusions Discusses whether or not data supports or Compares predicted result and actual refutes hypothesis result, and evaluates if hypothesis has been supported or refuted Discusses possible sources of error and Evaluates experimental methodology for how they may have affected results inherent sources of error Discusses what changes are necessary if Analyzes possible effect of error on result experiment is conducted again Designs controlled experiment Procedure is repeatable Procedure is detailed, clear, specific Includes diagrams where appropriate Includes data table/chart where appropriate to record observations and/or measurements: Heading quantity measured appropriate units Discusses whether hypothesis has been supported, refuted or is unclear using data from experimental work Uses mathematical models to find patterns in data Discusses possible sources of error and limitations of methods Calculates percent error as appropriate Revises explanation based on results of test and via public discussion Peer Review Provides constructive feedback to peers on how to improve methodology, representation of data, and evaluation of data Provides constructive feedback to peers on limitations of experimental work and write up Skill Area 3: Literacy 9th Grade 10th Grade 11th Grade Reading secondary sources for meaning and understanding Understanding biology and science Understanding chemistry and science Understanding physics and science vocabulary vocabulary vocabulary Dissecting word parts to help uncover Dissecting word parts to help uncover Dissecting word parts to help uncover meaning (roots/suffixes) meaning (roots/suffixes) meaning (roots/suffixes) Using context clues to determine Using context clues to determine meaning Using context clues to determine meaning meaning Identifying main ideas Using textbooks to gain information Using textbooks to gain information Using headings, key terms, etc. to make sense of a text Annotating/taking notes on texts Using other sources (e.g., internet, newspaper) to gain information Determining reliability of sources Determining reliability of sources Determining reliability of sources Connecting text to other ideas/themes Communicates connections between discussed in class (extrapolating beyond current and past aspects of curriculum. text) Identifying what they know/understand Evaluate preconceptions after reading from text and formulating questions for texts what they don’t understand Understanding directions or problems Understanding written directions Transforming word problems into Transforming word problems into visualizations visualizations Reading primary sources for information and understanding Identifying important sections of written experiment (e.g. hypothesis, data, etc.) Locate data from published sources to Interpret charts, graphs, and data tables. support/defend/explain patterns observed in natural phenomena. Articulate and revise scientific understandings in writing Articulate scientific concepts using Develop a written report that accurately Write scientific explanations for natural appropriate tone/vocabulary describes phenomena using qualitative phenomena or to argue a viewpoint and quantitative data based on scientific evidence. Labeling drawings and other work Refine ideas through peer feedback of Refine ideas through peer feedback of Refine ideas through peer feedback of written reports written reports written reports Write incisively and precisely Write incisively and precisely Use equations and words together Use equations and words together effectively effectively Cite sources Appendix A English/ Writing Arts Interim Assessment Name: ________________________________________________________ Section: _________ Date: 5 Development of Written Ideas Clear, focused topic Relevant and accurate supporting details are used Thorough analysis using accurate and insightful references 3 Broad topic Support is generalized or limited Analysis is satisfactory Depth is lacking at points 1 Unclear topic Lacking or irrelevant support Inaccurate evidence Writing shows little or no evidence of analysis Organization Language Use (Voice & Word Choice) Formal tone furthers purpose Writing is original and expressive Language is natural, effective and appropriate Word choice contributes to purpose and style Sentence Fluency Conventions Comprehension of Task Sentence structure contributes to purpose Sophisticated sentence structure used Natural flow and rhythm to sentences Grammar and usage are correct and contribute to clarity Use of conventions enhances ideas and style Writing demonstrates clear understanding of the task and/or text Task is achieved Essay revolves around task Uses literary devices and rhetorical structures effectively Sentences are constructed correctly Some sentences flow Grammar and usage mistakes do not impede meaning Writing shows some control Writing demonstrates some understanding of task Task is adequately completed Uses literary devices and rhetorical structures with some success Tone does not match Sentences are Grammar and usage purpose choppy, incomplete mistakes distract the or unnatural reader Writer’s voice is not apparent Run-ons and Grammar and usage fragments abound mistakes hinder Language is vague or meaning redundant Improper word choice/usage makes writing difficult to understand Assignment: ____________________________________________________ Writing does not demonstrate an understanding of task Task is not met Literary devices and rhetorical structures are not used or are misused Clear introduction, body and satisfying conclusion Thoughtful transitions show how ideas connect Order of writing enhances main idea Recognizable beginning, middle and end Transitions often work well; sometimes connections between ideas are fuzzy Writing is logically ordered No apparent structure Lack of transitions Order of ideas is illogical Tone is appropriate for purpose Voice is not fully developed Words adequately support meaning Language is general but functional