Introduction to Data Inquiry and BPS Data Warehouse SY 12-13 Data Facilitators Meeting #2 Office of Research, Assessment and Evaluation Agenda • Introduction to DESE’s Data-Driven Inquiry and Action Cycle • BPS Data Warehouse • MCAS Reports • Leading and Lagging Indicators 2 What is the Data-Driven Inquiry and Action Cycle? • According to DESE: “The Cycle provides a structure that takes data use from asking the right questions to getting results. It is an iterative process in which the district [school] uses data to target and support continuous improvement…. But analyzing data alone will not result in continuous improvement. Concrete actions that are grounded in evidence and continually monitored through the collection and analysis of multiple forms of data are critical to achieve improved results.”1 1 District Data Team Toolkit, Module 0: Introduction to the Toolkit, p. 1, http://www.doe.mass.edu/apa/dart/lg.html, retrieved 8/8/2012. 3 What does the Data-Driven Inquiry and Action Cycle look like? As used in the DESE District Data Team Toolkit, Module 0, p. 2. 4 Getting Ready: 4 primary domains of data Adapted from: Bernhardt, V. L. (2004). Data Analysis for Continuous School Improvement. Larchmont: Eye on Education , as used in the DESE District Data Team Toolkit, Module 1, p. 14. 5 DESE’s Data Team Toolkit • Designed for district data teams, but can be utilized within schools by shifting focus to the school level • Broken into 6 modules to guide teams through the Data-Driven Inquiry and Action Cycle • http://www.doe.mass.edu/apa/dart/lg.html 6 DESE’s Data Team Toolkit Module 1: Getting Ready http://www.doe.mass.edu/apa/ucd/ddtt/GettingReady.pdf Module 2: Inquiry http://www.doe.mass.edu/apa/ucd/ddtt/Inquiry.pdf Module 3: Information http://www.doe.mass.edu/apa/ucd/ddtt/information.pdf Module 4: Knowledge http://www.doe.mass.edu/apa/ucd/ddtt/Knowledge.pdf Module 5: Action http://www.doe.mass.edu/apa/ucd/ddtt/Action.pdf Module 6: Results http://www.doe.mass.edu/apa/ucd/ddtt/Results.pdf 7 Data Warehouse: Getting Ready How to Log In Go to bpsdepot.mybps.org Type in your User ID and Password, and click Login Browser tips: Turn off all popup blockers such as ones that come with browser, Yahoo, Google, etc. Uncheck the box for Show this page in the future to bypass this screen in subsequent logins Click My home to proceed 8 Data Warehouse: Getting Ready Site Layout & Navigation Prebuilt reports Home screen Your saved reports Download *Depending on access level, you may see more or less reports on your home page. 9 Save Data Warehouse: Inquiry and Information MCAS Reports – by Class Inquiry: How have my current and past students performed on the MCAS assessments? What areas of curriculum did they perform well on and what areas did they struggle with? Information: Results located in the MCAS Reports folder on the Home Screen 10 Data Warehouse: Inquiry and Information MCAS Reports – by Class 11 Data Warehouse: Inquiry and Information MCAS Reports – by Class 12 Data Warehouse: Inquiry and Information MCAS Reports – by Class My Elementary 13 Data Warehouse: Inquiry and Information MCAS Reports – by Class 14 Data Warehouse: Inquiry and Information MCAS Reports – by Class Which standard areas were strongly emphasized on the test? Which areas did my current students struggle with? On which areas did they perform well? Examine the report to answer the following: • Write three standard areas with the highest number of questions and points. • Write two standard areas with the highest percent correct. • Write two standard areas with the lowest percent correct. • Write two clarifying questions you want to pursue further. Discuss your ideas with your neighbor. 15 Data Warehouse: Inquiry and Information MCAS Reports – by Class & Item Inquiry: On the MCAS assessments, which questions gave my students the most difficulty? Information: Results located in the MCAS Reports folder on the Home Screen Tip While viewing the report, Class MCAS Results With Standards (in previous slide), scroll further down to the section on Class Responses as shown below. 16 Data Warehouse: Inquiry and Information MCAS Reports – by Class and Item • Every year, MA DESE only releases certain items from each MCAS subject test. 17 Data Warehouse: Inquiry and Information Which test items are associated with what standards? Are the difficult items associated with one of the emphasized standards? What can you learn about students’ strengths and weaknesses based on questions they answered correctly and incorrectly? Examine the report to answer the following: • List the difficult questions (with lowest percent correct) that are associated with the emphasized standards. • List the easy questions (with highest percent correct) that are associated with the emphasized standards. • Write possible reasons why students answered difficult items incorrectly. • Describe how you would address areas of strength and weakness in your teaching strategies. Discuss your ideas with your neighbor. 18 Data Warehouse: Inquiry and Information MCAS Item Bank I have identified Nonfiction as an area with strong emphasis on the grade 4 MCAS ELA test. Where can I find sample questions for this standard to use with my students? Tip While viewing Class MCAS Results (in previous slide), click on a standard link to drill through. 19 Data Warehouse: Inquiry and Information MCAS Item Bank Tip The questions are ordered from easiest to hardest based on State Percent Correct. Click on a question number to pull up the question on a new screen. 20 Data Warehouse: Inquiry and Information Which test questions have the same level of difficulty as the standard you are reviewing for your class? Are there commonalities or patterns among harder test questions? What concepts and skills are targeted in the harder test questions? Examine the report to answer the following: • List test questions that have a similar level of difficulty (State Percent Correct) as your students’ level (Class Percent Correct). • List five harder test questions (with lower State Percent Correct) and note any patterns or commonalities among them. • Write specific content and skill areas targeted in the harder questions. • For more test items, use the formative assessment web site: www.ati-online.com. Discuss your ideas with your neighbor. 21 Data Warehouse: Inquiry and Information MCAS Reports – by Student Inquiry: On the MCAS assessments, how have individual students performed? What areas of the curriculum are they struggling with and in what areas are they performing well? Information: Results located in the MCAS Reports folder on the Home Screen 22 Data Warehouse: Inquiry and Information MCAS Reports – by Student My Elementary (5555) Banker, Anna [987654] 987654 Banker, Anna Tip SGP: Student Growth Percentile (1-99). It measures how much a student's performance has improved from one year to the next relative to his or her academic peers: other students statewide with a similar MCAS test score history. If a 4th grade student has an SGP of 42 in ELA, that means that between grades 3 and 4 he/she improved more than 42 percent of students statewide with a similar MCAS test score history. CPI: Composite Performance Index. It is a 100-point index that assigns 100, 75, 50, 25, or 0 points to each student participating in MCAS and MCAS-Alt tests based on their performance. The average of all students’ points in a school or a group constitutes a school or student group's CPI for that subject. The CPI is a measure of the extent to which students are progressing toward proficiency (a CPI of 100) in ELA, mathematics, and science. 23 Data Warehouse: Inquiry and Information MCAS Reports – by Student Banker, Anna [987654] Tip For best printing results, choose View in PDF Format. 24 Data Warehouse: Inquiry and Information MCAS Reports – by Student 25 Data Warehouse: Inquiry and Information MCAS Reports – by Student 26 Data Warehouse: Inquiry and Information How well did the student perform compared to all students in the school, the district, and the state? In the areas of emphasis on the test, how well did the student perform? On which areas of emphasis does the student need to improve? Examine the report to answer the following: • Note the student’s number of points and percent correct overall and in the areas of emphasis. • Highlight student’s strengths and weaknesses in the emphasized areas of the test. • Describe the types of classroom work and homework assignments the student needs to focus on and what parents can do to reinforce the student’s learning in these areas. Discuss your ideas with your neighbor. 27 Data Warehouse: Inquiry and Information Leading and Lagging Indicators • New interactive data inquiry tool • Provides key indicators on students enrolled in schools • School and classroom planning • Progress monitoring • Data inquiry 28 Data Warehouse: Inquiry and Information Leading and Lagging Indicators • • • School Level Data Teacher Level Data Class Level Data • Plan, monitor, analyze by specific indicator 29 Data Warehouse: Inquiry and Information Leading and Lagging Indicators: Enrollment – Home Page • Each student group is linkable to a demographic profile table of those students 30 Data Warehouse: Inquiry and Information Leading and Lagging Indicators: Off-Track History • Drill down to specific subgroups of students • Available for each indicator 31 Data Warehouse: Scavenger Hunt Scavenger Hunt • Complete the Data Warehouse Scavenger Hunt using: • Class Listings by Teacher • Student MCAS Results • Leading and Lagging Indicators 32 BPS Personnel Contact Info Got a question? Call or email us! For data support and training, MCAS questions, and Leading and Lagging Indicators questions BPS RAE Office Erin Bridge, ebridge@boston.k12.ma.us, 635-7988 Katy Green, cgreen4@boston.k12.ma.us, 635-9447 Beth Nowak, enowak@boston.k12.ma.us, 635-9552 For BPS Data Warehouse access questions, technical support, website issues, and suggestions BPS OIIT LeRoy Wong, lwong2@boston.k12.ma.us, 635-7683 or datawarehouse@boston.k12.ma.us or OIIT Service Desk, 635-9200 33