CTP Science Alignment Study ERB is pleased to announce that we engaged WestEd in an alignment study of the CTP science items to the Next Generation Science Standards (2013). WestEd, a national educational organization has been a leader in moving research into practice through research and development programs; by providing training and technical assistance; and by working with policymakers and practitioners at state and local levels to carry out school improvement and innovative change efforts. WestEd is at the forefront of providing alignment services for states, districts, and several non-­‐profit organizations including ACT and College Board. WestEd also serves as the Project Management Partner for the multi-­‐state Smarter Balanced Consortium. CTP Science Items and the Next Generation Science Standards (NGSS) The National Research Council, the National Science Teachers Association, the American Association for the Advancement of Science, and Achieve partnered to create new science standards through a collaborative 26 state-­‐led process. The National Research Council’s, Framework for K-­‐12 Science Education (2012), serves as the foundation for the NGSS. The Framework is based on a rich and growing body of research on teaching and learning in science, as well as on nearly two decades of efforts to determine foundational knowledge and skills for K-­‐12 science and engineering. The NGSS, released in April 2013, describe specific goals for science learning in the form of performance expectations, statements about what students should know and be able to do at each grade level—and thus what should be tested at each grade level. Each performance expectation incorporates: (1) disciplinary core ideas; (2) cross-­‐cutting concepts; and (3) scientific and engineering Practices. Performance expectations, the assessable part of the NGSS, are at the intersection of the three dimensions. This intersection is referred to as three-­‐dimensional learning. Measuring the three-­‐dimensional learning described in the NGSS will require assessments that are significantly different from those in current use. The one-­‐to-­‐one correspondence of a single multiple-­‐choice item to content standards will not be adequate. In order to adequately measure three-­‐dimensional learning, new assessments need to be developed to contain multiple components, such as sets of interrelated questions and tasks. The CTP science items were developed to measure the intent of the National Science Education Standards (1996). Since the previous national standards were utilized in the development of the NGSS and there is some overlap in disciplinary core content and scientific inquiry, the CTP items were aligned to disciplinary core ideas and practices in NGSS. However, as stand-­‐alone items, the 1 CTP items do not align to the integrated nature of the performance expectations. They were found to align to some of the individual disciplinary core ideas and individual practices; no single select response item can integrate disciplinary core ideas, scientific and engineering practices and crosscutting concepts. NGSS Alignment Results • 30-­‐35% of the CTP science items were found to align at either a strong or a partial degree to disciplinary core ideas the NGSS. • 49% of the CTP science items were rated at DOK 2 (described below). Criteria Used to Align the CTP Science Items to the NGSS WestEd content experts analyzed the correspondence between the CTP science items and the NGSS disciplinary core ideas for grades 3-­‐high school. They rated the degree of match between the select-­‐response items and the NGSS and rated the cognitive complexity of the items. Degree of Match Criteria • Strong: The item assesses a fundamental skill or concept as explicitly stated in the standard. • Partial: The item addresses the standard at a lower level of complexity or does not cover the full intent of the standard. • None: The item does not address any of the content of the standard. Depth of Knowledge (DOK) Using Webb’s Scale for Science • Level 1: includes recalling information such as a fact, definition, term, or a simple procedure. • Level 2: includes engaging in mental processing that requires using more than one step to solve a problem. • Level 3: requires reasoning, planning, using evidence, and a higher level of thinking than the previous two levels. • Level 4: requires complex reasoning, planning, developing, and thinking most likely over an extended period of time. 2 In Detail: CTP Science Items Aligned to the Following Disciplinary Core Ideas in NGSS The ERB CTP science item pool consists of 666 select-­‐response items in an online environment. The items are organized within files identified by grades 3 through high school. According to the findings (see Table 1 below), items in the CTP science bank address the following NGSS disciplinary core ideas: • 32% Physical Science • 22% Life Science • 18% Earth Science • 1% Engineering and Technology Table 1: NGSS Disciplinary Core Ideas Grade Level Coverage
NGSS Disciplinary Core Ideas
3
4
5
6
7
8
9
10
HS
Total Items
Overall Percent
Physical
Science
41
24
31
40
34
29
7
5
4
215
32%
Earth Science
Life Science
13
19
12
27
22
17
3
3
3
119
18%
14
28
21
20
30
13
5
9
5
145
22%
Engineering &
Technology
0
0
0
2
0
2
0
0
0
4
1%
None
32
34
47
18
18
18
4
6
6
183
27%
est Percent of Disciplinary Core Ideas (DCI) by Grade Level Findings also showed that a number of CTP items were written to standards that are not included in the NGSS. These content areas include: lab safety; simple machines; friction; and taxonomy. As noted earlier, Scientific and Engineering Practices are integrated with Disciplinary Core Ideas and Crosscutting Concepts. In order to fully align to the NGSS, assessments must integrate these three dimensions. The scientific and engineering practices describe behaviors that scientists 3 engage in as they investigate and build models and theories about the natural world and the key set of engineering practices that engineers use as they design and build models and systems. The eight scientific and engineering practices include: 1. Asking questions and defining problems. 2. Developing and using models. 3. Planning and carrying out investigations. 4. Analyzing and interpreting data. 5. Using mathematics and computational thinking. 6. Constructing explanations and designing solutions. 7. Engaging in argument from evidence. 8. Obtaining, evaluation, and communicating information. While the CTP items were not designed to measure the integrated nature of the NGSS, some items were found to address inquiry skills covered by the practices. Findings show that 13% of the items align to Practice 2—using models, 10% to Practice 3—investigations, and 5% to Practice 6— explanations. What does this mean for my school? ! The alignment of the ERB CTP science items to the NGSS is a very valuable resource for teachers. It provides teachers with flexibility in how and when they assess their students, as well as assessing a particular standard. ! The findings of this study provide guidelines for ERB to bring the CTP science items into full alignment with the NGSS. Assessments fully aligned to the NGSS will need to include multiple components that reflect the connected use of different scientific practices in the context of interconnected disciplinary core ideas and crosscutting concepts. ERB is currently investigating the time frame for project completion and will provide updates as they become available. 4