Karen Hall
2008 WERA
Mathematics Assessment
Office of Superintendent
of Public Instruction
1
RCW 28A.305.215(3)
“…The recommendations [for revised EALRs and
GLEs in mathematics] shall be based on:
(a) Considerations of clarity, rigor, content,
depth, coherence from grade to grade,
specificity, accessibility, and measurability.”
Mathematics Assessment
Office of Superintendent
of Public Instruction
2
Standards
Curriculum
Assessment
Mathematics Assessment
Office of Superintendent
of Public Instruction
3





April 2008: New Grade 3-8 Math Standards approved
May - November 2008
◦ Develop Item Specifications for new grade-level standards
◦ Align or revise items in existing test items to new grade-level standards
◦ Write new items to fill in gaps
April-May 2009
◦ Last year of current Grade 3-8 math WASL
◦ Pilot new and rewritten items
April-May 2010: First administration of new Grade 3-8 math WASL
July 2010: State Board adopts new performance standards on Gr. 3-8
math tests
Mathematics Assessment
Office of Superintendent
of Public Instruction
4





August 2008: New High School Mathematics Standards approved
November 2008 –August 2009
◦ Align existing assessment items to new grade-level standards
◦ Write new items to fill in gaps
2009 and 2010: Pilot new and rewritten items
2011: First administration of new HS End of Course Assessments
July 2011: State Board adopts new performance standards on High School
End of Course Assessments
Mathematics Assessment
Office of Superintendent
of Public Instruction
5
Develop Item Specifications for
Grades 3-8 while retaining clarity and
specificity of Standards Document.
Mathematics Assessment
Office of Superintendent
of Public Instruction
6


5.3. Core Content: Triangles and quadrilaterals
(Geometry/Measurement, Algebra)
Students focus on triangles and quadrilaterals to formalize
and extend their understanding of these geometric shapes.
They classify different types of triangles and quadrilaterals
and develop formulas for their areas. In working with these
formulas, students reinforce an important connection
between algebra and geometry. They explore symmetry of
these figures and use what they learn about triangles and
quadrilaterals to solve a variety of problems in geometric
contexts.
Mathematics Assessment
Office of Superintendent
of Public Instruction
7
Grade Level
Area of Emphasis
Expectation
5.3.D
Mathematics Assessment
Office of Superintendent
of Public Instruction
8
Identify
which Performance Expectations
to assess on WASL.
Mathematics Assessment
Office of Superintendent
of Public Instruction
9

Identify restrictions, if any, for
◦ Vocabulary
 “Vocabulary First Used in Assessment Items”
 “Measurement Vocabulary”
◦ Computation
 Number of addends
 Denominators
 Decimal places
◦ Measurements
Mathematics Assessment
Office of Superintendent
of Public Instruction
10
 For
each Performance Expectation
Identify:
◦ Cognitive Complexity (Webb’s model)
◦ Item Type (MC or SA)
◦ Contextual Situation or not
◦ “Tools” or “No-Tools” day
Mathematics Assessment
Office of Superintendent
of Public Instruction
11
Items may ask students to:
C.C.
Format
Sit
5.3.A Classify quadrilaterals.
2
MC,SA
N
Too
l
I
5.3.B Identify, sketch, and measure acute, right, and obtuse angles.
1
MC,SA
N
I
5.3.C Identify, describe, and classify triangles by angle measure and number of co
ngruent sides.
5.3.D Determine the formula for the area of a parallelogram by relating it to the are
a of a rectangle.
5.3.E Determine the formula for the area of a triangle by relating it to the area of a
parallelogram.
5.3.F Determine the perimeters and areas of triangles and parallelograms.
1,2
MC,SA
N
I
NA
NA
NA
NA
NA
NA
NA
NA
1
MC
N
I
5.3.G Draw quadrilaterals and triangles from given information about sides and a
ngles.
5.3.H Determine the number and location of lines of symmetry in triangles and q
uadrilaterals.
5.3.I Solve single- and multi-step word problems about the perimeters and areas
of quadrilaterals and triangles and verify the solutions.
2
SA
I
I
1
MC
N
I
2
MC,SA
Y
I
Performance Expectations with Restrictions
Office of Superintendent
of Public Instruction
12
Stimulus, Stem, and Prompt Rules







Use Item Development Guidelines at the beginning of this document.
Answer choices will be stated in terms of the same system of
measurement.
Items will not require students to convert between U.S. customary and
metric units.
Exponents will not be used to express square units.
Items may tell students to use a straight edge or a protractor.
Items will not require use of a particular strategy to determine a derived
measurement.
Grids may be provided in items that require students to draw angles or
figures.
Office of Superintendent
of Public Instruction
13
Stimulus, Stem, and Prompt Rules



Items assessing 5.3.A
may include parallelograms,
kites, squares, rhombi, trapezoids, and rectangles.
Items assessing 5.3.C may require triangles to be classified
by angles as acute, right obtuse or by sides as scalene,
isosceles, or equilateral.
Items assessing 5.3.F may include side measures or may
require students to measure sides of figures.
Office of Superintendent
of Public Instruction
14
• OSPI Mathematics
Assessment Team
• Yoonsun Lee, Ph.D
Draft 1
Draft 2
• Review by Teachers and
Coaches
•Mathematics Assessment
Leadership Team (MALT)
• Review by Testing Experts
• Review by OSPI & other
interested educators
Draft 3
Mathematics Assessment
Office of Superintendent
of Public Instruction
15
Summer Item Writing
◦ 120 Item writers, ~900 items for grades 3-8
◦ Excellent feedback on item specifications
 Content Review of New Items
◦ 5 committees
◦ 2 weeks

Draft #8
Mathematics Assessment
Office of Superintendent
of Public Instruction
16
Test Experts Recommendation:
Assign Cognitive Complexity
to each Performance Expectation
Mathematics Assessment
Office of Superintendent
of Public Instruction
17

No Child Left Behind (NCLB) requires assessments
to “measure the depth and breadth of the state
academic content standards for a given grade level”
(U.S. Department of Education, 2003, p. 12)
Mathematics Assessment
Office of Superintendent
of Public Instruction
18

Mechanism to ensure that the intent of the standard
and the level of student demonstration required by
that standard matches the assessment items
(required under NCLB)

Provides cognitive processing ceiling for item
development
Mathematics Assessment
Office of Superintendent
of Public Instruction
19
Alignment
Standards
Standards
Assessment
Assessment
Standards
Assessment
Items
Assessment
Standards
Mathematics Assessment
Office of Superintendent
of Public Instruction
Adapted from Norman Webb, 2005
20
Develop assessment items
to align with
cognitive complexity of
Performance Expectation
Mathematics Assessment
Office of Superintendent
of Public Instruction
21
The demand on thinking the items requires:
Low Complexity
Relies heavily on the recall and recognition of previously learned
concepts and principles.
Moderate Complexity
Involves more flexibility of thinking and choice among alternatives
than do those in the low-complexity category.
High Complexity
Places heavy demands on students, who must engage in more
abstract reasoning, planning, analysis, judgment, and creative
thought.
Mathematics Assessment
Office of Superintendent
of Public Instruction
22
23


Adapted from the model used by Norman Webb,
University of Wisconsin, to align standards with
assessments
Used by the Council of Chief State School Officers
(CCSSO) for assessment alignment in more than ten
states
Mathematics Assessment
Office of Superintendent
of Public Instruction
24

The Depth of Knowledge is NOT determined by the
verb, but the context in which the verb is used and the
depth of thinking required.
Mathematics Assessment
Office of Superintendent
of Public Instruction
25

Level 1 (Low) Recall

Level 2 (Moderate) Skill/Concept

Level 3 (High) Strategic Thinking

Level 4 Extended Thinking
NOTE: Definitions with examples of skills at each level.
Mathematics Assessment
Office of Superintendent
of Public Instruction
26
DOK 1 requires recall of information, such as a fact,
definition, term, or performance of a simple
process or procedure.
Answering a Level 1 item can involve following a
simple, well-known procedure or formula. Simple
skills and abilities or recall characterize DOK 1.
Mathematics Assessment
Office of Superintendent
of Public Instruction
27
DOK 2 includes the engagement of some mental
processing beyond recalling or reproducing a
response. Items require students to make some
decisions as to how to approach the question or
problem.
These actions imply more than one mental or
cognitive process/step.
Mathematics Assessment
Office of Superintendent
of Public Instruction
28
DOK 3 requires deep understanding as exhibited
through planning, using evidence, and more
demanding cognitive reasoning. The cognitive
demands at Level 3 are complex and abstract.
An assessment item that has more than one
possible answer and requires students to justify
the response they give would most likely be a
Level 3.
Mathematics Assessment
Office of Superintendent
of Public Instruction
29
DOK 4 requires high cognitive demand and is very
complex. Students are expected to make connections
and relate ideas within the content or among content
areas—and have to select or devise one approach
among many alternatives on how the situation can be
solved.
Due to the complexity of cognitive demand, DOK 4
often requires an extended period of time.
Mathematics Assessment
Office of Superintendent
of Public Instruction
30
However, extended time alone is not the
distinguishing factor.
Task
Thinking
Collecting data samples over
several months
Recall
Organizing the data in a chart
Skills/
concepts
Strategic
Thinking
Using this chart to make and
justify predictions
Developing a generalized model
from this data and applying it to a
new situation
Extending
Thinking
Mathematics Assessment
Office of Superintendent
of Public Instruction
31
• Depth of Knowledge (DOK) is a scale of cognitive demand.
•DOK requires looking at the assessment item/standard
order to determine the level. DOK is about the
item/standard-not the student.
•The context of the assessment item/standard must be
considered to determine the DOK-not just a look at which
verb was used.
Mathematics Assessment
Office of Superintendent
of Public Instruction
32

Read example items and discuss cognitive
complexity of each item
Office of Superintendent
of Public Instruction
33
Mathematics Assessment
Office of Superintendent
of Public Instruction
34

Total Number of points

Total Number of Multiple-Choice Items

Total Number of Short-Answer Items

Distribution of Points by Area of Emphasis

Distribution of Points by Cog. Complexity
Mathematics Assessment
Office of Superintendent
of Public Instruction
35
Grades 3-5



38 Points Total
24 Multiple-Choice
7 Short-Answer
Grades 6-8



50 Points Total
30 Multiple-Choice
10 Short-Answer
Office of Superintendent
of Public Instruction
36
Areas of Empha
sis
5.1
MC
SA
24
7
24
14
Total Points
Points
Level 1
Points
Level 2
Points
Level 3
5.2
5.3
5.4
5.5
5.6
Total Number of
Items
Total Number of
Points
38
Mathematics Assessment
Office of Superintendent
of Public Instruction
37


Surveyed ~150 teachers, coaches and mathematics
coordinators
Workgroup to distribute test points among areas of
emphasis
◦
◦
◦
◦

Cathy Seeley, Dana Center
George Bright, OSPI, Mathematics Special Assistant
OSPI Teaching and Learning Mathematics
OSPI Mathematics Assessment
Test Map Review Committee
◦ Standards Revision Team
◦ Teachers, Coaches, Administrators
Mathematics Assessment
Office of Superintendent
of Public Instruction
38



Survey results
Paragraphs in Standards Document
Performance Expectations
Mathematics Assessment
Office of Superintendent
of Public Instruction
39
Items may ask students to:
C.C.
Format
Sit
5.3.A Classify quadrilaterals.
2
MC,SA
N
Too
l
I
5.3.B Identify, sketch, and measure acute, right, and obtuse angles.
1
MC,SA
N
I
5.3.C Identify, describe, and classify triangles by angle measure and number of co
ngruent sides.
5.3.D Determine the formula for the area of a parallelogram by relating it to the are
a of a rectangle.
5.3.E Determine the formula for the area of a triangle by relating it to the area of a
parallelogram.
5.3.F Determine the perimeters and areas of triangles and parallelograms.
1,2
MC,SA
N
I
NA
NA
NA
NA
NA
NA
NA
NA
1
MC
N
I
5.3.G Draw quadrilaterals and triangles from given information about sides and a
ngles.
5.3.H Determine the number and location of lines of symmetry in triangles and q
uadrilaterals.
5.3.I Solve single- and multi-step word problems about the perimeters and areas
of quadrilaterals and triangles and verify the solutions.
2
SA
I
I
1
MC
N
I
2
MC,SA
Y
I
40 Grade 5 Performance Expectations
34 BOLD text Performance Expectations
31 test questions
38 total points.
Office of Superintendent
of Public Instruction
40





Use results of test map review to finalize test maps for
grades 3-8
Submit test maps for approval to National Technical
Advisory Committee on January 11, 2009
Post Grades 3-8 Test and Item Specification Document
on OSPI website.
Review and feedback of assessment restrictions for
high school performance expectations – January
Conference
Write items to assess high school PEs
Mathematics Assessment
Office of Superintendent
of Public Instruction
41
GRADE
Total PE’s
PE’s Assessed
# of items Assessed 2010
3
37
32
31
4
43
37
31
5
40
33
31
6
39
35
40
7
35
32
40
8
33
31
40
Mathematics Assessment
Office of Superintendent
of Public Instruction
42