Heather Dorsey
Cheryl Vance
Goals
 Participants will:
• Explore SBAC Theory of Action
• Be Introduced to Evidence-Based Design and
Assessment Claims
• Review Depth of Knowledge
“Everyone is good at
mathematics because everyone
can think.
And mathematics is about
thinking.”
-Yeap Ban Har
National Institute of Education
Singapore
So the question is What are they thinking… and how do
we know if it is what we think they are
thinking? And is it what we thought
they were thinking about….
And the answer is
Assessment –
in its many forms
What is the purpose of
assessment?
... to gather evidence of learning.
National Assessment Consortiums
• Smarter Balanced Assessment
Consortium – SBAC
http://www.smarterbalanced.org/
• Partnership for Assessment of College
and Career Readiness – PARCC
• http://www.parcconline.org/
Implementation Timeline
2010-11
2011-12
2012-13
2013-14 2014-15
Phase 1: CCSS Exploration
Phase 2: Build Awareness & Begin
Building Statewide Capacity
Phase 3: Build State & District
Capacity and Classroom Transitions
Phase 4: Statewide Application and
Assessment
Ongoing: Statewide Coordination
and Collaboration to Support
Implementation
Gear Up - CCSS 6-27-12
8
Smarter Balanced Assessment
System: A National Consortium of States
•
•
•
•
27 states
representing 43%
of K-12 students
21 governing, 6
advisory states
Washington state
is fiscal agent
WestEd provides
project
management
services
9
A Balanced Assessment
System
English Language Arts/Literacy and Mathematics, Grades 3-8 and High School
School Year
Last 12 weeks of the year*
DIGITAL CLEARINGHOUSE of formative tools, processes and exemplars; released items and tasks; model
curriculum units; educator training; professional development tools and resources; scorer training modules; and
teacher collaboration tools.
Optional Interim
Assessment
Computer Adaptive
Assessment and
Performance Tasks
Optional Interim
Assessment
Computer Adaptive
Assessment and
Performance Tasks
PERFORMANCE
TASKS
• ELA/Literacy
• Mathematics
Scope, sequence, number and timing of interim assessments locally determined
COMPUTER
ADAPTIVE TESTS
• ELA/Literacy
• Mathematics
Re-take option
*Time windows may be adjusted based on results from the research agenda and final implementation decisions.
10
Time and format
• Summative:
For each content area - ELA & Math
– Computer Adaptive Testing (CAT)
• Selected response (MC), Constructed Response (openended), Technology enhanced (e.g., drag and drop,
video clips, limited web-interface)
– Performance Tasks (like our CBAs)
• Up to 2 per content area in grades 3-8
• Up to 6 per content area in High School
Gear Up - CCSS 6-27-12
11
Time and format
• Summative:
- Administration window is last 12 weeks of school
- For each content area - ELA & Math
– Shorter option for states (~3 hours ELA, ~2 hours Math)
• Scale score on comprehensive test (met/not met
determination)
– Longer option for states (~5 hours ELA, ~3 hours Math)
• Able to report data on claims for individual students
Gear Up - CCSS 6-27-12
12
Time and format
• Interim assessments
– Can be used as often as needed
– Can be customized by districts/schools
• To focus on selected strands
• To clone summative test
– Will use Computer Adaptive Technology
– Released items from summative item bank
Gear Up - CCSS 6-27-12
13
Washington’s Testing System Transition
Current Testing System
 Reading and Math: Grades 3–8 and 10
 Writing: Grades 4, 7, 10
 Science: Grades 5, 8, 10
SBAC/CCSS Testing System
 English/Language Arts and Math: Grade 3–8 and 11*
 Science exams are required under ESEA but are not
included in SBAC
*11th grade to measure college and career readiness. We are working with higher ed to explore the
possible use of these measures as an alternative for college placement (or entrance).
()
Gear Up - CCSS 6-27-12
14
Washington’s Context…
Proposed Summative Assessments in 2014–15
English/LA
Mathematics
Grade 3
SBAC
SBAC
Grade 4
SBAC
SBAC
Grade 5
SBAC
SBAC
Grade 6
SBAC
SBAC
Grade 7
SBAC
SBAC
Grade 8
SBAC
SBAC
MSP
Grades 9-10
HSPE
EOC
EOC
Reading & Writing
???
Algebra/Geometry
???
SBAC
SBAC
Grade 11
Science
MSP
SBAC=SMARTER Balanced Assessment Consortium
MSP= Measurements of Student Progress
HSPE = High School Proficiency Exams
EOC= End of Course exams
Gear Up - CCSS 6-27-12
15
Still to be worked out:
Washington’s Policy Discussion…
• Will 11th grade exam be used for graduation (exit exam) in
Washington?
• If these exams are our exit exams what will the CAA options
be?
• Will the Summative SBAC test replace our End of Course
exams or will SBAC have End of Course exams too?
• How will Washington’s science tests mesh with these tests?
Gear Up - CCSS 6-27-12
16
Calculator Use
• At grades 3–5, all items should be written so they can be answered
without using a calculator
• At grades 6-8, most items should be written so they can be
answered without using a calculator. However, some targets may
require the use of an online calculator tool in order to efficiently
problem solve. In these cases, the calculator tool will appear in the
specification table under “allowable tools.”
• Graphing and scientific calculators may be used for many items in
high school mathematics assessments, even if unnecessary to solve
the problem. An online version will be available for most items
during the CAT portion of the assessment, except when specifically
“turned off” because of the particular content of the item being
assessed.
Seven Key Principles
SBAC: Theory of Action, pp. 1 & 2
1.
2.
3.
4.
5.
6.
7.
An integrated system
Evidence-based approach
Teacher involvement
State-led with transparent governance
Focus: improving teaching and learning
Actionable information – multiple measures
Established professional standards
3. Teacher Involvement
• On-line video course for item writing
• Understand and be able to use sample items
and examples at a greater depth
• Increase ability to evaluate both instructional
materials and assessment items
• Create own classroom items aligned to new
standards
2. Evidence Based Design
1.2.2
1.3.2
2.1.3
2.1.5
2.1.7
Traditional Approach to Item
Development
Traditional Approach to Item Development
Content Standard 2.2.3: Perform addition accurately for single and two digit numbers.
Item:
Beth says that 2 + 4 = 6.
Explain why Beth is correct.
Applying Evidence-Centered Design to
Item and Task Development
Beth says that 2 + 4 = 6.
Explain why Beth is correct.
Weak Evidence
Content Standard 2.2.3:
Perform addition accurately for single and two digit numbers.
Applying Evidence-Centered Design to
Item and Task Development
2 + 4 = ____
Stronger Evidence
Content Standard 2.2.3:
Perform addition accurately for single and two digit numbers.
Applying Evidence-Centered Design to
Item and Task Development
Beth says that 2 + 4 = 6.
Explain why Beth is correct.
Content Standard 2.2.4: Perform
mathematical operations and justify
solutions.
2 + 4 = ____
Content Standard 2.2.3: Perform addition
accurately for single and two digit numbers.
Review of Cognitive Demand Depth of
Knowledge (DOK)
Cognitive Rigor and Depth of
Knowledge
• The level of complexity of the cognitive demand.
– Level 1: Recall and Reproduction
• Requires eliciting information such as a fact, definition, term,
or a simple procedure, as well as performing a simple algorithm
or applying a formula.
– Level 2: Basic Skills and Concepts
• Requires the engagement of some mental processing beyond
a recall of information.
– Level 3: Strategic Thinking and Reasoning
• Requires reasoning, planning, using evidence, and explanations
of thinking.
– Level 4: Extended Thinking
• Requires complex reasoning, planning, developing, and
thinking most likely over an extended period of time.
Level 1 Example
Grade 8
Select all of the expressions that have a value between 0 and 1.
87  8–12
74
7–3
1
2
3
(–5)6
(–5)10

1
3
9
Level 2 Example
Grade 8
A cylindrical tank has a height of 10 feet and
a radius of 4 feet. Jane fills this tank with water
at a rate of 8 cubic feet per minute. How many
minutes will it take Jane to completely fill the
tank without overflowing at this rate?
Round your answer to the nearest minute.
Level 3 Example
Grade 8
The total cost for an order of shirts from a company consists of the cost for
each shirt plus a one-time design fee. The cost for each shirt is the same
no matter how many shirts are ordered.
The company provides the following examples to customers to help them
estimate the total cost for an order of shirts.
• 50 shirts cost $349.50
• 500 shirts cost $2370
Part A: Using the examples provided, what is the cost for each shirt, not
including the one-time design fee? Explain how you found your answer.
Part B: What is the cost of the one-time design fee? Explain how you found
your answer.
Level 4 Example
Grade 8
During the task, the student assumes the role of an architect
who is responsible for designing the best plan for a park with
area and financial restraints. The student completes tasks in
which he/she compares the costs of different bids, determines
what facilities should be given priority in the park, and then
develops a scale drawing of the best design for the park and an
explanation of the choices made. This investigation is done in
class using a calculator, an applet to construct the scale drawing,
and a spreadsheet.
Assessment Claims for Mathematics
Overall Claim (Gr. 3-8)
Overall Claim (High School)
“Students can demonstrate progress toward college and
career readiness in mathematics.”
“Students can demonstrate college and career readiness in
mathematics.”
Concepts and Procedures
“Students can explain and apply mathematical concepts and
interpret and carry out mathematical procedures with
precision and fluency.”
Problem Solving
“Students can solve a range of complex well-posed problems
in pure and applied mathematics, making productive use of
knowledge and problem solving strategies.”
Communicating
Reasoning
“Students can clearly and precisely construct viable
arguments to support their own reasoning and to critique
the reasoning of others.”
Modeling and Data
Analysis
“Students can analyze complex, real-world scenarios and can
construct and use mathematical models to interpret and
solve problems.”
Claim 1
Concepts and Procedures
Students can explain and apply mathematical concepts
and interpret and carry out mathematical procedures with
precision and fluency.
Grade Level
Number of
Assessment Targets
3
11
4
12
5
11
6
10
7
9
8
10
11
16
Assessment Targets
= Clusters
F-IF.8
Write a function defined by an expression in different buy equivalent
forms to reveal and explain different properties of the function.
Assessment Targets
Claim 2 – Problem Solving
Claim 2: Students can solve a range of complex well-posed problems in
pure and applied mathematics, making productive use of knowledge
and problem solving strategies.
A. Apply mathematics to solve well-posed problems arising in
everyday life, society, and the workplace
B. Select and use tools strategically
C. Interpret results in the context of the situation
D. Identify important quantities in a practical situation and
map their relationships.
7.G.4
4.MD.3
Know the formulas for the area and circumference of a circle and use
them to solve problems; give an informal derivation of the relationship
between the circumference and area of a circle.
Assessment Targets
Claim 3 – Communicating Reason
Claim 3: Students can clearly and precisely construct viable
arguments to support their own reasoning and to critique the
reasoning of others.
A.
B.
C.
D.
E.
F.
G.
Test propositions or conjectures with specific examples.
Construct, autonomously, chains of reasoning that justify or refute
propositions or conjectures.
State logical assumptions being used.
Use the technique of breaking an argument into cases.
Distinguish correct logic or reasoning from that which is flawed,
and—if there is a flaw in the argument—explain what it is.
Base arguments on concrete referents such as objects, drawings,
diagrams, and actions.
Determine conditions under which an argument does and
does not apply.
4.NBT. 5
Multiply a whole number of up to four digits by a one-digit whole
number, and multiply two two-digit numbers, using strategies based on
place value and the properties of operations.
Assessment Targets
Claim 4 – Modeling and Data Analysis
Claim 4: Students can analyze complex, real-world scenarios and
can construct and use mathematical models to interpret and
solve problems.
A.
B.
C.
D.
E.
F.
G.
Apply mathematics to solve problems arising in everyday life, society,
and the workplace.
Construct, autonomously, chains of reasoning to justify mathematical
models used, interpretations made, and solutions proposed for a
complex problem.
State logical assumptions being used.
Interpret results in the context of a situation.
Analyze the adequacy of and make improvement to an existing model
or develop a mathematical model of a real phenomenon.
Identify important quantities in a practical situation and map their
relationships.
Identify, analyze, and synthesize relevant external resources to pose
or solve problems.
Simpson Park
Soda Cans
Art Project
Materials on the Internet
• Resources for writing CCSS-M like Assessment Items
www.smarterbalanced.org
• Item writing training
www.smarterbalanced.org/smarter-balancedassessments/item-writing-and-review/
• Sample items
http://www.smarterbalanced.org/sample-items-andperformance-tasks/
• PARCC sample items
http://parcconline.org/samples/item-taskprototypes#7
• Illustrative Mathematics Project
http://illustrativemathematics.org