Susan H. Hull, Cathy L. Seeley, Katey Arrington
Charles A. Dana Center
The University of Texas at Austin
April 24, 2012

•   Overview of the PARCC Mathematics Prototyping Project
•   VERY BRIEF overview of a new model for large-scale assessment (Evidence-Centered Design)
•   What makes an innovative summative assessment task?
•   Examples of possible innovations in summative assessment (technology, mathematics, other?)

•   Develop prototypes of possible assessment tasks/items that reflect PARCC’s priorities for summative assessment
•   Partners
•   Agile Mind, Inc.
•   University of Arizona (Bill McCallum; Center on
Mathematics & Education; Illustrative Mathematics Proj)
•   City University of New York
•   Dana Center Leadership: Cathy Seeley, Susan Hull, Kathi
Cook, Ann Roman, Lisa Brown, Katey Arrington & colleagues
•   Consultants/Authors: David Foster, Randy Charles, Steve
Leinwand, Sue Eddins, David Hughes, Vicki Massey, U of AZ

Keep in mind…
Prototyping is for learning, and it can be messy.

More Specifically…
PARCC Mathematics Prototyping Project
•   Provide examples to guide item/task development for the assessment
•   Generate a few task models for Grades 3, 4, 6, 7, HS
•   Generate 2 to 3 prototype tasks for each task model
•   Field test, gather data, and revise task models and their accompanying prototypes
Prototyping is for learning, and it can be messy.

•   Round 1 of Prototyping
Ready for Field-Test
•   Round 2 development beginning
•   Summer Public Release by PARCC o   Public release of
o   Public feedback on

Claims
Evidence
Design begins with the inferences
( claims ) we want to make about students
In order to support claims , we must gather evidence
Task Models
Tasks are designed to elicit specific evidence from students in support of claims
ECD is a deliberate, systematic approach to help establish validity of the assessments, increase comparability across years, and increase efficiencies/reduce costs .

Where have we been?
Standard
Standard
Standard
Standard
Standard
Standard
Task
Task
Task
Task
Task
Task

Where are we going?
Cluster Heading
Standard
Standard
Standard
Cluster Heading
Standard
Standard
Standard
Claims
Evidence
Task
Models
Task
Task
Task
Task
Task

What are they?
PARCC has defined the master claim and sub-claims to be used for grades 3-HS. (ITN Appendix D, pg 3)
Evidence statements are written using the language of the standards.
•   Some will match all or parts of standards/cluster headings.
•   Some will clarify what students are doing or producing.
•   Some will correspond to combinations stating that students can solve problems by combining knowledge, skills, and understandings.

Eliciting Evidence
The purpose of the task model is to describe a family of tasks that will elicit the desired evidence of student learning.
–   Can we create more assessment tasks (items) from the task model that would elicit the desired evidence?

Who will create them?
PARCC has defined the master claim and sub-claims to be used for grades 3-HS, and some evidence statements for sub-claims C and D.
The Dana Center will generate sample evidence statements for all sub-claims, sample task models and accompanying prototype tasks for grades 3, 4, 6, 7 and HS.
The approved vendor will produce a large and growing number of tasks based on an expanded set of task models.

•   See PARCC Item Development ITN Appendix D
–   Type I (Sub-Claims A, B, E): Machine-Scorable, including innovative, computer-based formats; balance of conceptual understanding, procedural knowledge, brief applications, includes MPs (esp. MP 5, 7)
–   Type II (Sub-Claim C): Hand or machine scored, written arguments/justifications, critique of reasoning, precision in communicating mathematically
–   Type III (Sub-Claim D): Hand or machine scored, realworld scenario, practice-forward highlighting modeling
(MP4), may also involve MP 2, 5, 7, 8

•   Agile Mind technological platform for prototyping
•   Vendors (ETS & Pearson) will use/create their own technological platform and capabilities
We are suggesting possible item formats that are enhanced by and make use of technology…

How will it be used?
•   To measure a wider range of standards cost effectively.
•   Technology enhancements may range from incremental to transformative.
•   Technology can draw students into the mathematics.
•   Enhancements can support wider accessibility for students.

(musings from Bill McCallum 4-14-12)
•   A situation or story external to mathematics in which the problem is placed – the background against which the main subject in a painting is placed
•   Designed to:
Make the problem interesting to the student
Describe a real-world problem that mathematics can solve
Set up a mathematical idea
Lead the student to a certain sort of mathematical work

•   Does not have to be realistic, but it has to serve a purpose, either mathematical or pedagogical.
•   Is not warranted: if it is distracting or phony,
if the task would be essentially the same without the context, or
if it distracts from the mathematics or points to a problematic practice.

•
•
•
•

What makes an innovative summative task?
Illustrative Math Project: Cash Box
Commentary on the item
We think this item may be a great instructional task, but not necessarily appropriate for summative assessment. At some point early in solving the problem, you have to make a guess as to which system to start with (e.g., do you start by solving the system as equal to 200, or 201). Therefore, the task favors those students who make a correct initial guess. They will need less time to answer the item, and have more time for the test.

What makes an innovative summative task?
Illustrative Math Project: Cash Box
Commentary on technology
What you want students to do in this task is figure out what strategy they need to use to answer the question. Therefore, you do not want to use technology in such a way that you guide the student to a strategy. So, you do not want to prompt the students to set up a system of equations using a puzzle type interaction.
We would recommend a more un-scaffolded approach. A first question would ask students to answer yes or no (through multiple choice, for example) “Does the dollar belong inside the cash box or not?” Then, you would prompt the student to indicate how many tickets were sold to individuals and how many to couples. This would be a fill-in-the-blank.

1.
Why I like this task
2.
Synergy between context and mathematics
3.
Synergy (possible) between technology and mathematics
4.
Things to work out

Susan H. Hull (Director of National Initiatives) shhull@austin.utexas.edu
Cathy Seeley (PARCC Prototype Project Director) cseeley@austin.utexas.edu
Katey Arrington (PARCC HS Lead) katey.arrington@austin.utexas.edu
Charles A. Dana Center utdanacenter.org
PARCC/Achieve http://www.parcconline.org/