A New Vision of Assessment
Association of American Publishers Annual Meeting
Laura Slover, PARCC
June 4, 2013
1
PARCC States
PARCC Priorities
1. Determine whether students are college and
career ready or on track
2. Connect to the Common Core State Standards
3. Measure the full range of student performance,
including that of high- and low-achieving students
4. Provide educators data throughout the year to
inform instruction
5. Create innovative 21st century, technology-based
assessments
6. Be affordable and sustainable
3
In the Last Year…
June 2012
 Minimum Technology Specifications, Version 1.0, Released
 Item Development Began
August 2012
 Item and Task Prototypes Released
September 2012
 Adopted Principles for Comparability with SBAC
October 2012
 College- and Career-Ready Determination Policy Adopted
November 2012
 Item Tryout and Field Testing RFP Released
December 2012
 Retest Policy Adopted
 Minimum Technology Specifications, Version 2.0, Released
January 2013
 Formed Executive Committee for Streamlined Decision-Making
March 2013




Assessment Administration Capacity Planning Tool Released
Estimated Time on Task Released
Item Development Research with 2,300 Students in 6 states
PARCC Non-Profit Formed
April 2013




Added Project Management & Procurement Capacity
Draft Performance Level Descriptors Released for Comment
Draft Accommodations Manual Released for Comment
Test Blueprints & Evidence Statements Released
4
Looking Ahead: 2013
June 2013
• Item Tryout Studies with 4,800+ Students in 4 States
• Guidance on Participation in Field Test and Practice Tests
July 2013
•
•
•
•
•
Final Subject- and Grade-Level Performance Level Descriptors Adopted
First Edition of PARCC Accommodations Manual
Estimates for Summative Assessment Costs Released
PARCC Technology Components RFP Released
Diagnostic & Formative Assessment RFPs Released
August 2013
•
•
•
•
•
Phase I Item Development Complete
Additional Sample Items Released
Schools/Districts Notified of Selection for Field Testing
Partnership Resource Center RFP Released
Additional Information about Testing Windows Released
Fall 2013
•
•
•
•
•
Sample Items Re-Released on Technology Platform
Design of Student Score Reports Released
Minimum Technology Specifications, version 3.0
Training Materials for IEP Writing Teams on Accommodations Manual
Timeline / Plan for Student Registration for Operational Testing
Winter
2013-2014
• Specifications for Online Testing Portal Released
• Final Information about Field Testing
• Final Information about Timing of Data Return
5
Looking Ahead: 2014 and Beyond
Spring 2014
•
•
•
•
Performance-Based Assessment Field Test Administration (Feb-March)
End-of-Year Field Test Administration (April-May)
Practice Test Available
Standard-Setting Methodology Released
Summer 2014
•
•
•
•
•
Final Information about Methodology for Calculating Student Scores
Final Test Security Policies Released
Final Registration Timeline and Process for Operational Testing Released
Final Technology Specifications Released
Phase II Item Development Complete
Fall 2014
• Operational Assessment Administration Manual
• Forms Construction for Operational Administration Complete
Winter
2014-2015
• 1st Operational Assessment – Fall 2014 Block Schedule Administration
Spring 2015
• 1st Operational Assessment – Spring 2014 Administration
Summer 2015
• Student Performance Levels and Associated Cut Scores for all Grades and
for College- and Career-Ready Determination (Standard-Setting)
6
Assessments
ELA/Literacy and Mathematics, Grades 3–11
Beginning of
School Year
End of
School Year
Flexible administration
Diagnostic
Assessment
Mid-Year
Assessment
PerformanceBased
Assessment
End-of-Year
Assessment
Speaking and
Listening
Assessment
Key:
Optional
Required
7
Evidence-Centered Design (ECD) for the
PARCC Assessments
Model Content Frameworks
Evidence Statements
To make claims
about what
students know, we Based on analysis,
must operationalize evidence drive task
the standards
development
Tasks
Tasks are designed
to elicit specific
evidence from
students
8
What is Different About PARCC’s
Development Process?
• PARCC states first developed the Model Content
Frameworks to provide guidance on key elements of
excellent instruction aligned with the Standards.
• Then, those Frameworks informed the assessment blueprint
design.
• Aligned evidence statements and task models followed.
So…
• PARCC is designing the assessments around exactly the
same content shifts the standards expect of teachers and
students.
• PARCC is communicating in the same voice to teachers as it
is to assessment developers
9
Advances in the PARCC
ELA/Literacy Assessment
10
What Are the Shifts at the Heart of PARCC
Design (and the Standards) for ELA/Literacy?
1. Complexity: Regular practice with complex text and its
academic language
2. Evidence: Reading and writing grounded in evidence from text,
literary and informational
3. Knowledge: Building knowledge through content rich
nonfiction
11
PARCC’s Four Core Commitments to
ELA/Literacy Assessment Quality
1.
Texts Worth Reading: The assessments will use authentic texts worthy of
study instead of artificially produced or commissioned passages
(commissioned passages used only if necessary to “round out” a research
task).
2.
Questions Worth Answering: Sequences of questions that draw students
into deeper encounters with texts will be the norm (as in an excellent
classroom), rather than sets of random questions of varying quality.
3.
Better Standards Demand Better Questions: Instead of reusing existing
items, PARCC will develop custom items to the Standards.
4.
Fidelity to the Standards (now in teachers’ hands): PARCC evidences are
rooted in the language of the Standards so that expectations remain the
same in both instructional and assessment settings.
12
Claims Driving Design: ELA/Literacy
Students are on-track or ready for college and careers
Students read and comprehend a
range of sufficiently complex texts
independently
Reading
Literature
Reading
Informational
Text
Vocabulary
Interpretation
and Use
Students write
effectively when using
and/or analyzing
sources.
Written
Expression
Conventions
and
Knowledge of
Language
Students
build and
present
knowledge
through
research and
the
integration,
comparison,
and synthesis
of ideas.
13
Evidence-Based Questions
Not Text-Dependent
Text-Dependent
In “Casey at the Bat,” Casey strikes out.
Describe a time when you failed at
something.
What makes Casey’s experiences at bat
humorous?
In “Letter from a Birmingham Jail,” Dr.
King discusses nonviolent protest.
Discuss, in writing, a time when you
wanted to fight against something that
you felt was unfair.
What can you infer from King’s letter
about the letter that he received?
From “The Adventures of Tom Sawyer,”
have students identify the different
methods of removing warts that Tom
and Huck talk about. Ask students to
devise their own charm to remove warts.
Are there cultural ideas or artifacts from
the current time that could be used in
the charm?
Why does Tom hesitate to allow Ben to
paint the fence? How does Twain
construct his sentences to reflect that
hesitation? What effect do Tom’s
hesitations have on Ben?
14
Which Prompt Exhibits Writing to
Sources?
In “Letter from Birmingham Jail,” Martin Luther King:
1. Gives several reasons to justify his presence in the city at that time.
Write an essay in which you relate a similar situation in your own life. Tell
about an experience in which you had to justify your reasons for being in
a particular place at a particular time.
2. Describes a process that he and his followers have recently undertaken.
Write an essay in which you describe this process and tell how the letter
shows that this process is important to the civil rights movement.
3. Is specifically responding to criticism about the goals of the civil rights
movement. Write an essay in which you relate these goals to aspects of
the modern-day civil rights movement.
15
Which Questions Require Rigorous
Analysis of Complex Texts?
1. When the author states that “Lee was acting like an angry bear,”
is he using simile, metaphor, hyperbole, or onomatopoeia?
2. What is the setting of the story about Lee’s adventures?
3. What is the relationship between the setting of the story and the
main event of the plot?
4. How did Lee’s decision to stay affect the outcome of the story?
5. Which of the following words describes Lee: brave, determined,
careful, or hopeful?
a.
b.
c.
d.
Questions #1 and #2
Questions #2 and #3
Questions #3 and #4
Questions #4 and #5
16
Sample Items Illustrating the Advances in
ELA/Literacy
17
Understanding the Research Simulation
Task (Grade 7 Example)
• Students begin by reading an anchor text that introduces the
topic. EBSR and TECR items ask students to gather key details
about the passage to support their understanding.
• Students read two additional sources (may include a
multimedia text) and answer a few questions about each text
to learn more about the topic so they are ready to write the
final essay and to show their reading comprehension.
• Finally, students mirror the research process by synthesizing
their understandings into an analytic essay using textual
evidence from several of the sources.
18
Texts Worth Reading?
• Range: Example of assessing reading across the disciplines
and helping to satisfy the 55%-45% split of informational text
to literature at the 6-8 grade band.
• Quality: The texts on Amelia Earhart represent content-rich
nonfiction on a topic that is historically significant.
• Complexity: Quantitatively and qualitatively, the passages
have been validated and deemed suitable for use at grade 7.
19
ELA/Literacy: Grade 7 Sample Item
Earhart and Noonan lived as castaways on Nikumaroro Island.
Claims
Earhart and Noonan’s plane crashed into the Pacific Ocean
People don’t really know where Earhart and Noonan died.
Part A: Highlight the claim that is supported by the most relevant and sufficient facts within
“Earhart’s Final Resting Place Believed Found.”
Part B: Click on two facts within the article that best provide evidence to support the claim
selected in Part A.
20
Questions Worth Answering?
Grade 7 Analytical Prose Constructed-Response Item #1
Based on the information in the text “Biography of Amelia
Earhart,” write an essay that summarizes and explains the
challenges Earhart faced throughout her life. Remember to use
textual evidence to support your ideas.
21
Questions Worth Answering?
Final Grade 7 Prose Constructed-Response Item #2
You have read three texts describing Amelia Earhart. All three include the claim
that Earhart was a brave, courageous person. The three texts are:
• “Biography of Amelia Earhart”
• “Earhart's Final Resting Place Believed Found”
• “Amelia Earhart’s Life and Disappearance”
Consider the argument each author uses to demonstrate Earhart’s bravery.
Write an essay that analyzes the strength of the arguments about Earhart’s
bravery in at least two of the texts. Remember to use textual evidence to
support your ideas.
22
Advances in the PARCC
Mathematics Assessment
23
What Are the Shifts in the Math
Standards at the Heart of PARCC Design?
1. Focus strongly where the Standards focus
2. Coherence: Think across grades and link to major
topics within grades
3. Rigor: In major topics, pursue conceptual
understanding, procedural skill and fluency, and
application.
24
PARCC’s Core Commitments to
Mathematics Assessment Quality

Focus: Instead of randomly sampling a mile-wide array of topics, PARCC
assessments will focus strongly. Teachers will be rewarded for teaching
crucial material in depth, not penalized for failing to “cover topics.”

Problems worth doing: Multi-step problems, conceptual questions,
applications, and substantial procedures will be common, as in an excellent
classroom.

Better Standards Demand Better Questions: Instead of reusing existing
items, PARCC will develop custom items to the Standards.

Fidelity to the Standards (now in Teacher’s hands): PARCC evidences are
rooted in the language of the Standards so that expectations remain the
same in both instructional and assessment settings.
25
Claims Driving Design: Mathematics
Students are on-track or ready for college and careers
Students solve problems
involving the major
content for their grade
level with connections to
practices
Students solve problems
involving the additional
and supporting content
for their grade level with
connections to practices
Students solve real
world problems
engaging particularly in
the modeling practice
Students express
mathematical reasoning
by constructing
mathematical arguments
and critiques
Student demonstrate
fluency in areas set forth
in the Standards for
Content in grades 3-6
26
The Special Role of Technology in
Advancing Assessment and the Shifts
• Technology enhancements supporting accessibility (e.g., the ability
to hover over a word to see and/or hear its definition, etc.)
• Transformative formats making possible what couldn’t be done at
all with paper (e.g., running a simulation to improve a model,
game-like environments, drawing/constructing diagrams or visual
models, etc.)
• Getting beyond the bubble and avoiding drawbacks of traditional
selected response such as guessing or choice elimination
• Capturing complex student responses through a device interface
(e.g., using drawing tools, symbol palettes, etc.).
• Machine scorable multi-step tasks are more efficient to administer
and score.
27
Sample Items Illustrating the Advances in
Mathematics
28
Overview of PARCC Mathematics Task
Types
Task Type
Description of Task Type
I. Tasks assessing
concepts, skills and
procedures
•
•
•
•
•
II. Tasks assessing
expressing
mathematical
reasoning
•
III. Tasks assessing
modeling /
applications
•
•
•
•
•
•
•
•
•
Balance of conceptual understanding, fluency, and application
Can involve any or all mathematical practice standards
Machine scorable including innovative, computer-based formats
Will appear on the End of Year and Performance Based Assessment
components
Sub-claims A, B and E
Each task calls for written arguments / justifications, critique of
reasoning, or precision in mathematical statements (MP.3, 6).
Can involve other mathematical practice standards
May include a mix of machine scored and hand scored responses
Included on the Performance Based Assessment component
Sub-claim C
Each task calls for modeling/application in a real-world context or
scenario (MP.4)
Can involve other mathematical practice standards
May include a mix of machine scored and hand scored responses
Included on the Performance Based Assessment component
Sub-claim D
For more information see PARCC Task Development ITN Appendix D.
29
Math Grade 3 Illustrative Sample Item:
Type II Task
• This a fairly traditional fraction task in a
computer-based setting.
• Unlike traditional multiple choice, it is
difficult to guess the correct answer or use a
choice elimination strategy and there is more
than one correct solution.
• Unlike paper and pencil tests, students can
create a visual representation even though
the task is scored automatically.
30
Math Grade 3 Illustrative Sample Item:
Type II Task
• Second part of multi-step problem, and, unlike traditional multiple choice, it
is difficult to guess the correct answer or use a choice elimination strategy.
31
Math Grade 7 Illustrative Sample
Item: Type I Task
32
Aligns to the Standards and
Reflects Good Practice
Grade 7 Sample Illustrative Item Key Features and Assessment Advances
• The PARCC assessment will seek to preserve the focus of the Standards by
thoroughly exploring the major work of the grade.
• In this case, a multi-point problem is devoted to a single standard about
proportional relationships, which are a major focus in grades 6 and 7.
• Unlike traditional multiple choice, it is difficult to guess the correct answer
or use a choice elimination strategy.
• Variants of the task could probe understanding of unit rates and
representations of proportional relationships by showing different scales on
the two graphs, and/or by presenting the data in tables C and D with the
ordered pairs not equally spaced in time.
33
Additional Sample Items
in ELA/Literacy
34
Item Types That Showcase Students’
Command of Evidence with Complex Texts
• Evidence-Based Selected Response (EBSR)—Combines a traditional selectedresponse question with a second selected-response question that asks
students to show evidence from the text that supports the answer they
provided to the first question. Underscores the importance of Reading Anchor
Standard 1 for implementation of the CCSS.
• Technology-Enhanced Constructed Response (TECR)—Uses technology to
capture student comprehension of texts in authentic ways that have been
difficult to score by machine for large scale assessments (e.g., drag and drop,
cut and paste, shade text, move items to show relationships).
• Range of Prose Constructed Responses (PCR)—Elicits evidence that students
have understood a text or texts they have read and can communicate that
understanding well both in terms of written expression and knowledge of
language and conventions. There are four of these items of varying types on
each annual performance-based assessment.
35
Evidence-Based Selected Response Example
Part A: In paragraph 4 of the text Who Was Marco Polo? , the author states
that an exaggeration became known as a “marco polo.” What is the meaning
of the word exaggeration?
A. a mistake or misunderstanding
B. a long journey
C. an exciting individual
D. an untruth or overstatement*
Part B: Which words from paragraph 3 and 4 best help the reader determine
the meaning of exaggeration?
A. didn’t know much
E. another phrase
B. strange lands
F. wild stories*
C. too incredible*
G. make the crowds laugh
D. the nickname
36
Technology-Enhanced Item Example:
Highlighting Evidence
From the The Great Fire: The passage provides
several reasons that Chicago was a city “ready to
burn.” The passage also gives reasons that
Chicago burned down when it did, rather than at
an earlier or later time.
• In the passage, highlight two sentences that
provide reasons that the city burned in
October 1871 rather than earlier or later.
37
Technology-Enhanced Item Example:
Drag and Drop
The passage gives reasons that explain why Marco Polo may have been
truthful in his book and also gives reasons that explain why he may have
made up his stories about China. The headings in the chart below list these
two different ideas from the text. Complete each row of the chart by
dragging and dropping facts and details from the text to support each idea.
The first row has been done for you.
Evidence from the text that Marco Polo Evidence from the text that Marco Polo
may have told the truth in his book
may not have told the truth in his book
But a list of his belongings around the
He said the Chinese city of Hangchow
time of death suggests that he did leave
had twelve thousand bridges, but it had
behind one of Kublai Khan’s gold
far fewer.
tablets.
38
Understanding the Narrative
Writing Task (Grade 6 Example)
• Students read one or two brief texts and answer a few
questions to help clarify their understanding of the text(s).
• Students then write either a narrative story or a narrative
description (e.g., writing a historical account of important
figures; detailing a scientific process; describing an account
of events, scenes, or objects).
39
Texts Worth Reading?
• Range: Example of assessing literature and helping to satisfy the
55%-45% split of informational text to literature at the 6-8 gradeband.
• Quality: Julie of the Wolves was a winner of the Newbery Medal in
1973. This text about a young Eskimo girl surviving on her own in
the tundra by communicating with wolves offers a story rich with
characterization and imagery that will appeal to a diverse student
population.
• Complexity: Quantitatively and qualitatively, the passages have
been validated and deemed suitable for use at grade 6.
40
ELA/Literacy: Grade 6 Sample Item
41
ELA/Literacy: Grade 6 Sample Item
42
Questions Worth Answering?
Grade 6 Prose Constructed Response from Narrative
Writing Task
In the passage, the author developed a strong character
named Miyax. Think about Miyax and the details the author
used to create that character. The passage ends with Miyax
waiting for the black wolf to look at her.
Write an original story to continue where the passage ended.
In your story, be sure to use what you have learned about the
character Miyax as you tell what happens to her next.
43
Additional Sample Items
in mathematics
44
Math High School Illustrative Sample
Item: Type I Task
 Item has two possible solutions
 Students have to recognize the nature of the equation to know how to solve
 Technology prevents guessing and working backward
45
Aligns to the Standards and
Reflects Good Practice
High School Illustrative Item Key Features and Assessment Advances
The given equation is quadratic equation with two solutions. The task does not clue the
student that the equation is quadratic or that it has two solutions; students must
recognize the nature of the equation from its structure. Notice that the terms 6x – 4 and
3x – 2 differ only by an overall factor of two. So the given equation has the structure
𝑄2 = 2𝑄
where Q is 3x – 2. The equation Q2 - 2Q is easily solved by factoring as Q(Q-2) = 0, hence
Q = 0 or Q = 2. Remembering that Q is 3x – 2, we have
3𝑥 − 2 = 0 or 3𝑥 − 2 = 2.
These two equations yield the solutions 23 and 43.
Unlike traditional multiple-choice tests, the technology in this task prevents guessing
and working backwards. The format somewhat resembles the Japanese University
Entrance Examinations format (see innovations in ITN Appendix F). A further
enhancement is that the item format does not immediately indicate the number of
solutions.
46
Math High School Illustrative Sample
Item: Type III Task
47
http://www.ccsstoolbox.com/parcc/PARCCPrototype_main.html
MP 1, 2, 3; F-BF.1; F-LE.2,3
Math High School Illustrative Sample
Item: Type III Task
48
http://www.ccsstoolbox.com/parcc/PARCCPrototype_main.html
MP 1, 2, 3; F-BF.1; F-LE.2,3
Math High School Illustrative Sample
Item: Type III Task
49
http://www.ccsstoolbox.com/parcc/PARCCPrototype_main.html
MP 1, 2, 3; F-BF.1; F-LE.2,3
www.PARCConline.org
50