Transition to PA Common Core:
Rigor in Learning and
Assessment of Learning
Copyright ©2010 Commonwealth of Pennsylvania
1
PA
Common
Core
Local Curriculum
Toolbox
Copyright ©2010 Commonwealth of Pennsylvania
2
Please do the Following:
• Connect to the Internet
• Navigate to: http://www.pdesas.org
– If a registered user, sign-in
– If not a registered user, join now
• Place your name and school
district/organization on your name tent
Your School District/Organization
Your Name
Copyright ©2010 Commonwealth of Pennsylvania
3
PA Common Core Introduction
Essential Questions
• What is rigor and how is it represented in
mathematical assessments?
• What does engagement look like for students in
mathematics and how is this captured in
assessments?
• What does relevance look like in terms of assessment
design?
• How can teachers design assessments that are
rigorous, engaging, and relevant that capture the
features of the “PA Common Core Standards?”
Copyright ©2010 Commonwealth of Pennsylvania
4
Rethinking Assessment
Activity #1
• How do you define rigorous assessment?
• Think-pair-share to create a working definition.
• Read 4 Myths about Rigor in the Classroom by
Ronald Williamson and Barbara R. Blackburn.
http://tinyurl.com/4RigorMyths
• With your partner, revise your shared working
definition of rigorous assessment. Be prepared to
report out to the group.
Copyright ©2010 Commonwealth of Pennsylvania
5
Depth of Knowledge
(DOK)
• Adapted from the model used by Norman
Webb, University of Wisconsin, to align
standards with assessments
• Used by the Council of Chief State School
Officers (CCSSO) for assessment alignment
in more than ten states
Copyright ©2010 Commonwealth of Pennsylvania
6
Why Depth of
Knowledge?
Why Depth of Knowledge?
• Mechanism to ensure that the intent of the
standard and the level of student demonstration
required by that standard matches the assessment
items (required under NCLB)
• Provides cognitive processing ceiling (highest level
students can be assessed) for item development
• Designed for use with formative and summative
assessments
Copyright ©2010 Commonwealth of Pennsylvania
7
Depth of Knowledge
(DOK)
Webb’s Depth of Knowledge
Recall and Reproduction
Level 1
Skills and Concepts
Level 2
Strategic Thinking
Level 3
Extended Thinking
Level 4
Copyright ©2010 Commonwealth of Pennsylvania
8
Depth of Knowledge
(DOK)
Recall and Reproduction: Level 1
• DOK 1 requires recall of information, such as a fact,
definition, term, or performance of a simple process
or procedure, as well as performing a simple
algorithm or applying a formula.
• Answering a Level 1 item can involve following a
simple, well-known procedure or formula. Simple
skills and abilities or recall characterize DOK 1.
Copyright ©2010 Commonwealth of Pennsylvania
9
Depth of Knowledge
(DOK)
DOK Level 1 Examples
1. Identify a diagonal in a geometric figure.
2. Multiply two numbers.
3. Find the area of a rectangle.
4. Convert scientific notation to decimal form.
5. Measure an angle.
Source: Kentucky Department of Education (2007). Support Materials for Core Content for
Assessment.
Copyright ©2010 Commonwealth of Pennsylvania
10
Depth of Knowledge
(DOK)
Skills/Concepts: Level 2
• DOK 2 includes the engagement of some mental processing
beyond recalling or reproducing a response. Items require
students to make some decisions as to how to approach the
question or problem.
• Keywords distinguishing Level 2 may include classify, organize,
estimate, make observations, collect and display data, and
compare data.
• These actions imply more than one mental or cognitive
process/step.
Copyright ©2010 Commonwealth of Pennsylvania
11
Depth of Knowledge
(DOK)
DOK Level 2 Examples
1. Classify quadrilaterals.
2. Compare two sets of data using the mean, median,
and mode of each set.
3. Determine a strategy to estimate the number of jelly
beans in a jar.
4. Extend a geometric pattern.
5. Organize a set of data and construct an appropriate
display.
Source: Kentucky Department of Education (2007). Support Materials for Core Content for
Assessment.
Copyright ©2010 Commonwealth of Pennsylvania
12
Depth of Knowledge
(DOK)
Strategic Thinking: Level 3
• DOK 3 requires reasoning, planning, using evidence,
and more demanding cognitive reasoning. The
cognitive demands at Level 3 are complex and
abstract.
• An assessment item that has more than one possible
answer and requires students to justify the response
they give would most likely be a Level 3.
Copyright ©2010 Commonwealth of Pennsylvania
13
Depth of Knowledge
(DOK)
DOK Level 3 Examples
1. Solve a multiple-step problem and provide support with a
mathematical explanation that justifies the answer.
2. Write a mathematical rule for a non-routine pattern.
3. Explain how changes in the dimensions affect the area and
perimeter/circumference of geometric figures.
4. Provide a mathematical justification when a situation has
more than one outcome. Interpret information from a series
of data displays.
Source: Kentucky Department of Education (2007). Support Materials for Core Content for
Assessment.
Copyright ©2010 Commonwealth of Pennsylvania
14
Depth of Knowledge
(DOK)
Extended Thinking: Level 4
• DOK 4 requires high cognitive demand and is very complex. It
requires complex reasoning, planning, developing, and
thinking.
• Students are expected to make connections - relate ideas
within the content or among content areas — and select or
devise one approach among many alternatives on how the
situation can be solved.
• Due to the complexity of cognitive demand, DOK 4 often
requires an extended period of time.
Copyright ©2010 Commonwealth of Pennsylvania
15
Depth of Knowledge
(DOK)
Extended Reasoning/Thinking Examples: Level 4
1. Specify a problem, identify solution paths, solve the problem,
and report the results.
2. Collect data over time taking into consideration a number of
variables and analyze the results.
3. Model a social studies situation with many alternatives and
select one approach to solve with a mathematical model.
4. Develop a rule for a complex pattern and find a phenomenon
that exhibits that behavior.
5. Complete a unit on formal geometric constructions, such as
nine-point circles or the Euler line.
6. Construct a non-Euclidean geometry.
Source: Kentucky Department of Education (2007). Support Materials for Core Content for
Assessment.
Copyright ©2010 Commonwealth of Pennsylvania
16
Depth of Knowledge
(DOK)
The Depth of Knowledge is NOT determined by the verb, but
the context in which the verb is used and the depth of
thinking required.
DOK 3- Describe a model that you might use to represent the
relationships that exist within the rock cycle. (requires deep
understanding of rock cycle and a determination of how best
to represent it)
DOK 2- Describe the difference between metamorphic and
igneous rocks. (requires cognitive processing to determine the
differences in the two rock types)
DOK 1- Describe three characteristics of metamorphic rocks.
(simple recall)
Copyright ©2010 Commonwealth of Pennsylvania
17
Depth of Knowledge
(DOK)
Students will identify and describe properties of and
apply geometric transformations within a plane to
solve real-world and mathematical problems.
DOK 1: Students will identify a transformation within a
plane.
Source: Kentucky Department of Education (2007). Support Materials for Core Content for
Assessment.
Copyright ©2010 Commonwealth of Pennsylvania
18
Depth of Knowledge
(DOK)
Students will identify and describe properties of and
apply geometric transformations within a plane to
solve real-world and mathematical problems.
DOK 2: Students will perform a compound
transformation of a geometric figure within a
coordinate plane.
Source: Kentucky Department of Education (2007). Support Materials for Core Content for
Assessment.
Copyright ©2010 Commonwealth of Pennsylvania
19
Depth of Knowledge
(DOK)
Students will identify and describe properties of and
apply geometric transformations within a plane to
solve real-world and mathematical problems.
DOK 3: Students will perform a geometric
transformation to meet specified criteria and then
explain what does or does not change about the figure.
Source: Kentucky Department of Education (2007). Support Materials for Core Content for
Assessment.
Copyright ©2010 Commonwealth of Pennsylvania
20
Depth of Knowledge
(DOK)
Students will identify and describe properties of and
apply geometric transformations within a plane to
solve real-world and mathematical problems.
DOK 4: Students will abstract the transformations
occurring in an Escher woodprint and then create a
simplified tessellation of their own.
Source: Kentucky Department of Education (2007). Support Materials for Core Content for
Assessment.
Copyright ©2010 Commonwealth of Pennsylvania
21
Depth of Knowledge
(DOK)
Applying Webb’s Depth of Knowledge to Bloom’s
Cognitive Process Dimensions for Math and Science
Hess’ Cognitive Rigor Matrix & Curricular Examples for
Math and Science
www.nciea.org/publications/CRM_math-sci_KH11.pdf
Source: Karen Hess (2009)
Copyright ©2010 Commonwealth of Pennsylvania
22
Depth of Knowledge
(DOK)
Activity #2
Depth of Knowledge Math Sorting Activity
1. Working individually, classify each mathematical
assessment task according to its DOK level.
2. Work with a partner to arrive at consensus as to
what DOK level each task reflects.
3. Report out responses and the rationale of such
responses.
Copyright ©2010 Commonwealth of Pennsylvania
23
Assessment AnalysisRigor
Activity #3: Your Turn!
Assessment Analysis - Rigor
1. Using the DOK levels, analyze one of your
assessments and classify each assessment
item according to one of the DOK levels.
2. Choose one question representing a DOK 1.
How might you revise that question to
elevate the rigor?
Copyright ©2010 Commonwealth of Pennsylvania
24
Depth of Knowledge
(DOK)
Questions to Consider When Assessing Learning…
1. What DOK levels should you be targeting with
assessment questions?
2. What would you consider to be an appropriate
distribution of the different DOK levels in an
assessment?
3. What concerns do you have regarding the different
levels of DOK?
4. How might you build DOK Level 4 questions into
your unit/lesson/assessment design?
Copyright ©2010 Commonwealth of Pennsylvania
25
Depth of Knowledge
(DOK)
Questions to Consider When Designing Learning…
1. What DOK levels should you be targeting in learning
activities?
2. What would you consider to be an appropriate
distribution of instructional time across the four
DOK Levels?
3. What concerns do you have regarding the different
levels of DOK with respect to learning?
4. How might you build DOK Level 4 activities into
your unit/lesson design?
Copyright ©2010 Commonwealth of Pennsylvania
26
Assessment AnalysisEngagement
Standards for Mathematical Practice
• Apply across Grades K-12.
• Describe ways in which students should
engage with the content, process, and
proficiencies of mathematics.
Copyright ©2010 Commonwealth of Pennsylvania
27
Assessment AnalysisEngagement
PACC Standards of Mathematical Practice
• Make sense of problems and persevere in solving them.
• Use appropriate tools strategically.
• Reason abstractly and quantitatively.
• Attend to precision.
• Construct viable arguments and critique the reasoning
of others.
• Look for and make use of structure.
• Model with mathematics.
• Look for and make sense of regularity in repeated
reasoning.
Copyright ©2010 Commonwealth of Pennsylvania
28
Assessment AnalysisEngagement
Activity #4: Your Turn!
Assessment Analysis-Engagement
1. Analyze assessment items through the lens of the mathematical
practices (www.pdesas.org/Standard/CommonCore).
2. Choose an assessment item to revise to better target one
specific mathematical standard of practice.
3. Share with a partner as to what that assessment item may look
like once it is revised.
4. Work with your partner to revise/expand the assessment item
to further address the mathematical standard of practice.
Copyright ©2010 Commonwealth of Pennsylvania
29
Assessment AnalysisRelevance
Characteristics of PA Common Core State Standards
• Building on a foundation before moving on.
• Hands-on exploration.
• Real world application emphasizing mathematical
modeling.
• Creating students who know math, do math, and use
math.
• Requiring students to make connections and relate ideas
within the content or among content areas while
selecting or devising one approach among many
alternatives on how the situation can be solved.
Copyright ©2010 Commonwealth of Pennsylvania
30
Assessment Analysis-Relevance
PACC 2.3.7.A.1 (Geometry)
Solve real-world and mathematical problems involving angle
measure, area, surface area, circumference, and volume.
Copyright ©2010 Commonwealth of Pennsylvania
31
Assessment AnalysisRelevance
Assessment Task Examples
• Perhaps high school students may build a swimming pool that can
meet the needs of ALL people who want to use it - from those who
have special needs to children; but at the same time, it needs to
meet certain criteria in terms of standard amounts of water and
size.
• Perhaps middle school students may be in charge of designing a
new and improved pyramid to be presented to the pharaoh complete with a variety of antechambers.
• Perhaps elementary students may be in charge of creating an
organic garden to sell certain products at the local farmer's
market.
Source: Andrew Miller (2012) http://www.andrewkmiller.com/2011/07/assessing-the-common-corestandards-real-life-mathematics/
Copyright ©2010 Commonwealth of Pennsylvania
32
Assessment AnalysisRelevance
Activity #5: Your Turn!
Assessment Analysis of Relevance
1. Analyze assessment items through the lens of relevance and real
world contexts. Do your assessments reflect real world learning
contexts for your students?
2. Choose an assessment item to revise to embed more of a real
world, relevant context.
3. Share with a partner as to what that assessment item may look
like once it is revised.
4. Work with your partner to revise/expand the assessment item to
reflect a real world, relevant context.
Copyright ©2010 Commonwealth of Pennsylvania
33
Building Assessment
Teams
Building Assessment Teams
• Working collaboratively to improve assessments to capture
rigor, engagement, and relevance
– Analyzing DOK levels of assessment
– Analyzing mathematical standards of practice in
assessments and assessment tasks
– Analyzing assessment design to reflect real-world context
• Using team members as a fidelity check to ensure that
assessment questions are written that align to the teacher’s
intent of the design of the task
Copyright ©2010 Commonwealth of Pennsylvania
34
Building Assessment
Teams
Building Assessment Teams
• Use each other as a resource. Writing DOK Levels of 3 or 4
assessment tasks may be different and more difficult than
writing DOK Levels 1 or 2.
• Create assessment questions based upon same Assessment
Anchor. Compare with colleagues to review differences in
assessment questions.
• Determine individual assessment areas of strength and areas
of need. As assessment teams, understand each other’s
strengths and use those strengths when designing assessment
items.
Copyright ©2010 Commonwealth of Pennsylvania
35
Building Assessment
Teams
Questions for consideration and extensions of
assessment teams:
1. What should be considered the target score on an
assessment in order to move forward with
instruction?
– What score is considered acceptable showing that your
students get it?
2. What actions should be taken as a result of the
assessment score? (remediation, enrichment, etc.)
3. How might these changes in assessment drive
changes in instruction?
Copyright ©2010 Commonwealth of Pennsylvania
36
Building Assessment
Teams
Building Assessment Teams
• What additional questions should assessment
teams be asking each other?
• What additional questions should teachers be
asking of their assessments?
Copyright ©2010 Commonwealth of Pennsylvania
37
Building Assessment
Teams
Activity #6:
Assessment Teams - Successes and Challenges
1. Write down two successes that you have experienced or
would like to experience as a result of working with an
assessment team.
2. On another card, write down two challenges that you
have experienced or might anticipate as a result of
working with an assessment team.
3. Share your successes and challenges with a shoulder
partner. Discuss strategies that could help assure the
successes and minimize the challenges.
Copyright ©2010 Commonwealth of Pennsylvania
38
Assessment
Questions???
Copyright ©2010 Commonwealth of Pennsylvania
39