Depth of
Knowledge –
Going Deeper
Into Your
Content with
ALL Students
David. A Johnson
Social Studies Consultant
Northern Michigan Learning Consortium
David.johnson@wmisd.org
June 2013
Welcome!
 Introductions
 Overview
of the day
 Turn and Talk – What do you hope to
leave today equipped to do?
The Top Questions So Far…
 Isn't
Smarter Balanced just an ELA and
Math test?
 How will the NGSS and C3 Framework
come into play?
 So the MEAP, MME, and ACT are going
away?
 Isn’t the Common Core going away? I
heard that the legislature…
 What else???
An Honest Assessment of Skills

Step 1: Do a gallery walk through your content area
(or closest match…special area teachers)
Step 2: Return to your seats – What did you notice?
What skills were being asked? How nervous are you?
Step 3: Grab a sticky note. Place one sticky note on
each poster in one of three locations: (Bottom left:
Piece of cake, bottom middle: One or two students
might struggle, but for the most part they’d be okay.
Bottom right: Generally, most of my students would
struggle with this.

What does the data we have just collected tell us.


Pre-Break Activity
 Pick
one item from the gallery walk and
look at the DOK handout.
 Quickly look at your performance task as
well as the handout, and rank where you
believe the task falls in terms of DOK. Give
a sentence or two explanation of why you
selected the level you did.
Learning Targets
 "I
can recall what Webb's DOK is."
 "I can recall critical characteristics of
each of the four DOK levels"
 “I can explain the attributes of each of
Webb's DOK levels"
Purpose
 Purpose
in studying DOK: To ensure instruction and
assessment match the depth of knowledge in our
standards.
Cognitive Demand
"I can recall what Webb's DOK is."
Why Depth of Knowledge
• Mechanism to ensure that the intent of the standard
and the level of student demonstration required by
that standard matches the assessment items
(required under NCLB)
• To ensure that teachers are teaching to a
level that will promote student achievement
"I can recall what Webb's DOK is."
DOK is NOT…
•
•
•
a taxonomy (Bloom’s)
the same as difficulty
about using “verbs”
"I can recall what Webb's DOK is."
It’s Not About the Verb
•
The Depth of Knowledge is NOT determined by the verb
(Bloom’s Taxonomy), but by the context in which the verb is
used and the depth of thinking required.
"I can recall what Webb's DOK is."
Verbs are not always used
appropriately...
Words like explain or analyze have to be considered in context.
• “Explain to me where you live” does not raise the DOK of a
simple response.
• Even if the student has to use addresses or landmarks, the
student is doing nothing more than recalling and reciting.
"I can recall what Webb's DOK is."
DOK is about what follows the
verb...
What comes after the verb is more important than the verb itself.
Analyze the accuracy /placement of the steps in the flowchart of
“How a Bill Becomes A Law”
The student can be taught the process by which a bill becomes a
law but it doesn’t provide the evidence that they can explain
each step
"I can recall what Webb's DOK is."
"I can recall critical characteristics of
each of the four DOK levels”
Same Verb – Three Different DOK
Levels – Social Studies

Level 1 Describe a core ideal of American
society

Level 2 Describe how peoples’ point of view
may differ on a core ideal of American
Society

Level 3 Describe how peoples’
misconceptions about American Core ideals
causes conflict over the role of government in
American Society
Same Verb – Three Different DOK
Levels – Science
 Level
1 Measure temperature of water
 Level
2 Measure temperature of water at
different times/places
 Level
3 Measure temperature of water at
different times/places to develop a
scientific model that accounts for the
temperature variations.
Same Verb – Three Different DOK
Levels – Mathematics

Level 1 Compare prices to destinations and
identify the lowest price.

Level 2 Compare and contrast the price of
airline tickets from multiple airlines to the same
destination.

Level 3 Compare prices to research and
recommend a travel plan that includes the
best price in airfare, hotel, and rental car
accommodations. Justify your answer.
Same Verb – Three Different DOK
Levels – ELA
 Level
1 Explain what a Simile is.
 Level
2 Explain how a simile is alike and
different from a metaphor.
 Level
3 Explain how similes and
metaphors can enhance a reader’s
understanding of a text.
DOK is about intended
outcome, not difficulty.
DOK is a reference to the complexity of mental
processing that must occur to answer a question,
perform a task, or generate a product.
• For example: Students could recall the date of the signing of the
Declaration of Independence
• Students could also recall the parties involved in signing the
Declaration of Independence
• Students could also recall the opposition to the signing of the
Declaration of Independence.
"I can recall what Webb's DOK is.”
"I can recall critical characteristics of each of the
four DOK levels”
DOK is NOT about difficulty
• Difficulty is a reference to how many students answer a
question correctly.
Words below taken from the Declaration of Independence
“How many of you know the definition of dissolve?”
DOK 1 – recall
If all of you know the definition, this question is an easy question.
“How many of you know the definition of evinces?”
DOK 1 – recall
If most of you do not know the definition, this question is a difficult
question.
"I can recall what Webb's DOK is."
"I can recall critical characteristics of each of the
four DOK levels”
DOK IS about Complexity
• The type of thinking and application of
intended student learning outcome determines
the DOK level.
• Every CCSS has been assigned a DOK level
for SBAC
• Instruction and classroom assessments must
reflect the DOK level of the objective or
intended learning outcome.
"I can recall what Webb's DOK is."
"I can recall critical characteristics of each of the four DOK
levels”
Quick Quiz
1) What is the relationship of verbs and Depth of Knowledge?
2) What is the difference between difficulty and complexity?
Turn and Talk…Discuss your answers with your partner…
"I can recall what Webb's DOK is."
"I can recall critical characteristics of each of the
four DOK levels”
Why are we studying DOK?
• Provides a scale of cognitive demand (thinking) with
which to align standards with assessments
• Defines the “ceiling” or highest DOK level for a CCSS
• Guides item development for SBAC
"I can recall what Webb's DOK is."
"I can recall critical characteristics of each of the
four DOK levels”
Webb’s Four Levels of Cognitive
Complexity
•
•
•
•
Level 1: Recall and Reproduction of
Information
Level 2: Basic Reasoning (Skills & Concepts)
Level 3: Strategic Thinking/Complex
Reasoning
Level 4: Extended Thinking/Reasoning
“I can explain the attributes of each of Webb's
DOK levels"
AB Each Teach
 Group
#1 – Read and summarize Level 1 and 2
Webb DOK
 Group #2 – Read and summarize Level 3 and 4
 Partner A reads one level; partner B reads the
other level; Teach one another and generalize
each level with supporting text
 Be prepared to report out
Level 1 – Recall and Reproduction
•
•
•
•
•
•
Recall elements and details of story structure such
as sequence of events, character, plot, and
setting.
Conduct basic mathematical calculations.
Label locations on a map
Represent in words or diagrams a scientific
concept or relationship.
Perform routine procedures like measuring length
or using punctuation marks correctly.
Describe the features of a place or people
“I can explain the attributes of each of Web's DOK
levels"
Level 2 – Skills and Concepts
•
•
•
•
•
•
•
Identify and summarize the major events in a
narrative.
Use context cues to identify the meaning of
unfamiliar words.
Solve routine multiple-step problems.
Describe the cause/effect of a particular event.
Identify patterns in events or behavior.
Formulate a routine problem given data and
conditions.
Organize, represent, interpret data.
“I can explain the attributes of each of Web's DOK
levels"
Level 3 –Strategic Thinking/Reasoning
•
•
•
•
•
•
Support ideas with details and examples.
Use voice appropriate to the purpose and audience.
Identify research questions and design investigations for a scientific
problem.
Develop a scientific model for a complex situation.
Determine the author’s purpose and describe how it affects the
interpretation of a reading selection.
Apply a concept in other texts.
“I can explain the attributes of each of Web's DOK
levels"
Level 4 – Extended Thinking
•
•
•
•
•
Conduct a project that requires specifying a problem,
designing and conducting an experiment, analyzing its data,
and reporting results/solutions.
Apply mathematical model to illuminate a problem or
situation.
Analyze and synthesize information from multiple sources.
Describe and illustrate how common themes are found across
texts from different cultures.
Design a mathematical model to inform and solve a practical
or abstract situation.
“I can explain the attributes of each of Web's DOK
Check In:
 What
questions, comments, or concerns might
you have about the content we’ve covered?
 Do
you still agree with the DOK level you
assigned to the sample prompt this morning?
Provide justification for your answer.
Directions for Practice Activity
 You
are about to see several statements
without a DOK level assigned.
 FIRST: Individually, rank each statement.
 SECOND: Take a moment to discuss each
statement in your table groups.
 THIRD: We will discuss as a whole group.
Practice
 Describe
the physical features of a place .
1
a problem, identify solution paths, solve the
problem and report the results.
4
 Specify
cause-effect of historical events .
2
or evaluate the effectiveness of literary
elements (plot, setting, conflict, point-of-view…).
3
 Explain
 Analyze
 Identify
and summarize the major events, problem,
2
solution, conflicts in a literary text.
 Using a graph, predict how many teeth would be lost 3
by all the 2nd grade classes in your schools and
justify your answer.
• Locate or recall facts found in text .
• Gather, analyze, organize and interpret data from multiple
(print and non-print sources) to draft a reasoned report .
• Analyze and explain multiple perspectives or issues within or
across time periods or events .
1
4
4
• Compare desert and tropical areas .
2
• Identify and summarize the major events, problem, solution,
conflict in a literary text .
• Research and design a system to lift heavy objects using the
conversion of electrical energy to mechanical energy. Build a
prototype of the system using materials found in the
classroom.
• Critique, compare, and contrast pieces of music from the
Baroque and classical periods.
2
3
3
Collaboration

Elementary group – Tomorrow’s Focus


Middle School group – Tomorrow’s Focus


Focus on literacy (reading and writing) and lesson
development design with a greater DOK that will impact
instruction.
Designing lessons and instructional practices wrapped
around the UDL Concepts to impact DOK
High School group – Tomorrow’s Focus

Develop assessment questions and format designed to
impact DOK
Use your remaining time today to discuss the following question in your
groups:
As we get together tomorrow to work collaboratively, we are going to
need some more information/resources on…
What changes in instructional focus and practices are needed to “go
deeper?”
Works Referenced
 Portions
of this presentation adapted from
Measured Progress presentation © 2011
 Portions of this presentation adapted from WexfordMissaukee ISD Curriculum Review (ELA) 2011-2012
 Portions of this presentation adapted from
Empowering SS Teachers for SMARTER Balanced
Assessment 2012-2013. Dave Johnson, Lindsay
Brindley, Rebecca Bush, Marshall Collins, David
Hales, Lena Nemeth, Roy Sovis, Cheryl Wyatt.