Increased Rigor
Through Questioning
& Classroom
ActivitiesAn Understanding of
Webb’s Depth of Knowledge & Bloom’s
Taxonomy
Housekeeping
Session 8 am-3 pm
Basic Needs
Common Courtesy
Lunch 11 am-12 pm
Open Forum
Participant Packet & Table
Sets
KWL
K
W
L
“He who learns but does
not think, is lost.
He who thinks, but does
not learn is in great
danger.”
Factors that Correlate to
Student Achievement Rates
National Assessment of Educational Progress (NAEP)
Efforts to Improve Student Learning
Class Size Reduction
Whole School Reform
Re-vamp Class time
(varied bell schedules, year-round schools, block schedules)
Innovative Curriculum
Traditional Curriculum (Back to Basics)
Remediation Programs (Tracking, two-year algebra, etc.)
Standards Based Education
(Pacing Guides, Benchmark Test, Data Driven Decision-Making, etc.)
High-stakes Accountability
(Rewards, Sanctions, Differentiated Accountability)
Choice (charter schools, magnet schools, etc.)
Centralized Leadership and Policies (state or
national)
Professional Learning Communities
So...what is the most significant
factor in student learning?
...the teacher
Teachers are the Key
“Teachers must be the primary
driving force behind change. They
are best positioned to understand
the problems that students face
and to generate possible
solutions.”
Quality Instruction Makes A Difference
“Good teaching can make a
significant difference in student
achievement, equal to one effect
size (a standard deviation), which
is also equivalent to the affect
that demographic classifications
can have on achievement.”
Paraphrase Dr. Heather Hill, University of Michigan
Differences in Instruction
“Our research indicates that there is
a 15% variability difference in
student achievement between
teachers within the same schools.”
Deborah Loewenberg Ball, Dean of Education, University of Michigan
“What Matters Very Much is
Which Classroom?”
“If a student is in one of the most effective classrooms
he or she will learn in 6 months what those in an
average classroom will take a year to learn. And if a
student is in one of the least effective classrooms in that
school, the same amount of learning take 2 years.”
Research has indicated
that... “teacher quality
trumps virtually all
other influences on
student achievement.”
(e.g., Darling-Hammond, 1999; Hamre and Pianta,
2005; Hanushek, Kain, O'Brien and Rivken, 2005;
Wright, Horn and Sanders, 1997)
What does the research
say…
about a “guaranteed and viable
curriculum?”
Curriculum: Student
Achievement
Marzano’s research on school
effectiveness shows that the
development of a guaranteed and
viable curriculum provides the
greatest impact on student
achievement.
Marzano, R. J. (2003). What works in schools: Translating research
into action. Alexandria, VA: Association for Supervision and
Curriculum Development.
Curriculum: School
Improvement
A guaranteed, viable curriculum is
“arguably the single most important
pre-condition for school
improvement” (p. 41).
Schmoker, M. (2006). Results now: How we can achieve
unprecedented improvements in teaching and learning. Alexandria,
VA: Association for Supervision and Curriculum Development.
Curriculum: Challenging
Standards
As Wiener and Hall (2004) traveled across the
country visiting schools that were successfully
closing gaps and raising achievement for
students, they found that one of the major
factors that had an impact on this academic
success was the presence of a rigorous
curriculum tied to challenging standards.
Wiener, R., & Hall, D. (2004, September/October). Accountability under no
child left behind. The Clearing House, 78(1), pp. 17-21.
Curriculum: Closing the
Gap
The secret to closing the achievement gaps
and making significant progress in student
assessment results is accomplished by holding
all students accountable to the same rigorous
standards regardless of demographic
subgroup.
Wiener, R., & Hall, D. (2004, September/October). Accountability under no
child left behind. The Clearing House, 78(1), pp. 17-21.
Toffler (1979)
The definition of “illiterate”
st
21
The illiterate of the
century will not
be those who cannot read and write, but
those who cannot learn, unlearn, and
relearn.
Making Sense &
Worthwhile Tasks
“What are our kids really being asked to
do?”
“How are we keeping up with
Cognitive Demand?”
Cognitive Demand
How does all of this connect?
Depth of Knowledge (DOK)
No Child Left Behind (NCLB) requires
assessments to “measure the depth
and breadth of the state academic
content standards for a given grade
level”.
(U.S.
Department of Education,
2003, p. 12)
How does all of this connect?
Depth of Knowledge (DOK)
The intent of the accountability
development process is to design a new
accountability system rather than modify
the current system. The new system may
look very different from the current state
accountability system.
State Board of Education, 2011
How does all of this connect?
Depth of Knowledge (DOK)
STAAR will focus on “clearer, fewer, and
deeper”Provide a more clearly articulated
assessment program
Focus on fewer skills
Address those skills in a deeper manner
Texas Education Agency, 2011
How does all of this connect?
Rigor on STAAR/EOC
FOCUS
Readiness & Supporting
assessing more focused student expectations but doing so multiple
times and in more complex ways
Emphasizing the content standards that best prepare students for
the next grade or course.
Focus is fewer skills, but addresses skills in a deeper manner
CLARITY
State standards (TEKS) are used to determine the STAAR
program’s structure and content
DEPTH
assessing content and skills at a greater depth and higher level of
cognitive complexity
using empirical data to link performance in specific courses to
college and career readiness
Projected Level of Rigor
on STAAR
Level 3
6% tested
Level 2
39% Tested
Level 1
55% tested
Why Depth of
Knowledge?
What is Depth of
Knowledge?
What does Depth of
Knowledge accomplish?
How does Depth of
Knowledge work?
Why Depth
of Knowledge (DOK)?
Mechanism to ensure that the intent of the
standard and the level of student demonstration
required by that standard matches the assessment
items and performance indicators
(required under NCLB)
To ensure that teachers are teaching to a
level that will promote student achievement
DOK
Wheel
Questioning Levels: A Cinderella Story
Level I: Recall
• The answer is in the text;
explicit, fact (fully and
clearly expressed;
leaving nothing implied)
• What were Cinderella's
slippers made out of?
• How did Cinderella get to
the ball?
Level II:
Analysis/Inference
Level III:
Synthesis
• Implicit or what is
implied; analysis, ask
how and why, require
analysis of the text,
reading between the
lines, hidden meaning.
(understood though not
directly expressed)
• Go beyond the text
and inquire into the
value, importance
and application of
the information
presented.
• Why does Cinderella's
stepmother care
whether or not she goes
to the ball?
• Why did everything turn
back the way it was
except the glass
slipper?
• Why don't the step
sisters like Cinderella?
• Does a woman's
salvation always
lie with a man?
• What does it
mean to live
happily ever
after?
• Does good
always
overcome evil?
Practice time! Activity #1
1. Take the Cinderella handout from your
packet and select at least one question
for which to refine the rigor.
2. How can you up the level of your
selected question(s)?
3. Be prepared to share.
Guidelines
Degree of Rigor
Requires instruction that targets
students’ abilities to infer,
organize, imagine, and revise
their understandings. It teaches
students skills they need to adapt
to any situation and become
lifelong learners-quality NOT
quantity.
Components of Rigor
Assists students in fulfilling predetermined
outcomes and competencies by challenging
them with high expectations.
Essential components of rigor in the classroom:
•
•
•
•
•
•
•
Content acquisition
Critical thinking
Relevance
Integration
Application of concepts
Long term retention
Responsibility
Let’s take a look…
Is this a low, moderate, or high level question?
Is this a low, moderate, or high level question?
DOK Activity #2: Leveling
Questions
1. In the large manila envelope, take
the questions and try to level them.
2. Discuss what you believe to be
the fundamental differences
between the 1’s, 2’s, and 3’s.
DOK is NOT...
•
a taxonomy (Bloom’s)
•
the same as difficulty
•
about using “verbs”
Bloom’s Taxonomy
CATEGORY
Knowledge (recalling-eliciting
factual answers)
Comprehension (grasping
meaning, translating,
interpreting, extrapolating)
ACTIONS/SKILLS DEMONSTRATED
Ask, cite, count, define, indicate, inquire, know, list,
locate, name, recite, state, tabulate, tell,
Associate, classify, compare, convert describe,
explain, extrapolate, give examples, identify,
interpret, match, measure, put in order, recognize,
report, restate, specify, stipulate, summarize,
translate
QUESTION STEMS
Who, What, Why, When, Where, How, How much,
What does it mean, Which one, Match, Choose
State in your own words, Give an example,
Condense the paragraph, What part doesn’t fit,
What seems to be, What exceptions are there,
Which are facts, Which are opinions, Translate,
Outline, Explain what is meant, This represents
Application (using knowledge in Apply, calculate, compute, demonstrate, do ,
situations that are new,
estimate, find, illustrate, manipulate, relate,
unfamiliar, or have a new slant)) simulate, solve, use, utilize
What would result, Chose the best statements that
apply, Estimate a solution, Apply a formula to,
Select the best solution, Use new information to
determine,
Analysis (taking it apart)
Analyze, categorize, classify, chart, code,
compare, contrast, diagram, derive, determine,
differentiate, dissect, draw conclusions, examine,
experiment, investigate, make inferences,
organize, question, separate, sequence, sort,
survey, test
What is the function, What is the main idea or
underlying theme, What statement is irrelevant or
extraneous to, What does the author believe or
assume, What ideas justify the conclusion, What is
the premise, What persuasive technique, What is
the relationship between
Synthesis (creating, combining
elements into a pattern not
clearly apparent before))
Arrange, assemble, change, combine, construct,
design, develop, formulate, generalize, integrate,
modify, plan, predict, produce, represent, set up,
write
How would you test, Propose an alternative,
Develop a plan, Design a model, Compose a song
or play, Formulate a theory or hypothesis
Evaluation (judging, evaluating
according to some set criteria)
Appraise, argue, assess, choose, conclude,
What fallacies, consistencies or inconsistencies
critique, deduce, evaluate, grade, justify, prioritize, appear, Find the errors in, Which is more
rate, rank, recommend, select, value
important, more logical, more appropriate,
It’s NOT about the verb...
The Depth of Knowledge is NOT
determined by the verb (Bloom’s
Taxonomy), but by the context in
which the verb is used and the
depth of thinking required.
Verbs are not always used
appropriately...
Words like explain or analyze have to be
considered in context.
• “Explain to me where you live” does not raise
the DOK of a simple rote response.
• Even if the student has to use addresses or
landmarks, the student is doing nothing more
than recalling and reciting.
DOK is about what follows the verb...
What comes after the verb is more
important than the verb itself.
“Analyze this sentence to decide if the commas have
been used correctly” does not meet the criteria for
high cognitive processing.
The student who has been taught the rule for using
commas is merely using the rule.
Same Verb—Three Different DOK Levels
DOK 1- Describe three characteristics of metamorphic
rocks. (Requires simple recall)
DOK 2- Describe the difference between metamorphic and
igneous rocks. (Requires cognitive processing to
determine the differences in the two rock types)
DOK 3- Describe a model that you might use to represent
the relationships that exist within the rock cycle. (Requires
deep understanding of rock cycle and a determination of
how best to represent it)
DOK is about intended outcome,
not difficulty
•
•
•
•
DOK is a reference to the complexity of mental
processing that must occur to answer a question,
perform a task, or generate a product.
Adding is a mental process.
Knowing the rule for adding is the intended
outcome that influences the DOK.
Once someone learns the “rule” of how to add, 4 +
4=8 is DOK 1 and is also easy.
Adding 4,678,895 + 9,578,885 is still a DOK 1 but
may be more “difficult.”
DOK is not about difficulty...
• Difficulty is a reference to how many students answer a
question correctly.
“How many of you know the definition of exaggerate?”
DOK 1 – recall
If all of you know the definition, this question is an easy
question.
“How many of you know the definition of prescient?”
DOK 1 – recall
If most of you do not know the definition, this question is a
difficult question.
DOK is about complexity
• The intended student learning outcome
determines the DOK level.
• Instruction and classroom assessments,
questioning and performance indicators
must reflect the DOK level of the objective
or intended learning outcome.
Quick Quiz
1) Give an example of a statement that
uses a verb that “sounds” like a high
DOK but is used inappropriately.
2) Fill in the blanks: What _____ the verb
is more _____ than the verb itself when
deciding the DOK level.
3) What is the difference between
difficulty and complexity?
4) What really determines the DOK level?
Quick Quiz Answers
1) Give an example of a statement that uses a verb
that “sounds” like a high DOK but is used
inappropriately. answers vary
2) Fill in the blanks: What follows the verb is more
important than the verb itself when deciding the
DOK level.
3) What is the difference between difficulty and
complexity? answers vary, but do not rely on the
verb
4) What really determines the DOK level? the
intended learning outcomes
So, let’s review- What is Depth
of Knowledge (DOK)?
• A scale of cognitive demand (thinking) to align
standards with assessments and performance
indicators
• Based on the research of Norman Webb,
University of Wisconsin Center for Education
Research and the National Institute for
Science Education
• Guides item development for many state
assessments
Webb’s Four Levels of
Cognitive Complexity
• Level 1: Recall and Reproduction
• Level 2: Skills & Concepts
• Level 3: Strategic Thinking
• Level 4: Extended Thinking
"To be, or
not to be:
that is the
question"
DOK Level 1:
Recall and Reproduction
•
•
•
Requires recall of information, such as a fact,
definition, term, or performance of a simple process
or procedure
Answering a Level 1 item can involve following a
simple, well-known procedure or formula
This also means following simple steps, recipes, or
directions. Can be difficult without requiring
reasoning. At DOK 1, students find “the right
answer,” and there is no debating the “correctness,”
it is either right or wrong.
Recall and Reproduction DOK Level
1 Examples:
• List animals that survive by eating other
animals
• Locate or recall facts found in text
• Describe physical features of places
• Determine the perimeter or area of rectangles
given a drawing or labels
• Identify elements of music using music
terminology
• Identify basic rules for participating in simple
games and activities
Examples of DOK 1 in
Music
Items
1.
2.
3.
Name the notes of the C
Major scale
Name 4 periods of classical
music.
Know that a sharp raises a
note ½ step
Why is this DOK 1?
1.
2.
3.
Simple recall of pre-learned
knowledge
Simple recall, but must be
taught
Identify a #, recognize that it
raises a pitch
Skills/Concepts: DOK Level 2
• Includes the engagement of some mental processing
beyond recalling or reproducing a response
• Items require students to make some decisions as to how to
approach the question or problem
• Actions imply more than one mental or cognitive
process/step
•Requires comparison of two or more concepts, finding
similarities and differences, applying factual learning at the
basic skill level. Main ideas – requires deeper knowledge
than just the definition. Students must explain “how” or
“why” and often estimate or interpret to respond.
Skills/Concepts: DOK 2 Examples
• Compare desert and tropical environments
• Identify and summarize the major events,
problems, solutions, conflicts in literary text
• Explain the cause-effect of historical events
• Predict a logical outcome based on information in
a reading selection
• Explain how good work habits are important at
home, school, and on the job
• Classify plane and three dimensional figures
• Describe various styles of music
Examples of DOK 2 in
Music
Item
1.
2.
Read and perform a simple
rhythm
Play a simple melody or
accompaniment
Why is this DOK 2?
1.
2.
If the student interprets the
rhythm (as opposed to
repeating) it is DOK 2.
Student must make sense
out of written notation and
perform
Strategic Thinking: Level 3
• Requires deep understanding exhibited through
planning, using evidence, and more demanding cognitive
reasoning
• The cognitive demands are complex and abstract
• An assessment item that has more than one possible
answer and requires students to justify the response
would most likely be a Level 3
•Students must reason or plan to find an acceptable
solution to a problem. More than one correct response
or approach is possible. Requires complex or abstract
thinking, and application of knowledge or skill in a new
and unique situation.
DOK Level 3 Examples
Develop a scientific model for a complex idea
• Propose and evaluate solutions for an
economic problem
• Explain, generalize or connect ideas, using
supporting evidence from a text or source
• Create a dance that represents the
characteristics of a culture
•
DOK Level 3: Strategic Thinking
Examples:
• Compare consumer actions and analyze how these
actions impact the environment
• Analyze or evaluate the effectiveness of literary
elements (e.g., characterization, setting, point of view,
conflict and resolution, plot structures)
• Solve a multiple-step problem and provide support
with a mathematical explanation that justifies the
answer
Examples of DOK 3 in
Music
Item
1.
2.
3.
Why is this DOK 3?
Improvise a simple melody
1.
Perform as a member of a
conducted ensemble
2.
Compose a single line
melody
3.
New application of complex
processes
Students make individual
choices about performance
New application of complex
processes
Extended Thinking: Level 4
• Requires high cognitive demand and is very complex
• Students are expected to make connections, relate ideas
within the content or among content areas, and select or
devise one approach among many alternatives on how the
situation can be solved
• Due to the complexity of cognitive demand, DOK 4 often
requires an extended period of time
•At this level, students typically identify a problem, plan a
course of action, enact that plan, and make decisions
based on collected data. Usually involves more time than
one class period. Multiple solutions are possible. Students
often connect multiple content areas to come up with unique
and creative solutions.
Extended Thinking: DOK 4 Examples
• Gather, analyze, organize, and interpret
information from multiple (print and non print)
sources to draft a reasoned report
• Analyzing author’s craft (e.g., style, bias, literary
techniques, point of view)
• Create an exercise plan applying the “FITT
(Frequency, Intensity, Time, Type) Principle”
Examples of DOK 4 in
Music
Item
1.
2.
3.
Compose using 2 or more
parts
Why is this DOK 4?
1.
Improvise over a given chord
progression
2.
Perform in a student-led
ensemble or solo with
accompaniment
3.
Requires application of
harmony, voice leading,
cadence
Requires student to apply all
previous learning in a new
and novel situation
Student makes all choices
Ask yourselves…
• How important are the questions you ask in
the classroom and on assessments?
• What is the benefit of raising the level of
awareness about the levels of different
questions?
Key Understandings and
Guiding Questions:
• Some organisms are capable of converting light
energy into chemical energy (glucose) through
photosynthesis.
•
•
What is the process of photosynthesis?
What organisms carry out photosynthesis?
• All organisms extract energy from glucose to
produce ATP.
•
•
What is the process of cellular respiration?
What organisms carry out cellular respiration?
Engage
•
Ask the students to think about and answer the following questions.
•
•
•
•
•
What did you eat this morning?
•
Answers will vary.
Why do you eat food? In other words, what is food?
•
(We eat food to extract nutrients for energy.)
How does your food get its food?
•
(If it is an animal (or animal product), it gets food from plants. If it
is a plant, it makes food from sunlight.)
What is the process called in which organisms make their own
food?
•
(Photosynthesis)
What do we call organisms that make their own food?
•
(Autotrophs or producers)
Which of the following is rigorous?
Why?
Read a description of a science experiment and list the
necessary material to perform the experiment.
Locate and read a current article on biotechnology,
summarize the innovation and the benefits to humans.
Read and analyze three original newspaper articles from
WWII and indentify reasons for public opposition to US
entry into the war.
Write directions on how to connect new television to
cable and DVD player.
Increased Rigor
Through Questioning
& Classroom
ActivitiesAn Understanding of
Webb’s Depth of Knowledge & Bloom’s
Taxonomy
Level 1
Level 2
“Recall of Information”
- This level generally requires students to identify, list, or
define.
- Recall who, what, when and where.
- Identify specific information contained in maps, charts,
tables, and drawings.
“Basic Reasoning”
- Convert information from one form to another:
Contrast and Compare
Cause and Effect
Categorize into groups
Distinguish between fact and opinion
DOK Levels
for Social
Studies
Level 3
“Complex Reasoning”
- Apply a concept in other contexts.
- Draw conclusions or form alternative conclusions.
- Analyze how changes have affected people or places.
- Analyze similarities and differences in issues or problems.
Level 4
“Extended Reasoning”
- Analyze and explain multiple perspectives or issues.
- Make predictions with evidence as support.
-Plan and develop solutions to problems.
- Describe, define, and illustrate common social, historical,
economic, or geographical themes and how they relate.
Depth of Knowledge
Level 1
Level 2
Level 3
Level 4
(recall) (skill/concept) (strategic thinking) (extended thinking)
Social Studies
Benchmark
DOK assessed
DOK needed
examine the role of the
Founding Fathers
Not identified by Texas
Dept. of Ed.
DOK 1 (name FF)
DOK 2 (identify positions)
DOK 3 (what would he be
today)
DOK 4 (create a 21st
century nation for him)
describe the history of
people who first lived in
Texas
Not identified by Texas
Dept. of Ed.
DOK 1 (name)
DOK 2 (identify location of
old and new home)
DOK 3 (what affect)
DOK 4 (what if Asians had
come first instead)
Depth of Knowledge
Level 1
Level 2
Level 3
Level 4
(recall) (skill/concept) (strategic thinking) (extended thinking)
Social Studies
Grade 5 or 8 Which of the following is the main reason many leaders in Great
Britain leaned toward supporting the Confederacy during the American Civil
War?
Level 1
Grade 8 There was a sharp decline in immigration into the US during the second
decade of the 20st century. Which of the following best accounts for that
decline?
Level 2
Grade 5 or 8 A well-known newspaper prints a story critical of the current
administration’s policies. The Bill of Rights allows a government official to
respond to this headline by which of the following. Level 3
Grade 5 or 8 A great shift in population from one area to another could
potentially cause problems and would change various structures now in
place. Describe the problems and indicate the changes that might take
place. Justify your answer with data.
Level 4
Let’s think…Sample Question
Social Studies
A newspaper prints a story that criticizes the current administration’s
Policies. The Bill of Rights allows a government official to respond to
this headline by
a) arresting the publisher of the newspaper
b) closing down the newspaper
c) demanding that the newspaper print a new headline
d) writing a letter of protest to the editor
This item is level 3 because it requires students to apply the
concepts of the Bill of Rights to a given situation represented
by the newspaper headline to determine the correct answer.
Level 1
Questions:
•
•
What was
George
Washington's
role in the
American
Revolution?
What factor was
most important
in growing
cotton in the
1800s?
DOK Social Studies Levels Examples
Level 2
•
The author of this article
most likely believed that
U.S. involvement in the
Spanish-American War
was necessary in order
to -
•
The provisions in this
excerpt were intended to
address which of the
following grievances in
the DOI?
•
HANDOUTS/
DITTOS
Memorization
using flash
cards
Questions:
•
•
Activities:
Have students compare
and contrast the causes
for the U.S.’s entrance
into WWI and WWII.
Explain the
similarities/differences
for genocide to occur
in Germany, Rwanda,
and Darfur.
If the world continues to
use fossil fuels at the
same rate what will be
the outcome on the
environment. Support
your answer with
evidence.
How did incidents like
the Gulf of Tonkin
Resolution and the
secret bombing of
Cambodia eventually
affect the relationship
between the legislative
and executive
branches of
government?
Activities:
•
•
Level 4
Questions:
•
Activities:
•
Level 3
Questions:
•
Have students write a
research paper to
explain, generalize,
and connect ideas.
Have students give
alternate solutions to
past outcomes in
history.
How does this pattern differ
from the early 20th
century? What are the
reasons for each?
Activities:
•
Have students plan and
develop a solution to the
war in Iraq.
Level 2
Level 1
Includes the recall of information such as a fact, definition, term
simple procedure, as well as performing a simple algorithm or
applying a formula. In mathematics a one-step, well-defined, and
straight algorithmic procedure is included at this lowest level.
Other key words that signify a Level 1 include “identify,” “recall,”
“recognize,” “use,” and “measure.” Verbs such as “describe” and
“explain” could be classified at different levels depending on what
is to be described and explained.
Keywords that generally distinguish a Level 2 item include
“classify,” “organize,” ”estimate,” “make observations,” “collect
and display data,” and “compare data.” These actions imply more
than one step. For example, to compare data requires first
identifying characteristics of the objects or phenomenon and then
grouping or ordering the objects. Some action verbs, such as
“explain,” “describe,” or “interpret” could be classified at different
levels depending on the object of the action.
DOK Levels
for
Mathematics
Level 3
Requires reasoning, planning, using evidence, and a higher level of
thinking than the previous two levels. In most instances, requiring students
to explain their thinking is a Level 3. Activities that require students to make
conjectures are also at this level. The cognitive demands at Level 3 are
complex and abstract. An activity, however, that has more than one possible
answer and requires students to justify the response they give would most
likely be a Level 3. Other Level 3 activities include drawing conclusions from
observations; citing evidence and developing a logical argument for
concepts; explaining phenomena in terms of concepts; and using concepts
to solve problems.
Level 4
At Level 4, the cognitive demands of the task should be high and the
work should be very complex. Students should be required to make
several connections—relate ideas within the content area or among
content areas—and have to select one approach among many
alternatives on how the situation should be solved, in order to be at
this highest level. Level 4 activities include designing and conducting
experiments; making connections between a finding and related
concepts and phenomena; combining and synthesizing ideas into
new concepts; and critiquing experimental designs.
Let’s think…Sample Question
Math
Each day that Jasmine turns in her homework on time, she earns 5points.
Jasmine has turned in her homework on time for the last 8 days. How
many points has Jasmine earned altogether?
a)30
b)35
c)40
d)45
The fact that this is a story problem does not make this more than a
level 1 item. The text here quickly reveals that the problem is
simple multiplication. However, story problems can often have
higher DOK levels even if the computations required are only
level 1, as long as there is some skillful or strategic thinking
required in determining what computations to perform.
Aligning DOK levels of standards and assessments
Standards ratings may serve as a “ceiling” for
assessment
•In general, the ratings of the benchmarks set a ceiling for
assessment. Thus, a benchmark that is rated at a
moderate level, could be assessed at a low level or a
moderate level.
•Ideally, the benchmark rating aligns with the assessment
level – this is the purpose for rating benchmarks and any
other questions (formative assessments, summative
assessments, journal prompts, class discussions).
•This helps teachers know to what depth students are
expected to master the benchmarks.
•It helps to build a common understanding of the
expectations of the benchmark.
Depth of Knowledge
Level 1
Level 2
Level 3
Level 4
(recall) (skill/concept) (strategic thinking) (extended thinking)
Mathematics
Benchmark
DOK assessed
DOK needed
describe a rule for sorting
objects- DOK 2
Not identified by Texas
Dept. of Ed.
DOK 1 (sort)
DOK 2 (identify/describe rule)
compose and decompose
three-digit numbers with
representations in words and
physical models- DOK 2
Not identified by Texas
Dept. of Ed.
DOK 1 (identify #)
DOK 2 (determine component
parts)
predict trends based on
graphical representation- DOK
3
Not identified by Texas
Dept. of Ed.
DOK 1 (determine how many)
DOK 2 (compare)
DOK 3 (make decisions)
simplify and evaluate numerical
and algebraic expressionsDOK 1
Not identified by Texas
Dept. of Ed.
DOK 1 (solve)
Level 1
Depth of Knowledge
Level 2
Level 3
Level 4
(recall) (skill/concept) (strategic thinking) (extended thinking)
Math
Which of the following numbers, when rounded to the nearest
thousand becomes 90,000?
Level 1
A car has traveled 23, 456.2 miles. The next exit is 1000 feet
ahead. What will the mileage gauge read then?
Level 2
Mr. Baxter wants to fence in an area for his dog. He can afford 36
feet of fencing. He is considering several different shapes for
the area but wants the shape to have four sides that are whole
number lengths and contain four right angles. What is the
largest area that Mr. Baxter can enclose with 36 feet of fencing?
Level 3
Identify a real world problem that requires the application of
mathematics, describe the possible procedure(s) for solving this
problem, and explain the outcome and your reasoning. Level 4
Determining DOK: Science Example
Sample Science Assessment Limit
DOK
Ceiling
Level
Example A: Perform a simple science
process or a set procedure to gather data
1
1
(Measure temperature of
water)
Example B: Represent data collected over a
period time, making comparisons and
interpretations
2
1
(Measure temperature of water at
different times/places)
2
(Construct a graph to organize,
display, and compare data)
Example C: Interpret data collected for a
research question for a scientific problem
related to your environment
3
1
(Measure temperature of water at
different times/places)
2
(Construct a graph to organize,
display, and compare data)
3
(Design an investigation to explain
the affect of varying temperatures of
the river in different locations)
Potential DOK Levels
for Assessment
Let’s think…Sample Question
Science
A scientist synthesizes a new drug. She wants to test its effectiveness in
stopping the growth of cancerous tumors. She decides to conduct a
series of experiments on laboratory mice to test her hypothesis. What
should she do?
a) Give half the mice the drug, the other half none, and compare their
tumor rates.
b) Give the drug to all mice, but only to half every other day, and record
tumor rates.
c) Double the dosage to all mice each day until tumors start to
disappear.
d) Give the drug only to those mice who have tumors and record their
weights.
This item is level 2. Students must at least apply knowledge of
controlled experiment design to this situation, or derive it from this
problem.
Depth of Knowledge
Level 1
Level 2
Level 3
Level 4
(recall) (skill/concept) (strategic thinking) (extended thinking)
Science
Benchmark
DOK assessed
DOK needed
distinguish between
radiating objects (the sun
and the stars) and reflecting
objects (the planets and
their moons).
Not identified by
Texas Dept. of
Ed.
DOK 1 (identify)
DOK 2 (sort)
DOK 3 (compare to)
DOK 4 (what could take away
radiating/reflective properties)
relate how internal factors
affect the Earth’s surface
including earthquakes and
volcanoes
Not identified by
Texas Dept. of
Ed.
DOK 1 (identify factors)
DOK 2 (what do factors do)
DOK 3 (what happens)
DOK 4 (what if)
Level 1
Depth of Knowledge
Level 2
Level 3
Level 4
(recall) (skill/concept) (strategic thinking) (extended thinking)
Science
What do we call the process plants use to manufacture sugar
from sunlight?
Level 1
Which graph of a heart rate represents that a person walked for 2
minutes, ran for 5 minutes, worked for 2 more minutes, and
ran for the last 2 minutes?
Level 2
In a lab experiment, an enzyme is combined with its substrate at
time zero. The absorbance of the resulting solution is
measured at five-minute intervals. In this procedure, an
increase in absorbance is related to the amount of product
formed during the reaction. The experiment uses three
preparations. The most likely reason for the failure of the
absorbance to increase significantly after 1- minutes in
preparation 3 is which of the following?
Level 3
Depth of Knowledge
Level 1
Level 2
Level 3
Level 4
(recall) (skill/concept) (strategic thinking) (extended thinking)
Language Arts
Benchmark
DOK assessed
DOK needed
use word recognition skillsDOK 2
Not identified by Texas
Dept. of Ed.
DOK 1 (recognize sound)
DOK 2 (read word)
analyze texts in order to
identify, understand, infer, or
synthesize information- DOK 3
Not identified by Texas
Dept. of Ed.
DOK 1 (read)
DOK 2 (understand)
DOK 3 (apply information)
apply knowledge of sentence
structure in composing or
editing- DOK 2
Not identified by Texas
Dept. of Ed.
DOK 1 (know parts)
DOK 2 (write sentence/edit
sentence)
use context clues to
determine the figurative
meanings- DOK 2
Not identified by Texas
Dept. of Ed.
DOK 1 (read)
DOK 2 (apply)
Depth of Knowledge
Level 1
Level 2
Level 3
Level 4
(recall) (skill/concept) (strategic thinking) (extended thinking)
Language Arts
Which word means about the same thing as exaggerate?
Level 1
Which of these statements best describes what the passage
you read is about?
Level 2
The style in this passage is characterized by use of similes
such as “ smooth as pudding, “crisp as an autumn apple,”
and “ rough as grated coconut.” Which of the following best
explains the author’s purpose in using these similes?
Level 3
You are to write an informative essay for your teacher using a
minimum of five resources including one source from
technology and one personal interview.
Level 4
Practice Time- Activity #3
Take a white envelope. You have 5-10 minutes to
complete the task:
If you have an assessment
question, determine the
DOK level of each
question.
•
Determine the steps or
thought process a student
must go through to get to the
correct response (You may
look at the SE if necessary)
•
•
Be honest, and think about
your students.
What are the instructional
implications for students? For
the teacher?
If you have a student
expectation, determine the
Bloom’s level of each question.
•
•
•
Develop a question that could
be used to evaluate this SE.
Determine the steps or
thought process a student
must go through to get to the
correct response to your
question.
Does the level of the
activity/steps/performance
match the level of the
assessment question?
• If at most 10-30% of the questions on
STAAR/EOC are at a Low to Moderate Level of
Complexity...How much class time should we
devote to DOK Level 1 thinking?
• If at least 30-80% of the questions on
STAAR/EOC (and in life) require Moderate to High
levels of Complexity....What are we doing to
promote these complex levels of higher order
thinking?
DOK Activity #4
1. In the medium envelope, take
the questions and try to level them.
2. Now level by level, determine if
you think the question came from a
STAAR released item, TAKS
release test, or CSCOPE unit test.
Key Points
• DOK 1 + DOK 1 + DOK 1 = 1
• Depths of knowledge classification is based
on the task, not the student
• DOK is different from task/item difficulty
• DOK ratings aid in alignment of standards
and assessment, and therefore instruction
•DOK is about complexity
•We don’t do enough level 3 and 4
Remember DOK is...
…descriptive
…focuses on how deeply a
student has to know the
content in order to respond
…NOT the same as difficulty.
…NOT the same as Bloom’s
Taxonomy
The Heart of the
Matter is the Depth of
Knowledge
Looking to the Future
1.How might teachers adjust instruction to increase
rigor? (Not go outside of your lane but depth within
it)
2.What is the greatest challenge you face in
addressing the increase in rigor and how might you
tackle it?
3.What are the benefits of increasing rigor in
instruction and assessment?
4.How can you ensure alignment between the
standards, your instructional practices and
resources and your formal/informal and
formative/summative questioning/assessment?
• Continue Doing
• Continue
w/Adjustments
Rigor
• Start Doing
• Stop Doing
Dr. Stephanie Zelenak,
CSCOPE Director- ESC Region 19
szelenak@esc19.net
915-780-5029