DOK and HOTS
Presented By:
Cisely Scott
Julian Davenport
Common Board Configuration (CBC)
DATE: June 23, 2011
BENCHMARK: DOK, HOTS, and
Questioning Strategies
OBJECTIVE:
Today we will examine the role
Higher Order Thinking, Depth of
Knowledge, and Questioning
Strategies on student
achievement.
ESSENTIAL QUESTION:
How does increasing the rigor in
your questioning enhance the
performance of your students in
the classroom? What are the
long term implications of using
these strategies?
VOCABULARY:
Depth of Knowledge, Cognitive
Complexity (low, medium, high),
Difficulty vs. Complexity, HOT
Verbs, Questioning Strategies,
Inquiry Based Learning,
AGENDA:
•What the Research Says….
•What is Depth of Knowledge?
•Application to FCAT
•Elements of Inquiry
•Activity-Developing a HOT
Question
•How Do We Ensure Use in the
Classroom?
•Journal Reflection: How does
the use of HOTs questions,
DOK, and Questioning
Strategies affect teacher
performance and student
achievement?
EXIT SLIP:
IN YOUR JOURNAL
Connect: How does the
use of HOTs questions,
DOK, and Questioning
Strategies affect teacher
performance and student
achievement?
Connect: 1 question you
still have about FCAT 2.0.
HOME LEARNING:
Review today’s lesson and
develop your next steps
to share this information
with teachers at your
school
ESSENTIAL QUESTION (EQ)
How does increasing the rigor in
teacher questioning enhance the
performance of your students in the
classroom? What are the long term
implications of using these
strategies?
REACHING YOUR VISION
This presentation will help you to
reach your vision as a coach by
providing you with an overview
DOK, HOT, and Questioning
Strategies so that you’re able to
monitor for and model these
strategies within the classroom.
What the Research
Says……
Teachers are the Key
“Teachers must be the primary
driving force behind change. They
are best positioned to understand
the problems that students face
and to generate possible
solutions.”
Quality Instruction Makes A Difference
“Good teaching can make a
significant difference in student
achievement, equal to one effect
size (a standard deviation), which
is also equivalent to the affect
that demographic classifications
can have on achievement.”
Paraphrase Dr. Heather Hill, University of Michigan
Differences in Instruction
“Our research indicates that there is
a 15% variability difference in
student achievement between
teachers within the same schools.”
Deborah Loewenberg Ball, Dean of Education, University of Michigan
“What Matters Very Much is
Which Classroom?”
“If a student is in one of the most
effective classrooms he or she will learn
in 6 months what those in an average
classroom will take a year to learn. And if
a student is in one of the least effective
classrooms in that school, the same
amount of learning take 2 years.”
Research has indicated
that... “teacher quality
trumps virtually all other
influences on student
achievement.”
(e.g., Darling-Hammond, 1999; Hamre and Pianta,
2005; Hanushek, Kain, O'Brien and Rivken, 2005;
Wright, Horn and Sanders, 1997)
So what are effective teachers
doing differently? What should you
be looking for as a coach?
Discuss with your neighbor the answer
to this question. (5-10 mins). Come
up with a list of 10 things to share.
Rigorous Classroom
Standards-based teaching
Look for Lots of high level activity- - High Levels of Questioning
 Reflecting
 Analyzing
 Doing experiments (science)
 Doing exploratory activities
 Discussing/Debates
 Writing
 Working in groups
And a Scoring Guide available to all students for all
major assignments

Sets expectations

12
Rigorous Classroom
Only by creating a culture of high
expectations and providing support so
students can truly succeed do you have a
rigorous classroom.
Barbara Blackburn 2008
13
What is Depth Of Knowledge?
Cognitive Demand
Why Depth of Knowledge?
Why Use a Depth of Knowledge?
• Used
to determine the level of the
expected outcomes of the Next
Generation Sunshine State
Standards and benchmarks
• Determines the complexity of FCAT
items (success with items leads to
AYP)
Verbs are not always used appropriately...
Words like explain or analyze have to be
considered in context.
• “Explain to me where you live” does not raise
the DOK of a simple rote response.
• Even if the student has to use addresses or
landmarks, the student is doing nothing more
than recalling and reciting.
DOK is about what follows the verb...
What comes after the verb is more
important than the verb itself.
“Analyze this sentence to decide if the commas have
been used correctly” does not meet the criteria for
high cognitive processing.”
The student who has been taught the rule for using
commas is merely using the rule.
Same Verb—Three Different DOK Levels
DOK 1- Describe three characteristics of metamorphic rocks.
(Requires simple recall)
DOK 2- Describe the difference between metamorphic and
igneous rocks. (Requires cognitive processing to determine
the differences in the two rock types)
DOK 3- Describe a model that you might use to represent the
relationships that exist within the rock cycle. (Requires
deep understanding of rock cycle and a determination of
how best to represent it)
Activity

In your groups, use the word
explain, analyze, or any of the other
words on the ladder to come up with
a low complexity question and a
higher complexity question using the
verb. (5 minutes)
DOK is about intended outcome,
not difficulty
DOK is a reference to the complexity of mental
processing that must occur to answer a question,
perform a task, or generate a product.
• Adding is a mental process.
• Knowing the rule for adding is the intended outcome
that influences the DOK.
• Once someone learns the “rule” of how to add, 4 +
4 is DOK 1 and is also easy.
• Adding 4,678,895 + 9,578,885 is still a DOK 1 but
may be more “difficult.”
DOK is not about difficulty...
• Difficulty is a reference to how many students answer a question
correctly.
“How many of you know the definition of exaggerate?”
DOK 1 – recall
If all of you know the definition, this question is an easy question.
“How many of you know the definition of vociferous?”
DOK 1 – recall
If most of you do not know the definition, this question is a difficult
question.
DOK is about complexity
• The intended student learning outcome
determines the DOK level.
• Every objective in the science and
mathematics frameworks has been assigned a
DOK level.
• Instruction and classroom assessments must
reflect the DOK level of the objective or
intended learning outcome.
Quick Quiz
1) Give an example of a statement that
uses a verb that “sounds” like a high
DOK but is used inappropriately.
2) Fill in the blanks: What _____ the verb
is more _____ than the verb itself when
deciding the DOK level.
3) What is the difference between difficulty
and complexity?
4) What really determines the DOK level?
Copy this down in your notebook?
Quick Quiz Answers
1) Give an example of a statement that uses a verb
that “sounds” like a high DOK but is used
inappropriately. answers vary
2) Fill in the blanks: What follows the verb is more
important than the verb itself when deciding the
DOK level.
3) What is the difference between difficulty and
complexity? answers vary, but do not rely on the
verb
4) What really determines the DOK level? the intended
learning outcomes
How does this apply to FCAT?
Correlation to FCAT
Percentage of Points by Cognitive Complexity
Level for FCAT SSS Reading
FCAT Science 2.0
Here is the new breakdown, based on Webb’s DOK:
Note: A high complexity question in 5th grade may not be a
high complexity question in 8th grade. The complexity is
ultimately based on the cognitive demand placed on the
student. Cognitive complexity also may depend on the level of
the distractor.
Cognition Classification
Schemes
Classification schemes for
cognition allow communication
about the nature and
characteristics of thinking and
learning.
Examples: Bloom’s Taxonomy and
Webb’s Depth of Knowledge
What is Webb’s Depth of Knowledge?
Webb’s Levels of Cognitive Complexity



Low Complexity relies on the recall and
recognition of previously learned concepts and
principles
Moderate Complexity involves more flexible
thinking and choice among alternatives
High Complexity requires students to engage in
more abstract reasoning, planning,
analysis, judgment, and
creative thinking
Complexity v. Difficulty
The degree of challenge of FCAT items is
currently
categorized in two ways:
 Cognitive Complexity



Determined by what the item requires the student to recall,
understand, analyze, and do.
Assume student is familiar with basic concepts of the task
and focuses on the task.
Depends on the task not the student.
• Item Difficulty

Depends on the percentage of students likely to answer
correctly.
• Easy – More than 70%
• Average – 40% to 70%
• Challenging – Less than 40%
DOK Level 1
Low Complexity




Low complexity items rely heavily on the recall and recognition of
previously learned concepts and principles.
Items typically specify what the student is to do, which is often to
carry out some procedure that can be performed mechanically.
It is NOT left to the student to come up with an original method or
solution.
Skills required to respond to low complexity items include:
•
•
•
•
•
•
•
•
•
•
•
solving a one-step problem;
computing a sum, difference, product, or quotient;
evaluating a variable expression, given specific values for the variables;
recognizing or constructing an equivalent representation;
recalling or recognizing a fact, term, or property;
retrieving information from a graph, table, or figure;
identifying appropriate units or tools for common measurements;
performing a single-unit conversion;
reproducing a diagram or standard representation;
completing a routine procedure, such as measure temperature; and
calculating using a common formula.
DOK Level 2
Moderate Complexity




FCAT Mathematics moderate complexity items involve more flexible and
choice among alternatives.
Items require a response that goes beyond the habitual, is not
explicitly specified in the text, and ordinarily has more than a single
step.
The student is expected to decide what to do using informal methods of
reasoning and problem solving strategies and to bring together skill
and knowledge from various domains.
Skills required to respond to moderate complexity items include:
•
•
•
•
•
•
•
•
•
•
•
•
•
•
solving a problem requiring multiple operations;
solving a problem involving spatial visualization and/or reasoning;
selecting and/or using different representations, depending on situation and purpose;
retrieving information from a graph, table, or figure and using it to solve a problem;
determining a reasonable estimate;
extending an algebraic or geometric pattern;
providing a justification for steps in a solution process;
comparing figures or statements;
representing a situation in more than one way;
formulating a routine problem, given data and conditions;
explaining relationships between facts, terms, or properties;
describing and explain examples and non-examples of science concepts;
making decisions and comparisons;
selecting a procedure according to criteria and perform procedure;
DOK Level 3
High Complexity




High complexity items make heavy demands on student thinking.
Students must engage in abstract reasoning, planning, analysis,
judgment, and creative thought.
Items require that the student think in a more abstract and
sophisticated way.
Skills required to respond correctly to high complexity items
include:
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
performing a procedure having multiple steps and multiple decision points;
solving a non-routine problem (as determined by grade-level appropriateness);
solving a problem in more than one way;
describing how different representations can be used for different purposes;
generalizing an algebraic or geometric pattern;
explaining and justifying a solution to a problem;
describing, comparing, and contrasting solution methods;
providing a mathematical justification;
analyzing similarities and differences between procedures and concepts;
formulating an original problem, given a situation;
formulating a mathematical model for a complex situation;
analyzing or producing a deductive argument;
identifying research questions and design experiments
developing a scientific model for a complex situation; and
forming conclusions and generalizations from experimental data.
How can I REALLY understand the
“depth” of the benchmarks?



Resources to help teachers gain a better
understanding of the benchmarks are
available.
Professional development, PLCs, Lesson
Study, common team planning, and
instructional coaching will support
teachers as they transition to the NGSSS.
What role should my textbook play as a
resource?
Elements of Inquiry:
Checks for Understanding and
Questioning Strategies
Top 10 Reasons to Ask
Questions
1. Increases student retention of knowledge, skills, and
understandings.
2. Fosters an active learning environment to maintain student
engagement.
3. Stimulates students’ interest and curiosity in content.
4. Encourages both individual and group accountability.
5. Assesses what students already know.
6. Diagnoses difficulties or road blocks to learning.
7. Helps students monitor their own understandings.
8. Supports students in making conjectures and arguing validity.
9. Allows students to hear different explanations by their peers.
10.Develops student communication skills and social development.
Quality Questioning




Quality questioning frames, delivers, times,
solicits, and responds to questions in a way that
reveals as much information as possible about
what students know or do not know.
It’s not, necessarily, about asking more questions.
In fact, research shows that teachers ask a lot of
questions in the course of a day…approximately
one every 72 seconds, on average.
However, almost half of these questions are
actually answered by the teacher, not the
students.
Ask Questions Better









Pose a question to the whole class, then call a student by name to
respond, “Question First, Name Last.”
Make and maintain eye contact with students.
Consider a “no hands up” approach.
Use a random technique for selecting students to avoid always
calling on the same students.
Limit if not eliminate unison response questions.
Provide students with “wait time” to think before responding.
Redirect student questions to other students to answer whenever
possible.
Ask all students regardless of gender, ethnicity, and/or
socioeconomic status equally challenging questions.
Vary pace of questioning as necessary to ensure lesson
momentum.
Activity
1.
2.
3.
4.
5.
Each group will select a benchmark from
either the Math or Science Test Item
Specifications.
Groups will develop a Higher Order
Thinking Question.
Each group will present.
Using Think-Pair-Share, groups will
discuss whether it is low, moderate, or
high complexity and why or why not
Using the whiteboards, you will vote L,
M, or H, and be asked to explain why.
How do we ensure that teachers
utilize Higher Order Thinking in the
Classroom?
Plan higher order questions in your
common planning sessions.
 Include HOTS questions in
promethean/smartboards to make
sure that teachers address the
questions.

Journal Reflection
How does the use of HOTs
questions, DOK, and Questioning
Strategies affect teacher
performance and student
achievement?