Science Curriculum Topic Study
Collaborative
Inquiry
into Examining
Student Thinking
(CIEST)
NCOSP Learning Community Forum
February 27-28, 2007
Black and Wiliam
“Standards are raised only by changes that are
put into direct effect by teachers and students in
classrooms. There is a firm body of evidence
that formative assessment is an essential
feature of classroom work and that development
of it can raise standards. We know of no other
way of raising standards for which such a strong
prima facie case can be made on the basis of
evidence of such large learning gains.”
From Inside the Black Box, 1998
http://www.pdkintl.org/kappan/kbla9810.htm
NCOSP Learning Community Forum
February 27-28, 2007
Three Goals
• (Learn) To learn about a professional
development strategy for collaboratively
examining student thinking.
• (Practice) To practice using the CIEST
protocol with a sample of student work.
• (Apply) To consider how you might use the
CIEST strategy to examine student
thinking in your context.
NCOSP Learning Community Forum
February 27-28, 2007
Balanced Assessment
Formative
Assessment
Summative
Assessment
CIEST- Examining students’ thinking for
the purpose of informing instruction
NCOSP Learning Community Forum
February 27-28, 2007
Collaborative Inquiry
…is a process by which all relevant groups
construct their understanding of important
problems and potential solutions through
asking questions, carefully analyzing all
relevant data, and engaging in
constructive dialogue with colleagues.
Wagner, 1998
NCOSP Learning Community Forum
February 27-28, 2007
Data: The Heart of Inquiry
“ Data enable us to be educational
detectives. We are ‘Columbos.’ We get
clues as to how students are doing. We
look at how to improve.”
Joe O’Reilly, Mesa Unified School District
NCOSP Learning Community Forum
February 27-28, 2007
CIEST PROTOCOL
ASSESSMENT
Data Driven
Dialogue
PROBES
CIEST
CURRICULUM
TOPIC STUDY
NCOSP Learning Community Forum
February 27-28, 2007
CIEST Tools and Resources
• Curriculum Topic Study Guide (CTS)
• National and State Standards and
Research on Student Learning
• Two-tiered Diagnostic Probes (ready
made probes or teacher-developed
probes)
• Student Work
NCOSP Learning Community Forum
February 27-28, 2007
CIEST DATA-DRIVEN
DIALOGUE- 3 Phases
Phase 1
Activating & Engaging
Surfacing assumptions, making
predictions, asking questions
Phase 2
Phase 3
Exploring &
Discovering
Organizing &
Integrating
Organizing & analyzing
the data
Generating inferences,
hypotheses, explanations
Adapted from Nancy Love’s Using Data, Getting Results and The Adaptive School,
Laura Lipton and Bruce Wellman
NCOSP Learning Community Forum
February 27-28, 2007
Establish Group Norms- Step 1
Generate “ground rules” for
conducting the CIEST protocol
* Include sticking to the protocol!
NCOSP Learning Community Forum
February 27-28, 2007
NCOSP Learning Community Forum
February 27-28, 2007
Examine and Complete the ProbeStep 2
Jot down any notes about:
 Your own thinking as you completed the probe
 Prior knowledge you accessed (e.g. formal or
informal learning, intuitive rules, topic study, etc.)
 Any difficulties you encountered or content
you are unsure of
 How and when your students might have
encountered (or will encounter) the idea(s)
targeted in this probe?
NCOSP Learning Community Forum
February 27-28, 2007
Probe Clarification Using
CTS - Step 3
What is the “best response” and scientific explanation?
What CTS Guide will you use?
What specific idea(s) from the standards are targeted
by the probe or may contribute to the scientific
knowledge used to respond to the probe ? (CTS
Section III)
What other learning goals are related to this probe?
(CTS Section III)
How do students typically encounter the idea targeted
by this probe? (CTS Section II)
NCOSP Learning Community Forum
February 27-28, 2007
Example: The Mitten ProblemProbe Clarification
The best response is “C”- Both thermometers
will have the same temperature reading.
See scientific explanation handout.
CTS Guide: Energy Transformation
NCOSP Learning Community Forum
February 27-28, 2007
Specific Ideas from the Mitten Problem (CTS
Section III)
Heat Production: Heat can be produced in many ways, such as
burning, rubbing, or mixing one substance with another (NSES K-4).
Heat is produced by mechanical and electrical machines and any
time one thing rubs up against another (BSL 3-5).
Conductors and Insulators: Heat can move from one object to
another by conduction (NSES K-4). Some materials conduct heat
better than others. Poor conductors can reduce heat loss (BSL 3-5).
Energy Transfer: When warmer things are put with cooler ones, the
warm ones lose heat and the cool ones gain it until they are all at
the same temperature (BSL 3-5). Energy is transferred in many
ways (BSL 6-8). Heat moves in predictable ways, flowing from
warmer objects to cooler ones, until both reach the same
temperature (NSES 5-8).
NCOSP Learning Community Forum
February 27-28, 2007
Other related learning goals (CTS Section III)
Energy is a property of many substances and
is associated with heat, light, electricity,
mechanical motion, sound, nuclei, and the
nature of a chemical (NSES 5-8).
Energy cannot be created or destroyed, but
only changed from one form into another (BSL
6-8).
NCOSP Learning Community Forum
February 27-28, 2007
CTS Section III Discussion
Examine the middle school learning goals
and the elementary prerequisite ideas
identified through CTS Section III.
How does the knowledge in the learning
goals contribute to middle school students’
understanding of the Mitten Problem?
NCOSP Learning Community Forum
February 27-28, 2007
How do the national standards describe
ways in which students encounter the
ideas related to the Mitten Problem?
(CTS Section II Essays)
NCOSP Learning Community Forum
February 27-28, 2007
Grades 3-5 BSL and Grades K-4
NSES Essay Excerpts
Children cannot distinguish between heat and temperature at
this age; therefore, investigating heat necessarily must focus
on changes in temperature (NSES K-4 Essay).
The one aspect of the energy story in which students of this
age can make some headway is heat. In their activities during
these years, students should be alerted to look for things and
processes that give off heat and then for those that seem not
to give off heat. Also, the time is appropriate to explore how
heat spreads from one place to another and what can be
done to contain it or shield things from it. Students' ideas of
heat have many wrinkles. Some materials may be thought to
be intrinsically warm (blankets) or cold (metals). Objects that
keep things warm-such as a sweater or mittens-may be
thought to be sources of heat. Only a continuing mix of
experiment and discussion is likely to dispel these ideas (BSL
3-5 Essay).
NCOSP Learning Community Forum
February 27-28, 2007
Benchmarks Grades 6-8 Essay
• At this level, students should be introduced to energy
primarily through energy transformations.
• At this early stage, there may be some confusion in
students' minds between energy and energy sources.
• The most primitive idea is that the energy needed for an
event must come from somewhere. That should trigger
children's interest in asking, for any situation, where the
energy comes from and (later) asking where it goes.
Where it comes from is usually much more evident than
where it goes, because some usually diffuses away as
radiation and random molecular motion.
NCOSP Learning Community Forum
February 27-28, 2007
NSES Grades 5-8 Essay
The understanding of energy in grades 5-8 will build on
the K-4 experiences with light, heat, sound, electricity,
magnetism, and the motion of objects. In 5-8, students
begin to see the connections among those phenomena
and to become familiar with the idea that energy is an
important property of substances and that most change
involves energy transfer. Students might have some of
the same views of energy as they do of force—that it is
associated with animate objects and is linked to motion.
In addition, students view energy as a fuel or something
that is stored, ready to use, and gets used up. The intent
at this level is for students to improve their
understanding of energy by experiencing many kinds of
energy transfer.
NCOSP Learning Community Forum
February 27-28, 2007
CTS Section II Discussion
How does CTS Section II help us
understand the instructional implications
related to The Mitten Problem?
NCOSP Learning Community Forum
February 27-28, 2007
CIEST Data-Driven
Dialogue Approach
Phase 1
Activating & Engaging
Surfacing assumptions, making
predictions, asking questions
Phase 2
Exploring & Discovering
Phase 3
Organizing & Integrating
Organizing & analyzing the data
Generating inferences,
hypotheses, explanations
Adapted by Nancy Love for Using Data, Getting Results from The Adaptive School,
Laura Lipton and Bruce Wellman
NCOSP Learning Community Forum
February 27-28, 2007
Data Dialogue
Phase 1: Activating and Engaging
 What assumptions do you have about how grade 8
students might respond to the Mitten Problem ? Each
group member share one individual assumption
 What group “quantitative” predictions can you make?
Each group make and share 1 quantitative prediction
about what they expect to see in the student work
(e.g. %’s of responses)
 What does your group want to find out? Each group
frame and share one question
NCOSP Learning Community Forum
February 27-28, 2007
Quick Scan of Student
Work- STEP 4
 Quickly scan through the student work and select 1
sample of student thinking that interests you.
 What was interesting or surprised you about the
student’s thinking?
 Briefly have each member of your small group
share 1 or 2 things (caution- no analysis yet! Just
the facts!)
 As each member shares, look for commonalities
NCOSP Learning Community Forum
February 27-28, 2007
CIEST Data-Driven
Dialogue Approach
Phase 1
Activating & Engaging
Surfacing assumptions, making
predictions, asking questions
Phase 2
Exploring & Discovering
Phase 3
Organizing & Integrating
Organizing & analyzing the data
Generating inferences,
hypotheses, explanations
Adapted by Nancy Love for Using Data, Getting Results from The Adaptive School,
Laura Lipton and Bruce Wellman
NCOSP Learning Community Forum
February 27-28, 2007
Sorting and Organizing
Data- STEP 5
Phase 2: Part I
 Tally Tier 1 responses
 Go Visual! Create bar graph of the Tier I
responses (A, B or C responses).
 Now, make a list of general categories of
reasoning used in the Tier 2 explanations
(student justification for selection of A, B, or C).
NCOSP Learning Community Forum
February 27-28, 2007
STEP 5
(Cont’d)
Phase 2- Part II
 Select 4-6 common categories of “rules” or
reasoning to focus on (one of the categories
should include a correct reason)
 Cross match your selected categories to the
Tier 1 responses (may have overlap in some
responses)
• Go Visual!- Post your data on large wall
graph or chart
NCOSP Learning Community Forum
February 27-28, 2007
Analyzing the Data- Step 6
 What prevalent student ideas
pop out at you from looking at
the data?
 What patterns or trends seem
to emerge from the data?
 What are some surprising or
interesting findings?
 Do correct Tier 1 responses
always have correct
reasoning?
“Just the Facts, Ma’am”, Jack Webb
NCOSP Learning Community Forum
February 27-28, 2007
Analyze CTS Research
Findings- Step 7
 Read through selected research summaries
from Section IV in CTS.
 Jot down notes or highlight sections of the
research that are related to students’ thinking
in the work you analyzed.
 Share research findings that appear to match
findings from your data.
 Were there any common ideas not described
in the research that you found in your student
work?
NCOSP Learning Community Forum
February 27-28, 2007
CTS Section IV Discussion
How did CTS Section IV, Research on
Student Learning, help you understand the
examples of 8th grade student thinking
from The Mitten Problem?
NCOSP Learning Community Forum
February 27-28, 2007
Analyzing Learning Goals Across a
K-12 Continuum- Step 8
 Examine Section V of CTS or your state
standards or curriculum materials
(Section VI)
 Are there any ideas in the “steps along
the way” that your students’ may have
missed?
 What implications does this have for
your curriculum and instruction?
NCOSP Learning Community Forum
February 27-28, 2007
CIEST Data-Driven
Dialogue Approach
Phase 1
Activating & Engaging
Surfacing assumptions, making
predictions, asking questions
Phase 2
Exploring & Discovering
Phase 3
Organizing & Integrating
Organizing & analyzing the data
Generating inferences,
hypotheses, explanations
Adapted by Nancy Love for Using Data, Getting Results from The Adaptive School,
Laura Lipton and Bruce Wellman
NCOSP Learning Community Forum
February 27-28, 2007
Organizing and Integrating- Step 9
Phase 3- Organizing and Integrating
Our Learning about Student Thinking
 What inferences, explanations, or
conclusions can we draw from the data?
 What else do we want to find out and
how could we do that?
 What hypotheses might we test?
NCOSP Learning Community Forum
February 27-28, 2007
Next Steps and
Reflection- STEP 10
 What else do we, as a group, want to learn from the
data we have collected?
 Do we need to disaggregate the data? How?
 What additional data should we collect? What other
information from CTS or other resources would be
useful to us?
 What will we do (group action steps) as a result of what
we have learned?
 What are the implications of our findings for ensuring
all our students achieve state and national standards
for science literacy?
NCOSP Learning Community Forum
February 27-28, 2007
Reflection
Individual Reflection
 Look back on your initial notes. How has your thinking
about teaching and learning related to the probe
changed or been enhanced by the CIEST experience?
 What would you do with the information you gained?
 What did you gain by interacting with your colleagues?
 What would you like to try differently in your classroom
or in your professional learning communities as a result
of this experience?
 How will you use the CIEST protocol in your work?
NCOSP Learning Community Forum
February 27-28, 2007
CIEST Is a PD Strategy that...
• Deepens teachers’ science or mathematics content
and pedagogical content knowledge
• Links standards and research to classroom context
• Engages teachers in evidence-based dialogue
about student learning and teaching practice
• Uses a structured data collection, organization, and
analysis approach
• Encourages reflection and promotes action
• Places high value on formative assessment
• Can be used with a variety of PD structures (e.g.
study groups, co-mentoring, action research)
NCOSP Learning Community Forum
February 27-28, 2007