TEACHER ANALYSIS AND USE
OF INTERIM ASSESSMENT
INFORMATION IN MATHEMATICS
A Presentation for the
Northern California Educational Leadership
Research Symposium
February 15, 2014
Kathy Pon, Ed. D.
Introduction
Use of student assessment data has
been heralded to improve instruction and
student achievement…
BUT
Teachers remain challenged when it
comes to linking data to actions that
improve student performance and
schools!
Purpose of Study
This study’s purpose was to deepen our knowledge about
• interim mathematics assessments ability to predict student
performance on high stakes tests
• how assessment results* are used to modify or improve
instruction through pedagogical and curricular conversations
of teachers
• the elements of conversations and behaviors related to
instructional decisions that inform and extend current datainformed decision-making theory.
*Analysis of this data can lead to the creation of “actionable knowledge”
that is, decisions and actions by teachers that modify and improve
instruction for students (Halverson, 2010).
District A – Contributions of Interim Assessments to CST
Math Scores By Grade Level
District B – Contributions of Interim Assessments to
CST Math Scores By Grade Level
Quantitative Findings Using a Regression
Analysis
Districts A and B respectively, both had significant and strong,
positive correlations to student performance on the California
State Tests for grades 4-6.
The results show that for grades 7-10, student performance in
both assessments had significant and moderate, positive
correlations to student performance to student performance on
the California State Tests.
These results support findings from Clune and White (2008)
suggesting that there may be a strong alignment between the
taught and tested curriculum. This alignment is evidenced in
relationship between benchmark scores and CSTs at the
elementary levels, but less so, as both districts’ students
progress into higher math courses. The assessment tools at
the secondary level appeared to be less useful both districts in
predicting student performance on CSTs, and as such, for
instructional purposes.
Qualitative Component of Study Focused on Behaviors
and Actions that Support Data Informed Decision Making
Reflect
Plan
Analyze Data
Implement
Assess
Weighting of Data Analysis Behaviors
Five Areas Impacting Analysis and Use of
Data
Assessment (Item) Selection Phase
Highly reflective conversations - Algebra teachers
discussing test items whereby students are not just
identifying or regurgitating the rules of exponents, but
instead must inductively come up with a rule after
seeing exponents expanded.
Conversations lacking depth whereby items did not
support the conceptual teaching of mathematics –
Algebra teachers examining a multiple choice question
adding negative exponents, and discuss a choral
strategy they use that the “fraction on the bottom is the
root and roots grow down, so the number on the top
raises the power”.
Analysis Phase
What contributes to higher levels of
analysis:
• Cycle of Inquiry
• Protocols
• Coaches who support and especially
model data results and implications to
instruction through probing questions
Supporting High Levels of Analysis
“The math coach from the District Office sat in on
meetings and helped me structure questions, and gave
me feedback afterwards. I was trying not to directly
point it out, but if no one else in the meeting noticed,
I’d step in with “What do you see in number 1? They
were more about leading questions. I needed to learn
to lead the discussion. Those Cycle of Inquiry
discussions were where we could talk in depth about
strategies, not just the percent of kids who did well on
questions, but about the actual teaching. That’s the
part that’s the most helpful, to get to the place where
you’re talking about strategies.”
Reflection Phase
• In almost all instances, pedagogical discussions
addressed the teaching of procedures.
• Evidence of need to identify “formulas” to ensure
students had a higher probability of solving word
problems.
• Reflection related to teachers’ capacity for
discussing pedagogy as it related to the teaching of
mathematical concepts.
Planning Phase
• High levels = Teachers placing students in an intervention
group and re-teaching in a different manner: “With algebra
last year, we saw kids weren’t doing well. We put
interventions into place and then measured if they made a
difference…”
• Basic levels = Teachers discussing how to support English
Learners who weren’t showing results using a “checking for
understanding” strategy or teaching academic vocabulary
for the “bubble kids”.
• Low levels (Predominantly used) = Teachers agreeing to
review items in a warm up activity the following day or
again before the next test.
Leadership Practices
• Evidence of “defined autonomy” in leadership (Marzano &
Waters, 2009) - the presence of central office systems of data
collection, but site discretion about how to support datainformed decision-making.
• District A’s co-occurrences of “Cycle of Inquiry” (COI) process
with “strategic district leadership” were weighted as “3s”
Frequency of Evidence of Strategic District Leadership in Excerpts
District B
District A
2.2
2.7
The Presence and Power of Distributed Leadership
Leaders who link visions of continuous improvement to data
use and who engage others to use the data to collaboratively
make decisions (Goodnow &Wayman, 2009) are
“transformational”. This was evidenced in both districts by
site and district level administrators.
• Setting expectations
• Release time
• Autonomy to make decisions and purchase
needed resources
• Culture of data use
Conclusions Related to Mathematics Instruction
from Qualitative Findings:
There are critical knowledge bases for practitioners in
mathematics:
a)
b)
c)
d)
content knowledge and understanding of items as
they represent mathematical concepts;
an understanding of analysis processes that best
facilitate deep analysis and understanding of
student thinking;
knowledge of protocols that support reflection,
especially on how to instructionally mediate
misconceptions;
Ideas about how to best to modify instruction as
needed.
Overall, What Does the Evidence Suggest about
Assessment Practices in Mathematics Instruction?
• Quantitative evidence suggests that interim assessments do
play a part in increasing student achievement.
• Qualitative evidence suggests that the teacher analysis of data
resulting from these assessments has contributed to an
increase in the alignment of taught and tested curriculum, more
than to improved mathematics instruction.
• There is not strong evidence in the qualitative results that data
analysis, reflections or decisions made in the data-informed
decision-making model result in stronger mathematics lessons,
modifications to instruction that address student
misunderstandings, concept development or an increased
ability of students to apply skills.
Implications
• In the end, the administration and analysis of interim
assessments provides frequent a measurement tool
which fulfills the “jump higher theory of improvement”, that
is, if educators measure student performance enough,
and do nothing differently pedagogically, then students will
surely increase their scores (Elmore, R., personal
communication, May 19, 2012).
Recommendations for Practitioners:
• Deepen content knowledge in mathematics, especially for
•
•
•
•
•
multiple subject teachers;
Construct interim assessments using better items: (enhanced
multiple choice, constructed response and performance tasks)
that inform instruction;
Use inquiry protocols that help teachers analyze students’
learning levels and misconceptions;
Provide structures that support teachers in the collection,
reflection and connection of student learning data to their own
teaching and delivery of instruction;
Identify how teachers can build re-engagement lessons and
offer better support for students after a lesson;
Continue to support professional learning with distributed
leadership structures that enable practitioners to do engage in
this work.
Going Forward
Additional research should identify assessment and data analysis
practices for the tasks provided by the Smarter Balanced
Assessment Consortium for mathematics. Additionally, research
that explores those protocols that best elicit conversations that
promote deeper conversations about pedagogy and instructional
decisions supporting Common Core will be invaluable to the field.
The Predictive and Instructional Value of Interim Assessments
Author(s): Pon, Kathleen
Date: 2013 February
Type: Dissertation
URI: http://scholarworks.csustan.edu/handle/011235813/110