A Provisional Agenda for Research
on Improving the Quality of
Mathematics Teaching on a Large
Scale
Paul Cobb and the MIST Team
Vanderbilt University
University of Washington
Michigan State University
McGill University
Purpose
• Outline a set of issues that need to be
addressed if research is to provide adequate
guidance for large-scale instructional
improvement efforts in mathematics
– Across a large urban district
Overview
• Preamble: what counts as high-quality
instruction
• Background: ongoing work as a setting for
appreciating the limitations of current
research
• Proposal for issues that need to be addressed
What Counts as Instructional Quality?
• Has to be justified in terms of students’
learning of mathematics that is worth
knowing
– Conceptual understanding as well as procedural
fluency
– Justifying solutions, evaluating the reasonableness
of solutions, generalizing from solutions, making
connections among multiple representations of
mathematical ideas
Research on Students’ Mathematical
Learning
• Rigorous mathematical tasks
• Individual or small group work
• Whole class discussion
– Teacher presses students to:
• Explain and justify their reasoning
• Make connections between different solutions
Goals for Teachers Learning
• High-leverage instructional practices
– Planning and conducting productive whole class
discussions
– Setting up rigorous mathematical tasks
• Specific types of knowledge implicated in the
enactment of these practices
– Mathematical knowledge for teaching
– Vision of high-quality mathematics instruction
– View of students’ mathematical capabilities
Challenge for Districts
• How to organize, support, and press for
teacher learning across the entire system
– What guidance can research provide?
Background: MIST Project
• Four-year collaboration with four large urban
districts – 360,000 students – 2007-2011
• Continued collaboration with two districts –
180,000 students – 2011-2015
• Investigate (and support) the districts’
instructional improvement efforts in middlegrades mathematics
Background: MIST Project
• High proportion of students from traditionally
underserved groups
– Limited financial resources
– High teacher turn over
– High proportion of novice teachers
• Atypical in one respect:
– Aiming at ambitious goals for student learning and
thus for teachers’ instructional practices
District Participants
• 30 middle-grades mathematics teachers in 610 schools in each district
• Mathematics coaches
• School leaders
– Principals, assistant principals
• District leaders
– Across central office units that had a stake in
mathematics teaching and learning
Collaboration with Districts
October
• Interview district leaders to document current
strategies for improving middle-school
mathematics
JanuaryMarch
• Audio-recorded interviews with the 200
participants to document how the districts’
strategies are actually playing out in schools and
classrooms
Collaboration with Districts
FebruaryMay
• Analyze transcripts of the 200 interviews
• Identify and explain gaps between each district’s
intended and implemented improvement strategies
• Develop a detailed report for leaders in each district
• Shared findings and made actionable recommendations
• Meet with district leaders to discuss our findings and
recommendations
May
Collaboration with Districts
• District leaders attempt to act on our
recommendations to a significant extent
• Become co-designers of district improvement
strategies
– Participants in as well as observers of the districts’
instructional improvement efforts
Collaboration with Districts
• Formulating recommendations: Have to
address concrete organizational design
problems
• Occasion to appreciate
– The types of problems that district leaders have to
address
– Extent to which current research can provide
guidance – hence this talk
Research Goal
• Develop an empirically grounded theory of
action for instructional improvement at scale
– Can inform other districts’ instructional
improvement efforts
Ongoing Analyses
• Initial conjectures about supports and
accountability relations
– Drew on then available literature
• Conjectures informed initial recommendations
to districts
• District leaders acted on recommendations –
opportunity to test and revise conjectures
Retrospective Analyses
• On-line surveys for teachers, coaches, and school leaders
• Video-recordings of two consecutive lessons in the 120
participating teachers’ classrooms
– Coded using the Instructional Quality Assessment
(IQA)
• Assessments of teachers’ and coaches’ Mathematical
Knowledge for Teaching (MKT)
• Video-recordings of select district teacher professional
development
• Audio-recordings of teacher collaborative planning time
• Student achievement data
Theory of Action for Instructional
Improvement at Scale
• A coherent instructional system:
– Instructional materials + professional
development + assessments to inform instruction
+ additional supports for struggling students
• Mathematics coaches’ practices in providing jobembedded support for teachers’ learning
Theory of Action for Instructional
Improvement at Scale
• School leaders’ practices as instructional
leaders in mathematics
• District leaders’ practices in supporting the
development of school-level capacity for
instructional improvement
Research Team
Paul Cobb
Kara Jackson
Ilana Horn
Tom Smith
Erin Henrick
Ken Frank
Research Team
Jessica Rigby
Jonee Wilson
Brooks Rosenquist
Britnie Kane
Brette Garner
Emily Kern
Mahtab Nazemi
Mollie Appelgate
Adrian Larbi-Cherif
Charlotte Munoz
Jason Brasel
Seth Hunter
Megan Webster
I-Chien Chen
Explicit goals for students’
mathematics learning
Vision of high-quality instruction
Teacher professional
development
Instructional materials
Component 1:
Coherent
Instructional System
(Bryk et al., 2010;
Newmann et al., 2001)
Additional supports for struggling
students
Assessments to inform ongoing
instructional improvement
Explicit goals for students’
learning
Vision of high-quality instruction
Instructional materials
Component 1:
Coherent
Instructional System
Vision of high-quality instruction:
• Small set of high-leverage
practices that are potentially
learnable in the context of highquality professional
development
Instructional materials:
• Grounded in student learning
trajectories that aim at
significant mathematical ideas
• Pull-out Professional Development
(PD):
• Specific PD designs – promising
findings
• Grounded in classroom practice –
pedagogies of investigation and
enactment
• Most work in pre-service – have
extrapolate to in-service
• Teacher Collaborative Time (TCT)
• Most researcher-led – potentially
productive types of activities
• Naturally occurring –
characteristics of productive
teacher groups
Teacher professional
development
Component 1:
Coherent
Instructional System
Component 1:
Coherent
Instructional System
Assessments to inform
instruction
• Formative assessment
systems
– Aligned with ambitious
goals for students’
learning
– Grounded in
trajectories of
students’ learning
• Goal: support
struggling students to
participate effectively
in mainstream
instruction
Component 1:
Coherent
Instructional System
Additional supports for
struggling students
Coherent Instructional System
• Collaborating districts: fragments of a coherent
instructional system
– Strengths: explicit goals for students’ learning, vision of
high-quality instruction, instructional materials
– Challenge: teacher professional development – district
capacity
– Challenge: TCT – expertise + leadership of meetings –
district capacity
– Weakness: additional supports for struggling students –
not aligned with mainstream classroom instruction
Needed Research: Developing District
Capacity
• Researchers typically assume full
responsibility for “building” particular
elements
• The problem of scale involves supporting
districts’ development of the capacity to
create, coordinate, and sustain the elements
of such a system
Developing District Capacity: Sacrificial
Offering
• Example: co-designing and co-leading PD for
coaches with district mathematics specialists
– Support the development coaches’ capacity to
design and lead high-quality teacher PD
– Gradual hand over of responsibility to district
mathematics specialists
• Overall goal: Investigate how to support districts’
development of capacity to develop and sustain a
cadre of mathematics coaches
Needed Research: Interrelations
Between Elements of the System
• Current research typically focuses on the
individual elements of a coherent instructional
system
• Also need to investigate interrelations
between various the elements, and between
elements and other components of ToA
– Which are preconditions for the development of
other elements/components?
Developing District Capacity: Sacrificial
Offering
• Example: co-designing and co-leading PD for
school leaders
– School leaders press for instructional
improvement
– Coaches support teachers in meeting those
expectations
• Investigate development of aligned support and
press for teachers’ improvement of their
instructional practices
Mathematics Coaching
• Finding: Teachers’ improvement of their
instructional practices depends crucially on
their access to colleagues who have already
developed accomplished practices
• Three of our four districts: Small proportion of
accomplished teachers
• Critical role of coaches as more accomplished
colleagues
Mathematics Coaching
• Design and lead pull-out PD
• Work with groups of teachers during TCT
– Current research + our findings indicate
importance of leadership/expertise in TCT
• Support teachers one-on-one in their
classrooms
– Build on pull-out PD and work in TCT
Current Research on Coaching
• Provides little guidance on:
– Types of activities in which coaches might engage
teachers
– Coaches practices as they enact these activities
– Supporting the development of a cadre of
accomplished mathematics coaches
• One of the collaborating districts: four years of
sustained professional development for coaches
• Only slightly ahead of the teachers they were expected
to support
Needed Research: Delineating Goals
for Coaches’ Learning
• Develop testable conjectures about potentially
productive coaching practices
• Working with groups of teachers:
• Draw on studies of researcher-led and naturally
occurring PLCs
– Kazemi and Franke: Potentially productive activities
– Horn: Press teachers of key issues
Needed Research: Delineating Goals
for Coaches’ Learning
• Working with teachers in their classrooms:
– Draw on research on teacher learning and
professional development
• Points to importance of modeling and especially coteaching
• Observation/feedback at specific points in teachers’
development
– e.g., Tuning their enactment specific practices
Needed Research: Delineating Goals
for Coaches’ Learning
• Important to explicate the forms of
knowledgeability implicated in the enactment
of proposed practices
– Researchers do not typically report what they
needed to know
Needed Research: Specifying Forms of
Knowledgeability
• Mathematical knowledge for teaching (MKT)
• Additional candidate: an envisioned trajectory
for teachers’ learning
– Classroom management
– Student engagement
– Teacher questioning
• Informs decisions about which aspects of
practice to work on with teachers
Needed Research: Designs for
Supporting Coaches’ Learning
• Both settings:
– Draw on very limited literature on coach PD
– Extrapolate from work on teacher learning and
teacher PD
Needed Research: Test and Revise
Designs for Coaches’ Learning
• Design experiments to:
– Test and refine conjectures about supports for
coaches’ development of target practices
– Assess in terms of improvements in:
• Coaches’ practices
• Quality of teachers’ classroom instruction
• Student learning
Coach PD: A Sacrificial Offering
MIST & District Math
Leaders collaboratively
plan for upcoming
session
Coach PD Session
MIST views videorecording of pilot PD in
light of goals for coaches’
learning
Coaches lead PD
with pilot group
School Instructional Leadership
• Standards-based reform: Principals and
assistant principals increasingly expected to
act as instructional leaders in specific content
areas
– Manage instruction rather than manage around
instruction
Current Research School Instructional
Leadership
• No consensus on what school leaders need to
know and be able to do in order to be effective
instructional leaders in mathematics
– General content-independent characteristics of
high-quality instruction
• Observe instruction and provide feedback
– MKT, student mathematical learning, high-quality
mathematics instruction, teacher learning
• Coach mathematics teachers
Findings
• Interviews – vision of high-quality
mathematics instruction (VHQMI)
– Form rather than function views
– Consistent with teachers’ accounts of the
feedback they receive from school leaders
• Extensive professional development
– Focused on general, content-independent
characteristics of high-quality instruction
Initial Findings
• General characteristics of high-quality inquiryoriented instruction
– Too abstract – not able to connect to concrete
instructional practices
• MKT, student mathematical learning, highquality mathematics instruction, teacher
learning
– Beyond the capacity of most districts
Needed Research: Delineating Goals
for School Leaders’ Learning
• Develop testable conjectures about potentially
productive school leadership practices
• Justify in terms of:
– Direct support/press for teachers’ learning
– Indirect support – developing conditions for
teacher learning
Current Bets
• Identify and capitalize on instructional
expertise in the school
• Observe mathematics instruction and provide
feedback
• Participate in teacher collaborative time (TCT)
• Support coach to support teachers’ learning
School Leadership Routine
Attend Teacher
Collaborative
Time
Observe
Classroom
Instruction
Meet with
Mathematics
Coach
School Leader
And Coach:
Quality of Individual
Teachers’ Instruction
+
How to
Support Instructional
Improvement
+
Jointly Plan for
Teacher
Collaborative
Time
Needed Research: Delineating Goals
for School Leaders’ Learning
• Important to explicate the forms of
knowledgeability implicated in the enactment
of the proposed practices
– Vision of high-quality mathematics instruction
• Observing instruction and giving feedback
• Identifying and leveraging instructional expertise
Needed Research: Designs for
Supporting School Leaders’ Learning
• Extrapolate from work on teacher learning
and teacher PD
– Vision of high-quality mathematics instruction
• Distinguish between high- and low-rigor tasks
• Distinguish between strong and weak enactments of
specific high-leverage instructional practices
Needed Research: Test and Revise
Designs for School Leaders’ Learning
• Design experiments to:
– Test and refine conjectures about supports for
school leaders’ development of target practices
– Assess in terms of improvements in:
•
•
•
•
Instructional leaders’ practices
Direct and indirect supports for teachers’ learning
Quality of classroom instruction
Student learning
Coaching and School Instructional
Leadership
• Important to take account of relations
between members of different role groups
– Coaches’ effectiveness in supporting teachers’
learning depends on relationship with school
leaders
– Coach is a potential support for school leaders’
learning
District Leadership
• Also important to take account of coaches’
and school leaders’ relations with district
leaders
– Supports for their learning
– What they are held accountable for
• Potential tension between raising student achievement
in the short term and improving the quality of
classroom instruction in the long term
Why Does Current Research Provide
Only Limited Guidance?
• Math education, teacher education, and the
learning sciences:
– Student learning, instructional activities and tools
– Teaching, teacher professional development,
teacher collaborative time
• Typically bracket out the school and district
contexts in which teachers’ learning occurs
Why Does Current Research Provide
Only Limited Guidance?
• Educational policy and leadership
– Typically instructionally agnostic
• Content + vision of high quality instruction matter
– Research in policy and leadership can be relevant
• Have to read through lens of what counts as highquality mathematics instruction
Supporting Instructional Improvement
at Scale
• A problem of both teacher learning and
organizational learning
– Need to specify goals for organizational as well
and teachers’ learning
• Differentiate between organizational change and
organizational learning
Organizational Learning
• Conjectured practices, social relations, tools,
and routines are provisional goals for
organizational learning
– Provisional goal for the organizational learning of
a district: The creation and ongoing refinement of
a coherent instructional system
– Provisional goal for the organizational learning of
a school: Establishment of the school leadership
routine
Papers and instruments downloadable at:
http://vanderbi.lt/mist