Collaborative Evaluation
Communities in Urban Schools
CEC Collaborators
University of Kansas – School of Education





Douglas Huffman, Ph.D. – Science Education
Kelli Thomas, Ph.D. – Mathematics Education
Danielle Barker – KU doctoral student
(Chemistry)
Carrie Hohl – KU doctoral student
(Engineering)
Anita Lundy - KU doctoral student & KCK
Teacher (Science Education)
Kansas City Kansas Public Schools
KCK science and mathematics coordinators
Bethel Elementary School – principal, IC, teachers
Emerson Elementary School – principal, IC,
teachers
Primary Goals
1. Improve the evaluation capacity of urban schools
by creating school-based collaborative evaluation
communities.
2. Develop graduate level educational leaders with
the knowledge and skills to evaluate science and
mathematics programs.
3. Provide teachers of science and mathematics
professional growth by improving their capacity to
use evaluation data and engage in continuous
improvement and data based decision-making.
4. Develop new models of evaluation capacity
building for K-8 schools and graduate students.
Purposes of CEC
 To create collaborative teams to examine teaching
and learning of math and science.
 To use evaluation and assessment data to inform
math and science instruction.
 To engage in collaborative “inquiry” as a means of
improving student achievement in math and/or
science.
 Help doctoral students in math and science education
engage in evaluation & assessment
Background
 Collaborative efforts that engage participants in a shared
evaluation experience of data gathering and analysis can lead
to changes in evaluation capacity of individuals and
organizations, science and mathematics teaching and learning.




Sarason (1996) urges districts and universities to create
opportunities for collaboration that move across current
hierarchies.
Cochran-Smith and Lytle (1999) contend, “Knowledge
emerges from the conjoined understandings of teachers
and others committed to long-term observation and
documentation of learners and their sense making” (p.
275).
Huffman and Kalnin (2003) found that when science and
math teachers engaged in collaborative inquiry, they not
only changed their instruction, but also began to make
school-wide changes.
If teachers don’t move beyond action research and
engage in school-wide changes they are “just playing in
the sandbox” (Elmore)
School Evaluation
Capacity Building
Explore District, State
and National Data
Monitor Results
Develop
Plan of Action
Analyze Data
Consider Implications
for District
Find Focus for
Evaluation
Collect Data
Graduate Student
Evaluation Capacity Building
University of Kansas
University of Minnesota
Year 1
Explore Evaluation
Collaborative
Evaluation
Communities
Professional
Conferences
Coursework
Year 2
Immersion in Evaluation
Collaborative
Evaluation
Communities
Coursework
Professional
Conferences
Videoconference
Videoconference
Eval Institute
The Bethel Elementary Case
Finding a Focus for Evaluation
1.
2.
3.
4.
5.
Curriculum alignment with state science
standards.
Pacing of instruction.
Instructional practices of teachers as they use
the curriculum.
Teachers’ science knowledge of content in the
curriculum.
Alignment with the state science assessment.
The Bethel Elementary Case
Data Collection
Bethel Team with our assistance created a
survey to gather initial data related to:
 the science curriculum
 teacher efficacy
 instructional practices
 assessment practices
 student learning outcomes
 teacher confidence with their own science
knowledge and understanding
The Bethel Elementary Case
Data Analysis
1.
2.
3.
4.
Teachers were not comfortable with using the FOSS kits.
Over 90% of the teachers responded that they struggle to
teach science as well as they teach other subjects.
Students rarely or never had the opportunity to either
design their own science investigation or use science
concepts to interpret and solve applied problems.
Teachers rarely used assessment to inform instruction in
science.
The Bethel Elementary Case
Plan of Action




Inventory FOSS kits and target specific grade levels
to use particular kits
Engage students in inquiry using the FOSS kits as a
framework for the learning experiences.
Write multiple choice mini-assessments for teachers
to use as embedded assessment throughout the
school year while using FOSS kits.
The intent for the mini-assessments is to provide
teachers with a means for gathering regular student
assessment data aligned with the curriculum, the
state science standards, and the state science test.
The Bethel Elementary Case
Monitor Results
 Sharing of instructional issues encouraged
collaborative reflection on changes that could be made
in the implementation of the FOSS kits
 Analysis of mini-assessments highlighted the need for
a deliberate approach to the sequencing of FOSS kits
from one grade level to the next
 Future work…..developing a sequence for FOSS kits
at each grade level that leads to greater depth of
knowledge and understanding, student achievement,
others?
The Emerson Elementary
Case
Finding a Focus for Evaluation
1. curriculum alignment with state mathematics
2.
3.
4.
5.
standards,
pacing of instruction,
instructional practices of teachers as they use the
curriculum,
teachers’ mathematical knowledge of content
contained in the curriculum,
curriculum alignment with the state mathematics
assessment.
The Emerson Elementary
Case
Data Collection
 Emerson Team with our assistance created a
survey to gather initial data related to:






the mathematics curriculum,
teacher efficacy,
instructional practices,
assessment practices,
student learning outcomes,
teacher confidence with their own mathematics
knowledge and understanding
The Emerson Elementary
Case
Data Analysis
1.
2.
3.
Teacher perceptions about Investigations in Number,
Data, and Space (TERC, 1998) varied widely among
teachers in the school.
Teachers agreed that they had the ability to effectively
teach mathematics. However, few teachers felt strongly
about this ability.
Responses to several questions about student learning,
teacher assessment of student learning, and instructional
practices lead to an emphasis on the issue of
assessment.
The Emerson Elementary
Case
Plan of Action
 Write multiple choice mini-assessments for teachers to
use as embedded assessment throughout the school
year while using Investigations in Number, Data, and
Space
 The intent for the mini-assessments is to provide
teachers with a means for gathering regular student
assessment data aligned with the curriculum, the state
mathematics standards, and the state mathematics
test.
The Emerson Elementary
Case
Monitor Results
 Team analysis of individual student assessments
highlighted areas to target for improvement
 Students need experiences working with money and
computations involving money
 Integrate mathematics with the positive behavior
support program
 Future work…..student achievement, correlation
between district selected norm referenced test and
state assessment, including all teachers in data
analysis, others?
Future…
 The CEC partnerships continue to meet
on a regular basis to discuss and debate
issues related to mathematics and
science teaching and learning.
 We anticipate repeating the CEC inquiry
process in each current school and with
new schools that join the project.
 Through this work, we hope to build a
culture for sustained evaluation and
assessment of educational programs in
the schools.
Discussion…
 What data would you suggest we collect
to study the impact of CEC?
 How best to collect?
 What are the most important impact
questions to ask?
 What are your ideas and suggestions?