School - University Partnerships for Science and Mathematics Reform
A one-year proposal for $100,000 submitted in Dec. 2001
to the Arizona Board of Regents: Eisenhower Math-Science Program
David Hestenes, Research Professor of Physics, ASU
Jane Jackson, Academic Associate, Department of Physics, ASU
480-965-8438, Jane.Jackson@asu.edu
Final Report by Jane Jackson. June 2003 (revised and updated in fall 2008, with underlining
to highlight Mesa Public Schools activities and results, since that was our primary partner
district)
Introduction and overview:
The idea that K-12 schools need help from universities to upgrade the science/math
curriculum has emerged as a major initiative in the National Science Foundation: the
Mathematics and Science Partnerships Program. It provides substantial federal funding for the
kind of school–university partnership created in this project. Such school–university
partnerships provide schools with a much-needed mechanism for continuous researchbased science/math education reform – – perhaps an essential mechanism
Schools and school districts are ill-equipped to conduct science and math professional
development on their own, because they lack necessary expertise in mathematics, science and
technology as well as resources to keep up-to-date with advances in science and math curriculum
materials and pedagogy. Those resources reside primarily in the nation’s universities.
It is imperative that we learn how to make such partnerships work effectively as soon as
possible so that federal money will be well spent and Congress will have justification for
continuing this program. For this reason we did not wait until the new program took effect last
year; we applied immediately for this “proof of concept” project on partnership design and
implementation, with high potential to serve as a model for other universities. As we have been
laying the groundwork for statewide partnerships for a decade, we are well prepared to get
Arizona partnerships up and running rapidly.
The Arizona Science and Technology Education Partnership (AzSTEP) began as a
statewide partnership between the ASU physics department and high school physics teachers to
support the teachers as local leaders of K-12 science and technology education reform. In the four
years since it started, AzSTEP has engaged half of Arizona high school physics teachers in
Modeling Workshops. Physics teachers cultivated in AzSTEP provide a cadre of leaders for
expansion of AzSTEP to a statewide network of school district - university partnerships.
 Immediate Goal: To set up trial school-university partnerships in local public schools and
demonstrate the effectiveness of this mechanism for providing school districts with access to
university resources to drive science/math/technology education reform.
 Long-range Goal: To establish the Arizona Science and Technology Education Partnership
(AzSTEP) to broker school-university partnerships statewide and secure their funding to
support continuous upgrades in science/math/technology education.
1
Although ultimately science and mathematics reform must encompass the entire twelve
years of schooling, this project addressed only one component, albeit a crucial one: the need for
an integrated physical science and mathematics workshop for seventh through ninth grade
teachers. Among other things, it established a much-needed curricular link between high schools
and their feeder schools. Physical science courses in middle school and ninth grade must provide
students with the conceptual underpinnings needed for success in high school physics and
chemistry. At the same time, these courses must develop basic scientific literacy for students who
will not go on to physics and chemistry.
As AzSTEP's goal is to establish a permanent mechanism for statewide reform affecting
all students, we targeted districts with the greatest likelihood of success in this “proof of concept”
project. Participating school districts were Mesa Public Schools, Peoria Unified, and Glendale
Union High School District (GUHSD) with its two feeder districts, Washington (WESD) and
Glendale Elementary (GESD). They were selected on the basis of administrative commitment
and teacher qualifications. Administrative commitment is, of course, essential for district-wide
reform. But the most important requirement is the existence of a cadre of teachers capable of
leading implementation of the program. Each of the selected districts already has such a cadre,
composed of high school teachers who have been active in the Modeling Program for years and
have shown that they are fully attuned to the aims and methods of the project. In fact, their
reputation for outstanding teaching was the most important factor in gaining administrative
support from their districts. Thus two expert high school teachers, one in physics and one in
math, were selected to lead the appropriate district-wide workshop. In Glendale, two workshops
were held, one including high schools to which GESD feeds and the other similarly with WESD.
Each district pledged to contribute $10,000 or more for teacher stipends. In fact, Mesa
and GUHSD contributed considerably more than this, whereas WESD contributed considerably
less. Each district applied for and was awarded an Arizona K-12 Center "Fast Track" grant of
$10,000, since AzSTEP was a pre-approved provider of this Modeling Workshop.
Evidence of school district needs:
Mesa Public Schools: Mesa is the third-largest city in Arizona. It has experienced
explosive growth in recent years. With 74,100 students in more than 90 schools and alternative
K-12 programs, Mesa Public Schools is the largest district in the state of Arizona. It is now an
urban district, with sections of the city inhabited almost exclusively by ethnic, mostly
disadvantaged families. Hispanic and American Indian youths constitute almost a third of the
student population, and 45% of Mesa’s students qualify for free or reduced lunch subsidies.
District data indicate that ethnic children in Mesa’s economically disadvantaged regions
perform poorer on measures of educational achievement. As of September 2002, 18 of Mesa’s 35
Title I K-12 schools failed to make Annual Yearly Progress (AYP) for two years or more, subject
to state intervention according to new regulations in President Bush’s No Child Left Behind
legislation. These data underscore the profound increase in diversity, and the accompanying
needs, of many students and schools in the district. If unchecked, this situation will result in
significant numbers of students failing to meet minimum competencies, in turn qualifying Mesa
as a “failing” district according to the new law. Thus Mesa’s school community is motivated to
find ways to help the district’s lowest-achieving students.
Similar data and needs exist in WESD, GUHSD, and especially in GESD, where 65% of
students are on the free lunch program.
2
Our telephone interviews in spring 2002 with science and mathematics teachers in all
thirteen Mesa junior high schools revealed that few mathematics and science teachers were
coordinating their courses. Moreover, junior high schools have been experiencing rapid
turnover of science teachers, with the result that a preponderance of teachers are inexperienced in
use of technology in instruction.
GUHSD: Our district-wide survey of mathematics teachers in August 2001 revealed that
only one-third of 34 ninth grade math teachers took physics in college. For the most part,
articulation in mathematics and science was lacking between GUHSD, GESD and WESD.
GESD and WESD: In spring 2001, only 3% of Glendale Elementary District eighth
graders passed the mathematics section of the AIMS test. Graphing calculators were not used in
most middle schools, thus depriving students of an important representational tool for learning
mathematics.
Peoria USD: In surveys returned to us by 42 science teachers in grades 6, 7, and 8 in
Aug. 2001, the median reported number of hours per year of school inservices related to science
or math content was zero! Only half of the teachers have taken a college-level course in physics
or earth science, and only one-fifth had astronomy. Most high school teachers of physical science
majored in life science, and some have only a minor in any science. Fewer than one-third of
ninth grade science teachers have had a calculus course.
Development of workshop:
Larry Dukerich, the Mesa Modeling Workshop leader, described development of the Modeling
Workshop in an e-mail to Jeremy Roschelle of SRI International (Jan. 4, 2003). Dr. Roschelle is
a collaborator with Professor James Kaput in the SimCalc mathematics project at the University
of Massachusetts - Dartmouth.
"The course development for Models of Physical Science originated years ago when physics
teachers in Modeling Workshops attempted to prepare a set of remedial materials for underprepared physics students patterned after the efforts of Jim Minstrell and Arnold Arons.
Teachers found these to be of use for their ninth grade physical science courses, where, as you
likely
know, the textbooks are abysmal. David Hestenes challenged Jeff Hengesbach (now at the Webb
School in Claremont, CA) and me to flesh out the course so that it would weave the threads of
modeling tools, structure of matter, and energy into a coherent course that could be used to
prepare students for a physics-first course in ninth grade. The design specifications also called
for greater integration with the math curriculum for eighth and ninth grades. David strongly
recommended that we work Sim-Calc materials into the course design. We met with you up at
NAU a year ago and we went to work.
The task was simply too large for two teachers working part-time to complete, but we finished
the first three units well enough (we think) that the materials could be used for the Physical
Science-Math Modeling Workshops that you and Jane discussed. Since our primary goal was to
develop a physical science course, we chose those Sim-Calc materials that seemed most relevant
to our needs in the first unit."
Recruitment of teachers:
In spring 2002, each of the five school districts recruited teachers using a two-page
application form. Leaders were sought: stated criteria were, for example (GUHSD): "Preference
3
will be given to applicants showing the greatest potential for improvement of student learning,
implementation of technology and modeling instruction on a regular basis, and leadership at the
local school for implementation of technology."
Pledged commitments were, for example (Peoria):
* Are you willing to give a 1/2 hour standardized test next year as a pre-test and post-test?
* Are you willing to serve as a resource in your school for science & math teachers, both on an
individual basis and in conducting training sessions on classroom technology for groups?
* Are you willing to be actively involved in a learning community of physical science and math
teachers?
GUHSD asked principals to choose a team of a math and a science teacher from among
applicants, and to try to provide common planning period. Other districts recruited too late for
this.
Summer modeling workshops:
Ninety-two teachers participated in four three-week Modeling Workshops in June at:
* Independence High School: 11 GESD teachers, 10 GUHSD teachers
* Washington High School: 12 WESD teachers, 11 GUHSD teachers
* Dobson High School: 23 Mesa teachers, 1 from Desert Eagle charter school, 1 from Gilbert HS
* Peoria High School: 20 Peoria teachers, 1 from Higley HS, Morenci HS, AZ School for Arts.
All teach grades 7, 8 or 9 except for six lower-grades GESD teachers and a few sixth
grade Peoria teachers. Half of the 92 teachers teach science, 35 teach mathematics, and the rest
teach both or all subjects. Forty-two schools were represented in the five districts, and almost
two-thirds of the schools had two or more participants. Six teachers at Cholla Middle School in
WESD participated, including almost the entire science department.
All workshops except Dobson High School met for six hours daily, with a total of 90
contact hours in 13 to 15 days. Dobson had four hours each afternoon for 15 days, for most Mesa
math teachers were committed to teach in mornings. The half-day workshop design was best:
teachers worked effectively and were still enthusiastically learning in the third week, whereas
teachers in the three all-day workshops were tiring and losing focus.
Since optional ASU graduate credit in physical science or mathematics was available,
teachers were assigned homework such that the total time devoted to the course by a participant
was at least 135 hours as required by the Arizona Board of Regents for a three-semester hour
course.
Workshop leaders were expert high school teachers in the respective districts, one each in
physics and chemistry (listed first) and mathematics.
* Independence High School: Hal Eastin and Cheryl Bryant.
* Washington High School: Sheila Ringhiser (with Dawn Harman) and Veronica Carlson.
* Dobson High School: Larry Dukerich and Stella Ollarsaba.
* Peoria High School: David Hill and Curt Aylward.
Workshop leaders communicated regularly via e-mail, sharing successes, problems, and
strategies.
The workshops provided teachers with education in standards-based content and
instructional strategies. ASU is well-qualified for this: in 2001 an Expert Panel of the U.S.
Department of Education designated the Modeling Instruction Program as one of two Exemplary
4
K-12 science programs in the nation, out of 27 projects reviewed. Ratings were based on these
criteria: (l) Quality of Program, (2) Educational Significance, (3) Evidence of Effectiveness, and
(4) Usefulness to Others. In 2000 a different Expert Panel of the U.S. Department of Education
designated Modeling Instruction as one of the seven best K-12 educational technology programs
in the nation (out of 134 applicants!). Links to panel reports are at http://modeling.asu.edu .
Participants were introduced to the Modeling Method as a systematic approach to the design
of curriculum and instruction. The name Modeling Instruction expresses an emphasis on making
and using conceptual models of physical phenomena as central to learning and doing science.
Adoption of "models and modeling" as a unifying theme for science and mathematics education
is recommended by both NSES and NCTM Standards as well as AAAS Project 2061. However,
to our knowledge, no other program has implemented it so thoroughly. Modeling workshops
meet or exceed many Arizona professional teaching standards, and grade eight and high school
performance objectives in the science and mathematics standards.
The essence of Modeling Instruction is that content is reorganized around basic models to
increase its structural coherence. Student activities are organized into modeling cycles, which
engage students systematically in all aspects of modeling. The teacher guides students
unobtrusively through each modeling cycle, with an eye to improving the quality of student
discourse by insisting on accurate use of scientific terms, on clarity and cogency of expressed
ideas and arguments. After a few cycles, students know how to proceed with an investigation
without prompting from the teacher. The main job of the teacher is then to supply them with
more powerful modeling tools. Lecturing is restricted to scaffolding new concepts and principles
on a need basis. Details of the modeling cycle are at http://modeling.asu.edu/modeling-HS.html.
In Modeling Workshops, participants are supplied with a complete set of course materials
(resources) for a semester, and they work through activities alternately in the roles of student or
teacher.
Workshop goals for teachers were to:
* improve their instructional pedagogy by incorporating the modeling cycle, inquiry methods,
critical and creative thinking, cooperative learning, and effective use of technology in instruction,
* understand content in structure/properties of matter, motion, energy, scientific thinking skills,
and related skills in each of the six Arizona Mathematics Standards (Strands),
* strengthen coordination between mathematics and physical science.
Anticipated student outcomes included improved understanding in geometrical and physical
properties of matter, motion, energy, graphing, and related mathematics and reasoning skills such
as measurement, proportional reasoning, and relation between graphs and equations.
Thematic strands woven into this course included scientific modeling, structure of matter,
energy, and use of calculators and computers as scientific tools. Mathematics instruction was
integrated seamlessly throughout the entire course by an emphasis on mathematical modeling.
The course included these models and modeling activities:
1. Modeling geometric properties of matter: length, area and volume
2. Modeling physical properties of matter: mass and density
3. Model of a point particle with constant velocity
4. An atomic model of solids, liquids and gases
5. Energy and the states of matter.
5
Larry Dukerich reflected on his workshop experience in his e-mail to Jeremy Roschelle of
SRI International (Jan. 4, 2003):
" The three-week summer workshop was an eye-opening experience for the workshop
leaders as well as the participants. Participants began to realize that it was possible for junior
high math and science teachers to use common vocabulary - that the math teachers could do
activities and the science teachers could use TI-83+ calculators (it was as if some taboos had
been lifted). The workshop leaders became more painfully aware of how fragmented our math
and science curricula were. Due to the pressures placed on math teachers for their students to
perform well on the state-mandated test, it's as if blinders have been placed on the majority of
math teachers. For many, their strategy has been to prepare for the test rather than to try an
approach that stressed a conceptual understanding rather than drill-and-kill. The workshop
leaders (and the math teachers who got it) tried to convince the rest that more iterations of what
hadn't worked in the past wasn't going to lead to success. Here's a sample exchange:
Traditional teacher, "How are we going to do this [Sim-Calc approach to slope] AND what we
usually do?" Reformed teacher, "No, you're going to do this INSTEAD of what you've been
doing."
Unsolicited comments by teachers, soon after the workshop ended (via e-mail):
 I really enjoyed the Math & Science Modeling Workshop. Hal and Cheryl did an excellent
job!! I learned so much. I have so many new and exciting modeling lessons and materials to
share with my students this upcoming school year.
 Had to let you know how much I enjoyed the class. Larry and Stella did a fabulous job in
every way. Thanks for getting me involved in this venture...my fear of physics has been
greatly reduced and I intend to continue with the program and your suggestions.
Teachers' workshop evaluations, measurement of teacher learning by site:
Workshop rating (10=superb): Peoria 9.7,WESD/GUHSD 9.3, Mesa 9, GESD/GUHSD 6.2.
GESD presented difficult challenges. They had to select six teachers of grades four to six
because 7th & 8th grade teachers taught summer school. This slowed the pace of the workshop,
and the classroom needs of the lower grades teachers weren't met because the level of activities
was too high for them. On the other hand, although the content and level were appropriate for the
ninth grade teachers, the pace was too slow for them. Also, one workshop leader, a novice leader,
worked diligently and tirelessly but was less than successful because of a leadership style that is
more conducive to working alone rather than with a co-leader.
Physical Science Concepts Inventory (PSCI) teacher pretest & posttest mean scores:
All teachers were pre-tested and post-tested on the Physical Science Concepts Inventory
(PSCI) and the Math Concepts Inventory (MCI). These instruments for students were developed
specifically to align with this course. The first eight questions are in pairs, from the Classroom
Test of Scientific Reasoning, a research-based test developed in the 1970’s by Anton Lawson, on
which extensive student data internationally are available (some are cited below). Most others are
released NAEP and TIMSS questions, and questions similar to the NCLB-required Arizona highstakes math test for eighth and tenth grades, the AIMS test. Considerable overlap exists between
the two inventories, for it is intended that the PSCI be given to science students and the MCI to
6
mathematics students. The PSCI and MCI were last revised in June 2003, as a result of these
workshops; they are in .pdf at http://modeling.asu.edu/MNS/MNS.html. The password to open
them can be obtained by e-mail to Jane.Jackson@asu.edu. (The most important revision is that
the harder pair of the four proportional reasoning questions, #7 and 8, was deleted.)
Teachers began the workshop with poor understanding of matter and energy; and teachers
with lowest pretest scores improved most, in general. In Mesa, the pretest mean was 69%,
posttest mean was 80%; WESD/GUHSD: pretest 73%, posttest 81%; GESD/GUHSD: pretest
69%, posttest 77%, Peoria: pretest 66%, posttest 83%. All teachers at each site except Peoria
took the posttest on the last workshop day. The Peoria posttest was given at the Saturday followup session in January; only six Peoria teachers' names could be matched pre- and posttest.
(Appendix A shows each teacher’s scores; it is available upon request to Jane.Jackson@asu.edu).
Content subscales of PSCI (but not determined for Peoria, since the sample was small) are:
* Conservation of mass (questions 1 & 2): everyone got these right on the posttest.
* Conservation of volume (questions 3 & 4): Eight out of 65 (12%) got #3 and #4 wrong on the
posttest. Five of the eight were from Mesa.
* Proportional reasoning (questions 5-8): at each site, the mean pretest score was about 75%;
mean scores improved to about 85% on posttest. Most teachers answered all four questions right
on both tests, but several teachers answered all questions wrong on the pretest and improved to
one right, indicating that they do not understand even the basics of proportional reasoning.
* Graphing skills/motion (four questions); at each site, pretest was about 90%, posttest about
95%.
* Geometrical & physical properties of matter (four questions): Mesa started lowest, at 70%;
WESD/GUHSD started highest, at 90%. Mesa and GESD/GUHSD ended at about 85%.
* Atomic model of matter (six questions): at each site, pretest and posttest mean scores were
about 65%. However, actual teachers' scores were diverse, from none right to all right, with little
improvement by anyone.
* Energy and states of matter (four questions): Tests showed tremendous improvement in most
Mesa teachers' scores; moderate or little improvement in individual teachers' scores at the two
Glendale sites. At each site, the mean pretest score was 40%. Mesa teachers improved by far the
most, to 75%; WESD/GUHSD to 50%. GESD/GUHSD didn’t include this content because of
slower pace due to six elementary school teachers; their mean posttest average was 45%.
Teachers' scores were diverse in the pretest at each site, from none right to three right.
In December 2002, after I related the above results to Larry Dukerich, he wrote to me,
"I would like to caution you against reading too much into the test results from the Physical
Science Concepts Inventory. Unlike the FCI [Force Concept Inventory], which was developed
over years based on interviews with students to come up with plausible distracters, this test is an
amalgam of items selected ad hoc from a variety of sources with different goals in mind. This is
not to say that the individual questions are not good, but that this collection of loosely related
items does not constitute a coherent test design.
Drawing conclusions about the relative effectiveness of the instruction in the four workshops
may be inappropriate given the test limitations. Here's my analysis of the sub-groupings in the
questions.
7
1. The questions borrowed from Lawson's Classroom Test of Scientific Reasoning (items 1-8)
have been banging around for quite some time. ... The majority of the teachers already knew how
to do items 5-8. Those who did not, showed only slight gains primarily because there was little
instruction on proportional reasoning in the workshop.
2. Items 9 - 12 [graphing/motion] were too easy for teachers of 8th and 9th grade - hence
they fail to discriminate between teachers who already understood the concepts and those who
learned as a result of the workshop.
3. On the positive side, I believe that questions 13-15 and 26 (geometric properties of matter)
could be used to test how much of an effect the workshop had on teachers as this concept was
explicitly covered in the workshop.
4. The items on the atomic model of matter are questions that probe deep misconceptions
about the atomic model in much the same way as does the FCI. I am not surprised that, given
the short duration of the workshop, that there was little or no gain. This should tell us more
about how we have to structure the workshop in the future.
5. The relative success of the Dobson [Mesa] workshop teachers was due, in great part to the
fact that we got through all three units, whereas teachers in the other workshops barely got to
energy, if at all. Stella told me that I was a slave driver, as I pushed the teachers to complete the
materials, knowing that the energy treatment would be novel for most of them. I have confidence
in these items simply because I know how long and hard Gregg Swackhamer worked on them.
So I urge caution in using these test results to draw conclusions about the workshops."
Follow-up meetings, and reflections by workshop leaders:
Each district planned and funded its own follow-up events, subject to AzSTEP's request of at
least two follow-up events of significant depth, preferably totaling ten hours.
GESD:
In August, most GESD teachers met for six hours on a Saturday with Hal Eastin, a workshop
leader, to develop lessons for their classes. In spring 2003, Renee Kopcha, the GESD curriculum
coordinator, organized three afternoons of classroom teaching demonstrations followed by
discussions. The workshop co-leader, Cheryl Bryant, reported that these model teaching meetings
were valuable, professionally and personally. She wrote, "Some GESD teachers do amazing
things with their students! Very good structure, policy, and procedures! My students can't fool
me any more, when they tell me they weren't taught well at GESD. Unfortunately, GESD has
huge turnovers of students, teachers, and principals; this is not conducive to student learning!"
Cheryl said she's convinced that a key to improved teaching is for teachers to observe other
teachers, followed by reflection and sharing of ideas. A great need exists for this in GESD, she
said, for teachers are isolated, unlike at GUHSD, where a mentoring program is in place at each
of the nine high schools.
GUHSD:
Debi Plum, GUHSD mathematics coordinator, organized a full day of follow-up in October,
including pay for substitutes for 18 teachers and five workshop leaders. Since ninth grade science
is primarily earth/space science, not physical science, workshop co-leader Dawn Harman
8
presented modeling activities in astronomy. Teachers worked on the following technology and
modeling instructional techniques:
Implementation discussion – what’s going well? What are our concerns?
Moon and daylight graphing activities
Downloading data to Graphical Analysis software
SimCalc MathWorlds software refresher
Parachute activity (modeling motion using CBR, presented by co-leader Veronica
Carlson.)
GUHSD teacher teams were asked to plan two lessons together, observe their team member's
teaching, and return lesson planning/observation report forms to the district office.
Mesa:
On Nov. 23, Mesa held the first of two Saturday morning meetings with workshop leaders
Larry Dukerich and Stella Ollarsaba; ten teachers participated. Agenda:
* Midcourse correction: testimonials, difficulties
* Math and energy activities
* Expressions of desired future activities. Teachers proposed these: "Get together as a
math/science team and focus on a concept (e.g., proportional relationships) in which you can
discuss the vocabulary, topic and ways that each subject matter can augment the other."
"Develop parallel worksheets from the curriculum that can be used in science/math."
Representative comments at this meeting by teachers were compiled by Larry Dukerich.
* "The workshop brought about a change in my teaching style. This is shown in the increase in
my students’ enthusiasm for science. They hardly ever work alone now. They are much more
engaged. My students’ test scores also show improvement."
* "Big whiteboards really promote interaction between the students. While one is writing, the
others are helping and checking." "While working in groups, the students are debating and
critiquing each other. Oral and written communication skills are being improved."
* A math teacher was so enthused that she and her principal made sure that every one of the math
teachers in her school had a classroom set of whiteboards.
* They all felt that they knew enough about whiteboarding that they could implement this aspect
of modeling in their classrooms.
* "The workshop experience changed the way I look at science in general. I find myself looking
for the models in the rest of the ninth grade science curriculum. It helps me realize the way
students learn so that I am a more effective teacher."
* Several teachers reported that they are “converting” colleagues who didn't participate.
* The level of the activities was too high for seventh grade students.
* Everyone expressed the concern that what was expected of them (either preparing for district
tests or conforming to the existing curriculum) hindered their ability to implement as much of the
materials as they would have liked.
* Several teachers commented that it would have been much better if a team of math and science
teachers from their school had participated in the workshop.
Peoria:
Peoria funded 10 hours follow-up at $22/hour, including an eSchool using Blackboard.com
and a four-hour Saturday meeting in January with workshop leaders David Hill and Curtis
9
Aylward. (David Hill is the science facilitator for Peoria USD, in addition to teaching physics
full time.) David reported that the in-person meeting, which was attended by seven participants,
was more successful than eSchool, in part because eSchool is a new modality for teachers, so it
was hard to get them to use it to ask questions of the instructors and to share ideas. The agenda
for the meeting included issues of teachers, using technology, and changes in activities and
sharing of experiences. Teachers expressed frustration at the district's slowness in installing
Graphical Analysis on networked computers, and science teachers were frustrated at not being
given commitment for time to do science activities because of focus on the math AIMS test. On
the other hand, math teachers found creative ways to use Modeling Workshop activities to teach
Arizona math standards, not by rote, but by science applications.
David Hill noted that the chief value of the Modeling Workshop was in process skills, rather
than content. Teachers reported to him that their classroom has become more student-centered.
Rather than students just doing a lab, filling out a worksheet, and turning it in, they engage in
whiteboarding and Socratic dialogue -- teachers ask students more questions. Rather than the
teacher giving a definition and doing a demonstration, followed by student verification, children
develop concepts through experimentation and discourse/reflection. (This is not quite model
development, he noted.) He said that it was hard for 7th and 8th grade science teachers to directly
implement most Modeling Workshop activities because 7th grade is earth science and 8th grade is
life science. Sixth and 9th grade are physical science, so these teachers had no problem, he said.
David recommended that, since fewer than half of the teachers were able to attend the wellpublicized Saturday meeting, early-release time or two after-school meetings would be better.
WESD:
In early August, WESD had two full days of follow-up with workshop leaders Sheila
Ringhiser and Veronica Carlson to integrate workshop learning into their curriculum. Teachers
wanted more follow-up sessions to strengthen their skills, but in mid-year the curriculum
coordinator, Jennifer Cruz, took another job in the district. Thus follow-up sessions did not
occur, to the disappointment of some teachers.
Listserv for participants:
I started a listserv for all 92 participants, workshop leaders, and district coordinators. Hardly
anyone ever posted. I posted occasionally on resources: how to borrow classroom sets of TI-83s
from Texas Instruments, how to write a $500 grant to the Wells Fargo Teacher Partner program,
recent research on 'not giving the answer' to middle school students, exemplary middle school
science programs, error-laden middle school science textbooks, and the like.
Implementation: unsolicited comments by teachers (via e-mail):
 "I introduced whiteboarding and the TI-83 into my classroom for the first time this year. I
cannot express strongly enough what a positive impact this has had on my classroom! Wow!
My students love sharing their work through whiteboarding and it has been an effective
method for me to encourage more risk-free participation in cooperative groups. In fact, I
keep spreading the word about how well it works and have even got my best friend
whiteboarding in her English class over at Metro Tech! In addition, the TI-83 has completely
captured my students' imagination."
10

(Received in May 2003) "I would have sworn that I didn't use much of the material I learned
last summer, but then I watched my kids take that test. I had many copies so I let the kids
work on the test[sheets]. They were on task during the test (I was in shock!) and then when I
looked over the tests, I realized that each child had attempted every problem and that they
had a method of approach for each problem. They extended graphs, drew pictures and wrote
notes to themselves. I was floored:)" [Note: this experienced mathematics teacher reported
little implementation but had a great deal of enthusiasm for the workshop. Her students
showed little gain on the Math Concepts Inventory.]
Measurement of student learning, by school district:
I preface this section by quoting Larry Dukerich (e-mail to me, June 2003):
"I am hesitant to draw far-reaching conclusions about the merits of individual questions (or
categories of questions) based on the scores of some teachers in Mesa who frankly admitted that
they were unable to implement the curriculum in any meaningful way. I cannot tell you how
dismayed I was to learn (in our follow-up sessions) that the teachers could only do this or that
activity here or there because they felt compelled to cover all the topics required of them in order
to prepare their students for the end-of-year tests. Having spent a significant chunk of a year
devising a set of activities designed to carefully build skills and a coherent view of the atomic
model of matter and energy, I felt as if it had been a real waste of time. Some of these teachers
may as well be a control group!
So, I would do my best to compare gains to the teachers' self-rating of the degree to which
they were able to implement the curriculum they learned - just as we did with the Modeling
Instruction in High School Physics workshops. It seems reasonable to me that one would only
find gains for students in classes in which teachers were able to implement the curriculum in a
coherent way."
The highest gaining district for grades 7, 8, and 9 was Peoria (eight percentage points
increase: for brevity, we call it an 8% gain in the PSCI). When disaggregated by grade, eighth
graders had this same PSCI gain.
A comparable gain was 9th grade math in Mesa (seven percentage points on the MCI, which
we call a 7% gain); most students tested were in alternative algebra, a two-period course for low
achievers (representative of the group most in need of improved instruction!).
Specifically, actual scores and gains in scores for the 135 9th grade Mesa students who
took the Math Concepts Inventory pretest and posttest are:
subcategory:
overall
Conservation of weight, volume
Proportional reasoning
graphing skills and motion
geometrical
and
physical
properties of matter
pretest
44%
57%
8%
61%
35%
posttest
51%
65%
15%
70%
41%
Percentage points gain
7%
8%
7%
9%
6%
I consider the increase of seven or eight percentage points to be roughly commensurate
with the typical ten percentage points increase in Force Concept Inventory scores of tens of
11
thousands of physics students in the first year of their teacher’s implementation of Modeling
Instruction. The Force Concept Inventory is simply a better test than the PSCI and MCI.
Modeling Instruction is a complex innovation, and my long-term study (unpublished as of 2008)
of physics teachers shows that student posttest mean scores continue to rise for three or four
years after the Modeling Workshop. Student gains are higher if the teacher takes a second
modeling workshop. Teachers report that the first year of implementation is hard; they feel
comfortable using Modeling Instruction only after three or four years. Modeling Instruction
is transformative, many teachers report. Satisfaction is high, and some write that it saved their
careers.
Appendix B (available by e-mailing Jane.Jackson@asu.edu) is a spreadsheet of grade 7,
8, and 9 student pretest and posttest mean scores for the PSCI and MCI in 2002-03, overall and
disaggregated by district and grade. All data are matched, pre- and posttest. Included are the
number of students in the sample, i.e., the number who took both tests (pre- and posttest). As of
2008, we have analyzed data by gender and race/ethnicity.
Teachers' scores in the atomic nature of matter subcategory were low and showed no
improvement, so we did not analyze them for students.
When looking at these student data, it is important to realize:
1) Some test questions needed revision or replacement to better reflect the actual content and/or
process skills of the Modeling Workshop. (This was done in June 2003.)
2) Workshop leaders expressed concerns like this to me, "Science teachers were frustrated at not
being given commitment for time to do science activities because of focus on the math AIMS test.
On the other hand, math teachers found creative ways to use Modeling Workshop activities to
teach Arizona math standards, not by rote, but by science applications."
3) Most teachers could not implement some content. They expressed three reasons for this. First,
in some districts, science in a certain grade was mostly life science, or earth science; and in other
districts the physical science content was specified to be other than what was taught in the
workshop (for example, Newton's laws in GESD, but not energy - a bad idea, because energy is
more fundamental to all sciences than forces). Second, workshop leaders reported that ALL
teachers "expressed the concern that what was expected of them (either preparing for district tests
or conforming to the existing curriculum) hindered their ability to implement as much of the
materials as they would have liked." Third: in Mesa, 7th and 8th grades had only one-half year of
science, and they had to race through an extensive list of topics.
4) Surveys returned in spring 2003 by teachers reveal great disparities in implementation of
modeling methodology.
5) Since grade nine is included on the spreadsheet, Mesa Public Schools, GUHSD, and Peoria
USD are expected to have higher scores than GESD or WESD, although most 9th grade Mesa
results are for alternative algebra for low-achievers.
The weakest subcategories of inventories that were analyzed are:
1) Proportional reasoning: only the easier pair of questions, #5 and 6, were analyzed; these
scores are typically under 10%. (Answers for the pair of #7 and 8 were random; most students
guessed.)
These scores are the same as those for 104 8th graders in suburban east Mesa in January1982,
as reported by Prof. Anton Lawson of Arizona State University (Lawson, Anton and Bealer,
12
Jonathan, "Cultural Diversity and Differences in Formal Reasoning Ability", Journal of Research
in Science Teaching 21, p.735-743, 1984). In that study, students were told that a given quantity
of water occupies four units in a wide cylindrical container and six units when poured into a
narrow one. They were asked to predict how high a given quantity of water that occupies six
units in the wide container would rise if poured into the narrow container, and state their reason.
(This is the essence of questions #5 and 6. Anton Lawson adapted his test to multiple choice
around 2000.)
Eighth graders in Berkeley, California scored twice as high on this task in spring 1978: 104
students averaged about 20% (reported by Anton Lawson in the article cited).
Sixth grade students tested in 2001-2002 by Dr. Kalyani Raghavan for her Model Assisted
Reasoning in Science (MARS) Program at the University of Pittsburgh scored about the same on
these two questions: 12% on the pretest, 14% on the posttest for both the reform group and the
control group (670 students in MARS, 297 in the control group) (private communication).
2) Energy and states of matter (actual scores are typically under 10%). Four research-based
conceptual questions about energy were included, one of which is the following:
As water in an ice cube tray freezes,
a. it absorbs energy from its surroundings.
b. its surroundings absorb energy from it.
c. it absorbs coldness from and releases energy to its surroundings.
d. it only absorbs the coldness from its surroundings.
e. it neither absorbs nor releases energy, because its temperature stays constant.
Students don't begin to understand energy. As with proportional reasoning, scores are below the
random score of 20%, indicating powerful alternative conceptions. Incidentally, the five choices
are expressed in the language of children who were interviewed. Middle school textbooks do a
poor job of treating energy. The Modeling Instruction Program improves instruction on energy,
the most fundamental concept in science. An enlightening article, “Making Work Work,” can be
downloaded at http://modeling.asu.edu/modeling-HS.html, and several other resources on energy
are posted on the modeling website, including a Basic Energy Concept Inventory for grades 8 to
12.
Response rates:
Of the 87 teachers in the five school districts, 61 gave the PSCI or MCI pretest to students.
Thirty-two submitted posttest answer sheets as well (18 in math, 15 in science, one in both math
and science). Response rates for both inventories were: GESD: 7 out of 11 teachers, GUHSD:
2/22, Mesa: 10/23, Peoria: 8/20, WESD: 5/11. GUHSD's response rate was low because many
math students change teachers quarterly, so it was difficult to give the posttest to the same
students. In addition, GUHSD has a week-long performance based assessment near the end of the
year in all subjects, plus the AIMS test, the Stanford 9, and one more test. That is too much
testing, leaving little time for teaching and learning, teachers say.
Correlation of gains of individual teachers' students with self-rating of degree of implementation:
No teachers reported that they implemented all content and all components of Modeling
Instruction consistently, and no teachers rated their implementation of the modeling cycle as very
13
good. Thus I could not discern strong correlations, unlike for physics. I present a few case
studies.
The two science teachers who had the highest student gains are a 9th grade (physical science)
and a 7th grade (earth science) in Peoria. They rated their overall implementation of the modeling
cycle as good or fair. As markers of implementation, both said that they regularly ask students to
work in groups in class, regularly ask different groups to discuss their ideas in class (a large
increase from the previous year), sometimes or frequently use hand-held whiteboards (a large
increase from the previous year), sometimes lecture, and seldom or never use a textbook. The 9th
grade teacher's PSCI score increased by 16 percentage points, from 74% to 90%. So that
teacher's knowledge improved, she implemented most components of Modeling Instruction
consistently, and her students achieved considerably.
A Mesa 9th grade algebra teacher with over 2 decades of experience said that the Modeling
Workshop enhanced her teaching maximally (i.e., 5, on a scale of 1 to 5). She implemented
Socratic questioning strategies regularly, used whiteboards frequently, had in-depth group work
sometimes, and focused student thinking on models equally with topics. Her implementation of
the math part (graphing, data gathering, and motion content) was 4, on a scale of 1 to 5. Her
student gains were moderate.
A brand-new Mesa 7th grade science teacher self-reported nearly as high a degree of
enhancement of his teaching (4), and a similar degree of implementation of methodology, except
that he used Socratic questioning only sometimes. His student gains were small; but 7th grade
science in Mesa is life science, not physical science, and it's only half a year.
Two GESD teachers who team-teach math and science in 7th and 8th grade reported fair
overall implementation of the modeling cycle. Their use of whiteboards increased from never to
regularly, and they frequently used the modeling cycle. Their 96 students had moderate gains on
the MCI.
Summary of self-reported trends among teachers in all districts this year:
 Large increase in use of whiteboards
 Some increase in student discourse (Socratic questioning)
 Less lecturing
 Less use of a standard textbook
 The workshop moderately enhanced their teaching (3 on a scale of 1 to 5)
 Little coordination of courses with colleagues
 No improvement in classroom technology. Typically teachers have one to three computers in
their classroom. Some have access to a computer lab (often with difficulty in scheduling).
Most teachers don't have graphing calculators, with the exception of GUHSD (and they don't
have TI-83+, but rather older models for which SimCalc can't be used).
I was able to provide StRUT-donated computers for a science teacher in Cholla Middle
School in WESD, where six teachers participated, and I helped get TI-83s for Shepherd Junior
High School in Mesa, where four teachers participated. They wrote a grant to the Mesa
Educational Foundation. My $10,000 proposal to the Medtronic Foundation for TI-83s was not
funded.
14
The degree of encouragement from school administration to implement the workshop learning
as reported by teachers was vastly different in different schools. It tended to be either low or
high; not moderate. Most WESD teachers reported little or no support from their school or
district.
I was impressed with the support at district level in Mesa, GUHSD, GESD, and Peoria.
I perceive that one factor that can reduce support in the school of any teacher, is a lack of
math and science background of the administrator. Many such administrators do not understand
science or math, nor the needs of teachers of these subjects, nor reforms that work.
To gather additional important information and insights, I conducted half-hour phone
interviews with four teachers from four districts. The teachers report the following:
 groups of students especially well served by Modeling Instruction are lower ability students concrete learners; shy, bright kids, ELL (mentioned by three teachers: visual representations;
input from other kids; whiteboarding helps a lot with vocabulary!); gifted students (they can
use software without supervision); special needs students -- special ed (SPED: graphing skills
increase their self-esteem), and girls (they enjoy science more).
 Obstacles to implementation are: modeling takes time -- block schedules are preferable to
traditional length classes; too much hassle to go to the computer lab to use MathWorlds or
Graphical Analysis software; Peoria school district refused to install SimCalc MathWorlds on
computers; lack of meter sticks and triple beam balances (in a GESD elementary school).
 One teacher used SimCalc MathWorlds software for two periods, and she'll use it for a week
next year in alternative algebra.
 Desires for future professional development are: review strategies in modeling, questioning,
and cooperative groups; review use of TI-83+ graphing calculator; review SimCalc
MathWorlds software; how to use modeling instruction in reading, social studies, and civics;
take three days in summer to develop lessons for one's classes and share them; specific
modeling strategies in mathematics; strategies for low-level math students.
 All said that it would be very valuable if they had an hour scheduled every other week to
coordinate their math and science courses with colleagues in their school.
 Technology desires: two would love to have a classroom set of TI-83+SE graphing
calculators (one said that calculators are preferable to computers because graphing calculators
force students to work, whereas students can be lazy if they're in a group that's working on a
computer); the other two would love nine computers in their classroom.
Long-term outlook:
Larry Dukerich wrote this to Rick Vanosdall, the science specialist in Mesa Public Schools,
during his workshop in June 2002:
" On the larger issue of meaningful change, I find it hard to be optimistic in the face of the
obstacles we face. The history of the fragmented approach we have been using in the Mesa
junior high program has shaped the way teachers view how instruction should take place. Most
see this workshop as a source of good activities that they could plug in here or there to
supplement or bolster what they already do. I had this vivid image of garage sale junkies
picking through the pile of dishware for a piece here and there, not recognizing that a whole set
was in front of them. It was with considerable difficulty that I managed to keep the look of
incredulity off my face as I listened to ____ (who has been a real pillar in this workshop) tell
15
me just how much she felt she was expected to do in 8th grade science. Force and motion,
energy, circulatory system, astronomy and ecology in 18 weeks???? In her defense, she is
gamely trying to "cover" the standards set forth for grades 6 - 8. It would require a major
paradigm shift for teachers to consider the possibility of using these materials as a complete
course. They universally express the view that they could be far more effective if they could
focus on just a few topics and do them well, but just don't see that in the cards. However,
expectations of performance on the current math test and future science test hangs like a sword
of Damocles over their heads. I expressed the view that perhaps we don't need to worry so
much about the tests; just focus on developing the skills, and the kids will do well enough. ..."
A second Modeling Workshop, identical in content to the 2002 one, was held in summer
2003 at Dobson High School, for 23 more Mesa participants. Apprentice leaders and mentors
were chosen from the 2002 participants to establish continuity and enhance the reform. Support
of principals was being enlisted by Rick Vanosdall and Sandra Nagy, the science and math
specialists; however, that process ended when Rick Vanosdall left the district in 2003. His
successor, a former teacher, did not have needed skills nor insight and commitment to the
reform. Thus the junior high reform paused. As of fall 2008, 10 of the 40 Mesa teachers are
gone. Another, Rosanne “Rosie” Magarelli, retired early due to injury, retrained by taking
another physical science modeling workshop in 2008, and is mentoring new modelers in
physical science and preparing to write model-based biology lessons for her community college
courses, BIO 100 and BIO 181, and helping Jane write grants to hold Modeling Workshops in
high school biology.
In GUHSD, Debi Plum later organized district-wide short Modeling Workshops for
math teachers that were well attended, she told me. Then she moved back into the classroom;
she now coordinates her math classes with the physics modeler in her school. The number of
GUHSD physics and chemistry teachers who use Modeling Instruction has grown through the
years.
The WESD district curriculum coordinator changed jobs, as did the GESD staff liaison.
Neither had a background in science or math, as I recall. With no district level commitment, the
reform died, at least collectively.
In Peoria, no further Modeling Workshops for middle school were held. The number of
physics and chemistry teachers who use Modeling Instruction has grown through the years, and
Gaylene Swenson, the successor to the prior science coordinator, actively supports district
physics and chemistry teachers via a professional learning community.
An enthusiastic and committed district level science or math coordinator can have a
powerful influence on junior high instruction (such as Rick Vanosdall and Debi Plum did), even
under adverse circumstances such as a low budget. But when the person changes jobs, their
work can easily die. Furthermore, knowledge of STEM at district level in elementary school
districts is hit or miss. Bottom line: school district infrastructure for STEM teaching/learning
reform is weak.
16