TPT Manuscript: MacIsaac & Falconer, Reform Teaching via RTOP p 1
A much-abbreviated version of this manuscript has been accepted for publication in The Physics
Teacher sometime in Fall 2002 (probably November). Please do not cite it until after it has been
published.
MANUSCRIPT FOR THE PHYSICS TEACHER
Reform Your Classroom Instruction via the Reformed Teaching Observation Protocol (RTOP)
Dan MacIsaac
Department of Physics & Astronomy, Northern Arizona University, Flagstaff AZ 86011-6010;
danmac@nau.edu, 520-523-5921 (voice) 520-523-1371 (FAX)
Kathleen Falconer
Department of Physics & Astronomy, Arizona State University, Tempe, AZ 85203-1504;
kathleen.falconer@asu.edu
Please direct correspondence regarding this manuscript to MacIsaac; authors' bios and photos are
attached as a separate document.
Abstract:
We report how physics teachers can use recent science education research developments to refine their own
teaching and improve their students’ conceptual achievement gains. The development of the Reformed
Teaching Observation Protocol (RTOP) has produced a valuable tool for reflecting upon and improving
physics teaching. Instructor RTOP scores have been found to strongly correlate with student conceptual
gains in introductory science and physics courses. We discuss the underpinnings of reformed teaching and
RTOP, how to use it to constructively evaluate and critique physics teaching and to guide personal teaching
improvement.
PACS Numbers: 01.40Ea,b; 0140Ga,b; 01.40H,J,K,R
Introduction: The call for reform
Professional associations of scientists, mathematicians, and educators -- for example the American
Association for the Advancement of Science (AAAS), the National Council of Teachers of Mathematics
(NCTM) and the National Research Council (NRC), have called for extensive reform in the teaching of
science and mathematics (REFS 1a,b,c,d,e,f,g,h,i,j). These reports critique US science and mathematics
curricula as largely incoherent, excessively repetitive and unfocused – ‘a mile wide and an inch deep’ (REF
1i, p3). International studies (REF 1j, p 35) have shown that US grade school science textbooks have many
times more topics than are typical in the rest of the world, and that American books are focused on
presenting simple information such as vocabulary, facts and simple equations, -- neglecting complex
information synthesis, analysis and relevant application, scientific reasoning, the use of scientific tools,
investigation and communication. There are also those who argue that the very culture of traditional
physics lectures (and science lectures in general) alienates a large number of students (REF 2a, 2b) and is
overdue for reform. Finally, Physics Education Research (PER) has found (REF 2c) that high school
physics students who studied without commercial textbooks and whose courses covered less material in
greater depth did better in their following college courses.
In 1995, the National Science Foundation funded a large five-year collaborative project called the Arizona
Collaborative for Excellence in the Preparation of Teachers (ACEPT) at Arizona State University (ASU)
(REF3a,b). The project goal was to better prepare K-12 teachers in science and mathematics. ASU faculty
collaborated with other Arizona university faculty working to reform the preparation of science and
mathematics teachers, K-20. Recognizing that most teachers teach as they were taught, ACEPT decided
to "break the cycle" by reforming the freshman science and mathematics classes taken by pre-service
teachers. Freshman science and mathematics courses would be reformed – that is, taught via the kinds of
D:\533568574.doc
2/17/2016
5:20 AM
TPT Manuscript: MacIsaac & Falconer, Reform Teaching via RTOP p 2
constructivist, inquiry-based methods advocated by the AAAS, NCTM and NRC so that these future
teachers would be taught as they were expected to teach.
In order to assess whether reformed teaching was occurring at the pre-service level (and later at the K-12
level when graduates entered the teaching field), the ACEPT evaluation team developed a classroom
observation instrument called the Reformed Teaching Observation Protocol (RTOP) that both measures
and operationally defines reformed teaching. Drawing upon existing observation instruments as well as the
literature surrounding reform, the RTOP was developed, fined, and validated over a period of two years. In
its present form, the RTOP is a highly reliable instrument with strong predictive validity (REF 4, 5). To
date, RTOP has been used in over 400 K-20 science and mathematics classrooms to provide a precise
quantitative reading of the degree to which teaching is reformed. RTOP both operationally defines and
assesses reformed teaching in the classroom – we henceforth explicitly reserve and define the term
reformed teaching to mean those classroom practices that result in a high RTOP score (REF 4).
Insert Figure 1 about here
In the evaluation of ACEPT, RTOP scores were found to strongly correlate with student conceptual gains
(REF 5, Figure 1) showing that reformed teaching is also effective teaching. Because the correlation
coefficients between RTOP and student achievement gains were so high (correlation coefficients in the
0.70 - 0.95 range were typical), it occurred to us that the items in the instrument might provide teachers as
well as researchers with a window into understanding reformed teaching. Could we use RTOP to help
teachers develop a deeper understanding of the nature of reformed teaching by observing themselves and
others using the RTOP? We started to use RTOP training sessions as an explicit science methods activity
for pre-service teacher courses and in professional development workshops for inservice teachers. While
much time and effort has been poured into reform teaching, there has been a lack of research linking
teacher self-knowledge, reform teaching and student achievement (REF 6). A review of the research lead
us to believe that RTOP was appropriate for this purpose.
There are two notable sets of research upon classroom behavior that are linked to student achievement –
the physics education research of Richard Hake, and the science education research done on cooperative
learning. In 1998, physicist Richard Hake (REF 7) published a large scale study of over 6000 introductory
mechanics students. In his study, Hake examined student conceptual gains in a series of curricula he
characterized with the label ‘Interactive Engagement’:
Interactive Engagement (IE) methods… [are] …designed at least in part to promote
conceptual understanding through interactive engagement of students in heads-on
(always) and hands-on (usually) activities which yield immediate feedback through
discussion with peers and or instructors… (REF7, p 65; our italics)
Hake found IE strategies produced increases in student achievement well beyond those produced with
traditional methods. Hake indicated that the four most popular nontraditional behaviors in his IE courses
studies were: 1) Collaborative Peer Instruction (REF 8a,b,c,e) (present in every IE course he studied), 2)
Microcomputer-Based Labs (REF 8d), 3) ConcepTests (REF 8e) and 4) Modeling Instruction (REF 8f).
Hake also repeatedly identifies curricula and methods from PER as critical to the stimulation of IE teaching
methods. (REF7, p 65)
The universal role of collaborative peer activities reported by Hake in IE physics courses is itself
noteworthy.. A large body of education research (REF 8a) reports that collaborative learning increases
retention, on-task behavior, promotes achievement, positive attitudes and self-esteem and produces higher
student achievement. The next three IE instructional innovations identified above by Hake all incorporate
collaborative learning. The RTOP instrument is designed to constructively critique details of classroom
practices including cooperative learning, interactive engagement and certain classes of PER activities and
findings collectively known as Pedagogical Content Knowledge.
D:\533568574.doc
2/17/2016
5:20 AM
TPT Manuscript: MacIsaac & Falconer, Reform Teaching via RTOP p 3
What do Teachers get from using RTOP?
RTOP is valuable because it can be used by both new and veteran teachers to not only score their own
teaching, but more importantly to acquire insight into their own teaching practices that guides their own
instructional improvement and professional teaching development. Teachers using RTOP must work with
a respected and trusted peer. RTOP scoring and initial reflection takes about ninety minutes, including one
hour spent observing the lesson scored.
RTOP specifies a set of 25 scored, observable classroom behaviors or items. Items catalyze teacher
development when each is used as a focus for reflection, discussion and debate upon observed teaching.
Constructively critical discussion and debate upon what these RTOP characteristics mean and how they
manifest in actual classroom activity underlies the development of a common language of reformed
teaching grounded in personal experiences. We consider the development of a common language
describing reformed teaching to be the most fruitful outcome of RTOP use by teachers – teachers are
unfamiliar with reformed teaching, and all meaningful learning requires the development and refinement of
precise conceptual language (REF9). RTOP items address behaviors that lie at the heart of learning science
and mathematics in the classroom, unlike broader instructional rubrics such as Madeline Hunters' Elements
of Effective Instruction (EEI - REF10). In some cases, these other classroom rubrics are incompatible with
inquiry science learning – e.g., Hunter's rubrics for direct instruction in the classroom are centered upon
teacher-directed behaviors such as "anticipatory sets" and "closure".
Teachers working with us have found that the RTOP is useful as a checklist for lesson planning purposes,
in the mentoring and professional development of new or student teachers and for their own personal
pedagogical growth. Another commonly cited use is in defense of instructional reform -- for example
justifying the modeling method to administrators and parents who may be familiar with traditional
instructional methods and require assistance in judging reformed teaching. Many aspects of inquiry
teaching challenge traditional practice and teachers tell us that RTOP helps validate and refocus their own
journey into reform.
Getting your own RTOP score via the Instrument
To obtain an RTOP score of one of your own lessons, 1) download the RTOP Training Manual (REF 4)
and print a copy for yourself and a teaching colleague whom you trust and respect, ideally familiar with
teaching your subject. 2) You and a colleague should read and discuss the instrument, then 3) arrange for
your colleague to visit your class to observe and RTOP an hour lesson. 4) While your colleague observes
your class, have a student or aide videotape your lesson. 5) RTOP this videotape yourself, before
discussing your colleague's RTOP score of your lesson. 6) Reciprocate -- perform an RTOP observation on
your colleague in turn. This will provide more needed classroom observation material for discussion and
genuine meaning in this experience for both of you. 7) Meet with your colleague to discuss and attempt to
reconcile the scores on each of the twenty-five items. Inevitably, you will disagree with your colleague.
Use the differences as a focus for re-examining your own teaching practices.
D:\533568574.doc
2/17/2016
5:20 AM
TPT Manuscript: MacIsaac & Falconer, Reform Teaching via RTOP p 4
The RTOP instrument items are divided into five sections: (1) lesson design and implementation
information, (2) propositional content knowledge, ( 3) procedural content knowledge, (4) classroom culture
(communicative interactions) and (5) classroom culture (student-teacher relationships). These five sections
include twenty-five observable items scored from 0 - 4 as follows:
0
1
2
3
4
the behavior never occurred
the behavior occurred at least once
occurred more than once; very loosely describes the lesson
a frequent behavior or fairly descriptive of the lesson
pervasive or extremely descriptive of the lesson
-- where the exact details of the intermediate scores differ for each of the twenty-five items and have been
rigorously defined by researchers. For your use, when in doubt scoring err on the side of a lower score. If
you feel uncomfortable with a five step gradation, try assigning only scores of 0, 2 or 4 (absent, sometimes
present, always) for an item. If you didn’t directly observe an item it scores as zero (do not make any
inferences without training).
When we conduct RTOP teacher workshops scoring video vignettes, teachers usually score lessons
artificially high on their first attempts; after more discussion and more video vignettes the scores typically
fall quite dramatically, indicating that teachers rapidly become more discerning. The scores you and your
colleague generate will not be as accurate as that of a trained observer and thus will not be useful for formal
research. However, your informal scores and discussion will generate comments, insights and ideas that
you can act upon for your own teaching growth, and this experience will enable you to discuss reform
teaching with colleagues.
Summing the twenty-five item scores results in an RTOP lesson score ranging from 0 – 100 describing the
degree of reform present. For physics lessons we have observed, some typical scores are:
traditional university lecture (passive)
university lecture with demonstrations (some student participation)
traditional high school physics lecture (with student questions)
partial HS reform (some group work; most discourse still with teacher)
medium sized (100 > n > 50) university lectures with Mazur-like groupwork (ConcepTests) and a student Personal Response System
the author’s modified (whiteboards etc) large (170 > n > 75) lectures
modeling curriculum (varies with amount and quality of discourse)
< 20
< 30
< 45
< 55
65-75
70-75
65-99
These totals are generalized and approximate, and large departures have been observed. Any RTOP score
greater than 50 indicates considerable presence of "reformed teaching" in a lesson.
The twenty-five items in the RTOP can be briefly summarized for physics teaching as follows:
Lesson Design and Implementation. The creation of physics lessons that: 1) respect student
preconceptions and knowledge; 2) foster learning communities; 3) explore before formal presentation;
4) seek and recognize alternative approaches; and 5) include student ideas in classroom direction.
Content (Propositional Knowledge). Teachers knowing their physics and teaching lessons that: 6)
involve fundamental concepts of physics; 7) promote coherent understanding across topics and
situations; 8) demonstrate teacher content knowledge (e.g. apparently "unrelated" questions); 9)
encourage appropriate abstraction; and 10) explore and value interdisciplinary contexts and real world
phenomena.
D:\533568574.doc
2/17/2016
5:20 AM
TPT Manuscript: MacIsaac & Falconer, Reform Teaching via RTOP p 5
Content (Procedural Knowledge). Physics lessons that use scientific reasoning and
teachers' understanding of pedagogy to: 11) use a variety of representations to represent
phenomena; 12) make and test predictions, hypotheses, estimates or conjectures; 13) are actively
engaging and thought-provoking and include critical assessment; 14) demonstrate metacognition
(critical self-reflection); and 15) show intellectual dialogue, challenge, debate negotiation,
interpretation and discourse.
Classroom Culture (Communicative Interactions). The use of student discourse to modify the locus
of lesson control such that: 16) students communicate their own ideas in a variety of methods; 17)
teachers' questions foster divergent modes of thinking; 18) lots of student, particularly inter-student
talk, is present; 19) student questions and comments shape discourse -- the "teachable moment" is
pursued; and 20) there is a climate of respect and expectation for student contributions.
Classroom Culture (Student-Teacher Relationships). Lessons interactions where: 21) students
actively participate (minds-on, hands-on) and set agendas; 22) students take primary and active
responsibility for their own learning; 23) the teacher is patient (plays out student initiatives, and is
silent when appropriate); 24) the teacher acts as a resource and students supply initiative; and 25) the
teacher is a listener.
Two sample RTOP item scores are described below from physics classes we have observed:
12) Students made predictions, estimations and / or hypotheses and devised means for testing them.
This item does not distinguish among predictions, hypotheses and estimations. All three terms are used so
that the RTOP can be descriptive of both mathematical thinking and scientific reasoning. Another word
that might be used in this context is “conjectures”. The idea is that students explicitly state what they think
is going to happen before collecting data.
Sample ratings for a lesson on 1-D non-uniform motion with students looking at position and velocity vs.
time graphs.
Score
Teacher Action
0
Teacher gives the students the formulas for non-uniform motion and the students solve problems
based on the graphs and problem sheets.
Teacher gives the students the theory and formulas for non-uniform motion, has the students
collect, analyze and interpret data using a step-by-step process.
Teacher sets up an appropriate experiment, has the students make predictions about what was
going to happen, the students do the experiment and analyze the data following the set procedure.
Teacher sets up an appropriate experiment, has the students make predictions about what was
going to happen, the students do the experiment with some procedural help and then analyze the
data to try and understand what relationship they see.
Teacher asks the students to look at the graphs and asks them to predict what type of motion gives
what graph. Teacher asks students to design and carry out experiments, which would develop the
relationships for non-linear motion. The teacher facilitates the experiment as required.
1
2
3
4
This item reflects student exploration of the complex social activity known as scientific inquiry. The world
is understandable through science, and scientific knowledge explains and predicts the world, is revisable,
testable, observable, does not rest upon appeal to authority (REF 60a, p13). For the teacher, this item
indicts the need for intensive teacher preparation upon the 'touchstone' concepts and situations identified
and described by physics education research (Ref 60b, p244-246).
D:\533568574.doc
2/17/2016
5:20 AM
TPT Manuscript: MacIsaac & Falconer, Reform Teaching via RTOP p 6
23) In general the teacher was patient with students.
Patience is not the same thing as tolerating unexpected or unwanted student behavior. Rather there is an
anticipation that, when given a chance to play itself out, unanticipated behavior can lead to rich learning
opportunities. A long “wait time” is a necessary but not sufficient condition for rating highly on this item.
Sample ratings for an introductory lesson on rotational kinematics when a student asks
"So how does this relate to the movement of the planets?"
Score
Teacher Action
0
1
2
Teacher informs the student –"Don't go there, that's for a later lesson" or ignores the student.
Teacher gives a short answer to the student's question.
Teacher turns the question back to the rest of the class, and awaits students' responses. After a few
responses, the teacher gives correct response incorporating some of the students' correct ideas.
Teacher turns the question back to the rest of the class. Gauging that students are interested, the
teacher asks students to get into their groups and discuss the question.
Teacher turns the question back to the rest of the class. Gauging that students are interested, the
teacher shows the students data for the earth's orbit around the sun and asks students to get into
their groups to try to figure out "How far does the earth travel in a year?" and "What is the earth's
angular velocity?"
3
4
There is an internal tension between patience and the teachers' pedagogical knowledge. The teacher may be
tempted to circumvent the inquiry process by not allowing the student enough time to explore their ideas
because the teacher knows where most students will have problems. However, having the students in
control of their learning does not imply that the teacher abdicates responsibility for the classroom and
learning. Rather, the teacher has to set up the appropriate problems, and know how and when to facilitate
student inquiries so that students reach a goal of common understanding. An expert physics teacher readily
identifies and aggressively pursues the "teachable moment" in a lesson, whether or not it is the scheduled
one. Although teacher expertise is required to score well on this item, the critical idea is to permit students
the time to explore apparently incorrect ideas, to wrestle with the language and to negotiate with peers.
Inquiry values the student's right to be explore and negotiate in a supportive environment. It is
extraordinarily difficult for teachers to shut up and allow students this freedom, yet teaching via student
dialog is a critical lesson for teachers to learn (Ref 60c, p493-494).
Lessons learned from RTOP
The twenty-five RTOP items lay out a set of nontraditional themes for physics lessons, which in turn
suggest particular opportunities for reforming physics teaching. We feel two of these themes are the most
worthwhile kinds of professional challenges that physics teachers should undertake.
First, and most important, these are NOT your father’s physics lectures anymore -- RTOP requires a
radically new kind of teaching with a radically new role for the teacher. This is a complete change from the
traditional culture of physics lectures (Ref 3b, 8d)
Reformed teaching looks quite different from traditional physics lessons; the biggest single change is that
the class is no longer focused on the teacher. The classroom is quite noisy, and the instructor works as a
group facilitator, which is a quite different skill than lecturing (REF 13a). Reformed classroom
management is quite different: considerable time must be found for student talk, usually by sharply
reducing course topical breadth and by largely eliminating lecture. The textbook is de-emphasized (or
abandoned), as there is insufficient time to present it in class. Rather than introducing content via lecture,
exploratory activities are organized and carried out. It is very hard for teachers to allow students to explore
and be wrong or incomplete for what seems to be excruciatingly protracted periods of time.
D:\533568574.doc
2/17/2016
5:20 AM
TPT Manuscript: MacIsaac & Falconer, Reform Teaching via RTOP p 7
Student talk is far more important than teacher talk -- high RTOP scores require cooperative student
learning through extended dialog. Teachers choose activities that foster such dialog, and manage, support
and reward dialog. These activities are carefully chosen to fit within time constraints, to be essential to the
concept and to be sufficiently challenging that collaboration is necessary. Students discuss, negotiate,
reflect upon and evaluate one another’s words and ideas in small groups. Students take time to negotiate
meaning, and teachers respect the student’s right to pursue blind alleys. In large classes, the teacher will
not be immediately available to help groups and groups must be prepared with self-help strategies. Groups
must exchange and negotiate amongst one another as well as within the group. Lectures are reshaped into
classroom learning communities, focused on group learning and student dialog.
Group participation and products are graded, though grading for correctness often is deferred to exams or
homework. Reform lesson materials and activities management is considerably more difficult than
traditional lecture, and some students will be highly resistant to taking on the additional work and
responsibility reformed teaching requires of them. Thankfully, there are benefits – greater conceptual
learning gains, greater participation and success for traditionally less successful physics students and
intrinsic motivation for learning physics.
Second, there is no "golden road" to physics teaching -- RTOP affirms the importance of specialized
preparation, knowledge and professional development for physics teachers. Specialized physics teaching
knowledge includes skills and content knowledge not required of physicists or of teachers of subjects other
than physics. This is called pedagogical content knowledge (PCK) in general science education literature
(REF 11).
Physics PCK includes those touchstone situations, activities and problems identified by PER as having
strong impacts upon physics learning. Physics PCK includes student preconceptions research (such as
students’ confuting position and velocity based on automobile riding experiences), topical emphasis issues
(e.g. – which kinematics concepts are critical to supporting Newton's Laws), and appropriate use of physics
examples and analogies (e.g. assimilating electrostatics by developing simpler ideas underlying gravitation
into generalized fields). Physics PCK is developed through specialized training and experience in physics
teaching, and is extended through professional development such as physics curricular workshops,
professional physics teaching journals, associations and meetings, and books about physics teaching (REF
12).
More general science education PCK includes knowledge of inquiry teaching and assessment, the nature of
science, and how to foster and support classroom dialogue so as to take advantage of those ‘teachable
moments’ you identify using physics PCK. Physics teachers require a thorough knowledge of course
content so as to be able to shift the lesson content in line with student thinking, often resulting in very
nontraditional sequences. Such shifts require expertise in identifying and underscoring real-world
examples as they spontaneously arise.
There are many teaching techniques and curricula that that foster the kinds of lessons that score well on the
RTOP rubric, and not all lessons can score high on RTOP. One technique that we strongly encourage
adopting as part of RTOP self evaluation is whiteboard use. Whiteboards were developed for use in
Hestenes' Modeling Physics Curriculum (Ref 8f), and whiteboards facilitate cooperative group learning by
anchoring student dialog in a shared, negotiated and explicit external representation. It is also possible to
foster dialog through the use of cooperative techniques such as think-pair-share, Mazur’s ConcepTests with
group reporting via personal response systems (REF 8e), or by cooperative group completion of touchstone
PER identified problems.
Like all worthwhile endeavors, reform teaching is a challenging, often difficult and richly rewarding
adventure. We encourage you to try this approach for your professional growth as a teacher and for the
conceptual growth of your students. You'll also learn lots of physics from listening to and reflecting on the
ideas of your students.
D:\533568574.doc
2/17/2016
5:20 AM
TPT Manuscript: MacIsaac & Falconer, Reform Teaching via RTOP p 8
Acknowledgements:
The preparation of this manuscript has been supported by the Arizona Teacher’s Excellence Coalition
(AzTEC) project funded by the US Dept of Education Teacher Partnership Grants program, and by the
Arizona Collaborative for Excellence in Preparing Teachers (ACEPT) funded through the NSF
Collaboratives for Excellence in Teacher Preparation program. We would like to thank Professors Dwain
Desbien, Julie Gess-Newsome, Richard Hake, David Hestenes, and Daiyo Sawada for their discussion and
commentary.
D:\533568574.doc
2/17/2016
5:20 AM
TPT Manuscript: MacIsaac & Falconer, Reform Teaching via RTOP p 9
References (I know, I used APA Sci Educ reference style but I will translate to APS)
1a. American Association for the Advancement of Science (AAAS) (1989). Project 2061: Science for All
Americans: A Project 2061 Report on Literacy Goals in Science, Mathematics, and Technology.
Washington, D.C.: AAAS. <http://www.project2061.org/tools/sfaaol/sfaatoc.htm>,
1b. AAAS (1993). Project 2061: Benchmarks for Science Literacy. Washington, D.C.:
AAAS.<http://www.project2061.org/tools/benchol/bolframe.htm>
1c. National Council Teachers of Mathematics (NCTM) (1989). Curriculum and Evaluation Standards for
School Mathematics. Reston, VA: NCTM. <http://standards.nctm.org/>
1d. NCTM (1991). Professional Standards for Teaching Mathematics. Reston, VA: NCTM.
1e. NCTM (1995). Assessment Standards for School Mathematics. Reston, VA: NCTM
<http://www.nctm.org/standards/buyonline.htm>.
1f. NCTM (2000). Principles and Standards for School Mathematics. Reston, VA: NCTM. <
1g. National Research Council (NRC) (1996). National Science Education Standards. Washington, D.C.:
National Academy Press <http://books.nap.edu/books/0309053269/html/>.
1h. NRC (2000). Inquiry and the National Science Education Standards. Washington, D.C.: National
Academy Press. <http://books.nap.edu/books/0309064767/html/>.
1i. NRC (1999). Designing mathematics or science curriculum programs: A guide for using mathematics
and science education standards. Washington, D.C.: National Academy Press.
<http://books.nap.edu/books/0309065275/html/>.
1j. NRC (1999). Global perspectives for local action: Using TIMSS to improve US Mathematics and
Science Education. Washington, D.C.: National Academy Press.
<http://books.nap.edu/books/0309065305/html/>.
2a. Seymour, E. (1996). Guest comment: Why undergraduates leave the sciences. American Journal of
Physics, 63, 199-202.
2b. Tobias, S. (1990). They’re not dumb, they’re different: Stalking the second tier. Tucson: The Research
Corporation.
2c. Sadler, P.M. & Tai, R.H. (2001). Success in introductory college physics: The role of high school
preparation. Science Education, 85(3), 111-137.
3a. ACEPT is described at <http://acept.asu.edu/> and the NSF CETP collaboratives maintain a continuing
centralized electronic archive at <http://ecept.net>. ACEPT goals have been largely supplanted by the
more recent and much larger AzTEC, see <http://purcell.phy.nau.edu/AZTEC/index.htm>, with most
ACEPT participants continuing in AzTEC.
3b. Wyckoff, S. (2001). Changing the culture of undergraduate science teaching. Journal of College
Science Teaching, XXX (6). Describes ACEPT, limited value of lecture in teaching physics,
interactive engagement.
4. Piburn, M., Sawada, D., Falconer, K., Turley, J. Benford, R., Bloom, I. (2000). Reformed Teaching
Observation Protocol (RTOP). ACEPT IN-003. The RTOP rubric form, training manual and reference
manual containing statistical analyses, (and eventually streamed video vignettes of physics teaching
practices) are all available from <http://purcell.phy.nau.edu/AZTEC/RTOP/>.
5. Lawson, A. E., Benford, R., Bloom, I., Carlson, M. P., Falconer, K. F., Hestenes, D. O., Judson, E.,
Piburn, M. D., Sawada, D., Turley, J., & Wyckoff, S. (2001). Reforming and evaluating college
science and mathematics instruction: Reformed teaching improves student achievement. Journal of
College Science Teaching, in press. Discusses links between RTOP scores and student achievement
gains for six physical science and four university physics classes, amongst many others.
6. Linn, R. L. (2000) Assessments and accountability, Educational Researcher; 29 (2), 4-16.
D:\533568574.doc
2/17/2016
5:20 AM
TPT Manuscript: MacIsaac & Falconer, Reform Teaching via RTOP p 10
7. Hake, R.R (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of
mechanics test data for introductory physics courses. American Journal of Physics, 66(1), 64-74. Also
Hake, R. (1998). Interactive-engagement methods in introductory mechanics courses, submitted to
the "Physics Education Research Supplement to AJP" (PERS), both available at
<http://www.physics.indiana.edu/~sdi/>.
8a. Johnson, D.W., Johnson, R.T. & Smith, K.A. (1991). Active learning: Cooperation in the college
classroom. : Edina, MN: Interaction Book Co. This includes a review of cooperative learning
literature, reporting several benefits including student achievement gains of 0.88 standard deviation
across many studies. CL is also described at <http://www.wcer.wisc.edu/nise/cl1/CL/default.asp>.
8b. Heller, P., Keith, R. & Anderson, S. (1992). Teaching problem solving through cooperative grouping,
Part 1: Group vs. individual problem solving. American Journal of Physics, 60, 627-636.
8c. Heller, P., & Hollabaugh, M., (1992). Teaching problem solving through cooperative grouping, Part 2:
Designing problems and structuring groups. American Journal of Physics, 60, 637-644.
8d. Laws, P. (1995). Workshop physics activity guide. NY: Wiley. The prototypical MBL curriculum,
workshop physics eschews lectures. See also Laws, P. (1991). Calculus-based physics without
lectures. Physics Today, 44(12) 24-31.
8e. Mazur, E. (1997). Peer instruction: A user’s manual. Prentice-Hall series in educational innovation. NJ:
Prentice-hall. See also <http://www.psrc-online.org/classrooms/papers/mazur.html> and Mazur’s
project Galileo website <http://galileo.harvard.edu/home.html>.
8f. Wells, M., Hestenes, D. & Swackhamer, G. (1995). A modeling method for high school physics
instruction. American Journal of Physics, 64, 114-119. Available at
<http://modeling.asu.edu/modeling/MalcolmMeth.html>. The Modeling Curriculum including
whiteboard activities is freely downloadable from <http://modeling.asu.edu>.
9. Vygotsky, L.S. (1997). (Original Revised and edited, A. Kozulin). Thought and language. Cambridge:
MIT.
10. Hunter, M. (1982). Mastery teaching: Increasing instructional effectiveness in secondary schools,
colleges and universities. El Segundo, CA: TIP Pubs. EEI is also described at
<http://www.humboldt.edu/~tha1/hunter-eei.html> and
<http://www2.bc.edu/~ruedaju/MadelineHunter.html>.
11. Barnett, J. & Hodson, D. (2001). Pedagogical context knowledge: Towards a fuller understanding of
what good science teachers know. Science Education 85(4), 426.
12. Arons, A.B. (1997). Teaching introductory physics. NY: Wiley.
13a. Whiteboard learning strategies are described at <http://purcell.phy.nau.edu/AZTEC/BP_WB/>.
Commercially manufactured whiteboards can be purchased from Playscapes, Inc (800) 248-7529 for
under $10 each or you can locally manufacture adequate versions for about $2 each. An overview of
whiteboard theory and some student reactions is available at <http://purcell.phy.nau.edu/pubs/CETP/>.
60a. Layman, J.W., Ochoa, G. & Heikkinen, H. (1996). Inquiry and learning: Realizing science standards
in the classroom. NY: The College Board.
60b. Trowbridge, D.E. and McDermott, L.C. (1981). Investigation of student understanding of the concept
of acceleration in one dimension. American Journal of Physics, 49, 242-253.
60c. Minstrell, J. (2000). Implications for teaching and learning inquiry: A summary. In Minstrell, J and
van Zee, E.H. (2000) (Eds). Inquiring into inquiry learning and teaching in science. Washington DC:
AAAS.
D:\533568574.doc
2/17/2016
5:20 AM
TPT Manuscript: MacIsaac & Falconer, Reform Teaching via RTOP p 11
Fig 1 (Review Quality):
Figure 1: ACEPT PHS 110 Fundamentals of Physical Science
Avg RTOP vs. Normalized Gain on Physics Concept Survey
RTOP Score/Normalized Gain (%)
100
90
80
70
60
50
Norm alized Gain
40
RTOP
30
20
10
0
Wkshp Partic 1
Wkshp Partic 2
Wkshp Partic 3
ASU
Control 1
RTOP scores are averaged across three observations by two or m ore expert observers (interrater reliability > 0.80).
The PCS contains a subset of the FCI. The correlation coefficient between Norm alized Gain and RTOP is 0.88.
These and sim ilar data published in REF 5.
Fig 2 (Review Quality):
Figure 2: Students using whiteboards in an introductory physics lesson
D:\533568574.doc
2/17/2016
5:20 AM
Control 2