Assessment of Preservice
Teachers’ Mathematics
Knowledge for Teaching the
Middle Grades
Margaret J. Mohr
Texas A&M University
February 23, 2006
Introduction
• In relation to education reform
recommendations, a survey of selected
teacher programs in the U.S. revealed few
changes in program characteristics (Graham,
Li, & Buck, 2000 ).
• At the middle school level, 68.5% of teachers
have no major certification in mathematics
and 21.9% do not have a minor in
mathematics (Seastrom, Gruber, Henke,
McGrath, & Cohen, 2005).
Introduction, cont’d.
• “To deliver the kind of mathematics content in ways
that respects middle grades students as learners
demands a well prepared and motivated teacher.
Few existing teacher preparation programs meet this
need, and certification requirements do not support
adequate content and pedagogical preparation”
(NRC, 2000a, p. 15).
• The improvement of middle school mathematics
teacher preparation must be grounded in research
that provides a theoretical understanding of what
prospective mathematics teachers for middle grades
need to learn, how they learn it, and how their
learning can be assessed (Kulm, Li, Allen, Goldsby, &
Willson, 2005).
Mathematics Knowledge for
Teaching
• RAND Mathematics Science Panel (2003)
report found a compelling relationship
between what a teacher can do with their
students and their own level of mathematics
competence.
• “either teachers do not have enough content
knowledge, or what they do know is not the
‘right’ content knowledge” (Sherin, 2002, p.
123)
Mathematics Knowledge for
Teaching
• NCTM (2000) emphasized teachers need different
kinds of knowledge
–
–
–
–
–
Knowledge of specific content
Curricular goals
The challenges students face in learning these ideas
Assessment
Pedagogical knowledge of effective teaching strategies
• “There is a positive connection between subject matter
preparation (in both content and specific teaching
methods) and teacher performance; however, for
some subjects, like mathematics, current subject
matter preparation (including an academic subject
major) may need to be reformed to increase reasoning
skills and conceptual knowledge” (ASCD, 2003, p. 1)
Mathematics Knowledge for
Teaching
• Shulman (1986) introduced “pedagogical
content knowledge”
– A conceived complementary relationship between
the pedagogical knowledge and the content
knowledge of the subject area
• Specific measures were not yet in place in
mathematics education (Hill, Schilling, & Ball,
2004)
– Set out to map out what elementary teachers knew
Mathematics Knowledge for
Teaching
• Hill, Schilling, and Ball (2004) found through
their multiple-choice assessment that teachers’
mathematics knowledge for teaching (PCK)
elementary grades was partly domain specific
rather than relating to their teaching or
mathematical ability
• In their study of first- and third-grade teachers,
they found that teachers’ mathematical
knowledge for teaching was significantly
related to student achievement gains in both
grade levels (Hill, Rowan, & Ball, 2005)
Mathematics Knowledge for
Teaching
• “By mathematical knowledge for teaching, we mean
the mathematical knowledge used to carry out the
work of teaching mathematics. Examples of this “work
of teaching” include explaining terms and concepts to
students, interpreting students’ statements and
solutions, judging and correcting textbook treatments
of particular topics, using representations accurately in
the classroom, and effects of teachers’ mathematical
knowledge on student achievement providing students
with examples of mathematical concepts, algorithms,
or proofs” (Hill, Rowan, & Ball, 2005).
• The implementation of this definition may very well be
the beginnings of the reform the ASCD (2003)
discusses.
Statement of the Problem
• Given the importance of mathematical content
knowledge and pedagogical content knowledge in
preparing preservice teachers, it is essential to know
the nature of this mathematics knowledge for
teaching
– Four main content strands: algebra, statistics, geometry, and
number and operations
• Help improve middle grades mathematics teacher
preparation programs
• Not only must the teacher understand the material
they are teaching, they must be able to communicate
it to the students as well.
Significance of Study
•
Previous research has provided insights into specific areas or parts
of teacher knowledge (i.e., Carter, 2005)
– No studies have been undertaken to collectively evaluate
mathematics knowledge for teaching at the middle grades levels
and across different cohorts.
•
The design of the study will allow for a snapshot of the
development of mathematics knowledge for teaching middle grades
during teacher preparation.
•
The current preservice middle grades mathematics certification
bachelor degree program at Texas A&M University represents a
model for teacher preparation recommended by professional
organizations (CBMS, 2001).
•
Research on this model will produce results which are intellectually
and scientifically sound; contribute to teacher preparation; and
have significant implications for current and future teacher
preparation programs.
Purpose of the Study
The purpose of this study is to:
• To develop an assessment instrument that
effectively evaluates and allows prospective
mathematics teachers to demonstrate their
mathematics knowledge for teaching
• Examine differences in mathematics
knowledge for teaching in four different
content strands across five different cohorts
of preservice middle grades mathematics
teachers at Texas A&M University
Goal of the Study
• The ultimate goal of this study is to
provide:
– An effective performance assessment
instrument,
– Data on preservice teachers’ mathematics
knowledge for teaching, and
– Recommendations that may be used to
help shape reform initiatives in teacher
education programs throughout the United
States
Research Questions
• What is preservice middle grades teachers’
mathematics knowledge for teaching number
and operations?
• What is preservice middle grades teachers’
mathematics knowledge for teaching algebra?
• What is preservice middle grades teachers’
mathematics knowledge for teaching
geometry?
• What is preservice teachers’ mathematics
knowledge for teaching statistics?
Ancillary Questions
• What is the effect of various sequencing of
mathematics courses on middle grades mathematics
teachers?
• What individual developmental differences are there
among prospective teachers as they progress
through the courses identified in the middle grades
mathematics program?
• Do some types of courses (e.g. algebra, geometry,
numerical, statistical or applied, theoretical) have
more impact than others upon the development of a
teacher’s mathematics knowledge for teaching
(MKT)?
• Does development of mathematics knowledge for
teaching happen at greater rates at certain stages of
the middle grades mathematics program than others?
Research Design
• Intent is to assess the nature of
mathematics knowledge for teaching
middle grades across five cohorts
(shown in future slide)
• Study will utilize a mixed-model design
Population
• Include all students who indicate their intent to be
teachers by enrolling in the first specialized
mathematics course at Texas A&M University
– MATH 365—Structure of Math I
• MATH 365, MATH 366, MATH 367, MATH 368,
MATH 403, MASC 351, MASC 450, MEFB 460, AND
MEFB 497
• 586 total students Spring 2005
• 590 total enrollment for Spring 2006
– Number will be less due to overlapping of students for some
courses
Spring ’05
Spring ‘06
MATH 365: Structure of Math I
127
MATH 366: Structure of Math II
152
124
140
MATH 367: Basics Concepts of
Geometry
?
18
MATH 368: Introduction to
Abstract Math
47
47
MATH 403: Math and Technology
64
44
MASC 351: Problem Solving
37
28
MASC 450: Integrated
Mathematics
40
56
4
Methods
Courses
MEFB 460: Methods of Teaching
Middle Grades Mathematics
66
36
5
Student
Teaching
MEFB 497: Teaching Middle
Grades
53
97
Cohort
1
Mathematics
Courses I
2
Mathematics
Courses II
3
Mathematics
Education
Courses
Course
TOTAL Participants:
590
Instrumentation
• Pilot tested
• Utilize ExamView Pro® Test Generator
– Test given on the computer via the internet
• Create a separate bank of questions for each content
strand
– Number and operation, geometry, statistics, and algebra
• Students randomly assigned a test from two different
test banks
– Even amount from each content area—7 questions
– Example: one student will take a test consisting of geometry
and statistics problems; another, algebra and geometry; etc.
Instrumentation, cont’d
Content will be different for each question;
all will have the following parts:
I.
Answer the above content knowledge
question and explain your solution of the
problem.
II.
How would you explain, model, and/or
demonstrate this item to someone who
did not understand?
Instrumentation, cont’d
Example Question:
The variables a, b, c, and d each represent a different
whole number. Given a = 3, use the properties of
whole numbers to determine the value for each
variable.
d+d=c
a∙d=a
c+d=a
c∙b=b
I.
Determine the value for each variable and explain
your solution of the problem.
II.
How would you explain, model, and/or
demonstrate this item to someone who did not
understand?
Instrumentation, cont’d
• Content questions from:
– Balanced Assessment (1999) series
– New York State Testing Program (1998)
Grade 8 Test Sampler Draft
– Learning from Assessment (Madfes & Muench,
1999)
• Rubrics from:
– New York State Published State Assessment
Rubrics and Exemplar Papers
– NCTM’s (2004) Mathematics Assessment: A
Practical Handbook for Grades 6-8
Procedures and Collection of
the Data
• Primary source of data will be the assessment
• Assessment will be a required part of the courses
involved
– Given right after mid-semester of their current courses
• Assessment delivered ExamView Pro® Software via
computer via internet
• Randomly will assign 14 questions from an already-
made test bank over two different content strands
– Number and operation, algebra, geometry, and/or statistics
Procedures and Collection of
the Data
• Question asked at beginning of test will
identify overlapping students
• If student has already taken test, will still take
it again, but the first time taken will be the
only data part of the study
• Also will be asked to provide the following
information: Math GPR, current courses
(identified from a given list), past courses
(identified from a given list), gender, and race
Analysis of the Data
• Mixed-Model design—data analyzed quantitatively
and qualitatively
• All the open-ended answers will be numerically
scored according to their corresponding rubric
– The first 5 assessments will be scored together in order to
obtain consistency
– Intra-rater and inter-rater reliability will be calculated and
reported
• Rubrics are being adapted from current valid and
reliable sources
– Both the NCTM and state of New York have field-tested their
rubrics several times; are published; currently implemented
across the country
Analysis of Data, cont’d
• Quantitative data will be analyzed using multivariate
analysis
– Possibly MANCOVA
• Cohort 1 can be thought of as the pretest and Cohort 5
can be thought of as the posttest
– TASP scores will be used in order to control for any
mathematical and/or educational differences
• Sample size should be around 450-500 with a majority
of the students being found in the first two cohorts
• Determined the number of assessment questions needed
to gain an adequate effect size at the end of the study
based on the number of students in the last 3 cohorts
– Cohorts will be restructured if after the initial analysis of the
data, there is not enough power in the last two cohorts
Analysis of the Data, cont’d
• Analyze open-ended questions
qualitatively also
– Emergent themes and/or trend
– Constant comparison (Denzin & Lincoln,
2000)
• Follow-up interviews with 5-10 students
– Verbally asking the same and/or similar
content questions
Definition of Terms
•
Algebra: the branch of mathematics that deals with
computations performed with variables, usually
designated by letters which stand for numbers in
arithmetic operations such as addition and multiplication. It
is a “generalized arithmetic…a structured system for
formulating and manipulating variables and formal
mathematical statements” (Driscolli, 1988, p. 119).
•
Assessment: the process of collecting, interpreting, and
synthesizing information to aid in decision making
(Airasian, 1991).
– The purpose of assessment is to “find out what each student
is able to do, with knowledge, in context” (Wiggins,
1996/1997, p. 19)
•
Content: what preservice teachers should demonstrate
that they know, think, or can do (Martin-Kniep, 2005).
Definition of Terms, cont’d
• Criterion: a standard on which a judgment or
decision may be based.
• Form: what perservice teachers’ products,
performances, or processes should look like
(Martin-Kniep, 2005).
• High-stakes test: “an assessment for which
important consequences ride on the test’s
results” (Popham, 2005, p. 15).
Definition of Terms, cont’d
•
Mathematical achievement: the attainment and success
of a student in mathematics as indicated by scores that go
beyond the number of correct responses. One example of
mathematical achievement is a score on a text-embedded
chapter test.
•
Middle grades certification: Math/Science specialist
program: Middle grades certification program leading to
the Bachelor of Science degree (B.S.) with a major in
Interdisciplinary Studies (INST) through the Middle Grades
Certification Program in the Department of Teaching,
Learning and Culture at Texas A&M University.
– It is a field-based program, with students spending extensive
time in middle schools.
– Credit hours required for graduation in the
Mathematics/Science strand total 133-134 credit hours.
Definition of Terms, cont’d
•
Mathematics knowledge for teaching: the mathematical
knowledge “used to carry out the work of teaching mathematics”
(Hill, Rowan, & Ball, 2005, p. 373).
•
NCTM: The National Council of Teachers of Mathematics, an
organization composed of classroom teachers, supervisors,
educational researchers, teacher educators, university
mathematicians, and administrators involved in the mathematics
education of students.
•
Performance Assessment: a form of testing requiring students
to demonstrate their achievement of understandings and skills
by completing a demanding task(s) in which they are asked to
respond to the task(s) orally, in writing, or by constructing a
product (Gronlund, 2003; Nitko, 2001; Popham, 2005).
•
“Specialized” content knowledge: mathematical knowledge,
not pedagogy (Hill, Rowan, & Ball, 2005).
Delimitations
• Not possible to test all preservice
middle grades mathematics teachers
• Study is limited to the number of
preservice teachers and topics feasibly
available
• Study did not consider gender or
ethnicity
Spring ’05
Spring ‘06
MATH 365: Structure of Math I
127
MATH 366: Structure of Math II
152
124
140
MATH 367: Basics Concepts of
Geometry
?
18
MATH 368: Introduction to
Abstract Math
47
47
MATH 403: Math and Technology
64
44
MASC 351: Problem Solving
37
28
MASC 450: Integrated
Mathematics
40
56
4
Methods
Courses
MEFB 460: Methods of Teaching
Middle Grades Mathematics
66
36
5
Student
Teaching
MEFB 497: Teaching Middle
Grades
53
97
Cohort
1
Mathematics
Courses I
2
Mathematics
Courses II
3
Mathematics
Education
Courses
Course
TOTAL Participants:
590
Conclusions/Implications
• There are no studies to directly compare the results to
– Hard to initially generalize the results to the population
• More longitudinal testing will have to be completed in
order for the results to become widely accepted
amongst mathematics educators.
• Some of the results may be comparable to similar
studies conducted at the elementary levels such as
those by Hill, Rowan, and Ball (2005).
• Other comparisons could possibly be made to
assessment efforts and findings of Bill Bush, University
of Louisville.
Thank You!
Margaret J. Mohr
Texas A&M University
Department of Teaching, Learning and Culture
MSMP Project Supervisor
Research Assistant and Assistant to Dr. Gerald Kulm
runmohr@yahoo.com