LMT Sharing Session: April 2011
Vicky Kukuruda
Riverside County Office of Education
Patty O'Driscoll
Public Works, Inc.
Susan Tucker
Evaluation & Development Associates LLC
AGENDA
• Content of LMT
• Appropriate use and fidelity to research
embedded in the LMT project
• Strengths and challenges of using LMT
• Overview of options
• Project examples
LMT Overview
• Developed by researchers led by Deborah
Ball at the University of Michigan
• Builds off of research begun at the MPDI
(Math Professional Development Institutes)
• Supported by NSF since 2002
• Focuses on nature of the mathematical
knowledge needed for teaching
– Special focus on instructional practice that can
intervene on significant patterns of educational
inequality in mathematics education
Design of the LMT
• Designed to measure effectiveness of PD and its
impact on the mathematical content knowledge
that teachers need to teach mathematics well
• LMT Design team develops the test through
writing, piloting, and analysis of problems that
reflect real math tasks facing teachers
• LMT piloted with the help of over 2000 teachers
– Mapping item pool against the NCTM and California
standards
• Can be used to study:
– content-focused PD
– teacher learning from pre-service coursework
– curriculum materials and projects exploring contribution
of teacher knowledge to student achievement
What you learn in a training
• Training at UM and national conferences
– no cost for training or for the measures
themselves, but users must cover transportation
and accommodation expenses
• To use the LMT in a research project, you
must attend an LMT training
• Literature and research framework for the
development of the assessment system
• How test design is embedded in
mathematics professional development
• How it is different from past efforts to
measure effectiveness of PD
• When to use it and when not
Training Topics
• LMT measures & their development
• Early statistical work and their measures
• Item Response Theory (IRT) basics
– focus on its use in scoring the assessment
• Orientation to scales and forms
• Using technical reports to understand measures
• Terms of use of the LMT
• Administration & lessons learned from experience
• Teaching Knowledge Assessment System (TKAS)
(new on-line version of LMT)
Content covered in LMT about
teacher content knowledge
•
•
•
•
Number and operations (K-6, 6-8)
Patterns, functions, and algebra (K-6, 6-8)
Geometry (3-8)
Topic-specific modules (4-8) in:
–
–
–
–
Rational number
Proportional reasoning
Geometry
Data, probability, and statistics
Appropriate Use of the LMT
• LMT does NOT offer measures that can be used
for hiring, promotion, pay, or tenure
• LMT is NOT designed to make highly accurate
statements about individuals' math knowledge
• LMT does NOT participate in study design or data
analysis or provide PF to teachers or schools
• LMT is for comparing groups of teachers'
mathematical knowledge, or examine how a group
of teachers' knowledge develops over time
LMT Assumptions
• To make progress in developing theory of
knowledge for teaching
• To develop items that measured math that
teachers use in teaching, not just what they
teach
• To orient items around problems that all
teachers might face in teaching math
• To produce useable measures:
– With items that are not grade-specific
– With items that do not represent any single view of
teaching (e.g., “reform”)
– Which can discriminate among teachers’ capability in this
area
Fidelity of Use of the LMT
• Want to measure range of teacher ability
reliably
– Need easy, medium, difficult items
– No evidence that teachers must answer any of these
items to be effective
• Limits conclusions
– 50% average ≠ “teachers failing”
– No conclusions about individuals’ competency
Using the LMT Making Selections
• Choose content domain
• Identify forms that match your professional
development content
• Consider the ability of your participants
– Consider short AND long term development
• Select forms
– match to ability and content of PD
– look for high reliability scales
Available Scale Measures
• Elementary number and operations (K-6, 6-8)
• Elementary patterns, functions, and algebra (K-6,
6-8)
• Geometry (3-8)
• Rational number (4-8)
• Proportional Reasoning (4-8)
• Data, probability, and statistics (4-8)
– Each K-6 item has each been piloted with over 600 elementary teachers,
– Middle school items have each been piloted with over 300 middle school
teacher
Sample Item- Elementary
7. Which of the following story problems could be used to illustrate
1
1
1 divided by ? (Mark YES, NO, or I’M NOT SURE for each possibility.)
4
2
1
pies evenly between
4
two families. How mu ch should each
family get?
Yes
No
I’m n ot
sure
1
2
3
1
2
3
1
2
3
a) You want to split 1
b) You have $1.25 and may soon double
your money. How mu ch money would
you end up with?
c) You are making some homemade taffy
1
and the recipe calls for 1 cups of
4
butter. How many sticks of butter (each
1
stick = cup) will you need?
2
Sample Item- Elementary
5. Mrs. Johnson thinks it is important to vary the whole
when she teaches fractions. For example, she might use
five dollars to be the whole, or ten students, or a single
rectangle. On one particular day, she uses as the whole
a picture of two pizzas.
What fraction of the two pizzas is she illustrating
below?
Mark ONE answer
a)5/4
b) 5/3
c)5/8
d)1/4
Sample Item- Elementary
2. Ms. Chambreaux’s students are working on the
following problem:
Is 371 a prime number?
As she walks around the room looking at their papers,
she sees many different ways to solve this problem.
Which solution method is correct? (Mark ONE answer.)
Sample Item-Middle School
18. Mrs. Smith is lo oking through her textbook for problems and solution methods that
draw on the distributive property as their primary justification. Which of these familiar
situations could she use to demonstrate the distributive property of multiplication over
addition [i.e., a (b + c) = a b + ac]? (M ark APPLIES, DOES NOT APPLY, or I’M NOT
SURE for each.)
Applies
Does not
apply
I’m n ot
sure
3 5

4 4
1
2
3
b) Solving 2x – 5 = 8 for x
1
2
3
c) Combining like terms in the expression
3x2 + 4y + 2x2 – 6y
1
2
3
1
2
3
a) Adding
d) Adding 34 + 25 using this method:
34
+25
59
Other considerations/Examples
• Pros and Cons
– On-line system
• online using the Teacher Knowledge Assessment
System (TKAS) available to those who have
previously attended an item training workshop.
• contact Katherine Mikesell at kmikesel@umich.edu
– Computer Adaptive testing
– Paper-based assessments
• Examples to Share
– Riverside
– NSF MSP ACES grant at CSUSB
– Downey
• Discussion and Questions
Augmenting the LMT at CSUSB
• NSF MSP ACES grant directed by Dr.
Davida Fischman
• Working in Ontario-Montclair School District
• Demonstrating certain LMT items with think
aloud protocol
• Video taping lesson study
– Adapting MQI protocol to analyze videos
– Mathematical Quality of Instruction (MQI)
– Online training for MQI: Nina Cohodes at
nina_cohodes@gse.harvard.edu or 617.496.4815.
Example of LMT use in Riverside:
• 110 Teachers grades 3 - Algebra 1
• 4 small feeder K-8 districts, High School
district and County Special Education and
Alternative Education teachers
• Objective 1: Teacher Content Knowledge
• 3 year Content Focus:
– Numbers and Expressions
– Proportional Reasoning
– Functions and Equations
Project DELTA Design Diagram
Jan.
2010 –
June
2011
Spring
July
Sept – Nov.
Jan. – Mar.
Feb. – May
June
Kick Off
Event
Intensive
8-days/
2 weeks
(M-Th,
M-Th)
Site/Grade level based
lesson study days
Intensive
Saturday Sessions
Site/Grade level based
lesson study days
Culminating
Event
2
hrs
May 26,
2010
60
hrs
(specific dates TBD by
specific lesson study groups)
6
hrs
6
hrs
6
hrs
Numbers &
Expressions
6
hrs
60
hrs
6
hrs
6
hrs
Proportional
Reasoning
60
hrs
Functions &
Equations
6
hrs
6
hrs
6
hrs
Feb. 5,
2011
12 quarter-units MCPT coursework*
6
hrs
6
hrs
6
hrs
Jan. 21,
2012
March 10,
2012
6
hrs
6
hrs
6
hrs
12 quarter-units MCPT coursework*
July 18-21
July 25-28
July
2012 –
June
2013
6
hrs
Jan. 22,
2011
July 19-22
July 26-29
July
2011 –
June
2012
(specific dates TBD by
specific lesson study groups)
6
hrs
6
hrs
6
hrs
6
hrs
6
hrs
Jan. 12,
2013
March 9,
2013
6
hrs
6
hrs
6
hrs
2
hrs
June 15,
2013
8 quarter-units MCPT coursework*
July 16-19
July 23-26
* For participants seeking Mathematics Supplemental Authorization or Subject Matter Authorization
Cycles of Administration:
•
First Administration – Pre A
3 surveys on first day of institute (first hour)
• Second Administration- Post B
Saturday before we released them
• Third Administration- Post A
2 hour open session with a series of
afterschool choices/locations
Project DELTA LMT MKT Schedule
Spring
July
Kick Off
Event
Summer
Institute
(8-days/
2 weeks)
2010 –2011
Year 1
Elem. Number
Concepts/Ops.
CK 2008-A
Sept – Nov.
3 Lesson Study Days
MS Number
Concepts/Ops.
CK 2007-A
Year 2
Summer
Institute
2011
July 18-22
July 25-28
Feb. – May
Intensive
Saturday Sessions
MS Patterns/
Functions/Alg.
CK 2005-A
2011 – 2012
Jan. – Mar.
Elementary
Number Concepts
and Operations
CK 2008-B
June
Culminating
Event
3 Lesson Study Days
Elementary
Number Concepts
and Operations
CK 2008-A
Numbers and Expressions
3 Lesson Study Days
Middle School
Number Concepts
and Operations
CK 2007-B
3 Lesson Study Days
Middle School
Number Concepts
and Operations
CK 2007-A
Proportional Reasoning
2012 –2013
Year 3
Summer
Institute
2012
3 Lesson Study Days
Middle School
Patterns,
Functions and
Algebra
CK 2005-B
3 Lesson Study Days
Functions and Equations
Middle School
Patterns,
Functions and
Algebra
CK 2005-A
Pre and Mid Year Results on the LMT MKT Teacher
Questionnaire
29
27
25
23
Number of Items Correct
21
19
17
15
13
11
9
7
5
3
1
pre
mid
n =80
n=
DELTA REG ED
16
18
Regular Ed pre = .51; Regular Ed mid = .69 (IRT coefficients)
n=
n=
19*
DELTA ALT ED/SPED (RCOE)
14
18
ALT ED/SPED (RCOE) pre = .17; ALT
*Interpret differences with caution due to small sample size of RCOE group.
Objective 1: Teacher Content
Knowledge
•DELTA teachers increased performance on the LMT
MKT from pre to mid (ss)
•DELTA Regular Ed teachers performed better than
Alt Ed/SPED (RCOE) teachers on the pre LMT
MKT, however, Alt Ed/SPED teachers made greater
gains and were performing at the same level by mid
year.
Q & A about LMT
• What are questions you have about:
–
–
–
–
Content
Use
Challenges
Ways to leverage with other data sources
Citation for the work
• Copyright © 2006 The Regents of the University of
Michigan. For information, questions, or permission
requests please contact Merrie Blunk, Learning
Mathematics for Teaching, 734-615-7632. Not for
reproduction or use without written consent of
LMT. Measures development supported by NSF grants
REC-9979873, REC- 0207649, EHR-0233456 & EHR
0335411, and by a subcontract to CPRE on Department
of Education (DOE), Office of Educational Research
and Improvement (OERI) award #R308A960003.
Contacts for more info:
• Vicky Kukuruda:
– VLKUKURUDA@rcoe.us
• Patty O'Driscoll:
– patty@publicworksinc.org
• Susan Tucker:
– sutucker1@mac.com