CONSTRUCTING AUTHENTIC AND MEANINGFUL
ASSESSMENT SYSTEMS
FOR MEASURING TEACHER EFFECTIVENESS
AND STUDENT ACHIEVEMENT
Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D.
Janice Janz, Ph.D, & Richelle Voelker, M.Ed.
University of New Orleans (UNO)
Kristin A. Gansle, Ph.D.
Louisiana State University (LSU)
Purpose
CONTRIBUTE TO THE NATIONAL DISCUSSION OF
THE ACCOUNTABILITY FOR STUDENT LEARNING
THROUGH EFFECTIVE TEACHER PREPARATIONS
Assessment
• Demand for accountability
• Opportunity to examine the
relationship between
teaching and learning
“The processes
of teaching
and learning
stimulate
one another.”
(Confucius)
• Evidence of strengths,
needs for improvement, and
impact
Agenda
Louisiana Department of Education
LA Teacher Preparation Program (TPP) Assessment
Value-Added Teacher Evaluation Model
Data and Results
UNO Collaborative Redesign Project
Collaborative Processes
Results: In-Progress
Subsequent Plans
Conversations with Colleagues
Discussion Questions
Open Discussion
LOUISIANA METRICS
OF TEACHER PREPARATION PROGRAM EFFECTIVENESS
INCORPORATING STUDENT ACHIEVEMENT
Kristin A. Gansle , Ph.D.
Louisiana State University (LSU)
LA Teacher Preparation Program (TPP) Assessment
2000-2001 & Following
• How to measure our program completers’ success in terms
of the achievement of their students?
• Especially important given statewide redesign of academic
programs including SpEd by TPPs
• BoR (Jeanne Burns), Blue Ribbon Commission, LSU
(George Noell, Kristin Gansle)
• Have been using Value-Added metrics to evaluate TPPs
since 2003
What Do We Do?
• Predict achievement of individual students based on
prior achievement, demographics, and attendance (using
State achievement tests: HLM, mean approx 300, sd
approx 50)
• Assess actual student achievement
• Link the difference between predicted achievement and
actual student achievement to their new teachers and the
TPPs that taught them
• Act on results
Predictors for LDOE Value-Added
Teacher Evaluation Model
(Current Model)
• Prior Student Achievement
• Attendance
• Gifted Classification
• Free/Reduced Lunch Status
• Section 504 Status
• Disability Status
• Discipline Records
• Retained students are included in the analysis
Value-Added Model
• State Achievement Tests (Math, Science, Social Studies, Reading,
& Language Arts - Grades 4-9)
• New Teachers:
• 1st and 2nd year teachers with regular certificates
• Teaching within area of certification
• Full academic year with their students
• Experienced Teachers:
• 3rd or subsequent year teachers with regular certificates
• Teaching within area of certification
• Each year: all districts, 250,000 students, 7000+ teachers, 1300+
schools
Data We Provide to TPPs
• (October, 2006) By content areas (Reading, Mathematics, Science,
Social Studies, English/Language Arts)
• (March, 2011) By certification/grade spans within content areas
(Grades 1-5, Grades 4-8, Grades 6-12, & Special Education)
• (Fall, 2011)
• By student performance subsets within content areas (Low,
Middle, and High)
• By special education status (yes/no) for student groups
• By free lunch status (free lunch/pay own lunch)
• By LEP status
Data We Provide to TPPs
(continued)
• (Fall 2011) Deidentified individual teacher data for programs
• Value-Added Score for each teacher
• Mean content standards scores from standardized testing
program (LEAP, iLEAP) for ELA, Math, Reading, Science,
Social Studies
• What does that look like?
Overall Results
content
ELA
Mathematics
Reading
Science
Social Studies
75
103
overall
mean
effect
1.0
5.1
sem
0.7
0.9
48
88
66
-2.2
2.7
0.7
0.9
1.0
1.7
N
FOR ADDITIONAL INFORMATION
Kristin A. Gansle
kgansle@lsu.edu
Jeanne M. Burns
jeanne.burns@la.gov
George H. Noell
gnoell@lsu.edu
http://regents.la.gov/value-added-teacher-preparation-programassessment-model/
Certification Endorsements
Elementary Grades
Middle School
Secondary Grades
Special Education
mean
effect
mean
effect
mean
effect
mean
effect
content
n
sem
n
sem
n
sem
n
sem
ELA
42
-0.8
1.1
30
3.1
1.6
41
-1.0
1.1
12
1.2
2.8
47
1.4
1.3
69
4.2
1.2
28
3.8
1.3
12
1.0
1.8
Reading
40
-0.6
0.9
8
3.2
2.9
9
3.0
2.6
8
3.7
2.9
Science
38
-1.6
1.1
40
2.5
1.6
20
-0.4
1.3
5
-2.9
1.9
Social
Studies
35
-2.7
1.7
10
2.0
4.2
19
4.1
3.2
5
-5.1
4.2
Mathematics
Achievement Bands
Lowest 25%
Middle 50%
mean
effect
content
ELA
Mathematics
Reading
Science
Social Studies
n
Highest 25%
mean
effect
25
24
sem
-4.7
1.7
-5.2
1.9
17
27
30
-1.8
-2.4
-0.9
1.6
1.0
1.5
n
34
31
24
29
37
mean
effect
sem
1.0 1.0
2.1 1.2
0.3
0.0
0.0
0.9
1.1
1.4
n
21
21
17
19
25
sem
-2.2 1.2
-0.6 1.5
-0.2
-0.9
-0.1
0.9
1.2
2.1
Free Lunch/Not SpEd/no SpEd
Free Lunch
n
mean
effect
Paid Lunch
sem
n
mean
effect
Special
Education
sem
n
mean
effect
No Special
Education
Services
sem
n
mean
effect
sem
50
-0.5
0.8
47
1.3
0.7
40
0.3
1.4
44
-1.3
0.9
40
-1.4
1.0
31
2.0
1.1
20
-1.2
1.1
45
-1.9
1.1
Reading
34
-1.9
0.7
43
1.5
0.8
17
-0.9
1.5
38
-2.1
0.9
Science
38
-0.2
1.0
38
0.8
0.8
23
1.0
1.0
32
0.3
1.0
Social
Studies
45
-1.4
1.1
40
0.7
1.0
29
-0.8
1.5
40
-1.5
1.1
ELA
Mathematics
Deidentified Data in Excel File
TPPs Can Use as They Wish
UNO: PROGRAM DATA
Brenda Burrell, Ph.D.
Paul Bole, Ed.D.
Janice Janz, Ph.D.
& Richelle Voelker, M.Ed.
University of New Orleans (UNO)
Collaborative
Processes
• Analyzing and
synthesizing professional
standards and literature
• Interviews, surveys, and
discussions with multiple
and diverse stakeholders
• Individual and committee
design and development
work sessions
Results: Work In-Progress
• Response to Louisiana TPP Assessment
• Constructive Principles
• Concordance of Professional Standards
• Revised Conceptual Framework
• Answers to Key Questions: Perspectives
Response to LA TPP Assessment
Concerns/Opportunities
• School administrators
may not know what to
look for when
observing special
education instruction
Potential Strategies
• Provide professional
development
opportunities for school
leadership personnel
• Develop guidelines
aligned with research
based special education
practices
Response to LA TPP Assessment
Concerns/Opportunities
• It may be difficult to
determine how to assign
effectiveness scores to special
education teachers working in
inclusive settings
Potential Strategies
• Research
• Disseminate findings
• Make recommendations
• Facilitate systemic
changes
• Teachers in high-need
schools may not be rated
similarly as those in more
advantaged schools
Response to LA TPP Assessment
Concerns/Opportunities
• It may be difficult for
special education teachers
to score “highly effective”
on some elements of the
Danielson framework
• Some teachers may
develop Student Learning
Targets (SLTs) that are
problematic
Potential Strategies
• Advocate for revised
descriptions in rubric to align
with special education practices
and provide corresponding
examples to assist principals
when observing effective
instructional practices
• Review samples of SLTs
developed by special education
teachers to determine specific
issues and develop
corresponding training materials
Constructive Principles
• Collaboratively Designed
• Informed by Multiple and Diverse Resources
• Linked to Professional Standards
• Useful for Multiple Assessment Points
• Yields Data to Inform/Improve Teacher
Education
Concordance of Professional Standards
• Danielson Framework for Teaching
• Council for Exceptional Children (CEC)
• International Reading Association (IRA)
• National Council of Teachers of English (NCTE)
• National Science Teachers Association (NSTA)
• National Council for the Social Studies (NCSS)
• National Council of Teachers of Mathematics (NCTM)
• Louisiana Components of Effective Teaching
• UNO Roles and Responsibilities of Teachers
Revised Conceptual Framework
Roles & Responsibilities of Effective Teachers
• Manage Classroom Contexts/Environments
• Design Curriculum and Instruction
• Deliver Instruction and Assess Learning
• Participate in Professional Responsibilities
(e.g., Advocacy, Collaboration, Using Data to Improve
Practice)
Answers to Key Questions: Perspectives
Questions
Sample Responses
• How should IHE faculty
assess teachers’ use of
evidence-based practices
and teacher effectiveness
• How should IHE faculty
assess program
completers’ impact on
student learning and
student behavioral
progress?
• Video Sample of Teaching
• Surveys/Interviews
• Informal Observations
• Progress Monitoring
• Student Work Samples
• Student/Teacher Portfolios
Subsequent Plans
• Develop our own teacher evaluation rubric
• Use same teacher evaluation rubric across
concluding semesters of preparation and during
initial years of teaching after program completion
(induction); using evaluation to provide support
• Conduct field testing and research
• Make recommendations to the Louisiana DOE
regarding rubric currently used for State teacher
assessment and/or training provided to evaluators
Conversations with Colleagues
Question: Set 1
• What data related to teacher effectiveness and
student achievement should and can be collected
by faculty in teacher education programs?
• What data will represent authentic and meaningful
measures of academic and behavioral performance
and growth of students with high incidence
disabilities and is also available to university
faculty?
Conversations with Colleagues
Question: Set 2
• Are administrators and other teacher evaluators
sufficiently informed to assess teachers with
students with special needs in an inclusive
setting or other settings?
• What professional development opportunities
would be most helpful and how could or should
they be delivered?
Conversations with Colleagues
Question: Set 3
• How can IHEs assess positive teacher
impact on students who do not
demonstrate academic or behavior
progress?
• Should teacher performance be aligned
exclusively with teacher effectiveness, as
indicated by student performance?
Conversations with Colleagues
Open Discussion
Contact Information
University of New Orleans (UNO)
Department of Special Education
and Habilitative Services
Brenda Burrell
bburrell@uno.edu
Janice Janz
jgjanz@uno.edu
Paul Bole
pbole@uno.edu
Richelle Voelker
rvoelker@uno.edu