Preparation Program Accountability
EPSB Retreat
July 15, 2013
Terry Hibpshman
Martin School for Public Policy and Administration
University of Kentucky
(c) 2013 Education Professional Standards Board
1
Inference
Expert Judgment
Based on multiple sources of information
Derived from noisy and incomplete sources
So that decisions are nonmonotonic
Applying bright lines only where appropriate
(c) 2013 Education Professional Standards Board
2
Influences
•
•
•
•
•
•
•
•
Product-oriented accountability trends
Post-Sanders perceptions of teacher effects
Title II of the Higher Education Act
The Elementary and Secondary Education Act – NCLB
Race to the top
KDE PGES
CAEP proposed program quality standards
EPSB Goals and Themes
And of course, the ubiquitous NCTQ
(c) 2013 Education Professional Standards Board
3
EPSB Goals
Goal 1: Every approved educator preparation program meets or exceeds all accreditation
standards and prepares knowledgeable, capable teachers and administrators who
demonstrate effectiveness in helping all students reach educational achievement.
Goal 2: Every professional position in a Kentucky public school is staffed by a properly
credentialed educator.
Goal 3: Every credentialed educator exemplifies behaviors that maintain the dignity and
integrity of the profession by adhering to established law and EPSB Code of Ethics.
Goal 4: Every credentialed educator participates in a high quality induction into the
profession and approved educational advancement programs that support effectiveness
in helping all students achieve.
Goal 5: The EPSB shall be managed for both effectiveness and efficiency, fully complying
with all statutes, regulations and established federal, state, and agency policies.
(c) 2013 Education Professional Standards Board
4
EPSB Themes

Diversity (with specific attention to exceptional children including the gifted and
talented, cultural and ethnic diversity)

Assessment (developing skills to assess student learning)

Literacy/Reading

Closing the Achievement Gap (identify what courses emphasize strategies for
closing the gap)
I would suggest there are others
Awareness of local conditions
Productivity
(c) 2013 Education Professional Standards Board
5
How do we know when we’ve succeeded?
EPSB Goals are operationalized by strategies
Goal 4: Every credentialed educator participates
in a high quality induction into the profession and
approved educational advancement programs
that support effectiveness in helping all students
achieve.
Strategy 4.1. Develop and utilize reliable measures
of teacher effectiveness and student achievement
that may be used in evaluation of induction and
professional advancement activities.
(c) 2013 Education Professional Standards Board
6
Strategies are operationalized by measurements
Proposed Measurement Item 25: Mean performance score on internship performance
record
Which requires a valid, reliable, and sensitive intern performance measurement
methodology
And there might be additional measures
Teacher effectiveness
Student achievement
(c) 2013 Education Professional Standards Board
7
General Accountability Approach
Horizontal accountability
Coordinated with KDE
Incorporates federal requirements
Eventually incorporate CAEP stadards
Sensitive to conditions of practice
Segmented teacher labor market
School and district characteristics
Weighted by year post-completion
Addresses matters within the control of preparation programs
Differentiates between poor judgment and lack of capacity
Measures are practice-relevant
Consequences administered within a “just culture” framework
Incentives and technical assistance to help programs develop effective internal accountability systems
Subject to ongoing review and revision
(c) 2013 Education Professional Standards Board
8
Consequences
Reporting
Technical assistance
Program review
Sanctions
Identification as a poorly-performing program
Limitations on the program’s privileges
Decertification
And an escalation component
Slide 12
9
Implementation

Create scales for the reporting of existing methods, such as accreditation
processes

Identify additional elements for the dashboard, and develop display
formats

Develop methods for creating the disaggregated values using SGP data

Develop generalized statistical procedures for analysis of longitudinal data

Develop procedures for review and revision of the accountability model

Develop a yearly program progress report format

Develop inference guidelines, train staff
(c) 2013 Education Professional Standards Board
10
Examples
Production
Effectiveness
Praxis
Completions
(c) 2013 Education Professional Standards Board
11
Questions/Comments
(c) 2013 Education Professional Standards Board
12
(c) 2013 Education Professional Standards Board
13
HEA Title II State Annual Report
Number of Report Headings
2002-2011
Reporting Year Number of Headings
2002
8
2003
8
2004
8
2005
8
2006
8
2007
8
2008
8
2009
7
2010
25
2011
26
And the rules keep changing . . .
(c) 2013 Education Professional Standards Board
14
Kentucky HOUSSE Scoring Protocol
College-Level Coursework: Maximum=87 points
Must be documented by official college transcript and specific to the teaching assignment listed above.
Must be coursework in core content – i.e., not pedagogy. May be coursework taken prior to or after
initial certification.
3 points per credit hour earned
Teaching Experience: Maximum=45 points
Must be in the teaching assignment and in an accredited school
3 points per year
Professional Development: Maximum=45 points
Must be consistent with the definition of “high quality professional development” delineated in Section
9101 of NCLB
5 points per documented activity during the past 10 years
Achievements/Awards: Maximum=35 points
Must be specific to the teaching assignment listed above (e.g., department chair; cooperating teacher
for student teacher; KTIP resource teacher; documented student learning as demonstrated via state/
national test scores; teaching awards).
5 points per documented activity during the past 10 years
Source: Kentucky Educational Professional Standards Board; Identifying Highly Qualified Teachers
(c) 2013 Education Professional Standards Board
15
Race to the Top Reforms
•Designing and implementing rigorous standards and high-quality assessments.
•Attracting and keeping great teachers and leaders in America’s classrooms
•Supporting data systems that inform decisions and improve instruction
•Using innovation and effective approaches to turn-around struggling schools
•Demonstrating and sustaining education reform
(c) 2013 Education Professional Standards Board
16
17
Proposed CAEP Accountability Standards
Content and pedagogical knowledge
Clinical partnerships and practice
Candidate quality, recruitment and selectivity
Program impact
Provider quality, continuous improvement, and capacity
In addition, a CAEP data acquisition and reporting mechanism to monitor
the quality of teacher education programs nationally
(c) 2013 Education Professional Standards Board
18
An example - Production
Program Completers by Content Area
Southern State University
Content area
Elementary Education
Middle School Mathematics
Middle School Science
Middle School Social Studies
Middle School Language Arts
Music
Art
Physical Education
Chemistry
Physics
Biology
English
Completers Completers Completers Completers Completers Completers Completers Completers
2008
2009
2010
2011
2012
2013
2014
2015
62
61
65
63
59
58
56
59
20
28
25
21
17
15
17
15
18
18
17
15
18
21
22
19
28
27
31
29
26
28
32
29
31
30
32
35
37
39
41
42
10
12
9
11
10
10
12
9
8
5
9
11
8
7
9
7
25
22
21
19
20
21
20
19
6
5
6
7
4
6
8
7
0
1
0
0
0
2
0
1
15
18
*12
15
16
17
14
18
30
28
32
34
31
30
32
33
(Simulated)
(c) 2013 Education Professional Standards Board
19
(Simulated)
(c) 2013 Education Professional Standards Board
20
TPP/School accountability attribution by practice year
Practice Year
TPP component School component
1
100%
0%
2
80%
20%
3
60%
40%
4
40%
60%
5
20%
80%
6 or more
0%
100%
(c) 2013 Education Professional Standards Board
21
(Real historical data)
22
This is a relatively small, rural, nonpublic institution
(Real historical data)
(c) 2013 Education Professional Standards Board
23
This is a relatively large, rural, public institution
(c) 2013 Education Professional Standards Board
24
This is a medium-sized, rural, nonpublic institution
(c) 2013 Education Professional Standards Board
25
This is a small, urban, nonpublic institution
(c) 2013 Education Professional Standards Board
26
(c) 2013 Education Professional Standards Board
27
(c) 2013 Education Professional Standards Board
28
(c) 2013 Education Professional Standards Board
29
(c) 2013 Education Professional Standards Board
30
(c) 2013 Education Professional Standards Board
31
(c) 2013 Education Professional Standards Board
32