Impacts of Teacher Preparation on
Student Test Scores in North Carolina:
Teacher Portals
June 10, 2010 (revised and updated)
Gary T. Henry, UNC at Chapel Hill
Charles L. Thompson, East Carolina University
Kevin C. Bastian, C. Kevin Fortner, David C. Kershaw, Kelly M. Purtell, & Rebecca A. Zulli,
UNC at Chapel Hill
Alan R. Mabe & Alisa Chapman, UNC General Administration
Carolina Institute
for Public Policy
A Strategic Priority of the University
 UNC Overall Priority: Preparing More and Better Teachers
and School Leaders for North Carolina Public Schools
 Key Strategies to address the goal:

Recruitment

Preparation

New Teacher and School Leader Support
 Research approach to address quality preparation:

Entry Model, Persistence Model, and Impact Model(s)
 The latest Teacher Portals analysis from the UNC Impact
Research Model will be presented today
Carolina Institute
for Public Policy
2
Study Purpose and Acknowledgements
We set out to answer:
 How does the performance of teachers prepared in UNC teacher preparation
programs compare to the performance of teachers who enter North Carolina
classrooms with other types of preparation, for example, out of state, lateral
entry, Teach For America, and NC private & independent colleges?
• For the report, we analyzed 1,556,982 million test scores, 939,016 students,
and 19,940 teachers with less than 5 years of experience from all school
districts in North Carolina.
We are grateful to the North Carolina Department of Public Instruction, NC
Public School Forum, Teach For America, and our partners, the UNC General
Administration, for providing data and expert advice that were essential for this
report.
Carolina Institute
for Public Policy
3
How are teachers prepared to enter
North Carolina classrooms?
In 2007-08, 104,424 teachers were employed in North Carolina public schools
Distribution of Teachers by Portal, 2008
UNC Undergraduate
Out of State Undergraduate
Lateral Entry
NC Private Undergraduate
Unclassifiable
Out of State Masters
UNC Masters
Visiting International Faculty
UNC Licensure Only
Other Licensure Only
NC Private Masters
Teach for America
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
 This study “benchmarks” performance of UNC undergraduates to 11 other
“Portals” through which NC teachers entered classrooms.
 Portals are combinations of formal education and formal preparation to teach.
Carolina Institute
for Public Policy
4
What are the trends in the way teachers are prepared to enter
NC classrooms? (All NC public school teachers)
UNC Undergrad
Out of State Undergrad
NC private Undergrad
Lateral Entry
Unclassifiable
Out of State Masters
UNC Masters
VIF
UNC Licensure Only TFA
NC Private Masters
Other Licensure Only
The University of North Carolina at Chapel Hill
Carolina Institute for Public Policy
5
What are the trends in the way recently hired teachers are prepared to enter
NC classrooms? (less than 5 years of experience)
UNC Undergrad
Lateral Entry
Out of State Undergrad
NC private Undergrad
Unclassifiable
UNC Masters
Out of State Masters
VIF
UNC Licensure Only TFA
NC Private Masters
Other Licensure Only
The University of North Carolina at Chapel Hill
Carolina Institute for Public Policy
6
Where are out of state teachers concentrated?
Highest concentrations:
Polk County, Kannapolis, Dare
County, Charlotte-Mecklenburg,
Mooresville City, Union County
& Wake County
The University of North Carolina at Chapel Hill
Carolina Institute for Public Policy
7
Where are newly hired lateral entry
middle school teachers concentrated?
*
Highest concentrations:
Gates County, Scotland County,
Pasquotank County, Halifax County,
Anson County & Bertie County
*Excludes Teach For America & Visiting International Faculty
The University of North Carolina at Chapel Hill
Carolina Institute for Public Policy
8
Preview of Findings
 UNC Undergraduates
– Largest single source of NC teachers (32% of the teacher workforce).
– Performance in the middle of the pack (better in 14 comparisons, worse in 9 & no different in 74).
 Out of state undergraduates
– Second largest source of NC teachers (23% of the teacher workforce)
– Perform worse than UNC undergraduate prepared teachers in 5 out of 11 comparisons, including
elementary mathematics and reading
 Lateral entry teachers (other than Teach For America & Visiting International Faculty)
– Third largest source of North Carolina teachers (15% of the teacher workforce)
– Even higher concentrations in middle and high schools (in HS they teach 36% of the students
taking End Of Course subjects)
– Perform worse than UNC UG prepared in 3 of 11 comparisons, notably in high school overall
 Teach For America teachers
– Smallest source of NC teachers (0.3% of the teacher workforce)
– TFA corps members serve in Charlotte & 12 eastern North Carolina school districts
– Outperform UNC traditionally prepared in 5 out of 9 comparisons (no different in 4).
 Teaching Fellows
– Comprise 3% of NC teacher workforce
– State carefully selects HS seniors and provides scholarships in exchange for 4 years of teaching
– Perform slightly better than UNC UG teachers in 3 out of 11 comparisons & worse in 1 out of 11.
 Inexperienced teachers, particularly in their first year, are much less effective
The University of North Carolina at Chapel Hill
Carolina Institute for Public Policy
9
UNC Undergraduate performance
compared to other “portals”
UNC UG Outperform
Teachers from Other
Portals
Teachers from Other
Portals Outperform
UNC UG
UNC Masters
0
0
NC private undergraduate
1
0
NC private Masters
0
3
Out of state undergraduate
5
0
Out of state Masters
1
0
UNC Licensure only
1
0
Other Licensure only
0
0
Teach For America
0
5
Visiting Intl Faculty
2
1
Lateral Entry
3
0
Unclassifiable
1
0
Teacher Portals
The University of North Carolina at Chapel Hill
Carolina Institute for Public Policy
10
Focus for Today’s Briefing
1.
2.
3.
4.
5.
How effective are teachers that are imported from
other states?
Are Teach For America corps members effective in
North Carolina classrooms?
Do North Carolina Teaching Fellows make effective
teachers?
What are the costs of inexperienced teachers?
Middle school malaise – slowing gains from earlier
years.
The University of North Carolina at Chapel Hill
Carolina Institute for Public Policy
11
Where do out of state teachers come from?
The University of North Carolina at Chapel Hill
Carolina Institute for Public Policy
12
Top 10 Out of State Teacher Sources (undergraduate)
Source
New York
Pennsylvania
Ohio
Virginia
South Carolina
Michigan
Florida
West Virginia
Tennessee
Indiana
Teacher Count
2,714
2,395
2,237
1,996
1,993
1,441
1,322
1,138
864
728
Carolina Institute
for Public Policy
13
How does the performance of
out of state teachers compare?
 23 % of North Carolina teachers come from out of state
with undergraduate preparation
 Out of state teachers perform worse than UNC
undergraduates in elementary school mathematics and
reading; in high school overall, mathematics and social
studies
 How much does being taught by an out of state teacher
affect elementary mathematics students?
– Students taught by out of state teachers lose the equivalent
of 6.1 days of schooling
Carolina Institute
for Public Policy
14
Effectiveness of
Teach For America Corps Members
HS Overall
HS Math
HS English
HS Science
HS Social Studies
MS Math
MS Reading
MS Algebra
MS Science
Elementary Math
Elementary Reading
Teach for America
Coefficient
Teach for America as Compared
to UNC Traditional
0.172*
0.139*
0.085*
0.222*
0.079
0.148*
0.024
NR
NR
0.042
0.040
better
better
better
better
no difference
better
no difference
NR
NR
no difference
no difference
 What are the effects of TFA corps members?
–
–
TFA corps members increase student test scores for middle school
mathematics by approximately ½ year of learning.
TFA corps members increase HS pass rates for their students by
approximately 3 percentage points.
Carolina Institute
for Public Policy
15
Teaching Fellows and Other Scholarships
3,125 Teaching Fellows in North Carolina Classrooms in 2008 (3% of NC Teacher Workforce)
Level
Teaching Fellows
Other Scholarships**
High School Overall
578
10.16%
150
2.64%
High School Math
223
3.92%
76
1.34%
High School English
151
2.65%
28
0.50%
High School Science
71
1.25%
26
0.46%
High School Social Studies
140
2.46%
20
0.35%
Middle School Math
149
4.88%
78
2.55%
Middle School Reading
183
4.85%
56
1.48%
Middle School Algebra 1
40
8.00%
14
2.81%
Middle School Science
14
4.02%
11
3.12%
Elementary School Math
357
4.65%
150
1.96%
Elementary School Reading
364
4.68%
150
1.93%
** Non-TF Scholarships include NC Millennium Teacher Scholarship, Future Teachers of NC Scholarship, and Prospective Teacher Scholarship.
Carolina Institute
for Public Policy
16
Effects of Teaching Fellows and Other Scholarships
Comparisons with UNC Undergraduate
Prepared Teachers
Comparisons with
“Other Teachers” †
Teaching Fellows to
Other UNC Traditionally
Prepared Teachers
Non-TF Scholarship
Holders** to Other
UNC Traditionally
Prepared Teachers
Teaching
Fellows to
Other
Teachers
Non-TF
Scholarship
Holders** to
Other Teachers
All Public &
Private NC
Prepared
Teachers to Other
Teachers
High School Overall
0.019*
-0.014
0.039*
0.017
0.018*
High School Math
0.020
-0.012
0.061*
0.033
0.036*
High School English
0.005
-0.013
0.011
-0.007
0.004
High School Science
0.020
0.010
0.038
0.014
0.014
High School Social Studies
0.021
-0.021
0.048*
0.004
0.028
Middle School Math
0.024*
0.019
0.018
0.014
-0.007
Middle School Reading
-0.021*
-0.022
-0.010
-0.014
0.010
Middle School Algebra 1
0.026
-0.067
0.055
-0.039
0.010
Middle School Science
-0.110
-0.035
-0.149*
-0.015
0.002
Elementary Math
0.032*
0.072*
0.044*
0.070*
0.013*
Elementary Reading
0.000
0.029*
0.007
0.029
0.007
†Note: “Other Teachers” include Lateral Entry (including TFA & VIF), Out of State, Licensure Only, and Unclassifiable Teachers
** Non-TF Scholarships include NC Millennium Teacher Scholarship, Future Teachers of NC Scholarship, and Prospective Teacher Scholarship.
17
High School: Comparing Performance of Portals
to UNC Undergraduate Prepared
Overall
Math
English I
Science
Social Studies
UNC Masters
--
--
--
--
--
NC private undergrad
--
Worse
--
--
--
NC private Masters
Better
--
Better
Better
--
Out of state undergrad
Worse
Worse
--
--
Worse
Out of state Masters
--
--
--
--
Worse
UNC Licensure only
--
NR
--
--
--
Other Licensure only
--
NR
NR
NR
NR
Teach For America
Better
Better
Better
Better
--
Visiting Intl Faculty
Worse
Worse
--
--
NR
Lateral Entry
Worse
Worse
--
--
Worse
Unclassifiable
--
Worse
--
--
--
Teacher Portals
Dashed line ( -- ) indicates that the teachers in portal performed neither better not worse than UNC undergraduate prepared teachers.
The University of North Carolina at Chapel Hill
Carolina Institute for Public Policy
18
Middle School: Comparing Performance of Other
Portals to UNC Undergraduate Prepared
Math
Reading
Algebra I
8th Grade Science
UNC Masters
--
--
NR
NR
NC private undergrad
--
--
--
NR
NC private Masters
NR
NR
NR
NR
Out of state undergrad
--
--
--
--
Out of state Masters
--
--
--
--
UNC Licensure only
--
Worse
NR
NR
Other Licensure only
NR
NR
NR
NR
Teach For America
Better
--
NR
NR
Visiting Intl Faculty
--
--
--
NR
Lateral Entry
--
--
--
--
Unclassifiable
--
--
NR
NR
Teacher Portals
Dashed line ( -- ) indicates that the teachers in portal performed neither better not worse than UNC undergraduate prepared teachers.
The University of North Carolina at Chapel Hill
Carolina Institute for Public Policy
19
Elementary School: Comparing Performance of Other
Portals to UNC Undergraduate Prepared
Teacher Portals
Math
Reading
UNC Masters
--
--
NC private undergrad
--
--
NC private Masters
--
--
Worse
Worse
Out of state Masters
--
--
UNC Licensure only
--
--
Other Licensure only
--
--
Teach For America
--
--
Visiting Intl Faculty
--
Better
Lateral Entry
--
--
Unclassifiable
--
--
Out of state undergrad
Dashed line ( -- ) indicates that the teachers in portal performed neither better not worse than UNC undergraduate prepared teachers
The University of North Carolina at Chapel Hill
Carolina Institute for Public Policy
20
Other Teacher and Classroom Influences
on Student Achievement: Preliminary Findings
1. Inexperienced teachers





first year teachers are less effective in 10 out of 11 comparisons
second year teachers are less effective in 6 out of 11 comparisons
elementary reading teachers are less effective until 5th year as teachers
elementary school math students lose the equivalent of 21 days of schooling
middle school math students lose the equivalent of 47 days of school
2. Out-of-field teachers


students do worse when taught by out-of-field teachers in HS (overall, math, English,
and science) and in middle school math and reading
more teachers prepared to teach math and reading/English could increase student
achievement
3. Masters degrees acquired after beginning teaching (supplementary
Masters degree)

better in high school (mathematics and English)
4. Praxis II scores predict effectiveness in 7 out of 11 comparisons
Carolina Institute
for Public Policy
21
Effects of Other Teacher Characteristics
Infield
1st YR
2nd YR
3rd YR 4th YR
High School Overall
Better
Worse
Worse
--
High School Math
Better
Worse
Worse
High School English I
Better
Worse
High School Science
Better
MA
NBC
PRX
--
--
--
Better
--
--
Better
--
--
--
--
--
Better
--
Better
Worse
--
--
--
--
--
Better
--
Worse
--
--
--
--
--
Better
MS Math
Better
Worse
Worse
--
--
--
--
Better
MS Reading
Better
Worse
Worse
--
--
--
--
--
MS Algebra I
--
Worse
--
--
--
--
--
--
8th Grade Science
--
--
--
--
Better
--
NR
--
ES Math
--
Worse
Worse
Worse
--
--
--
Better
ES Reading
--
Worse
Worse
Worse
Worse
--
--
Better
High School SS
The University of North Carolina at Chapel Hill
Carolina Institute for Public Policy
22
Summary
1. TFA represents an opportunity for UNC & NC to learn and improve. How to
move from a smaller scale “boutique” operation to an industrial model.
–
Top college graduates across the U.S. apply
–
Selection of top 10% of applicants that have traits related to successful teaching
(persistence; ability to engage students; leadership)
–
Summer preparation: teaching in summer and planning to teach NC Standard
Course of Study
–
TFA corps members assigned to schools in clusters, form local social support
network
–
Continuing professional development throughout tenure as corps members
2. Out of State teachers
–
–
–
–
Little knowledge of or exposure to the NC Standard Course of Study
Less understanding of NC students & schools
Fewer classroom experiences in schools similar to those in which they teach
May over-represent teachers without option to teach in home state
3. High concentrations of lateral entry in high and middle schools
4. Serious consequences of inexperienced teachers
Carolina Institute
for Public Policy
23
Guiding Principles for
Transforming Research into Action
 UNC will take ownership and responsibility for “evidence based” policies and program
improvements, including:
– Improve existing UNC teacher preparation program
– Develop, pilot & evaluate innovations in UNC teacher preparation programs
– Increase UNC productivity where other large portals perform worse
 Identify specific remedies for gaps (e.g. training to teach the NC Standard Course of Study or
providing frequent & rigorous feedback to beginning teachers) in lower performing portals that
could be addressed through UNC programs
 Coordinate with K-12 education partners to develop, pilot & evaluate innovations
 Identify aspects of TFA that are “portable and scalable” to UNC programs:
– Selection based on “soft skills” e.g. perseverance, engaging audience and leadership
– More focus on preparation to teach specific objectives in assigned course/grade based on
NC Standard Course of Study
– Intensive observation, collaboration & feedback during pre-service preparation & first year
– Institute “evidence based approach” to teacher preparation
Carolina Institute
for Public Policy
24
More Details on Research Findings
Additional Information
Carolina Institute
for Public Policy
25
Current Findings: Finer-Grained “Routes”

12 “Portals” through which North Carolina teachers entered classrooms:
combinations of formal education and formal preparation to teach
–
Type of degree: Bachelors or Masters
–
Source of degree: UNC, Private Colleges & Universities in North Carolina, and
Out-of-State Colleges and Universities
–
Completion of stand alone certification by UNC or others with prior college
degree
–
Not licensed: Teach For America, Visiting International Faculty & Other Lateral
Entry
–
Unclassifiable

Focus of analysis: students’ gains; teachers with less than 5 year of
experience; adjusts for differences in students, classrooms & schools
Carolina Institute
for Public Policy
26
Sources of North Carolina Teachers
Portals
Certified Personnel
2000-2001 through Percentage
2007-08
Teachers
2007-08
Count
Percentage
1. UNC undergrad prepared
54,103
28%
33,031
32%
2. UNC graduate degree
7,862
4%
2,792
3%
3. NC private undergrad prepared
22,257
12%
12,793
12%
988
.5%
479
.5%
5. Out of state undergrad prepared
43,954
23%
24,101
23%
6. Out of state graduate degree
12,583
7%
5,780
6%
7. UNC licensure only
961
.5%
671
.6%
8. Other licensure only
1,414
.7%
647
.6%
535
.3%
310
.3%
10. Visiting International Faculty
1,925
1%
867
.8%
11. Lateral Entry
27,008
14%
15,268
15%
12. Unclassifiable
17,102
9%
7,685
7%
4. NC private graduate degree
9. Teach For America
TOTAL
190,692
104,424
27
Sources of Teachers in North Carolina:
Teacher Portal Definitions
Teacher Portal
Definition
1. UNC undergrad prepared
A North Carolina public school teacher who completed the requirements for initial
licensure at a UNC institution by earning (a) a bachelors degree in education or (b) a
bachelor’s degree in another major while simultaneously completing the necessary
education-related coursework, before beginning teaching, including Teaching Fellows
2. UNC graduate degree
A North Carolina public school teacher who earned a graduate degree from a UNC
system institution and qualified for an initial license before beginning teaching,
including Teaching Fellows
3. NC private undergrad
prepared
A North Carolina public school teacher who completed the requirements for initial
licensure at a private (independent) institution in North Carolina by earning (a) a
bachelors degree in education or (b) a bachelor’s degree in another major while
simultaneously completing the necessary education-related coursework, before
beginning teaching, including Teaching Fellows
4. NC private graduate
degree
A North Carolina public school teacher who earned a graduate degree from a private
(independent) North Carolina institution and qualified for an initial license before
beginning teaching
5. Out of state undergrad
prepared
A North Carolina public school teacher who completed the requirements for initial
licensure at an out of state institution by earning a bachelors degree before beginning
teaching
6. Out of state graduate
degree
A North Carolina public school teacher who earned a graduate degree from an out of
state university and qualified for an initial license before beginning teaching
Carolina Institute
for Public Policy
28
Sources of Teachers in North Carolina:
Teacher Portal Definitions (cont.)
Teacher Portal
Definition
7. UNC licensure only
A North Carolina public school teacher who, after earning a bachelor’s degree at any
public or private institution in any state, then separately completed the educationrelated requirements for teacher licensure at a UNC system institution, before beginning
teaching
8. Other licensure only
A North Carolina public school teacher who, after earning a bachelor’s degree at any
public or private institution in any state, then separately completed the educationrelated requirements for initial teacher licensure at a non-UNC system institution, before
beginning teaching
9. Teach For America
A North Carolina public school teacher who began teaching in NC after earning a
bachelors degree but before completing the remaining requirements for initial licensure
and did so through the Teach for America program
10. Visiting International
Faculty
A North Carolina public school teacher who entered teaching in NC through the Visiting
International Faculty program
11. Lateral Entry
A North Carolina public school teacher who entered the profession prior to completing
requirements for initial licensure (Teach For America corps members excluded)
12. Unclassifiable
A North Carolina public school teacher who cannot be classified into one of the portals
above on the basis of available evidence
Carolina Institute
for Public Policy
29
High School Teacher Counts by Portal
(<5 years exp.)
Teacher Portals
All Subjects
Math
English 1
Science
Social Studies
UNC Undergraduates
1458
517
381
210
376
UNC Masters
257
54
69
57
80
NC private undergrad
372
140
94
44
106
NC private Masters
64
15
14
15
20
Out of state undergraduate
868
313
182
181
217
Out of state Masters
244
62
53
76
60
UNC Licensure only
93
10
25
15
44
Other Licensure only
18
4
4
10
1
Teach For America
88
21
24
24
20
Visiting International Faculty
93
47
14
34
1
Lateral Entry
2057
689
504
637
342
Unclassifiable
76
17
17
Carolina Institute
17
27
for Public
Policy
30
Middle Grades Teacher Counts by Portal
(<5 years exp.)
Teacher Portals
Math
Reading
Algebra I
Science
UNC Undergraduates
716
845
132
69
UNC Masters
25
61
3
6
NC private undergrad
177
187
15
9
5
7
4
1
Out of state undergraduate
661
782
127
86
Out of state Masters
112
174
20
20
UNC Licensure only
26
42
4
2
Other Licensure only
8
8
1
1
Teach For America
39
65
8
1
Visiting International Faculty
101
81
17
7
Lateral Entry
1140
1438
Unclassifiable
46
87
NC private Masters
159
Carolina
138
Institute
for Public Policy
9
8
31
Elementary Grades Teacher Counts by Portal
(<5 years exp.)
Teacher Portals
Math
Reading
UNC Undergraduate
2420
2448
UNC Masters
119
127
NC private undergrad
1016
1022
18
19
Out of state undergraduate
2457
2484
Out of state Masters
469
478
UNC Licensure only
92
96
Other Licensure only
33
33
Teach For America
45
49
Visiting International Faculty
170
168
Lateral Entry
617
641
Unclassifiable
211
211
NC private Masters
32
Findings presented today:
Assessment of Impacts of Teacher Preparation

Teachers with fewer than five years of experience in NC classrooms

High School: Overall, Math, English I, Science, and Social Studies

Middle School: Math, Reading, Algebra I, and 8th Grade Science

Elementary School: Math and Reading

Teacher value added models including prior test scores

Models compare gains for teachers prepared in each portal with UNC
undergraduate prep programs

Additional models assess marginal effects of:

–
Teaching Fellows Program and Other Teacher Scholarships
–
NBC, Supplemental Masters, Praxis II scores
Models account for numerous variables that are beyond the control of
the teacher prep programs (list updated from prior models)
Carolina Institute
for Public Policy
33
Control Variables Used in the Impact Model
Other factors in the model to isolate the effect of teacher portals
Student
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
Prior test scores (reading & math)
Classmates prior test scores (peer
effects)
Days absent
Structural mobility
Other between year mobility
Within year mobility
Race/ethnicity
Poverty
Parental education
Gifted
Disability
Currently limited English proficient
Previously limited English proficient
Overage for grade (held back or
retained at least once)
Underage for grade (promoted two
grades)
Grade level
Classroom & Teacher
School
Years of experience
Teaching infield
Number of students
Advanced curriculum
Remedial curriculum
Heterogeneity of prior
achievement within classroom
7. Additional: Teaching Fellows &
Other Teacher Scholarships
8. Supplemental Masters, NBC,
Praxis II
1. School size (ADM)
2. Suspension rate
3. Violent acts per 1,000
students
4. Total per pupil expenditures
5. District teacher supplements
6. Racial/ethnic composition
7. Concentration of poverty
1.
2.
3.
4.
5.
6.
Carolina Institute
for Public Policy
34