What goes on inside the classroom in Africa?
Assessing the relationship between what
teachers know, what happened in the
classroom, and student performance.
Deon Filmer
Ezequiel Molina
Brian Stacy
February 10, 2015
Abstract
In this paper, we examine the effects of teacher human capital, instruction time, and teacher practices on student achievement in Sub-Saharan
African countries. As we show, there is a substantial amount of variation in
student performance between classrooms even after controlling for student,
teacher, and school characteristics. Using a unique survey data set containing information on student achievement, teacher knowledge and pedagogical skill, teacher instruction time, and teacher practices, we examine the role
of these factors in raising student achievement. We find that a 1 standard
deviation increase in teacher knowledge can increase student achievement
between .05 and .11 standard deviations in math and between a null effect
and .06 standard deviations in reading. We find that a 1 standard deviation increase in the pedagogical skill of the teacher increases mathematics
achievement by around .05 standard deviations and reading achievement by
.10. Additionally, the length of instruction time, the practices of instilling
trust in students and challenging the students intellectually also have positive impacts on achievement. However, we find that these factors explain
only a small part of between classroom differences in achievement.
Acknowledgments: The authors gratefully acknowledges support from The William and Flora
Hewlett Foundation and the World Bank. This work is a product of the Service Delivery Indicators
initiative (www.SDIndicators.org,www.worldbank.org/SDI)and the staff of the International Bank
for Reconstruction and Development/The World Bank. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of
Executive Directors, or the governments they represent.
1
1
Introduction
A consistent finding in education research, at least in the context of the United
States, is that teachers matter for student achievement. A puzzle in this research
is that common observable characteristics of teachers explain little of the variation in teacher effectiveness. While teacher experience, education, and credentials
are often used by policy makers and researchers as measures of teacher quality,
these characteristics explain little of the variation in teacher performance at raising student achievement, making finding characteristics that do predict student
performance important for policy makers.1
Using a new survey data set linking teacher data to student test score performance in Sub-Saharan African countries, this paper makes two major contributions. First, while most of the research involving the effects of teachers on
students takes place in the context of the United States, ours is the first paper of
which we are aware to examine the size of differences in classroom effectiveness
in Sub-Saharan African countries.2 Second, our unique survey data links student
test scores to the subject knowledge level of their teachers as well as a measure of
pedagogical skill and classroom observations of teacher practices, which allows
us to take a deeper look at education production function than can be seen using
proxies such as experience, education, and credentials. With this information, we
are then able to examine the extent that teacher human capital, instruction time,
and classroom practices explain the variation in the classroom effectiveness.
Our data contain measures of student test score performance in 1,869 schools
in Mozambique, Uganda, Togo, Nigeria, and Kenya. The data also contain links of
students to teachers, along with data containing teacher mathematics and reading
1 See
Hanushek & Rivkin (2006) for an overview of the evidence.
context where, as discussed in Filmer, Martin, Molina, Stacy, Rockmore, & Wane (2014),
only 26% of 4th grade students in Togo for instance can correctly read a sentence. Additionally,
less than 15 percent of teachers know the basic minimum knowledge required to teach language
and mathematics in 3rd and 4th grade. Also, using classroom observations, the authors find that
teachers spend less than 45% of the time they are supposed to teach in a given day actually instructing their students. In addition, during unannounced visits to the school only around 6 in 10
teachers were actually found teaching when enumerators arrived at the school.
2A
2
knowledge test scores, as well as teacher pedagogical skill exam scores based on
lesson preparation and assessing student ability. Further, our unique data contain
rich information on teacher instruction, including a measure of effective instruction time formed during a classroom observation and particular instruction practices. These instruction practices we examine include whether or not the teacher
instills trust in the students, instills discipline in the student, or challenges students
intellectually.
Using this unique data set, we find that classroom effects, after controlling for
student non-verbal reasoning ability, as well as other student, teacher, and school
characteristics, explain around 15% of the total variation in student achievement,
suggesting that teachers matter in the context of Sub-Saharan Africa and that improving what takes place inside the classroom can potentially have large benefits
in raising achievement.
We find evidence that increasing teacher knowledge in mathematics can raise
math student achievement, confirming a finding found in Metzler & Woessmann
(2012). A 1 standard deviation increase in teacher mathematics knowledge can
increase student achievement by around .05 to .11 standard deviations of achievement, depending on the specification. We find a range between a null effect and an
effect of .06 standard deviations for teacher reading knowledge on student reading achievement. We also find a 1 standard deviation increase in pedagogical
knowledge increases student achievement by roughly .05 standard deviations in
math and .10 standard deviations in reading. We find that an additional hour of
effective teaching time per day increases student achievement by .034 standard
deviations in math and .054 standard deviations in reading. Finally, we find evidence that a teacher instilling trust in their students increases student achievement
in math and reading by .073 and .069 standard deviations respectively and that
challenging the students intellectually increases math achievement by .093 and
reading achievement by .141 standard deviation.
While teacher knowledge and classroom practices do have moderate impacts
on student achievement, we find that, after accounting for these factors, unobserv-
3
able characteristics of teachers and classes still explain roughly 13% of the total
variance of student achievement, down from 15% without controlling for these
variables. This means that a considerable amount of differences in classroom effectiveness are left unexplained even after accounting for teacher knowledge and
practices.
2
Background and Previous Literature
A large recent literature has found that teachers are one of the most important factors affecting student achievement. Nye, Konstantopoulos, & Hedges (2004) in an
analysis using random assignment created by the Project STAR class size experiment in the United States found that teacher explain roughly 11% of the variation
in math achievement and 7.5 to 10% of the variance in reading achievement. McCaffrey, Lockwood, Koretz, Louis, & Hamilton (2004) found that teacher explain
between 4.5% to 14% of the total variance of achievement in mathematics. The
results in Aaronson et al. (2007) suggest that teachers explain roughly 5% of the
total variance in student achievement in mathematics. Estimates in the range of
around 5% of the total variance in student achievement seem common in the context of the United States.
Almost no research is available on the importance of teachers on student
achievement in Sub-Saharan Africa, and the relative importance of teachers may
differ in this context for several reasons.3 First, schools in Sub-Saharan Africa
surely draw teachers from different segments of the teacher quality distribution
than in say the United States because of labor market conditions, differences in
human capital development for potential teachers, and host of other issues. Additionally, the returns to moving up higher in the teacher quality distribution may
3 The
only exception is Hein & Allen (2013), who examine the impact of teachers on the gap
between student achievement in math and student achievement in reading. These estimates identify
differential teacher effectiveness for the two subjects, and not the overall effectiveness levels of
the teachers in the subject. The authors find that teacher assignment explains around 3 to 12 % of
a standard deviation of the difference in subject knowledge for students.
4
differ in these countries compared to more developed countries, which could generate differences in the variation in student achievement explained by teachers.
While it is known that teachers can have a large impact on student achievement, at least in the United States, a common finding is that common variables
such as experience, education levels, and training of teachers are only weakly related to student achievement, a fact documented in Hanushek & Rivkin (2006)
and Hanushek & Wößmann (2007).4 Hanushek & Rivkin (2006) provides an
overview of the evidence of the impacts of teacher experience and credentials on
student achievement. The authors show that among 34 high quality teacher valueadded model studies, 0 showed statistically positive effects of teacher education
on student achievement and 9% showed negative effects. For experience, the authors found that only 41% showed a positive effect and 3% showed a negative
effect.
One characteristic that does seem to predict better student performance is the
teacher’s knowledge level of the subject they are teaching. Metzler & Woessmann (2012), in the context of Peru, found that a one standard deviation increase
in subject specific teacher knowledge raises student achievement by .09 standard
deviations in mathematics and has a null effect in reading. Shepherd (2013) examined teacher subject knowledge in South Africa and found that teacher knowledge
improves student achievement in the wealthiest quintile of schools.
Also, there is some evidence that teacher pedagogical practices and instruction
time affect student achievement. Lavy (2010) examines the impact of length of
instruction time in explaining student achievement using PISA 2006 and Israeli
administrative data and finds that instructional time has a positive and statistically
significant impact on student achievement.5 The estimates suggest an additional
4 Between
the three, there is more evidence that teacher experience increases student achievement. Wiswall (2013) finds that teacher experience has a statistically significant impact in mathematics but not reading. Harris & Sass (2011) also finds that experience increases student performance. They find no statistically significant evidence that professional development training,
undergraduate training, or college entrance exam scores improve student achievement. Both papers were in the context of the United States.
5 None of the countries in our survey data participated in the 2006 PISA examinations.
5
hour of instruction per week increases test scores by .07 standard deviations. Lavy
(2011) examines pedagogical practices of teachers and finds that the practice of
“instillment of knowledge and comprehension” and endowing pupils with analytical and critical skills have large impacts relative to other teaching practices.
Dobbie & Fryer Jr (2013) find increased instruction time and high expectations
predict school effectiveness is a sample of charter schools in New York City. A
25% increase in instruction time was associated with an increase in mathematics achievement of .05 standard deviations. They also find that schools with high
academic and behavioral expectations have .044 standard deviation higher mathematics and .03 standard deviation higher reading scores.
Our paper adds to this literature by estimating the fraction of the variance
in student achievement explained by teacher effects in a group of Sub-Saharan
African countries, something not done before to our knowledge. Additionally,
we are able to examine what teachers know, whether the teachers know how to
teach, how they spend time in the classroom, as well as their practices in the
classroom and the interactions between them in the context of Sub-Saharan Africa
using our unique survey data. We then examine to what extent these observable
characteristics explain differences seen across classrooms in student achievement.
3
Data
The Service Delivery Indicators initiative is an Africa-wide program that collects
facility-based data from schools and health facilities. It is a partnership of the
World Bank, the African Economic Research Consortium and the African Development Bank to develop and institutionalize a set of robust measures of service
delivery. The survey instruments are underpinned by rigorous research and embraces the latest innovations in measuring provider competence and effort. The
survey instruments were piloted in Tanzania and Senegal in 2012. SDI has been
implemented to date in Kenya and Uganda on 2013 and Nigeria, Mozambique,
6
and Togo in 2014.6
The Service Delivery Indicators (SDI) survey consist of two visits to the school.
One announced visit, the first visit, and a second announced visit, which occur
days after the announced visit to collect information on absenteeism. The SDI
survey is comprised of six modules described in Table 1.
Table 2 provides an overview of the number of observations for each module.
On average, for each survey we collected information on 374 schools in each
country. With around 8 teachers per school and 1 observation per teacher, we have
an average of 3110 observations on teacher attendance by country.7 Additionally,
around 4 teachers and 9 students have been tested to assess their knowledge in
each school.
The SDI approach to sampling is a multistage, cluster sampling approach. The
SDI surveys produce indicators with sufficient precision to identify changes in the
indicators of around 5 − 7 percentage points over time.
Table 3 reports summary statistics for the sample.8 On average students were
only able to correctly answer around 42% of the questions for mathematics and
45% for reading on tests targeted for the 4th grade level. Teachers could only
answer 53% of the questions correctly in mathematics and reading. When given a
pedagogical exam, the teachers only were able to get around 23% of the points on
average. In an unannounced visit to the school, only 72% of teachers were found
in the classroom they were supposed to be in. The effective instruction time is
only 3.25 hours per day on average, despite the scheduled duration of the school
day being 5.2 hours per day on average. The instruction time measure combines
data from the staff roster module (used to measure absence rate), the classroom
observation module, and reported teaching hours for the school day. The teaching
6 More
information on the SDI survey instruments, how to access the data, and more generally
on the SDI initiative can be found at: www.SDIndicators.org and www.worldbank.org/SDI, or by
contacting sdi@worldbank.org.
7 We only count the announced observation but we actually observe each teacher twice, in the
first and in the second visit.
8 Due to the multistage, cluster sampling approach the data are weighted by the inverse probability the unit is selected for sampling.
7
Table 1: The SDI Questionnaire
Module
Module
Interviewee
Module 1
School Information
Principal/Head
Teacher
Module 2A
Module 2B
Module 3
Module 4
Module 5
Module 6
Content
Information about school type,
facilities, school governance,
student numbers and school
hours.
Teacher Roster
Principal/ Head List of all school teachers.
Teacher
Teacher Roster
(10 randomly) Measure absence rates and colSelected Teach- lect information about teacher
ers
characteristics.
School Finances
Principal/Head
Information about school fiTeacher/Finance nances (Public Expenditure
Officer
Tracking Module).
Classroom Observa- Observation
An observation module to assess
tion
(teacher,
stu- teaching activities and classdents)
room conditions.
Pupil Assessment
(10 randomly) An assessment of 10 randomly
Selected
4th selected 4th grade students on
Grade Students mathematics and language.
Teacher Assessment Selected Teach- An assessment of teachers covers*
ering mathematics and language
subject knowledge and teaching
skills.
Note:* All 4th grade teacher that teaches mathematics or language currently, all teachers that taught 3th
grade mathematics or language the previous academic year and 3 to 5 teachers that teach 5th or higher
grade.
8
Table 2: SDI Surveys: Observation Counts Across Countries
Schools
Mozambique
Uganda
Togo
Nigeria
Kenya
Total
203
400
200
760
306
1869
Number of Observations
Teachers
Teachers
Classroom
Absenteeism Assessment Observations
1948
673
203
3805
2151
398
1086
831
192
5753
2256
730
2958
1678
306
15550
7589
1829
Pupil
Assessment
1761
3966
1938
6644
2953
17262
Notes: In the case of Tanzania and Senegal, since we use a different survey instrument
for teacher assessment, classroom observation and pupil assessment
hours for the school day is adjusted for the time teachers are absent from the
classroom, on average, and for the time the teacher teaches while in classrooms
based on classroom observations recorded every 1 minute in a teaching lesson.
Finally, in classroom observations panel of Table 3 we show three measures
of classroom practices which were formed using our classroom observations and
we found to be important for explaining differences in student achievement or to
closely match measures of classroom practice explored in other literature. The
first, which we name “instill trust”, is an indicator for whether the teacher called
on students by name and praised students while giving feedback. The second,
which we name “instill discipline”, is an indicator for whether the teacher hit or
slapped students during the lesson and scolded while giving feedback during a
lesson. Finally, an indicator named “challenge student intellectually”, is an indicator for whether the teacher assigned or reviewed homework in class and asked
questions or students during the lesson. We view this third measure as similar to
the high academic and behavioral expectations measure that was found to be important by Dobbie & Fryer Jr (2013). For the measures, 83% of teachers instilled
trust in their students, 42% instilled discipline, and 45% challenged students in9
tellectually.
3.1
Teacher and Student Knowledge Exams
The SDI survey data contain test scores of student and teacher knowledge. Within
each school, 10 students in 4th grade were randomly selected and given a mathematics, reading, and non-verbal reasoning exam. Additionally, all teachers that
teach 4th grade mathematics or reading in the current year or taught 3rd grade in
the previous year, as well as 3 to 5 additional teachers in 5th grade or higher are
given exams testing their knowledge of mathematics, reading, and pedagogy.
The student language test is composed of 6 multi-part items on letter and word
identification, sentence and paragraph reading, and comprehension of written material. An average score across the 6 items is computed for each student. The
Cronbach’s alpha, a commonly used measure of test score reliability, for the language test is .872.9 The student mathematics test is composed of 15 items related
to number identification, addition, subtraction, multiplication, and division, sequences, and word problems. The Cronbach’s alpha measure of reliability is .802.
The student’s non-verbal reasoning test is composed of 4 items related to pattern
recognition, with a reliability of .398.
The teacher language test consists of 22 items involving grammar, Cloze, and
composition tasks and has a reliability of .831. The teacher mathematics test
consists of 15 items related to addition, subtraction, multiplication, division, fractions, interpreting graphs and data, and one variable algebra and has a reliability of
.864.10 The teacher pedagogy test consists of 11 items on preparing lesson plans,
9 The
Cronbach’s alpha measure can be interpreted as an estimate of the ratio of the
variance of true student knowledge divided by the variance of observed student knowledge, which contains measurement error. For comparison the 2013 SAT critical reading
exam has a reliability of .91-.93. The mathematics section has a reliability of .92-.94.
Source: Test Characteristics of the SAT: Reliability, Difficulty Levels, Completion Rates, The
College Board. https://secure-media.collegeboard.org/digitalServices/pdf/sat/sat-characteristicsreliability-difficulty-completion-rates-2014.pdf
10 For comparison, the reliability of the teacher knowledge exams used by Metzler & Woessmann (2012) was .64 for reading and .74 for mathematics in their Peruvian data.
10
Table 3: Summary Statistics for SDI Surveys in Uganda, Mozambique, Togo,
Nigeria, and Kenya
Variable
Mean
Student Characteristics
Mathematics % Correct
Reading % Correct
Pupil non-verbal reasoning % Correct
Student’s Age
Student Female
Student Ate Breakfast
Observations
Std. Dev.
41.55
44.83
53.91
11.03
0.49
0.72
19.64
37.30
23.99
2.98
0.50
0.45
17262
Teacher Knowledge
Math Teacher Knowledge % Correct
Reading Knowledge % Correct
Pedagogical Skill % Correct
Observations
53.80
52.68
22.87
28.09
19.74
17.76
7589
Teacher Characteristics
Teacher Male
0.40
Experience
11.97
University Degree
0.55
Teacher has Bachelor of Education Degree
0.10
Head Teacher
0.06
Contract Teacher
0.26
Absent from Classroom (Unannounced Visit)
0.28
Observations
Classroom Observations
Effective Instruction Time (hours)
Scheduled Duration of School Day
Instill Trust
Instill Discipline
Challenge Students Intellectually
Observations
0.49
9.50
0.50
0.30
0.24
0.44
0.45
15550
3.25
5.20
0.83
0.42
0.45
1.76
1.10
0.37
0.49
0.50
1829
School Characteristics
School is in urban location
School Infrastructure Index
School Equipment Index
Pupils per Teacher
School w/ PTA or Gov Board
Private School
School Level NVR Ability
Observations
0.21
0.36
0.64
28.51
0.97
0.22
53.91
0.40
0.48
0.48
21.39
0.17
0.42
13.00
1869
Weighted summary statistics in Uganda, Mozambique, Togo, Kenya, and Nigeria.
11
assessing student abilities, and evaluating student progress and has a reliability of
.665.
4
How Much Do Differences across Classrooms Explain Differences in Student Achievement?
We begin by examining the fraction of the variance in student achievement explained by classroom or teacher effects, which can tell us to what extent these
factors are responsible for differences in student achievement. If a large fraction
of the variance is explainable by classroom factors, then this suggest that policies
targeted at improving these classroom factors could produce large improvements
in student achievement.
We begin with a model of student achievement based on the education production function.
Ai jd = Xi jd δ + µi + τ j + αd + εi jd ,
(1)
where Ai jd is the achievement level of student i in classroom j and district d.
The vector Xi jd contains observable factors affecting achievement. The variable
µi is unobservable student motivation or ability. The variable τ j represents fixed
factors within classrooms, αd is a district effect, and εi jd represents idiosyncratic
unobserved factors affecting achievement.11 The classroom effect, τ j , picks up the
impact of teachers and other fixed factors within classrooms after conditioning on
Xi jd and the district fixed effects.
We estimate the classroom effects using a variety of specifications. In table 4,
we report the fraction of the variance explained by classroom factors, which is the
variance in classroom effects, i.e. the variance of τ j across classrooms, divided by
11 The
districts contain around 25 sampled schools on average, with a maximum of 52 schools.
The definition of a district varies somewhat from country to country. In Uganda, the definition of
a district is sub-county/division geographic unit. In Kenya, it is the district unit. In Nigeria, it is
the LGA. In Togo, it is the region. In Mozambique, it is the district.
12
the total variance of student achievement under these different specifications.
Our identification strategy for the classroom effects is based on a cross sectional regression with a detailed set of covariates. We acknowledge upfront that
our estimates may not be causal, but do believe our covariates may be effective
at reducing a large amount of bias. Two identifying assumption are necessary
to estimate the classroom effects using this approach without bias. First is that
student assignment to classrooms is random conditional on the control variables.
This means that unobserved student ability and motivation must be uncorrelated
with classroom assignment and the other covariates. We offer some suggestive evidence in table 10 in the appendix that this assumption may not be too unrealistic.
As shown in table 10, in many of the countries student’s appear to be nearly randomly distributed in terms of non-verbal reasoning ability, which may suggest that
unobservable factors of student achievement, µi , could also be roughly randomly
distributed.
Second, an assumption needed for the cross sectional estimator used below
is that the covariates controlled for in the regressions are uncorrelated with the
remaining fixed unobserved classroom factors in τ j .12 One way this assumption
could be violated would be if teachers sort to schools based on unobservable characteristics. While we cannot rule this out, we maintain that most sorting of teachers to schools is likely done on the basis of observed factors of teachers, such as
teacher gender, experience, and education, and note that previous research shows
that observable characteristics of teachers are only weakly related to teacher unobservables.13 Therefore, we believe this second assumption may be plausible as
12 We
cannot perform a classroom fixed effects estimator which would relax this assumption,
because we do not have multiple years of data needed to generate variation within classrooms in
teacher and school characteristics.
13 Winters et al. (2012) using data from the state of Florida found that while selection is present
in estimating the impacts of teachers, the bias created is quite small in magnitude. Hanushek
& Rivkin (2006) provides an overview of the evidence of the impacts of teacher experience and
credentials on student achievement. The authors show that among 34 teacher value-added model
studies, 0 showed statistically positive effects of teacher education on student achievement and 9%
showed negative effects. For experience, the authors found that only 41% showed a positive effect
and 3% showed a negative effect.
13
well.
To begin we present the fraction of the variance explained controlling for district fixed effects in column (1) of Table 4. The underlying variance of the classroom effects is estimated directly by MLE, which is equivalent to an HLM regression with students nested within classrooms and a random classroom intercept.
The fraction explained by the classroom effects is .333 in mathematics and .346
in reading. District fixed effects alone may not adequately capture differences in
student background and school conditions, and so the classroom effects may be
overstated.
Our preferred approach is based on a classroom random effects regression
controlling for district fixed effects as well as the student’s non-verbal reasoning
ability, age, gender, whether the student ate breakfast, the native language of the
student, teacher gender, experience, university degree status, education degree status, head teacher status, contract teacher status, whether the school is in an urban
location, whether it is a private school, the school infrastructure index, equipment
index, pupil teacher ratio, the school non-verbal reasoning level, and whether the
school has a PTA or governing board.14 Again, the underlying variance of the
classroom random effects is estimated by MLE.15 With these controls the fraction explained by classroom effects, shown in column (2) of Table 4 is .152 in
mathematics and .139 in reading.
As another check, we also show the results for schools in which there is only
one classroom in columns (4) and (5). In these cases, no within school sorting of
teachers to students is possible, so only between school non-random assignment
can take place. Overall the results are similar to the sample with all teachers.
14 While
some of these factors are classroom factors that affect student achievement, such as
teacher experience and degree status, we choose to control for these variables because sorting of
teachers to students and schools is likely based on these characteristics of teachers. By controlling
for these characteristics, the conditional random assignment assumption of students to classrooms
needed to identify the classroom effects is more likely to be met.
15 The estimation approach is done using Stata’s xtreg command with the mle option specified.
The “panel” variable is the teacher ID. The likelihood is weighted by the inverse probability the
teacher is selected for the sample. The MLE approach for estimating the variance of classroom
effects is similar to that taken in Nye, Konstantopoulos, & Hedges (2004).
14
Table 4: Fraction of Variance in Achievement Explained by Classroom Effects
Mathematics
Fraction Explained
District FE
Student Covariates
Teacher Covariates
School Covariates
All Teachers
.333
.152
X
One Classroom Sample
.327
.156
X
X
X
X
Observations
X
11190
X
X
X
X
9382
Reading
All Teachers
One Classroom Sample
Fraction Explained
.346
.139
.358
.145
District FE
Student Covariates
Teacher Covariates
School Covariates
X
X
X
X
X
X
X
X
X
X
Observations
12395
10258
Variance of classroom effects estimated using a classroom random effects
MLE (HLM) estimator. The likelihood is weighted by the inverse probability of selection for survey. Student covariate set includes: student non-verbal
reasoning ability, age, gender, whether the student ate breakfast, and the native language of the student. Teacher covariate set includes: teacher gender,
experience, university degree status, education degree status, head teacher status, and contract teacher status. School Covariates set includes: whether the
school is in an urban location, whether it is a private school, the school infrastructure index, equipment index, pupil teacher ratio, the school non-verbal
reasoning level, and whether the school has a PTA or governing board.
15
The estimates suggest that fixed classroom factors explain a sizable portion of
the total variation in student achievement. The most plausible estimates indicate
that fixed classroom factors explain anywhere around 15% of the total variation
in student achievement. This is larger than what is typically found in the context
of the United States, where a fraction explained of around 5% is more common.
This may suggest that policies targeting these fixed classroom factors could potentially substantially improve achievement. In the coming sections, we examine
the particular roles of teacher knowledge and classroom instruction practices on
student achievement.
5
Effects of Teacher Human Capital, Instruction Time,
and Practices on Student Achievement
Returning to equation 1, we can express the classroom effects, τ j , as a function of
observable teacher characteristics, as well as unobservable characteristics.
τ j = β Tj + η j ,
(2)
where T j are observable teacher characteristics, and the vector of coefficients β is
the effects of these characteristics on student achievement. The teacher characteristics we examine are teacher human capital, which includes teacher knowledge
and pedagogical skill, effective instruction time in the classroom, and particular
instruction practices of the teacher. All unobserved classroom factors affecting
students is captured by η j .
Equation 1 can be rewritten as:
Ai jd = Xi jd δ + β T j + η j + µi + αd + εi jd ,
(3)
In the remainder of the paper, our goals are to identify β , the effects of teacher
human capital, instruction time, and practices on student achievement, as well as
identify the extent that the observable teacher characteristics explain the variation
16
in the classroom effects. We begin by examining the effects of teacher knowledge
and then move to the effects of teacher knowledge and pedagogical skill, which we
group as teacher human capital, as well as instruction time and teacher practices.
5.1
Effects of Teacher Knowledge
We take two approaches to estimating the effects teacher knowledge on student
achievement. We present estimates from a cross sectional estimator which includes district fixed effects and student, teacher, and school covariates. We also
present estimates using a correlated random effects estimator similar to that used
in Metzler & Woessmann (2012). These two alternate estimators rely on different assumptions for identification. Both estimators provide estimates suggesting
a moderate impact of teacher knowledge on student achievement.
To begin, in Table 5 we present the results from the cross sectional regression
of student achievement on the teacher, student, and school characteristics. The
coefficients are estimated using a classroom random effects regression similar to
column (2) of Table 4, except augmented with the teachers mathematics or reading knowledge. The important identifying assumption under these cross sectional
regressions is that the set of covariates is sufficiently rich to render the remaining unobserved factors affecting achievement uncorrelated with the covariates of
interest. We condition on district fixed effects, student’s age, gender, non-verbal
reasoning ability, whether the student ate breakfast, and the student’s native language. We also condition on the teacher’s gender, experience, education level,
whether the teacher has an education degree. Finally, we condition on whether
the school is in an urban location, a school infrastructure index and an equipment
index, the pupil-teacher ratio, whether the school has a PTA or governing board,
and the school level non-verbal reasoning score. Results are reported in units of
standard deviations of student achievement.
Results for all teachers are presented in the first two columns. The estimate of
the effect of math teacher knowledge is .105, meaning that a one standard deviation increase in teacher mathematics knowledge increases student achievement by
17
.105 standard deviations. This estimate is statistically significant at the 1% level.
The effect for reading is .064 and is also statistically significant at the 1% level.16
In the next two columns, we report results restricting the sample to teachers that
serve as both the mathematics and reading teacher. This restriction is needed for
the correlated random effects results presented below, so we present these results
to provide a direct comparison. The results restricting the sample to only students
with the same teacher in both subjects are very similar, with an effect for math of
.117 and an effect in reading of .062.17 In the final two columns, the results for
students who have the same teacher in both subjects and are in schools with only
one classroom per grade are presented. For this set of teachers, no within school
sorting of teachers to students is possible. The results are slightly larger again for
math, with an effect of .134 and effect of .072 in reading.18
16 In similar regressions, Metzler & Woessmann (2012) estimate an effect of .094 for math and
.027 for reading.
17 Metzler & Woessmann (2012) estimate an effect of .073 and .032 in this case.
18 Metzler & Woessmann (2012) estimate an effect of .061 and .017 with this restriction.
18
Table 5: Classroom Random Effects Regression Results of Effects of Teacher
Knowledge on Student Achievement
All Teachers
variable
Same Teacher Sample
Same Teacher - One Class
Mathematics
Reading
Mathematics
Reading
Mathematics
Reading
Teacher Knowledge
0.105***
(0.017)
0.064***
(0.016)
0.117***
(0.0204)
0.062***
(0.019)
0.134***
(0.023)
0.072***
(0.020)
District FE
Classroom RE
Student Covariates
Teacher Covariates
School Covariates
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Observations
Number of Teachers
11,297
1,343
12,251
1,427
7,434
862
7,438
864
6,334
735
6,328
739
Effects estimated using a classroom random effects effects regression. Student covariate set includes: student non-verbal reasoning ability, age, gender, whether the student ate breakfast, and the native language of
the student. Teacher covariate set includes: teacher gender, experience, university degree status, education
degree status, head teacher status, and contract teacher status. School Covariates set includes: whether the
school is in an urban location, whether it is a private school, the school infrastructure index, equipment
index, pupil teacher ratio, the school non-verbal reasoning level, and whether the school has a PTA or governing board. Standard Errors clustered at school level in parenthesis. * Denotes significant at 10% level.
** Denotes significant at 5% level. *** Denotes significant at 1% level.
19
A concern with the cross sectional regressions is that unobserved student ability µi is correlated with the covariates. As a check, we estimate the effects using
a correlated random effects approach, similar to the approach laid out in Metzler
& Woessmann (2012), in Table 6. The correlated random effects approach relies
on stronger functional form assumptions but can also remove unobserved student
ability if the assumptions are met.
The CRE approach involves the same regression as the cross sectional regression, except the teacher’s knowledge level in the other subject is added (as well
as potentially any other covariate that varies across subject).19 In this case, following Metzler & Woessmann (2012), we use only student observations where
the same teacher teaches both mathematics and reading. The estimated effect for
math is generated by subtracting the estimated coefficient for teacher mathematics
knowledge from the reading regression from the estimated coefficient for teacher
mathematics knowledge from the math regression. The effect for reading is generated by subtracting the estimated coefficient on teacher reading knowledge from
the math regression from the estimated coefficient for teacher reading knowledge
from the reading regression. Intuitively, if the teacher’s knowledge in the alternate
subject only is related to student achievement through the unobserved student heterogeneity, µi , then by subtracting the estimated coefficient in the alternate subject
equation from the estimated coefficient in the actual subject, bias from the unobserved heterogeneity is differenced away, although this relies on strong functional
form assumptions.20 More details of the correlated random effects approach can
be found in the appendix.
The original coefficients prior to the subtraction are shown at the top of Table
6. The coefficient on mathematics teacher skill in the reading regression is .059
19 As
it turns out, none of the other covariates vary across subject, so teacher knowledge is the
only variable added in the CRE approach.
20 This approach requires the following functional form assumptions: unobserved student ability
must be identical for both mathematics and reading, teacher knowledge must be related linearly
to unobserved student ability, and teacher knowledge in the alternate subject must have no relationship to the student’s achievement in the actual subject except through the relationship with the
unobserved ability of the student.
20
and is statistically significant at the 1% level. The coefficient on reading skill in
the mathematics regression is .029 and insignificant.
The estimated effect for teacher knowledge in mathematics is .050, and this
is statistically significant at the 1% level. The coefficient for reading is .020 and
insignificant.21 The estimates using the CRE estimator tend to be smaller than
the cross sectional results. This could be an indication of bias due to unobserved
student heterogeneity in the cross sectional results, or it could be that the stronger
functional form restrictions imposed by the CRE estimator are not met. Although
the CRE estimates are smaller, both sets of estimates suggest a moderate impact
of teacher knowledge on student achievement. The two estimates give a range
of the effect of mathematics knowledge on student achievement of between .05
and roughly .11 for mathematics and a range of a null effect up to roughly .06 for
reading.
Table 6: CRE Regression Results of Student Achievement on Teacher Knowledge:
Same Teacher Sample
variable
Teacher Math Knowledge
Teacher Reading Knowledge
Implied Effect of Teacher Knowledge
Mathematics
Reading
0.109***
(0.021)
0.029
(0.021)
0.059***
(0.020)
0.048**
(0.020)
0.050***
(0.017)
0.020
(0.016)
Regressions include district fixed effects and student, teacher, and school covariates. Standard Errors clustered at school level in parenthesis. * Denotes significant at 10% level.
** Denotes significant at 5% level. ** Denotes significant at 1% level.
21 Metzler
& Woessmann (2012) estimate an effect of .064 for mathematics and an insignificant
.014 for reading using a similar approach.
21
5.2
Effects of Teacher Human Capital, Instruction Time, and
Practices
In the next section, we examine the roles of human capital, instruction time, and
particular teacher practices in raising student achievement. These three dimensions of teacher quality capture the following: do the teachers have the skills to
teach (human capital), do they spend time in the classroom (effective instruction
time), and what do the teachers do in the classroom. Returning to equation (2),
we can write the teacher effect as a function of the three dimensions and unobservables η j .
τ j = F(H j , I j , Pj ) + η j ,
(4)
where H j is a vector of human capital characteristics of the teacher, I j is instruction time, and Pj are the instruction practices of the teacher. We model (4) as a
linear function of the characteristics, but allow for an interaction between human
capital and instruction time to test whether additional instruction time is particularly effective after a teacher reaches a threshold in terms of human capital.
τ j = H j β1 + I j β2 + I j H ∗j β3 + Pj β4 + η j ,
(5)
where H ∗j denotes a teacher with high levels of human capital.22
We use the subject knowledge examinations and the teacher pedagogical examinations as measures of teacher human capital. As discussed previously, the
pedagogical tests examined a teachers ability to prepare a lesson, assess student
abilities, and evaluate student progress in learning.23 We form a measure of effective instruction time measure by combining data from the staff roster module
22 For
our purposes, this means that the teacher answered more than 50% of the teacher subject
knowledge questions correctly and more than 25% of the pedagogy questions correctly. These
levels of subject knowledge and pedagogical skill are roughly at the average for the sample of
teachers.
23 The teachers are asked to prepare a lesson based on a news article presented to them. Then
they are asked to read two paragraphs from two hypothetical students and judge the strengths and
weakness. Finally, they are asked to evaluate a set of test scores for students.
22
(used to measure absence rate), the classroom observation module, and reported
teaching hours for the school day. The instruction time for the school day is adjusted for the time teachers are absent from the classroom, on average, and for
the time the teacher teaches while in classrooms based on classroom observations
recorded every 1 minute in a teaching lesson.24 We examine three particular classroom practices: the teacher instilling trust in the students, the teacher instilling
discipline in the student, and the teacher challenging students intellectually.25
The classroom observation only took place for either a mathematics lesson
or a reading lesson but not both. This means that it is not possible to produce
CRE estimates of the effects of teacher practices on student achievement, since
we would need variation across subjects to identify the effects. However, given
that in the previous analysis that the estimated effects of teacher knowledge on
student achievement using the cross sectional regression were broadly similar to
the estimates formed using the CRE estimator as well as the other analysis on
student sorting in the data, we feel encouraged that widespread non-random selection of teachers to students based on unobservable factors is not taking place.
That said, we present the cross sectional results below, acknowledging that they
not be causal.
The first two columns of Table 7 shows the effects of human capital alone on
student achievement. In the middle two columns, human capital and instruction
time are included as covariates. Additionally, in order to test for a particularly
strong effect for instruction time if instruction time is paired with high human
capital, instruction time is interacted with an indicator for whether the teacher has
a high level of human capital, which is defined as whether the teacher answered
more than 50% of the teacher subject knowledge questions correctly and more
than 25% of the pedagogy questions correctly. These levels of subject knowl24 The
school average classroom absence rate is used a measure for absenteeism in this measure.
“instill trust” measure is an indicator for whether the teacher called on students by name
and praised students while giving feedback. The “instill discipline” measure is an indicator for
whether the teacher hit or slapped students during the lesson and scolded while giving feedback
during a lesson. The “challenge student intellectually” measure is an indicator for whether the
teacher assigned or reviewed homework in class and asked questions or students during the lesson.
25 The
23
edge and pedagogical skill are roughly at the average for the sample of teachers.
Finally, in columns 5 and 6, particular teacher practices are also included as covariates.
Both forms of human capital, teacher knowledge and pedagogical skill, have
a statistically significant impact on student achievement. The estimates suggest
that teacher subject knowledge is an important predictor of student achievement
even after conditioning on pedagogical skill. The coefficient in mathematics is
.087 in mathematics and .033 in reading in columns 1 and 2, which is modestly
smaller than the estimates of .105 and .064 reported in the previous section. The
coefficient on teacher pedagogical skill for mathematics is .046 and statistically
significant at the 1% level, meaning a one standard deviation increase in pedagogical skill increases students achievement in math by .046 test score standard
deviations. The coefficient for reading is .101 and significant at the 1% level. The
estimates for teacher knowledge and pedagogical skill are similar in columns 3
through 6 after controlling for instruction time and practices.
Teacher instruction time also has a statistically significant positive relationship for math and reading. The estimated coefficient for mathematics, shown
in column 3, is .034, suggesting that an additional hour of effective instruction
time increases student achievement by .034 test score standard deviations. The
estimated coefficient on reading is .054. Interestingly, the estimated interaction
effects with human capital are statistically insignificant, meaning there is little evidence that instruction time has a particularly strong effect for teachers with high
human capital. Lavy (2010) reports an effect size of .07 standard deviations for
an additional hour of instruction time per week in the context of PISA countries.
This implies the effect of an additional hour per day is roughly .35 in his data,
which is much larger than the effect estimated in our data. Dobbie & Fryer Jr
(2013) reports an effect of .05 standard deviations in math for an increase of 25%
in instruction time over the average number of hours in public schools. Assuming
the average number of hour per day in public schools is around 6.5, which implies
an increase of around 1.625 hours, this implies an effect per additional hour of
24
around .03 standard deviations, which is similar to what we find.
Of the specific classroom practices, which are all binary indicators of whether
or not the teacher engaged in this practice during their classroom observation,
the practice of instilling trust has a positive and statistically significant impact on
mathematics and reading. The estimated coefficient for mathematics is .073 and
the estimate for reading is .069. The effect for the practice of instilling discipline
has a negative point estimate for both mathematics and reading but is insignificant. The effect of challenging students intellectually by assigning homework and
asking questions is statistically significant for math and reading. The estimated
effect for mathematics is .093 and the estimate for reading is .159. For comparison, Dobbie & Fryer Jr (2013) found that charter schools with high academic and
behavioral expectations had .044 standard deviation higher mathematics and .03
standard deviation higher reading achievement. This measure we view as broadly
similar to our measure of challenging students intellectually.
5.3
How Much are Classroom Effects Explained by Teacher
Knowledge and Practices
In the next section, we examine the extent that human capital, instruction time, and
classroom practices explain the variation in classroom effectiveness documented
in section 4. In section 4, we examined the fraction of the variance in student
achievement explained by fixed classroom factors, τ j in equation 1:
Ai jd = Xi jd δ + µi + τ j + αd + εi jd .
(1)
With τ j rewritten as a function of observables and unobservables,
τ j = β Tj + η j ,
(2)
Ai jd = Xi jd δ + β T j + η j + µi + αd + εi jd ,
(6)
and rewriting 1 as
25
Table 7: Estimated Effects of Teacher Human Capital, Instruction Time, and
Teacher Practices on Student Achievement
Variables
Teacher Knowledge
Pedagogical Skill
Human Capital
Human Capital &
Instruction Time
Human Capital,
Instruction Time,
& Practices
Math
Reading
Math
Reading
Math
Reading
0.087***
(0.018)
0.046***
(0.015)
0.033**
(0.016)
0.101***
(0.015)
0.085***
(0.021)
0.054**
(0.022)
0.034***
(0.011)
-0.013
(0.012)
0.030
(0.021)
0.105***
(0.022)
0.054***
(0.011)
-0.018
(0.011)
0.088***
(0.021)
0.051**
(0.022)
0.034***
(0.011)
-0.014
(0.012)
0.073**
(0.037)
-0.138
(0.092)
0.093**
(0.041)
0.028
(0.021)
0.104***
(0.022)
0.053***
(0.011)
-0.018
(0.011)
0.069*
(0.040)
-0.087
(0.076)
0.159***
(0.040)
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Effective Instruction Time (hours)
Instruction Time - High Human Capital
Instill Trust
Instill Discipline
Challenge Students Intellectually
District FE
Classroom RE
Student Covariates
Teacher Covariates
School Covariates
Observations
11274
12216
8284
8574
8080
8564
Number of Teachers
1342
1424
959
967
933
966
Effects estimated using a classroom random effects effects regression. Student covariate set includes: student non-verbal reasoning ability, age, gender, whether the student ate breakfast, and the native language of
the student. Teacher covariate set includes: teacher gender, experience, university degree status, education
degree status, head teacher status, and contract teacher status. School Covariates set includes: whether the
school is in an urban location, whether it is a private school, the school infrastructure index, equipment
index, pupil teacher ratio, the school non-verbal reasoning level, and whether the school has a PTA or governing board. High human capital is defined as teacher answering more than 50% of the subject knowledge
questions correctly and 25% of the pedagogy questions.
26
we can examine the fraction of the variance in student achievement explained by
η j , which is the remaining impact of teachers after netting out human capital,
instruction time, and practices. If, hypothetically, these teacher characteristics are
driving differences in classroom effectiveness, then once these factors are netted
out, there should be little remaining variation in classroom effectiveness captured
by η j .
In Table 8, we compare the estimates of the fraction of the variance explained
by τ j to the fraction explained by η j . Again we estimate the variance of τ j using MLE in a classroom random effects regression controlling for district fixed
effects, as well as student, teacher, and school characteristics. To estimate the
variance of η j , we perform the same classroom random effects regressions, except additionally control for teacher knowledge and human capital, instruction
time, and classroom practices. Because the regressions that include human capital, instruction time, and teacher practices have fewer observations available than
the regressions without these variables, we estimate all regressions using the subset of observations available for the human capital, instruction time, and practices
regression. Column 1 of Table 8 shows the fraction explained by τ j . Column 2
shows the estimates of η j , after controlling only for teacher knowledge. Column
3 shows the estimates of η j , after controlling for human capital, instruction time,
and practices. The specification of column 3, corresponds to that of columns 5
and 6 of Table 7.
The estimates of the fraction explained by τ j versus η j suggest that the observable characteristics of teachers explain only a small amount of the variation in
classroom effectiveness. In mathematics, the fraction explained by τ j is .14, while
the fraction explained by η j in the specification with only teacher knowledge is
.135, and the fraction explained in the specification with teacher knowledge and
practices is also .13. In reading, the fraction explained by τ j is .137, while the
fraction explained with only teacher knowledge is .137, and the fraction explained
with teacher knowledge and practices is .132.
The estimates suggest that while improving teacher knowledge and practices
27
can improve student achievement, unobservables of teachers and classroom seem
to account for a much larger share of the variation in classroom effectiveness.
Table 8: Fraction of Variance in Achievement Explained by Classroom Effects
after Accounting for Teacher Knowledge and Practices
Mathematics
Overall
Knowledge
Human Capital,
Instruction Time,
& Practices
Fraction Explained
.139
.135
.13
District FE
Student Covariates
Teacher Covariates
School Covariates
X
X
X
X
X
X
X
X
X
X
X
X
Observations
8080
Reading
Fraction Explained
.137
.137
.132
District FE
Student Covariates
Teacher Covariates
School Covariates
X
X
X
X
X
X
X
X
X
X
X
X
Observations
8564
Classroom effects estimated using a classroom random effects effects regression. Student covariate set includes: student non-verbal reasoning ability, age, gender, whether the student ate
breakfast, and the native language of the student. Teacher covariate set includes: teacher gender,
experience, university degree status, education degree status, head teacher status, and contract
teacher status. School Covariates set includes: whether the school is in an urban location,
whether it is a private school, the school infrastructure index, equipment index, pupil teacher
ratio, the school non-verbal reasoning level, and whether the school has a PTA or governing
board.
28
6
Conclusions
In this paper, we find that classroom effects explain roughly 15% of the total
variance of student achievement in Sub-Saharan African countries. This is the
first paper that we know of to do this calculation. The percentage explained is
larger than what is typically seen in the context of the United States, which is
typically roughly 5%.
Additionally, we find evidence that the classroom effects are at least partially
driven by what teachers know, whether the teachers know how to teach, how they
spend time in the classroom, as well as their practices in the classroom. A one
standard deviation increase in teacher knowledge in mathematics increases student achievement by around .05 to .11 standard deviations. The effect for reading
is between null and .06 standard deviations. A 1 standard deviation increase in
pedagogical skill increases achievement by around .05 standard deviations in math
and .10 standard deviations in reading. An additional hour of effective teaching,
or time on task, increases student achievement by .034 standard deviations in math
and .054 standard deviations in reading. Instilling trust in the students is associated with an increase in student achievement of .073 standard deviations in math
and .069 in reading. Challenging students intellectually also has a statistically
significant positive effect. The estimated effect for math is .093 and the estimated
effect for reading is .159.
Finally, we find that even though these characteristics of teachers do predict
student achievement, they account for only a small part of the differences between
classrooms in effectiveness. After accounting for these variables, remaining unobserved variation in classroom quality explains only a slightly smaller amount
of variation in student performance than previously. The fraction explained drops
from around .14 to .13.
This paper provides a detailed look at factors affecting differences in classroom effectiveness in Sub-Saharan Africa. The results suggest that ways of improving teacher knowledge and practices should be examined by policy makers,
although there may be more room to improve teacher quality in other ways. For
29
instance, this could involve coming up with ways to attract teachers from higher
in the teacher quality distribution or by finding ways to remove particularly ineffective teachers. More research is needed to examine the remaining unobservable
factors affecting classroom performance in Sub-Saharan Africa.
References
Aaronson, D., Barrow, L., & Sander, W. (2007). Teachers and student achievement
in the chicago public high schools. Journal of Labor Economics, 25(1), 95–135.
Chamberlain, G. (1982). Multivariate regression models for panel data. Journal
of Econometrics, 18(1), 5–46.
Dobbie, W., & Fryer Jr, R. G. (2013). Getting beneath the veil of effective schools:
Evidence from new york city. American Economic Journal: Applied Economics, 5(4), 28–60.
Filmer, D., Martin, G., Molina, E., Stacy, B., Rockmore, C., & Wane, W. (2014).
The missing link: Why Increasing Resources Alone will not Amount to Higher
Learning in Sub-Sahara Africa. The World Bank.
Goldhaber, D., & Hansen, M. (2013). Is it just a bad class? Assessing the longterm stability of estimated teacher performance. Economica, 80(319), 589 –
612.
Hanushek, E. A., & Rivkin, S. G. (2006). Teacher quality. Handbook of the
Economics of Education, 2, 1051–1078.
Hanushek, E. A., & Wößmann, L. (2007). The role of education quality for economic growth. World Bank Policy Research Working Paper, (4122).
Harris, D. N., & Sass, T. R. (2011). Teacher training, teacher quality and student
achievement. Journal of public economics, 95(7), 798–812.
30
Hein, C. F., & Allen, R. (2013). Teacher quality in sub-saharan africa: Pupilfixed effects estimates for twelve countries. Department of Quantitative Social
Science-Institute of Education, University of London.
Lavy, V. (2010). Do Differences in Schools Instruction Time Explain International
Achievement Gaps? Evidence from Developed and Developing Countries. National Bureau of Economic Research, Inc.
URL http://ideas.repec.org/p/nbr/nberwo/16227.html
Lavy, V. (2011). What Makes an Effective Teacher? Quasi-Experimental Evidence. National Bureau of Economic Research, Inc.
URL http://ideas.repec.org/p/nbr/nberwo/16885.html
McCaffrey, D. F., Lockwood, J., Koretz, D., Louis, T. A., & Hamilton, L. (2004).
Models for value-added modeling of teacher effects. Journal of educational
and behavioral statistics, 29(1), 67–101.
Metzler, J., & Woessmann, L. (2012). The impact of teacher subject knowledge
on student achievement: Evidence from within-teacher within-student variation. Journal of Development Economics, 99(2), 486–496.
URL
http://ideas.repec.org/a/eee/deveco/
v99y2012i2p486-496.html
Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large are teacher
effects? Educational evaluation and policy analysis, 26(3), 237–257.
Shepherd, D. (2013). The impact of teacher subject knowledge on learner performance in south africa: A within-pupil across-subject approach.
Winters, M. A., Dixon, B. L., & Greene, J. P. (2012). Observed characteristics
and teacher quality: Impacts of sample selection on a value added model. Economics of Education Review, 31(1), 19–32.
Wiswall, M. (2013). The dynamics of teacher quality. Journal of Public Economics, 100, 61–78.
31
A
Sorting of Teachers and Students Across Schools
One may worry to what extent teachers with high levels of subject knowledge sort
to be with similar teachers. For instance, one may be worried that teachers with
high levels of subject knowledge sort into schools with high performing students.
In order to examine this, we performed an analysis similar to what was done in
the Aaronson et al. (2007), who examine student sorting in their data. To examine
sorting, we compute the within-school standard deviation of teacher knowledge
in the actual data. If there is a lot of sorting, the within-school standard deviation
should be low, since similar teachers are teaching at schools. We then compared
this to the within-school standard deviation that would exist if teachers were randomly sorted into schools using a simulation, and also compared it to the case
where teachers are perfectly sorted into schools by their subject knowledge.
After comparing the numbers, reported in table 9 it looks like the data looks
more random than perfectly sorted. Nigeria is an outlier, where it looks like teachers are sorted more than the other countries.
32
Table 9: Average Within-School Standard Deviations of Teacher Knowledge for
Actual Distribution of Teachers Across Schools, Randomly Distributed Teachers
Across Schools, and Perfectly Sorted Teachers Across Schools
variable
Actual
Random Perfect Sorting
Kenya
Math Teacher Skill
Reading Teacher Skill
Teacher Ped Skill
.725
.796
.801
Number of Teachers
Number of Schools
.846
.899
.974
.009
.018
.006
1679
306
Nigeria
Math Teacher Skill
Reading Teacher Skill
Teacher Ped Skill
.588
.612
.597
Number of Teachers
Number of Schools
.914
.891
.943
.004
.004
.004
2406
759
Uganda
Math Teacher Skill
Reading Teacher Skill
Teacher Ped Skill
.879
.886
.852
Number of Teachers
Number of Schools
.953
.950
.961
.006
.006
.005
2162
400
Togo
Math Teacher Skill
Reading Teacher Skill
Teacher Ped Skill
.795
.796
.859
Number of Teachers
Number of Schools
.957
.934
.93
.008
.011
.014
831
198
Mozambique
Math Teacher Skill
Reading Teacher Skill
Teacher Ped Skill
.71
.875
.749
Number of Teachers
Number of Schools
.929
.897
.868
.015
.018
.013
550
190
33
In table 10, we perform a similar exercise to examine the amount of student
variation within and across schools. This analysis can help get at whether high
performing/ability or low performing/ability students are concentrated in certain
schools. If one is to take the non-verbal reasoning scores of students as a measure
of innate IQ that isn’t affected directly by schooling, then the degree of sorting for
non-verbal reasoning may indicate whether or not more innately gifted students
are concentrated in certain schools.
Overall, again the actual distribution of student knowledge looks surprisingly
random. In particular, the actual within school standard deviation of non-verbal
reasoning ability is much closer to the case of randomly distributed ability than
perfectly sorted ability. Again, Nigeria is the outlier. In this case, the actual
within school standard deviation of non-verbal reasoning knowledge is .813, while
the standard deviation for the randomly distributed case is .975. The standard
deviation for the perfectly sorted case is .001 however.
34
Table 10: Average Within-School Standard Deviations of Student Knowledge for
Actual Distribution of Students Across Schools, Randomly Distributed Students
Across Schools, and Perfectly Sorted Students Across Schools
variable
Actual Random Perfect Sorting
Kenya
Student Non-Verbal Reasoning Knowledge
Student Math Knowledge
Student Reading Knowledge
.898
.822
.682
Number of Students
Number of Schools
.966
.966
.915
.005
.007
.005
2951
306
Nigeria
Student Non-Verbal Reasoning Knowledge
Student Math Knowledge
Student Reading Knowledge
.813
.625
.542
Number of Students
Number of Schools
.975
.969
.992
.001
.003
.002
6568
759
Uganda
Student Non-Verbal Reasoning Knowledge
Student Math Knowledge
Student Reading Knowledge
.91
.764
.625
Number of Students
Number of Schools
.968
.978
.998
.004
.005
.003
3957
399
Togo
Student Non-Verbal Reasoning Knowledge
Student Math Knowledge
Student Reading Knowledge
.903
.774
.704
Number of Students
Number of Schools
.976
.971
.997
.006
.012
.005
1927
195
Mozambique
Student Non-Verbal Reasoning Knowledge
Student Math Knowledge
Student Reading Knowledge
.881
.843
.667
Number of Students
Number of Schools
.984
.964
.925
1761
203
35
.006
.01
.006
B
Correlated Random Effects Methodology
In order to examine the effects of teacher knowledge and potentially other teacher
and classroom factors, assume the following education production function:
Ai jd1 = β1 T j1 + γU j1 + Zi α + Xi jd1 δ + µi + τ j1 + εi j1
(7a)
Ai jd2 = β2 T j2 + γU j2 + Zi α + Xi jd2 δ + µi + τ j2 + εi j2 ,
(7b)
where again Ai jdm is the achievement level of student i with teacher j in district
d and subject m. The covariate T jm is the teacher j’s subject knowledge level in
subject m, and U jm consists of other subject specific teacher characteristics. The
covariates in Zi are non-subject specific covariates and Xi jdm are subject specific
covariates of students, teachers, and schools. In the error term, we have unobserved student specific heterogeneity µi , remaining teacher and classroom specific
unobserved heterogeneity τ jm , and an idiosyncratic error εi jdm .
As noted in Metzler & Woessmann (2012), a correlation between the unobserved characteristics and the covariates T jm , U jm , Zi , and Xi jdm makes identification of the effects of these characteristics much more difficult. We can take the
approach of Chamberlain (1982) to deal with correlation with µi by writing:
µi = η1 T j1 + η2 T j2 + θ1U j1 + θ2U j2 + Zi χ + Xi jd1 φ1 + Xi jd2 φ2 + ωi ,
(8)
where ωi is the projection error and is uncorrelated with the regressors. Collecting
terms we have:
Ai j1 = (β1 + η1 )T j1 + η2 T j2 + (γ + θ1 )U j1 + θ2U j2 + Zi (α + χ)
(9a)
+Xi jd1 (δ + φ1 ) + Xi jd2 φ2 + ωi + τ j1 + εi j1
Ai j2 = η1 T j1 + (β2 + η2 )T j2 + θ1U j1 + (γ + θ2 )U j2 + Zi (α + χ)
+Xi jd1 φ1 + Xi jd2 (δ + φ2 ) + ωi + τ j2 + εi j2
36
(9b)
Metzler & Woessmann (2012) propose examining only students who have the
same teacher in both math and reading to simplify equations (9a) and (9b). After
making this restriction, U j and τ j are constant across subjects and we have the
simplified model:
Ai j1 = (β1 + η1 )T j1 + η2 T j2 + (γ + θ1 + θ2 )U j + Zi (α + χ)
(10a)
+Xi jd1 (δ + φ1 ) + Xi jd2 φ2 + ωi + τ j + εi j1
Ai j2 = η1 T j1 + (β2 + η2 )T j2 + (γ + θ1 + θ2 )U j + Zi (α + χ)
(10b)
+Xi jd1 φ1 + Xi jd2 (δ + φ2 ) + ωi + τ j + εi j2 .
While the correlated random effects specification is useful for dealing with
a correlation between µi and the covariates, a remaining issue is the potential
correlation between unobserved teacher attributes, such as ability or motivation,
contained in τ j .
In order to remove τ j , we can use the sample of students who have the same
teacher in math as reading and if we assume that β1 = β2 , η1 = η2 , and φ1 = φ2 ,
then we can first difference the equations (10a) and (10b) to produce:
Ai j1 − Ai j2 = β (T j1 − T j2 ) + φ (Xi jd1 − Xi jd2 ) + εi j1 − εi j2 .
37
(11)