What Makes an Effective Teacher? Quasi-Experimental Evidence
August 2011
Victor Lavy
Department of Economics, Hebrew University of Jerusalem
University of Warwick, NBER and CEPR
Abstract
This paper measures empirically the relationship between classroom teaching practices and student
achievements. Based on primary- and middle-school data from Israel, I find very strong evidence that
two important elements of teaching practices cause student achievements to improve. In particular,
classroom teaching that emphasizes the instilment of knowledge and comprehension, often termed
―traditional‖-style teaching, has a very strong and positive effect on test scores, particularly among girls
and pupils of low socioeconomic background. Second, the use of classroom techniques that endow pupils
with analytical and critical skills (―modern‖ teaching) has a very large positive payoff, evidenced in
improvement of test scores across subgroups differentiated by gender and socioeconomic background.
However, a second element of modern teaching, instilment of the capacity for individual study, has no
effect while transparency, fairness, and proper feedback in teachers‘ conduct with their students improve
marginally academic performance, especially among boys. Apart from identifying ―what works‖ in the
classroom, these findings yield two insights for the debate about the merit of ―traditional‖ versus
―modern‖ approaches to teaching, which are often discussed as rival classroom pedagogical approaches.
First, both may coexist in the classroom production function of knowledge. Second, it is best to target the
two teaching practices differentially to students of different genders and abilities. The effect of the
effective teaching practices estimated is very large, especially in comparison with that of other potential
interventions such as reducing class size or increasing school hours of instruction.
JEL No. I2, I21, J24

I thank Michael Beenstock, Brian Jacob, Daniel Paserman, Steve Pischke, Jonah Rockoff, Yona Rubinstein,
Analia Schlosser and seminar participants at LSE, Hebrew University, Paris School of Economics, Tel Aviv
University, University of London, CEP 2010 Annual Conference, Israeli Economic Association 2011 Meeting and
the NBER 2011 Summer Institute Education Conference for most useful discussions and suggestions. Special
thanks go to Daniel Schuchalter and Agnia Galesnik for their excellent research assistance.
1.
Introduction
While teacher quality may be important, it is driven by characteristics that are difficult or impossible to
measure. This is often the conclusion of many past and recent studies that failed to produce consistent
evidence linking pupils‘ achievement to observable teacher characteristics (Hanushek, 1986). As an
alternative, researchers have tried recently to separate student achievements into a series of ―fixed
effects,‖ assigning importance to teachers, schools, pupils, and so on. For example, Rockoff (2004),
Rivkin et al. (2005), Kane et al. (2008b), and Aaronson et al. (2007) demonstrate substantial and
persistent variations in achievement growth among students assigned to different teachers. An even more
recent strand of research tries anew to identify specific characteristics of teachers that affect pupils‘
achievements. In contrast to older studies that examined mostly the effect of teachers‘ demographic and
educational characteristics , these new studies (e.g., Kane, Rockoff and Staiger, 2008b, Rockoff et al.,
forthcoming, Rockoff and Staiger, forthcoming) focus on characteristics such as cognitive ability,
content knowledge, personality traits, and personal beliefs regarding self-efficacy.
In this paper, I shift the attention from teachers‘ personal characteristics and attributes to what they
do in the classroom in an attempt to identify the most effective teaching practices. In particular, I
measure teaching practices on the basis of data obtained from surveys among primary- and middleschool students in Israel in 2002–2005. Students are asked about the intensity that their teachers engage
in specific activities. The idea that students can characterize what their teachers are doing in the
classroom raises a number of measurement and interpretation issues which I discuss in the data and
methodology sections. However, other researchers and organizations recently endorsed and implemented
this approach in other settings. For example, The Bill &Melinda Gates Foundation project ―Measures of
Effective Teaching,‖ launched in fall 2009 also uses a survey among students to measure perceptions of
the classroom instructional environment (Bill and Melinda Gates Report, 2010) and their effect on
students‘ value added. Their Tripod survey, like the Israeli surveys, asks students if they agree or
disagree with a variety of statements about their teachers‘ classroom practices. Many of its questions
resemble those in the Israeli survey.
1
I use the conceptual categorization of teachers‘ pedagogical practices as developed in the
educational-psychology literature (Bloom, 1956) to summarize information based on 29 dimensions of
pedagogy in five aggregated measures of teaching practices. These measures are (1) instilment of
knowledge and enhancing comprehension; (2) instilment of applicative, analytical, and critical skills;
(3) instilment of the capacity for individual study; (4) transparency, fairness, and feedback, and
(5) individual treatment of students.1 I use three different ways of aggregating the students‘ responses and
then I examine which of these characteristics cause students‘ test-score outcomes to improve. Even
though individually only a few of the 29 teaching characteristics have statistically significant effects on
student outcomes, the first two factors of these teaching practices clearly have a large and statistically
significant effect on students‘ test scores. I also find significant effects for some of the other factors by
allowing for heterogeneity in treatment by students‘ gender and socioeconomic status.
This paper makes its main contribution by identifying multiple ways of measuring effective
teaching for pupils‘ learning. The evidence that I present here, however, also provides insights of
relevance for the ongoing policy debate about the relative merit of ―traditional‖ versus ―modern‖
methods of classroom teaching. In some countries, recent policy action has replaced traditional teaching
methods with modern ones or vice versa. In the U.S., for example, the National Standards recommend
modern teaching practices that engage students in self- and group-learning activities (National Council of
Teachers of Mathematics, 1991, National Research Council, 1996).2 In England, conversely, Michael
1
The Tripod questions in The Bill &Melinda Gates Foundation project are summarized in eight principal
components. One of them, ―Consolidate,‖ strongly resembles Bloom‘s definition of ―instilment of knowledge and
enhancement of comprehension.‖ A second component, ―Challenge,‖ largely overlaps Bloom‘s ―instilment of
applicative, analytical and critical skills.‖ The other three principal components that I use do not overlap perfectly
with the other components in the Gates Foundation report although, as I noted above, many other similar individual
items appear in both. See Bill and Melinda Gates Report (2010) for further detail.
2
Zemelman, Daniels, and Hyde (1993 and 2005) provide a normative typology of teaching practices for schools in
the U.S. Traditional practices that should be decreased, they say, include rote practice, rote memorization of rules
and formulas, single answers and single methods of finding answers, the use of drill worksheets, repetitive written
practice, teaching by telling, teaching computation out of context, stressing memorization, testing for grades only,
and being the dispenser of knowledge. Modern teaching practices that should be put to greater use are manipulative
materials, cooperative group work, discussion of mathematics, questioning and making conjectures, justification of
thinking, writing about mathematics, a problem-solving approach to instruction, content integration, use of
calculators and computers, facilitating learning, and assessing learning as an integral part of instruction.
2
Gove, the Secretary of Education, announced shortly after taking up his post in the summer of 2010 a
reform that would reintroduce traditional teaching and learning in schools.3 In the summer of 2008, the
Israel Ministry of Education unveiled an opposite reform.4 The empirical evidence that I present in this
paper addresses these policy initiatives directly, since ―instilment of knowledge and enhancement of
comprehension‖ includes a variety of classroom practices that are viewed as the core of traditional
teaching, requiring students to acquire knowledge through drill, practice, and memorization (Salomon,
Perkins, and Theroux, 2001). The ―instilment of applicative, analytical, and critical skills‖ measure
captures the main elements of modern teaching style, focusing on the instilment of learning skills and
creative thinking (Resnick, 1987).
I use in the empirical analysis panel data on pupils in Israel who are in fifth grade in 2002 (primary
school) and in eighth grade (middle school) in 2005. These students were tested in four subjects (English,
Hebrew, mathematics, and science) in both grades as part of a national testing program. I base the
identification on within-pupil analysis (using pupil fixed effects) together with primary- and middleschool fixed effects that net out any confounding factors among schools or individuals. Therefore,
variations in teaching practices within schools/grades and across classes are natural variations in teaching
styles of teachers. I use this heterogeneity to estimate the effect of interest and demonstrate below that it
does not correlate with any of the pupil or class characteristics that may affect potential outcomes. In
practice, the within school estimation eliminates most of the selection of pupils into primary and middle
3
For details of the recent reform, see, for example, the Daily Mail of November 25, 2010. Michael Gove‘s
prescription for the improvement of state schools (expressed while he was still the shadow Secretary of Education)
focuses on return to learning based on memorization and comprehension, e.g., reciting multiplication tables,
learning to conjugate verbs, and memorizing important dates and figures in national history. Gove also claimed, ‖It
is often the poorest children who suffer most from trendy teaching. When synthetic phonics was abandoned as the
way of teaching children to read because it was too authoritarian, children from book-rich backgrounds survived
but those who were already book poor fell behind.‖ (See the Daily Telegraph, October 20, 2007, Rachel Sylvester
and Alice Thomson interview with England‘s shadow Education Secretary, and, more recently, Michael Gove
article in the Times, March 17, 2010.)
4
‗Pedagogical Horizon,‘ a 2008 circular from the Ministry Director General, advised that as of September 1, 2008,
teaching at the post-primary level would change from being based on memorization, repetition, and practice to
emphasis on development of deep understanding and acquisition of learning and thinking skills. The facilitation of
pedagogical reform entailed other changes, including in curriculum and standards, teacher training, and student
evaluation.
3
schools but the estimates change further when students‘ fixed effects are added to the estimated
equations. Additional evidence that supports the causal interpretation of the findings presented in this
paper is the institutional rules that forbid school choice and tracking by ability in primary and middle
public schools in Israel and that classes are actually formed randomly. The findings about heterogeneity
of the effect of treatment, by type of teaching practices and by types of students, specifically by gender
and ability, support the claim about the causal nature of our finding because boys and girls, for example,
are sorted similarly across schools and among classes within schools. If the results are produced by
sorting, they should be similar for males and females, but they are not. This heterogeneity in treatment
effects constitute also an evidence against the possibility that it is overall teacher effectiveness and not
specific teaching practices that is causing students‘ value added.
As mentioned, the evidence presented in this paper suggests considerable pupil heterogeneity by
gender and socioeconomic background in the effect of both teaching practices. Thus, while instilment of
knowledge and enhancing comprehension has a generally positive effect, it is much larger for girls and
for pupils from less-educated families; conversely, the effect of the instilment of applicative, analytical,
and critical skills is positive for both genders and both levels (low and high) of student socioeconomic
status. The coexistence of positive and heterogeneous effects of these seemingly contradictory teaching
practices has important policy implications for the potential improvement of pupils‘ knowledge by the
targeting of teaching methods. Another interesting outcome of the study is that the three other teaching
practices studied have no systematic relation with the improvement of pupils‘ test scores.
The effect size of the two statistically significant teaching practices is very large relative to other
educational interventions such as reducing class size or improving teacher training. For example, if the
proportion of classroom teachers that practice ―traditional teaching‖ increases from the mean to the
maximum observed in the sample, the average test score in each subject increases by 0.27 standard
deviation of the test-score distribution. As similar change for ―modern teaching‖ practice increase test
scores by 0.25 standard deviation. These large effect sizes of teaching practices and the consistency of
the findings reported in this paper are important as many countries search for ways to promote ―teacher
4
quality‖ or ―teacher effectiveness.‖ This ardent interest, partly occasioned by anxiety in some countries
about students‘ poor performance in international tests and comparisons such as TIMMS, PISA and
PIRLS, is not being addressed by much reliable evidence. The findings reported in this paper are a step
toward meeting this demand.
The paper proceeds as follows: Section 2 presents brief review of the most relevant literature,
Section 3 describes the data, and Section 4 explains the empirical methodology. The results, robustness
checks, and heterogeneity in the treatment effects of teaching practices are presented and discussed in
Section 5. The last section concludes and suggests policy implications.
2.
Related Literature
A yhtgnel list of studies represents the efforts of researchers to use non-experimental data to estimate
teacher effects on pupils‘ learning outcomes (e.g., Hanushek, 1971; Murnane and Phillips, 1981;
Rockoff, 2004; Hanushek, Rivkin, and Kain, 2004; Jacob and Lefgren, 2005; Aaronson, Barrow, and
Sander, 2007; Kane, Rockoff, and Staiger, forthcoming; Gordon, Kane, and Staiger, 2006, Cantrell et al
2008). Several additional studies used random assignment to estimate the variation in teacher effects. In
this kind of analysis, Nye, Konstantopoulous, and Hedges (2004) revisited the results of the STAR
experiment in Tennessee. After accounting for the effect of different classroom-size groupings, their
estimate of the variance in teacher effects was well within the range typically reported in the nonexperimental literature. Chetty et al. (2010), based on the STAR experiment data as well, followed the
project participants to adulthood and have shown that students who had a more experienced teacher in
kindergarten have higher earnings.
Some researchers (e.g., Clotfelter et al., 2006, 2007) found that teachers with stronger academic
backgrounds produce larger performance gains for their pupils; others did not find this relationship (e.g.,
Harris and Sass, 2006, on graduate coursework and Kane et al., 2008a, on college selectivity). A small
number of studies (e.g., Clotfelter et al., 2006, 2007; and Goldhaber, 2007) found a link between
5
teachers‘ scores on certification examinations and their effectiveness; Harris and Sass (2006) found no
such relation.
Researchers acknowledge the possibility that non-random assignment of students to teachers may
distort measures of teacher effectiveness. Some teachers are given better students who would achieve
well in many different classrooms. Some researchers question whether a teacher‘s specific contribution
can be accurately estimated at all, given the possibility that students are assigned to teachers on the basis
of unmeasured characteristics not captured by test scores and demographics (Rothstein, 2010). Other
researchers, while recognizing the potential of bias, are more optimistic (Koedel and Betts (2007). One
recent study (Kane, Rockoff and Staiger, 2008b) compared experimental (i.e., classes randomly assigned
to teachers) and non-experimental estimates of teachers‘ effects on student achievement growth for a
small sample of teachers in Los Angeles. In that sample, the non-experimental or observational measures
predicted the experimental measures with little bias—as long as the observational models controlled for
each student‘s prior achievements.
In several studies, the effect of teachers in one grade fades out as students progress to higher grades
(McCaffrey et al, , 2004; Kane, Rockoff and Staiger, 2008a; Jacob, Lefgren, and Sims, 2008; Rothstein,
2010). Hypotheses for the fadeout range from artifacts of empirical strategy to the heterogeneity of
teacher quality within schools to the relevance of skills gained in one year for skills tested the next year
(Kane, Rockoff and Staiger, 2008b).
A few recent studies found a relationship between a teacher‘s measured effect on student
achievements and overall subjective administrator ratings (Jacob and Lefgren, 2005; Rockoff and
Speroni, 2010; Rockoff, Jacob, Kane and Staiger, forthcoming). These studies, however, do not identify
the criteria or behaviors that the school principals use to make their judgments. Two papers that are close
in motivation and measurement of treatment to this study is Tyler et al., (2010) and Kane et al., (2011).
The first paper describes the early and the second the final results from the same study where teachers‘
evaluation and their teaching practices are related to student achievement. The study finds that measures
6
of teaching effectiveness are substantively related to student achievement growth and that some observed
teaching practices predict achievement more than other practices.
The Learning about Teaching (Bill and Melinda Gates foundation, December 2010) report about
initial findings from Measures of Effective Teaching Project suggests that a teacher‘s past track record of
value-added is among the strongest predictors of their students‘ achievement gains in other classes and
academic years. The teachers who lead students to achievement gains on one year or in one class tend to
do so in other years and other classes. The report also suggests that student perceptions of a given
teacher‘s strengths and weaknesses are consistent across the different groups of students they teach and
that student perceptions in one class are related to the achievement gains in other classes taught by the
same teacher.
3.
Data
The empirical analysis uses two samples, one of fifth-grade primary-school students and one of eighthgrade middle-school students. Both samples were culled from Jewish secular schools to the exclusion of
others, because this part of the Israeli school system places Grades 5 and 8 in different schools—primary
and middle, respectively. The distinction is important because it requires students to change schools in
the middle and because the secular school system does not allow school choice at either the primary or
the middle level, except in a few cities where charter schools that allow opting out of neighborhood
schools. I address this issue again in the Methodology section below.
I drew the two samples from the Growth and Effectiveness Measures for Schools (GEMS)—Meizav
in Hebrew—datasets for 2002 and 2005. GEMS, composed of a series of tests and questionnaires
administered by the Evaluation and Measurement Division of the Ministry of Education,5 is administered
at the midterm of each school year to a representative 1-in-2 sample of all primary and middle schools in
Israel, so that each school participates in GEMS once every two years.
5
The GEMS assessments are not administered for school accountability purposes; only aggregate results at the
district level are published. For more information on GEMS, see the Evaluation and Measurement Division website
(in Hebrew): http://cms.education.gov.il/educationcms/units/rama/odotrama/odot.htm.
7
The GEMS student data include test scores of fifth- and eighth-graders in mathematics, science,
Hebrew, and English, as well as the responses of fifth- through ninth-grade students to questionnaires. In
principle, all students except those in special-education classes are tested and required to complete the
questionnaire. The proportion of students tested is above 90% and the rate of questionnaire completion is
roughly 91%. The raw test scores use a 1–100 scale that we transform into z-scores to facilitate
interpretation of the results. The tests in all four subjects measure facts, analytical skills and also critical
thinking, both at low and high level. These properties of the achievement tests are important because we
may expect traditional teaching to matter more for learning about facts while we would expect modern
practices to be more important for critical thinking.
The GEMS student questionnaire addresses various aspects of the school and learning environment.
The section I used, focusing on teaching style and practices, includes 29 items that are listed in the
Appendix. These items ask students to rate on a six-point scale to what proportion of their teachers the
statement is appropriate. A score of 1 indicates ‗none of the teachers‘, 2 indicates ‗very few teachers‘, 3
indicates ‗few teachers‘, 4 indicates ‗part of the teachers‘, 5 indicates ‗large part of the teachers‘, and 6
indicates ‗almost all teachers‘. The items refer to the current year‘s teachers so it is not a retrospective
report but on the other hand it has the limitation of ignoring a student‘s history of teachers. Perhaps we
can view this limitation as an advantage here because students, particularly those in primary grades, are
less likely to be particularly retrospective. Another point to emphasize is that these survey questions
provide information about the average teaching practices of all teachers that teach a particular class and
not about a particular teacher. We should note this distinction when interpreting the evidence presented
later sections. Although we probably care also about the practices of the teacher in the subject of interest,
rather than the proportion of a student‘s teachers who exhibit a certain practice, the latter measure is
much less likely to be endogenous. The average teaching styles of teachers of a given classroom is less
likely to be a function of student abilities, class size and so on. Nevertheless, I describe in the next
section the identification strategy where I control for student‘s ability by a pupil‘s fixed effect and I
provide evidence that classes are formed randomly out of the students in a given grade. Both of these
8
elements limit very much the possibility that the variation in teaching practices that I use for estimation
reflects some endogenous placement of certain teaching practices.
I group the items under five categories that describe teachers‘ pedagogical practices in the
classroom: (1) instilment of knowledge and enhancement of comprehension (seven items); (2) instilment
of applicative, analytical and critical skills (nine items); (3) instilment of capacity for individual study
(three items); (4) transparency, fairness, and feedback (three items); and (5) individual treatment of
students (seven items). These categories of teachers‘ pedagogical practices correspond to common and
accepted terminology in the educational-psychology literature, dating back to Bloom (1956), on major
categories in the taxonomy of educational objectives (―Categories in the Cognitive Domain‖). Each
category comes with detailed list of outcomes and a list of outcome-illustrating verbs. Knowledge, for
example, is defined as the remembering (recalling) of appropriate previously learned information; the
associated verbs include define, describe, enumerate, and others that are listed in Appendix B.6 Bloom
(1956) defines comprehension as grasping (understanding) the meaning of informational materials; some
of the associated verbs in this category are classify, convert, describe, discuss, and explain. Bloom uses
both concepts, ―knowledge‖ and ―comprehension,‖ to define his first teaching practice, ―instilment of
knowledge and enhancement of comprehension.‖ In relying on Bloom‘s logic and categorization, I avoid
some arbitrariness in grouping the items in different categories even though some people could disagree
with Bloom as to the appropriate placement of certain items. However, it is important to note here that
the results reported in this paper are stable with respect moving items that may seem to some as less
definitive in terms of the category to which they ‗belong‘. For example, it can be argued that item 3 in T1
("The teachers commend students who know the material well") or item 7 in T1 ("I understand the
teachers' scholastic requirements well") could legitimately be placed in the transparency, fairness and
feedback domain. Changing the locations of these items does not affect the basic results reported below.
6
Retrieved from (http://faculty.washington.edu/krumme/guides/bloom.html).
9
Such robustness of the estimated effect of the various categories is also an indication that the number of
items included in each category is not affecting the reliability of the composite.
In this paper, I focus on the first four teaching-practice measures and do not report evidence on the
fifth measure (―individual treatment of students‖) because it reflects the level of the students in class and
its estimated effect is prone to reverse causality. I do, however, include this measure in all the estimated
regressions that I report in the paper even though the estimated effect of the other four teaching-practice
measures does not change when the fifth measure is omitted from the estimated equations—partly
because the estimates of this measure are almost always small and not significantly different from zero.
I aggregate the student‘s responses to a class level measure in three different ways. First, I treat the
teaching practices variables as cardinal treatment by assigning a proportional value to each categorical
response as follows: ‗none of the teachers‘ = 0, ‗very few teachers‘ = 0.2, ‗few teachers‘ = 0.4, ‗part of
the teachers‘ = 0.6, ‗large part of the teachers‘ = 0.8 and ‗almost all teachers‘ = 1.0. It is important to
note here that here is an obvious justification for treating the numerical values assigned to the categorical
responses as meaningful because students are asked about the proportion of their classroom teachers that
use certain teaching practices. I then simply average the students‘ responses to each question given the
above proportional values and then use the mean of all questions that form each of the teaching practices
as the treatment variables. However, as an alternative treatment measure I use the proportion of students
that give a given or higher categorical response in each of the items that form a teaching practice. I use
three thresholds for computing this measure: 4 (‗part of the teachers‘ or more), 5 (‗large part of the
teachers‘ or more), and 6 (‗almost all teachers‘). This is an ordinal measure and it will be useful to
compare the results obtained when using it to those I obtain when using the first cardinal measure. A
third measure that I use is to simply average the categorical values (1 to 6) without translating them to
their proportion counterpart. It is important to note that all these three alternative methods of measuring
classroom level teaching practices yield identical evidence about their effect on test score outcomes.
However, I find the first method most appealing because it is intuitive and the estimates it yields are
10
meaningfully interpretable as the treatment is measured as the proportion of classroom teachers that use a
given teaching method.
I linked the student questionnaire data and the 2002 and 2005 test scores data to student
administrative records collected by the Israel Ministry of Education, which include student demographic
background characteristics. Using the linked datasets, I built one data set for primary schools and another
for middle schools. The primary-school file for 2002 includes data from 415 primary schools with test
scores and student questionnaires. The means of students‘ characteristics in this sample are presented in
column 1 of Table 1. The panel sample includes students from 359 primary schools and their mean
characteristics are presented in column 2 of Table 1. Restricting the panel sample to primary schools with
at least five students leaves 122 such schools and their students' mean characteristics are presented in
column 3 of Table 1. The mean characteristics of pupils included in the panel dataset (in column 2 or 3)
are not very different from those of the full sample though some of these differences are significant. But
note that the differences in father and mother year of schooling as well as the gender composition are
very small and they are not statistically different from zero. Therefore, we can conclude that the panel
sample is representative of the full sample in terms of the background characteristics that are most
important as determinants of cognitive outcomes.
The middle school file for 2005 includes data from 176 schools with eighth grade students‘
questionnaires and test scores. The panel sample with at least five students in each school includes 192
schools, 122 primary schools and 70 middle schools.
4.
Empirical Strategy
The structure of GEMS makes it possible to track a sample of students from primary schools (fifth grade
in 2002) to middle schools (eighth grade in 2005).7 I used this feature to construct a longitudinal dataset
at the student level to examine how changes in teaching practices (styles and methods) induce changes in
pupils‘ test scores. We know from prior work (e.g., Rivkin et al., 2005) that there exists dramatic cross-
7
I did not link datasets from consecutive years because almost all localities were sampled once every two years.
11
grade variation in measures of teacher quality and I use in this paper such variation for estimating the
effect of teaching practices. However, it is important to note that that this change is due to the
compulsory transition of students from primary to middle school. Also important is the fact that Israel
does not allow school choice at the primary and middle levels; pupils are assigned to their neighborhood
primary school and middle school, the latter often having a catchment area that includes several primary
schools.
Since the estimated regression includes a student fixed effect and a school fixed effect, the
identification is based on contrasting the change in exposure to the various teaching practices during
grades five and eight among students who followed the same transition path from primary to middle
school. More formally, I assume that the cognitive achievements of pupils in grades five and eight are
determined by the following equation:
(1)
yics  i s  Scs' 2    (TeachingPractice) sc   ics  cs
where i denotes individuals, c denotes class (within a grade), and s denotes schools. Since the school
indicator is perfectly correlated with the grade indicator (fifth or eighth grade), there is no need to add a
grade effect in Equation (1). yics is an achievement measure for student i in class c and school s;  i is a
'
pupil effect,  s is a school effect, Scs is a vector of characteristics of class c in school s; it includes a set
of variables for average characteristics of students in the class (mother‘s and father‘s years of schooling,
number of siblings, immigration status, and five groups of ethnic origin), characteristics of class learning
environment and climate (such as levels of classroom noise, violence, lack of discipline and class size).
(TeachingPractice) sc is a vector of four teaching practices (  1...4) in class c and school. These are
measured by the proportion of the classroom teachers who use each practice. The error term in the
equations is composed of a school-specific random element cs that allows for any type of correlation
within observations of the same school across classes and an individual random element  ics . The
coefficient of interest is θ, which captures the effects of the different teaching practices. For the purpose
12
of comparison, I will also present OLS estimate of regressions that do not include pupil‘s fixed effects
but include instead individual characteristic as controls:
(2)
'
yics   s  xics
1  Scs' 2    (TeachingPractice) sc   ics  cs
'
Where xics is a vector of student‘s covariates that includes mother‘s and father‘s years of schooling,
number of siblings, immigration status, and ethnic origin, and indicators for missing values in these
covariates.
To estimate Equations (1) and (2), I need to observe students while they are in fifth and eighth
grade. For the estimates in Equation (1) to have a causal interpretation, however, the unobserved
determinant of achievement must be uncorrelated with the treatment variable. The inclusion of school
fixed effects and pupil fixed effects controls for the most obvious potential confounding factor—the
endogenous sorting of students across schools. However, there may be unobserved within-school and
across-class factors that also correlate with changes in teachers‘ teaching practices. If some classroom
characteristics are not controlled, the estimated effects of interest will be biased. Random assignment of
students and teachers to classrooms solves this problem by breaking the link between teaching practices
and extraneous effects on the class such as unobserved peer quality. True random-assignment variation
is rare in an education context and unavailable in many countries. However, students in Israel‘s primary
and middle schools are rarely grouped into classes on the basis of ability or family background; in fact,
such practices are forbidden by law. Therefore, classes in primary schools with multiple classrooms at
the same grade level are typically formed on a more-or-less random basis; classes in middle schools are
formed in a way that creates social integration by mixing students from different socioeconomic
backgrounds.8 Since all classes within a grade are of equal average ability, teachers are assigned to
classes more-or-less randomly and the possibility of better teachers avoiding assignment to lower-
8
A 1968 education reform established a three-tier structure of schooling in Israel: primary (grades 1–6), middle (7–
9), and high (10–12). The reform established neighborhood school zoning as the basis of primary enrollment and
integration, sometimes with busing, of students out of their neighborhoods in middle school. Tracking and sorting
of students in primary- and middle-school classes were outlawed and the law is strictly enforced.
13
performing classes is irrelevant, as is the possibility for ―teacher-shopping‖ by parents. I note here also
that the lack of tracking in primary and middle schools in Israel rule out as well the possibility that class
composition changes across subjects. Therefore, the students in a given class rank the same teachers.
The foregoing implies that (TeachingPractice) sc will be uncorrelated with class-level shocks cs
conditional on a set of school fixed effects, pupil fixed effects, and class-mean characteristics. Thus, the
basic identifying assumption in this study is that the systematic components of teaching practices in
school arise only at the school level and not at the class level. A necessary condition for the withinschool estimation to work is, of course, that there is sufficient variance in teaching practices within a
school, as is the case in our data.
The identification strategy I use in this paper is most closely related to that of Ammermueller and
Pischke (2009), who use a school-fixed-effects framework to estimate peer effect based on within-school
and across-class variation in peer ability. They demonstrate that conditioned on a school fixed effect,
class composition within a grade is random. However, I also include a pupil fixed effect in the
regressions, which accounts for any selection based on pupil specific attributes. This addition is very
important in the context of this paper because it is possible that the teaching practices vary based on
students‘ ability. The student fixed effects that I include in the regression are therefore appropriate
controls that rule out a bias due to potential endogeneity of teaching practices. Including also in the
'
regressions the class level characteristics ( Scs ) is also useful in this regard.
Evidence of the Validity of the Identification Strategy
The key identifying assumption I make in this paper postulates that, conditional on pupil fixed effects,
changes in teaching practices within a school are uncorrelated with changes in unobserved factors that
may affect students‘ outcomes. I assess here, from different angles, the plausibility of this assumption. I
first discuss the assignment of students both between and within schools and present evidence that sheds
light on the question of whether classes are formed (more-or-less) randomly and whether different
classrooms systematically get different resources. Even if the variation in teaching practices within a
14
school resembles a random process, however, these variations may be correlated with additional class-toclass changes that may affect student outcomes. To assess this possibility, we check whether changes in
teaching practices within a school are associated with changes in student background characteristics such
as parental education, family size, ethnicity, and student's immigration status.
Students in Israel attend primary school from initial enrollment to grade 6 and middle school from
grades 7 to 9. Generally speaking, primary-school assignment depends on place of residence. Each
middle-school catchment area includes several primary schools in order to achieve social integration by
blending pupils from different socioeconomic backgrounds. Parents can affect choice of school in certain
ways, e.g., by choosing to live near the school of their choice. The school administration is responsible
for assigning students to classes within schools. Extra resources are allocated to schools that have a high
share of disadvantaged or recent-immigrant students but class size cannot exceed 40 students in all
schools. An important regulation from the Ministry of Education forbids grouping of students by ability
in primary and middle school.9 Even when parents fund additional weekly instruction hours, these
resources cannot be used for the formation of study groups by ability (tracking) or any other criterion.10
A similar regulation from the Ministry applies to middle schools and requires heterogeneous classes. For
example, a circular from the Director General outlining the responsibilities of a middle-school principal
relative to the responsibilities and authority of a secondary-school principal states explicitly that it is the
responsibility of the former to create heteroegenous classes and that ability tracking is allowed only after
ninth grade.11 These institutional rules are supported by evidence from PIRLS 2003, based on the item in
the school questionnaire that asks whether the school forms classes on the basis of ability. The fraction of
students in schools that report some ability grouping at the class level is close to zero and it does not vary
by gender. I obtained similar evidence from TIMSS 1999 and PIRLS 2003, which included a similar
question about the extent to which classes are formed based on students‘ ability. Jakubowski (2009),
9
Ministry of Education, Circular from the Director General, March 2000:
http://cms.education.gov.il/EducationCMS/applications/mankal/arc//s7bk3_1_8.htm.
10
See http://cms.education.gov.il/EducationCMS/applications/mankal/arc//sc3ak3_11_9.htm.
11
See http://cms.education.gov.il/EducationCMS/Units/Sherut/Takanon/Perek7/Chativa/.
15
using PIRLS 2003 data to study the effect of tracking, included Israel among countries that do not track
students by ability in their primary- and middle-school systems.
Having obtained this institutional evidence of random formation of classes within schools, I used
the sample of all primary and middle schools to test whether the data I use in this study also support this
claim. In particular, I checked class assignment to see whether it correlates systematically with students‘
2
characteristics. For this purpose, I performed a series of Pearson Chi-Square (  ) tests for eight
characteristics: gender, father‘s years of schooling, mother‘s years of schooling, number of siblings, and
three ethnic origin indicators. If a school forms its classes randomly, any particular characteristic of a
student should be statistically independent of his or her class assignment. In father‘s schooling, for
2
example, the Pearson  test asks whether there are more pupils with high father‘s schooling in a
particular class than is consistent with independence, given the size of the school‘s enrollment.
Ammermuler and Piscke (2009) describe and apply this test in their study of peer effects. Formally, I
performed the Pearson test for each school and, under the assumption that schools in Israel are
independent, I also added up the test statistics for all schools to obtain an aggregate test statistic such as
that described by DeGroot (1984). Obviously, I performed this test only on the basis of the subsample of
schools (482 out of 605) that had two or more classes within the relevant grade. Of the 1928 p values for
primary schools, 83—only 4% of the sample—were lower than 5%. Furthermore, only two schools had
two or more of characteristics with p-values equal to or lower than 5%. The aggregate p-values for each
of the four characteristics far surpassed 20 %. The middle-school data yielded similar results: of 230
middle schools with two or more classes, 82 were equal or lower than 5% out of 920 p-values. Therefore,
in 9% of the cases I cannot reject that there is non-random assignments. However, only in 13 of the 230
schools (exactly 5% of all schools) two or more p-values are equal or lower than 5%. Overall, I conclude
that there is no evidence of systematic formation of classrooms with respect to the four measures of
student family background measures.
16
The second question I investigate in this section is whether classrooms that differ in teaching
practices differ in pupils‘ characteristics as well. To test whether classroom teaching practices are
statistically independent of each of the student characteristics, I ran balancing tests as are run in
randomized trials, using the following OLS regression model:
(3a)
xics     (TeachingPractice) sc  vics
and the following school fixed effect model:
(3b)
xics    S   (TeachingPractice) sc  vics
where  S are the school fixed effects. Tables 2 and 3 present estimates from regressions where the
dependent variable is a student characteristic and the explanatory variable is one of the four teachingpractice measures. Table 2 presents the results for the fifth-grade sample and Table 3 does the same for
the eighth-grade sample. Each table presents the results from two specifications: an OLS regression
without any additional control variables and another that includes school fixed effects. I could not add
individual fixed effects because the nature of the pupil-background characteristics (father‘s years of
schooling, mother‘s years of schooling, number of siblings, an indicator of recent immigration, a gender
indicator, an indicator of whether a pupil‘s parents are native Israeli and four other indicators of ethnic
origin) generally rules out changes in characteristics between grades 5 and 8. I also ran these balancing
tests for class size in search of evidence of selection in allocating school resources to classes.
Tables 2 and 3 offer little evidence for the proposition that students of different family backgrounds
are more likely to be in classes that invoke certain teaching practices, conditional on the school they
attend. Only eleven of the 72 balancing coefficient estimates presented in Table 2 and Table 3 are
significantly different from zero at the 10 percent level of statistical significance when school fixed
effects are included in the regressions. Four of these imbalance characteristics are with respect to the
teaching practices ‗transparency, fairness and feedback‘ and five of them relate to the gender variable.
This pattern stands in sharp contrast to the balancing estimates obtained from the OLS equations (without
school fixed effects), which yielded 32 estimates that were significantly different from zero. It useful to
17
demonstrate this lack of signs of selection based on the parental human capital variables. The pattern of
selection between primary schools with respect to parental schooling is negative, meaning that primary
schools with large enrollments of pupils whose fathers or mothers have many years of schooling have
lower intensities of all four teaching practices; All eight of the OLS estimates were significantly different
from zero. However, after I added the school fixed effects to the regressions, three estimates changed
signs to positive and no one of these 8 estimates is significantly different from zero. The respective
pattern of selection with respect to parental school is different in middle schools as only two of the eight
OLS estimates and two of the school fixed effects estimates are significantly different from zero, all
related to the forth teaching practice (transparency, …). The between- and within-school selection pattern
with respect to number of siblings are inconsistent as well, as two of the eight OLS estimates and three of
the eight school-fixed-effect estimates were positive. Further, only 2 of the OLS estimates and one of the
school fixed effect estimates were significant. Similarly inconsistent is the positive selection pattern in
the distribution of teaching practices between schools based on the balancing-test estimates (some
positive, some negative) of the proportion of immigrant pupils. I view these inconsistencies in the
selection patterns across the various socio-economic proxies as suggestive evidence that even across
schools the variation in teaching practices is not an indication of clear meaningful selection that can
confound the effect of teaching practices on pupils‘ academic outcomes in a certain direction. I should
emphasize again, however, that once I added school fixed effects to the regressions, all evidence or signs
of potential selection disappeared.
This evidence largely confirms that classes in the sample schools are formed randomly within the
schools. There is little evidence that students of different family backgrounds are more likely to be
grouped in certain classes depending on the school they attend. However, even if classes are formed
randomly, they may receive other school resources differentially. For example, if a class ends up with
more children from less advantaged family backgrounds purely by chance, the school may assign this
class a smaller class size. To shed light on this question, I ran a set of regressions of the teaching-practice
variables described in the previous section on class size and its square. The estimates presented in
18
columns 17-18 in Table 2 and Table 3 show that there are no meaningful within schools correlations
between class size and any of the four teaching-practice variables. All eight estimates are not
significantly different from zero and two are even positive.
5.
Results
I now analyze the effects of the various measures of teachers‘ teaching practices on students‘ test scores.
Table 4 presents descriptive statistics for the four teaching-practice measures. In the Appendix, I present
the items that I averaged into each of these indices. Even though each of the items ranges from 0 to 1, the
range is narrowed when aggregated into the four teaching-practice measures, from about 0.2 to about 1.
There are no significant differences between the descriptive statistics of the panel and the full samples
and the means of T1-T4 are marginally lower in grade 8. Based on the means from the full sample
(presented in panel B of Table 4) we see that a higher proportion of classroom teachers practice T1 than
T2-T4. In primary school, the mean of T1 is 0.76 and in middle-school it is 0.65. The respective means
of T2 are 0.57 and 0.47. The respective means of T3 they are 0.58 and 0.42. The similarity between the
mean proportion of teachers using T2 and T3 is a result of these two practices being different dimensions
of modern teaching.
Table 5 reports estimates based on pooling of all four subjects together and each specification in the
table includes subject fixed-effect indicators. I report the effect of each category of teaching practice.
Although estimates for all individual items that I used to construct the aggregate teaching-practice
measure are not reported here, I should note that most of these estimates are not precisely measured and
some have negative values, partly because of the high correlation among the various items. Therefore, it
is appropriate and necessary to aggregate the items in several principal components as I do in this paper.
Having no prior information with which to justify a particular weighting, I assign equal weight to all
items grouped in a given teaching-style characteristic in order to provide a more transparent
interpretation. I computed the class-level mean of these teaching-practice characteristics for each student
while excluding the student‘s own answer. When I used measures based on means that included also
19
students‘ own answers, I obtained the same results exactly. I also tried averaging the z scores of each
item instead of the absolute value of the students' response to each of the questions and I obtained again
the same results. I tried as well an alternative measure of the teaching practices, using (0/1) dummy
variables that divide the sample to two groups based on alternative thresholds values of the proportion of
teachers that use each of the teaching practices. The results from these alternative measures are fully
consistent with the continuous measures and I will provide more details on this issue later in this section.
The first and second columns in Table 5 report OLS estimates from an equation that included the
following control variables: pupil‘s background characteristics (gender, father‘s and mother‘s years of
schooling, number of siblings, an indicator of immigration status, and five indicators of ethnic origin—
Europe/America, Asia/NorthAfrica, Soviet Union, Ethiopia, and Israel). I also included in the regression
as controls the class-level means of all these indicators, an indicator for each of the four subjects, and
measures of class climate and learning environment such as level of noise, disturbances, violence, and
lack of discipline. In Column 1, the estimates are from separate regressions in which each teaching
practice enters as a single treatment variable. In Column 2, the estimates come from one regression that
includes all the teaching practice measures as multiple treatments. In Columns 3 and 4, I add primaryand middle-school fixed effects to the specification of columns 1-2. In the specification presented in
Columns 5 and 6, I replace all the student‘s control variables with students‘ fixed effects. In Panel A, the
results are based on a sample of schools that have at least five pupils in the panel data (the Five plus
sample); in Panel B, the sample is further restricted to include schools that have at least ten pupils in the
panel data (the Ten plus sample). The estimates in Panel B provide a robustness check to the sensitivity
of the estimates to the precision of the school-fixed-effect estimates.
Focusing first on the OLS estimates of the effects of teachers‘ pedagogical methods in panel A, we
see that all eight estimates are positive but only two (the estimated coefficients of T1) are significantly
different from zero. This suggests that there is no obvious selection-bias pattern for three of the treatment
measures even if based on the OLS estimates. It is also noticeable that the Column 1 and Column 2
estimates show no clear differences in sign and precision. However, adding the primary and middle
20
school fixed effect to the equation induces a major change in the size and sign of the estimates of the
teaching-practice measures. Focusing on the specification that includes all treatment measures in the
regression, we observe the following: the estimate of T1 (Instilment of Knowledge and Enhancement of
Comprehension) drops marginally to 1.010 (se=0.268) and the estimate of T2 (Instilment of Analytical
and Critical Skills) climbs to 0.810 (se=0.265) relative to its respective OLS estimate. The estimate of T3
is almost unchanged but become more precise and the estimate of T4 (Transparency, Fairness, and
Feedback) is reversed in sign and becomes significant. Adding the pupil‘s fixed effects to the regression
leads to an equal proportional decline in the point estimates of T1, to 0.792 (se=0.292), and of T2, to
0.520 (se=0.281). The other two estimates (of teaching practices T3 and T4) are reversed in sign, become
very small and not statistically different from zero.
The estimates reported in Columns 5–6 have two remarkable features. The first is the similarity
between the estimates when only one of the teaching-practice measures is used as a treatment measure
and when all four are used jointly. The estimates of T1 and T2 are only 20-30 percent lower when all
teaching practices enter jointly as treatments in the regression. This pattern suggests that there is very
little omitted variable bias when three of the four teaching-practice measures are left out of the equation
once we include pupil and school fixed effects. Apparently, even though T1 and T2 are highly correlated,
the conditional (on schools and students fixed effects) partial correlation between these two variables is
much smaller which allows measuring precisely their independent effect on test scores. The second is the
difference in the estimates of the two measures that capture two distinct elements of modern teaching. T2
has positive and significant estimated effect while the estimated effect of T3 is practically zero (small,
negative and insignificant). By implication, if it is selection that generates the positive effect of T2, it
must be very different for each of these two measures, which is very unlikely. Another noteworthy
feature of the estimates in Column 5-6 is the relative similar sensitivity of the estimates of T1 and T2 to
the addition of pupils‘ fixed effects to the regressions, as T1 falls by about 60 percent and T2 by about 40
percent. I should also note that the estimates of the other two teaching practice measures (T3 and T4)
21
also change when pupil‘s fixed effects are added, from large and significantly different from zero to
much smaller and imprecise.
Panel B of Table 5 presents estimates based on a sample restricted to schools that had at least ten
pupils in the panel data. The estimated effects do not change at all in comparison to those presented in
Panel A. This restriction allows greater precision in estimating school fixed effects and it is important
that the estimated effects of T1 and of T3–T4 are not sensitive to this sample restriction.
Before discussing the effect size of the various estimates, it is important to rule out the possibility
that if there is a correlation between specific teaching practices and overall teacher effectiveness, our
findings then simply reflect the underlying teacher effectiveness and not the effect of the practice itself.
For example, if a teacher with strong thinking skills is more effective than one with weak thinking skills,
using either ―modern‖ or ―traditional‖ practices, we would expected to see the patterns reported above.
However, this does not seem to be driving our results as demonstrated by the very different estimated
effects of T2 and T3. Even though these two teaching practice are highly correlated (estimated
correlation coefficient of 0.873, see Table A1) and both are integral elements of modern teaching, T2 has
a significant effect on test score while T3 has a zero effect. If it is overall teachers effectiveness and not a
specific teacher practice that improve student outcomes, we should have observed T3 being equally
effective as T2. I will return to this issue in the next section when presenting evidence about the
heterogeneity of the effect of T1 and T2 across sub-groups of students, by gender and by ability. We
would not expect such treatment heterogeneity if it is the overall teacher‘s effectiveness that is causing
students‘ value added instead of some specific teaching practices.
Overall, the evidence in Table 5 strongly suggests that two of the four teaching styles and methods
have positive and meaningful effects on pupils‘ learning. The more important of them in terms of effect
size is the indicator of the extent to which teachers make sure that their students know and understand the
material by using examples, memorization techniques, homework, classwork, and so on. When the mean
(65 percent of the classroom teachers) of this teaching-practice measure rises to the maximum (99
percent of the classroom teachers) observed in the data (in the full sample), the test score changes by an
22
average of 0.27 (=0.34 * 0.792) standard deviation of the test-score distribution. The effect of elevating
teachers‘ instilment of analytical and critical skills in the classroom from the mean (0.47) to the
maximum (0.95) observed in the data (full sample) is only marginally smaller at 0.25 (=0.48 * 0.52)
standard deviation. A simultaneous change in these two teaching practices, which is not unreasonable
because there are many classes with large proportion of teachers using both practices, leads to a again of
over half a standard deviation of the test score distribution. A more drastic policy change is to improve
the use of T1 or T2 from the lowest to the highest proportion observed in the data. The gain for such a
change in T1 is 0.53 (0.67*0.792) and for a respective change in T2 it is 0.39 (0.75*0.52). These results
suggest that the gains associated with changes in these two effective teaching practices are much bigger
than the effect of most other effective interventions, such as reducing class size, increasing schooling
time, or providing teachers and students with conditional financial incentives. It is also probably less
expensive to train teachers in the appropriate use of these two effective teaching practices than to
implement these alternative interventions. Before further discussing the policy implications of the
findings, however—I do this in the Conclusions section—I present additional evidence about the
heterogeneity of the effect of T1 and T2.
Results Based on Ordinal Measures of Teaching Practices
I have treated so far the teaching practices variables as cardinal treatment measuring the
proportion of classroom teachers that use a given teaching practice. To assess how sensitive are the
results to this particular way of measurement of teaching practices, I also estimated models with the two
alternative measures of the treatment variables. I first used the simple mean of the categorical ranking of
students without converting them to proportional terms. The results based on these cardinal measures are
identical completely to the results presented in Table 5 and are available from the author. Second, in
Table 6, I present treatment estimates where I measure the intensity of each teaching practice by the
proportions of answers above a certain level (4, 5 or 6) in all the questions that the teaching practice
consist of. Column 1 presents estimates where the proportion count includes all answers from 4 and up
23
(students who said that part or large part or almost all of the teachers apply the relevant teaching
practice). The proportion of students for this level of treatment of T1 is 0.816 and for T2 it is 56.1 The
effect of T1 and T2 are positive and significantly different from zero and the former is larger. The effect
of T3 and T4 are practically zero. The evidence presented in columns 2 and 3 depicts the same pattern.
Comparing the estimates in the three columns suggests that T1 has a large effect already when 40 percent
of the classroom teachers are practicing T1 and any increment in T1 has declining marginal effect. On
the other hand, the effect of T2 is largest when all teachers practice it in class.
Overall, the results using the two alternative measures of the teaching practices are consistent
with the evidence presented in the last column of Table 5 and therefore I will continue using in the rest of
the paper the measures used in Table 5.
Estimated Effects by Subject
The results reported so far assume that the effect of each of the teaching practices is the same in all
subjects. To test this assumption, columns 1–2 in Table 7 present evidence based on the pooling of math
and science test scores and columns 3–4 on the basis of pooling Hebrew and English test scores. All
estimates in columns 1–4 of Table 7 are based on regression specifications that include pupil and
primary- and middle-school fixed effects. I view these groupings as less restrictive than the pooling of all
four subjects together because pedagogy and teaching style may well be more similar in the first two
subjects, which are intensive in math and rigorous analysis, and in the two language subjects.
Comparison of the estimates in Columns 2 and 4 reveals a surprisingly close similarity in the respective
estimates of the effect of all four teaching practices. Based on the estimates in Panel A, the effect of
instilment of knowledge/comprehension measure (T1) is 0.788 for math and science and 0.804 for
Hebrew and English. The estimate of enhancement of analytical and critical skills (T2) is 0.406 for math
and science and 0.553 for Hebrew and English. The estimates for the other two teaching style measures
are small, practically zero, and insignificant for both sets of subjects. Note also the consistency in sign of
these two measures in both sets of subjects: T3 is negative in math and science and in Hebrew and
24
English, and T4 is positive in both. The results presented in Panel B strengthen these conclusions. In fact,
in the 10+ sample the effect of T2 in math and science is even more similar to that in Hebrew and
English (0.089 and 0.118, respectively). This suggests that pooling all four subjects in estimation is not
overly restrictive and offers the advantage of estimating the parameter of interest more precisely.
In Table A2, I present evidence based on a separate regression for each subject. The four estimates
of T1 are again similar, highest for English (1.030) and lowest for Hebrew (0.550). However, running
separate regressions for each subject comes at the expense of precision of estimates because of the
smaller sample size used in each regression. The estimates of T2 are not very different for English,
Hebrew and science but it is much lower in math. I therefore present in Table A3 column 1 results based
on pooling all subjects except math. The estimates in this column are very similar to those presented in
Table 5 with the exemption that the estimated effect of T2 is now 25 percent higher and all estimates
have lower standard errors.
Heterogeneous Treatment Effects
To gain further insights on the extent of effects of teaching styles and methods on students‘ test scores, I
explore heterogeneous effects across different dimensions. Table 8 reports heterogeneous treatment
effects of the teaching-style measures by gender and by father‘s years of schooling (above or equal
to/below the median, i.e., 13 years).12 I prefer to stratify the sample by these subgroups instead of using
interaction terms for these subgroups with the treatment effects because in the latter approach the
treatment-interaction terms may pick up variations by gender or parental schooling in the effects of other
covariates included in the regressions. The stratifying approach comes at the price of estimating the
heterogeneous treatment effects on the basis of a smaller sample. Since the sample-size issue is
especially troubling in estimating fixed-effects models, Panel A of Table 8 shows the results from
samples that are restricted in each subgroup to the inclusion of at least five pupils per school.
12
Students with missing values in parental education (4% of the total sample) are excluded from this analysis. The
results are not sensitive to the inclusion of these students in the low or high education group. Estimating
heterogeneous treatment effects by mother‘s schooling, the results obtained are very similar to those based on
father‘s schooling.
25
In Columns 1 and 2 of Table 8, I report the effect of each of the four teaching-practice measures on
boys and girls, respectively. The estimates of T1 (―traditional‖ teaching) presented in the first row reveal
a striking difference in the effect of this teaching practice on boys and on girls. The effect on boys is not
statistically different from zero (0.213, se= 0.439) while the estimated effect for girls is very large, 0.997
(se=0.426). The estimated standard errors of these parameters clearly does not allow us to accept the
hypothesis that the practice is more effective for girls than for boys but nevertheless this gender gap is
quite obvious. The effect of T2 (―modern‖ teaching), on the other hand, has a much smaller gender gap,
though here in favor of boys. The estimate for boys 0.742 (se=0.437) and for girls it is 0.303 (se=0.400).
Clearly the confidence intervals of these two estimates overlap greatly because the two estimates are
imprecisely measured. Another apparent gender difference is in the estimates of the transparency,
fairness, and feedback practice (T4), which are positive and significant for boys and small, negative, and
not different from zero for girls. These gender differences are evident as well when math test scores are
not included in the sample. These results are presented in columns 2-3 of Table A3.
The implications of this result—that solving problems and exercises routinely in class and teaching
based on repetition of material until most students attain knowledge and comprehension affect girls in the
main, as opposed to boys—are important and interesting. First, they are consistent with several studies,
which show that other schooling interventions either affect only girls (e.g.., pupil monetary incentives) or
affect both genders equally (e.g., teacher financial incentives or class-size reduction). Second, this
heterogeneity in treatment effect is evidence that the effect of T1 and T2 are not simply capturing the
overall teacher‘s effectiveness because in that case we would expect to observe each teaching practice to
affect similarly boys and girls. Note that boys and girls are studying in the same classes within schools
and there is no within school systematic differences between the measures of teaching practices based on
boys and girls assessment of the (same) classroom teachers.
Next, I stratified the sample by high or low father‘s years of schooling. Since the father‘s schooling
variable has fewer missing values, I used it to define this indicator even though the evidence based on
mother‘s years of schooling is very similar. Overall, there is large heterogeneity between the groups in
26
the effect of T1 but not in the effect of T2. The effect of T1 on pupils of low socioeconomic status (SES),
0.940 (se=0.412), is larger and marginally statistically different from the effect on high-SES pupils,
0.119 (se=0.503).13 Yet the economically meaningful much higher effect of T1 on low-SES pupils is
intuitive since this teaching style, which emphasizes practice and rote learning in the classroom, most
likely replaces relatively scarce home and parental guidance in the production of knowledge among lowSES families. However, the similarity in the effect of the modern style of teaching on the two groups
may be unexpected but is encouraging because it provides an important indication that pupils from low
SES-backgrounds can be equally motivated and challenged by a teaching style that emphasizes the
instilment of learning skills and creative thinking. It would be useful to stratify the samples of low and
high SES by gender as well in order to gain more insights about the heterogeneity of these two very
quantitatively important teaching practices in the classroom, but it is not possible in this study due to
sample-size limitations.
Table 9 presents evidence about the heterogeneous effect of T1 and T2 on students‘ ability as
measured by their ranking in the test-score distribution. I defined a new outcome measure as a product of
the z score and the 1/0 indicator based on the percentile ranking of the pupil‘s test score in a given
subject. The table offers estimates for five regressions: the 25th, 50th, 75th, 80th and 90th percentiles. The
pattern of the results based on these percentiles is a good representation of the evidence obtained for
other percentiles. Clearly, instilment of knowledge and enhancement of comprehension is very effective
in the classroom for pupils below median ability; its effect drops sharply after this threshold is surpassed.
In contrast, the effect of instilment of analytical and critical skills is not very effective at very low ability
(below the 25th percentile) but its effect picks up and remains high until the very high level of ability,
although it is highest at around the 75th percentile and then declines gradually. However, the differences
in the effect across the distribution of ability levels above the first quartile are not large and sharp enough
13
These differences by family SES background are evident as well when math test scores are not included in the
sample. These results are presented in columns 4-5 of Table A3.
27
to draw firm conclusions about such heterogeneity. The results in Table 9 are consistent with the
evidence presented in Columns 3–4 of Table 8 due to the positive and large correlation that exists
between ability (ranked by test scores) and socioeconomic background.
The evidence about the heterogeneous treatment effects of T1 and T2 is important because it adds
credibility to the causal interpretation of the estimates. I showed in Section 4 that classes within schools
are formed randomly with respect to parental schooling and student‘s gender; therefore, any
unaccounted-for sorting or selection of teaching-practice measures across classes within schools should
not be different by gender or by parental schooling. The evidence of differential gender and parentalschooling effects is an indication that potential omitted selection or sorting factors, as well as the
possibility of endogeneity of the teaching practices, cannot account for the results I present in this paper.
The heterogeneity in treatment effects is also an indication that the estimates do not trace to a pattern in
which students prefer the teaching practices that their favorite teachers use. This conclusion is enhanced
by the way I computed the measures of teaching practices—averages based on the assessment of all
students in the class—and by the insensitivity of the results to excluding or including own assessment in
computing the class average. Finally, the heterogeneity in treatment effects is also an indication that the
estimates do not reflect overall teacher effectiveness. If it is the latter, we would have expected to see
similar effects for boys and girls, and also for high and low ability students.
6.
Conclusions
In this paper, I measured empirically the relationship between classroom teaching practices and student
achievements. I found very strong evidence that two important teaching practices cause student
achievement growth. In particular, classroom teaching that emphasizes ―instilment of knowledge and
comprehension" has a very strong and positive effect on test scores, especially of girls and of pupils from
low socioeconomic backgrounds. Second, the use of classroom techniques that endow pupils with
―analytical and critical skills‖ has a very high payoff, especially among pupils from educated families.
Transparency in the evaluation of pupils, proper and timely feedback to students, and fairness in
28
assessing pupils also lead to cognitive achievement gains, especially among boys. However, ―instilment
of the capacity for individual study‖ does not cause any gain in value added of students learning. These
results are robust to the three different methods I use to aggregate the students‘ responses about their
teachers‘ teaching practices to class level means.
Beyond estimating models with school and student fixed effects that control for all sorts of selection
and sorting of students into schools, this paper provides several additional reasons to believe that
selection and sorting are not responsible for the results summarized above. I show, for example, that boys
and girls were sorted similarly across schools; thus, if the results trace to sorting, they should have been
similar, rather than very different, for males and females. In addition, there does not appear to be any
sorting within the sample by parental schooling yet the effects of treatment are again heterogeneous.
Similarly, if the effect of teaching practices is simply capturing overall teacher effectiveness, we would
not expect to see heterogeneity in effect by teaching practices and by gender ability groups of students,
but we do. All this evidence supports a causal interpretation of the estimated effects summarized above.
The evidence I presented in this paper provides important insights about what does and does not
work in the classroom; the set of results is one of the first that clearly identifies actions of teachers that
―pay off‖ versus others that do not. This study has additional policy relevance because its evidence sheds
light on the merits of traditional versus modern approaches to teaching, a contrast that has featured in
recent policy debates and educational reforms in several countries. This study may be the first to
demonstrate that one approach need not crowd out the other and that the two can coexist. The estimated
heterogeneity in treatment effects of the two styles and the essence of teaching that I estimated in this
paper implies that it is best to target certain teaching practices to relevant customers and also to mix the
two in the classroom. However, a limitation of this study that perhaps can be addressed in the future is
that teaching practices are measured as a class average and not for individual teachers.
The effect sizes estimated for some of teaching practices are truly impressive, especially relative to
the effect sizes of other potential interventions such as reducing class size, increasing school hours of
instruction, and providing more teacher training. These three alternative interventions and other possible
29
educational programs are much more expensive or difficult to implement than the installation of
appropriate teaching practices in the classroom. Although this change would entail some teacher training,
it should not be too costly since teachers in most education systems around the world routinely engage in
on-the-job training. Therefore, re-directing the syllabus relating to enhancement of teachers‘ human
capital toward training in adequate use of ―instilment of knowledge and enhancement of comprehension‖
and ―instilment of analytical and critical skills‖ should be neither too difficult nor too costly. The
potential gains seem enormous and worth the effort to sway away teachers from teaching practices that
suit their comparative advantages but may not be effective.
7.
References
Ammermueller, Andreas and Jorn-Steffen Pischke, (2009). ―Peer Effects in European Primary Schools:
Evidence from PIRLS,‖ Journal of Labor Economics, vol. 27, no. 3, pp. 315–348.
Aaronson, Daniel, Lisa Barrow and William Sander (2007). ―Teachers and Student Achievement in
Chicago Public High Schools,‖ Journal of Labor Economics Vol. 24, No. 1, pp. 95–135.
Bill and Melinda Gates foundation. 2010. ―Learning about Teaching: Initial Findings from the Measures
of Effective Teaching Project‖ MET project, Research Paper, December.
Bloom, Benjamin S. (ed.), 1956, Taxonomy of Educational Objectives, Handbook 1: Cognitive Domain,
New York, N.Y: Longmans, Green and co.
Cantrell Steven, Kane, J. Thomas, Jon Fullerton, and Douglas Staiger. ―National Board Certification and
Teacher Effectiveness: Evidence from a Random Assignment Experiment,‖ NBER Working Paper
#14608, December 2008.
Chetty, Raj, John N. Friedman, Nathaniel Hilger, Emmanuel Saez, Diane Whitmore Schanzenbach, AND
Danny Yagan, ―How Does Your Kindergarten Classroom Affect Your Earnings? Evidence From
Project Star‖, 2010, forthcoming, Quarterly Journal of Economics.
Clotfelter, C.T., Ladd, H.F., Vigdor, J.L. ―Teacher-Student Matching and the Assessment of Teacher
Effectiveness‖ Journal of Human Resources 41(4):778–820 (2006).
Clotfelter, C.T., Ladd, H.F., Vigdor, J.L. (2007). ―How and Why Do Teacher Credentials Matter for
Student Achievement?‖ NBER Working Paper 12828.
DeGroot, Morris. 1984. Probability and statistics. 2nd ed. Reading, MA: Addison-Wesley.
Goldhaber, Dan. 2007. ―Everyone's Doing It, but What Does Teacher Testing Tell Us about Teacher
Effectiveness?‖ Journal of Human Resources 42(4): 765–94.
Gordon, Robert, Thomas J. Kane and Douglas O. Staiger, (2006). ―Identifying Effective Teachers Using
Performance on the Job‖ Hamilton Project Discussion Paper, Published by the Brookings
Institution.
Hanushek, Eric A. (1971). ―Teacher Characteristics and Gains in Student Achievement; Estimation
Using Micro Data‖. American Economic Review, 61, 280-288.
Hanushek, Eric A., John F. Kain, Steven G. Rivkin, ―Why Public Schools Lose Teachers‖, Journal of
Human Resources 39(2), Spring 2004b.
Hanushek, Eric A. ―The Economics of Schooling: Production and Efficiency in Public Schools.‖ Journal
of Economic Literature, September 1986, 24(3), pp. 1141–1117.
30
Harris, Douglas, and Tim R. Sass. 2006. ―Value-Added Models and the Measurement of Teacher
Quality.‖ Florida State University. Unpublished.
Jacob, Brian and Lars Lefgren (2005). ―Principals as Agents: Student Performance Measurement in
Education‖ NBER Working Paper No. 11463.
Jacob, B.A., L. Lefgren, and D. Sims, ―The Persistence of Teacher-Induced Learning Gains,‖ NBER
working paper #14065, June 2008.
Jakubowski Maciej, ―Early tracking and achievement growth‖ Directorate for Education, OECD,
December 2009.
Kane, Thomas J., Jonah E. Rockoff, and Douglas O. Staiger (2008a). ―What Does Certification Tell Us
about Teacher Effectiveness? Evidence from New York City.‖ Economics of Education Review,
27(6): 615–631.
Kane, Thomas J., Jonah E. Rockoff and Douglas Staiger, (2008b). ―Estimating Teacher Impacts on
Student Achievement: An Experimental Evaluation‖ NBER Working Paper #14607, December.
Kane Thomas J., Eric S. Taylor, John H. Tyler and Amy L. Wooten, ―Identifying Effective Classroom
Practices Using Student Achievement Data‖, Journal of Human Resources, 2011, 587-613.
Koedel Cory and Julian Betts, 2007. "Re-Examining the Role of Teacher Quality In the Educational
Production Function," Working Papers 0708, Department of Economics, University of Missouri.
Konstantopoulos, S. ―How Long Do Teacher Effects Persist?‖ IZA Discussion Paper 2893, 2007.
McCaffrey, D. F., Lockwood, J. R., Koretz, D., Louis, T. A., and Hamilton, L. (2004). Models for valueadded modeling of teacher effects. Journal of Educational and Behavioral Statistics, 29(1), 67-101.
(2004).
Murnane, R. J. & Phillips, B. R. (1981). ―What Do Effective Teachers of Inner-City Children Have in
Common?‖ Social Science Research, 10, 83–100.
National Council of Teachers of Mathematics, (9119) ―Professional Standards for Teaching
Mathematics‖, Published by the National Council of Teachers of Mathematics.
National Research Council, 1996, ―National Science Education Standards‖, National Academy Press
Washington, D.C.
Nye Barbara, Spyros Konstantopoulos and Larry V. Hedges ―How Large Are Teacher Effects?‖,
Educational Evaluation and Policy Analysis Fall 2004, Vol. 26, No. 3, pp. 237-257.
Resnick, L. (1987). Education and Learning to Think, Washington D.C.: NationalAcademy Press.
Rivkin, S., Hanushek, E. A. and Kain, J. (2005). ―Teachers, Schools, and Academic Achievement,‖
Econometrica, 73(2): 417–458.
Rockoff, J. E. (2004). ―The Impact of Individual Teachers on Student Achievement: Evidence from
Panel Data,‖ American Economic Review, 94(2): 247–252.
Rockoff, Jonah E., Brian Jacob, Thomas J. Kane, and Douglas O. Staiger. ―Can You Recognize an
Effective Teacher When You Recruit One?‖ Education Finance and Policy, 2011, 6(1): 43-74.
Rockoff, Jonah E., and Cecilia Speroni (2010). ―Subjective and Objective Evaluations of Teacher
Effectiveness.‖ American Economic Review, 100(2): 261–66.
Rockoff, E. Jonah and Douglas Staiger. ―Searching for Effective Teachers with Imperfect Information,‖
forthcoming, Journal of Economic Perspectives.
Rothstein, Jesse (2010). ―Teacher Quality in Educational Production: Tracking, Decay, and Student
Achievement.‖ Quarterly Journal of Economics, 125(1): 175–214.
Salomon, G. Perkins, D. Theroux, P. (2001). Comparing Traditional Teaching and Student Centered,
Collaborative
Learning
[Online,
retrieved
14/6/02]
URL:
http://shaw.ca/priscillatheroux/collaborative.html.
Tyler, John H., Eric S. Taylor, Thomas J. Kane, and Amy L. Wooten. (2010) "Using Student
Performance Data to Identify Effective Teachers and Teaching Practices." American Economic
Review 100(2):256-260.
Zemelman Steven, Harvey Daniels, and Arthur Hyde, Best Practice, Today's Standards for Teaching and
Learning in America's Schools, 1993 and 2005, Heinemann, Reed Elsevier Incorporation.
31
Appendix A: Questionnaire Items
Instilment of knowledge and enhancement of comprehension
1. The teachers give exercises and assignments that help memorize the material.
2. The teachers ask many questions in class that check whether we know the material well.
3. The teachers commend students who know the material well.
4. The teachers provide many examples that help understand the material.
5. The teachers hold discussions in class that help understand the material.
6. During lessons, the teachers ask many questions that check whether we understand the material
well.
7. I understand the teachers' scholastic requirements well.
Instilment of analytical and critical skills
1. The teachers give exercises and assignments whose answers have not been studied in class and
are not in the textbooks.
2. The teachers require that we use what we have studied to explain various phenomena.
3. The teachers ask that we find new examples by ourselves for the material we have studied.
4. The teachers ask that we try to find several ways to solve a certain problem.
5. The teachers teach us to find a single common explanation for different phenomena.
6. The teachers give assignments where it is required to analyze material and to relate it to other
things we have studied.
7. When there are several ways to solve a problem, the teachers require that we check them all and
find the best one.
8. The teachers expect us to ask ourselves whether what we have learned is correct.
9. The teachers teach us how to know whether information we have found is important, relevant,
and usable.
Instilment of capacity for individual study
1. The teachers teach us how to learn new topics by ourselves.
2. The teachers require students to utilize many and varied sources of information (newspapers,
books, databases etc.).
3. The teachers teach us to observe our environment and to follow phenomena that occur in it.
Transparency, fairness and feedback
1. The teachers explain to me exactly what I have to do to improve my studies.
2. The teachers explain according to what they determine the grades / assessments.
3. The teachers often tell me what my situation is regarding schoolwork.
Individual treatment of students
1. The teachers know what the educational difficulties of each student are.
2. When a student has difficulty with a certain topic, the teachers give him more time to study it.
3. The teachers give homework to every student according to his place in the material.
4. The teachers help every student to learn topics interest him.
5. The teachers give me a feeling that if I make an effort I will succeed more at studies.
6. When a student fails, the teachers encourage him to try again.
7. The teachers always assist me when I need help with studies.
32
Appendix B: Major Categories in the Taxonomy of Educational Objectives
(Bloom, 1956) (http://faculty.washington.edu/krumme/guides/bloom.html)
Categories in the Cognitive Domain: (with Outcome-Illustrating Verbs)
1. Knowledge of terminology; specific facts; ways and means of dealing with specifics (conventions,
trends and sequences, classifications and categories, criteria, methodology); universals and abstractions
in a field (principles and generalizations, theories and structures): Knowledge is (here) defined as the
remembering (recalling) of appropriate, previously learned information.
● defines; describes; enumerates; identifies; labels; lists; matches; names; reads; records;
reproduces; selects; states; views.
2. Comprehension: grasping (understanding) the meaning of informational materials.
● classifies; cites; converts; describes; discusses; estimates; explains; generalizes; gives
examples; makes sense out of; paraphrases; restates (in own words); summarizes; traces;
understands.
3. Application: the use of previously learned information in new and concrete situations to solve
problems that have single or best answers.
● acts; administers; articulates; assesses; charts; collects; computes; constructs; contributes;
controls; determines; develops; discovers; establishes; extends; implements; includes; informs;
instructs; operationalizes; participates; predicts; prepares; preserves; produces; projects;
provides; relates; reports; shows; solves; teaches; transfers; uses; utilizes.
4. Analysis: the breaking down of informational materials into their component parts, examining (and
trying to understand the organizational structure of) such information to develop divergent conclusions
by identifying motives or causes, making inferences, and/or finding evidence to support generalizations.
● breaks down; correlates; diagrams; differentiates; discriminates; distinguishes; focuses;
illustrates; infers; limits; outlines; points out; prioritizes; recognizes; separates; subdivides.
5. Synthesis: Creatively or divergently applying prior knowledge and skills to produce a new or original
whole.
● adapts; anticipates; categorizes; collaborates; combines; communicates; compares; compiles;
composes; contrasts; creates; designs; devises; expresses; facilitates; formulates; generates;
incorporates; individualizes; initiates; integrates; intervenes; models; modifies; negotiates; plans;
progresses; rearranges; reconstructs; reinforces; reorganizes; revises; structures; substitutes;
validates.
Evaluation: Judging the value of material based on personal values/opinions, resulting in an end
product, with a given purpose, without real right or wrong answers.
● appraises; compares and contrasts; concludes; criticizes; critiques; decides; defends; interprets;
judges; justifies; reframes; supports.
33
Table 1: Descriptive Statistics and Mean Differences Between the Panel and the Full Sample
Means,
sample of all
5th grade
students in
2002
Means, panel
sample of 5th
grade students
in 2002
Means, panel
sample of 5th
grade students
in 2002:
schools with
at least 5
students in
final sample
T-test,
differences
between
means in
column 1 and
column 2
T-test,
differences
between
means in
column 1 and
column 3
(1)
(2)
(3)
(4)
(5)
Father's years of
schooling
12.586
12.735
12.783
0.173
(0.136)
0.223
(0.160)
Mother's years of
schooling
12.924
13.089
13.076
0.193
(0.124)
0.173
(0.147)
Number of siblings
2.125
2.004
2.021
-0.141
(0.053)
-0.119
(0.064)
Gender (female=1)
0.498
0.503
0.504
0.006
(0.008)
0.006
(0.009)
Immigration status
(immigrant=1)
0.136
0.199
0.186
Parents born in Israel
0.525
Characteristics
0.445
0.446
0.073
0.056
(0.018)
(0.022)
-0.092
-0.089
(0.021)
(0.025)
Schools and pupils
Number of schools
Number of pupils
415
359
122
26964
3824
3117
Notes: Each parameter estimate presented in columns 4 and 5 is obtained from a separate regression. Standard deviations
are presented in parenthesis in columns 1-3. Standard errors, clustered by school, are presented in parenthesis in columns 45.
Table 2: Balancing Tests, Fifth Grade
OLS
School FE
Father's years of
schooling
OLS
School FE
Mother's years of
schooling
OLS
School FE
Number of siblings
(1)
(2)
(3)
(4)
(5)
(6)
T1: Instilment of knowledge and
enhancement of comprehension
-6.791
(2.686)
-2.759
1.918
-6.329
(2.069)
-2.358
(2.114)
-2.670
(1.128)
-0.551
(0.947)
T2: Instilment of analytical and
critical skills
-9.155
(2.377)
-1.946
(2.001)
-7.207
(2.058)
0.479
(1.857)
-1.314
(0.947)
-1.361
(0.690)
T3: Instilment of capacity for
individual study
-5.291
(1.996)
-1.008
(1.744)
-4.544
(1.551)
-0.208
(1.483)
-1.451
(0.819)
-0.650
(0.616)
T4: Transparency, fairness and
feedback
-6.717
(1.560)
1.370
(1.345)
-6.362
(1.194)
0.271
(1.116)
-0.892
(0.575)
-0.347
(0.432)
Recent immigration
Boy
Parents born in Israel
(7)
(8)
(9)
(10)
(11)
(12)
T1: Instilment of knowledge and
enhancement of comprehension
1.319
(0.323)
-0.103
(0.258)
-0.010
(0.187)
-0.476
(0.349)
-0.997
(0.394)
0.101
(0.356)
T2: Instilment of analytical and
critical skills
1.726
(0.303)
0.341
(0.189)
0.042
(0.173)
0.160
(0.300)
-1.387
(0.329)
-0.391
(0.298)
T3: Instilment of capacity for
individual study
1.044
(0.263)
0.179
(0.139)
-0.048
(0.135)
-0.138
(0.281)
-0.927
(0.289)
-0.187
(0.250)
T4: Transparency, fairness and
feedback
0.794
(0.219)
-0.131
(0.141)
0.045
(0.111)
-0.471
(0.203)
-0.905
(0.239)
-0.287
(0.193)
Parents born in Asia
or Africa
Parents born in
Europe or US
Class size
(13)
(14)
(15)
(16)
(17)
(18)
T1: Instilment of knowledge and
enhancement of comprehension
-0.155
(0.231)
-0.209
(0.281)
-0.177
(0.211)
0.335
(0.275)
-3.515
(12.601)
4.391
(7.762)
T2: Instilment of analytical and
critical skills
-0.047
(0.193)
-0.131
(0.205)
-0.371
(0.158)
0.217
(0.285)
-9.495
(8.922)
3.495
(3.791)
T3: Instilment of capacity for
individual study
-0.069
(0.154)
-0.248
(0.181)
-0.098
(0.121)
0.279
(0.209)
-14.399
(6.041)
5.383
(4.087)
T4: Transparency, fairness and
feedback
0.136
(0.143)
0.119
(0.142)
-0.067
(0.127)
0.350
(0.157)
-8.607
(6.879)
-0.838
(3.942)
Notes: The table reports OLS and school fixed effects estimates from separate regressions of the relevant
dependent variable on each of the four teaching practices. Robust standard errors clustered at the school level
are reported in parentheses. The sample includes 5th grade pupils in 2002, from secular schools with 5 or more
pupils in the panel data set.
Table 3: Balancing Tests, Eighth Grade
OLS
School FE
Father's years of
schooling
OLS
School FE
Mother's years of
schooling
OLS
School FE
Number of siblings
(1)
(2)
(3)
(4)
(5)
(6)
T1: Instilment of knowledge and
enhancement of comprehension
-1.875
(1.774)
1.202
(1.316)
-0.272
(1.804)
1.524
(1.411)
-0.414
(0.464)
-0.221
(0.673)
T2: Instilment of analytical and
critical skills
-3.400
(2.469)
-1.827
(1.420)
-1.974
(2.041)
-1.715
(1.245)
0.709
(0.575)
0.594
(0.810)
T3: Instilment of capacity for
individual study
-0.456
(1.633)
-0.298
(0.748)
0.457
(1.357)
-0.134
(0.714)
0.299
(0.369)
0.164
(0.498)
T4: Transparency, fairness and
feedback
-4.966
(1.507)
-1.504
(0.919)
-4.212
(1.422)
-2.202
(1.090)
-0.106
(0.387)
0.322
(0.465)
Recent immigration
Boy
Parents born in Israel
(7)
(8)
(9)
(10)
(11)
(12)
T1: Instilment of knowledge and
enhancement of comprehension
0.849
(0.300)
0.237
(0.141)
-0.136
(0.144)
-0.425
(0.214)
-0.719
(0.288)
0.120
(0.185)
T2: Instilment of analytical and
critical skills
0.875
(0.394)
0.336
(0.248)
-0.310
(0.183)
-0.497
(0.252)
-0.465
(0.393)
-0.231
(0.211)
T3: Instilment of capacity for
individual study
0.368
(0.274)
-0.012
(0.143)
-0.165
(0.135)
-0.348
(0.191)
-0.115
(0.295)
0.074
(0.128)
T4: Transparency, fairness and
feedback
0.703
(0.230)
0.178
(0.146)
-0.361
(0.118)
-0.687
(0.198)
-0.841
(0.166)
-0.111
(0.124)
Parents born in Asia
or Africa
Parents born in
Europe or US
Class size
(13)
(14)
(15)
(16)
(17)
(18)
T2: Instilment of analytical and
critical skills
-0.239
(0.202)
.
-0.377
(0.160)
-0.284
(0.115)
.
-0.024
(0.196)
0.125
(0.112)
.
-0.038
(0.167)
-0.050
(0.145)
.
-0.091
(0.166)
-2.950
(4.789)
.
-12.694
(6.487)
0.879
(5.271)
.
7.089
(8.395)
T3: Instilment of capacity for
individual study
-0.187
(0.140)
-0.009
(0.103)
-0.055
(0.098)
-0.063
(0.106)
-11.097
(4.980)
-2.631
(5.205)
T4: Transparency, fairness and
feedback
0.164
(0.162)
0.049
(0.119)
-0.053
(0.107)
-0.133
(0.132)
6.511
(4.214)
3.766
(4.462)
T1: Instilment of knowledge and
enhancement of comprehension
Notes: The table reports OLS and school fixed effects estimates from separate regressions of the relevant
dependent variable on each of the four teaching practices. Robust standard errors clustered at the school level
are reported in parentheses. The sample includes 8th grade pupils in 2005, from secular schools with 5 or more
pupils in the panel data set.
Table 4: Descriptive Statistics, Proportion of Classroom Teachers Using Each of the Teaching
Practices
Grade 5
Grade 8
Mean
Min
Max
Mean
Min
Max
(1)
(2)
(3)
(4)
(5)
(6)
T1:Instilment of knowledge and
enhancement of comprehension
0.76
(0.05)
0.61
0.93
0.65
(0.07)
0.44
0.84
T2:Instilment of analytical and
critical skills
0.57
(0.05)
0.42
0.85
0.47
(0.05)
0.35
0.62
T3:Instilment of capacity for
individual study
0.58
(0.07)
0.40
0.92
0.42
(0.08)
0.24
0.65
T4:Transparency, fairness and
feedback
0.64
(0.07)
0.40
0.88
0.61
(0.08)
0.35
0.82
B: Full sample
T1:Instilment of knowledge and
enhancement of comprehension
0.76
(0.06)
0.50
0.99
0.65
(0.07)
0.32
0.99
T2:Instilment of analytical and
critical skills
0.57
(0.06)
0.32
0.95
0.47
(0.05)
0.20
0.83
T3:Instilment of capacity for
individual study
0.58
(0.09)
0.26
1.00
0.42
(0.08)
0.20
0.87
T4:Transparency, fairness and
feedback
0.63
(0.09)
0.28
1.00
0.61
(0.08)
0.29
1.00
A: Panel sample
Notes: This table presents means, standard deviations and minimum and maximum values for the four teaching
practices measures. The figures in the table measure the proportion of classroom teachers using a given practice.
Column 1-3 present results for the fifth grade (2002) sample and columns 4-5 for the eight grade (2005) sample.
Panel A reports results for the secular schools that are represented in the panel data with 5 or more pupils and panel
B reports results for the population of all secular schools. Standard deviations are presented in parentheses.
Table 5: Estimates of the Effect of Teaching Practices on Pupils’ Test Scores
OLS
School fixed effect
School and pupil fixed
effect
Each
All
measure
measures
included
included
separately
jointly
Each
measure
included
separately
All
measures
included
jointly
Each
measure
included
separately
All
measures
included
jointly
(1)
(2)
(3)
(4)
(5)
(6)
T1: Instilment of knowledge and
enhancement of comprehension
1.659
(0.558)
2.258
(0.647)
1.518
(0.230)
1.910
(0.268)
1.111
(0.248)
0.792
(0.292)
T2: Instilment of analytical and critical
skills
0.087
(0.452)
-0.215
(0.608)
1.000
(0.209)
0.810
(0.265)
0.768
(0.219)
0.520
(0.281)
T3: Instilment of capacity for individual
study
-0.015
(0.351)
0.410
(0.475)
0.384
(0.149)
0.320
(0.192)
0.214
(0.159)
-0.274
(0.206)
T4: Transparency, fairness and feedback
0.287
(0.318)
0.459
(0.348)
-0.248
(0.156)
-0.510
(0.178)
0.473
(0.164)
0.123
0.191
T1: Instilment of knowledge and
enhancement of comprehension
2.049
(0.620)
2.499
(0.716)
1.687
(0.247)
1.970
(0.285)
1.156
(0.263)
0.879
(0.309)
T2: Instilment of analytical and critical
skills
0.165
(0.526)
-0.329
(0.684)
1.128
(0.227)
0.940
(0.286)
0.734
(0.235)
0.544
(0.299)
T3: Instilment of capacity for individual
study
0.080
(0.389)
0.395
(0.513)
0.345
(0.161)
0.148
(0.206)
0.114
(0.169)
-0.407
(0.217)
T4: Transparency, fairness and feedback
0.493
(0.358)
0.522
(0.386)
-0.107
(0.171)
-0.408
(0.192)
0.478
(0.176)
0.126
(0.202)
A. Five plus sample (N= 20660)
B. Ten plus sample (N= 18168)
Notes: This table reports OLS (columns 1-2), school fixed effects (columns 3-4) and pupil and school fixed
effects(columns 5-6) estimates of the effect of the four teaching practice measures on pupils test scores measured as z
scores, The z scores are computed based on the full sample. For the OLS regressions the standard errors are clustered at
the class level and robust standard errors are reported for the pupil and pupil and school fixed effects regressions. The
OLS regressions include as controls pupils' personal characteristics (parental education, gender, number of siblings,
immigrant status and three ethnic indicators), four subject dummies, class size, class means of the pupils' characteristics
and several class climate measure (class means of level of noise, incidence of violence, discipline). The estimates
presented in the odd columns are from regressions when each of the teaching practices is used as the only treatment
variable in the regression. The estimates presented in the even columns are from regressions where all four teaching
practices measures are used simultaneously as treatment variables in the regressions. The estimates presented in panel A
are based on the five plus sample and the estimates in panel B are based on the ten plus sample.
Table 6: Estimates of Effect of Alternative Measures of Teaching Practices on
Pupils’ Test Scores
Proportion of
answers 4
and above
Proportion of
answers 5
and above
Proportion of
answers 6
and above
(1)
(2)
(3)
T1: Instilment of knowledge
and enhancement of
0.668
(0.195)
[0.816]
0.352
(0.173)
[0.556]
0.138
(0.220)
[0.257]
T2: Instilment of analytical
and critical skills
0.363
(0.193)
[0.561]
0.289
(0.217)
[0.332]
0.965
(0.316)
[0.141]
T3: Instilment of capacity for
individual study
-0.106
(0.145)
[0.532]
-0.191
(0.159)
[0.318]
-0.022
(0.211)
[0.136]
T4: Transparency, fairness
and feedback
0.098
(0.138)
[0.674]
0.198
(0.126)
[0.488]
0.147
(0.158)
[0.265]
N
20660
Note: The estimates reported in this table are obtained from one regression. The treatment variables
are proportions of answers above a certain figure among all the questions the treatment conssists of.
Standard errors are reported in parentheses .The figures in square parenthesis are the average
proportions in the panel. The regression specification includes pupil and school fixed effects. See
note to Table 5 for additional controls included in the regression. The regression is based on the five
plus sample.
Table 7: Effects of Teaching Practices by Pooled Math and Science and pooled Hebrew and
English Samples, Based on a Model with Pupil and School Fixed Effects
Math and Science
Hebrew and English
Each measure
included
separately
All measures
included jointly
Each measure
included
separately
All measures
included jointly
(1)
(2)
(3)
(4)
T1: Instilment of knowledge and
enhancement of comprehension
0.990
(0.335)
0.788
(0.395)
1.224
(0.345)
0.804
(0.406)
T2: Instilment of analytical and
critical skills
0.604
(0.299)
0.406
(0.382)
0.876
(0.301)
0.553
(0.390)
T3: Instilment of capacity for
individual study
0.126
(0.214)
-0.288
(0.277)
0.289
(0.222)
-0.257
(0.289)
T4: Transparency, fairness and
feedback
0.352
(0.224)
0.037
(0.259)
0.599
(0.227)
0.217
(0.264)
10464
10464
10196
10196
T1: Instilment of knowledge and
enhancement of comprehension
1.027
(0.356)
0.959
(0.419)
1.286
(0.366)
0.814
(0.429)
T2: Instilment of analytical and
critical skills
0.548
(0.322)
0.470
(0.408)
0.863
(0.323)
0.527
(0.415)
T3: Instilment of capacity for
individual study
-0.021
(0.227)
-0.443
(0.292)
0.230
(0.236)
-0.388
(0.305)
T4: Transparency, fairness and
feedback
0.254
(0.241)
-0.051
(0.275)
0.709
(0.243)
0.306
(0.280)
9178
9178
8990
8990
A. Five plus sample
N
B. Ten plus sample
N
Notes: Columns 1-2 report the estimates and robust standard errors (in parentheses) from regressions of the four
treatment variables on pupils' achievements based on a sample that includes the Math and Science test scores.
Columns 3-4 report the respective estimates from a sample that includes the Hebrew and English test scores. The
estimates are based on the pupil and school fixed effect model. See notes to Table 5 for the controls included in
these regressions. The estimates presented in panel A are based on the five plus sample and the estimates in panel B
are based on the ten plus sample.
Table 8: Effects of Teaching Practices on Pupils’ Test Scores By Gender and Parental
Education Based on a Pupil and School Fixed Effect Model
Boys
Girl
High SES
Low SES
(1)
(2)
(3)
(4)
T1: Instilment of knowledge and
enhancement of comprehension
0.213
(0.439)
0.997
(0.426)
0.119
(0.503)
0.940
(0.412)
T2: Instilment of analytical and
critical skills
0.742
(0.437)
0.303
(0.400)
0.362
(0.527)
0.357
(0.388)
T3: Instilment of capacity for
individual study
-0.591
(0.316)
-0.220
(0.289)
-0.018
(0.388)
-0.409
(0.288)
T4: Transparency, fairness and
feedback
0.443
(0.282)
-0.174
(0.278)
0.308
(0.327)
0.116
(0.277)
N
10230
10438
7860
12740
T1: Instilment of knowledge and
enhancement of comprehension
0.241
(0.468)
1.220
(0.439)
-0.101
(0.538)
1.137
(0.424)
T2: Instilment of analytical and
critical skills
0.887
(0.468)
0.163
(0.414)
0.631
(0.575)
0.459
(0.400)
T3: Instilment of capacity for
individual study
-0.885
(0.339)
-0.291
(0.296)
-0.097
(0.405)
-0.547
(0.298)
T4: Transparency, fairness and
feedback
0.503
(0.303)
-0.244
(0.286)
0.367
(0.358)
0.120
(0.284)
8922
9254
6645
11468
A. Five plus sample
B. Ten plus sample
N
Notes: This table reports pupil and school fixed effects estimates of the effect of the four teaching practice
measures on various subsamples of students. All four teaching practices are entered simultaneously as treatment
variables in the regressions. The High and low SES samples refer to high and low parental schooling,
respectively. Robust standard errors are presented in parenthesis. See notes to Table 5 for list of controls that
are included in each of the four regressions.
Table 9: Effect of Teaching Practices by Percentiles of Test Scores
Distribution
The outcome measures are the
product of the z score and a
0/1 dummy indicator of
percentile ranking of students
Above 25th percentile
Above 50th percentile
Above 75th percentile
Above 80th percentile
Above 90th percentile
N
T1: Instilment of
knowledge and
enhancement of
comprehension
T2: Instilment of
analytical and
critical skills
(1)
(2)
0.420
0.108
(0.155)
(0.150)
0.367
0.271
(0.181)
(0.175)
0.123
0.311
(0.165)
(0.159)
0.038
0.247
(0.154)
(0.149)
0.074
0.178
(0.121)
(0.117)
20660
Notes: This table reports pupil and school fixed effects estimates of the effect of the first
two teaching practice measures on test scores. The two measures are entered jointly as
treatment variables in the regressions. The estimates in each row are from one regression
based on the five plus sample. The dependant variable is a product of the z_score and the
dummy indicator for the specified percentile. The Panel includes pupils from secular
schools, with 5 or more pupils in the panel. Robust standard errors are reported in
parenthesis.
Table A1: Correlations Between Treatment Variables
T1
(1)
T2
(2)
T3
(3)
T1
1
T2
0.810
1
T3
0.809
0.873
1
T4
0.543
0.439
0.436
Notes:
T1: Instilment of knowledge and enhancement of comprehension
T2: Instilment of analytical and critical skills
T3: Instilment of capacity for individual study
T4: Transparency, fairness and feedback
T4
(4)
1
Table A2: Effect of Teaching Practices on Pupils Test Scores by Subjects
English
Hebrew
Math
Science
(1)
(2)
(3)
(4)
1.030
0.550
0.747
0.869
(0.492)
(0.549)
(0.482)
(0.579)
0.578
0.631
0.015
0.898
(0.476)
(0.527)
(0.464)
(0.565)
5100
5096
5292
5172
T1: Instilment of knowledge and
enhancement of comprehension
0.987
(0.527)
0.556
(0.583)
0.931
(0.509)
1.008
(0.613)
T2: Instilment of analytical and
critical skills
0.470
0.715
0.060
1.025
(0.510)
(0.561)
(0.490)
(0.603)
4494
4496
4640
4538
A. Five plus sample
T1: Instilment of knowledge and
enhancement of comprehension
T2: Instilment of analytical and
critical skills
N
B. Ten plus sample
N
Notes: Columns 1-4 report the estimates and robust standard errors (in parentheses) from regressions of the four
treatment variables on pupils' achievements for each subject separately. All four teaching practices are included jointly
as treatment variables in the regressions. See notes to Table 5 for the controls included in these regressions. The
estimates presented in panel A are based on the five plus sample and the estimates in panel B are based on the ten plus
sample.
Table A3: Estimates of the Effect of Teaching Practices on Pupils’ Test Scores:
English, Hebrew and Science
All
Boys
Girls
High SES
Low SES
(1)
(2)
(3)
(4)
(5)
T1: Instilment of knowledge and
enhancement of comprehension
0.824
(0.341)
0.072
(0.520)
1.198
(0.492)
-0.201
(0.588)
1.062
(0.483)
T2: Instilment of analytical and
critical skills
0.679
(0.329)
0.864
(0.519)
0.446
(0.462)
0.630
(0.616)
0.476
(0.456)
T3: Instilment of capacity for
individual study
-0.180
(0.241)
-0.381
(0.375)
-0.195
(0.335)
0.024
(0.454)
-0.364
(0.339)
T4: Transparency, fairness and
feedback
0.212
(0.222)
0.469
(0.334)
-0.004
(0.321)
0.403
(0.381)
0.131
(0.324)
N
15374
7605
7769
5878
9448
T1: Instilment of knowledge and
enhancement of comprehension
0.848
(0.360)
0.045
(0.554)
1.313
(0.506)
-0.511
(0.630)
1.201
(0.494)
T2: Instilment of analytical and
critical skills
0.712
(0.350)
1.024
(0.555)
0.320
(0.478)
1.020
(0.673)
0.598
(0.469)
T3: Instilment of capacity for
individual study
-0.301
(0.254)
-0.705
(0.402)
-0.230
(0.342)
-0.049
(0.475)
-0.514
(0.350)
T4: Transparency, fairness and
feedback
0.248
(0.236)
0.548
(0.359)
-0.057
(0.330)
0.497
(0.418)
0.087
(0.333)
N
13534
6639
6895
4975
8514
A. Five plus sample
B. Ten plus sample
Notes: This table reports pupil and school fixed effects estimates of the effect of the four teaching
practice measures on pupils test scores measured as z scores. Test scores in mathematics are excluded
from these regressions. The z scores are computed based on the full sample. The controls added are four
subject dummies, class size, class means of the pupils' characteristics and several class climate measure
(class means of level of noise, incidence of violence, discipline). The estimates presented in panel A are
based on the five plus sample and the estimates in panel B are based on the ten plus sample.