SCHOOL EFFECTIVENESS IN DEVELOPED
AND DEVELOPING COUNTRIES; A REVIEW
OF THE RESEARCH EVIDENCE
Jaap Scheerens
University of Twente/ The World Bank
June 1999
Abstract
Research approaches and substantive outcomes of different strands of educational
effectiveness research in industrialized countries are summarized in the first part of the
paper. Concerning education production function research, the debate on the significance of
resource input factors like teacher/pupil ratio and teacher qualifications has flared up during
the last five years, challenging the message “money does not matter” that emerged from
earlier research reviews. Research on effective schools, is developing into more integrated,
multi-level studies in which school environmental variables, school inputs, school
organizational conditions and instructional practices are all included. The study of effective
teaching and instruction has focused on classroom management and structured didactic
approaches; a perspective that is challenged by constructivist views on teaching.
Results altogether indicate a modest impact of resource input and school organizational
factors and a medium-size impact of instructional conditions.
Reviews of studies on educational effectiveness in developing countries show a strong
predominance of the production function type of study. Only a small minority of studies
include school organizational and instructional variables. Results show greater frequency of
resource input factors having a significant impact than in the case of industrialized countries.
There is little evidence so far on the impact of instructional conditions. It is argued that a
blending of research orientations in the choice of independent variables and methodology
between the two “worlds” could improve research practice in both contexts. The available
knowledge base can be applied as a basis for selecting relevant input and process conditions
of schooling in different types of evaluation studies. Synergy between program evaluation,
effectiveness research and monitoring is seen as an important condition for furthering applied
research. A second type of application, the use of the school effectiveness knowledge base for
the planning of education projects in developing countries, should be approached with much
prudence and sensitivity of cultural contingencies. A tentative set of suggestions is presented.
JS.99.016 - 15-2-2016
1
Introduction
This paper is divided into two parts. In the first part, after an introduction of the concept of
school effectiveness, a categorization of educational effectiveness research in industrialized
countries is presented. The main research outcomes are summarized.
Research traditions and outcomes of effectiveness research in OECD-countries are used as a
basis for comparison with studies conducted in developing countries. Here the results of
available reviews are complemented by an analysis of more recent studies by the author.
In the discussion the comparison is used as a source of inspiration for substantive priorities
and methodology of future effectiveness studies and suggestions for application of the
knowledge base.
PART 1
EVIDENCE FROM OECD (ORGANIZATION
DEVELOPMENT) COUNTRIES
OF
ECONOMIC CO-OPERATION
AND
The Concept of School Effectiveness
School A can be called more effective than school B when school A does better in achieving
its core objectives. This is the general definition of school effectiveness that emerges from
about three decades of research conducted under this label.
It is a definition that requires more precision, and, moreover, even when further clarified,
remains debatable.
First of all, the comparison in the general definition should be “fair”, which means that goalattainment measures should be adjusted for possibly diverging entrance characteristics of the
units (i.e. the students) on which these measures are taken. In research practice this means
that outcome measures that reflect goal-attainment are to be corrected for prior achievement,
proxy’s like scholastic aptitude or social-economic status or both (Bosker, 1995). This is also
known as the “value-added” perspective in determining school effectiveness. The fact that the
determination of school effectiveness is generally conceptualized as a comparative endeavour
should also be explicitly underlined. Schools are compared among themselves on value-added
effectiveness criteria rather than being judged by employing absolute standards.
Secondly, “goal attainment” in schooling could mean many different things. What goals?
Being the obvious question to be answered. Cheng (1996) illustrates the complexity of this
question by referring to various functions of schooling (technical/economic functions, human/
social functions, political functions, cultural functions and educational functions), each of
which likely to emphasize different categories of educational objectives. Goal-attainment of
schooling can thus be defined in terms of varying long-term societal effects and in terms of
more direct attainment categories at the end of a fixed period of schooling. But also with
respect to these more direct attainment categories there are various possibilities and priorities
to be set among them: cognitive vs. non-cognitive outcomes and, within the cognitive
JS.99.016 - 15-2-2016
2
domain, various types of knowledge and skills, varying from basic subject-matter mastery to
higher order problem solving skills.
Thirdly, from an organization-theoretical perspective the concept of school effectiveness
could be defined in even broader terms. According to typologies on organizational
effectiveness (Cameron & Whetten, 1983; Scheerens, 1992; Cheng, 1996). The “goalattainment” model, implied in the above general definition, is just one of several models of
organizational effectiveness. The goal-attainment model uses “productivity” of the
organization’s primary process as the central effectiveness criterion. Other models, like the
resource-input model and the organization process model emphasize other criteria, namely
procured resources and student intake and smooth internal functioning respectively.
Finally, it should be noted that in school effectiveness research, rather than in some more
applied contexts of determining school effects, the inquiry does not stop by, for example
rank-ordering schools on the basis of their value-added performance but, in addition, aims to
answer the question to which specific characteristics of school organization or instruction,
such differences could be attributed to. School effectiveness is intrinsically a causal concept,
in which the black box of “a school” is opened in order to reveal specific variables that relate
to the effect criterion. Gradually, as will be explained in subsequent sections school
effectiveness research has lead to the development of causal models in which these various
characteristics are related to each other and the effect criterion.
These four points show that in order to use school effectiveness in more operational terms in
a research context, there are various choices to be made:
a) choice of an effect criterion, where one could choose among indicators of the more long
term outcomes of schooling, between goal-attainment or alternative criteria of
organizational effectiveness and between various domains of goal-attainment;
b) choice of one or more variables to adjust effect measures;
c) choice of hypothetical effectiveness enhancing conditions as possible attributes of
differences in effect measures and a particular causal model in which these conditions are
interrelated.
Choices with respect to (a) are of a substantive nature, whereas choices with respect to (b)
and (c) are of a more technical nature. In the various strands of educational effectiveness
research in which school effectiveness research has become integrated, a rather limiting
choice with respect to the effectiveness criterion is evident. In the large majority of
educational effectiveness studies, achievement in basic school subjects, reading and writing in
the native language and mathematics, is used as effect-criterion. It should also be noted that
the bulk of educational effectiveness research is carried out at the level of primary and lower
secondary schools. In this context a discussion on the educational significance of this limiting
choice of effect criteria in educational effectiveness research will be left out (see Scheerens,
1992; Cheng, 1996). The implicit position in continuing this review, however, being that
achievement in basic school subjects is sufficiently important to figure out how such
outcomes are best accomplished.
JS.99.016 - 15-2-2016
3
The choice of hypothetical effectiveness enhancing conditions depends on the disciplinary
background from which research is conducted. The evolvement of different traditions of
educational effectiveness research in what shall be referred to as “integrated school
effectiveness research” and the implications for conceptual modelling will be addressed in the
next section.
Before describing this development, however, it is necessary to briefly refer to a set of critical
questions about the scope of the concept of school effectiveness, some of which were raised
in an article by Ralph and Fenessey (1983). Can a school be called effective on the basis of
achievement results measured only at the end of a period of schooling, or should such a
school be expected to have high performance at all grade levels? Can school effectiveness be
assessed by examining results in just one of two school subjects, or should all subject matter
areas of the curriculum be taken into account? And: shouldn’t one restrict the qualification of
a school being effective to consistently high performance over a longer period of time, rather
than a “one shot” assessment at just one point in time?
Fortunately all of these questions are amenable to empirical research, This type of studies that
have to do with the consistency of school effects over grade-levels, teachers, subject-matter
areas and time have sometimes be referred to as “foundational studies” (Scheerens, 1993),
because they are aimed at resolving issues that bear upon the scope and “integrity” of the
concept of school effectiveness.
A recent review of such foundational studies is given in Scheerens & Bosker, 1997 (ch. 3).
Their results concerning primary schools are presented in Table 1. Consistency is expressed
in terms of the correlation between two different rank orderings of schools. Results are based
on arithmetic and language achievement.
Table 1: Consistency of school effectiveness (primary level); from Scheerens & Bosker, 1997
(ch. 3).
Type of consistency
across time (stability)
(1 or 2 years)
across grades
across subjects
Average correlation
r = .70 (.34 - .87)
r = .50 (.20 - .69)
r = .70 (.59 - .83)
The results summarized in Table 1 indicate that there is a reasonable consistency across
cohorts and subjects, while the consistency across grades is only average. Results measured at
the secondary level likewise show reasonably high stability coefficients (consistency across
cohorts), somewhat lower coefficients for stability across grades (for example in a French
study (Grisay, 1996) coefficients based on value-added results were .42 for French language
and .27 for mathematics. The average consistency between subjects at the secondary level
JS.99.016 - 15-2-2016
4
was somewhat lower than in the case of primary schools (r about .50). This phenomenon can
be explained by the fact that, at the secondary level, different teachers usually teach different
subjects, so that inconsistency is partly due to variation between teachers.
The few studies in which factor analysis was used to examine the size of a stable school
factor relative to year specific and subject specific effects have shown results varying from a
school factor explaining 70% of the subject and cohort specific (gross) school effects (Bosker,
1990), to 39% (Van der Werf & Guldemond, 1995) and 25% (Luyten, 1994).
The picture that emerges from these studies on the stability and consistency of school effects
is far from being undifferentially favourable with respect to the unidimensionality of the
school effects concept. Consistency is fair, when effects at the end of a period of schooling
are examined over a relatively short time interval. When grade-level and subject-matter area
are brought into the picture consistency coefficients tend to be lower, particularly when
different teachers teach different grades or subjects. Especially at the secondary school level
school effects are to a considerable amount to be seen as teacher effects.
The message from these “foundational studies” is that one should be careful not to
overgeneralize the results of school effectiveness studies when only results in one or two
subject matter areas at one point in time are measured. Another implication is that
hypothetical antecedent conditions of effects are not only to be sought at the school
organizational level, but also at the level of teaching and the teaching and learning process.
Turning back now to the definition of school effectiveness that was given in the first sentence
of this section, the results of the subsequent conceptual analysis can be summarized as a set of
qualifications of this general characterization:
 the comparison of schools is to be based on value-added effect measures;
 the choice of just a few core objectives as a basis for the effect criterion indicates a
limitation of most empirical school effectiveness studies with respect to more
encompassing perspectives on organization effectiveness;
 not just the effectiveness of school “as a whole”, but also specific school and instructional
characteristics are the antecedent conditions in school effectiveness research;
 as empirical studies have shown that the unidimensionality of school effects is somewhat
uncertain, care should be taken not to overgeneralize the results of studies involving just a
few subject-matter areas, sub-systems of the school and one point in time when effects are
measured.
Integration of Research Traditions in Educational Effectiveness
The elementary design of school effectiveness research is the association of hypothetical
effectiveness enhancing conditions of schooling and output measures, mostly student
JS.99.016 - 15-2-2016
5
achievement. A basic model from systems theory, where the school is seen as a black box,
within which processes or “throughput” take place to transform this basic design. The
inclusion of an environmental or context dimension completes this model (see Figure 1). The
major task of school effectiveness research is to reveal the impact of relevant input
characteristics on output and to “break open” the black box in order to show which process or
throughput factors “work”, next to the impact of contextual conditions. Within the school it is
helpful to distinguish a school and a classroom level and, accordingly, school organizational
and instructional processes.
context
inputs
process or throughput
outputs
school level
classroom level
Figure 1: A basic systems model of school functioning
Research tradition in educational effectiveness varies according to the emphasis that is put on
the various antecedent conditions of educational outputs. These traditions also have a
disciplinary basis. The common denominator of the five areas of effectiveness research that
will be distinguished is that in each case the elementary design of associating outputs or
outcomes of schooling with antecedent conditions (inputs, processes or contextual) applies.
The following research areas or research traditions will be considered:
1) Research on equality of opportunities in education and the significance of the school in
this.
2) Economic studies on education production functions.
3) The evaluation of compensatory programs.
4) Studies of unusually effective schools.
5) Studies on the effectiveness of teachers, classes and instructional procedures.
re 1) School effectiveness in equal educational opportunity research
Coleman’s research into educational opportunity, about which a final report known as the
Coleman report was published in 1966, forms the corner-stone for school effectiveness
studies (Coleman et al., 1966). While this study was intended to show the extent to which
JS.99.016 - 15-2-2016
6
school achievement is related to students’ ethnic and social background, the possible
influence of the “school” factor on learning attainment was also examined.
In the survey three clusters of school characteristics were measured: (a) teacher
characteristics; (b) material facilities and curriculum; and (c) characteristics of the groups or
classes in which the pupils were placed. After the influence of ethnic origin and socioeconomic status of the pupils had been statistically eliminated, it appeared that these three
clusters of school characteristics together accounted for 10 percent of the variance in pupil
performance. Moreover, the greater part of this 10 percent variance was due to the third
cluster that was operationalized as the average background characteristics of pupils, which
means that again the socio-economic and ethnic origin - now defined at the level of the school
- played a central role. In reactions to the Coleman report there was general criticism on the
limited interpretation of the school characteristics. Usually, only the material characteristics
were referred to, such as the number of books in the school library, the age of the building,
the training of the teachers, their salaries and expenditure per pupil. Nevertheless there were
other characteristics included in Coleman’s survey, such as the attitude of school heads and
teachers towards pupils and the attitude of teachers towards integrated education, i.e.
multiracial and classless teaching.
Other large-scale studies that were primarily focused at providing data on equality of
opportunity are those by Jencks et al. (1972, 1979), Alexander and Eckland (1980), and
Hauser, Sewell and Alwin (1976). Thorndike’s (1973) study, although not explicitly
dedicated to equality of opportunity, also examined school careers in relationships to the
environmental background of pupils.
The overall results of these studies indicated a relatively high correlation between socioeconomic and ethnic family characteristics and learning attainment, and a small or even
negligible influence from school and instruction characteristics. The outcomes were criticized
by educationalists for the rather narrow choice of school characteristics and on
methodological grounds (cf. Aitkin & Longford, 1986), for multi-level associations not being
properly modelled and analyzed.
re 2) Economic studies on educational production functions
The focus of economic approaches towards school effectiveness is the question of what
manipulative inputs can increase outputs. If there was stable knowledge available on the
extent to which variety of inputs is related to variety of outputs it would also be possible to
specify a function which is characteristic of the production process in schools - in other
words, a function which could accurately indicate how a change in the inputs would affect the
outputs.
This leads to a research-tradition that is identified both by the term input-output studies as by
the term research into education production functions. The research model for economicsrelated production studies hardly differs from that for other types of effectiveness research:
JS.99.016 - 15-2-2016
7
the relationship between manipulative school characteristics and attainment is studied while
the influence of background conditions like social class and pupils’ intelligence is eliminated
as far as possible. The specific nature of production-function research is the concentration on
what can be interpreted in a more literal sense as input characteristics: the teacher/pupil
relationship, teacher training, teacher experience, teachers’ salaries and expenditure per pupil.
In more recent observations of this research type one comes across the suggestion to take
effectiveness predictors known from educational psychology research into account
(Hanushek, 1986). It should be noted that the Coleman-report (Coleman et.al.1966) is often
included in the category of input-output studies. In view of its emphasis on the more material
school characteristics, the association is an obvious one.
The findings of this type of research have often been referred to as being disappointing.
Review studies like those from Mosteller and Moynihan (1972), Averch et al. (1974),
Glasman and Biniaminov (1981), Hanushek (1979 and 1986) always produce the same
conclusions: inconsistent findings throughout the entire available research and scant effect at
most from the relevant input variables.
From reanalysis of Hanushek’s (1986) dataset, Hedges et al. (1994), however, conclude that
there is an effect of per pupil expenditure of “considerable practical importance” (an increase
of PPE by $510 would be associated with a 0.7 s.d. increase in student outcome).
But this conclusion in its turn is contested by Hanushek.
In table 5 cited from Hanushek, 1997, the most recent “vote count” overview of education
production function studies is given.
Table 2: Percentage Distribution of Estimated Effect of Key Resources on Student
performance, Based on 377 Studies (cited from Hanushek, 1997, p. 144)
Resources
Real classroom resources
Teacher pupil ratio
Teacher education
Teacher experience
Financial aggregates
Teacher salary
Expenditure per pupil
Number of
estimates
Statistically significant
Positive Negative
Statistically insignificant
Positive Negative
Unknown
sign
277
171
207
15%
9
29
13%
5
5
27%
33
30
25%
27
24
20%
26
12
119
163
20%
27
7%
7
25%
34
20%
19
28%
13
Hanushek’s interpretation of these results is that there can be little confidence that adding
more of any of the specific resources or, for that matter of the financial aggregates, will lead
to a boost in student achievement. The variable that shows relatively the highest proportion of
positive effects is teacher experience, but here, “reverse causation” could be at play, since
more experienced teachers might have selected schools with better performing pupils (ibid, p.
144).
JS.99.016 - 15-2-2016
8
In other reviews, e.g. Verstegen & King (1998), a more positive interpretation is given on
largely the same set of studies that was analyzed by Hanushek (1997). During the last decade
several studies drew attention to the fact that certain resource input factors did show
significant positive associations with pupil achievement or other educational outcomes. The
most important of these are the studies by Card & Krueger (1992), which indicated a positive
association between school resources and differences in earnings among workers, Hedges,
Laine & Greenwald (1994) who conducted a statistical meta-analysis on a sub-set of
Hanushek’s 1979 data set and found significant effects for several resource input variables,
among which is rather large positive effect of Per Pupil Expenditure, Ferguson (1991) who
found particularly large effects of variables related to teacher qualifications (specifically
scores on a teacher recertification test), and Achilles (1996) who reported the sustained
effects of reduced class-size (14-16 as compared to 22-24) in Kindergarten and the first three
grades of primary school) on student achievement.
That these differences in interpretation are to a certain degree of the kind: “the cup is half
full” as compared to “the cup is half empty” is illustrated by Verstegen & King’s (1998)
presentation of table 6, cited from Hanushek, 1997.
Table 3: Verstegen & King’s (1998) rendering of Hanushek’s (1997, p. 144) tabulation.
Percentage Distribution of Significant Estimated Effects of Key Resources on
Student Achievement, Based on 377 Studies
Statistically significant
Number of
Positive
Negative
Estimates (no.)
(%)
(%)
Real Classroom Resources
Teacher-pupil ratio
78
54
46
Teacher education
24
64
36
Teacher experience
70
85
15
Financial aggregates
Teacher salary
32
74
26
Expenditure per pupil
55
79
21
By omitting the large proportions of studies showing insignificant results, and “blowing up”
the relatively small numbers of studies showing significant results to percentages, these
authors appear to be keen to see (or construct) the bright side of things.
Unfortunately, as in other types of educational effectiveness studies, the critics and those who
present the more conservative interpretation appear to have the best arguments. Hanushek,
1997, presents most of them:
JS.99.016 - 15-2-2016
9
-
-
-
when outcome measures, such as student achievement scores are properly adjusted for
student background characteristics, and “value added” outcome indicators are used, the
number of positive effects declines;
if data at high aggregation levels (e.g. individual states) is used misspecification bias is
likely to produce overstatement of effects (this criticism would apply to both the Ferguson
and Card & Krueger studies). This problem frequently occurs for the variable Per Pupil
Expenditure which is usually only defined at the district level;
in statistical meta-analysis the null-hypothesis that is addressed is that resources or
expenditure differences never, under whatever circumstances, affect student performance;
clearly this hypothesis is to be rejected also in cases where only a minority of studies
shows a significant positive association with the outcome variable.
Many of the recent contributions to summarizing the research evidence on education
production function studies mention the need to search for answers to the question “why
money does or does not matter”, for example by looking for combinations and interactions
between resource input levels and school organizational and instructional variables. In a
recent collection of articles on class size (Galton, 1999) reference is made to differences
between educational cultures in the degree to which large classes are considered a burden to
teachers.
Another desirable extension of the basic education production function type of study would
be to address questions of cost-effectiveness more directly, by comparing cost-effectiveness
or even cost-benefit ratio’s for different policy measures. A comparison of education
production function studies between industrialized and developing countries is particularly
interesting, since a “restriction of range” phenomenon (little variance in, for example, teacher
salaries between schools) might surpress the effects in relatively homogenous school systems.
Results of education production function studies in developing countries will be presented in
a subsequent section.
re 3) The evaluation of compensatory programs
Compensatory programs may be seen as the active branch in the field of equal educational
opportunity. In the United States compensatory programs like “Head Start” were part of
President Johnson’s “war on poverty”. Other large-scale American programs were “FollowThrough” - the sequel to Head Start - and special national development programs that resulted
from Title 1 of the Elementary and Secondary Education Act, enacted in 1965. Compensatory
programs were intended to improve the levels of performance of the educationally
disadvantaged. In the late sixties and early seventies there were also similar programs in the
Netherlands like the Amsterdam Innovation project, the Playgroup Experiment project,
Rotterdam’s Education and Social Environment (OSM) project and the Differentiated
Education project (GEON) of the city of Utrecht.
JS.99.016 - 15-2-2016
10
Compensatory programs manipulate school conditions in order to raise achievement levels of
disadvantaged groups of pupils. The level in which this is achieved demonstrates the
importance of the school factor - and in particular the conditions and educational provisions
within it.
However, it proved to be not that simple to redress the balance with effective compensatory
programs. In fact no overwhelming successes could be established. There was heated debate
on the way available evaluation studies should be interpreted.
The key question is: what results can be realistically expected from compensatory education
given the dominant influence in the long run of family background and cognitive aptitudes on
pupils’ attainment level? Scheerens (1987, p. 95) concluded that the general image provided
by the evaluation of compensatory programs reveals that relatively small progress in
performance and cognitive development can be established immediately after a program
finishes. Long-term effects of compensatory programs cannot be established by and large.
Moreover, it has been occasionally demonstrated that it was the “moderately” disadvantaged
in particular that benefited from the programs, while the most educationally disadvantaged
pupils made the least progress, relatively speaking.
In view of the variety of compensatory programs the evaluation studies gave some insight into
the relatively best type of educational provision. When comparing the various components of
Follow Through, programs aimed at developing elementary skills like language and
mathematics and which used highly structured methods turned out to be winners (Stebbins et
al., 1977; Bereiter & Kurland, 1982; Haywood, 1982).
As will appear later, there is a remarkable similarity between these characteristics and the
findings of other types of effectiveness research. In any case, when interpreting the results of
evaluations of compensatory programs one should be aware that the findings have been
established among a specific pupil population: very young children (infants or first years of
junior school) from predominantly working-class families.
re 4) Effective schools research
Research known under labels like “identifying unusually effective schools” or the “effective
schools movement” can be regarded as the type of research that most touches the core of
school effectiveness research. In Coleman’s and Jencks’ surveys the inequality of educational
opportunity was the central problem. In economic-related input-output studies the school was
even conceived as a “black box”. In the still to be discussed research on the effectiveness of
classes, teachers and instruction methods, education characteristics on a lower aggregation
level than the school are the primary research object.
Effective school research is generally regarded as a response to the results of studies like
Coleman’s and Jencks’ from which it was concluded that schools did not matter very much
when it came down to differences in levels of achievement. From titles such as “Schools can
make a difference” (Brookover et al., 1979) and “School matters” (Mortimore et al., 1988) it
appears that refuting this message was an important source of inspiration for this type of
JS.99.016 - 15-2-2016
11
research. The most distinguishing feature of effective schools research was the fact that it
attempted to break open the “black box” of the school by studying characteristics related to
organization, form and content of schools.
The results of the early effective schools research converged more or less around five factors:
 strong educational leadership;
 emphasis on the acquiring of basic skills;
 an orderly and secure environment;
 high expectations of pupil attainment;
 frequent assessment of pupil progress.
In the literature this summarizing is sometimes identified as the “five-factor model of school
effectiveness”. It should be mentioned that effective schools research has been largely carried
out in primary schools, while at the same time studies have been largely conducted in inner
cities and in predominantly working-class neighborhoods.
In more recent contributions effective schools research became more integrated with
education production function and instructional effectiveness research, in the sense that a
mixture of antecedent conditions was included, studies evolved from comparative casestudies to surveys and conceptual and analytical multi-level modeling took place to analyze
and interpret the results. Numerous reviews on school effectiveness have been published
since the late seventies. Early reviews are those by Anderson (1982), Cohen (1982),
Dougherty (1981), Edmonds (1979), Murnane (1981), Neufeld et al. (1983), Purkey and
Smith (1983), Rutter (1983), Good and Brophy (1986), Ralph and Fenessey (1983), Kyle
(1985), and Sweeney (1982). More recent reviews are those by Levine and Lezotte (1990),
Scheerens (1992), Creemers (1994), Reynolds et al. (1993), Sammons et al. (1995), and
Cotton (1995).
The focal point of interest in the reviews is the “what works” question; typically the review
presents lists of effectiveness enhancing conditions.
There is a fairly large consensus on the main categories of variables that are distinguished as
effectiveness enhancing conditions in the reviews, also when earlier and more recent reviews
are compared.
Table 4 summarizes the characteristics listed in the reviews by Purkey and Smith (1983),
Scheerens (1992), Levine and Lezotte (1990), Sammons et al. (1995), Cotton (1995).
JS.99.016 - 15-2-2016
12
Table 4: Effectiveness enhancing conditions of schooling in five review studies (italics in the
column of the Cotton study refers to sub-categories).
Purkey & Smith,
1983
Levine & Lezotte,
1990
Achievementoriented policy;
cooperative
atmosphere, orderly
climate
Productive climate
and culture
Clear goals on basic
skills
Focus on central
learning skills
Frequent evaluation
Appropriate
monitoring
In-service training/
staff development
Practice-oriented
staff development
Strong leadership
Outstanding
leadership
Salient parent
involvement
Time on task,
reinforcement,
streaming
Effective
instructional
arrangements
High expectations
High expectations
JS.99.016 - 15-2-2016
Scheerens, 1992
Sammons, Hillman
& Mortimore, 1995
Pressure to achieve, Planning and
Shared vision and
consensus,
learning goals,
goals, a learning
cooperative
curriculum planning environment,
planning, orderly
and development
positive
atmosphere
reinforcement
Planning and
Concentration on
learning goals school teaching and
wide emphasis on
learning
learning
Evaluative potential Assessment (district, Monitoring progress
of the school,
school, classroom
monitoring of pupils’ level)
progress
Professional
A learning
development
organization
collegial learning
Educational
School management Professional
leadership
and organization,
leadership
leadership and
school improvement,
leadership and
planning
Parent support
Parent community
Home school
involvement
partnership
Structured, teaching, Classroom
Purposeful teaching
effective learning
management and
time, opportunity to organization,
learn
instruction
Teacher student
High expectations
interactions
Pupil rights and
responsibilities
Distinct-school
interactions
Equity
Special programs
External stimuli to
make schools
effective
Physical and
material school
characteristics
Teacher experience
School context
characteristics
13
Cotton, 1995
Consensus is largest with respect to the factors:
 achievement orientation (which is closely related to “high expectations”);
 co-operation;
 educational leadership;
 frequent monitoring;
 time, opportunity to learn and “structure” as the main instructional conditions.
Behind this consensus on general characteristics hides considerable divergence in the actual
operationalization of each of the conditions. Evidently concepts like “productive,
achievement-oriented climate and educational leadership are complex concepts and individual
studies may vary in the focus that different elements receive.
Scheerens and Bosker (1997, ch. 4) provide an analysis of the meaning of the factors that are
considered to work in schooling apparent from the actual questionnaires and scales as used in
ten empirical school effectiveness studies.
Their summary table, in which the main components of thirteen general factors are
mentioned, is cited below as Table 5.
Table 5: Components of fourteen effectiveness-enhancing factors
Factors
Achievement, orientation,
high expectations
Educational leadership
Consensus and cohesion
among staff
Curriculum quality/
opportunity to learn
School climate
JS.99.016 - 15-2-2016
Components
 clear focus on the mastering of basic subjects
 high expectations (school level)
 high expectations (teacher level)
 records on pupils’ achievement
 general leadership skills
 school leader as information provider
 orchestrator or participative decision making
 school leader as coordinator
 meta-controller of classroom processes
 time educational/administrative leadership
 counselor and quality controller of classroom teachers
 initiator and facilitator of staff professionalization
 types and frequency of meetings and consultations
 contents of cooperation
 satisfaction about cooperation
 importance attributed to cooperation
 indicators of successful cooperation
 the way curricular priorities are set
 choice of methods and text books
 application of methods and text books
 opportunity to learn
 satisfaction with the curriculum
a) orderly atmosphere
 the importance given to an orderly climate
 rules and regulations
 punishment and rewarding
 absenteeism and drop out
 good conduct and behaviour of pupils
 satisfaction with orderly school climate
14
b)









Evaluative potential
Parental involvement
Classroom climate
Effective learning time






















climate in terms of effectiveness orientation and good internal relationships
priorities in an effectiveness-enhancing school climate
perceptions on effectiveness-enhancing conditions
relationships between pupils
relationships between teacher and pupils
relationships between staff
relationships: the role of the head teacher
engagement of pupils
appraisal of roles and tasks
job appraisal in terms of facilities, conditions of labour, task load and general
satisfaction
facilities and building
evaluation emphasis
monitoring pupils’ progress
use of pupil monitoring systems
school process evaluation
use of evaluation results
keeping records on pupils’ performance
satisfaction with evaluation activities
emphasis on parental involvement in school policy
contacts with parents
satisfaction with parental involvement
relationships within the classroom
order
work attitude
satisfaction
importance of effective learning
time
monitoring of absenteeism
time at school
time at classroom level
classroom management
homework
re 5) Studies on instructional effectiveness
As the most relative strands of research on teaching and classroom processes for the topic at
hand are studies on characteristics of effective teachers, and studies that go under the label of
“process-product studies”. This latter category of studies was also inspired by Carroll’s
(1963) model of teaching and learning and off-springs of this model, such as the models of
mastery learning (Bloom, 1976) and “direct teaching” (e.g. Doyle, 1985).
The research results have been reviewed by, among others, Stallings (1985), Brophy and
Good (1986), and Creemers (1994) and quantitatively synthesized in meta-analyses by
Walberg (1984), Fraser et al. (1987) and Wang, Haertel and Walberg (1993). These latter
authors incidentally have also included variables outside the classroom situation, like the
student’s relationships with peers, and the home environment (e.g. television viewing) in their
analyses which they label under the heading of “educational productivity”.
JS.99.016 - 15-2-2016
15
In the sixties and seventies the effectiveness of certain personal characteristics of teachers
was particularly studied. Medley & Mitzel, 1963; Rosenshine & Furst, 1973 and Gage, 1965
are among those who reviewed the research findings. From these it emerged that there was
hardly any consistency found between personal characteristics of the teacher like
warmheartedness or inflexibleness on the one hand, and pupil achievement on the other.
When studying teaching styles (Davies, 1972), the behavioural repertoire of teachers was
generally looked at more than the deeply-rooted aspects of their personality. Within the
framework of “research on teaching” there followed a period in which much attention was
given to observing teacher behaviour during lessons. The results of these observations,
however, in as far as they were related to pupil achievement, seldom revealed a link with
pupil performance (see Lortie, 1973, for instance). In a following phase more explicit
attention was given to the relation between observed teacher behaviour and pupil
achievement. This research is identified in the literature as “process-product studies”.
Variables which emerged “strongly” in the various studies were the following (Weeda, 1986,
p. 68):
1. Clarity: clear presentation adapted to suit the cognitive level of pupils.
2. Flexibility: varying teaching behaviour and teaching aids, organizing different activities
etc.
3. Enthusiasm: expressed in verbal and non-verbal behaviour of the teacher.
4. Task related and/or businesslike behaviour: directing the pupils to complete tasks, duties,
exercises etc. in a businesslike manner.
5. Criticism: much negative criticism has a negative effect on pupil achievement.
6. Indirect activity: taking up ideas, accepting pupils’ feelings and stimulating self-activity.
7. Providing the pupils with an opportunity to learn criterion material - that is to say, a clear
correspondence between what is taught in class and what is tested in examinations and
assessments.
8. Making use of stimulating comments: directing the thinking of pupils to the question,
summarizing a discussion, indicating the beginning or end of a lesson, emphasizing
certain features of the course material.
9. Varying the level of both cognitive questions and cognitive interaction.
In later studies effective teaching time became a central factor. The theoretical starting points
of this can be traced back to Carroll’s teaching-learning model (Carroll, 1963). Chief aspects
of this model are:
 actual net learning time which is seen as a result of:
perseverance and opportunity to learn;
 necessary net learning time as a result of:
pupil aptitude, quality of education and pupil ability to understand instruction.
The mastery learning model formulated by Bloom in 1976 was largely inspired from Carroll’s
model, and the same goes for the concept of “direct teaching”.
Doyle (1985) considers the effectiveness of direct teaching, which he defines as follows:
JS.99.016 - 15-2-2016
16
1. Teaching goals are clearly formulated.
2. The course material to be followed is carefully split into learning tasks and placed in
sequence.
3. The teacher explains clearly what the pupils must learn.
4. The teacher regularly asks questions to gauge what progress pupils are making and
whether they have understood.
5. Pupils have ample time to practice what has been taught, with much use being made of
“prompts” and feedback.
6. Skills are taught until mastery of them is automatic.
7. The teacher regularly tests the pupils and calls on the pupils to be accountable for their
work.
The question whether this type of highly structured teaching works equally well for acquiring
complicated cognitive processes in secondary education as for mastering basic skills at the
primary school level was answered in the affirmative (according to Brophy & Good, 1986, p.
367). However, progress through the subject matter can be taken with larger steps, testing
need not be so frequent and there should be space left for applying problem-solving strategies
flexibly. Doyle (ibid) emphasized the importance of varying the learning tasks and of creating
intellectually challenging learning situations. For the latter an evaluative climate in the
classroom, whereby daring to take risks even with a complicated task is encouraged, is a good
means.
In the domain of classroom organization Bangert, Kulik & Kulik’s meta-analysis (1983)
revealed that individual teaching in secondary education hardly led to higher achievement and
had no influence whatsoever on factors like the self-esteem and attitudes of pupils. “Bestevidence-syntheses” by Slavin (1996, p. 57) indicated a significantly positive effect of cooperative learning at the primary school level.
Meta-analyses by Walberg (1984) and Fraser et al. (1987) found the highest effects for the
following teaching conditions:
 reinforcement
 special programs for the educationally gifted
 structured learning of reading
 cues and feedback
 mastery learning of physics
 working together in small groups
It should be noted that more recently developed cognitive and particularly constructivist
perspectives on learning and instruction challenge the behaviouristically oriented approach
and results of the process-product research tradition (Cohen, 1988; Resnick, 1987; Collins et
al., 1988; Duffy & Jonassen, 1992; Scheerens, 1994; Brophy, 1996). According to the
constructivist approach independent learning, meta-cognition (e.g. learning to learn), “active
learning”, learning to model the behaviour of experts (“cognitive apprenticeship”) and
learning from real life situations (“situated cognition”) should be emphasized. The
JS.99.016 - 15-2-2016
17
effectiveness of teaching and learning according to these principles has not been firmly
established as yet. Authors who have addressed this issue (Scheerens, 1994; De Jong & Van
Joolingen, 1998) however, point out that a straightforward comparison with more structured
teaching approaches may be complicated, since constructivist teaching emphasizes different,
more higher order cognitive objectives. Moreover, structured versus “active” and “open”
teaching had probably be better conceived as a continuum of different mixes of structured and
“open” aspects, rather than as a dichotomy.
Integration
Of the five effectiveness-oriented educational research types, which were reviewed, two
focused on “material” school characteristics (such as teacher salaries, building facilities and
teacher/pupil ratio). The results were rather disappointing in that no substantial positive
correlations of these material investments and educational achievement could be established
in a consistent way across individual studies. On the basis of more recent studies these rather
pessimistic conclusions have been challenged, although methodological critique indicates that
the earlier pessimistic conclusions are more realistic. In-depth process studies connected with
large-scale evaluations of compensatory programs pointed out that programs which used
direct, i.e. structured, teaching approaches were superior to more “open” approaches. The
research movement known as research on exemplary effective schools (or briefly: effective
schools research) focused more on the internal functioning of schools than the earlier tradition
of input-output studies.
These studies produced evidence that factors like strong educational leadership, emphasis on
basic skills, an orderly and secure climate, high expectations of pupil achievement and
frequent assessment of pupil progress were indicative of unusually effective schools.
Research results in the field of instructional effectiveness are centered around three major
factors: effective learning time, structured teaching and opportunity to learn in the sense of a
close alignment between items taught and items tested.
Although all kinds of nuances and specificities should be taken into account when
interpreting these general results they appear to be fairly robust - as far as educational setting
and type of students is concerned. The overall message is that an emphasis on basic subjects,
an achievement-oriented orientation, an orderly school environment and structured teaching,
which includes frequent assessment of progress, is effective in the attainment of learning
results in the basic school subjects.
Table 6 summarizes the main characteristics of the five research traditions.
JS.99.016 - 15-2-2016
18
Table 6: General characteristics of types of school effectiveness research
independent
variable type
a. (un)equal
socio-economic
opportunities status and IQ of
pupil, material
school
characteristics
b. production
material school
functions
characteristics
c. evaluation
specific curricula
compensatory
programs
d. effective
“process”
schools
characteristics of
schools
e. effective
characteristics of
instruction
teachers,
instruction, class
organization
dependent
variable type
attainment
Discipline
Sociology
main study type
Survey
achievement
level
achievement
level
Economics
Survey
interdisciplinary
pedagogy
quasi-experiment
achievement
level
interdisciplinary
pedagogy
case-study
achievement
level
educational
psychology
Experiment
observation
In recent school effectiveness studies these various approaches to educational effectiveness
have become integrated. Integration was manifested in the conceptual modelling and the
choice of variables. At the technical level multi-level analysis has contributed significantly to
this development. In contributions to the conceptual modelling of school effectiveness,
schools became depicted as a set of “nested layers” (Purkey and Smith, 1983), where the
central assumption was that higher organizational levels facilitated effectiveness enhancing
conditions at lower levels (Scheerens & Creemers, 1989). In this way a synthesis between
production functions, instructional effectiveness and school effectiveness became possible, by
including the key variables from each tradition, each at the appropriate “layer” or level of
school functioning [the school environment, the level of school organization and
management, the classroom level and the level of the individual student]. Conceptual models
that were developed according to this integrative perspective are those by Scheerens (1990),
Creemers (1994), and Stringfield and Slavin (1992). Since the Scheerens model was used as
the starting point of the meta-analyses described in subsequent sections it is shown in Figure
2.
JS.99.016 - 15-2-2016
19
Context
. achievement stimulants from higher administrative levels
. development of educational consumerism
. 'covariables', such as school size, student-body composition,
school category, urban/rural
PROCESS
Inputs
. teacher
experience
. per pupil
expenditure
. parent support
School level
. degree of achievement-oriented
policy
. educational leadership
. consensus, cooperative planning
of teachers
. quality of school curricula in
terms of content covered, and
formal structure
. orderly atmosphere
. evaluative potential
Outputs
Student
achievement,
adjusted for:
. previous
achievement
. intelligence
. SES
Classroom level
. time on task (including
homework)
. structured teaching
. opportunity to learn
. high expectations of pupils'
progress
. degree of evaluation and
monitoring of pupils' progress
. reinforcement
Figure 2: An integrated model of school effectiveness (from Scheerens, 1990)
The choice of variables in this model is supported by the “review of reviews” on school
effectiveness research that will be presented in the next section.
Exemplary cases of integrative, multi-level school effectiveness studies are those by
Mortimore et al. (1988), Brandsma (1993), Hill et al. (1995), Sammons et al. (1995) and
Grisay (1996).
JS.99.016 - 15-2-2016
20
Summary of meta-analyses
In table 7 (cited from Scheerens and Bosker, 1997) the results of three meta-analysis and a reanalysis of an international data set have been summarized. The results concerning resource
input variables are based on the re-analysis of Hanushek’s (1989) summary of results of
production function studies that was carried out by Hedges, Laine & Greenwald, 1994. As
stated before this re-analysis was criticized, particularly the unexpectedly large effect of per
pupil expenditure.
The results on “aspects of structured teaching” are taken form meta-analyses conducted by
Fraser, Walberg, Welch and Hattie, 1987. The international analysis was based on the IEA
Reading Literacy Study and carried out by R.J. Bosker (Scheerens & Bosker, 1997, ch. 7).
The meta-analysis on school organizational factors, as well as the instructional conditions
“opportunity to learn”, time on task”, “homework” and “monitoring at classroom level”, were
carried out by Witziers and Bosker and published in Scheerens & Bosker, 1997, Ch. 6. The
number of studies that were used for these meta-analyses varied per variable, ranging form 14
to 38 studies.
The results in this summary of reviews and meta-analyses indicate that resource-input factors
on average have a negligible effect, school factors have a small effect, while instructional
have an average to large effect. The conclusion concerning resource -input factors should
probably be modified and “nuanced” somewhat, given the results of more recent studies
referred to in the above, e.g the results of the STAR-experiment concerning class-size
reduction.
There is an interesting difference between the relatively small effect size for the school level
variables reported in the meta-analysis and the degree of certainty and consensus on the
relevance of these factors in the more qualitative research reviews.
It should be noted that the three blocks of variables depend on types of studies using different
research methods. Education production function studies depend on statistics and
administrative data from schools or higher administrative units, such as districts or states.
School effectiveness studies focussing at school level factors are generally carried out as field
studies and surveys, whereas studies on instructional effectiveness are generally used on
experimental designs. The negligible to very small effects that were found in the re-analysis
of the IEA data-set could be partly attributed tot the somewhat “proxy” and superficial way in
which the variables in question were operationalized as questionnaire items.
JS.99.016 - 15-2-2016
21
Table 7: Review of the evidence from qualitative reviews, international studies and research
syntheses
Qualitative
reviews
Resource input variables:
Pupil-teacher ratio
Teacher training
Teacher experience
Teachers’ salaries
Expenditure per pupil
School organizational factors:
Productive climate culture
Achievement pressure for basic subjects
Educational leadership
Monitoring/evaluation
Cooperation/consensus
Parental involvement
Staff development
High expectations
Orderly climate
Instructional conditions:
Opportunity to learn
Time on task/homework
Monitoring at classroom level
Aspects of structured teaching:
-cooperative learning
-feedback
-reinforcement
Differentiation/adaptive instruction
PART II
+
+
+
+
+
+
+
+
+
+
+
+
International
analyses
Research
syntheses
-0.03
0.00
0.02
-0.03
0.04
-0.07
0.20
0.02
0.04
0.00
-0.02
0.08
0.14
0.05
0.15
0.03
0.13
0.20
0.04
0.11
0.15
0.00/-0.01 (n.s.)
-0.01 (n.s.)
0.09
0.19/0.06
0.11 (n.s.)
0.27
0.48
0.58
0.22
EVIDENCE FROM DEVELOPING COUNTRIES
In this part the evidence about effectiveness enhancing conditions of schooling in developing
countries will be reviewed. The review sets out by referring to earlier review articles,
particularly those by Hanushek (1995) and by Fuller and Clarke (1994), which in itself
incorporates results of reviews by Fuller (1987), Lockheed & Hanushek (1988), and
Lockheed & Verspoor (1991). Next a schematic description of 13 studies conducted after
1993 is provided. Conclusions are drawn about the state of the art of educational effectiveness
research in developing countries, in terms of predominance of the type of factors that are
studied, outcome comparison with results from industrialized countries, relevant research
innovations and implications for policy and practice applications.
JS.99.016 - 15-2-2016
22
Scope and limitations of the school effectiveness model, particularly when applied to
developing countries
Although the integrated model of school effectiveness is comprehensive in that it
encompasses input, process, output and context conditions and recognizes the multi-level
structure of educational systems it has a number of limitations.
1. The model has the level of the individual school as its focus, and leaves important issues
of a proper functioning of national education systems untreated; I shall refer to this as the
aggregation limitation. To the extent that subsidiarity*) is applied and schools are
autonomous this limitation is less severe, since, by definition, the school would have more
formal responsibilities.
2. The model has a strongly instrumental focus, treating educational goals and objectives as
largely “given”. Extending the model according to the larger perspective of organizational
effectiveness, as briefly referred to in part I, can partly compensate for this limitation, by
taking into account the responsiveness of the school vis à vis changing environmental
constraints. It is again dependent on the pattern of functional decentralization in an
educational system, to what extent adoption mechanisms at school level are important as
compared to the provision of such levels at the macro level. We shall refer to this
limitation as the instrumentality limitation.
3. Although the model is amenable to include questions of equity and efficiency, the actual
research practice has not lived up to expectations in this area. Moreover, the way school
effectiveness research is dealing with these issues is also determined by the other two
limitations concerning level of aggregation and instrumentality. The argument is that,
particularly in developing countries, these issues deserve to be dealt with from a broader
perspective than the school effectiveness model. This limitation will be referred to as the
relatively narrow quality orientation.
Before discussing these three limitations further and examining their seriousness for
developing countries, some clarification will be provided with respect to the concepts of
“functional decentralization” and “subsidiarity”. These concepts provide a basis to determine
the relative importance of the school as a decision-making level in education systems, and
moreover differentiate the answer to this question according to particular domains of
decision-making.
In the history of education in the Netherlands the term subsidiarity was used to refer to a
specific way in which denominational pressure groups in education linked to see the
relationship between the state and corporations representing interest groups in the educational
field. According to the subsidiarity principle the state should not interfere in matters that can
be dealt with by organized units of professionals. In the original case these organized units
*)
see discussion and explanation of this concept further on
JS.99.016 - 15-2-2016
23
were the denominationally based corporations or pressure groups of representatives in the
education field, their umbrella organizations in particular. “Subsidiarity” was the term
preferred by the Roman-Catholic denomination, while the Protestants spoke of “sovereignty
in one’s own circle”. Leune (1987, 379-380) points at the corporatistic nature of this kind of
concepts. According to the subsidiarity principle the state only acts subsidiary, that is, it only
interferes as a replacement, when needed. A simple example of subsidiarity is a drivinginstructor, who takes over the steering of a vehicle when the trainee makes a mistake, but in
all other cases quietly watches without interference. Within the context of the European
Commission the term subsidiarity is used to express the principle that what can be
accomplished by the member states should not be done by the central organs of the Union.
Of course it is debatable to what extent subsidiarity should be applied to schooling, in other
words which functions the schools could accomplish without interference from higher
administrative levels. The concept of functional decentralization helps in nuancing this
discussion by taking into account that a system can decentralize in some domains, but not in
others.
Although various classifications are available in the literature (cf. Van Amelsvoort &
Scheerens, 1997) the most commonly recognized educational domains are:
 the curriculum (including goals and standards)
 finance
 the conditions of labor and personnel policy
 school management
 teaching methods
 quality control
A well-known pattern of functional decentralization is a liberalization of finance (e.g. block
grants), management (cf. “school-based management”), and teaching methods, accompanied
by a centralized core curriculum. In actual practice it appears hard to relax central regulations
concerning the conditions of labor of educational personnel, under conditions of collective
bargaining by trade unions.
A further qualification with respect to the degree of decentralization is possible by
recognizing that sometimes government units are merely dispersed (“deconcentration”), that
decision-making authority is sometimes only partly shed (“delegation”) and in other cases is
completely given to local bodies (“devolution”) (cf. Bray, 1994).
re 1) aggregation limitation
At first sight it might look trivial to point out that the school effectiveness approach does not
address all that is relevant in developing and improving education and that there is a place for
macro level educational policy measures. The statement is made in this context for the
following reasons:
 to explicitly avoid the impression that the school effectiveness approach could be seen as
a paracee for all problems in education in developing countries;
JS.99.016 - 15-2-2016
24

to introduce the idea of optimal patterns of functional centralization and (functional)
school autonomy.
When considering macro-level stimulants and constraints, next to the school as a lever for
educational improvement radical decentralization and privatization seems to be left aside as a
policy option a priori. The relevant literature (e.g. James, 1991) points out that private
education is likely to be either regulated or funded (or both) by the state, so that mixtures of
state regulation and school autonomy are the most common situation, even in private and
semi-private government-dependent school systems. Indeed the very decision as to the degree
of privatization and decentralization is an example of a relevant state-controlled measure.
Other relevant issues that are likely to be settled at the state level, particularly in developing
countries, are:
 dealing with trade-offs between quantity and quality of schooling (enlarging enrolments
or investing in quality) and between quality and efficiency, for example, with respect to
discussions on school and class size;
 providing teacher training, or at least setting standards for teacher training;
 arranging for equitable distribution of scarce resources (e.g. by stimulating poor rural
areas on the basis of a progressive income tax measure);
 setting (core) educational objectives and standards to provide vertical coordination
between levels of schooling, e.g. between primary and secondary schools;
 stimulating accountability, by introducing evaluation and feedback mechanisms, which
can serve information requirements of consumers of education and administrators.
Although the empirical evidence is scarce, there appears to be some support for the
hypothesis that functional centralization on curriculum standards and assessment enhances
educational performance (e.g.Conley, 1997). Setting achievement standards and assessing
student achievement relate favorably to effectiveness enhancing conditions at the school
level. Having clear, accessible objectives can add to the overall purposefulness and
achievement orientation of the school. It can, likewise, be seen as a supportive condition for
“instructional leadership”, and, if information is properly fed back to stakeholders, as a basis
for organizational learning, accountability and improved “consumerism”.
A further hypothesis, regarding developing countries is that the lower the level of schooling
of parents and the poorer the catchment area of the school the more effective these measures
of functional centralization are likely to be.
re 2) instrumentality limitation
Another aspect of the school effectiveness model is the “goal immanent” orientation. A
function of “goal detection” or adaptation of goals according to changing societal and
contextual conditions is missing. When the school effectiveness model is broadened in scope,
by taking into account additional criteria such as responsiveness, participant satisfaction and
JS.99.016 - 15-2-2016
25
formal structure (cf. Faerman & Quinn, 1985) this situation is improved. In developing
countries material support from the local community appears to be particularly important, and
part of the schools’ effort would be needed to acquire this support.
Given its technical and instrumental orientation the school effectiveness model is not strongly
oriented towards incentives, and trade-offs between task-related and person-related interest.
This is one of the reasons to attempt to connect microeconomic theory and school
effectiveness modeling (cf. Scheerens & Van Praag, 1998).
Again, in developing countries “adaptability” and provisions of conditions that create
incentives for good performance also deserve to be dealt with at macro level.
re 3) relatively narrow quality orientation
The school effectiveness models is, at its core, an instrumental model of direct school outputs
(as compared to more long term, societal outcomes of schooling), in other words quality is
addressed as technical effectiveness. The origin of school effectiveness research lies in
improving education in poorer “inner city” districts in US cities, and, among studies, there is
definitively a bias towards less “privileged” educational contexts, and therefore the research
findings have a certain relevance to creating more equal educational provisions. Equity is
more directly addressed in studies on so-called “differential effectiveness”, where the
effectiveness of a school is differentiated according to sub-groups; i.e. boys/girls and children
with high and low SES backgrounds. These studies are scarce, and the results inconclusive,
however. The same applies to studies that have addressed cost-effectiveness. This state of
affairs underlines a previous conclusion that the school effectiveness model inadequately
addresses equity and efficiency of educational provisions at large and that, particularly in
developing countries, these issues should be addressed primarily at the level of macro level
educational policies.
Production function studies in developing countries
Hanushek (1995) provides the following tabulation of the effects of resources in 69 studies in
developing countries (see table 8).
JS.99.016 - 15-2-2016
26
Table 8: Summary of ninety-six studies on the estimated effects of resources on education in
developing countries, cited from Hanushek, 1995
Input
Teacher-pupil ratio
Teacher’s education
Teacher’s experience
Teacher’s salary
Expenditure per pupil
Facilities
Number of
studies
30
63
46
13
12
34
Statistically
significant
Positive Negative
8
35
16
4
6
22
8
2
2
2
0
3
Statistically
insignificant
14
26
28
7
6
9
When the number of positive significant associations are expressed in percentages the
comparison depicted in table 9 with the results shown in table 5 concerning studies in
industrialized countries can be made in a more straightforward way.
Table 9: Percentages of studies with positive significant associations of resource input
variables and achievement for industrialized as compared to developing countries
(sources: Hanushek, 1995, 1997)
Input
Teacher/pupil ratio
Teacher’s education
Teacher’s experience
Teacher’s salary
Per pupil expenditure
Industrialized countries
% sign. positive associations
15%
9%
29%
20%
27%
developing countries
% sign. positive associations
27%
55%
35%
30%
50%
The relevance of facilities in education in developing countries, not shown in the comparison,
amounts to no less than 70 when expressed as the percentage of significant positive studies.
The larger impact of these resource input factors in developing countries can be attributed to
larger variance in the independent as in the dependent variables. Both human and material
resources in education in industrialized countries are distributed in a relatively homogeneous
way among schools, in other words: schools do not differ that much on these variables.
Regarding the outcome variables (e.g. educational achievement) Riddell (1997) has shown
JS.99.016 - 15-2-2016
27
that schools in developing countries vary on average 40% (raw scores) and 30% (scores
adjusted for intake variables). This is a considerably larger variation than is usually found in
industrialized countries; where values of 10% to 15% between school variance on adjusted
outcomes are more common (cf. Bosker & Scheerens, 1999).
The positive outcomes of production function studies in developing countries make intuitive
sense (if basic resources and facilities are not present this will obviously be detrimental to the
educational endeavor as a whole). At the same time the outcomes give rise to interesting
interpretations when they are brought to bear on the theoretical principles of micro-economic
theory. Jimenez & Paquea (1996), for example, present findings that support the thesis that
local involvement in school finance stimulate both achievement orientation as economy in
spending. Pritchett and Filmer (1997) point at the political advantages of spending on human
resources (diminishing class size in particular) as compared to spending on instructional
materials, despite the much larger efficiency of the latter approach, while Picciotto (1996)
criticizes the narrow set of educational performance criteria that is used in most education
production function research and states that “program design must be informed by
assessments of overall educational performance against societal objectives; by evaluations of
the relevance of the objectives themselves and by judicious design of institutions to deliver
the needed services” (ibid, 5). Micro-economic theory has interesting conjectures with respect
to control mechanisms in education as well; where the argument is that bureaucratic control
measures are expensive and faulty and community involvement and “direct democracy”
would present a better alternative. Particularly when studies are becoming more theory-driven
and cost-benefit analyses are more frequently included, production function research is to be
considered as a viable approach to school effectiveness studies in developing countries.
Reviews of school effectiveness research in developing countries
The most recent review of school effectiveness studies in developing countries is the one by
Fuller and Clarke (1994), see table 10.
JS.99.016 - 15-2-2016
28
Table 10: School input and process variables that showed significant positive associations
with achievement in at least 50% of the studies in developing countries, analyzed
by Fuller and Clarke, 1994*)
Number of significant effects divided by
the number of analyses
Primary Schools
Secondary Schools
School/teacher factor
School spending
Expenditure per pupil
Total school expenditure
Specific school inputs
Average class size
School size
Availability of textbooks
Supplementary readers
Exercise books
Teaching guides
Desks
Instructional media
Quality of facilities
School library
Science laboratories
Child nutrition and feeding
Teacher attributes
Total years of schooling
Earlier measured achievement
Tertiary or teacher college
In-service teacher training
Teacher subject knowledge
Teacher gender (female)
Teacher experience
Teacher salary level
Teacher social class
Classroom pedagogy and organization
Instructional time
Frequent monitoring of pupil performance
Class preparation time
Frequency homework
Teacher efficacy
Cooperative learning task student
School Management
School cluster membership
Principal’s staff assessment
Principal’s training level
School inspection visits
Tracking or pupil segregation
*)
Source: Fuller & Clarke, 1994.
JS.99.016 - 15-2-2016
29
3/6
2/5
3/5
-
9/26
7/8
19/26
1/1
3/3
0/1
4/7
3/3
6/8
16/18
5/12
7/8
2/22
1/5
7/13
2/2
0/1
1/1
3/4
1/1
1/1
9/18
1/1
21/37
8/13
4/4
1/2
13/23
4/11
7/10
5/8
1/1
8/14
3/4
2/4
1/12
2/11
-
15/17
3/4
5/8
9/11
1/1
-
12/16
0/1
½
2/2
0/1
3/3
2/2
3/4
3/4
2/3
1/1
0/1
1/2
0/1
-
The review considered about 100 studies and drew upon earlier reviews by Fuller (1987),
Lockheed & Hanushek, 1988, Lockheed & Verspoor and an analysis of 43 studies in the
period 1988-1992 conducted by the authors themselves.
Only studies that controlled achievement for students’ family background were included;
and only significant associations at the 5% level were reported.
What table 10 indicates is, first of all, that there were more studies about primary schools than
about secondary schools. Also, financial, material and human resource input variables were
investigated more frequently than school and classroom process variables, with the exception
of instructional time.
This predominance of relatively easily assessable input characteristics is also evident from
table 11 where the number of times a particular variable was included in a total of 43 studies
is indicated.
Table 11: The number of times out of a total of 43 studies conducted between 1988 and
1992 (primary and secondary schools taken together) a particular type of school
input or process variable was investigated. Source: Fuller & Clarke, 1994
Enrolments/staff
School size
Class size
Teacher variables
Teacher training
Teacher salaries
Teacher experience
Teacher preparation
Teacher efficacy
Teacher gender
Inservice training
Instruction
Instructional time
Homework
Specific pedagogy
Testing of pupils
School organization
Public/Private
Tracking
Headmaster supervision
Equipment and facilities
Library facilities
General facilities and equipment
6
25
24
3
9
1
1
5
7
13
3
12
5
4
1
3
3
15
On the basis of their review of significant positive effects Fuller and Clarke (ibid) conclude
that rather consistent school effects can be found in three major areas: availability of
textbooks and supplementary reading material, teacher qualities (e.g. teachers’ own
JS.99.016 - 15-2-2016
30
knowledge of the subject and their verbal proficiencies) and instructional time and work
demands placed on students.
Policy relevant factors that showed inconsistent or lack of effects appeared to be class size
and teacher salaries.
The findings summarized in tables 10 and 11 once more underline the predominance of
production function type of effectiveness studies in developing countries. Riddell (1997), in a
more methodologically oriented review, observes that a “third wave” of school effectiveness
research in developing countries is “in danger of being lost without ever having been
explored”. By this third wave she refers to, what I have described as “integrated school
effectiveness studies”, comprising resource inputs, organizational factors and instructional
characteristics, in which multi-level modeling is a vital methodological requirement.
An innovative set of suggestions that Fuller & Clarke develop in their interpretation of the
research evidence, is to pay more attention to cultural contingencies when studying school
effectiveness in developing countries. Such contingencies might help in explaining why
school and classroom level variables “work” in one country but not in the next. They
distinguish four broad categories of cultural conditions:
a) the local level of family demand for schooling;
b) the school organization’s capacity to respond to family demand “while offering forms of
knowledge that are foreign to the community’s indigenous knowledge” (Fuller & Clarke,
1994, p. 136);
c) the teacher’s capacity and preference for mobilizing instructional tools;
d) the degree of consonance between the teacher’s pedagogical behavior and local norms
regarding adult authority, didactic instruction and social participation within the school
(ibid, p. 136).
These ideas, as well as the appeal to overcome other weaknesses of school effectiveness
studies (lack of cost benefit analyses, shortage of longitudinally designed studies) have
demanding implications for the design of studies. According to Riddell (1997) Fuller and
Clarke fail to present clear research alternatives.
A tabulation of studies after 1993
In order to update the presented reviews of school effectiveness studies in developing
countries, I summarized the results of 13 studies, conducted after 1993. The schematic
description is presented in table 12.
JS.99.016 - 15-2-2016
31
Table 12: School effectiveness studies in developing countries, since 1993
Project title (year of
publication, authors)
Explanation
Country
Input/Process indicators
(significant)
Label outcome variable
Education Achievements
and School Efficiency in
Rural Bangladesh, 1996,
Shahidar R. Khandker
Indirect (community-development) and
direct investment in schooling were
compared. It was concluded that direct
investment in schooling (teacher
qualifications) was relevant
Bangladesh
school attainment in
terms of survival rates
Primary school
achievement in English
and Mathematics in
Zimbabwe, 1993, L.M.
Nyagura & A. Riddell
Modeling between school variance in
mathematics was quite successful. Not all
between school variance in English could
be explained.
Zimbabwe
Student Achievement
and Schooling Choice in
Low-Income Countries.
Evidence from Ghana,
1994, P. Glewwe & H.
Jacoby
Study models selection bias school
effectiveness and follows up with costbenefit analysis.
Ghana
 mothers’ education
 teacher qualifications
 proportion of female teachers
 water and sanitation facilities
 electricity
mathematics
 teacher training
 instructional time
 pupil/teacher ratio
 textbook to pupils ratio
 supervised study
English
 teacher training
 textbook to pupil ratio
 pupil/teacher ratio
 conditions of classrooms
 presence of blackboard
(teacher schooling and teacher
experience were not statistically
significant)
JS.99.016 - 15-2-2016
32
Adjusted
outcome
yes/no
no
mathematics and English
achievement tests
yes
Math. and reading testscores (middle school
level)
yes
Project title (year of
publication, authors)
Explanation
Country
Input/Process indicators
(significant)
When girls learn more
than boys: The influence
of Time in school and
Pedagogy in Botswana,
1994, Fuller, Hua &
Snyder
Cultural conditions are referred to in
order to explain why instructional
practices, effective in Western cultures,
are ineffective in other parts of the world
Botswana
Improving Primary and
Secondary Education in
Madagascar, 1995,
Jaekel & Carceles
Elements of a strategy for improving the
quality of primary and secondary
education:
 focus on learning outcomes
 strengthen school leadership
 providing more learning materials
 fostering community support
 creating positive climate for reform
 shifting budget allocations towards
instructional purposes
 stimulate private education
(performance contracts)
 finding ways to get teachers to serve
in rural areas
 increase substitutes at lower income
groups, through more efficient and
greater mobilization of household
expenditure
Madagascar
Time at school. Urban schools do
better . Students perform better with
female teachers. Supplementary
reading material in-service training.
Little effect of teaching practices &
teacher attitudes. Negative effect of
frequency of asking open-ended
questions to girls.
management of the school director
JS.99.016 - 15-2-2016
33
teaching materials (teachers’ guides
and textbooks)
community support (especially for
facilities and equipment provision and
maintenance)
Note: qualitative (case-study) research
24 secondary schools
Label outcome variable
Adjusted
outcome
yes/no
yes/
achievement
tests in English
and math.
repetition rates
drop-out rates
academic results
(examination results)
No
Project title (year of
publication, authors)
Explanation
Country
Input/Process indicators
(significant)
Label outcome variable
An eclectic approach to
estimating the
determinants of
achievement in Jamaican
Primary Education,
1995, Glewwe, Grosh,
Jacoby & Lockheed
(primary schools)
As inputs may have reached the point of
disminitive returns in Jamaican primary
schools, pedagogical process variables
are relatively the most important in
determining achievement differences
Jamaica
California Achievement
Test (CAT) vertically
equated over all grade
levels
Do local contributions
affect the efficiency of
public primary schools?,
1996, Jimenez & Paquea
Micro-economic theory: local funding
goes together with increased local
demands on school achievement and a
more efficient use of scarce resources
(less investment in personnel as
compared to operations)
The
Philippines
significant for one or two subjects
(math, language)
 household per capita expenditure
 vision testing of students
 textbooks arriving in past 3 years
 availability of desks for all pupils
 doing written assignments (-)
 testing students
 time spent in whole-class
instruction (-)
 intensity of textbook use
 discussing curriculum &
pedagogic issues at staff meetings
 hours of instruction and assistance
by the principal
 frequency with which teachers
help each other
Local funding was significantly
positively related to (ran) achievement
(math. English, Philippino)
JS.99.016 - 15-2-2016
34
Adjusted
outcome
yes/no
yes
no (cost
function, rather
than
production
function was
estimated)
Project title (year of
publication, authors)
Explanation
Country
Input/Process indicators
(significant)
Label outcome variable
Is there a QuantityQuality trade-off as
enrolments increase?,
1997, Duraisaamy,
James, Lane & Tan
(primary & secondary)
As enrolments increase, resources need to
increase as well. If this does not occur
equipment levels become too low and
classes too big, with a predicted decline
of pass rates on examinations.
Positive effect of “aided” schools to
select their own teachers, to hire
substitute teachers and to fill vacancies
expeditiously
Study primarily directed to enrolments
and community background factors.
School quality defined in terms of pupilteacher rates and pupil –desk ratios.
Core findings: the higher the age of
enrolment, the lower the grade
attainment.
Tamil Nada,
India
pupil/teacher ratio (range: 36-47)
participation of teachers in nonteaching activities.
State of school buildings.
negative effect of increased
pupil/teacher ratio’s
positive effect of private schooling
examination results
Honduras




In a decentralized setting parents will be
better able to monitor teacher behavior
El Salvador
Primary Education
Efficiency in Honduras.
What remains to be
done, 1997, J.H.Y.
Edwards, B. Fuller & S.
Parandenker
EDUCO, 1998, Jimenez
& Sawada
school provides free lunches
(grade repetition)
(not significant:
 proportion of female teachers
 student/teacher ratio
 distance from school
 length of school day
 school has textbooks)
Student/teacher ratio has a negative
impact on the conditional probabilities
of leaving school.
 decentralization school autonomy
 availability of infrastructure
services
 multigrade classrooms
grade attainment
grade repetition
survival level
number of school days
missed by the student
due to teacher absences
math & language
achievement
JS.99.016 - 15-2-2016
35
Adjusted
outcome
yes/no
yes / success
rates adjusted
for growth of
examination
candidates
no
Yes
Project title (year of
publication, authors)
Explanation
Country
Input/Process indicators
(significant)
Label outcome variable
Expansion of Private
Secondary Education in
Tanzania, 1998,
Lassibille, Tan & Sumra
Evaluation of School
Improvement through an
Educational
Effectiveness Model: the
Case of PEQIP
Indonesia, 1999, G. van
der Werf, B.P.M.
Creemers, R. de Jong &
E. Klaver
no incentives for better performing
schools
Tanzania
expansion private & public schools
no effect on value added
achievement
The study found a negative effect of
teacher experience and a small effect of
textbook availability; both could be
explained from specific conditions in the
Indonesian context.
Indonesia
Bahasa Indonesia,
Mathematics, Science,
grade 6
School inputs in
secondary education and
their effects on
Academic Achievement:
A study in Colombia,
1999, Piñeros &
Rodriguez
Better school resources lead to better
achievement, after appropriate control for
socio-economic background variables
Colombia
innovative teaching
(grouping, a variety of teaching
methods, asking comprehensive
questions, keeping pupils actively
involved, stimulating interaction)
educational leadership
(classroom observations by principal,
evaluating the quality of teachers,
evaluating the quality of the school)
parents’ engagement
 range of socio-economic
background variables
 range of school infrastructural
variables (e.g. the existence of a
library, most of them significant;
surprisingly supply of textbooks
was insignificant
 time of contact of the student with
the school (positive)
 time to comment to school
(negative)
JS.99.016 - 15-2-2016
36
Adjusted
outcome
yes/no
yes
yes
yes,
achievement in
math and
(Spanish)
language
The studies summarized in table 12 confirm the picture that is drawn on the basis of earlier
reviews. The majority of studies has concentrated on material and human resource input
factors. A minority of four studies include school organizational factors that are other than
structural (school culture, school management characteristics) and pedagogical or
instructional variables (Glewwe et al., 1995; Nyagura & Riddell, 1993; Fuller et al., 1994;
Van der Werf et al., 1999).
Only these last three studies and the study by Piñeros & Rodriguez (1999) have used multilevel modeling and analysis.
As far as the theoretical and conceptual background of the study is concerned (see the
entrances in the second column of table 12), micro-economic theory is referred to in a few
cases (Lassibille et al., 1998; Jimenez & Paquea, 1996). In most cases there is no explicit
theoretical framework, although interesting interpretations of the results, contextualized in the
particular national context are sometimes given (e.g. Fuller et al., 1994; Van der Werf et al.,
1999). The studies by Glewwe et al., Fuller et al. and Van der Werf et al. are most explicit in
referring to the knowledge base on school effectiveness developed from empirical studies in
industrialized countries.
A striking point when reviewing these studies and examining the earlier reviews is that
instructional and pedagogical theory is practically missing as a source of inspiration for
educational effectiveness studies in developing countries. Only the study by Van der Werf et
al. (1999) is explicit in referring to instructional theory.
One last point concerning methodology is that sophisticated designs that are more frequently
used in quantitative effectiveness studies in OECD-countries, like the use of structural
equation modeling and multi-level analyses are sparse among the studies in developing
countries. On the other hand econometric models used to adjust for selection bias as in the
study by Glewwe et al. (1995) and the rendering of the outcomes of effectiveness studies in
terms of “elasticities” (the increments in school input factors needed to attain a specified
increment on the outcome variable) are quite innovative with respect to the state of the art
methodology used in school effectiveness research in OECD-countries.
The research outcomes of the studies summarized in table 12 confirm the importance of
equipment (textbooks) and the human resource factor (teacher training) for schooling in
developing countries. From the limited set of studies that have looked more inside the blackbox of schooling no convincing pattern of significant impact of the school organizational and
instructional variables emerges. In the next section a more detailed discussion of this sub-set
of studies is given.
A closer look at school organizational and instructional factors
Given the fact that school organizational variables and instructional variables have been
studied in a small minority of effectiveness studies in developing countries and given the
JS.99.016 - 15-2-2016
37
expectations one might have considering their impact on the basis of research results obtained
in OECD countries (see Part I) a closer look at studies that have investigated these factors is
warranted. Several authors, reflecting on the direction educational effectiveness studies
should take in developing countries, Lockheed & Longford, 1989; Riddell, 1997, also refer to
these expectations.
Among the studies since 1993, summarized in table 12, only those by Glewwe et al. (1995),
Fuller et al. (1994), Nyagura & Riddell (1993) and Van der Werf et al. (1999) studied more
cultural and managerial school organizational variables and variables reflecting teaching
practices.
Apart from more descriptive characteristics like qualifications of head teachers, Nyagura &
Riddell (ibid) studied the following more “substantive” school organizational variables in
Zimbabwen secondary schools: teacher stability, time devoted to school-based in-service
activities and professional support to teachers through supervision by the head teacher. Data
were collected by means of survey methods, yielding information at school, classroom and
student level. In a preliminary regression analysis all individual variables were related to
English and math achievement. Among the more substantive, theoretically interesting class
and school level variables the amount of instructional time devoted to mathematics (class
level) and the amount of supervision of teachers had a significant association with
achievement. Contrary to expectations the association of the latter variable with achievement
was negative, however. Further multi-level modeling pointed out that just one other classlevel variable, the amount of supervised study time afforded by the head teacher had a
positive and significant effect in mathematics (not in English). Both at school and classroom
level textbook availability and teacher training stood out as the most important factors for
either subject.
Glewwe et al. (1995) examined, apart from more physical input variables, pedagogical inputs
(curriculum, instructional time and teacher quality), pedagogical processes (teaching practices
in the classroom) and school organization, climate and control (school autonomy, workcentered environment, community involvement, orderly environment and school type) in their
study of primary schools in Jamaica.
Data on school and classroom level variables were collected by means of a school
administrator’s and a teacher questionnaire. Effect variables were measured by means of the
California Achievement Test (CAT) which measures mathematics computation and reading
comprehension. This test yielded comparable scores across all grade levels.
Data were analyzed by means of econometric methods. The results pointed out that physical
and pedagogical inputs only “played a marginal role in explaining cross-sectional differences
in cognitive skills in Jamaica” (ibid, 249).
In the domain of the pedagogical process variables “doing written assignments in class” had a
strong negative effect on achievement. Testing students had a weakly significant (10% level)
effect and time spent in whole-class instruction a weakly significant negative effect on math
JS.99.016 - 15-2-2016
38
achievement. In reading intensity of textbook use and the percentage of teacher time spent
testing students had positive effects, and time spent doing written assignments had a negative
effect.
Among the school organization, climate and control variables there were no significant
effects that reached the 5% level. Significant at the 10% level were: discussing curriculum
and staffing issues at staff meetings (positive for mathematics); hours of instructional
assistance by the principal and average frequency with which teachers help each other
(positive for reading).
The authors conclude that overall, variables measuring pedagogical processes are more often
significantly related to student achievement than are physical and pedagogical input variables
and school organizational variables (ibid, 250). They refer to the high levels that important
inputs in Jamaica are at to explain the relatively small impact of such variables.
In the study of Indonesia primary schools Van der Werf et al. (1999) used observation and
interview methods to study school organizational and teaching variables. Data were analyzed
by means of multi-level modeling (VARCL-program). Student achievement in mathematics,
Bahasa Indonesia and Science was measured in grade 6, by means of standardized tests. Out
of 27 school and classroom level variables, all selected on the basis of the school and
instructional effectiveness literature, 4 variables had a significant (5% level) association with
achievement in the expected (positive) direction for mathematics (time spent on subject,
frequent questioning by the teacher, evaluation of teachers and help with homework); for
Bahasa Indonesia there were 3 such variables (innovative teaching, observations in
classrooms and voluntary work of teachers); with respect to science 3 variables were
significant and positive (innovative teaching, evaluation of school quality and availability of
student books), however, 4 other variables had a significant effect with the “wrong” sign
(time spent to subject, presentation of content, pupils working and evaluation of teachers.
In two cases variables were significantly positive for two out of three subjects (innovative
teaching and evaluation of teachers).
In the study of Botswana’s junior secondary schools Fuller et al. (1994) observation methods
and survey methods were used to study four blocks of school organizational and teaching
variables for their effect on language and math. achievement:
A. material conditions and school inputs
B. teacher background, gender and training levels
C. teaching practices and classroom rules
D. teacher effort and pedagogical beliefs
In block C the following pedagogical behaviors were observed: the complexity of
instructional tools utilized by teachers, task demands placed on pupils by the teachers,
especially the frequency of active reading and writing exercises, the frequency and
JS.99.016 - 15-2-2016
39
complexity of questions asked of pupils, the consistency of the teachers “pedagogical
technology” and the use of instructional time.
In block D (teacher effort and pedagogical beliefs) pedagogical philosophies, teacher selfperceptions of competence, job satisfaction and level of efficacy were addressed.
Altogether the set of variables in blocks C and D appeared to have “little explanatory power”
(ibid, 368). The only variable in these domains that had a significant effect was teachers’
average use of open-ended questions, but contrary to expectation, the sign of the association
was negative.
How should these results be interpreted? First of all, four studies is obviously too small a
number to draw firm conclusions. Nevertheless a reason to attach some importance to these
studies is the fact that they used in-depth and partly observational methods to study school
and classroom processes and used appropriate state of the art methodology (econometric and
multi-level modeling, use of achievement tests that allowed for vertical equating in the case
of the study by Glewwe et al.).
The results are somewhat disconcerting with respect to the assumption that school
organizational and particularly classroom level instructional variables will account for a
sizeable part of the variance in achievement outcomes, as could be expected (for the class
level conditions) on the basis of research in industrialized countries. Apparently an earlier
review, by Anderson, Ryan and Shapiro (1989), reached a similar conclusion stating that
“variations in teaching practice in developing countries are only rarely found to be associated
with variations in student learning” (Anderson et al., 1989, cited by Glewwe et al., 1995).
The 1994 review by Fuller and Clarke, however, indicate significant effects for a large
proportion of the limited occasions when classroom pedagogy and school management
variables were included (see table 10).
Two lines of reasoning could be considered to explain a relatively low impact of
organizational and instructional variables.
 The first is the reference to cultural contingencies put forward by Fuller and Clarke,
already discussed in a previous section.
 The second calls upon bearing in mind the comparative nature of the school effectiveness
studies that were reviewed. It could well be that, in some developing countries, the range
of variation in teaching practices is quite limited.
The implications for future research of the first would be the exploration and measurement of
the most relevant cultural contingencies, preferably in internationally comparative studies.
With respect to the second supposition, a more focused study of the between school variances
on relevant process indicators should be considered. Such an approach is in line with the
identification of process indicators as carried out by Heneveld & Craig (1994).
JS.99.016 - 15-2-2016
40
Discussion
Educational effectiveness studies in OECD-countries are carried out in three main strands:
studies in the education production function traditions, studies of effective schools and
studies about instructional effectiveness. During the last decade these three approaches are
being integrated in more comprehensive multi-level studies. Broadly summarized, the results
indicate a relatively minor impact of resource input variables and school organizational
conditions and a larger impact of instructional characteristics at classroom level.
In school effectiveness studies in developing countries there is a strong predominance of
studies carried out in the education production function tradition. When the results of the
small minority of studies that include school organizational and instructional conditions is
taken into account the overall picture of results is that facilities and the quality of human
resource inputs have a larger impact than variation in instruction and pedagogy.
The implication of this state of affairs for the future of effectiveness research in developing
countries should not be that – given the relatively small impact demonstrated so far – school
organizational and instructional conditions deserve less attention. On the contrary, as others
have pointed out (Fuller & Clarke, 1994), as material and quality of human resources
conditions are improving in developing countries it is to be expected that managerial and
instructional conditions will start to make more of a difference. One could argue that in early
stages of development primacy ought to be given to putting in place basic provisions in terms
of buildings, equipment, textbooks and qualified teachers. After this would have been
realized conditions of education would be more comparable to conditions in OECDcountries, with a greater likelihood of effectiveness studies beginning to show the impact of
instructional factors.
The strength of the school effectiveness knowledge base does not just depend on the
accumulation of evidence on the strengths of associations between antecedent conditions of
schooling and outcomes. The existence of an integrated conceptual framework and theoryembedded principles that could explain “why” certain factors work, is just as important. In
this sense the study of educational effectiveness in OECD-countries is still very weak.
Although there is some conceptual multi-level modeling, where the interactions between
school environmental, school managerial and instructional conditions are substantively
interpreted and analytically modeled (Scheerens & Bosker, 1997) use of theories is largely
eclected. At the level of classroom instruction there is a latent controversy between
behavioristic principles that seem to be better able to account for the outcomes of empirical
studies so far and currently popular constructivist approaches.
The influence of economics and micro-economic theory on effectiveness studies in
developing countries provides a more unilateral framework for these studies as compared to
the eclectic use of theory in OECD-countries, where micro-economic theory is just one of the
theories that is referred to.
JS.99.016 - 15-2-2016
41
A perspective that is interesting and innovative – also for studies in OECD-countries – is the
use of cultural contingencies in interpreting results of effectiveness studies (Fuller & Clarke,
1994).
Concerning methodology a blending of experiences and approaches from studies in the two
“worlds” would be desirable too. As emphasized by Riddell (1997), multi-level modeling
should be applied more frequently in effectiveness studies in developing countries. (A
condition that would be enforced if conditions at school and classroom level would be taken
into account in study-designs).
Econometric modeling of selection-bias, seen more frequently in studies in developing
countries, has a lot of potential for studies elsewhere too, as do cost-benefit analyses, and
simulations. Another valuable idea from econometrics is the rendering of effects in terms of
elasticities.
As far as research “logistics” is concerned, a lot is to be said for exploiting two kinds of
synergy:
 integrating effectiveness studies with evaluations of development projects (the study by
Van der Werf et al. (1999) that was referred to earlier, is an example);
 integrating project evaluation with the development and implementation of monitoring
systems at national level, possibly exploring the “spin-off” of such monitoring instruments
for school self-evaluation purposes as well*).
When such integrations can be realized it is more likely that core data on inputs, processes
and outcomes of schooling will be available on a longitudinal basis, and this would be a
condition that would greatly enhance the validity and applicability of effectiveness studies.
Finally the value and strength of the school effectiveness knowledge base for practical
application should be looked at. Two types of application can be considered: planning on the
one hand and monitoring and evaluation on the other. The first is a stronger and more
pretentious type of application than the latter.
When the input and process conditions that have received empirical support in educational
effectiveness research are merely used as a basis for identifying variables to be included in
indicator, monitoring and school self-evaluation instruments there is only an indirect link
with prescribing policy and practice. At the same time there would be ample room for local
and within-culture interpretation.
Given the weaknesses and “blank spots” in the empirical knowledge base and the need to
contextualize results, the evaluative use of educational effectiveness knowledge seems to be
the most appropriate one.
*)
In a forthcoming paper by the author, together with J.P. Tan and R. Shah these ideas will be worked out in
more detail, as in forthcoming reports on projects in Belize and the Dominican Republic where these approaches
are being implemented.
JS.99.016 - 15-2-2016
42
Nevertheless there is robustness in the general principles that emerge from the research-based
knowledge of more than three decades of educational effectiveness research in industrialized
and developing countries. From these principles the following tentative suggestions for
educational projects in developing countries could be derived:
 describe the general conditions of education on the basis of a core set of indicators,
including poverty conditions per region, participative rates and the availability of basic
resources;
 at early stages of development emphasize conditions that stimulate intended participation
levels and basic resources and facilities (e.g. buildings, classroom);
 invest in substantive educational programs containing four well-integrated parts: a national
examination or assessment program, national curriculum priorities in core subjects, teacher
training (focused at subject matter mastery and instructional principles) and a national
monitoring system;
 decentralize school management, create conditions of local participation and control over
financial conditions and teachers’ conditions of labor;
 use different media (distance education, training courses, model-curricula, school selfevaluation) to enhance classroom management, effective learning time and structured
teaching (with diagnosis, feedback and immediate remediation at its core) to stimulate
active learning; adapt these general instructional conditions to aspects of the local culture.
For various reasons (larger between school variances in outcomes, practical relevance,
blending of methodological principles, synergy with program evaluation and monitoring) it is
to be expected that the “cutting edge” of educational effectiveness research in the near future
will be situated in developing countries, more than in the industrialized part of the world.
References
Achilles, C.M. (1996). Students achieve more in smaller classes. Educational Leadership,
76-77.
Aitkin, M., & N. Longford (1986). Statistical modelling issues in school effectiveness studies.
Journal of the Royal Statistical Association, Series A (General), 149, Part 1, 1-43.
Alexander, K.L., & Eckland B.K. (1980). The "explorations in quality of opportunity" sample
of 1955 high school sophomores. In A.C. Kerkhoff (Ed.), Research in sociology of
education and socialization. Vol. I: Longitudinal perspectives on educational
attainment. Greenwich: JAI Press Inc.
Amelsvoort, H.W.C.H. van, & Scheerens, J. (1997). Policy issues surrounding processes of
centralization and decentralization in European education systems. Educational Research
and Evaluation, 3(4), 340-363.
Anderson, C.S. (1982). The search for school climate: A review of the research. Review of
Educational Research, 52(3), 368-420.
Anderson, L.W., Ryan, D.W., & Shapiro, B.J. (1989). The IEA Classroom Environment
Study. Oxford: Pergamon Press.
JS.99.016 - 15-2-2016
43
Averch, H.A., Carroll, S.J., Donaldson, T.S., Kiesling, H.J., & J. Pincus (1974). How
effective is schooling? A critical review of research. Englewood Cliffs, NJ: Educational
Technology Publications.
Bangert, R.L., Kulik, J.A., & Kulik, C.C. (1983). Individualized systems of instruction in
secondary schools. Review of Educational Research, 53, 143-158.
Bereiter, C., & Kurland, M. (1982). A constructive look at follow through results.
Interchange, 12, 1-22.
Bloom, B. (1976). Human characteristics and school learning. New York: McGraw Hill.
Bosker, R.J. (1990). Extra kansen dankzij de school? Nijmegen: ITS/OoMO.
Bosker, R.J. (1995). De stabiliteit van Mattheus-effecten. Paper presented at the annual
conference of the Dutch Association for Educational Research (Onderwijsresearchdagen), Groningen, June.
Bosker, R.J., & Scheerens, J. (1999). Openbare prestatiegegeven van scholen; nuttigheid en
validiteit. Pedagogische Studiën, 76(1), 61-73.
Brandsma, H.P. (1993). Basisschoolkenmerken en de kwaliteit van het onderwijs
(Characteristics of primary schools and the quality of education). Groningen: RION.
Bray, M. (1994). Centralization/decentralization and privatization/publicization: conceptual
issues and the need for more research. In W.K. Cummings & A. Riddell (Eds.),
Alternative policies for the finance, control, and delivery of basic education. Special
issue of the International Journal of Educational Research, 21(8), 817-824.
Brookover, W.B., Beady, C., Flood, P. et al. (1979).School social systems and student
achievement - schools can make a difference. New York: Praeger Publishers.
Brophy, J. (1996). Classroom management as socializing students into clearly articulated
roles. Paper presented at AERA Annual Meeting, New York.
Brophy, J., & Good, Th. L. (1986). Teacher behavior and student achievement. In M.C.
Wittrock (Ed.), Handbook of research on :teaching [pp. 328-375]. New York:
Macmillan.
Cameron, K.S., Whetten, D.A. (Eds.) (1983). Organizational Effectiveness. A comparison of
multiple models. New York: Academic Press.
Card, D. & Krueger, A.B. (1992). Does school quality matter? Returns to education and the
characteristics of public schools in the United States. Journal of Political Economy,
100, 1-40.
Caroll, J.B. (1963). A model of school learning. Teachers College Record, 64, 722-733.
Cheng, Y.C. (1996). The pursuit of school effectiveness. Hong Kong: The Chinese University.
Cohen, M. (1982). Effective schools: Accumulating research findings. American Education.
January-February, 13-16.
Cohen, D.K. (1988). Teaching Practice ... Plus ça Change ... In Ph. Jackson (Ed.),
Contributing to Educational Change: Perspectives on Research and Practice. Berkeley,
CA: McCutchan.
Coleman, J.S. et al. (1966). Equality of educational opportunity. Washington D.C.: U.S.
Government Printing Office.
Collins, A., Brown, J.S., & Newman, S.E. (1988). Cognitive apprenticeship: teaching the
craft of reading, writing and mathematics. In L.B. Resnick (Ed.), Cognition and
Instruction: Issues and Agendas. Hillsdale, NJ: Lawrence Erlbaum Associates.
Conley, D.T. (1997). Road Map to Restructuring. Charting the course of change in American
Education. University of Oregon: ERIC Clearing House on Educational Management.
Cotton, K. (1995). Effective schooling practices: A research synthesis. 1995 Update. School
Improvement Research Series. Northwest Regional Educational Laboratory.
Creemers, B.P.M. (1994). The effective classroom. London: Cassell.
JS.99.016 - 15-2-2016
44
Davies, J.K. (1972). Style and effectiveness in education and training: a model for organizing,
teaching and learning, Instructional Science, 2.
Dougherty, K. (1981). After the fall: Research on school effects since the Coleman report.
Harvard Educational Review, 51, 301-308.
Doyle, W. (1985). Effective secondary classroom practices. In M.J. Kyle (Ed.), Reaching for
excellence. An effective schools sourcebook. Washington DC: US Government Printing
Office.
Duffy, Th.M., & Jonassen, D.H. (1992). Constructivism and the technology of instruction: a
conversation. Hillsdale, NJ: Lawrence Erlbaum Associates.
Duraisaamy, James, Lane, & Tan, J.P. (1997). Quality trade-offs as enrolments increase.
Working Papers Series on Impact Evaluation of Education Reforms. The World Bank.
Edmonds, R.R. (1979). A discussion of the literature and issues related to effective schooling.
Cambridge: Center for Urban Studies. Harvard Graduate School of Education.
Edwards, J.H.Y., Fuller, B., & Parandenker, S. (1997). Primary Education Efficiency in
Honduras. What remains to be done? Human and Social Development Group, The
World Bank.
Faerman, S.R., & Quinn, R.E. (1985). Effectiveness: the perspective from organization
theory. Review of Higher Education, 9, 83-100.
Ferguson, R.F. (1991). Paying for Public Education: New Evidence on How and Why Money
Matters. Harvard Journal on Legislation, 28 (465-498).
Fraser, B.J., Walberg, H.J., Welch, W.W., Hattie, J.A. (1987). Syntheses of educational
productivity research. Special Issue of the International Journal of
EducationalResearch, 11(2).
Fuller, B. (1987). What factors raise achievement in the Third World? Review of Educational
Research, 57, 255-292.
Fuller, B. & Clarke, P. (1994). Raising school effects while ignoring culture? Local
conditions and the influence of classroom tools, rules and pedagogy. Review of
Educational Research, 64 (119-157).
Fuller, B., Hua, H., Snijder, C.W. (1994). Focus on Gender and Academic Achievement.
When Girls learn more than Boys: The Influence of Time in School and Pedagogy in
Botswana. Comparative Education Review, 38, 347-376.
Gage, N. (1965). Desirable behaviors of teachers. Urban Education, 1, 85-95.
Galton, M. (1999). Class Size and Pupil Achievement. International Journal of Educational
Research, vol. 29.
Glasman, N.S., & Biniaminov, J. (1981). Input-output analysis of schools. Review of
Educational Research, 51, 509-539.
Glewwe, P., Grash, M., Jacoby, H., Lockheed, M. (1995). An Eclectic Approach to
Estimating the Determinants of Achievement in Jamaican Primary Education. The Work
Bank Economic Review, 9, 231-258.
Glewwe, P., & Jacoby, H. (1994). Student Achievement and Schooling Choice in LowIncome Countries. The Journal of Human Resources, xxix-3, 843-864.
Good, Th.L., &Brophy, J. (1986). School effects. In M.C. Wittrock (ed.), Handbook of
research on teaching (p. 328-375). New York: McMillan Inc.
Grisay, A. (1996). Evolution des acquis cognitifs et socio-affectifs des eleves au cours des
annees de college. Liège: Université de Liège.
Hanushek, E.A. (1979). Conceptual and empirical issues in the estimation of educational
production functions. Journal of Human Resources, 14, 351-388.
Hanushek, E.A. (1986). The economics of schooling: production and efficiency in public
schools. Journal of Economic Literature, 24, 1141-1177.
JS.99.016 - 15-2-2016
45
Hanushek, E.A. (1995). Interpreting Recent Research on Schooling in Developing Countries.
The World Bank Research Observer, 10(227-46).
Hanushek, E.A. (1997). Assessing the Effects of School Resources on Student Performance:
An update. Educational evaluation and Policy Analysis, 19(141-164).
Hauser, R.M., Sewell, W.H., & Alwin, D.F. (1976). High School effects on achievement. In
W.H. Sewell, R.M. Hauser & D.L. Featherman (Eds.), Schooling and achievement in
American Society. New York: Academic Press.
Haywood, H.C. (1982). Compensatory education. Peabody Journal of Education, 59, 272301.
Hedges, L.V., Laine, R.D., & Greenwald, R. (1994). Does money matter? A meta-analysis of
studies of the effects of differential school inputs on student outcomes. Educational
Researcher, 23(3), 5-14.
Heneveld, W., & Craig, H. (1996). Schools Count. World Bank Project Designs and the
Quality of Primary Education in Sub-Saharan Africa. Washington D.C.: World Bank.
Hill, P.W., Rowe, K.J., & Holmes-Smith, P. (1995). Factors affecting students' educational
progress: multilevel modelling of educational effectiveness. Paper presented at the
International Congress for School Effectiveness and School Improvement, Leeuwarden,
the Netherlands, January 1995.
Jaekel & Carceles (1995). Improving Primary and Secondary Education in Madagascar. The
World Bank.
James, E. (1991). Public Policies toward Private Education: an International Comparison. In
E. Jimenez & M.E. Lockheed (Eds.), Private versus Public Education: an International
Perspective. International Journal of Educational Research, 15(5), 359-376.
Jencks, C. et al. (1972). Inequality. New York: Basic Books.
Jencks, C. et al. (1979). Who gets ahead? The determinants of economic success in America.
New York: Basic Books.
Jimenez, E. & Paquea, V. (1996). Do Local Contributions Affect the Efficiency of Public
Schools? Economics of Education Review, 15, 377-386.
Jimenez, E. & Sawada (1998). El Salvador’s EDUCO-Program. Working Papers Series on
Impact Evaluation of Education Reform.
Jong, T.A.B. de, & Joolingen, W.R. van (1998). Scientific discovery learning with computer
simulations of conceptual domains. Review of Educational Research, 68, 179-202.
Khandker, G.R. (1996). Education Achievements and School Efficiency in Rural Bangladesh,
World Bank Discussion Paper, 319.
Kyle, M.J. (Ed.) (1985). Reaching for excellence. An effective schools sourcebook.
Washington D.C.: U.S. Government Printing Office.
Lassibille, G., Tan, J.P., & Sumra, S. (1998). Expansion of Private Secondary Education:
Experience and Prospects in Tanzania. Working Papers Series on Impact Evaluation of
Education Reforms. The World Bank.
Leune, J.M.G. (1987). Besluitvorming en machtsverhoudingen in het Nederlandse
onderwijsbestel. In J.A. van Kemenade et al. (Red.), Onderwijs, bestel en beleid deel 2.
Groningen: Wolters-Noordhoff.
Levine, D.K., & Lezotte, L.W. (1990). Unusually Effective Schools: A Review and Analysis of
Research and Practice. Madison, Wise: Nat. Centre for Effective Schools Research and
Development.
Lockheed, M. & Hanushek, E. (1988). Improving educational efficiency in developing
countries: What do we know? Compare, 18(1), 21-38.
Lockheed, M.E., & Longford, N.T. (1989). A Multi-Level Model of School Effectiveness in a
Developing Country. Discussion paper no. 69. Washington D.C.: World Bank.
JS.99.016 - 15-2-2016
46
Lockheed, M. & Verspoor, A. (1991). Improving Primary Education in Developing
Countries. London: Oxford University Press.
Lortie, D.C. (1973). Observations on teaching as work. In R.M.W. Travers (Ed.), Second
Handbook of Research on Teaching. Chicago: Rand McNally.
Luyten, J.W. (1994). School effects: stability and malleability. Enschede: Universiteit
Twente, dissertatie.
Medley, D., & Mitzel, H. (1963). Measuring classroom behavior by systematic observation.
In N.L. Gage (Ed.), Handbook of research on teaching. Chicago: Rand McNally.
Mortimore, P., Sammons, P., Stoll, L., Lewis, D., & Ecob, R. (1988). School matters: the
junior years. Somerset: Open Books.
Mosteller, F., Moynikan, D.D. (Eds.) (1972). On equality of educational opportunity. New
York: Random House.
Murmane, R.J. (1981). Interpreting the evidence on school effectiveness. Teachers College
Record, 83, 19-35.
Neufeld, E., Farrar, E., & Miles, M.B. (1983). A review of effective schools research: The
message for secondary schools. Washington D.C
Nyagura, L.M., & Riddell, A. (1993). Primary School Achievement in English and
Mathematics in Zimbabwe. A multilevel analysis. The World Bank, Policy Research
Working Paper.
Picciotto, R. (1996). Wat is Education Worth? From Production Function to Institutional
Capital. World Bank: Human Capital Development Working Paper, no. 75.
Piñeros, L., & Rodrígues, A. (1999). School Inputs in Secondary Education and their Effects
on Academic Achievement: A Study in Colombia. The World Bank, Latin America and
the Carribean Regional Office. LCSHD Paper Series.
Pritchett, L. & Filmer, D. (1997). What Educational Production Functions Really Show. A
positive theory of education. The World Bank. Policy Research Working Paper of the
Development Research Group Poverty and Human Resources.
Purkey, S.C., & Smith, M.S. (1983). Effective schools: a review. The Elementary School
Journal, 83(4), 427-452.
Ralph, J.H., & Fennessey, J. (1983). Science or reform: some questions about the effective
schools model. Phi Delta Kappan, 64(10), 689-695.
Resnick, L.B. (1987). Education and learning to think. Washington D.C.: National Academic
Press.
Reynolds, D., Hopkins, D., & Stoll, L. (1993). Linking school effectiveness knowledge and
school improvement practice: towards a synergy. School Effectiveness and School
Improvement, 4(1), 37-58.
Rosenshine, B., & Furst, N. (1973). The use of direct observation to study teaching. In R.M.
Travers (Ed.), Second Handbook of Research on Teaching. Chicago: Rand McNally.
Rubin Riddell, A. (1997). Assessing Designs for School Effectiveness Research and School
Improvement in Developing Countries, Comparative Education Review, 41(2).
Rutter, M. (1983). School effects on pupil progress: research findings and policy
implications. Child Development, 54 (1), 1-29.
Sammons, P., Hillman, J., & P. Mortimore (1995). Key characteristics of effective schools: a
review of school effectiveness research. London: OFSTED.
Scheerens, J. (1987). Enhancing educational opportunities for disadvantaged learners.
Amsterdam/Oxford/New York: North-Holland Publishing Company.
Scheerens, J. (1990). School effectiveness research and the development of process indicators
of school functioning. School Effectiveness and School Improvement,1(1), 61-80.
Lissse: Swets & Zeitlinger.
JS.99.016 - 15-2-2016
47
Scheerens, J. (1992). Effective Schooling, Research, Theory and Practice. London: Cassell.
Scheerens, J. (1994). The school level context of instructional effectiveness. A comparison
between school effectiveness and restructuring models. Tijdschrift voor
Onderwijsresearch, 19(1), 26-38.
Scheerens, J., & Bosker, R.J. (1997). The Foundations of Educational Effectiveness. Oxford:
Elsevier Science Ltd.
Scheerens, J., & Creemers, B.P.M. (1989). Towards a more comprehensive conceptualization
of school effectiveness. In B.P.M. Creemers, T. Peters & B. Reynols (Eds.), School
Effectiveness and School Improvement [pp. 265-278]. Lisse: Swets & Zeitlinger.
Scheerens, J., & Praag, B.M.S. van (Eds.) (1998). Micro-economic Theory and Educational
Effectiveness. Enschede/Amsterdam: Universiteit Twente, OCTO/Universiteit van
Amsterdam, SEO.
Slavin, R.E. (1996). Success for all. Lisse: Swets & Zeitlinger.
Stallings, J. (1985). Effective elementary classroom practices. In M.J. Kyle (Ed.), Reaching
for excellence. An effective schools sourcebook. Washington, DC: US Government
Printing Office.
Stebbins, L.B., St. Pierre, R.G., Proper, E.C., Anderson, R.R., & Cerva, T.R. (1977).
Education as experimentation: a planned variation model, Vol. IV-A. An evaluation of
Follow Through. Cambridge, Mass.: Abt Associates Inc.
Stringfield, S.C. ,& Slavin, R.E. (1992). A hierarchical longitudinal model for elementary
school effects. In B.P.M. Creemers & G.J. Reezigt (Eds.), Evaluation of Educational
Effectiveness [pp. 35-39]. Groningen: ICO.
Sweeney, J. (1982). Research synthesis on effective school leadership. Educational
Leadership, 39, 346-352.
Thorndike, R.L. (1973). Reading comprehension education in fifteen countries. Stockholm:
Alonqvist & Wiksell.
Verstegen, D.A. & King, R.A. (1998). The Relationship Between School Spending and
Student Achievement: A Review and Analysis of 35 Years of Production Function
Research. Journal of Education Finance, 24, 243-262.
Walberg, H.J. (1984). Improving the productivity of American Schools. Educational
Leadership, 41, 19-27.
Wang, M.C., Haertel, G.D., & Walberg, H.J. (1993). Toward a knowledge base for school
learning. Review of Educational Research, 63, 249-294.
Weeda, W.C. (1986). Effectiviteitsonderzoek van scholen. In J.C. van der Wolf & J.J. Hox
(Red.), Kwaliteit van het onderwijs in het geding. Publicaties van het Amsterdams
Pedogogische Centrum, nr. 2. Lisse: Swets & Zeitlinger.
Werf, G. van der, Creemers, B., de Jong, R., & E. Klaver (1999). Evaluation of School
Improvement Through an Educational Effectiveness Model: the Case of PEQIP
Indonesia. Manuscript submitted for publication.
Werf, G. van der, & Guldemond, H. (1995). Omvang, stabiliteit en consistentievan
schooleffecten in het basisonderwijs. Paper presented at the annual conference of the
Dutch Association for Educational Research (Onderwijsresearchdagen), Groningen,
June.
JS.99.016 - 15-2-2016
48