Evaluating a National Curriculum Reform
Ian Westbury
University of Illinois at Urbana–Champaign
The introduction to this volume describes its origin in a seminar in Oslo centered
on the evaluation of Norway’s educational reform of 1997 (Reform 97). This program of
evaluation studies, sponsored by the Norwegian Research Council and described by Peder
Haug (ch. 1) and Lars Monsen (ch. 2), will continue until 2003. The seminar sought, in
its original context, to frame the issues in this long-term, theory- and research-based
evaluation program and to contextualize the larger issues implicit in the work of the 25
separate but related studies that make up the program.
My task in this concluding chapter is to reflect on these reflections on the larger
tasks that circle around an evaluation of Reform 97, a major and contentious, national
educational reform. In that spirit I will not be attempting to summarize the chapters in
this volume. They speak for themselves. Instead I will be considering some of the larger,
more explicitly educational and policy questions that surround these disparate chapters by
exploring how we might think about the educational and educational policy issues that
face any evaluation of a national educational reform. In that spirit, I will not be
commenting on the on-going evaluation of Reform 97. Instead I will build a parable
around some ideas snatched from what is very much a reading-from-a-distance of the
Norwegian reform as a way of developing a perspective on the tasks facing any
evaluation of large-scale educational reform. As I develop my analysis, I will focus my
discussion around two themes: the intersection between reform, policy-making, and
schooling; and, in the light of that exploration, the questions that seem to follow as we
consider the object of evaluation in an educational reform.
Reform 97: Framing the evaluation
2
. . . the point of departure for the national evaluation of the primary and lower
secondary school is the full breadth of the school’s objectives. The main point of
departure is the degree to which the national objectives for the primary and lower
secondary school are realized as laid down in acts and regulations and in the national
curriculum.
Kirke-, Utdannings- og Forskningsdepartementet (1999)
As I was pondering how I might frame a response to the chapters that make up
this volume, a survey of drug education programs in the U.S. was receiving substantial
attention in the American press (Hallfors and Godette, 2002). The study sought to
evaluate whether or not schools receiving federal funding for these programs were using
“research-based” substance-abuse prevention programs, as the U.S. Department of
Education (USDE) requires them to. Overwhelmingly they were not doing so, although
most district office coordinators who were surveyed reported that they had adopted the
federal standards for program development, which required such programs, as their
standards. The study also concluded that the few districts that were claiming to use
research-based programs were “implementing” them without “fidelity.”
These findings are, of course, all too predictable, at least in the United States. And
the conclusions of the evaluators are equally predictable: There is a gap between
“research” and “practice;” even large school systems lack a “well-organized infrastructure
to select, disseminate, and monitor the quality of substance-abuse prevention
implementation” (p. 468); and schools in the decentralized US educational system lack
“quality controls.”
As we reflect on these conclusions of one evaluation of a contemporary national
educational “reform,” we need to recognize the ways in which a language frames how we
think about schools and schooling, and about the success or failure of “reforms.” As
Lundgren (ch. 5) and Hopmann (ch. 6) suggest, during the last 20 years, the language, the
public ideologies, and the forms of discourse around schooling and education, have
changed dramatically across the OECD countries.1 Thus the American “reform” of
substance-abuse programs I sketched above requires schools receiving federal funds for
3
such programs to: (1) conduct needs-assessments; (2) set measurable objectives based on
their assessments; (3) chose research-based programs; and (4) evaluate progress towards
objectives (Hallfors & Godette, 2002). In the United States these requirements crystallize
what is, despite 40 years of concern with program planning and evaluation, a new way of
thinking for school administrators, one symbolized by a new language code and a set of
managerial and programmatic technologies that are seen as instantiating the ideas that the
new language reflects. With this paradigm shift we see systems of schools being
(re)incorporated into the state, and into political structures, in new ways in order to
support an overriding political concern for the “quality” of schooling. Working within the
perspective of this new architecture, Hallfors and Godette firmly judge the schools they
investigated as failing the test of the reform.2
With this shift in frames, the language of “administration” becomes replaced by a
new language of public sector “management” (see Halligan, 1995). Thus Hallfors and
Godette (2002) write unselfconsciously about the absence in American school districts of
a “well-organized infrastructure to select, disseminate, and monitor the quality of
substance-abuse prevention implementation.” Schooling is no longer a public service,
which is offered to a citizenry to be used, or not used, as individuals decide. Instead
schooling has become an agency that has to be managed by way of rational processes
towards explicit goals, and its outcomes assessed in the light of stated goals, with these
tasks framed in terms of the language of systematic, planful management.
Reform 97 is not framed in terms the terms that are immediately parallel to those
of the USDE’s vision of substance-abuse education, and the evaluation of the reform is
not couched in the quite the frame that Hallfors and Godette’s evaluation instantiates.
However, Norway’s approaches to evaluation are being framed in terms of tasks like
“supplying relevant control data to the state, to municipalities, and to the schools,”
“obtaining information as a basis for development,” and “encouraging the more targeted
use of resources” (Kirke-, Utdannings- og Forskningsdepartementet, 1999). And within
this larger set of purposes the questions around Reform 97 center on the degree to which
4
the curriculum that is at the heart of the reform is “realized”—the core concern that also
lies behind Hallfors and Godette’s evaluation.
But what is the “real” relationship between the work of administrative centers,
i.e., the activities of policy-making, “reform,” resource provision, and the like, and the
work of schools as local sites in local communities, with their own needs, visions, and
on-going tasks? It is a necessary fiction that, within democratic states, a ministry or
parliament decides and public servants implement these decisions. The language of both
the USDE substance-abuse program and Norway’s national program of evaluation
highlight what I will be arguing is the ever-problematic relationship between the
abstracted frames which circle around contemporary educational policy-making and the
practices and patterns of schooling and the curriculum.
Thus, as all too many evaluations have shown, this story of the national legislature
deciding and local educational agencies and schools implementing is too simple: often
schools and teachers simply cannot meet the expectations of the center, because they do
not have the fiscal and human resources that are required, teachers do not have the skills
that are asked of them, and/or they are not given the training and education required to
develop those skills (see, e.g., Cohen, Raudenbusch & Ball, 2002). But, more important
because it is less often acknowledged, reforms with moral and ideological loading in the
political arena must become, as they move to the administrative arena, procedural and
technical. The notion of individualizing instruction for students with special needs, which
was symbolized at the political level in the U.S. by mandating Individual Education Plans
(IEPs), becomes at the school level the action of writing plans, which may or may not be
fulfilled because the regulations under the relevant legislation focused on the creation of
IEPs, not on the actual instruction they symbolized in the public and political arena. And
it is the presence or absence of plans that is the concern of auditors as compliance with
the legislation is evaluated, not how teachers work with students.
In other words, more often than not the outcomes of “reform” become ambiguous
when reform enters the “real world.” Ever-present institutional (vested) interests and
capacities are always at work, and all too often result in symbolic or administrative moves
5
and countermoves, but no mutual accommodation. But we also have to recognize that
such reactions to all but the most explicitly procedural or structural reforms are more or
less inevitable. Even when the mandates of a legislature or central agency seem to be
quite explicit, they are necessarily embodied in texts, which are typically ambiguous in
many ways. These texts in their turn become the “reform,” and are necessarily
interpreted and reinterpreted in every state, school district and school, and by every
administrator and teacher, in the light of their communities’ interests, their collective
ideologies and purposes, and their on-going practices and experiences. In one setting a
mandated curricula can be used to justify and actualize agendas that are already in place;
at other times they are fully assimilated into the curricular, pedagogical, and moral order
of an unchanging school. Or a district or school can shrewdly have it both ways: it can
segment its response by “reforming” in one place while leaving other places alone.
Japanese can be added to the curriculum to symbolize a response to a mandate of
globalism, while the English (or mother-tongue) literature curriculum remains firmly
embedded within the traditional canon.
But, over and beyond such inevitably piece-meal responses to any reform, there is
a larger, more fundamental issue. Cohen, Raudenbusch and Ball (2002) have recently
offered an analysis which seeks to explain why, in the American context, the resources
that are provided by centers, such as the Federal government—like the research-based
programs Hallfors and Godette (2002) describe or Federal mandates that seek to steer
resources towards desired ends by way of incentives or sanctions—typically have all too
little impact on the outcomes they target. They argue that this is inevitable in that the
impact of resources or mandates must be seen as mediated by what is done with the
resources that are made available. “We argue that resources have no direct effects, but
that their effects depend on their use” (p. 81; emphasis added). And “use” is, in their
argument, local and situated: “Teaching is what teachers do, say, and think with learners,
concerning content, in a particular organization of instruction, in environments, over
time” (p. 90). There are only local practices that abstracted prescriptions, regulations, and
6
mandates enable or constrain in the light of the range of teachers’ and students’
capacities, knowledge, norms, and incentives in particular places.
These all-too-obvious paradoxes around curriculum guides and policy guidelines
that do not guide and mandates that do not mandate, and curriculum reform, i.e., planned
change that brings about no change, has led to many attempts to understand what in fact a
“curriculum” is as the formal symbol of what schools seek to do, and why it is so
compelling a topic in educational thinking and policy-making. Doyle (1996), for example,
has sought to differentiate three levels of curricular action and discourse that reflect very
different worlds. He identifies
• an institutional curriculum, centered on policies at the intersection between
schooling, culture and society;
• a programmatic curriculum, centered on (a) the specification of subject content
for schools, school types and tracks, with their core and elective course
requirements or expectations, subject specifications, etc., and (b) the
construction of appropriate content for classroom coverage within these
subjects; and
• a classroom curriculum, centered on the elaboration of the programmatic
curriculum for the worlds of schools and classrooms in their real-world,
particular contexts.3
Doyle observes that all institutional work around the curriculum, around, for
example, the scope and rationale of an optimal curriculum addressing national cultures
and histories, centers on metaphors that reflect idealized norms for an imagined society
and school (see also McEneaney & Meyer, 2000). Such discourse is typically framed in
terms of the need for a convergence between the normative ideal and the ongoing work of
schools. Such discussion rarely, if ever, connects in any direct way to the central issues
around both programmatic and classroom curricula. But, as Doyle notes, the imagemaking that is characteristic of the curriculum “policy” debates within and between
interest groups is important in that such debates symbolize and instantiate what
communities should or do value. In this sense curriculum discussion, debate, and
7
planning—and the public, political, and professional processes involved in such work—is
a social form for clarifying the role schooling as an idea plays in the social and cultural
order.
Programmatic curriculum work has a quite different set of tasks. On the one hand
it addresses the socio-cultural, political, and organizational processes through which the
educational visions that are accepted by élites are translated into operational frameworks
for schools. Such work seeks to precipitate social, cultural, and educational symbols into
a workable and working organizational interpretation and framework. Thus in the United
States, an ideology and a related policy language of “educational excellence” becomes the
introduction of “gifted” programs in elementary schools or university-level courses in
high schools. However, such organizational frameworks remain only indirectly linked to
actual classroom teaching. Curriculum work of this kind frames the character of schools
and classrooms organizationally, and also the ways in which schools might be seen within
their communities. It does not direct the work of schools or teachers in any
straightforward way.
At the classroom level the curriculum is, as Cohen, Raudenbusch and Ball (2002)
also suggest, a body of activities, jointly developed by teachers, students, parents and
communities. This curriculum instantiates their understanding of the potential for them of
the programmatic framework or curriculum. At this level teachers and schools are active
interpreters, not passive agents, of their educational system’s programmatic, or
organizational frameworks. Such interpretations may or may not be well articulated with
the curriculum as imaged at the institutional level or as laid out (and rationalized) at the
policy and programmatic levels. However, the legitimacy of such local interpretations is
not derived from the institutional and organizational frameworks. Instead it derives from
the match between what a local school is, and seems to be, doing and the understandings
of its community about what their school can and should be doing.
In other words, the terms the term “curriculum” must always be seen as
symbolizing a loosely-coupled system of ideologies, symbols, discourses, organizational
forms, mandates, and subject and classroom practices. It instantiates collective, and often
8
very different, understandings about what is to be valued about the idea and the practice
of education. However, at the same time the myth of an authoritative and hierarchical
structure in which legislative bodies and/or ministries determine classroom work, with
the curriculum as the agent of the linkage, is widely understood—as it is in Norway—as
necessary for a public schooling that is subject to political control (see, e.g., Imsen, ch. 8).
It is this paradox that gives all discussion of the curriculum its emotional force! It is this
understanding which also poses the greatest challenge to anyone who would undertake a
centrally- (i.e., ministry- or agency-) sponsored evaluation of an educational reform!
Thus, if the account of curriculum that I have been sketching is correct, some
major questions emerge and circle all around a curriculum reform like Norway’s Reform
97. By implication we must always ask: What is this educational or curricular “reform” a
reform of? What is its locus? Where is it directed? Is its focus “institutional,” in Doyle’s
(1996) sense? Or is it “programmatic?” Is it about changing classroom and school
practice? If it is, is there capacity in the schools to do the job, and/or have the resources
been made available to make this a realistic expectation? If that does not seem to be the
case, what is the (latent) object of the “reform?” What might be motivating it? Is this
reform an attempt to articulate a “new” public ideology, in Lundgren’s (ch. 5) sense, for a
school system? Or is symbolic in some other way, and what interests might be being
served by this reform of the symbolic order? As Elmore (1997) has pointed out, in the
United States there has been a major disconnect over the past two decades between the
political rhetoric of on-going but symbolic educational reform and an unreformed
schooling, which he suggests threaten the legitimacy both of the political class and of
school people!
How do these reflections, derived from curriculum theory and research, link with
the tasks of evaluating Reform 97? The Reform contains elements that are explicitly
programmatic—as with its creation of new frame for education in schools in the north of
Norway where there are significant numbers of Sami and Finnish-speaking children. But
in an important sense these aspects of the reform are peripheral, although important in
principle in ways that I will discuss later. What is more important for the Norwegian
9
educational system is that the Reform has been seen as purposively breaking with the
trajectory of curriculum policy-making, and as such has been controversial (see, for
example, Løvlie, 1998; Karlsen, 2000; Broadhead, 2001; Koritzinky, 2001). To pick up
but one these controversies: the mandated curriculum that is at the heart of the Reform is
a “subject-based” curriculum with an expansive content. This leads Broadhead (2001),
for example, to wonder if the expansive content focus of Reform 97 will lead to a greater
concern on the part of teachers for coverage at all costs rather than for the active learning
and topicality that the curriculum also mandates. But as I have noted, Cohen,
Raudenbusch and Ball (2002) argue that centers cannot prescribe what teachers will do if
those teachers do not know how to do what they are being asked to do, and have no
inclination or local incentives to do it. And in this context we can always ask if a center
cannot steer or direct practice, why would it seek to mandate a curriculum that creates
controversy? Why would educators debate, pro and con, what many would regard as a
non-issue? How does one approach the evaluation of a planned, intentional curriculum
change like Reform 97 that may or not have the potential for impacting on schools in
significant ways?
Evaluating Reform 97
The national evaluation program must be developed in such a way that it helps to provide
the state, the local political and executive authorities and the individual schools with a
better basis for working consciously to ensure that the resources are utilized in the best
possible way to achieve the national goals for education.
Kirke-, Utdannings- og Forskningsdepartementet (1999)
I never intended we should have perfect foresight. . . . You view the introduction of a
new curriculum as a learning process in itself. That it is not finished, it is not something
that is thought through, the implications aren’t there; what type of project work you
should do. It’s not something which is given.
10
. . . you have to do these experiments and see what works and when you find something
that works, teachers have to communicate amongst themselves and tell each other. . . If
the Reform is to be successful, it has to be a learning experience among teachers.
Gudmund Hernes, (former) Norwegian Minister of Education4
The above statements, one taken from a report by the Norwegian Ministry of
Education on the National Evaluation Program for education and the others from
statements by Gudmund Hernes, the Norwegian minister (1990--1995) who took much
personal responsibility for the development of the Reform 97 curriculum, offer two views
of the important tasks in the evaluation of Reform 97. The Ministry writes expansively of
the need to document “the relationship between input and the capacity to meet national
objectives” and, in the context of Reform 97, highlights the quality of content and
experiences, the centrality of the Norwegian policies around equality, and local
adaptation of the national curriculum in order to meet the needs of pupils of different
backgrounds, interests and abilities (Kirke-, Utdannings- og Forskningsdepartementet,
1999). For the Ministry, these expectations come together in the question of whether the
Reform is being realized.
Hernes, on the other hand, identifies potential problems around specific elements
in the reform curriculum, and of the need to seek feedback to address possible problems
with these elements. The evaluation he outlines would be, in a sense, formative—
designed, on the one hand, to gather feedback that could be used by curriculum writers in
the next iteration of the curriculum, on the other hand, focused on gathering, and
communicating, examples of “best practice”—for the use of teachers working within the
reformed curriculum.
However while both of these understanding of the tasks of an evaluation of
Reform 97 are appropriate, prudent and, in a sense, incontrovertible, we must also ask if
these are the only issues that circle around the Reform. As I have suggested, there are
other stakeholders who see the Reform in different ways, and would demand a different
kind of evaluation?
11
Thus, as I have suggested, Reform 97 was, and is, controversial (see Koritzinsky,
2001). The imperative that underlay the work of the educational reform Hernes initiated
as minister and actively directed was embedded a larger policy concern for “the content
and quality of education” in Norway (Koritzinsky, 2001; Broadhead, 2002). In this larger
context, the reform curriculum was a response to the understanding on the part of
contemporary élites that a rigorous education, with a focus on mathematics, science and
modern technologies, is the key to national economic and social well-being and to the
challenges and opportunities posed by globalism and internationalism. For Norway,
education is also one way by which the citizens of a small country can both strengthen
social integration and reaffirm the place of their culture and nation in the European and
global community. The re-assertion of the prerogative of the Ministry to outline and
prescribe a national curriculum can be seen as a response to these starting points.5 The
reform curriculum is seen, moreover, as directed at the total community, and particularly,
at parents, and not merely as an inward policy document of the school system (Sivesind,
2002). It sought, by implication, a firm engagement on the part of parents with the school.
But, for others, the very idea of a national curriculum was seen as a repudiation of
a ten-year policy trajectory of devolution of authority from the Ministry towards counties
and regions, and to teachers, within the very broad guidelines of a central curriculum.
Others would reject the assumption that the quality of education is determined by the
“quality” and quantity of the “national” content that is taught -- and of the significance of
access to, for example. technology as a hallmark of a “quality education.” Furthermore, in
a context in which Norway’s centralized curriculum-making had long been a leisurely and
collegial activity, the active, directive role of the Minister and the Ministry seemed
inappropriate: the process implied the deprofessionalization of teachers and the teaching
profession in favor of the empowerment of the Minister and Ministry.
Such concerns have also elided with substantive and ideological differences
around the educational and curricular preferences that were seen as pervading the
curriculum. Thus there are critics who have questioned its Norwegian nationalism and
“chauvinism,” its prominent emphasis on Christianity6 and western values, its
12
expansiveness and specificity7 and its aggressive embrace of such long-standing but
imperfectly realized curricular ideologies as thematic teaching and activity methods,
without regard to their short-term institutional practicality or feasibility, etc.. Such
responses to Reform 97 problematize any notion of evaluating the straightforward
“realization” of the reform curriculum. They also highlight questions which clearly go
beyond the kind of incremental implementation issues which Hernes raised as a central
focus in the evaluation of Reform 97. They go beyond the issues of capacity which we
saw Cohen, Raudenbusch, and Ball (2002) raising to ask, as I did in the earlier discussion
of Hallfors and Godette’s (2002) assumptions, how much consensus there was among
Reform 97’s stakeholders about what the reform was a reform of, and thus what it meant
for them.
What does the presence of such issues around Reform 97 imply for the framing of
an evaluation of the reform. Should the foci of the evaluation pick up only the questions
that the minister and ministry identified? Or should it embrace the “issues” raised by the
critics of the curriculum in a “responsive evaluation?” But do the issues that the critics of
the curriculum have so vigorously raised yield questions that would be amenable to
“answers,” deliberation, or future policy-making around the reform rather than political
and ideological debate? In other words, are many of the differences in ideological and
institutional perspective around Reform 97 merely metaphors, in Doyle’s (1996) terms,
reflecting normative positions which have no direct, immediate bearing on what most
schools do? Are they issues which bear on the role of a curriculum as what Lundgren (ch.
5) terms the “ideological system” or the normative order around schooling, always
potentially a system that can be symbolically reconstructed, but not something that will
immediately steer the programmatic or classroom curricula of schools? Debate is part
and parcel of the processes by which a society develops an ideological consensus around
the abstract idea of the school and schooling. Does evaluation have a role in clarifying
such issues?
13
Evaluating an “institutional” curriculum
To the extent that Reform 97 is seen to be primarily an ideological platform for
the Norwegian school system, and only secondarily as something that is meant to steer
schooling, real issues emerge about what the reform might be. If we make a shift in
language we can gain some leverage on this problem and, as we engage in this shift, the
issues do become amenable to theory, research, and evaluation in some significant ways.
Thus if we see a curriculum as a narrative rather than an ideology, we secure a
different understanding of the ideological role of a curriculum. In the words of Lindblad,
Johannesson and Simola (2002), such narratives express “images, myths, and sagas that
are to place people in a collective whole” (p. 241). And, as many have observed—
although they have valued the consequences in different ways—modernity, globalization,
and the like have stimulated transformations of both collective and individual narratives,
as Lindblad et al. (2002) use that term. To take one distant example, the idea of the nation
and/or the individual in the global world creates narratives in, say, the new states of
Central Asia of the geopolitics of the nation and of myself as someone who must know
English—or Russian, Turkish, or Arabic—because . . . . An authoritative national
curriculum crystallizes the questions around such narratives, and often gives them a
resolution. In Norway the incorporation in the Reform 97 curriculum of programs for the
Sami and Finnish communities and of Norwegian sign language gives a particular
authoritative collective interpretation to the narrative of Norway, although few
individuals are served by these elements of the curriculum. On the other hand, the
programmatic prescription that English will be taught from grade 1—and German and
French from grade 8—gives an explicit endorsement of a notion of the world that is most
salient to Norway’s future, one that will be noted by all parents who have children
entering the elementary school.
Let us assume that the reformed Norwegian curriculum was developed to
reconstruct and resymbolize just such a “new,” public narrative for the country. Although
not a member of the European Union, the Norwegian élite is part of the European élite
that is, to different degrees, embracing “modernity” and “globalization” and facing the
14
forces that the new world is creating—problems of unemployment, of working within the
demands of a Europe-wide and global trading system with the accompanying difficulties
in controlling the economy, etc.. And, as in many countries, the values of this new order
do not always sit easily with those of older orders: divides are opening between the
groupings and interests that are directly (or potentially) advantaged and/or disadvantaged
by such forces, between “traditional” and “modern” groupings and interests (see Vike,
1997) and “educated” and “less educated” publics. The challenge facing the Norwegian
élites centers on how these different sets of interests are to be reconciled in an immediate
sense and, more fundamentally, on how they are to accommodated in an ideological and
moral sense. The narrative of the Reform 97 curriculum highlights modernity, but at the
same time buffers its globalism and concern for competence with a ringing endorsement
of the narrative of the nation. How is this new narrative being received, by whom, and in
what ways, and to what effect (see Ball, 1990; Elmore, 1997; Lindblad, Johannesson &
Simola, 2002)?
In other words, one can envisage ways in which evaluation might explore the
ways in which the elements and the totality of the curriculum’s narrative interact with the
political-cum-cultural “needs” of different social groupings, communities, and interests as
they view the school system. But with the question becomes one which centers on the
formation (or otherwise) of a consensus in which the public’s schools, with their
reformed curriculum, are seen as a vital collective agency in which the larger community
can find an identity (see Vike, 1997). And, following Ball’s (1990; see also Shore &
Wright, 1997; Lindblad, Lundahl, Lindgren, & Zackari, 2002) lead, evaluation studies can
explore how, in what ways, and to what extent the narrative that is the reformed
curriculum finds its way into the classroom and school—and ask what one or another
kind of penetration can mean for the narratives students develop about their place in what
world. Such questions might not be responsive to a ministry’s mandates for an
evaluation, but they raise questions for “the nation”—which may, as I have been
suggesting, be more immediately salient to what a curriculum is than any questions we
15
might ask about how, to what extent and under what conditions the letter of any
curriculum is “realized” in any immediate sense.
Evaluating an “organizational” curriculum
I contended earlier that the ideological mandates at the institutional/political level
become frames and procedures for the programmatic level. And, as Haug (ch. 1) notes, it
is organizational questions, and the values and goods seen as residing in one other
another organization form, which provided the core frame of reference for the evaluation
of Reform 97:
Should children start school at 6 or 7 years of age? What could be gained
by introducing the teaching of English as a foreign language in the first
year of school, compared to the 4th class, as it was before Reform 97? Is
team-teaching and project work a better way of teaching for all students,
compared to traditional classroom teaching? Has Reform 97 made the
school more inclusive?
But, as Haug also notes, these questions have faded in significance since 1997,
and will likely be moot by the conclusion of the formal evaluation program in 2003.
They cannot provide a useful frame of reference for a significant, long-term evaluation.
But if such questions cannot be the important questions, how should the evaluation be
understood?
It is with this question in view that Haug (ch. 1) suggests that the evaluation
program around Reform 97 be seen as framing a program of research designed to
understanding the working of the Norwegian school system; in other words, the reform
becomes a lens by which we might examine the working of the system. It is over the long
rather than the short run that such a program can make its contribution. The evaluation
program will lead, Haug hopes, to greater understanding of Norway’s schools and school
system -- although the program may also have an interim function of giving feedback to
the system.
But what does such a program for an evaluation program imply? We can concede
that evaluation is a form of research, which derives practicality and utility from its
16
engagement with notions of judgment with all that that word and concept means. But if
evaluation is a form of research that finds its meaning in a judgmental engagement with
practice, the questions become How is that practice engaged? and What values should
animate the engagement? In this volume these questions are joined in terms of the
Nordic debates about the contemporary transformation in the management of schooling
and public sector more generally, the so-called New Public Management, and, in more
muted form, in terms of the prerogatives of centers and regions and/or élites and clients in
contemporary societies (see Stangvik, ch. 3; Karlsson, ch. 7: Imsen, ch. 8). These
concerns lead to questions about the merits of externally-driven, “political,” “goalachievement” frames for evaluation that drive a concern for short-term, and even
expedient, judgments rather than a concern for longer-term “educative” understanding
and engagement with the total system (see Schwandt, ch. 9)
But does the issue around Reform 97 need to be so starkly framed? Evert
Vedung’s (ch. 10) classification of evaluation models offers other ways in which the
evaluation task can be seen, and that list could be extended.8 The utility that is a
necessary part of evaluation only requires that an evaluator identify an issue, and a group
for whom the issue is salient, as a basis for an engagement with that issue. Thus, as we
have seen, Gudmund Hernes identified a cluster of such issues around the project-based
teaching that the reform advocated. Such teaching has, of course, been a hallmark of all
progressivism throughout this century, and has recently experienced a renaissance as an
ideal, but has all too rarely found its way into classrooms in an authentic form. Hernes
saw a need to understand and document instances of “best practices” around which the
further development of project-based teaching might cathect. We might also infer that he
would be also interested in the problems that might face teachers and schools in engaging
with such best practices -- to ask if those problems could be minimized by policy action.
“Inclusion” of children with special needs is a newer example of a policy that has met
fierce “resistance” from teachers, at least in North America, although it has achieved
notable successes. It is an issue that flows more or less naturally from the value that
Norway has placed on equality as a fundamental value for all government programs. We
17
might readily ask how well the practices around “inclusion” are working in the context of
the Reform, and ask, If not, why not?
We might also ask, given the concern in Norway that Haug (ch. 1) notes around
the consequences and effects of introducing English as a foreign language from grade 1,
whether or not something meaningful is taking place in the early grades as a result of this
new mandate: What might it be? and How can the practices which are associated with
the significant effects that we might find be supported, and extended more widely, if that
is what is needed?
In all such cases our focus would be on the ideological and practical “issues”
which inhere is practices and/or settings: what, for example, inhibits teachers’ support for
inclusion, not an ideal but as a practice, or what has long led teachers to reject (or
profoundly modify) project and activity methods? What are and have been the barriers to
the ready acceptance of these pedagogical models on the part of teachers and can policy
makers address these barriers in some way. Or might it be best to re-examine the values
that lead to the official advocacy of such policies? In all cases such evaluations should be
facilitative: gathering “data,” presenting “findings,” in order to facilitate and support
discussion, conversation and learning on the part of all of the stakeholders in the
situation.
In other words, an evaluation might seek to unpack the dilemmas which circle
around the “translation” of policies and ideals into practice, and thereby probe both policy
and practice.9 Is the problem the absence of understanding on the part of many teachers
of the range and character of “best practice” in any of these areas, and/or is it the lack of
immediate supports, i.e., resources, time, etc., which teachers can draw upon or use as
they face the task of “implementation” in their local worlds? Or does the problem lie in
other places? In the structures of pre- or in-service teacher education? In the absence of
networks that might support implementation? In the absence of systems which can
demand that teachers pay (sustained) attention to these ideas and practices and their
realization? Or is it, as I have suggested as a possibility, in the ideas themselves?
18
But a major caveat circles around all speculation about what such a responsive
evaluative engagement with teachers’ working practices might achieve. While
“progressive” change, or a more rigorous engagement with a curriculum, might be
desirable in the best of all worlds, teachers work in very demanding, local places, not in
the best of all worlds. Their time and energy is necessarily devoted to the maintenance of
their classrooms and schools, and changing their practices is invariably an afterthought,
which may or may not be enthusiastically supported by their schools and communities. In
other words, what can we expect of schools as settings that are fundamentally conceived
of, structured for, and evaluated by parents and communities in terms of service delivery
rather than as creators and re-creators of their services?
Such an observation gives us a way of understanding the findings of Bähr et al.
(2000) in their evaluation of cantonal and school-level curriculum-making in Switzerland.
They found that that many of the schools they observed could not use the expansive space
that many cantonal and community curricula had granted them. It was, they argued, all
too easy to develop a “vision” for a school, but much more difficult to develop a viable,
locally-based curriculum from that vision.10
Furthermore, for Bähr et al. (2000), and for Cohen, Raudenbusch, and Ball (2002)
in the United States, even the effective directive role of curricula over classroom practice,
i.e., the classroom curriculum, was also limited. As I noted above, when a new
curriculum ratified pre-existing practice, it was accepted; when it was ahead of practice, it
was rejected. We might ask if this is also the experience that emerges in the response of
Norwegian schools to Reform 97, and what findings along these lines might mean for the
reform goal of a common national curriculum with local adaptations.
But we can also hypothesize that speculations of this kind do not open up the most
important questions about the outcomes and consequences the Reform 97—because the
classroom and school is not the most important locus of the effects of educational
reforms. Thus Bähr et al. (2000) argue that the most significant direct consequence of
new curricula is their effect on educational administrations, not their effect on schools
and teachers. New formal curricula give “school authorities a new frame of reference for
19
the allocation of resources, the licensing of teaching materials, or the issuing of
regulations for examinations and progression to the next level” (p. 26).
In the case of Reform 97 there are explicit administrative stipulations about the
frame of the school day, week and year: about how much time should be devoted to one
or another subject in each year. Such things can be easily monitored but, as Haug (ch. 1)
notes, are not the grist for the mill of any major evaluation program; they are the stuff of a
ministry or central office’s on-going audits. But there are a multitude of very different
issues in the Reform 97 curriculum which are, while part of the curriculum’s narrative or
ideology in the senses that I suggested above, nevertheless authoritative, and have clear
implications for administration. Thus, to use the example I introduced above, if we turn
our interest in the requirement for “inclusion” from teachers to administrations, we can
ask how much support, in terms of resources of all kinds, administrations are giving
schools and teachers and why there might be differences in the provision of such
resources. This example can be generalized in very important ways. How much support
is being given teachers and schools in the areas which the curriculum has explicit
mandates, but which are clearly pose problems for schools and teachers, for example,
projects and thematic teaching, bullying, inter-ethnic tolerance, local adaptations of the
curriculum, and so on. If administrations seem to be falling short in the priority or
resources they are giving to the support of schools in such areas, why might that be the
case? And what might that mean? Are, for example, the priorities of Norway’s counties
and cities, which administer the schools, aligned with the ambitions of the Reform? or are
they directed, right or wrong, towards other problems and issues? And why?
Conclusion
Those of us who center our gaze and professional concern on schools and school
systems as they are and might be, see education as a practical field where doing good
rather than theorizing well is the goal. We want to know “what is” as a preamble to “what
should be,” and for this reason evaluation is compelling. It requires us to figure out what
we know about how schools work, how we can judge what we see, how we get from
where we are to where we might be, and then how we can then adjudicate our sense of
20
what should be in the light of what can be. We believe that traditional educational
research and theory—at least in the Anglo-American tradition—have not been
particularly helpful in that regard. For these reasons the idea of a national evaluation of a
reforming, or at least changing, educational system poses fascinating challenges as we
ponder issues like the relationship between political action directed (seemingly) towards
the betterment of a system and the on-going, day-by-day work being undertaken in
classrooms; between the management of change and the management of the environment
of the school system; between ideology—in both Lundgren’s (ch. 6) sense and in the
more traditional sense of Anglo-American “progressivism” and European “reform
pedagogy”—and what schools actually do.
I have tried to suggest here that there are bodies of substantive theory and
empirical findings about how schools work as institutions and organizations, and how
teachers function within schools as institutions and organizations, which must inform the
evaluation of any complex educational reform. I have also tried to suggest there are many
approaches to the task of evaluation that can yield very different things to different
stakeholders. But what we still need, and do not have, is a comprehensive realistic theory
of schooling. I have tried here to sketch some aspects of what I believe might be the
beginnings of such a theory, and how such a theory might be further developed, by way of
a reflection of the tasks around the evaluation of educational reforms.
But as I have reflected on about the chapters in this volume I have also sensed an
unease and uncertainty about the many issues around paradigms, approaches,
implications, contexts, and politics that tends to obscure the central substantive issues
around evaluating the Norway’s Reform 97. Of course, that unease about the tasks of
evaluation and about educational research, and the contexts we are working within,
extends far beyond Norway. What I have tried to do here is address that mood by
highlighting the endlessly absorbing issues that are being explored in the Reform 97
program, and the promise that that work has for a realistic theory of schooling.
21
Notes
1. For an extended discussion of this issue, see Halligan (1995).
2. But these local systems could be equally well seen as players in a complex political
system which required the performance of meaningless rituals in order to secure monies
which were theirs by right, custom, or political power. Or they could be seen as
responding appropriately to a new remedy adopted by the federal government prior to any
meaningful identification of the problem -- which could, in its turn, be understood as the
need to symbolize concern by both the legislators and schools for the problem of
substance abuse.
3. A parallel but, in intent, somewhat different understanding is offered by Sivesind
(2002); see also Bähr et al. (2000).
4. As quoted (as the “ex-minister”) in Broadhead (2001).
5. The previous curriculum (1987), while a national curriculum, had emphasized local
curriculum development. It was also broadly progressive with, in Koritzinky’s (2001)
words, “cultural, ethical, cross-disciplinary, and social pedagogical challenges . . . as
important as the academic subjects.” (p. 209).
6. It needs to the emphasized that Norway has an established, i.e., state (Lutheran)
church. Indeed the ministry of education is also the ministry of church affairs.
7. Koritzinsky (2001) observes that “ . . . in the subject Christian knowledge and religious
and ethical education 144 names (persons, gods, and other religious figures) were
mentioned. In Norwegian the pupils should be familiar with 68 names of authors, in art
and crafts with 41 names from the visual arts, design, and architecture (p. 213).
22
8. See, for example, Tiller (1990: 222).
9. For models of such interrogation, see, e.g., Phillion (2002); Ross (in press).
10. In the light of this Bähr et al. (1999) suggest that the relationship between an
authoritative formal curriculum and the individual school be reconceived in terms of the
development only of a “local profile,” to be operationalized in terms of a set of tightly
time-limited activities.
23
References
Bähr, K. Fries, A.-V., Ghisla, G., Künzli, R., Rosenmund, M., & Seliner-Müller, G.
(2000). Curriculum-making: Structures, Expectations, Perspectives:
Implementation Report. National Research Programme 33: Effectiveness of Our
Education Systems. Aarau, Switzerland: Schweizerische Koordinationsstelle für
Bildungsforschung.
Ball, S. J. (1990). Politics and Policymaking in Education. London: Routledge.
Broadhead, P. (2001). Curriculum change in Norway: Thematic approaches, active
learning and pupil cooperation: From curriculum design to classroom
implementation. Scandinavian Journal of Educational Research, 45 (1), 19-36.
Broadhead, P. (2002). The making of a curriculum: How history, politics, and personal
perspectives shape emerging policy and practice. Scandinavian Journal of
Educational Research, 46 (1), 47--64.
Cohen, D. K., Raudenbusch, S. W. & Ball, D. L. (2002). Resources, instruction and
research. In F. Mosteller & R. Boruch (Eds.), Evidence Matters: Randomized
Trials in Education Research (pp. 80--119). Washington, DC: Brookings
Institution Press.
Doyle, W. (1996). Curriculum and pedagogy. In P. W. Jackson (Ed.), Handbook of
Research on Curriculum (pp. 486--516). New York: Macmillan.
Elmore, R. F. (1997). The origins and problems of education reform in the United States.
In W. K. Cummings & N. F. McGinn (Eds.), International Handbook of
Education and Development: Preparing Schools, Students and Nations for the
Twenty-first Century (pp. 293-303). Oxford: Pergamon.
Hallfors, D. & Godette, D. (2002). Will the ‘Principles of effectiveness” improve
prevention practice? Early findings from a diffusion study. Health Education
Research: Theory and Practice, 17, 461--470.
Halligan, J (1995). The process of reform: Balancing principle and pragmatism. In J.
Stewart (Ed.), From Hawke to Keatinge: Australian Commonwealth
Administration 1990--1993 (pp. 1--14). Canberra: Centre for Research in Public
24
Sector Management, University of Canberra, and Royal Institute of Public
Administration Australia.
Karlsen, G. E. (2000). Decentralized centralism: framework for a better understanding of
governance in the field of education. Journal of Education Policy, 15 (5), 525-538.
Koritzinsky, T. (2001). Educational reforms in Norway in the 1990s: Civic pluralism and
national unity in decision-making and curriculum contents. In S. Ahonen & J.
Rantala (Eds.), Nordic Lights: Education for National and Civic Society in the
Nordic Countries, 1850-2000 (pp. 204-225). Helsinki: Finish Literature Society.
Lindblad, S., Johannesson, I. A., & Simola, H. (2002). Educational governance in
transition. Scandinavian Journal of Educational Research, 46 (3), 237--245.
Lindblad, S., Lundahl, L., Lindgren, J., & Zackari, G. (2002). Educating for the new
Sweden. Scandinavian Journal of Educational Research, 46 (3), 283--303.
Løvlie, L. (1998). Paradoxes of educational reform: The case of Norway in the 1990s. In
B. B. Gundem & S. Hopmann (Eds.), Didaktik and/or Curriculum: An
International Dialogue (pp. 195--214). New York: Peter Lang.
Kirke-, Utdannings- og Forskningsdepartementet [Ministry of Education, Research and
Church Affairs] (1999). Information Dossiers on the Structure of Education
Systems in Europe, Ch. 9: Evaluation of the Education System
http://www/odin.dep.no/ufd/engelsk/publ/veiledninger/014005-990621/indexdok000-b.n.a.html [Visited September 16, 2002].
Kirke-, Utdannings- og Forskningsdepartementet [Ministry of Education, Research and
Church Affairs] (n.d.). The Development of Education 1991 to 2000: National
Report for Norway: 1. The Education System at the End of the 20th Century
http://www.odin.dep.no/ufd/engelsk/publ/rapporter/014001-220012/indexhov001-b-n-a.html [Visited September 16, 2002].
25
McEneaney, E. H. and Meyer, J. W. (2000). The content of the curriculum: An
institutionalist perspective. In M. T. Hallinan (Ed.), Handbook of the Sociology
of Education (pp. 189--211). New York: Kluwer.
Phillion, J. (2002). Narrative Inquiry in a Multicultural Landscape: Multicultural
Teaching and Learning. Westport, CT: Ablex.
Price, J. N. & Ball, D. L. (1997). “There’s always another agenda:” Marshaling resources
for mathematics reform. Journal of Curriculum Studies, 29, 637--666.
Ross, V. D. (in press). Walking around the curriculum tree: An analysis of a grade 3/4
mathematics lesson. Journal of Curriculum Studies.
Royal Ministry of Education, Research, and Church Affairs (1999). The Curriculum for
the 10-Year Compulsory School in Norway. Oslo: Royal Ministry of Education,
Research, and Church Affairs.
Shore, C, & Wright, S. (1997). Policy: A new field of anthropology. In C. Shore & S.
Wright (Eds.), Anthropology of Policy: Critical Perspectives on Governance and
Power (pp. 3--39). London and New York: Routledge.
Sivesind, K. (2002). Tasks and themes in communication about the curriculum: The
Norwegian compulsory school reform in perspective. In M. Rosenmund, A.-V.
Freis, & W. Heller (Eds.), Comparing Curriculum-Making Processes (pp. 319-332). Bern: Peter Lang.
Tiller, T. (1990). Evaluation in a decentralised school system: Where do we stand? Where
are we heading? In M. Granheim, M. Kogan, & U. P. Lundgren (Eds.), Evaluation
as Policymaking: Introducing Evaluation into a National Decentralised
Educational System (pp. 219--233). London: Jessica Kingsley Publishers.
Vike, H. (1997). Reform and resistance: A Norwegian illustration. In C. Shore & S.
Wright (Eds.), Anthropology of Policy: Critical Perspectives on Governance and
Power (pp. 195--216). London and New York: Routledge.