Sustainability and Learning: developing effective research
My presentation will explore the inter-relationships between evaluation and research, drawing
particularly on work with environmental and sustainability-focused NGOs, (but also with research
councils, government and others). I’ll argue that more care is needed when drawing up evaluation
strategies, not only so that they can be fully integral to the development process, but also in order that
we maximize opportunities for more extensive learning. I’ll also draw on recent surveys of researchers
in order to explore what our priorities now might be for the future.
My background is that of someone interested, in both a research and evaluation sense, in the study of
environmental learning, and in the phenomenon that we know as environmental education (and ESD).
By environmental education I mean that broad church conception which was set out in the 1977 Tbilisi
1
Declaration . This includes those who wish to share the joy and fulfillment that can be derived from
experience of nature in order to bring about life-enhancing and life-changing experiences, and those
interested in stimulating socio-economic change in order to re-balance the human-environmental
relationship – and many more in between these.
I’ll start by saying something about conceptions of research, and evaluation. It’s instructive to see how
we think about research as a social process has changed over the past 30 years. Compare these two
views. One is from the early 1980s. The other is a contemporary perspective from the OECD:
Research comprises:
… investigations employing systematic methods to study or interpret phenomena. It is databased and employs valid observations with an intent to generalise results or build new models
[ 1980s ]
… creative work undertaken on a systematic basis in order to increase the stock of
knowledge, including knowledge of humanity, culture and society, and the use of this stock of
knowledge to devise new applications [ 2007 ]
Although obviously broadly similar, the modern view is much more open in terms of outcomes, with a
change of emphasis from generalisation and modelling, to a focus on knowledge generation and its
application. This reflects a shift in our understanding of what counts as knowledge and hence, as
research. Or, more accurately, perhaps, a change in what’s allowed to count as research. My own
university uses two definitions in order to determine whether an enquiry is to be classified as research or
2
not. The first, reasonably, I suppose, is from the UK’s Research Assessment Exercise ; the other from
Her Majesty’s Customs and Revenue service. The two definitions are essentially the same, however,
with research seen in this way: original investigation undertaken in order to gain knowledge and
understanding. The Revenue’s view is significant as research is zero-rated for VAT, whilst other enquiry
isn’t. And, as the university lives in fear of the inspector’s early morning call, it tends to be conservative in
its classifications. At the end of the day, of course, it doesn’t matter what academics think, it all comes
down to how the umpire in the university’s research office calls it.
The RAE view has a broad scope. It includes work of direct relevance to the needs of commerce,
industry, and to the public and voluntary sectors; scholarship; the invention and generation of ideas,
images, performances, artefacts including design, where these lead to new or substantially improved
insights. Scholarship, here, is defined as the creation, development and maintenance of the
intellectual infrastructure of subjects and disciplines. Research in this sense is restricted to activities
that are directed towards opening up new areas of knowledge or understanding rather than towards
mere quantitative additions to human knowledge. For example, in my university’s view, in the
humanities and social sciences, an activity is seen as research if it aims at widening the general
understanding of society, and its structures and processes – rather than at the solution of particular
local problems. As you’ll appreciate, such distinctions are far from watertight.
That’s about research in general – what about my own field: environmental education research? This
is the definition used in a review published in 1981:
Environmental education research concerns developing or analysing environmental awareness,
valuing, or problem-solving behaviour. [ 1981 ]
And this is the definition from a review of the subsequent ten years of research, published in 1990.
Environmental education research concerns developing and / or measuring environmental
awareness, ecological and issue-related scientific knowledge, issue investigation and decisionmaking skills, the empowerment of learners as environmental change agents, responsible
1
2
See http://www.gdrc.org/uem/ee/tbilisi.html
http://www.rae.ac.uk/
Galway Talk 091002
environmental behavior …" [ 1990 ]
This is more detailed as you see, but the definition’s still about awareness-raising, and on what
individuals can do – for example as change agents and problem-solvers, with a focus on their moral
responsibility. This 1990 definition also included a statement about the breadth of focus of
environmental education research and its outputs:
Environmental education research focuses on ... studies of programs, projects, personnel,
facilities, curricular materials, and other resources involved in the delivery of environmental
education … including systematic reviews, critiques, summaries, and syntheses of research
studies. [ 1990 ]
Here, you see an emphasis on researching formal educational provision, its delivery and the
resources needed. Of course, all such definitions tend to reflect the interests of those doing the
defining, but I think that any over-arching view today would be much broader, and more socially-aware
than this, in order to reflect the considerable shifts in perspective that have taken place – about what
research is, for example, what it might usefully focus on, and whose interests it needs to serve.
Environmental education research now claims a broad social relevance, as it has shifted from
narrowly-constructed research based in the scientific-realist paradigm, to include enquiry based on
wider and more sophisticated grounds. There’s also been a change of emphasis from generalisation
and modelling, to knowledge generation and its application. The field is now rich and complex and
has seen significant changes in who researches, where research is carried out, and where it’s
reported. And, as we’ve seen, significant shifts in the underlying conception of research itself as a
social process that has broadened out and become more inclusive. Cleary, what environmental
education research is, and how its done, is now representative of research conducted across the
whole of the social sciences.
So, where does this leave evaluation? It seems to me that evaluation involves making judgements
about the quality, value, merit or worth of a process or activity so that informed decisions about future
activities or developments can be made. Evaluations can involve feedback during an activity, thus
allowing an understanding of the processes involved – and this is often viewed as quality assurance.
It can also focus on outcomes, usually occurring at the end of a process. Research and evaluation
can both involve the planned and systematic collection, analysis and interpretation of data. The chief
difference between the two is the nature of the question which is being addressed, rather than,
necessarily, the processes which are being used to address the question. With evaluation, however,
there is usually an ‘evaluation question’ for which an answer is being sought, and it can sometimes be
helpful to think of evaluation as a particular, restricted kind of research enquiry. And there’s a useful
distinction that can be made between evaluations aimed at proving quality, and those aimed at
improving quality;
In the former sense, this might mean demonstrating quality in terms of:
• Value: payback from an activity – especially monetary;
• Merit: the degree of excellence of programme or activity;
• Worth: the overall usefulness of, or need for, the activity.
In the latter sense an evaluation might aim to:
• Pin-point weaknesses to be remedied;
• Identify strengths and achievements which might be disseminated;
• Facilitate the formulation of strategies for further development;
• Lead to an understanding of the process of change.
I think that great care is needed when drawing up evaluation strategies, not only, so that they can be
fully integral to the development process but also in order that we can be creative in maximizing
opportunities for research, and hence more extensive learning. We can do this by bringing our
scholarly knowledge and understanding to bear on evaluation questions and process, thus increasing
the leverage we have in revealing insights. This is not always a welcome process, however,
especially to those who would rather like their activities to be scrutinized on their own narrow terms,
and not in relation to wider thought – and it’s easy to run into opposition if you do this. For example,
just because participants say something was great, doesn’t mean that it was. We can also report
evaluation studies in research forums and outlets, and this brings considerable benefits to all
concerned.
Whether we think about evaluation or research, however, our biggest problem is that we are still
largely a minority activity in isolation from mainstream. Philip Payne commented on this:
2
“… by and large, the mainstream discourses of education, educational research and teacher
education remain immune to, resistant or disinterested in the enclave of thought, research and
practice in environmental education research.”
All this seems disturbingly true, but maybe nobody else believes our persistent rhetoric that achieving
global social justice whilst living within the Earth’s carrying capacity is something that will be built
mainly through formal education. There is also this persistent division between environment and
development which is mirrored, in the UK, at least, by separate government ministries, distinct overarching associations and research centres, and specialist NGOs. This is less true of curriculum
authorities, but the Welsh devolved administration feels it has to make a distinction between ESD and
global citizenship education. It sometimes feels as if there has been a long-standing state of tension
between hostile camps, each ready to fight for territory and influence, but this is probably a reflection
of something of much greater significance, the tension between social justice and environmental
protection arguments within the discourse around sustainable development. You only had to read the
early drafts of the ESD Decade documentation to appreciate the palpability of such tensions. And all
this raises a question about what (and who) environmental and developmental education is now for –
and the roles that schools, colleges and universities might have in it.
John Foster has made a telling contribution to our understanding by arguing that Sustainable
Development makes no sense other than as a social learning process of improving the human
condition that can be continued indefinitely without undermining itself. He argues that Sustainable
Development doesn’t depend on learning. Rather it’s inherently a learning process of making the
emergent future ecologically sound and humanly habitable, as it emerges through the continuous
responsive learning which, Foster says, is the human species’ most characteristic endowment. The
sort of learning we’re talking about here is about how we live (and might live) with each other and the
planet, and we still know very little about the ways in which learning impacts on people’s lives – or the
ways in which people’s living affects their learning. To do this, we’ll need to work with other
researchers and agencies and recognize that the problems we face cannot be labeled ‘environmental’,
‘developmental’ ‘social’ or ‘economic’, but need to be seen as something much more subtle, and taken
in the round.
The last 15 to 20 years have seen a marked deepening of interest in the development of research
skills, a greater exploration of the nature of research as a practice, and some welcome lessening of
tensions between entrenched methodological camps. It seems clear, both within our field, and in
wider educational contexts, that the last few years have seen a realisation that merely focusing on
education in relation to sets of pre-figured outcomes – without a complementary focus on the learners
themselves and what they want to learn, is perverse, and there have been a series of significant
reviews and projects addressing this. The problem has been that, too great a focus on process and
input (that is on pedagogy and teaching), tends to indulge those who are convinced that they know
what others need to know – but this is a conviction as old as time, of course.
3
Recently, my colleague Alan Reid and I edited a review of research reported in the first 10 years of
the journal Environmental Education Research. This involved reprinting what seemed to be seminal
papers, and then commissioning future-focused synthesis articles around these. In doing this, we
asked authors to make a series of recommendations about future research foci and approaches, and
I’m going to end my talk by saying a little about what emerged from this, and inviting you to think about
how this relates to your work.
We thought that 3 key ideas emerged:
[1] Environmental education / ESD research as connected across interests, preferences,
approaches, time and distance
Authors called for more openness towards new ways of doing research, whilst constructively engaging
with work already done. They urged us to be driven by questions, rather than by preferred methods or
even methodologies. The point, they argued, is to recognise the multiple paradigms of educational
thought as unique, historically situated forms of insight which require reconciliation at the paradigm
level, with a commitment to respect conflicting ideas and values. Authors argued a need to research
multiple epistemological approaches to the point that members of different communities of research
practice can understand and engage with one another through their differences and a need to build
bridges between different kinds of research to see how they might be usefully combined to create new
space and opportunity. Authors argued that we needed to see ourselves as members of a community
of reflective scholars, and not just as aggregates of individuals, competing camps, or as unconnected
3
Reid AD & Scott WAH (Eds.) (2009) Researching Education and the Environment: retrospect and prospect;
London: Routledge [paperback edition]
3
researchers without common interests. They urged us to be open-minded, respectful and generous,
acknowledging the goodwill and intelligence of each other, whilst being critical and exercising
judgment. They said that we needed to have the courage to discuss the ‘holes’ in research, and
research that doesn’t go to plan, and called for more openness on the part of reviewers, editors and
publishers. And finally, they urged us to research in much wider education and learning contexts than
those traditionally associated with the field.
[2] Research foci needing attention
Authors argued that there is a need here to research what we know and how we have come to know
this, and how such knowing, whether linguistic or embodied, serves to produce who we are, and might
yet become. They argued that attention needs to be focused on approaches to research and
pedagogy that interrogate how dominant educational discourses make it difficult, or (im)possible, for
teachers and students to engage in environmental education or ESD. They argued for the
encouragement and support of research that can shed light on the relationship between race / power /
and culture and between action / experience /and technology. Authors urged us to acknowledge that
research as a process has to provide opportunities for learning for everyone involved: researchers,
practitioners and other users. They also asked us to work towards educational and research
discourses that are informed by practitioners and practice in diverse cultural contexts, starting with real
issues and experiences in local communities, and what (young) people feel are the most appropriate
courses of action.
[3] Sustainable development as inherently a learning process that needs researching by / with
those involved in the dynamics of such learning
Authors argued that there’s a need to acknowledge that we still know little about the nature and
dynamics of learning in relation to sustainability, and to recognise that we are engaged in an
incremental and somewhat erratic process of learning our way into an uncertain future, where a key
aspect is the development of learners’ abilities to make sound choices. Authors argued that if our
understanding of learning is to develop and deepen, we need to shift what we do in 3 ways:

to become more reflexive about what we mean by learning, and more wide-ranging in
where, when and how we seek to research such learning

to become more sophisticated in our use of theory and existing knowledge

and to become much more creative in how we seek to integrate knowledge generation with
knowledge transformation and utilisation.
Authors argued for us to recognise the tendency for environmental concerns to be stripped of their
historical complexities, and for a global agenda to be imposed on local, culturally situated, issues.
And finally, they argued for a shift from making policy pronouncements on what ESD should be, as
abstractions decontextualized from practice, that are to be enacted by educators who have not
participated in the formulation of goals and concepts, to engaging educators and young people so it is
constructed with them, and contextualized historically and politically.
There is a lot here. In reality, there is much more, of course, as this is a distillation down from some
50,000 words of close argument. I imagine there may be issues here you don’t agree with, but I think
that the wide canvas upon which all this analysis is based does have something of significance for all
our work.
………………..
William Scott
University of Bath, Centre for Research in Education and the Environment
October 2009
4