Working Paper 4.2
Report of the working group on
A National Evidence
Centre for Education
Richard Andrews
Andrew Morris
On behalf of the NECE working group
December 2005
1
The National Educational Research Forum (NERF) is an independent
organisation. Its role is to oversee the development of a coherent strategy for
educational research and its use.
NERF organises projects, seminars and workshops inspired by and engaging
with its partners. The outcomes of these can be found in NERF’s Working
Paper series, copies of which can be downloaded from the NERF website.
Contact NERF by email on info@nerf.org. Website address: www.nerf-uk.org.
Working papers: list of themes
1 Capacity building
2 Systematic reviewing
3 Research strategy
4 Evidence centre
5 D&R programmes
6 Priorities
7 Practitioner engagement
8 Policy and research
9 About NERF
2
Foreword
This report has been prepared by a working group established by the National
Educational Research Forum in 2005. It worked from July to November 2005.
Its members are listed below.
The working group was established by NERF to explore in greater detail the
strategic proposal it had made in 2004 for the creation of a national evidence
centre for education (NERF Strategic proposals, 2004).
The working group was chaired by Richard Andrews, Head of the Department
of Educational Studies at the University of York. Its members had
backgrounds in teaching and research in schools, colleges and universities
and teacher training, and in the organisation of evidence for policy and
practice in the social and healthcare fields.
Richard Andrews
Andrew Morris
24 November 2005
Members of the working group:
Richard Andrews
James Durran
Amanda Edwards
Vicky Hames
Marilyn Leask
Andrew Morris
Mike Prosser
Nancy Rowland
Professor of Education, University of York (Chair)
Advanced Skills Teacher, Parkside Community College,
Cambridge (Vice-Chair)
Head of Knowledge Services, Social Care Institute for
Excellence
Primary Teacher, Copmanthorpe Primary School
Head of Effective Practices and Research Dissemination,
Training and Development Agency for Schools
Director, National Education Research Forum
Director, Research and Evaluation, Higher Education
Academy
Head of Research, British Association for Counselling &
Psychotherapy (ex-National Health Service Centre for
Reviews and Dissemination)
Administrative support:
Alison Robinson
Information Officer, English, Maths and Science Review
Groups, University of York
3
Table of contents
Executive Summary
5
Introduction
7
Role
8
Function
9
Scope
10
Scale
11
Partnership
13
Outputs
14
Impact
15
Governance
17
Transformation of evidence into action
18
Quality
19
Conclusion and recommendations
20
References
21
Appendix I
Policy background
22
Appendix II Recent research background
23
Appendix III Existing evidence centres
24
Appendix IV Examples of output
26
Appendix V Quality framework
27
Appendix VI Contact with Scotland, Wales and Northern Ireland
28
Appendix VII Diagram of NECE
29
4
Executive Summary
Rationale and aims
Following the establishment of the National Institute for Clinical Excellence
(NICE) in the 1990s, and the Social Care Institute for Excellence (SCIE) in
2000, the time is now ripe for the creation of an analogous body for
Education. This body, which we have provisionally entitled the National
Evidence Centre for Education (NECE), will build on and coordinate existing
initiatives in the field.
The principal aim is to provide better evidence to inform teaching, and thus to
improve the learning of all students in the UK. There is therefore a social
justice dimension to the proposal: to contribute to the removal of inequities in
the education system by providing a more secure, and more accessible
evidence base for teaching and learning.
Associated aims include enhancing the awareness of research evidence for
teachers and lecturers; increased efficiency and effectiveness in the
transformation of research findings into practice and policy; and ensuring that
leading edge teaching is supported and challenged by the best evidence
available.
Needs and actions
The working group recommends that an evidence centre be established for
education that:
1. embraces all aspects and phases of education
2. is designed for use by all involved in education – lay and professional
3. produces outputs designed to help a range of types of user, including
practitioners, policymakers and researchers, translate research and
evidence into action
4. provides an on-line search facility for research-based evidence and a
range of publications based on syntheses of high quality research
5. attends to the transformation of research evidence for the benefit of
users.
To achieve this it should:
6. combine a small core staff with wider networking, exploiting electronic
and regional resources
7. work in partnership with organisations already offering support and
guidance for practitioners using evidence
8. act as a node for networks and e-communities encouraging both the
application of research evidence to practice and policy, and the
accumulation of research evidence reliable enough for national
application
5
9. commission the bulk of its outputs from external providers
10. take account, in its governance arrangements, of the wide range of
stakeholders with vital interests
11. be developed gradually, building on the experience of existing centres
and projects.
12. develop and apply procedures to ensure that users are not
overwhelmed with an excess of unfiltered material.
Recommendations
This report outlines what is a radical and multi-dimensional idea. It is intended
to assist NERF and DfES in developing their proposals. Members from the
social care and healthcare fields were particularly struck by the opportunity
presented in education to build upon the experience of existing initiatives in
other sectors.
The working group believes that many challenges would best be addressed
by starting to “get on with the job”. It is for this reason that it recommends that
the beginnings of a centre be established soon and that stages of
development and piloting be used to resolve technical issues as they arise.
The group hopes that this report will assist in the establishment of an
embryonic centre and in the dialogues that will be necessary to ensure it has
support from, and learns from, all interested parties.
The working group therefore recommends that NERF and DfES:
a) take forward the ideas set out in this report and begin to
establish a National Evidence Centre for Education
b) engage potential partners and stakeholders in a dialogue in
order to design the development of the centre.
6
Introduction
1. Key initiative. The National Educational Research Forum was established
by the Department for Education and Skills in 1999 with a remit to develop
a coherent strategy for educational research in England. The priority that
emerged from early discussion was the need to strengthen the connection
between research, policy and practice. This approach was subsequently
reinforced by recommendations in an OECD examination of educational
R&D in England (OECD, 2000). After discussion within and beyond NERF
a key strategic initiative was proposed: the establishment, at national
level, of a centre or system for education that would bring together
research evidence scattered across disparate organisations and
make it available in easily useable formats.
2. A national centre or system. The proposal was for a centre or system to
cover all phases and aspects of education, useable by all kinds of
education professional, whether involved in research or not. Evidence was
to be primarily research-based, though links to inspection, administrative or
other kinds might be included. No particular view was held about whether
the centre should be a physical place, a virtual system or a mixture of the
two.
3. The working group. In 2005, a working group was set up, chaired by
NERF member, Richard Andrews, to investigate in greater depth some of
the issues such a proposal raises, drawing where relevant on experience
elsewhere. The fields of social care and healthcare have proved instructive
in this regard.
4. Outcomes. This document reports the outcomes of the working group. It is
written for the members of NERF in the first instance. After discussion
within NERF, the report could be made generally available to stimulate
public discussion and consultation on whether such a resource should be
brought into being for education.
5. A National Evidence Centre for Education. The report begins by
considering the role of such a centre, what it would do and for whom. It
then addresses questions of the scope, scale and organisation of such a
centre and finally assesses its potential impact. The working title – National
Evidence Centre for Education – is used in this report for convenience.
The working group felt that a different title could be sought if key
stakeholders so wished, but, from a teacher’s point of view, the working
title seems clear and to the point.
7
Role
6. Purpose of the centre. The primary purpose of an evidence centre is to
inform people who are making decisions or trying to improve practice by
making research-based evidence readily available. A number of initiatives
already exist for this purpose in many branches of education and social
science. This proposal seeks to build upon these by:

making evidence currently held in many different sources
available through one access point

facilitating the accumulation of evidence through research
synthesis

using this evidence as a basis for useful guidance materials
for specific groups, such as teachers.
7. Audience. The group proposes that it should be designed for the benefit of
all interested in education – professional and lay – including all involved in
teaching, supporting, managing and leading learning, developing policy
and materials, teaching teachers, researching, guiding learners and
helping them at home, in the workplace and the community.
8. Brokerage and value for money. A centre would necessarily play a
brokerage role between organisations that produce research evidence and
those that need to use it. To succeed in this it would need to work in close
partnership with many kinds of organisation including universities, research
councils, governments, agencies and others in the voluntary, public and
private sectors. The role of a centre would be to secure greater value from
the separate efforts of partner organisations rather than to attempt to
appropriate any of their functions.
9. Evidence for policy. For policymakers the centre would need to have a
clear location in the chain of decision-making from ministers, through
officials to those who provide educational services. For the highest level,
where items of high quality research evidence are already made available
through the work of analytical officials, the role of the centre would be to
commission syntheses or interpretations, particularly where the evidence
appeared to be divided or where absolute independence of judgement
needed to be clearly seen.
10.Rapid response for policy purposes. For policymaking and for
practitioners at all levels it would offer a rapid response facility for issues
arising at short notice. For this purpose it is proposed that the centre
maintains up-to-date information on where expertise lies on given topics
(via expert networks as well as through individuals) and develops methods
for commissioning rapid syntheses of evidence, on a politically realistic
timetable.
11.Evidence for practitioners. For practitioners and those that work around
them, it is more important that evidence is accumulated that is both
relevant and sound, rather than available at short notice. Although some
research evidence is now assessed and written up specifically for
teachers, most is not yet available in a useable form or accessible through
8
a single point of contact. Practitioners need useful materials to support
them in their daily practice. Such materials are already provided by the
many organisations that support them but they vary in the degree to which
they are informed by evidence from sound research. The working group
proposes that, over time, the evidence centre works in partnership with the
responsible bodies, such as TDA and GTC, to develop innovative teacherfocussed materials, informed by synthesised research evidence, designed
for practical use.
12.A portal. What is needed urgently by both practitioners and policymakers,
as well as by all other users, is a web-based portal providing access to
evidence on a given topic from diverse sources. An example from the
social care field, social care on-line (www.scie.org.uk), demonstrates the
potential value of such a facility. A NERF project is currently working on
an education version.
Function
13.Online service. Arising from this description of its role, it is clear that a
centre would have a number of key functions. First, it would need to be
able to draw on research reports, articles, reviews, summaries and other
kinds of material from disparate sources. This will require a sophisticated
search facility to enable materials to be brought together on a screen in
response to a search request. The technical problems to be overcome in
achieving this, in the areas of quality assessment, meta-tagging and
presentation are currently being worked on in a NERF project: the National
Educational Evidence Portal (NEEP). The example of social care online
(SCIE, 2005) demonstrates the ways in which different kinds of material
suited to different kinds of user can be clearly distinguished and quickly
accessed.
14.Guidance services. The second key function would be to produce
published or web-based materials informed by high quality research.
These might include, for example:
a) guidance for practitioners on priority issues
b) advice to policymakers on the state of knowledge on a topic
c) advice to research funders on topics where research evidence is
lacking
d) tools to help people make effective use of research
15.Synthesis service. To produce these kinds of output a number of inhouse processes would need to be developed. Information would have to
be identified from many sources and classified according to various
aspects of quality (see section on quality). Where appropriate and
possible, evidence from primary studies would need to be synthesised to
provide users with a concise appraisal of research information on a topic.
The knowledge from such syntheses would subsequently need to be
transformed for use in the contexts of practice and policy. In the view of the
9
working group, substantive work on synthesising evidence and publishing
materials should be commissioned, but not undertaken by, the centre.
Specific capacities that would be required in–house include:
a) Technical resource to develop and maintain electronic search
facilities
b) Decision-making apparatus for selecting topics for synthesis
c) Expertise on design of materials for different user types
d) Commissioning capability for syntheses and production of materials
16.Network liaison and support. The working group took the view that the
tasks of assembling evidence and producing materials would, of
themselves, not be sufficient to impact significantly on practice. Studies of
research impact (Nutley 2004, Rickinson, 2005) suggest that practitioners
need to engage actively with, not merely be exposed to, research
evidence. The group recommend that the centre should engage with the
networks that already exist amongst practitioners, policymakers and
researchers to encourage the use of the materials and to monitor their
success. There may also be areas in which new networks would need to
be established.
17.National research strategy service. In addition to the primary function of
making research-based evidence available, the working group suggest that
the centre could usefully provide information relating to the wider
“business” of research. Acting as single point of contact, serving all sectors
and communities in education, the centre could collate and make available
information on the research agendas of disparate organisations with a view
to identifying areas for collaboration and areas of overall weakness. Such
a function could potentially assist in the identification of overall priorities for
educational research in the future and perhaps for collating work on
horizon-scanning or scenario-building.
Scope
18.Comprehensive scope, but through a phased approach. The working
group recommend that the scope of an evidence centre should extend to
all age phases and aspects of education – from early years to higher
education, from the nursery to the workplace. It should, in principle, be of
interest to anyone with an interest in the education system, including
parents, employers, learners and community groups in addition to
professionals. In practice, however, resources would be limited at any
given time and tough choices would be have to be made about topics to be
covered and clients to be addressed.
19.Phase 1. Initially, it would be advisable for the centre to identify topic areas
for which there is believed to be both a reasonable body of sound evidence
and a lively demand from users. It would be advantageous if topics were
chosen initially that transcend sector boundaries and were of interest to
both practitioners and policymakers. Later, after the value of the centre’s
10
services had become more widely appreciated, the range of topics and
client interest might be expected to increase.
20.National dimension. The working group favoured the use of the word
‘national’ in the title to denote its embrace of local, regional, thematic and
sectoral information. Whether the word also referred to England or the UK
as a whole would depend on the interest shown from the four constituent
countries that make up the UK and on who funds it. The group felt that
although there are significant differences between England, Northern
Ireland, Scotland and Wales in educational provision, curricula etc., the
underlying issues are common and that a UK-wide centre devoted to the
gathering, dissemination and translation of research-based evidence would
be desirable. Initial contact has been made by the working group with
colleagues in the other UK countries, and in-principle support has been
received (see appendix VI)
Scale
21.Both physical and virtual. The working group deliberated on whether the
centre should be conceived primarily as a physical building with a central
staff team or as a largely virtual enterprise. It recommends a mixture of
both. The ‘centre’ will need a core staff located in an actual building to
develop expertise and a way of working. These might include researchers,
teachers, knowledge managers, communicators and administrators. At the
same time e-communities, and perhaps some live groups, should be
developed to facilitate networking and impact activities regionally,
nationally and, for some purposes, internationally.
22.Size. The working group found it difficult to estimate an appropriate size for
a centre at such an early stage of planning. However it did feel able to put
forward one or two suggestions in relation to size.
23.A small start. It would advisable for the centre to begin on a small scale,
addressing a very limited range of issues, in order to concentrate on
developing procedures agreeable to all parties and to demonstrate its
value rapidly. Once this had been achieved, the centre could expand to
cover a wider range of topics and client groups, as resources were made
available.
24.Staffing – roles. However, a minimum range of capacities seems
inescapable in a complex brokerage operation. For a centre to hold
together and function at all it would need:
a) a director
b) experts in synthesis
c) an expert in policy analysis for the ‘rapid response’ function
d) an ICT network specialist to develop and maintain electronic
systems
e) a publications and communications specialist for editorial, design
and marketing issues
f) administrative/PA staff, including financial and commissioning
11
capability
g) part-time seconded teachers/lecturers, officials and researchers to
develop materials and networking
25.Staffing – numbers. The scale of staffing could not be finely determined
by the working group at this stage. However, it estimated a range of 5 – 10
fte staff to operate a centre initially and noted that the National Institute for
Clinical Excellence, a pioneer in the field, began with just four staff but
grew rapidly, whereas the Social Care Institute of Excellence was
launched with a complement of 20 staff, drawn largely from predecessor
organisations. The group suggested that the number of staff needed at
subsequent stages would depend on the way the centre evolved. The
group suggests that a pilot be started drawing on the resources of a host
organisation for personnel, finance, contracting and other functions. If this
approach were to be adopted, however, the centre would need to be
careful to preserve its independence from the host, and to be seen to do
so..
26.Growth options. Whatever the scale of funding, the services provided by
a centre would need to be phased in over time. The sheer quantity of
material to be processed and the multiplicity of user demands would be
likely to outstrip the capacity available initially. The working group
considered two fundamentally different growth options:
a) developing a system in which suppliers freely submit their research
material to an open database. This offers the possibility of most
topic areas being covered from the outset but risks an
unmanageable quantity of unfiltered material for users.
b) choosing priority topics and selecting material according to agreed
criteria. This would be more likely to result in a manageable quantity
of relevant material for users, but coverage would be more patchy
initially. Suppliers would have to submit to an extra selection
process, and wait for their topic area to be given priority
In the light of the proposal to ‘start small’ and concentrate on quality (see
earlier in this paper), the working group proposes that the second
approach be adopted for the production of syntheses and resources for
users. However, the on-line facility, building on the NEEP pilot project,
could provide more extensive coverage of topics, based on documents
supplied by the research community. Technical development work on
these approaches would need to be tackled in a pilot phase of the centre.
12
Partnership
27.Collaboration with agencies. The education world is blessed with an
abundance of agencies and bodies to support the work of the teaching
service and many of these have an evidence-producing and/or evidenceusing function. At present these provide guidance and other services for
specific client groups – schoolteachers, post-16 leaders, teacher trainers,
curriculum developers, examiners etc. The working group was concerned
to ensure that an evidence centre should provide a useful and specific
service for users rather than cause confusion. A new centre would need to
work in partnership with these many organisations and to demonstrate its
additional value. .
28.Benefits for agencies – pooling of evidence. In one important respect
the advantage of a single national level evidence centre is unambiguous
for these organisations: the pooling of evidence. Each has limited capacity
for R&D itself, yet needs to be able to call upon evidence from a wide
range of sources and to disseminate its outputs widely. A large number of
organisations has already joined together on a voluntary basis in the NEEP
project because each sees the long-term merit in a common approach to
classifying, retrieving and displaying research information. The working
group proposes that the evidence centre builds on and takes forward the
preliminary work of NEEP. This involves addressing the technical
difficulties of pooling research materials and developing an on-line facility.
A further stage could involve inviting additional research organisations to
join the consortium and developing more sophisticated functionality in the
portal. See paragraph 56.
29.Quality and kite-marking. In a different respect the work of the centre
would require more detailed one-to-one negotiations with partners.
Producing guidance materials for specific client groups is a function that
many organisations already perform. The value of the proposed centre
would be to see that these were informed by high quality research
evidence. In doing this it would need to avoid confusing the market with
apparently overlapping materials. The working group proposes that the
centre should act in partnership with relevant organisations to commission
syntheses and produce materials. It would need to work closely with
bodies that are developing methods of synthesis, such as the EPPI-Centre
(also the ESRC centre for research synthesis) and with those that produce
materials, such as government departments, NDPBs, universities and
others in the voluntary, public and private sectors. It would need to be
decided at a later stage whether the centre should have a distinct
publication identity and/or should confer a publicly recognisable kitemark
on materials produced by partners.
13
Outputs
30.Internet services. The presence of the evidence centre would be
experienced by most people in education through its outputs. The mostly
widely used would probably be its on-line portal, available to all through the
internet. This would be located on the home page of a website and would
enable people to reach research evidence on a topic. Other services would
be publicised via the same website.
31.Print-based service. The working group recommends that printed
publications should also be made widely available as there is evidence that
both teachers and senior managers use these extensively. The production
of these should be sub-contracted, either by the centre itself or by partner
organisations with which it had agreed to collaborate. It is suggested that
in the first instance these should be few in number and focus on guidance
on topics of major concern. A distinct publication identity is likely to be very
important to the recognition of the centre.
32.Types of document. A consultation document published by the National
Institute of Clinical Excellence (NICE) during summer 2005 provides a
useful starting point for classifying how evidence might be presented. It
proposes four types of output document, aimed at readers with different
needs:
a) a quick reference guide
b) a longer summary/effective practice guide
c) a version for the general public
d) full guidance, including any technical report.
The working group suggests that these might also be appropriate types for
an evidence centre in education.
33.Not a ‘what works’ approach. The working group felt that guides to the
evidence would be well received by users, if recommendations were to be
couched as ‘implications for practice, policy or further research’ rather than
as guidance on ‘what to do’. Where evidence is equivocal or inconclusive,
this information should be included alongside positive and negative
findings. The group felt that the ‘what works’ approach over-simplifies the
relation between evidence and action.
34.Rapid reviews. A distinct output of the centre, particularly for
policymakers, would be rapid reviews on urgent issues. The centre would
be in a position to use its growing knowledge of where expertise lay and of
technical developments to commission these on behalf of government,
agencies or other organisations.
35.Strategies for utilization. The working group felt that, in addition to the
production of useful materials for different client groups, the centre should
also encourage their active use. Studies of evidence utilisation suggest
that this may well involve social processes in which groups of
professionals adapt the materials within their various communities of
practice. The working group proposes that networks associated with such
communities be encouraged to associate with the centre to help stimulate
14
this kind of activity. It is anticipated that these would largely be existing
networks, though where necessary the centre might also stimulate new
ones.
36.Research strategy information and outputs. A further kind of output
would be information on the ‘business’ of research, collated across the
various partner organisations. This might include information about current
research programmes, future plans, publications, key research questions
etc. Such a role might be appropriate for the centre because it would be
independent and active across all sectors, with growing expertise in
information handling.
37.A journal of record. A final suggestion put to the group as a possible
output from the centre was a “journal of record” for education in both print
and electronic formats. The medical profession uses a journal of record,
The Lancet, to disseminate and provide an arena for debate.
Alongside outputs, it could provide a digest of research, which maintains
the highest standards of peer review, which will have interdisciplinary
presence, and which will respond to current issues in education (see
appendix IV).
Impact
38.Experience from other professions. The potential impact of an evidence
centre would depend on many unpredictable factors, and would have to be
estimated during the early developmental period. Initial confidence in the
idea of an evidence centre comes largely from experiences in other
sectors: first in medicine, health promotion and public health, subsequently
in social care. NICE and SCIE have grown from small beginnings,
surrounded by uncertainties. By choosing priorities carefully and attending
to a range of stakeholder interests, they have become generally accepted
as valuable.
39.Comparison with SCIE. Greatest impact is likely to be with practitioners
and policy-makers. The case of SCIE provides a useful comparison. In
both the education and social care sectors, there is a disparate research
and evidence base, a dispersed workforce, a developing – but not
embedded – culture of using research in practice, and a need to improve
access to knowledge and research evidence. SCIE is now seen as central
and essential to further development of practice and policy in social care.
See the report, A Quality Strategy for Social Care (DoH 2000) which made
the case for the establishment of SCIE and which set out the rationale and
context for such a development.
40.Risks. There are several potential risks associated with a comparable
initiative in the education field. First amongst these is the risk that the key
stakeholder groups fail to see its merit. The practitioner community is not
accustomed, as a whole, to using evidence regularly to inform practice; a
demand side weakness highlighted in a 2005 OECD conference on linking
evidence to practice. The policymaking community might see an evidence
centre as unnecessary if it perceives itself as adequately served by
present arrangements. Parts of the research community might find itself
15
unwilling to cooperate fully with a government-funded centre because of its
concerns about intellectual independence and the nature of evidence.
41.Challenges. Perhaps the greatest difficulties might lie in the technical
challenges of assembling, synthesising and interpreting evidence in a
partnership structure, Clearly, to be effective the centre will need to work
between the providers and users of evidence. This would involve
collaboration with a large number of organisations with significant
differences of outlook and purpose. To produce sound and useful outputs
in this context would be a formidable management task.
42.Agents for transformation. The practical application and transformation
of subject and pedagogic research evidence needs to be embedded in
initial teacher training, in continuing professional development and in
lecturing through the Higher Education Academy’s subject centres. The
‘research champions’, mentioned later in paragraph 53(e), will play an
important part in this work. The influence may, in due course, also be felt
to the way teaching is inspected, standards set and interventions
designed..
43.Demand from practitioners. Despite the risks and challenges outlined
above, the desire for such a facility is widely felt. Amongst practitioners,
some leading teachers and teacher trainers, identifiable through such
activities as the NTRP teacher research conference, the NERF Bulletin
and local networks, are actively seeking a coherent base of evidence with
which to influence their practice. Organisations close to practitioners, such
as the Higher Education Academy, Learning and Skills Development
Agency, General Teaching Council (England) and the Training and
Development Agency for Schools are supporting this through the
development of the evidence base for their client groups.
44.Accumulation of evidence. In the policymaking area, several
organisations are calling for, and working towards, a commonly held
evidence base. Early benefits are likely to be felt simply in the growing
awareness of the research outputs of other bodies. Currently, research
evidence from the many agencies, HEIs and research councils are rarely
assembled systematically. Combining evidence from organisations in the
schools, post-16 and HE sectors and from independent social and
economic studies, on, for example, young peoples’ attitudes to HE, would
provide a richer base of evidence than at present.
45.Identification of gaps. A further impact on organisations that commission
research, would be a better sense of where research is lacking.
Weaknesses in the evidence base would be more clearly visible and
information about what research is underway or planned would be readily
available.
16
Governance
46.The NECE Board. NECE would need governance arrangements that
reflect its purpose and mode of operation. It needs to be independent, in
the intellectual sense, so its board would need to be free to form views
without regard to the particular interests of its funders1. At the same time,
to be relevant and useful it would need to work closely with government
and providers. If it is to be a partnership organisation, making use of
existing resources and working primarily through linkages with other
bodies, its board would need to reflect the diversity of such bodies, in all
sectors.
To be effective the board would nevertheless need to be as small as
possible. It should focus on strategic questions such as prioritisation,
approaches and future funding and draw on stakeholder groups and
expertise in key business areas, for members. A relevant model of
governance might be that of Teachers’ TV in which a Board of Governors’
role is to uphold editorial independence; ensure value for money; set
targets for performance and measure it against them; support strategic
development; and provide a complaints appeals process. The governance
arrangements for SCIE, for example, establish it as a company limited by
guarantee and a registered charity governed by a board of trustees, one of
whom is chair. SCIE is funded by an annual grant from the Department of
Health, Wales and Northern Ireland, governed by a service level
agreement.
Distinct arrangements will be needed for scientific, as opposed to
corporate governance. These will need to be considered in the next stage
of development.
47.The Executive. To be efficient and effective, the working group
recommends that a small executive would be needed to run the centre on
a day-to-day basis. This would oversee the work of specialists in
knowledge management, communications, ICT, research synthesis and
administration, as well that of part-time staff seconded from educational
bodies.
48.Advisory Group. The working group felt that in addition to the relatively
small executive and strategic groups, a partnership body of this kind,
straddling all sectors of education, would also benefit from a much wider
advisory group with members drawn from all sectors and stakeholder
groups. Such a group would be too unwieldy for decision-making purposes
but, with the benefit of e-communications, would provide a rich resource of
expertise and advice as required.
1
It is particularly important that NECE should be intellectually independent, as is the case with NICE
and SCIE. The early work of SCIE emphasized open processes that were independent of and
transparent to all stakeholder groups.
17
Transformation of evidence into action
49.Usability of research evidence. It is unusual for research as originally
published to be immediately useable by teachers or policymakers. There
often needs to be synthesis and then transformation of the results into key
points. Much work is already going on at organisations such as HEA, TDA,
DfES, NFER, , LSC, GTCE and at the EPPI-Centre with regard to the best
forms of summary for practitioners. The working group recommends that
developments at the centre should make use of and, where necessary,
add value to, existing prototypes. To inform and improve practice, attention
will need to be given to each node in the system: the individual teacher,
the delivery institution (nursery, school, college, training provider,
university etc), the senior team, local or national officers and ultimately, the
pupil/student themselves.
50.Caution with guidelines. From the teachers’ point of view, caution needs
to be exercised in the production of guidelines; they can be perceived as a
constraint rather than as an interpretation of evidence. Teachers are more
likely to be receptive to materials that are well synthesized and
transformed for application in the classroom/seminar room. There is also
some evidence that they may be more receptive to research if they have
some experience of it, either locally, in initial teacher education
programmes or as part of their continuing professional development. Such
readiness to engage with research exists and needs to be developed
(Rickinson 2005).
51.Role of networks. Locally-coordinated research networks operate in the
health service to interpret and disseminate research findings. In education
such a distributed system would need to involve teachers/lecturers, CPD
professionals and teacher trainers to act as champions for the use of
research evidence within their institutions.
52.Learning from experience. The working group suggests that plans for the
evidence centre take account of recent experience in the transformation of
research knowledge for use; in particular that:
a) summaries of synthesized research evidence need to be written by
representatives of specific user groups in order to ‘speak to’ their
various constituencies
b) mechanisms are needed at local or regional level whereby
translation of evidence into practice is discussed, implemented and
reviewed
c) the centre should second people from user communities for fixed
periods, to contribute to research transformation and for their own
professional development
d) existing networks (eg, GTC, LSRN, subject associations) should be
encouraged to associate with the centre to encourage the
development and use of research evidence.
e) preparedness for the use and transformation of research evidence
is developed through organisations such as the TDA as part of the
18
initial teacher training programme, and through CPD routes, so that
there is a ‘research champion’ in each school or education
institution who will promote and mediate research for practitioners.
Quality
53.Volume constraints. A core function of the proposed centre will be to
manage the transformation of a rapidly growing body of research evidence
for practice and policy. There will, however, be limits to the quantity of
information that can be handled. At the input end, the resources available
for processing it will act as a limit; at the output end the pressure to select
evidence, relevant and sound enough to recommend to users, will do the
same.
54.Quality assurance procedures. Evidence is likely to be made available to
users both through an on-line search facility and through the materials
produced from syntheses. The working group suggests that distinct quality
procedures will be needed for each of these processes.
55.A staged approach. The search facility currently being developed in
NERF’s Evidence Portal project (NEEP) is expected to simply provide
information about the quality assurance process through which each
document has already passed, prior to publication, rather than offer an
additional assessment of quality. The working group suggests that initially,
the centre might adopt this expedient. At a later stage, procedures
developed at the Social Care Institute of Excellence (SCIE), with its on-line
search facility, Social Care Online, might be studied to assess their
suitability for education. The issue of differentiation of material for different
constituencies of user will have to be addressed.
56.Quality of output. For the second type of output, the resources to be
published by the centre, criteria and processes will need to be developed
for the selection of evidence that is both scientifically robust and of
practical value to specific kinds of user. Processes and formats for the
production of high quality published materials will also need to be
developed. Again the working group suggests that the centre should draw
upon the experience of SCIE, where guidelines have been developed for
the synthesis of evidence and the writing and design of resource packs.
57.Categories of material. In general, procedures will need to be developed
for characterising material so that readers can find their way to evidence
with the specific qualities they require. The quality framework for applied
and practice-orientated research developed by Furlong and Oancea (2005)
for ESRC may provide a starting point for this. It proposes four dimensions
of quality each with a number of distinctive attributes (see appendix V).
The dimensions (plus one illustrative attribute) are:
a) epistemic (eg trustworthiness)
b) technological (eg purposivity)
c) value for people (eg plausibility) and
d) economic (eg feasibility).
19
58.A framework for practice-oriented research. The working group
suggests that this framework be used as a starting point for developing
quality marks to be associated with items of evidence from practiceoriented research.
Conclusion and recommendations
59.A radical and timely idea. This report outlines what is a radical and multidimensional idea. It is intended to assist NERF and DfES in developing
their proposals. Members from the social care and healthcare fields were
particularly struck by the opportunity presented in education to build upon
the experience of existing initiatives in other sectors.
60.The need for action. The working group believes that many challenges
would best be addressed by starting to “get on with the job”. It is for this
reason that it recommends that the beginnings of a centre be established
soon and that stages of development and piloting be used to resolve
technical issues as they arise. The group hopes that this report will assist
in the establishment of an embryonic centre and in the dialogues that will
be necessary to ensure it has support from, and learns from, all interested
parties.
61.The working group therefore recommends that NERF and DfES:
a) take forward the ideas set out in this report and begin to
establish a National Evidence Centre for Education
b) engage potential partners and stakeholders in a dialogue in
order to design the development of the centre.
20
References
CERI (1995) Educational Research and Development: Trends, issues and
challenges. Paris: CERI
Demos (2005) About Learning: Report of the Learning Working Group.
London: Demos
DoH (2000) A Quality Strategy for Social Care. London: Department of
Health, August 2000
Eraut, M. (2004) Factors influencing the transfer of good practice. Paper given
at Department of Education and Skills Research Conference, 19th November
2004. London: DfES (Research report CR2004)
ESRC symposium on the non-academic impact of social science: see
www.esrc.ac.uk/ESRCInfoCentre
Furlong, J. and Oancea, A. (2005a) Assessing Quality in Applied and
Practice-Based Educational Research: a framework for discussion. Oxford:
Department of Educational Studies.
Hillage, J., Pearson, R., Anderson, A. and Tamkin, P. (1998) Excellence in
Research on Schools. London: DfES (‘The Hillage Report’)
Furlong, J. and Oancea, A. (2005b) Assessing Quality in Applied and
Practice-Based Educational Research (ESRC Research Briefing). Swindon:
Economic and Social Research Council.
Millar, R. (2005) Is evidence-based medicine a useful model for educational
research and practice?. Paper prepared for staff seminar, Department of
Educational Studies, University of York, 22 June 2005.
NERF (2003) see Peckham, M. and Morris, A. NERF Strategic Proposals.
NERF Working Paper no 3.1, available on www.nerf-uk.org
NICE (2005) Consultation paper: Operating Model for Centre for Public Health
Excellence. London: National Institute for Clinical Excellence.
Nutley, S. (2004) Delivering research-informed practice: lessons from the
crime reduction and social care fields. Keynote address, Department for
Education and Skills Research Conference, 19th November 2004. London:
DfES (Research report CR2004) and see also www.standrews.ac.uk/%7ruru/publications.htm
Rickinson, M. (2005) Practitioners’ Use of Research: A Research Review for
the National Evidence for Education Portal (NEEP) Development Group.
NERF working paper 7.5 (Dec 2005)
SCIE (2003) Interim Guidelines for SCIE Systematic Knowledge Reviews.
London: Social Care Institute for Excellence.
SCIE ( 2005) website: www.scie.org.uk
21
Appendix I
Policy background
There are many views about the efficacy and viability of a national evidence
centre. Sceptics suggest that teachers don’t demand evidence, academics
dispute its very nature whilst politicians reject any that is contrary! NERF,
however, has found the idea welcomed by representatives of all three groups:
teachers, researchers, policymakers. Each has an interest in making research
evidence more widely available in a useable form.
Extensive use of evidence in healthcare began taking off about 1990 following
recommendations of a House of Lords select committee. The National
Institute of Clinical Excellence was launched, funded by the Department of
Health. Subsequent moves towards a comparable body for social care led to
the establishment of the Social Care Institute of Excellence in 2001, funded by
the same department.
In education, the most relevant move was the report Excellence in Research
on Schools (the “Hillage” report) published in 1998. This identified as a
fundamental weakness the “lack of people and processes to help distil and/or
interpret research findings for a practitioner and/or policymaker audience”. It
recommended, amongst many other things, the “establishment of an
education information unit to coordinate and support the collation of
educational research and ….a strategy to ensure that different users have
access to the information they need in a form they can use”. A further
development has been the creation in 2002 of the What Works Clearing
House in the USA which provides such information and interpretation
services. It has generated considerable debate by only admitting as evidence
“studies that provide the strongest evidence of effects: primarily well
conducted randomized controlled trials and regression discontinuity studies,
and secondarily quasi-experimental studies of especially strong design”.
NERF reached the conclusion in 2004 that of all the many improvements
needed in the educational R&D system, the establishment of an evidence
centre or system at national level, as suggested in the Hillage report, was of
paramount importance. At that time there was no specific description of its
nature or scope; these remaining for NERF, its working group and others to
consider.
More recently, the Demos report (2005), About Learning, argues for closer
links between the evidence from science and the evidence from professional
practice. It suggests that understanding the nature and practices of learning,
rather than of teaching, is at the heart of the problem; and that there needs to
be “a rethinking of how development and research into learning are
organised” (p15). The report identifies an inadequate knowledge base about
learning for the teaching profession; confusion about the evidence base for
new practices; and a lack of coordination between development and research.
22
Appendix II
Recent research background
There has been considerable interest recently in research into the relationship
between evidence and practice. Two bodies of research are presented here.
Nutley (2004), for example, aims to answer two questions: Why does it take
practice so long to adopt best evidence? And what can researchers do to
increase research impact?
Deriving from work in social care, she and colleagues have created a typology
of kinds of research utilisation, with categories of critical practitioner,
embedded and institutional use of evidence. These kinds of research
utilisation are helpful when considering what best suits the education field. In
the first model, the research-based practitioner has personal responsibility for
keeping up to date with research evidence. This model “assumes that
individuals have professional autonomy – that they can change their practice
in line with research evidence” (2004, p14). There is a strong emphasis on
initial professional education and on continuing professional development in
this model, with appropriate infrastructure in place to allow easy access to
research evidence. The infrastructure ensures that the research exists, that
there are good systems for accessing it, appraising it and applying it to
everyday practice.
In the embedded research model, research is embedded in the systems and
processes within which practitioners work. Individual practitioners may remain
unaware of the research that is guiding their practice. The responsibility for
research translation lies with policy-makers and managers at national and
regional/local levels. Research use “is still viewed as a linear process: the
research exists, it is converted into systems and tools, and then applied” (ibid,
p14).
In the third model, “responsibility for research use lies with local service
delivery organisations” (ibid, p15) and thus there is a strong emphasis on
leadership at these levels. Research use is seen as “an ongoing learning
process” (ibid) with local commissioning and adaptation. Local organisations
are much more in the driving seat of the research agenda, working often with
higher education institutions to determine and carry out the research. The key
to such an organisational excellence model is partnership.
No one model is intrinsically better than any other, and circumstances will
dictate which is most appropriate. It is also the case that a holistic
combination of the three models is the best way forward when considering
how such models might apply to the education field: “activities underpinned by
the three models of research use are likely to be required at different times
and in different places” (ibid, p18).
Eraut’s research (2004) focuses on the challenges of the transfer of good
practice in education from the standpoint of the receiver; the nature of
23
practice and its reception; and the challenges for the originating institution.
In particular, the nature of ‘practice’ is problematised, with the suggestion
that it is a holistic concept, honed over time and operating at different
levels of consciousness.
The conclusions arising from Eraut’s research suggest that there are four sets
of contextual conditions that are usually required for the successful transfer of
practice: good trust relationships; respect for the personal and professional
identities of teachers, and recognition of the intrusiveness and demanding
nature of the transfer process; recognition of teachers’ preparedness to risk
their time and reputation; and an understanding of the need for learner
engagement and creating the time to construct new practices. These will
apply as much to the translation of research evidence into practice as to the
transfer of ‘good’ or ‘best’ practice from one teacher to another.
24
Appendix III
Existing evidence centres
The working group has taken note of the work of existing evidence centres in
the UK.
The Social Care Institute of Excellence (SCIE) was established in 2000 by the
Department of Health. SCIE works with people and organisations throughout
the social care sector in children’s and adult’ services to identify useful
information, research and examples of good practice. Using this information,
SCIE produces resources which evaluate knowledge in a particular area of
social care, draw out key messages for good practice and identify areas
where more research is needed to inform good practice. SCIE also hosts
Social Care Online – the UK’s most complete range of information and
research on all aspects of social care. Practitioners, researchers and service
users rely on SCIE’s resources as a central and trusted point for knowledgebased good practice guidance. Its audience is primarily individual practitioners
in social care and its aim is to improve practice by providing the confidence
that comes from knowing that actions are based on sound evidence. It found
that gaining active support at junior minister level proved important in getting
the Institute off the ground.
The NHS Centre for Research and Dissemination (CRD) and the National
Institute of Clinical Excellence (NICE) were established by the Department of
Health in 1990s. The context for NICE differs significantly from that in social
care or education, not least in that there is a statutory duty on healthcare
providers to take note of NICE guidelines. There is therefore a greater
external incentive for practitioners to use the evidence distilled by NICE.
Methodologically, NICE and NHS CRD operate with a more rigid hierarchy of
evidence than SCIE. It is likely that because of the nature of the field, a centre
for education would need to draw on evidence of many different kinds.
In the field of Education, the Evidence for Policy and Practice Information and
Coordinating Centre (EPPI-Centre) was set up by the DfES in 2000 and has
commissioned a number of systematic research reviews, both on behalf of
DfES and in collaboration with the Training and Development Agency for
Schools (TDA) and other bodies. Its function is largely methodological, but it
also undertakes training for and monitoring of systematic reviews.
25
Appendix IV
Examples of output
There are many kinds of output that NECE could produce, including rapid
research reviews for policy-makers, teacher materials, sets of guidelines,
ajournal of record and downloadable information from a website .
Rapid reviews are the subject of discussion across government and would be
brief reports on topical issues completed by expert groups over a matter of
months. The process to be used remains to be developed.
Guidance, advice and materials for teachers are currently produced by many
organisations. NECE would need to work with these to determine how best to
create useful additional publications. These might set out clear suggestions
for practice, where the evidence is strong, or act as a guide to different
approaches where the evidence is divided.
A journal of record, comparable to The Lancet in medicine, has also been
suggested which could be produced through the proposed evidence centre.
Initially, it could appear (say) twice-yearly, in order to ensure quality and wide
distribution. Subsequently it could appear more frequently. It could act as an
archive for authoritative and highly significant research in the field of
Education and would probably include an editorial, systematic research
reviews, articles containing key primary research, a methodological section, a
section where debates on the nature and use of evidence can take place,
book reviews and other sections as appropriate. It would appear in hard copy
and online formats.
26
Appendix V
A framework for quality in applied and practiceorientated research
In 2005, ESRC commissioned Professor John Furlong to analyse various
concepts of quality in research that was applied or orientated to practice and
to develop some kind of tool to assist in assessing it. The study carried out by
himself and Alis Oancea, resulted in a framework comprising four dimensions
of quality and a number of attributes in each. These have been widely
welcomed and are being actively used in preparation for the 2008 Research
Assessment Exercise. Because it may be of assistance in the development of
an evidence centre, the framework is reproduced below.
The dimensions and sub-dimensions of applied and practice-based research quality proposed
here can be summarised as follows:
Quality sub-dimensions
Epistemic:
methodological and
theoretical robustness
Dimensions of quality
Technological
Capacity development
and value for people
Economic
Trustworthiness
Purposivity
Plausibility
Builds on what is
known +contribution to
knowledge
Salience/ timeliness
Partnership,
collaboration and
engagement
Cost-effectiveness
Explicitness
Specificity and
accessibility
Reflexivity, deliberation
and criticism
Auditability
Propriety
Concern for enabling
impact
Receptiveness
Feasibility
Paradigm-dependent
criteria
Flexibility and
operationalisability
Transformation and
personal growth
Originality
Scientific robustness
Social & economic robustness
27
Marketability and
competitiveness
Appendix VI
Contact with Scotland, Wales and Northern Ireland
In the time available to the group, contact has been made with a limited
number of the organisations in these countries: two in Scotland and one each
in Wales and Northern Ireland.
Both the Scottish Executive Education Department and the Scottish Council
for Research in Education Centre recognise the need for initiatives that
facilitate knowledge transfer and exchange into policy and practice, and
broadly welcome the proposal for a National Evidence Centre for Education.
Diane Wilkinson, Senior Researcher at the Scottish Executive Education
Department, is familiar with the SCIE model (and other social work and social
care initiatives) “and [is] impressed by its multi-dimensional approach”
[correspondence, 29.9.05]. She comments that she would want this type of
initiative extended to Scotland, given the differences in curricula, as there are
commonalities that would enable Scottish practitioners to benefit from
‘English’ evidence (and vice-versa).
The Department for Training and Education (DfTE) in the Welsh Assembly
agrees that there is a need for a central research/evidence team in the UK
and supports the idea of NECE in principle. Scotland and Wales have
expressed the need for such a centre to recognise and reflect the specific
needs of each country, and for the initiative not to be England-biased.
The Northern Irish Central Survey Unit at NISRA referred the proposal to the
Department of Education and the Department of Employment and Learning.
The Analytical Services Group has set its own research agenda that would be
of interest to NECE, and urges that vocational and training issues are
considered along with school and higher education issues.
Concerns about the difficulties on reflecting the diversity found within the UK
are real. Paul Brna, Director of SCRE, comments “Of the many models of how
such a centre could be set up in ways that respect this diversity, which ones
both preserve the independence of the different devolved areas of the UK and
provide mechanisms for cooperation?” There are substantial issues with
regard to the transformation of research into practice and policy, not least the
difficulty of extracting “evidence from research which is rooted in different
socio-cultural contexts” [correspondence of 16.10.05]. He suggests the
establishment of an independent body to scrutinise the work of the Centre on
a regular basis, ensuring that the diverse needs of the UK as a whole are
represented.
Further consultation with the other jurisdictions will be required if the centre is
to be truly ‘national’.
28
Appendix VII
Diagram of NECE
Inputs
Programmes
and centres
Articles and
reports
Reviews
Databases
Quality of
input
Quality
assurance
process
National
Evidence
Centre for
Education
Search,
retrieval
and
transformation
processes
Quality of
output
Publications
Interactive web
portal (NEEP)
Advice for
policymakers
Guidance and
tools for
practitioners
Outputs
Existing resources
Many of the elements above are already in existence or under development. A few
indicative examples are given here:
Programmes/centres: TLRP, WBL, CEE, NRDC and many others
Reviews: EPPI centre, BERA, ESRC Evidence network and others
Reports: From the many organisations in the public, voluntary and private sectors
Databases: British Education Index, ESRC society today, DfES research
Web portal: National Evidence for Education Portal (NEEP)
Publications, guidance and tools: through many organisations in all sectors
29