Working Paper number 5.2
Modelling D&R
programmes: initial
exploration of features
Andrew Morris
October 2004
1
Modelling D&R programmes
Phase 1: initial exploration of features
A. Summary
1.
Background
NERF has put forward two strategic proposals to increase the coherence and
effectiveness of educational research:

creation of a national evidence system accessible to all

piloting of new forms of Development & Research (D&R) programme
A modelling exercise is being carried out to refine what the nature of the latter
might be. Key features initially identified for such programmes are that they:
1.1.
address priority problems in educational practice
1.2.
combine practitioners, such as teachers and managers, and/or policy
officials with researchers in setting the research agenda
1.3.
draw upon a range of academic disciplines, as required by the problem,
across and beyond the social sciences
1.4.
operate through iterations of closely connected developmental and
research activity
1.5.
provide a range of outputs capable of both improving practice and
contributing to theory
Preliminary discussions have been held with a number of people currently
engaged in development or research with one or more of these features. Key
points arising from these are set out here.
2. Priorities
2.1. There is generally weak conceptualisation of the way in which research
priorities are established, though sources can be cited to explain choices
2.2. Priorities exist at many levels – from broad intentions at government level,
through local and institutional objectives to individual classroom
imperatives.
2.3. The effort expended in establishing priorities needs to be proportionate to the
scale of the research envisaged.
2.4. Discussion about research options often takes place within the general
decision-making processes of organisations and many have methods for
establishing priorities.
2
2.5. Priorities need to “keep abreast of changing policy” and may need to alter
2.6. Space also needs to be allowed for projects that develop ideas rather than
solutions
3. Practitioners
3.1. There is widespread experience of engaging practitioners with research,
throughout the branches of education
3.2. A variety of roles for practitioners can be identified in relation to the
different stages of the research process. These include involvement in
3.2.1. initiating or shaping research questions.
3.2.2. designing projects
3.2.3. gathering data and interpreting findings.
3.2.4. enhance the impact of research on practice
3.3. Distinct modes of practitioner engagement can be identified, including:
3.3.1. top-down: where an external body runs a project in which teachers are
the key resource for ideas and testing
3.3.2. support-for-teachers: where whole schools or colleges are engaged in
implementing and studying a change
3.3.3. networks: in which practitioners share knowledge and experiences
outside their home institution
3.3.4. small-scale projects: to investigate issues, develop products or enhance
understanding
3.3.5. benchmarking: in which practitioners submit local data about students
to a central analytical team and receive back local and national
findings from the aggregate data.
3.3.6. teacher workshops: in which teachers discuss their methods and
organise collective investigations and test hypotheses.
3.3.7. integrated R&D: in which research is integrated into local development
activity through interviews, workshops and electronic communications.
3.4. A variety of distinct approaches, both theoretical and practical, have been
identified for practitioner engagement:
3.4.1. An overarching enquiry-based approach can be taken to small scale
projects within a developmental scheme
3.4.2. Teaching and learning units in colleges and schools explore practical
pedagogic issues from which research questions or small scale projects
emerge.
3.4.3. Broadly-based partnerships between universities and schools or
colleges can be designed to include a research strand
3.4.4. Research engagement can be considered as an act of learning, in which
participants identify their existing conceptualisations and develop from
there
3
3.5. Strengths and weaknesses of some mechanisms used in engaging
practitioners have been identified:
3.5.1. to release teachers from classrooms, part-time posts or secondments are
more effective than occasional cover
3.5.2. sophisticated e-communications enable different sets of stakeholder
(parents, teachers, officials etc) to discuss issues across institutions and
sectors
3.5.3. communication activities, such as seminars, newsletters and websites
reach practitioners, though to be effective funding is needed for
communications expertise
3.5.4. advisory panels can be used to involve practitioners but it may prove
difficult to sustain regular attendance.
4. Multi-disciplinary working
4.1. knowledge, tools and theories from a wide range of discipline are often
needed to adequately address problems of practice
4.2. Contextual problems and specific obstacles to multi-disciplinary working
include:
4.2.1. factors within academic research that militate against it, such as:

the range of skills and knowledge represented in RAE panels

the prestige attached to single-discipline journals

funding and publication incentives that favour single discipline
career development

reluctance among academics to work in collaborative teams.
4.2.2. antagonism between particular disciplines because of methods, such as
field trials, associated with them.
4.2.3. weak scientific and mathematical backgrounds amongst some
professionals which may affect capacity to engage with quantitative
colleagues
4.2.4. incompatible world views making collaboration ineffective
4.2.5. funding that switches on and off too rapidly to foster the build up of
trust over long periods of time
4.2.6. The machinery of open competitive tendering working against the
gradual build-up of collaborative partnerships
4.3. Several approaches to overcoming these obstacles are identified, including:
4.3.1. arrangements within universities that enable academics to teach in a
department but research in an inter-departmental research centre.
4.3.2. situations where subject disciplines are perceived as less significant
than development of relevant competences across a team
4
4.3.3. Holistic approaches incorporating research within broader strategic
partnerships
4.4. Multidisciplinary development may need to be conceptualised as much as
multidisciplinary research
5. D&R methods
5.1. The two arms – development and research – need to interact continuously.
An agreed theoretical framework for this can be helpful
5.2. methods for combining research and development need to be explored. The
“design study” is a possible candidate.
5.3. Development projects can attract significant funding but may be weaker
when not linked to research.
5.4. Research elements can be incorporated into local development projects; one
college unit doing this has a £1million turnover
5.5. sustained centres of excellence, rather than one-off projects, may be needed
to illuminate the process of D&R as well as the substantive topic.
5.6. The D&R concept can be applied to trials and evaluations
5.7. The benefits found in preliminary studies may not obtain when scaled-up for
mainstream use:
5.7.1. Consideration needs to be given to the reasons why practitioners might
be expected to change
5.7.2. stable intermediate states may need to be established before a preferred
state is reached
5.7.3. Trailing-edge as well as leading-edge practice needs to be studied
5.8. The merits of particular methodological choices may be perceived differently
by practitioners and academics.
5.9. Examples of ways in which D and R are being combined include:
5.9.1. exploiting data about student performance, demographics, attitudes and
satisfaction to inform decision-making
5.9.2. Local partnerships involving practitioners with employers, academics
and community groups
5.9.3. Staff groups focussing on evidence about teaching and learning
5.9.4. teachers and teacher-researchers identifying features of good practice
from which research questions are expected to emerge and materials be
produced.
5.10. Major challenges for combining D and R include:

Difficulties in securing funding simultaneously from both R and D
sources

Allowing sufficiently long timescales for the work to take effect.
5
6. Management
6.1. The way in which multi-party programmes are managed is critical to success.
6.2. Project management is an area in which practice and policy leaders may have
special expertise
6.3. Sophisticated email (and possibly bulletin board) communications as well as
meetings are needed to hold together complex R&D programmes.
6.4. Regular, structured newsletters are also important and can increase impact by
cascading through linked networks
6.5. Accountability arrangements present a fundamental challenge to cross-sector
working because traditions, cultures and reward systems differ profoundly
between the communities.
6.6. D&R programmes have to contend with these tensions. Moving forward
involves gradual change, empathy, long-term vision and a sense of urgency.
6
B. Main Report
1.
Background
NERF has put forward two strategic proposals to increase the coherence and
effectiveness of educational research. They are for the creation of a national
evidence system accessible to all, and for piloting of new forms of
Development & Research (D&R) programme (NERF 2003). A modelling
exercise is being carried out to refine what the nature of the latter might be.
Key features initially identified for such programmes are that they:
1.1.
address priority issues in educational practice
1.2.
combine practitioners, such as teachers and managers, and/or policy
officials with researchers in setting the research agenda
1.3.
draw upon a range of academic disciplines, as required by the problem,
across and beyond the social sciences
1.4.
operate through iterations of closely connected developmental and
research activity
1.5.
provide a range of outputs capable of both improving practice and
contributing to theory
To begin the modelling, one-to-one discussions have been held with a number
of people currently engaged in development or research. They have been
asked about difficulties experienced in trying to work in any of the ways listed
above and about any benefits and opportunities in doing so. A list of those
involved appears in appendix 1.
2. Priorities
2.1. All those involved in the discussions agreed that research is needed that
addresses priority issues in practice and were primarily engaged with such
work. Examples included:

Improving learning opportunities through the children’s fund

Understanding the links between learning and health and social
benefits

Improving literacy and numeracy amongst adults

Using ICT effectively in the classroom
2.2. No specific ideas were put forward about the methods by which specific
priorities are arrived at, but a number of sources were identified for the actual
topics chosen:

Key reports on the need for improvements
7

Government reports and Acts of Parliament

Government evaluation and research contracts

Priorities within schools, colleges or adult learning centres

Priority lists established by consultations within organisations

Ideas and outcomes emerging from previous research

The outcomes of preliminary feasibility studies
2.3. Some general thoughts also emerged. Firstly, priorities exist at many levels –
from broad intentions at government level, through institutional objectives to
individual classroom imperatives. The effort expended in establishing
priorities has to bear some relation to the scale of the research envisaged. On
the whole there is little evidence of organisations attempting to
systematically rank and select from competing priorities. At most there is
discussion about alternatives within the general decision-making processes of
organisations.
2.4. Secondly, priorities need to “keep abreast of changing policy”. Longer term
projects may well have to alter course as the original rationale for them
changes. Thirdly, some projects are needed that develop ideas rather than
produce recommendations or tools - “going fishing” is also important.
2.5. A range of methods were described for establishing priorities locally. In
some larger organisations internal consultations take place across
departments. Others hold expert seminars involving a mixture of external and
internal participants. On-the-ground priorities in one scheme arise from
structured e-communications between different practitioner groups and the
centre.
3. Involving practitioners
There is widespread experience of activity that engages practitioners with research
in primary and secondary schools, colleges, adult learning centres and related
health and social services. The discussions revealed many examples and
demonstrated the great variety of modes as well as success factors and pitfalls. The
following analysis draws out themes of relevance to the D&R concept.
3.1. Roles
3.1.1. Some colleges have developed a variety of roles for practitioners in
relation to the different stages of the research process; for example:
fieldworker, analyst, interpreter, project designer, project manager,
mentor, subject.
3.1.2. A role identified in several discussions was the initiator or shaper of
research questions. To this task they can bring experience of the
authentic problems of learners and some views about solutions that
work. However, as one interviewee pointed out, “not everything of
importance to practice will be obvious to the practitioner”.
3.1.3. In some cases practitioners are involved in the design of projects and in
data gathering.
8
3.1.4. Another important aspect of engagement is the potential for
practitioners to enhance the impact that research evidence ultimately
has on practice –either through directly influencing its quality or by
championing its outcomes.
3.2. Modes
3.2.1. Two modes are identified by an academic with long experience of
working with schoolteachers: top-down and support-for-teachers. In
the first, an external body, such as an examination board, runs a project
in which teachers are the key source of ideas and carry out some testing
of novel approaches. In the second, whole schools or colleges are
engaged in implementing and studying a change (eg introducing IT). A
key to success is to ensure that multiple levels of the organisation are
engaged from the outset because they will all need to change ultimately
– leadership, cross institutional coordinators, budget-holders and
classroom staff.
3.2.2. Intermediary organisations organise many networks in which
practitioners share knowledge and experiences outside their home
institution, Small scale projects engage practitioners in projects. These
may be to investigate an issue, develop a product or process or simply
enhance the understanding and skill of participants.
3.2.3. In one university centre, practitioners in thousands of schools and
colleges are involved in submitting local data (statistical and survey
responses) about students to a central processing team which then
produces statistical information with local detail and benchmark
averages. From this the central team is also able to produce nationally
important findings from the aggregate data. The resulting network has
been used to create teacher workshops in which teachers discuss their
methods and organise collective investigations. Some of these involve
whole schools in setting up and testing hypotheses.
3.2.4. In a cross –sector evaluation project on children 5 – 13 years old, a
thorough strategy for maintaining the engagement of practitioners was
implemented. The research was integrated into local development
activity and involved visits to sites, intensive interviews, development
workshops and electronic communications.
3.3. Mechanisms
Various means have been developed to facilitate engagement of practitioners
with research. Some examples are given here.
3.3.1. Releasing teachers from their front-line duties is a central problem.
Using cover teachers for occasional periods can be disruptive. More
effective ways include:

Funding a substitute part-time teacher

Funding a part-time secondment

Enabling leave of absence

Using deputy heads or others with some non-contact time
9
3.3.2. Several methods for engaging practitioners in research communications
have been tried. One research centre organises conferences, seminars,
newsletters and a website open to practitioners. It also has four
advisory panels which draw in practitioners and policymakers in equal
proportion to academics. The limitations are that practitioners are often
unable to attend and when they do their role tends to be to comment on
propositions put forward by researchers. The inadequate level of
funding for communications activities and the expertise to support
them is an underlying difficulty. In another centre, practitioners were
brought together in a group without academic researchers present to
comment on draft materials prepared by the research team.
3.3.3. In a third example, sophisticated e-communications were installed at
the outset to enable different sets of stakeholder in a development and
evaluation project (parents, teachers, social workers, officials etc) to
collaborate. Web discussion took place across the institutions and
sectors and email groups containing different selections of stakeholders
enabled targeted messages and responses to be communicated.
3.4. Approaches
3.4.1. As well as distinct mechanisms, a variety of approaches are taken to
engaging practitioners in research. One is to perceive the activity as a
learning experience itself. Practitioners can engage in activities to
identify their existing conceptualisations of an issue and then develop
them in the light of particular research findings.
3.4.2. A different approach is taken in a development scheme linking
community and private sector professionals with practitioners in
schools. It has an overarching enquiry-based approach and places a
requirement on each local scheme to build-in a research project.
3.4.3. Other approaches have been adopted in colleges and schools. One
college has organised a teaching and learning team comprising its
advanced practitioners. This is exploring practical pedagogic issues
form which important research questions are emerging. Another
college runs a Learning Development Unit to which teachers can turn
for advice on their practical problems, which frequently lead on to
small scale research investigations. Positive inspection comments have
alerted the college leadership to the “bottom-line” importance of this
unit. A secondary school is adopting a whole-school approach
involving teachers, leaders, support staff and governors. One senior
member of staff leads the activity and a researcher-in-residence has
been seconded from a local university.
3.4.4. Cautions
A number of specific cautions were expressed about teachers’ engagement
in research:

the contributions of enthusiasts need to be distinguished from
those of the novice or less competent.

Teachers often to want to adapt new ideas or materials too early
and may unconsciously filter out uncomfortable ideas
10

Practitioner engagement can lead to “re-goaling”. It is important to
maintain the tension between teacher and researcher

In the words of one interviewee, sometimes practitioner
“expectations of research have to be managed”

In mixed settings practitioners may tend to be less vocal than
academic researchers

Practitioner views expressed by those who have been away from
the classroom for some time can be misleading.

Sometimes practitioner contributions can be deferential or
stereotypical – effectively subtracting from rather than adding to
understanding
4. Multi-disciplinary working
4.1. The central concept, that to address problems of practice effectively the
resources of many disciplines are required, was accepted in each of the
discussions. As well as the social sciences and education, disciplines such as
architecture, information technology, healthcare, social care, management,
business studies and law might have important contributions to make as well
as curriculum subject areas themselves –in the arts, humanities, engineering,
medicine and so on. Disciplines offer theories and analytical tools as well as
bodies of knowledge.
4.2. However, those with experience of multi-disciplinary working pointed out
some of the practical difficulties involved. Within the academic community
the HEFCE Research Assessment Exercise was described in one discussion
as unhelpful to such working. The fear was that assessment panels may lack
the wide range of skills and knowledge required for research that draws on a
wide range of disciplines. More generally, funding and journal publication
were perceived as favouring single discipline working. The incentives that
shape research careers do not encourage multi-disciplinary working. Perhaps
as a consequence of this, there may be a reluctance among UK academics to
work in collaborative teams.
4.3. Particular methodologies may be associated with particular disciplines.
Divided views about such methodologies, for example field trials, or
grounded approaches, may discourage multi-disciplinary working. In one
discussion it was suggested that for effective working, the “world views” of
members of a team need to be compatible
4.4. In most discussions combinations of qualitative and quantitative methods
were seen as important. In one case it was suggested that a weak scientific
and mathematical background amongst some professionals (including the
speaker himself) might affect the capacity to engage effectively with
quantitative colleagues.
4.5. Building an effective team, particularly with unfamiliar colleagues from a
wide range of disciplines, calls for high levels of trust and mutual respect.
This takes time to develop and is jeopardised by short-term funding. The
rapid switching on and off of research teams, especially as a result of
11
competitive tendering, works against the gradual build-up of collaborative
partnerships
4.6. Nonetheless, several approaches have been tried to overcome these
difficulties. In one discussion it was observed that some US universities are
organised so that academics teach in a subject-based department but research
in an inter-departmental research centre. In another discussion, an example
was given in an English university of a problem-focussed research unit that
seconds-in academics from departments to work together according to the
needs of each particular project.
4.7. In the case of one multi-disciplinary centre, the differences according to
subject discipline were seen as, and treated as, less significant than those of
competences across the whole team. The development of skills and
understanding across the team were more of an issue than working across
disciplines. In another centre, the question of combining disciplines, like that
of combining of different kinds of participant and end-user is addressed
within a broader strategic partnership that involves all of these groups. ,
4.8. One discussion led to the concept of a multidisciplinary approach to
development as well as research. In a well balanced D and R programme
perhaps this would need to be conceptualised and acted upon.
5. D&R methods
5.1. In each of the discussions the combining of developmental with research was
seen as important. Just as there is a huge variety of definitions of and
approaches to research, so there is with development. For some centres it
involved development of materials for teaching and learning, for others
development of actual practice in classrooms, for others development of
understanding amongst the many groups involved in changing practice in the
light of research evidence. In all cases, the two arms – development and
research – need to interact continuously. Development activity can throw up
questions that need investigation through research. Equally the outcomes of
research may call for developmental activity. Evaluation of development
activity may help to shape and improve it, if the two are intimately linked. In
one programme a clear theoretical framework was central to the D and R
work and helped develop coherence across the various parties.
5.2. Particular methods for combining research and development need to be
explored. The “design study” approach developing particularly in the USA
attempts to provide a framework for both improving practice in specific
classrooms and developing theoretical understanding that could be more
generally applied.
5.3. Nationally, development is a larger-scale activity than research and is more
substantially funded. Research can be envisaged as a crucial element of
effective development projects and incorporated as a strand in bids. One
college-based unit uses this approach and currently has a £1million turnover.
5.4. The discussions demonstrated many ways in which D and R are being
combined.
12
5.4.1. In one centre, where data from schools and colleges are aggregated and
analysed, the results are used locally to inform institutional
development.
5.4.2. In one scheme, formal evaluation of an initiative is combined with
development of thinking and tools locally. Local partnerships
involving practitioners, academics, service leaders and community
groups engage with research findings continuously. This helps both by
providing practitioners with detailed evaluation information about
effectiveness and by involving those on the ground in the conceptual
development.
5.4.3. In a number of schools, colleges and adult learning centres, groups of
staff are brought together to identify features of good practice and to
develop useful research questions from them. These are used to
influence the research agenda in parallel with the views of
policymakers and academic researchers.
5.4.4. In some centres practitioners are also involved in shaping and
promoting the use of the materials produced as result of research.
5.5. A number of challenges to combined D&R working were thrown up in
discussions.
5.5.1. One major issue is how to ensure that benefits found in preliminary
studies still obtain when scaled-up for mainstream use. One centre
suggested that thought needs to be given to the reasons why
practitioners should be expected to change in the light of research
evidence. There may well be legitimate, pragmatic reasons for them
not to do so. One way suggested for moving towards a desired change
is to avoid aiming for the desired final state in one leap, but to establish
stable intermediate states beforehand. This can be likened to
environmental change, such as reforestation, where intermediate states
of ecological balance may be reached, from which further development
becomes possible. To ensure that pilot studies are capable of being
scaled up, it can be useful to include examples of “trailing-edge” as
well as leading-edge practice in D&R programmes.
5.5.2. A second issue is that the merits of particular methodological choices
may be perceived differently by practitioners and academics. To
reconcile perspectives it may be that sustained centres of excellence,
rather than one-off projects, are needed to illuminate the processes of
D&R as well as the substantive topic.
5.5.3. A challenge of a practical, but fundamental kind is to secure funding
simultaneously from both research and development sources.
Organisations responsible for each are quite distinct and the outlooks
of their respective budget-holders may differ significantly. Timescale is
a key problem. Holders of development budgets need to appreciate the
timescale required for rigorous research and their opposite numbers in
research need to understand the sense of urgency facing people
working in direct contact with current daily practice and its
deficiencies.
13
6. Management
6.1. In each of the discussions it was clear that the way in which complex
programmes or centres are managed is critical to their success. The
provenance of leaders varies – some have academic backgrounds, some
practice-based. The responsibilities of project management may be usefully
distinguished from those of intellectual leadership of the substantive topic
itself.
6.2. Sophisticated communications were widely held to be essential to the
effective management of complex multi-party programmes or centres. Email, with clearly differentiated sub-groups, is an important means. But
equally, face-to-face business meetings, seminars and workshops are each
needed to hold together and progress programmes. Methods of
communicating with interested parties beyond the programme or centre are
also seen as important. Regular, structured newsletters, for example, can
increase impact by cascading through linked networks
6.3. The question of accountability presents a fundamental challenge to crosssector working. A sense of accountability to the peer community and to the
development of the body of knowledge in disciplines is central to the
academic role. The reward systems of universities and the HEFCE funding
exercise embody this. Accountability to colleges and schools, and ultimately
to the national bodies that fund them, affect the behaviour of developers and
practice-based researchers. D&R programmes have to contend with these
tensions. However, the discussions reported here revealed a widespread and
strong sense of accountability, shared by all, to the community of learners for
whose benefit the entire D&R enterprise is ultimately mounted. Moving
forward will involve gradual changes, boundless empathy with the positions
of others, blended with patience about the long-term vision and urgency
about the need for change.
14
C. Appendix
People involved in discussions
The discussions upon which this report is based have been held with the following.
Person
Role
Organisation
Jim Ridgway &
NI colleague
Professsor of Education Durham University
John Vorhaus
Associate Director of
Research
National R&D Centre for
Adult Literacy and Numeracy
Leon Feinstein
Director of Research
Wider Benefits of Learning
Research Centre
Anne Edwards
Professor
University of Birmingham
Peter Tymms
Head of Centre
CEM, University of Durham
Richard Andrews
Head of Education
University of York
Mary Rimington
& Anna Douglas
Vice principal
City & Islington College
Deputy Director
Islington Sixth Form College
Jill Wilson &
Paul Ticehurst
Headteacher and
Research coordinator
Oathall Community College
(secondary school, 11-16)
Mary Coyle &
Helen Kenwright
Leaders of Learning
Development Unit
York College
Joe Hallgarten &
David Parker
Director and Head of
Research
Creative Partnerships (DCMS
and Arts Council)
Ken Warman and
Ian Hooper
Head and Senior
lecturer
Tower Hamlets College
Kate Anderson &
Jenny Burnette
Directors of research
and curriculum
LSDA
Geoff Stanton
Consultant
Freelance, post-16 education
and training
15