Working Paper 7.5 Practitioners’ Use of Research A Research Review for the National Evidence for Education Portal (NEEP) Development Group Mark Rickinson October 2005 1 The National Educational Research Forum (NERF) is an independent organisation. Its role is to oversee the development of a coherent strategy for educational research and its use. NERF organises projects, seminars and workshops inspired by and engaging with its partners. The outcomes of these can be found in NERF’s Working Paper series, copies of which can be downloaded from the NERF website. Contact NERF by email on info@nerf.org. Website address: www.nerf-uk.org. Working papers: list of themes 1 Capacity building 2 Systematic reviewing 3 Research strategy 4 Evidence centre 5 D&R programmes 6 Priorities 7 Practitioner engagement 8 Policy and research 9 About NERF 2 Contents Acknowledgements Executive Summary 2 3 1. Introduction 1.1 Background 1.2 Aims 1.3 Scope and Methods 1.4 Structure of the Report 6 6 6 7 8 2. An overview of the evidence base 2.1 Previous gaps 2.2 Current Evidence 2.3 Summary 9 9 9 11 3. How do practitioners access research? 3.1 How frequently do practitioners access research? 3.2 Where do practitioners access information about research? 3.3 What kinds of practitioners are likely to access research? 3.4 Summary 12 12 13 15 17 4. How and why do practitioners use research? 4.1 Conceptual models of research use 4.2 How do practitioners engage with research? 4.3 What do practitioners use research for? 4.4 Summary 18 18 19 21 23 5. What can help practitioners to access and use research? 5.1 Nature of the research 5.2 Nature of the practitioner 5.3 Nature of the professional context 5.4 Nature of the wider context of support 5.5 Summary 24 24 26 27 29 30 6. Conclusions and Implications 6.1 Support for online research resources 6.2 Guidance on content and design issues 6.3 Relationships to other initiatives 6.4 Beyond dissemination and access? 31 31 32 32 33 References Appendix 1: Database Search Strategy 34 41 Acknowledgements I would like to thank several people who have contributed to this piece of work: Victoria White (DfES) for her supportive management of this project on behalf of the Department; Pauline Benefield (NFER) for her advice on the initial database searches; DfES library staff for providing copies of many of the references; and members of the NEEP Development Group for their comments on an earlier version of this report. About the Author Dr Mark Rickinson is an independent educational research consultant, who specialises in research reviews, programme evaluation and research training. He is also a Research Fellow at Oxford University Department of Educational Studies. Contact details: m.rickinson@dsl.pipex.com 2 Executive Summary This document summarises a review of recent UK and US research on education practitioners’ use of research. The review was commissioned by the Department for Education and Skills (DfES) as part of early discussions about the development of a National Evidence in Education Portal (NEEP). The aims of the review were to establish what is known about the ways in which practitioners access and use research, and to identify implications for the future development of NEEP. The work was undertaken by an independent educational research consultant between May and August 2005. What evidence is available? Empirical work on research utilisation by practitioners in education is most definitely an area of research that is newly-emerging rather than wellestablished. Until quite recently, conceptual interest in the research-practice interface had not been matched by in-depth empirical inquiry. The last five years, however, have seen a growth of empirical studies into the ways in which teachers, managers and other practitioners access, engage with and make use of research. The current evidence base can now be seen to span both the conceptual and the empirical, and to provide information about both research access and research use (see below). An Overview of the Evidence Base Accessing Using Empirical Studies of practitioners’: • sources of research information • views on different information sources Studies of practitioners’: • perceptions of research and research outputs • awareness of research findings Studies of: • practitioners’ use of research • initiatives to support research utilisation Reflective writing by: • research-active practitioners Theories/models of: • research dissemination Theories/models of: • research utilisation • research impact • strategies for supporting research use Conceptual How do practitioners access research? Surveys suggest that practitioners who read and use published research are 3 considerably more numerous than those who do not. Where studies have explored this issue in more depth, though, there is concern that such engagement may be superficial or limited. A range of information sources seem to be important in communicating research to practitioners, including INSET, accredited courses, conferences, online databases and various kinds of publications. The internet appears to have become a popular source of research information for practitioners, although poor search skills can limit its effectiveness. There is some evidence to suggest that teachers have a preference for research sources that are indirect and informal (such as research-based teaching materials and communication with colleagues). Recent studies suggest that research use is more likely amongst staff who are more senior/experienced and have postgraduate qualifications and positive beliefs about improvement. How do practitioners use research? Much has been written about how to conceptualise the process of research or knowledge utilisation. A commonly-cited typology distinguishes between research use that is instrumental, conceptual, strategic or through wider influence. Until recently, little was known about the nature and dynamics of research utilisation amongst education practitioners. The small number of studies that have now been undertaken suggest that the process is more complex than some might have assumed. Practitioners seem to use research in ways that are active, selective and variable, and the process can be both values-rich and rewarding. This suggests that the development of NEEP needs to be informed by careful thinking about the ways in which the portal can contribute to practitioners’ professional learning and be useful to those responsible for supporting such learning. What can help practitioners to access and use research? It is clear that there is a wide range of factors that can facilitate or hinder practitioner access to and engagement with research. These include: (i) the nature of the research – factors relating to the focus and form of the research evidence; (ii) the nature of the practitioners – factors relating to the interests, needs and background of the practitioner users; (iii) the nature of the professional context – factors relating to the institutional context in which the research is being utilised; and (iv) the nature of the wider context of support – factors relating to the wider context of knowledge transformation, transfer and communication. Taken together, these factors highlight the complexity of the processes surrounding research utilisation and the breadth of thinking that will be needed in the design and development of NEEP. Implications for a National Evidence in Education Portal (NEEP) 1. Support for online research resources - Several of the studies in this review highlight how difficult it is for practitioners to access information generated by research. This suggests that there is a demand for high quality, well designed and easily navigable online research resources such as that envisaged by the NEEP Development Group. 4 2. Guidance on content and design issues - The development of NEEP does not need to start from scratch. There is now a developing evidence base on the kinds of research outputs that are appealing and useful to education practitioners. This literature needs to be drawn upon and contributed to by those involved with NEEP. 3. Relationships to other initiatives - The ‘evidence-into-practice’ agenda in education is associated with an increasingly complex variety of interventions, programmes and initiatives. The strategy underpinning NEEP will need to consider its points of convergence and divergence with other existing and planned developments within education. 4. Beyond dissemination and access? – According to Louis and Dentler (cited in Seashore Louis and Jones, 2001, p. 13), ‘dissemination systems that are designed to spread information as widely as possible may be different from those that are intended primarily to encourage use’. The evidence presented in this review suggests that careful consideration for how NEEP can work towards encouraging use as well as spreading information will be critical. 5 1. Introduction 1.1 Background The National Educational Research Forum (NERF) is currently working with a consortium of interested organisations, led by the Training and Development Agency for Schools (TDA), to develop a National Evidence in Education Portal (NEEP). NEEP aims to provide all professionals involved in education and learning access to a range of research and evidence materials held in a variety of online sources. Ultimately, the vision is to improve access to the evidence base for education, as has happened in various areas of health care, in order to facilitate an evidence-informed profession. This comes as part of wide-ranging efforts to improve the quality, accessibility and usefulness of educational research in the UK (e.g., DfES, 2002; OECD, 2002; Sebba, 2004) The NEEP Development Group, comprising representatives of all the interested organisations, is in the process of initiating a pilot project to explore the technical options for making existing research evidence accessible and searchable via one portal (see Sheffield and Saunders, 2004 for an earlier scoping study). As part of the background to this pilot phase, the Department for Education and Skills (DfES) on behalf of the Development Group commissioned a research consultant (Dr Mark Rickinson) to undertake a review of what is known about how education practitioners access and use research. This work was undertaken during a three month period between May and August 2005. 1.2 Aims The aim of the project was to review and synthesise recent UK and US research evidence on the ways in which education practitioners access and use research. For the purposes of this review, ‘education practitioners’ was taken to mean teaching and other staff in schools, colleges and universities, as well as other education professionals such as advisors in local education authorities (LEAs). More specifically, the project sought: 1. To establish what is known (and not known) about: the ways in which, and reasons why, practitioners access research the ways in which, and reasons why, practitioners use research the factors that can support or hinder practitioners to access and use research. 2. To identify implications for the future development of NEEP. 6 1.3 Scope and methods The scope of the review was determined by a series of search parameters decided through discussion with NERF and the NEEP Development Group at the start of the project (Table 1.1). Table 1.1: The Search Parameters Overall focus Time-scale Geographical scope Type of literature Empirical studies on the ways in which education practitioners access and use research Work published 1999-early 2005 Work published in the UK and USA Published journal articles, books and research reports and unpublished conference papers, if particularly relevant These search parameters were designed to strike a balance between, on the one hand, being comprehensive in what was included and, on the other hand, being realistic as to what could be achieved within the time-scale and resources of the project. The focus on work published since 1999 reflected the Development Group’s concern with empirical research published since the critiques of educational research in the mid-late 1990s (see Oancea, 2005 for a recent overview of these in the UK and internationally). The inclusion of work published in the United Kingdom (UK) and the United States (US) reflected a desire for evidence that was UK-specific, as well as an interest in some relevant international evidence (time and cost constraints meant that this was restricted to literature from one country). It should be pointed out that although the searches were limited to UK and US literature, where studies from other countries arose from databases or other searches they were included if relevant to the overall focus (e.g., Costa et al., 2000; DETYA, 2000, 2001). With regard to the type of literature included, while the main focus was on published journal articles, books and research reports, unpublished conference papers were also included in order to cover the most recent research developments. Finally, the focus on empirical studies reflected the Development Group’s primary interest in empirical as opposed to conceptual evidence, although discussions of conceptual models of research utilisation were included. In accordance with the search parameters, relevant research literature was identified using a number of complementary search methods. These included: bibliographic database searches of education/social science research databases, including British Education Index, British Education Internet Resource Catalogue, ERIC and SwetsWise (see Appendix 1 for full details) hand searches of previous reviews and bibliographies of relevance to this review (e.g., CUREE, 2003a; Nutley et al., 2003; Hemsley-Brown and Sharp, 2003) online searches using specialist websites such as the Research Unit for 7 Research Utilisation at St Andrews, as well as general internet search engines. These searches identified a large number of potentially relevant studies (1475 records in total), from which a smaller number of 93 studies was selected to order or download in full. Of these, about 60 were judged to be sufficiently relevant to the focus of the review to be worth reviewing in full. The selection processes were based on whether the focus of the publication was in line with the parameters of the review. Examples of excluded studies (from the list of 93 that were ordered or downloaded in full) included: articles dealing with researchers’ or research students’ use of research information sources; research on teachers’ and schools’ use of performance data (as opposed to research outputs); and theoretical discussions of evidence-based practice and policy. The practice of research reviewing is a challenging one and there is ‘a great deal of variety and no established typology of reviews’ (Boaz et al., 2004, p. 2). With a short time-scale and a modest budget, this piece of work is clearly different in scale and scope from the kinds of systematic review funded by the Evidence for Policy and Practice Information and Co-ordinating (EPPI) Centre. That said, it has sought to use systematic search procedures and inclusion/exclusion criteria, and has drawn on review procedures developed as part of several other previous research reviews (see, for example, Rickinson, 2003). An important part of the review process was critical analysis of the available evidence, both in terms of validity or trustworthiness of individual studies’ findings and the strengths and weaknesses of the evidence base as a whole. 1.4 Structure of the report The findings of this review are presented in five sections. Section 2 considers the current evidence as a whole and outlines the kinds of research that have been carried out during the period of this review. Section 3 looks at what is known about the extent and ways in which practitioners access research. In Section 4, the focus shifts to research use with discussion of conceptual models of research utilisation and recent empirical evidence concerning how and why practitioners make use of research. Section 5 presents an overview of the factors that can facilitate or hinder practitioner access and use of research. The report ends with a series of issues and implications for the future development of NEEP. 8 2. An overview of the evidence base This section provides an overview of the research evidence that was included within this review. It begins by describing a previous lack of empirical work in this area, before outlining the types of studies that have been carried out in recent years. 2.1 Previous gaps Empirical work on research utilisation by practitioners in education is most definitely an area of research that is newly-emerging rather than wellestablished. Until very recently, conceptual interest in the research-practice interface had not been matched by in-depth empirical inquiry. As a US researcher argued recently: While there are very many normative perspectives on what the relationship of research and practice should be, there is surprisingly little data about what it actually is. (St. Clair, 2004, p. 1) [original emphases] This observation is echoed by researchers in the UK. John and Prior (2003), for example, bemoan the fact that ‘Only a few studies cast light on the ways in which teachers understand and use research in their practice’ (p. 232). The NERF sub-group report on research impact spoke of ‘a surprisingly small literature on the impact of educational research on policy and practice’ (NERF, 2000, p. 1). Likewise, Everton et al. (2002) point out that ‘surprisingly little attention has been given to establishing what teachers already know about research and researchers’ (p. 374). The literature in higher education (HE) has been described as ‘strong on rhetoric and light on the empirical nature of the relationship between teaching and research’ (Coate et al., 2001, p. 159). Meanwhile, in the context of initial teacher education, Pendry and Husbands (2000) emphasise that ‘We do not know very much about what [research] beginning teachers access [or] the use to which they put it’ (p. 322). This lack of empirical work was also evident in an earlier systematic review on research utilisation, which looked at work published between 1988 and 2001 in the UK, the USA, Canada, Australia and Europe (Hemsley-Brown and Sharp, 2003). In relation to the use of research findings by headteachers and teachers, for example, this review found only two empirical studies to discuss in depth (and neither of these was carried out in the UK). 2.2 Current Evidence Through comparing the situation of four years ago (as described by HemsleyBrown and Sharp, 2003) with the literature identified for this current review, it is clear that empirical studies of practitioners’ perceptions, and use, of research have increased in number. The evidence base in the UK and US 9 can now be seen to include: studies of practitioners’ sources of research information (e.g., Everton et al., 2002) and attitudes towards different information sources (e.g., Williams and Coles, 2003) studies of practitioners’ perceptions of research (e.g., Ratcliffe et al., 2004) and research outputs (e.g., Cordingley, 2000) studies of practitioners’ awareness of research findings (e.g., Costa et al., 2000) studies of the nature and extent of practitioners’ use of research (e.g., St. Clair et al., 2003) and research-based practices (e.g., Gersten et al., 2000) reflective writing by research-active practitioners about research utilisation (e.g., Bevan, 2004) studies of initiatives to support practitioner research access and use (e.g., Sanders et al., 2005) theoretical/conceptual writing about research utilisation (e.g., Nutley et al., 2002), research impact/dissemination (e.g., NERF, 2000) and strategies for supporting research use (e.g., Walter et al., 2003) systematic reviews on research utilisation (e.g., Hemsley-Brown and Sharp, 2003) and research impact (e.g., Nutley et al., 2003). Figure 2.1: An Overview of the Current Evidence Accessing Using Empirical Studies of practitioners’: • sources of research information • views on different information sources Studies of practitioners’: • perceptions of research and research outputs • awareness of research findings Studies of: • practitioners’ use of research • initiatives to support research utilisation Reflective writing by: • research-active practitioners Theories/models of: • research dissemination Theories/models of: • research utilisation • research impact • strategies for supporting research use Conceptual Taken together, these various types of study can be seen to span both the conceptual and the empirical, and to provide information about both research access and research use (Figure 2.1). Broadly speaking (and with a risk of over-simplifying), it could be argued that the last four years (particularly within the UK) have seen a movement from the bottom left to the top right of Figure 2.1. In other words, there has been a growth of empirical inquiry (moving 10 from the bottom to the top of the diagram) and an increased focus on the utilisation of research (moving from the left to the right of the diagram). In the context of this review, this means that there is more empirical evidence to inform initiatives such as NEEP than there was five years ago. However, the research base in this area is still developing and conclusions that are drawn need to be seen as preliminary as opposed to conclusive. 2.3 Summary Until recently, the literature on the research-practice interface in education has tended to be conceptual rather than empirical. There have been few attempts to understand how practitioners perceive, access, engage with and use research. There are signs that this situation is changing due to a growth of empirical inquiry and an increased focus on the utilisation of research by teachers and school leaders. 11 3. How do practitioners access research? This section considers what is known about the extent and ways in which education practitioners access research. 3.1 How frequently do practitioners access research? Data on how often teachers and other education practitioners access research are not plentiful. The information that is available tends to be fairly basic, such as the percentage of survey respondents reporting that they read research publications. Recent UK studies include a MORI telephone poll of 3,000 primary and secondary school teachers in England and Wales (MORI, 2004), and a NFER postal survey of 215 primary, secondary, special and nursery school teachers in England (an 18 per cent response rate) (Taggart et al., 2004). Both studies questioned respondents about the frequency with which they read and/or use research. Their findings (Table 3.1) indicate that the majority of teachers read and use published research findings to inform their professional development or classroom practice at least ‘occasionally’ (51-68 per cent) and, in some cases, ‘frequently’ (22-42 per cent). Only a minority of teachers in both studies (5-7 per cent) claimed never to read or use published research. This was also the case with a small-scale pilot study of research use amongst 35 teachers in four English schools, all but two of whom answered ‘Yes’ to the question ‘Do you ever consult any newspapers/ newsletters/ magazines/ journals about research in education?’ (Hannan et al., 2000). Table 3.1: Frequency of reading and using research by teachers in England and Wales MORI (2004) Taggart et al. (2004) 3,000 primary and secondary teachers in England and Wales Use research to inform professional development or classroom practice 215 primary, secondary, special and nursery teachers in England ‘Frequently’ 42% Read published research findings 26% Use research to inform professional development or classroom practice 22% ‘Occasionally’ 51% 68% 67% ‘Never’ 5% 6% 7% The impression of an encouraging level of teacher self-reported engagement with research corresponds with a US study of 143 adult literacy teachers in Texas (St. Clair et al., 2003). Respondents in this study reported reading research-related items either weekly (22 per cent), monthly (28 per cent) or 12 several times a year (38 per cent), and only 12 per cent stated that they rarely read research. What these figures don’t tell us, however, is what type of research is being consumed and in what depth it is being read or used. Indeed, where studies have explored this issue a little deeper, there has been concern that practitioners’ engagement with research can in fact be superficial or limited. Williams and Coles (2003), in a mixed-method study of research use by UK teachers, noted ‘a distinct contrast between the more positive expression of confidence in the survey and the way teachers talked about their experiences in interviews and groups’ (p. 20). A study of experienced science teachers in Portugal found that their knowledge of education research findings was ‘generally very limited’ (Costa et al., 2000, p. 37). Biddle and Saha’s (2000) study of headteachers in Australia and America concluded that although their research knowledge was broad, it was also shallow. Similarly, the aforementioned study by St. Clair et al. (2003) noted that ‘research consumption, while relatively frequent, is not particularly extensive’ (p. 6). This corresponds with concerns expressed by Pendry and Husbands (2000) about trainee history teachers in England, who were found to be reliant on a very limited number of sources (i.e., one of three books) for information about research. Another issue, raised by Everton et al. (2002), is how current the research is that practitioners are using; their surveys found many respondents referring to research that was ‘undertaken in the late 1970s and early 1980s’ (p. 389). 3.2 Where do practitioners access information about research? If teachers are accessing research, then what sources of information are they using and why? This question has been investigated through surveys (e.g., Everton et al., 2002) and focus groups (e.g., Sanders et al., 2005) with school leaders (e.g. CUREE, 2003a), teachers (e.g., Ratcliffe et al., 2004) and trainee teachers (e.g., Pendry and Husbands, 2000). Within England and Wales, the most comprehensive survey data comes from two Teacher Training Agency (TTA)-funded studies undertaken in the late 1990s (Everton et al., 2000, 2002). Based on a combined sample of 572 school senior managers, teachers and support staff, the study noted a range of sources of information about educational research (Table 3.2). Table 3.2: Sources of research for teachers in England (Everton et al., 2002) Source of Research Relative Popularity In-service training (INSET) 73% Official publications 66% Accredited courses 66% Books 62% Newspapers 57% Journals 56% Other teachers 43% TV 26% 13 A similar picture emerged from a more recent survey of 390 teachers and headteachers in Scotland (240 teachers, 44 headteachers), England (53 teachers, 24 headteachers) and Wales (16 teachers, 10 headteachers) (Williams and Coles, 2003). The researchers found that: ‘the most frequently used sources are colleagues, in-service events, and the kind of reports and professional journals that they are already likely to find in school’ (ibid., p. 13). The internet was also noted as a relatively popular source due to its accessibility and speed, availability of specialist knowledge and currency of information. However, it was also stressed that: a significant proportion of the survey sample also indicated that they never used the internet: 19.9 per cent for subject related information, 35.6 per cent for teaching and learning research. (ibid., p. 14) Furthermore, follow-up interviews with 28 respondents found uncertainly about how to develop search strategies for online information beyond ‘sort of clicking fairly aimlessly on the web’ (ibid., p. 10). This is an important issue for the development and dissemination of NEEP which needs to consider how practitioners’ skills in using ICT as a tool for accessing information for their own professional learning can be enhanced and how the current ‘under-use of the professional information skills of school librarians’ can be challenged (ibid., p. 22). It should be noted that the survey component of this study had low response rates (10 per cent for teachers; 15 per cent for headteachers) and a limited sample size for Wales and England. Several of the sources noted above also featured in a study of science teachers undertaken as part of the ESRC Teaching and Learning Research Programme (TLRP) (Bartholomew et al., 2003; Ratcliffe et al., 2004). Through interviews with 62 primary and secondary science teachers as well as others involved in science education, they found evidence of four main sources of research knowledge: reading INSET interaction with colleagues activities/publications of the Association for Science Education (ASE). More recently still, a series of focus groups undertaken by NFER with 40 staff in research-active schools nationally suggested that ‘the internet [is seen] as the first point of call for finding relevant research’ (Sanders et al., 2005, p. 11). However, and again with pertinence for NEEP, several respondents also ‘commented on the difficulty of finding exactly what they wanted’ through search engines (ibid., p. 4). Alongside online access, though, importance was also attached to: printed materials such as newspapers, union and professional association publications, research summaries/newsletters and professional journals continuing professional development such as conferences, accredited courses, and collaborative research projects 14 mediators such as headteachers, LEA advisors and university-based partners. Another strong theme in the literature on research access is the importance that practitioners attach to research sources that are informal and indirect. A review of the literature on school leaders’ use of research, for example, reported a number of ‘everyday contexts which leaders cited as means of accessing research outcomes: professional association literature; professional development meetings; websites of educational organisations; media coverage of educational issues; and neighbourhood school or special interest networks’ (CUREE, 2003b, pp. 2-3). A study of 127 experienced teachers enrolled in university-based courses in mid-west America concluded that: Perhaps the most salient finding of this study is the consistency with which teachers rated professional journals and college coursework as generally less trustworthy than their own colleagues and workshops or inservice publications. (Landrum et al., 2002, p. 46) The authors see this as part of ‘the small but growing body of evidence that suggests that teachers may be more likely to get information they use in their classroom from the relatively informal sources in their professional lives’ (p. 47). This is supported by several other studies looked at for this review: When asked which format they prefer, respondents identified the internet (49 per cent), newsletters (58 per cent) and conferences (48 per cent) as their favourite means of finding research information, with academic books (9 per cent) and research reports (18 per cent) being the least popular. (St. Clair et al., 2003, p. 7) When invited to identify which publication was consulted [for information about research], 25 (71 per cent) referred to the Times Educational Supplement […] A total of 44 other publications were identified […] Academic journals focused on education studies did not get a single mention. (Hannan et al., 2000, pp. 4-5) In addition to direct research involvement, the teachers decisions were strongly influenced by sources that are themselves directly impacted on by research, specifically ITT [initial teacher training], professional reading, the advice of other teachers, CPD courses and formal postgraduate study. (DETYA, 2000, no page number) 3.3 What kinds of practitioners are likely to access research? There are a number of surveys which suggest that levels of research engagement can vary for different kinds of teachers. Three studies carried out in England, for example, have indicated that: newly-qualified teachers (NQTs) are significantly more likely to never use research as compared with teachers with more experience (MORI, 2004) senior teachers are more interested in using research, while junior staff 15 are more interested in doing research (Everton et al., 2002) the breadth of research information sources is greater for secondary school teachers, deputy headteachers, and staff with postgraduate qualifications (Everton et al., 2002) ‘a hard core of teachers who do not engage with research are likely to be found among female classroom assistants or assistant teachers working in primary schools’ (Everton et al., 2002, p. 378) secondary school teachers are the least likely to use research, while nursery teachers are the most likely to use research (Taggart et al., 2004). One study that looked specifically at practitioners’ attitudes and skills in relation to accessing research information highlights the significance of individuals’ confidence in locating and evaluating information (their ‘information literacy’) (Williams and Coles, 2003). Based on questionnaire responses from 390 teachers and headteachers in Scotland (240 teachers, 44 headteachers), England (53 teachers, 24 headteachers) and Wales (16 teachers, 10 headteachers), this study found that: all respondents were consistently more confident about finding and using general information than finding and using research information confidence levels were lower in the primary and nursery sectors than in the secondary sector headteachers were more confident about locating research information and more likely to use research journals than teachers individuals who had experience of doing research were more confident in handling research information than those who had no such experience. An Australian study identified the following as characteristics of educators who are likely to set out to access new information: observe and question their own accomplishments/practice have a sense of hope that something can be done scan the environment for understanding of the bigger picture use internal problems coupled with optimism as a trigger have a sense of obligation/moral purpose continually ask good questions focus effort by making single tasks serve many purposes (DETYA, 2000; NCSL, 2003). Similarly, studies of research use amongst school leaders internationally indicate that: Motivation to engage with research often sprang from the belief that a core leadership role was to build a learning community and create new professional knowledge to suit the distinct needs of schools. Above all the leaders making use of research believed that improvement was possible. (CUREE, 2003b, p. 1). 16 3.4 Summary Surveys suggest that practitioners who read and use published research are considerably more numerous than those who do not. Where studies have explored this issue in more depth, though, there is concern that such engagement may be superficial or limited. A range of information sources seem to be important in communicating research to practitioners, including INSET, accredited courses, conferences, online resources and various kinds of publications. The internet appears to have become a popular source of research information for teachers, although poor search skills can limit its effectiveness. There is some evidence to suggest that teachers have a preference for research sources that are indirect and informal (such as research-based teaching materials and communication with colleagues). Propensity to seek out research information would seem to vary across groups of teachers. Recent studies suggest that research use is more likely amongst staff who are more senior/experienced and have postgraduate qualifications and positive beliefs about improvement. 17 4. How and why do practitioners use research? In this section, the focus shifts from accessing research to using research. It looks briefly at conceptual models of research utilisation, before considering recent empirical evidence concerning how and why practitioners make use of research. 4.1 Conceptual models of research use There is a well-developed literature on the relationships between research and practice/policy and a wide range of models of research/knowledge utilisation (e.g., Wittrock, 1991; Huberman, 1994; 2002; NERF, 2000; Estabrooks, 2001; Ginsburg and Gorostiaga, 2001; Hammersley, 2001; Nutley et al., 2002, 2003; Hemsley-Brown and Sharp, 2003). A detailed examination of such writing is beyond the scope of this report, but a brief consideration of some of the more prominent ways of conceptualising research use will be a helpful context for later empirical insights. A commonly-presented framework for understanding research utilisation is in terms of instrumental, conceptual and strategic research use. These are summarised by Estabrooks (2001, pp. 283-4) as follows: instrumental research use – which ‘implies a concrete application of research, where the research has often been translated into a material or usable form’ and ‘is used to direct specific decisions and/or interventions’ conceptual research use – where ‘research may change one’s thinking but not necessarily one’s particular action … In this kind of research utilization, research informs and enlightens the decision-maker’ strategic (or symbolic) research use – which ‘involves the use of research as a persuasive or political tool to legitimate a position or practice’. Each of these is underpinned by different assumptions about the nature of research and the contribution that research can make to practice (Wittrock, 1991; Estabrooks, 2001; Hammersley, 2001). Instrumental utilisation involves research providing specific and immediately applicable technical solutions, and sees research as the main or only knowledge source to guide practice. With conceptual use, meanwhile, research is providing concepts which come to play a part in how practitioners define problems, and research is one among several sources of knowledge upon which practitioners can draw. Where the use is strategic, research is providing ammunition for partisan battles, and again is one of many kinds of information that can be used to justify a position. In addition to these three types of use, Nutley et al. (2002) present a fourth kind under the heading ‘wider influence’. This, they argue, reflects the fact that: Research can have an influence beyond the institutions and events 18 being studied. Evidence may be synthesised. It might come into currency through networks of practioners and researchers, and alter policy paradigms or belief communities. (no page numbers) Alongside these conceptual perspectives, recent years have seen the emergence of empirical investigation into the use of research within practice communities. As noted earlier, this has come in response to concern that ‘education […] has wrestled so rarely with the actual day to day utilization of research’ (St. Clair, 2004, p. 226). The findings of such work can shed light on ‘the how’ (i.e. the processes) and ‘the why’ (i.e. the purposes) of research utilisation by practitioners. 4.2 How do practitioners engage with research? There is now a developing empirical evidence base relating to the ways in which education practitioners approach, engage with and make use of research. This comes from studies of the ways in which school staff read, evaluate and respond to pieces of research (e.g., Kennedy, 1999), collaborative investigation into teachers using research to inform their practice and thinking (e.g., Rickinson et al., 2003), reflective writing by research-active practitioners (e.g., Bevan, 2004), and studies that ‘backtrack’ from teachers’ practice to research (e.g., Figgis et al., 2000). The pictures of research utilisation that emerge from this literature suggest a process that is active, selective, values-rich, rewarding and developing (see below). These characteristics should not be taken as hard-and-fast descriptors, but rather as emerging features of a process that is still coming to be understood. They also need to be seen as potentially distinctive from the ways in which researchers might read and use research. To quote from Bartels’ (2003) small-scale US study of how three language teachers and three language researchers evaluated journal articles: For academics, the primary purpose of reading and communicating findings is to build a public base of abstract, generalisable knowledge. […] For teachers, on the other hand, the primary purpose in reading and writing up research is to expand their personal, context-specific bases of knowledge about their teaching and their students’ learning. (ibid., p. 751) 1. An active process - A common theme in utilisation studies is the active manner in which teachers read and evaluate research outputs and ideas. The process, then, is talked about using terms such as ‘translation’ (St. Clair, 2004), ‘drawing analogies’ (Kennedy, 1999; John and Prior, 2003) and ‘filtering, fragmenting and fiddling’ (Bevan, 2004). The common point is that the practitioner user is active in making sense of, and finding meaning in, the research. This is well illustrated by the literature on school leaders which highlights a wide range of active processes associated with research engagement. Examples include: ‘acquiring and interpreting information, including that from research findings’; ‘enquiry, including collaborative action research and analysing data’; ‘planning development work’; and ‘professional 19 dialogue’ (CUREE, 2003a, p. 6). 2. A selective process – Perhaps not surprisingly, teachers (like other users) are found to be idiosyncratic in the ways they read, process and make use of research findings. An Australian study that examined teachers’ practices for evidence of the impact of educational research found utilisation to be ‘a very individual process’ (DETYA, 2000, no page numbers). Kennedy’s (1999) study of 100 US teachers’ responses to different genres of research found that individuals ‘varied in their reasons for perceiving studies as persuasive and relevant’, thus challenging ‘the tacit assumption that teachers are homogenous in their perceptions of the value of research’ (p. 27). Along similar lines, Bevan (2004, p. 329) talks about ‘the subjective selection and rejection of research according to personal reading’ amongst teachers in the early phases of their careers. Furthermore, a project exploring researchpractice connections in science education, found that ‘new materials and approaches are judged first on their practicality and fit to the context of application’ (Millar and Hames, 2003, no page numbers). 3. A values-rich process – Research use is not only active and selective, it also appears to be personal and values-dependent. Work with science teachers, for example, reported that ‘weight of evidence is rarely sufficient to change educational practice – claims must also resonate with prior beliefs and experience’ (Bartholomew et al., 2003, p. 1). Similarly, Australian research concluded that ‘teachers generally seek out the sources they believe will inform their existing knowledge base’ (DETYA, 2001, p. 2). It has also been reported that research engagement can ‘evoke strong feelings and involve challenging emotions’ on the part of practitioners (Rickinson and Reid, 2003, p. 15). 4. A rewarding process - Kennedy’s work with US teachers found that: ‘Many used the studies to reinterpret their own experiences, and many volunteered that they had already made copies of one or more of the studies to share with colleagues’ (Kennedy, 1999, p. 27). A similar story was reported by John and Prior (2003) for their sub-sample of eight research-active teachers: ‘Not only did they seek to connect the findings to their practice, but they also perceived the purpose of reading as educative and enlightening’ (p. 239). In addition, participants of a project looking at research-practice connections in environmental education reported benefits such as: ‘I got far more from the research than I anticipated’; ‘I appreciated the opportunity to discuss the research findings and test ideas and assumptions with practitioner colleagues’ (Rickinson et al., 2004, p. 207). 5. A developing process - In a recent paper within a special issue of the journal Teacher Development, Bevan (2004) presents a model of practitioner research engagement in terms of three phases during a teachers’ career. Drawing on his own biography as a practitioner researcher as well as ideas from research into teachers’ life cycles, Bevan argues that research engagement develops through: ‘filtering’ in early career – ‘subjective selection and rejection of research with personal interpretation’ 20 ‘fragmenting’ during mid career (typically between the fourth and eighth years of teaching) – ‘research findings isolated, simplified and removed from context’ ‘fiddling’ during the time when professional practice becomes established and experimentation becomes the norm – ‘applying findings, action research, tinkering and transforming’ (p. 327). Arguably the key message emerging from all of the above points is the idea that research use by practitioners is more complex than some might have hoped or assumed. As Bell et al. (2002) have argued, research use is about learning and so needs to be seen ‘as a pedagogic problem’ rather than a transmission problem. Put another way: Evidence-informed practice does not merely mean bringing new information about what works to bear on professional practice, it becomes part of an ongoing learning process on the part of the practitioner. (Cordingley, 2004, p. 80) This suggests that the development of NEEP needs to be informed by careful thinking about the ways in which the portal can contribute to practitioners’ professional learning and be useful to those responsible for supporting such learning. 4.3 What do practitioners use research for? Until recently, little was known about what education practitioners actually did with research when they used it. Four studies of different groups of teachers in England and the US have provided some preliminary evidence on this topic (Table 4.1). This is clearly an evolving area of understanding, but some insights can be drawn from the examples in Table 4.1. It is clear, firstly, that research can be used for a range of different purposes. The idea that research use can range from the more conceptual (i.e., raising questions) to the more instrumental (i.e., providing answers) is certainly evident in all of the studies. So too is the notion of strategic use (i.e., providing validation). Interestingly, though, it seems that, in three of the studies, validation can take two forms: a personal affirmation of one’s practices, or evidence to convince others of the value of a particular approach. The former is more about personal verification, while the latter is more about public advocacy. Another theme in two of the studies is where using research is about doing research. Both the Rickinson et al. (2003) and the Sanders et al. (2005) work cite examples of practitioners describing research as a stimulus for further school-based investigation and inquiry. 21 Table 4.1: Empirical Studies of Research Use by Practitioners St. Clair et al. (2003) 143 adult literacy teachers in USA Pendry and Husbands (2000) 141 history PGCE students in England Three categories of use: Two categories of use: Validation – ‘Mainly it’s towards the grant writing entity or proving to United Way why or why not you can make certain gains’ Design – ‘The research helped direct me in my teaching strategies or curriculum’ Improvement – ‘It just helps the program to run a little more smoothly and little more cheaply and a little more efficiently’ Rickinson et al. (2003) 6 environmental educators in England Practical use – ‘It helped me to devise teaching strategies’ Prompting reflection – ‘It made me think more deeply about what was happening in my lessons’ Sanders et al. (2005) 40 research-active teachers England Six categories of use: Five categories of use: To support/justify – ‘I realised we had many practices already in place which support aspects of the research’ To challenge – ‘The research raised emotive and powerful questions for me as a practitioner’ To reflect – ‘It has helped me reflect on what I was doing and how I do it’ To investigate – ‘I wanted to find out if this [research finding] was true for some of my students’ To change – ‘I wanted to find information that might help to explain why these misconceptions were happening and how I could change my practice to tackle them’ To inform – ‘The research pointed us towards various criteria which we must address in order to make future projects successful and worthwhile’ in For general information To follow-up items of particular interest with further reading To recommend items to colleagues To contribute to further study To inform their own research To validate or celebrate their practice To change their practice Perhaps the key conclusion that can be drawn is the idea that research use is 22 multi-dimensional and can involve practitioners not only as teachers but as planners, thinkers, leaders, coaches, researchers and learners. In other words, research utilisation is not simply about what teachers do in the classroom or the lecture hall but also about what they do in the many other aspects of their professional roles. 4.4 Summary Much has been written about how to conceptualise the process of research or knowledge utilisation. A commonly-cited typology distinguishes between research use that is instrumental, conceptual, strategic or through wider influence. Until recently, little was known about the nature and dynamics of research utilisation amongst education practitioners. The small number of studies that have now been undertaken suggest that the process is more complex than some might have assumed. Practitioners seem to use research in ways that are active, selective and variable, and the process can be both values-rich and rewarding. The purposes for which research is used are also wide-ranging in the sense that it can be about validating practice, challenging practice, prompting reflection about practice, stimulating research about practice or improving/changing practice. Furthermore, this can involve practitioners not only as teachers, but also as planners, thinkers, leaders, researchers and learners. 23 5. What can help practitioners to access and use research? This section presents an overview of the factors that can facilitate or hinder access to and use of research by education practitioners. There is a considerable amount of evidence relating to this issue, which can be considered under four broad headings: 5.1 nature of the research – factors relating to the focus and form of the research evidence nature of the practitioners – factors relating to the interests, needs and background of the practitioner users nature of the professional context – factors relating to the institutional context in which the research is being utilised nature of the wider context of support – factors relating to the wider context of knowledge transformation, transfer and communication. Nature of the research Evidence of research utilisation being influenced by the nature of the research has often come in the form of barriers rather than facilitators. Two issues that recur in studies of practitioners’ perceptions of research are complaints about the inaccessibility of the language and the challenge of locating work that is relevant. Statements along the lines of ‘practitioners sometimes sounded frustrated as they talked about how difficult it was to get information already generated by research’ (St. Clair et al., 2003, no page numbers) are common. A recent study of science teachers found that ‘the inaccessibility of many research reports in both location and style [was] seen as a barrier to the impact of research’ (Ratcliffe et al., 2004, p. ii). Similar issues arose in a NFER study on the use of research for school improvement within 19 local education authorities (LEAs) in England and Wales (Wilson et al., 2003; Wilson, 2004). Based on interviews, focus groups and surveys of school, university and LEA staff in LEAs with and without specific strategies in place to encourage the use of research, they found evidence of six main barriers to research use. Three of these were related to research itself: ‘problems with accessing research’, ‘a perceived lack of relevance and practicality in published research’ and ‘uncertainly about the results of research’ (Wilson et al., 2003, pp. 11-12). The difficulty of deciphering the language of research, particularly for practitioners with little previous experience of research, is further highlighted by other recent studies with practitioners in the UK (John and Prior, 2003; Rickinson et al., 2003; Williams and Coles, 2003) and the US (Bartels, 2003). The latter uses Gee’s theory of Discourses to frame the problem in terms of teachers and researchers being members of different professional Discourse communities. Problems arise where research papers fail to ‘incorporate [the] kind of textual features and qualities that are central to teachers’ Discourse’ (Bartels, 2003, p. 751). 24 Another issue that has been explored is that of whether studies using particular types of methodologies have more or less credibility in the eyes of practitioners. A US study that questioned 100 school teachers about examples of different research genres found no evidence for ‘the superiority of any particular research genre, whether the criterion for superiority is persuasiveness, relevance, or ability to influence practitioners’ thinking’ (Kennedy, 1999, p. 26). What was of concern to the teachers in this study was the research topic: The studies that teachers found to be most persuasive, most relevant and most influential to their thinking were all studies that addressed the relationship between teaching and learning. (ibid., p. 26) This raises the question of what features of educational research and its outputs are likely to aid utilisation in practice? Based on the studies analysed for this review, it would seem that engagement and use by educational practitioners will be more likely for research outputs that: provide ‘brief summaries [that can] help teachers decide quickly whether the study is likely to be useful’ (Cordingley, 2000, p. 3; Rickinson et al., 2003) as well as ‘clear layout and good signposting’ (Sanders et al., 2005, p. 18) come in a format that is tangible and useful, such as ’curriculum materials and teaching approaches, resulting from transformation of research findings into practical strategies’ (Bartholomew et al., 2003, p. 3) and ‘clear implications for practice’ (Rickinson and Reid, 2003, p. 8; Wilson et al., 2003) focus on ‘real classroom situations’ (Wilson et al., 2003, p. 13) and ‘the relationship between teaching and learning’ (Kennedy, 1999, p. 26) and have the potential to ‘enhance pupils’ learning’ (Cordingley, 2004, p. 82) use ‘crisp, plain language’, ‘case study portraits of strategies’ and ‘short relevant video clips of authentic teaching and learning situations’ (Cordingley, 2000, p. 2; Olivero et al., 2004) feature ‘use of colour and visual illustrations’ and, for websites, ‘a limited amount of text on screen, clear sections with easy navigation, good printing capabilities and tool such as search facilities and “email to a friend”’ (Sanders et al., 2005, p 18; Taggart et al., 2004) have convincing findings, that is ‘from studies with clear, rigorous methods which seem likely to generalise to other contexts’ (Ratcliffe et al., 2004, p. iv) are made accessible via ‘a well organised access point to research information located within the school’ (William and Coles, 2003, p. 22). This list could easily be extended or reconfigured. Perhaps the most important point, though, is that almost all of the characteristics above involve the judgement of the practitioner as to the usefulness, trustworthiness and relevance of the research. This brings us onto the second set of factors that 25 can influence whether and how practitioners access and use research: the practitioners themselves. 5.2 Nature of the practitioner A recurring message emanating from research on knowledge utilisation within the field of education and beyond is that ‘“users” are far from empty vessels to be filled with the wisdom of research’ (Huberman, 1994, no page numbers). As we saw earlier, practitioners tend to be active and selective in the ways they read, evaluate and make use of research outputs. Hence the identity and background of the practitioner can influence the nature and extent of research use. Recent studies highlight two ways in which this can happen: as a result of practitioners’ professional knowledge and beliefs; and as a result of practitioners’ research training and experience. Several studies of practitioners’ engagement with research have emphasised the importance of individuals’ professional knowledge and beliefs. Amongst school leaders, for example, ‘beliefs about leadership roles and responsibilities’ have been shown to be a major factor in their engagement with research (CUREE, 2003a, p. 3). Ratcliffe et al.’s (2004) work with science educators identified four conditions that research findings must fulfil in order to make an impact on classroom practice. The second of these was that research must ‘resonate with, or acknowledge, teachers’ professional experiences’ (ibid., p. iv). This was because: For many teachers, the principal factor in determining whether a piece of research might influence their own practice was not its quality as research, but rather the extent to which it accorded with their own gut reactions, departmental policy and practice, or personal experience and beliefs. (Ratcliffe et al., 2003, p. 14) Along similar lines, Bartels’ (2003) study of language teachers in the US found that ‘ideas in journal articles were seen as legitimate not on the basis of empirical evidence, but according to the ease with which they could integrate that knowledge into their personal knowledge base’ (p. 743). This corresponds with earlier work cited by Hemsley-Brown and Sharp (2003) in their systematic review on research use in education and medicine, as well as a synthesis of five studies that, in different ways, examined the impact of educational research in Australia (DETYA, 2000, 2001). A key conclusion of the latter was that: ‘Individual attitudes and beliefs […] are important in the promotion of research usage. The converse is also true, these can act as blocks to the use of research’ (ibid., no page numbers). Further evidence of the ways in which practitioner research audiences ‘weigh new information against the constructs and experience they have built up throughout their lives’ (Huberman, 1994, no page numbers) is provided by studies with school and community educators (St. Clair et al., 2003; Rickinson et al., 2003; Williams and Coles, 2003) and Further Education (FE) college lecturers (Goodrham, 2004). A second area of influence comes in terms of practitioners’ familiarity and 26 confidence in using and doing research. There is considerable empirical evidence which suggests that training in research methods or involvement in doing research can help to develop more positive attitudes to, and sophisticated skills in, the use of research amongst practitioners. HemsleyBrown and Sharp (2003) cite a 1991 study of school principals in America and Australia, which found that postgraduate training contributed towards a principal’s regard for educational research knowledge even though such research might be considered problematic (Biddle and Saha, 2000). Everton et al.’s (2002) survey of 572 school teachers in England found that respondents who reported never using research were more likely to have no postgraduate qualifications as compared with those who did report using research. The authors concluded that: ‘it is reasonable to infer that an extended period of full- or part-time study is likely to make a teacher more responsive to research’ (p. 393). This corresponds with studies of research utilisation in fields such as nursing, which have shown that ‘education about research may be one way to overcome negative attitudes towards research as a barrier to research utilization’ (McLeary and Brown, 2003, p. 556; see also Rogers, 2000). The same can be said of involvement in doing research of some kind. Williams and Coles (2003), who questioned 390 UK teachers and headteachers, reported a significant positive correlation between previous involvement in research and confidence in handling research information. In an interview-based study with two groups of teachers (one with considerable research experience and the other with none), John and Prior (2003) found that the ‘research-orientated group read the articles in a more varied and extended way’ (p. 238). By contrast, those with little or no research experience ‘had difficulty in reading […] and could not understand how the articles overtly connected with their practice’ (p. 236). Similarly, science teachers with prior experience of research were found to be ‘more prepared than others to consider a range of data collection and analysis methods as constituting research’ and ‘better able to view professional practice through a different “evidence-based” lens’ (Ratcliffe et al., 2003, p. 26). Other studies highlighting the value of action research experiences in this respect include Elliott (2000), Hannan et al. (2000) and Jaworski (2000). The idea of postgraduate study facilitating research use is, however, challenged by one recent study of Texan literacy educators. The research team found that: ‘Somewhat to our surprise, practitioners with more education did not use research more, are not more likely to use academic sources and are not more likely to have made changes based on research’ (St. Clair et al., 2003, p. 8). What this study did find, though, was that more experienced practitioners were more likely to read and use research than more junior ones. 5.3 Nature of the professional context Any discussion of factors influencing research use would be incomplete without consideration for the professional contexts in which practitioners are working. Hemsley-Brown and Sharp (2003) were very clear on this point in their review of the pre-2001 literature: 27 The conclusions from empirical research, in both education and nursing, confirm that the main barriers to knowledge use in the public sector are not at the level of individual resistance but originated in an institutionalised culture that does not foster learning. […] The key recommendation that emerges from empirical studies is that managers are viewed as key facilitators, and there is a need to create and develop an organisational culture that values and uses research. (p. 460) A key organisational barrier to research use cited by practitioners in many studies of is a lack of time. Wilson et al.’s (2003) investigation into the use of research for school improvement reported that ‘While teachers said they would like to use research, they felt they were prevented from doing so by the volume of day-to-day work’ (p. 11). A very similar message was communicated by three quarters of the science teachers in Ratcliffe et al.’s (2003) study. Interviewees highlighted ‘a lack of thinking time to sit and think and discuss things’ and ‘the amount of time that it takes to read and make sense of articles published in research journals’ (ibid., p. 23). Likewise, a recent study of research-engaged schools (which is discussed in more detail below) found that ‘By far the greatest difficulty in taking on a research activity was in finding time for teachers and others to engage in research’ (Sharp et al., 2005, p. 52). Underpinning issues of time, though, are questions of funding and resources. Williams and Coles (2003), for example, found that tied to the issue of a lack of time was ‘the lack of ready access to research in one place (i.e. the school), lack of readily accessible information on what is available and the fact that the school library did not provide access to research information’ (p.13). Participants in Wilson et al.’s (2003) study called not only for the provision of time, but also for funding and expertise and for research to be prioritised as an important aspect of professional practice. Furthermore, the editorial of a recent special issue of the journal Teacher Development focused on teachers and research stated that: One of the most compelling ‘take away’ messages from this whole journal issue is the need for strategic leadership, practical support and sources of funding at national level for research-informed professional practice. (Saunders, 2004, p. 124) Coupled with time and resources, though, are a series of other factors that go to make up what Hemsley-Brown and Sharp (2003) call ‘an organisational culture that values and uses research’. Evidence demonstrating the influence of school culture or ethos on research engagement has emerged from studies of teachers (e.g., Williams and Coles, 2003) and school leaders (e.g., CUREE, 2003a). A key argument is that ‘research is more readily integrated into school life when systems are in place to enable the school to operate as a learning organisation’ (CUREE, 2003a, p. 5). This is well illustrated by the concept of ‘research-engaged schools’ which are described as schools that have ‘research and enquiry at the heart of their outlook, 28 systems, and activity’ (Handscomb and Macbeath, 2003, p. 4). A recent indepth study of 15 such schools in England, identified the following ‘key conditions for becoming a research-engaged school’: A culture that values openness, reflection and professional debate A commitment to using evidence for school improvement A commitment of resources (especially staff time) A willingness to embed research activity into existing school systems, such as school development planning, staff meetings, staff development activities and performance targets Access to sources of expertise and support (Sharp et al., 2005, p. 53). The importance of these sorts of conditions is supported by a number of other studies (Abbott et al., 1999; Gersten et al., 2000; Cordingley et al., 2002; St. Clair et al., 2003; Goodrham, 2004). 5.4 Nature of the wider context of support An Australian study that sought to explore the effect of research on policy and practice by backtracking from educators’ practices, put forward a ‘user-centric’ model of the impact of educational research (Figure 5.1) (Figgis et al., 2000). Based on extensive interviewing, observation and documentary analysis, they concluded that: The gap between research knowledge and researchers is linked by a connecting web to the practitioner or policy-maker who deals with professional problems. The connecting web has many nodes that represent linking activities […] for example, conferences, taskforces, reports, media coverage and internet sites. (DETYA, 2001, p. 3) Figure 5.1 A User-centric Model of the Impact of Educational Research (Figgis et al., 2000) This is important in the context of considering what can support practitioners 29 to access and use research as it extends the discussion beyond the earlier issues of research, practitioners and institutional cultures to the wider context of support (or otherwise) for research engagement. Pertinent issues here include, for example: 5.5 the significant role played by mediators such as professional associations, teacher unions, LEAs and CPD facilitators (Wilson et al., 2003; Ratcliffe et al., 2004) the need for ‘sustained interactivity’ and ‘dialogue’ between and among researchers and practitioners as part of the research process (Huberman, 2002, p. 263; Ginsburg and Gorostiaga, 2001, p. 192) the importance of ‘professional learning networks’, ‘collegial networks’ and ‘practitioner dissemination and research networks’ (CUREE, 2003a, p. 2; Gersten and Dimmino, 2001, p. 122; Smith et al., 2002) the need for recognition and reward for practitioners and institutions who use research (Wilson et al., 2003) the potential for inspiring and enthusiastic researchers to draw practitioners into using research (Sanders et al., 2005). Summary This section makes clear the complex mix of factors that can facilitate or hinder practitioner access to and engagement with research. These relate not only to the nature of the research outputs and the practitioner users, but also the institutional contexts and wider contexts of support in which utilisation takes place. With respect to the development of NEEP, the evidence presented here suggests a need for careful thinking as to how a national evidence for education portal might build upon and connect with the many other parts of the research utilisation picture. 30 6. Conclusions and Implications This final section suggests a number of implications for the development of NEEP. A report into dissemination practices within the USA made an interesting point about the development of the Educational Resources Information Centres (ERIC). It stated that: The ERIC system was developed in response to the deeply held (and accurate) belief that one of the problems with the low levels of research utilisation in education that were routinely discussed in the 1950s and 1960s was the inaccessibility of research. However, ERIC was developed without any corresponding thought given to how educators would actually use the system. ERIC, as designed, was not, and is still not, “user friendly”. (Seashore Louis and Jones, 2001, p. 6) [emphasis added] This is helpful in highlighting the sorts of shortcomings that NEEP must seek to avoid. Unlike ERIC, NEEP needs to give every thought to ‘how educators would actually use the system’, and, judging by the contents of this review, there is at least some research evidence that can help to inform this. From this review, there seem to be four issues that have potential implications for the future development of NEEP. 6.1 Support for online research resources Several of the studies in this review highlighted ‘how difficult it was [for practitioners] to get information already generated by research’ (St. Clair et al., 2003; no page numbers; Williams and Coles, 2003; Wilson et al., 2003; Ratcliffe et al., 2004). This is true for electronic media as much as for other information sources: Participants referred to the usefulness of search engines as a starting point for finding research, although they all commented on the difficulty of finding exactly what they wanted. (Sanders et al., 2005, p. 4) These kinds of challenges suggest that there is a demand for high quality, well designed and easily navigable online research resources such as that envisaged by the NEEP Development Group. Indeed, Williams and Coles (2003, p. 22) state specifically that: the key to overcoming access problems appear to be to have a single point of access, organised and navigable to allow teachers to find their way to what they want in the limited time available, and to site that resource within the school. 31 6.2 Guidance on content and design issues The development of NEEP does not need to start from scratch. There is now a developing evidence base on the kinds of research outputs that are appealing and useful to education practitioners. This was discussed earlier (Section 5.1), but the following suggestions from recent studies are worth reiterating: 6.3 provide ‘brief summaries [that can] help teachers decide quickly whether the study is likely to be useful’ (Cordingley, 2000, p. 3; Rickinson et al., 2003) as well as ‘clear layout and good signposting’ (Sanders et al., 2005, p. 18) use ‘crisp, plain language’, ‘case study portraits of strategies’ and ‘short relevant video clips of authentic teaching and learning situations’ (Cordingley, 2000, p. 2; Olivero et al., 2004) feature ‘use of colour and visual illustrations’ and, for websites, ‘a limited amount of text on screen, clear sections with easy navigation, good printing capabilities and tool such as search facilities and “email to a friend”’ (Sanders et al., 2005, p. 18; Taggart et al., 2004) have convincing findings, that is ‘from studies with clear, rigorous methods which seem likely to generalise to other contexts’ (Ratcliffe et al., 2004, p. iv). Relationships to other initiatives The ‘evidence-into-practice’ agenda is associated with an increasingly complex variety of interventions, programmes and initiatives (Nutley et al., 2002). Education is no exception, which means that NEEP will need to consider how it relates to other existing and planned developments. As Atherton (1999, p. 3) has argued in the relation to similar development in social care: A consequent task for all these initiatives is to be clear between themselves, and to others, about what each offers, what is the same and what is different and to then be clear to potential users about what can be expected from each service. A framework that may prove helpful in exploring these issues is a taxonomy developed by Walter et al. (2003) to make sense of the many interventions designed to enhance the impact of research within the criminal justice, health, education and social care sectors. The taxonomy is based on eight ‘mechanisms’ that drive research impact, and 32 ‘intervention types’ that are similar in form and content (Figure 6.1). The detail of the different types featured in Figure 6.1 is not important to go into here; the purpose of sharing this framework is to highlight the complexity that will surround NEEP and the need to consider carefully how NEEP can support and connect with other related developments. 32 Figure 6.1: A taxonomy of research impact interventions (adapted from Walter et al., 2003) MECHANISMS Dissemination Education Social influence Collaboration Incentives Reinforcement Facilitation Multifaceted 6.4 INTERVENTION TYPES Written materials General Feedback Oral education Reminders presentations Endorsement Changes in Alternative Lobbying structures formats Client-mediated Office systems Mass media interventions Computer Research access Boundary support systems Research-based spanners guidance Networks Use of internet Educational Expert support Quality systems materials Research-inOrganisational Passive practice Multi-component education Research initiatives Staff planning development Collaboration Educational Research outreach incentives Interactive Financial education incentives Beyond dissemination and access? This final issue picks up on what has been a recurring theme throughout the review, namely that practitioner research utilisation involves much more than ‘bringing new information to bear on professional practice’ (Cordingley, 2004, p. 80). This raises the question of the extent to which NEEP will be concerned with promoting research use as well as dispersing research information. In other words, will the remit of NEEP go beyond dissemination and access? To what extent will NEEP be concerned not just with providing ‘a single point of access to aggregated information’ (Devedzic, 2005, p. 96), but also with: facilitating practitioners’ research information literacy and professional learning; supporting research mediation by leaders, advisors and information specialists; and enhancing research cultures within and between organisations? According to Louis and Dentler (cited in Seashore Louis and Jones, 2001, p. 13), ‘dissemination systems that are designed to spread information as widely as possible may be different from those that are intended primarily to encourage use’. In the light of the evidence presented in this review, careful consideration for how NEEP can work towards encouraging use as well as spreading information will be critical. 33 References ABBOTT, M., WALTON, C., TAPIA, Y. and GREENWOOD, C. R. (1999). ‘Research to practice: a “blueprint” for closing the gap in local schools’, Exceptional Children, 65, 3, 339–52. ATHERTON, C. (1999). ‘Getting a GRIP’, Research Policy and Planning. [online]. Available:http://www.elsc.org.uk/socialcareresource/rpp/articles/1711999art1. htm [10 August, 2005] BARTELS, N. (2003). ‘How teachers and researchers read academic articles’, Teaching and Teacher Education, 19, 737-753. BARTHOLOMEW, H., HAMES, V., HIND, A., LEACH, J., OSBORNE, J. and RATCLIFFE, M. (2003). Towards Evidence-based Practice in Science Education 4: Users’ Perceptions of Research (Teaching and Learning Research Briefing 4) [online]. Available: http://www.tlrp.org/pub/documents/no4.pdf [11 August, 2005]. BELL, M., CORDINGLEY, P., CURTIS, A., EVANS, D., HUGHES, S. and SHREEVE, A. (2002). ‘Bringing research resources to practitioner users via web technology: lesson learned to date.’ Paper presented at the British Educational Research Association Annual Conference, University of Exeter, 12-14 September. BEVAN, R. B. (2004). ‘Filtering, fragmenting and fiddling? Teachers’ life cycles and phases in their engagement with research’, Teacher Development, 8, 2/3, 325–39. BIDDLE, B. J. and SAHA, L. J. (2000). ‘Research knowledge use and school principals.’ Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans, 24 April. BOAZ, A., SOLESBURY, W. and SULLIVAN, F. (2004). The Practice of Research Reviewing: an Assessment of 28 Review Reports. London: ESRC UK Centre for Evidence Based Policy and Practice. COATE, K., BARNETT, R. and WILLIAMS, G. (2001). ‘Relationships between teaching and research in higher education in England’, Higher Education Quarterly, 55, 2, 158–74. CORDINGLEY, P. (2000). ‘Teacher perspectives on the accessibility and usability of research outputs.’ Paper presented at the British Educational Research Association Annual Conference, University of Cardiff, 7-9 September. CORDINGLEY, P. (2004). ‘Teachers using evidence: using what we know 34 about teaching and learning to reconceptualize evidence-based practice.’ In: THOMAS, G. and PRING, R. (Eds) Evidence-Based Practice in Education. Maidenhead: Open University Press. CORDINGLEY, P., WILSON, J., SEDDON, K. and THOMAS, F. (2002). ‘Encouraging discerning research involvement by schools and developing the role of CPD co-ordinators – the contribution of the National Teacher Research Panel’, Paper presented at the British Educational Research Association Annual Conference, University of Exeter, 12-14 September. CENTRE FOR THE USE OF RESEARCH AND EVIDENCE IN EDUCATION (CUREE) (2003a). Leading the Research-engaged School: Why and How School Leaders Engage with Educational Research [online]. Available: http://www.ncsl.org.uk/media/F7B/94/randd-engaged-full.pdf [3 September, 2005]. CENTRE FOR THE USE OF RESEARCH AND EVIDENCE IN EDUCATION (CUREE) (2003b). How do leaders and their teacher colleagues become interested in using research evidence to inform their practice? What conditions encourage active engagement in research? [online]. Available: http://www.curee-paccts.com/dynamic/curee43.jsp?m=46 [3 September, 2005]. COSTA, N., MARQUES, L. and KEMPA, R. (2000). ‘Science teachers’ awareness of findings from education research’, Research in Science and Technological Education, 18, 1, 37–44. DEPARTMENT FOR EDUCATION AND SKILLS (DfES) (2002). Research and Development in England: Background Report Prepared for the OECD Review. London: DfES. DEPARTMENT OF EDUCATION, TRAINING AND YOUTH AFFAIRS (DETYA) (2000). The Impact of Educational Research [online]. Available: http://www.dest.gov.au/archive/highered/respubs/impact/overview.htm [12 August, 2005]. DEPARTMENT OF EDUCATION, TRAINING AND YOUTH AFFAIRS (DETYA) (2001). Educational Research: In Whose Interests? (Higher Education Series Report No. 39). Canberra: DETYA Higher Education Division. DEVEDZIC, V. (2005). ‘Research community knowledge International Journal of Knowledge and Learning, 1, 1/2, 96-112. portals’, ELLIOTT, J. (2000). ‘How do teachers define what counts as ‘credible evidence’? Some reflections based on interviews with teachers involved in the Norwich Area Research Consortium.’ Paper presented at the British Educational Research Association Annual Conference, University of Cardiff, 7-9 September. 35 ESTABROOKS, C.A. (2001). ‘Research utilization and qualitative research.’ In: MORSE, J.M., SWANSON, J.M. and KUZEL, A.J. (Eds) The Nature of Qualitative Evidence. London: Sage. EVERTON, T., GALTON, M. and PELL, T. (2000). ‘Teachers’ perspectives on educational research: knowledge and context’, Journal of Education for Teaching, 26, 2, 168–82. EVERTON, T., GALTON, M. and PELL, T. (2002). ‘Educational research and the teacher’, Research Papers in Education, 17, 4, 373–401. FIGGIS, J., ZUBRICK, A., BUTORAC, A., and ALDERSON, A. (2000). ‘Backtracking practice and policies to research.’ In: DEPARTMENT OF EDUCATION, TRAINING AND YOUTH The Impact of Educational Research [online]. Available: http://www.dest.gov.au/archive/highered/respubs/impact/overview.htm [12 August, 2005]. GERSTEN, R., CHARD, D. and BAKER, S. (2000). ‘Factors enhancing the sustained use of research-based instructional practices’, Journal of Learning Disabilities, 33, 5, 445–57. GERSTEN, R. and DIMINO, J. (2001). ‘The realities of translating research into classroom practice’, Learning Disabilities Research and Practice, 16, 2, 120–30. GINSBURG, M. B. and GOROSTIAGA, J. M. (2001). ‘Relationships between theorists/researchers and policy makers/practitioners: rethinking the twocultures thesis and the possibility of dialogue’, Comparative and International Education Society, 45, 2, 173–96. GOODRHAM, M. (2004). ‘Using research to enhance professionalism in further education (FE)’, Paper presented at the 5th Annual Conference of the Teaching and Learning Research Programme, Cardiff, 22-24 November. HAMMERSLEY, M. (2001). Educational Research, Policy Making and Practice. London: Paul Chapman. HANNAN, A., ENRIGHT, H. and BALLARD, P. (2000). Using Research: the Results of a Pilot Study Comparing Teachers, General Practitioners and Surgeons [online]. Available: http://www.leeds.ac.uk/educol/documents/000000851 [29 July 2005]. HANDSCOMB, G. and MACBEATH, J. (2003). The Research Engaged School. Chelmsford: Essex County Council. HEMSLEY-BROWN, J. and SHARP, C. (2003). ‘The use of research to improve professional practice: a systematic review of the literature’, Oxford Review of Education, 29, 4, 449–70. 36 HUBERMAN, M. A. (1994). ‘Research utilization: the state of the art’, Knowledge and Policy, 7, 4, 13–34. HUBERMAN, M. A. (2002). ‘Moving towards the inevitable: the sharing of research in education’, Teachers and Teaching: Theory and Practice, 8, 3/4, 257-68. JAWORSKI, B. (2000). ‘Research or research and its relation to mathematics teaching.’ Paper presented at the Conference of the British Society for Research into Learning Mathematics. JOHN, P. and PRIOR, J. (2003). ‘Conceptions, contentions and connections: how teachers read and understand different genres of educational research.’ In: SUTHERLAND, R., CLAXTON, G. and POLLARD, A. (Eds) Learning and Teaching: Where Worldviews Meet. Stoke on Trent: Trentham Books. KENNEDY, M. (1999) ‘A test of some common contentions about educational research’, American Educational Research Journal, 36, 3, 511–41. LANDRUM, T. J., COOK, B. G., TANKERSLEY, M. and FITZGERALD, S. (2002). ‘Teacher perceptions of the trustworthiness, usability and accessibility of information from different sources’, Remedial and Special Education, 23, 1, 42–8. MARKET AND OPINION RESEARCH INTERNATIONAL (MORI) (2004). MORI Teachers' Omnibus 2004 (Wave 2). London: MORI. McLEARY, L. and BROWN, G. T. (2003). ‘Association between nurses’ education about research and their research use’, Nurse Education Today, 23, 556–65. MILLAR, J. and HAMES, V. (2003). ‘Using diagnostic assessment to enhance teaching and learning: a study of the impact of research-informed materials on science teachers’ practices.’ Paper presented at ESERA Conference, Noordwijkerhout, Netherlands, 19-23 August. NATIONAL COLLEGE FOR SCHOOL LEADERSHIP (NCSL) (2003). Backtracking Practice and Policies to Research. Nottingham: NCSL. NATIONAL EDUCATIONAL RESEARCH FORUM (NERF) (2000). The Impact of Educational Research on Policy and Practice: Sub-group Report. London: NERF. NUTLEY, S., WALTER, I. and DAVIEW, H. (2002). From Knowing to Doing: a Framework for Understanding the Evidence-Into-Practice Agenda [online]. Available: http://www.st-andrews.ac.uk/%7Eruru/KnowDo%20paper.pdf [20 August, 2005] NUTLEY, S., PERCY-SMITH, J. and SOLESBURY, W. (2003). Models of Research Impact: a Cross-sector Review of Literature and Practice (Building 37 Effective Research: 4). London: Learning and Skills Research Centre. OANCEA, A. (2005) ‘Criticisms of educational research: key topics and levels of analysis’, British Educational Research Journal, 31, 2, 157–83. OLIVERO, F., JOHN, P. and SUTHERLAND, R. (2004). ‘Seeing is believing: using videopapers to transform teachers’ professional knowledge and practice’, Cambridge Journal of Education, 34, 2, 179–91. ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT (OECD) (2002). Educational Research and Development in England. Examiners’ Report (OECD Review). Paris: OECD. PENDRY, A. and HUSBANDS, C. (2000). ‘Research and practice in history teacher education’, Cambridge Journal of Education, 30, 3, 321–34. RATCLIFFE, M., BARTHOLOMEW, H., HAMES, V., HIND, A., LEACH, J., MILLAR, R. and OSBORNE, J. (2003). ‘Evidence-based practice in science education: The researcher-user interface.’ Paper presented at the British Educational Research Association Annual Conference, September. RATCLIFFE, M., BARTHOLOMEW, H., HAMES, V., HIND, A., LEACH, J., MILLAR, R. and OSBORNE, J. (2004). Science Education Practitioners’ Views of Research and its Influence on their Practice. York: University of York. RICKINSON, M. (2003). ‘Reviewing research evidence in environmental education: some methodological reflections and challenges’, Environmental Education Research, 9, 1, 257–71. RICKINSON, M., ASPINALL, C., CLARK, A., DAWSON, L., McLEOD, S., POULTON, P., ROGERS, J. and SARGENT, J. (2003). Connecting Research and Practice: Education for Sustainable Development. Southwell: British Educational Research Association. RICKINSON, M., CLARK, A., MCLEOD, S., POULTON, P. and SARGENT, J. (2004). ‘What on earth has research got to do with me?’ Teacher Development, 8, 2/3, 201–20. RICKINSON, M. and REID, A. (2003). ‘What’s the use of research in environmental education?’ Paper presented at the Annual Meeting of the American Educational Research Association, Chicago, 21-25 April [online]. Available: http://www.nfer.ac.uk/publications/other-publications/conferencepapers/pdf_docs/AERAMR.pdf [28 September, 2005]. ROGERS, S. E. (2000). ‘A study of the utilization of research in practice and the influence of education’, Nurse Education Today, 20, 279–87. SANDERS, D., WHITE, K., SHARP, C. and TAGGART, G. (2005). Evaluation of the NERF Bulletin Trial. Phase Two Report. Slough: NFER. 38 SAUNDERS, L. (2004). ‘Doing things differently?’ Teacher Development, 8, 2/3, 117–26. SEASHORE LOUIS, K. and JONES, L. M. (2001). Dissemination with Impact: What Research Suggests for Practice in Career and Technical Education. Columbus, OH: National Center for Career and Technical Education. SEBBA, J. (2004). ‘Developing evidence-informed policy and practice in education.’ In: THOMAS, G. and PRING, R. (Eds) Evidence-Based Practice in Education. Maidenhead: Open University Press. SHARP, C., EAMES, A., SANDERS, D. and TOMLINSON, K. (2005). Postcards from Research-engaged Schools. Slough: NFER. SHEFFIELD, P. and SAUNDERS, S. (2004). Scoping Study Related to the Development of a Web Portal for Users of Education Evidence Bases. Unpublished report. SMITH, C., BINGMAN, M. B., HOFER, J., MEDINA, P. and PRACTITIONER LEADERS (2002). Connecting Practitioners and Researchers: an Evaluation of NCSALL’s Practitioner Dissemination and Research Network. Cambridge, MA: NCSALL. ST. CLAIR, R. (2004). ‘From abstract to action: how adult literacy and ESL educators make use of research’, Perspectives: The New Your Journal of Adult Learning, 2, 1, 49-61. ST. CLAIR, R., CHEN, C. and TAYLOR, L. (2003). How Adult Literacy Practitioners Use Research [online]. Available: http://wwwtcall.tamu.edu/orp/orp2.htm [25 May, 2005]. TAGGART, G., KENDALL, L., BENTON, T. and SHARP, C. (2004). Evaluation of the NERF Bulletin Trial. Phase One Report (Unpublished report). WALTER, I., NUTLEY, S. and DAVIES, H. (2003). Developing a Taxonomy of Interventions used to Increase the Impact of Research [online]. Available: http://www.standrews.ac.uk/~ruru/Taxonomy%20development%20paper%200 70103.pdf [25 May, 2005]. WILLIAMS, D. and COLES, L. (2003). The Use of Research by Teachers: information literacy, access and attitudes. Final Report [online]. Available: http://www.nerf-uk.org/word/Fullreport.doc?version=1 [3 October, 2005] WILSON, R. (2004). ‘Taking control: how teachers use research’, TOPIC: Practical Applications of Research, Issue 31. Slough: NFER. WILSON, R., HEMSLEY-BROWN, J., EASTON, C. and SHARP, C. (2003). Using Research for School Improvement: the LEA’s Role (LGA Research 39 Report 42). Slough: NFER. WITTROCK, B. (1991). ‘Social knowledge and public policy: eight models of interaction.’ In: WAGNER, P., WEISS, C.H., WITTROCK, B. and WOLLMAN, H. (Eds) Social Sciences and Modern States. Cambridge: Cambridge University Press. 40 Appendix 1: Database Search Strategy A number of relevant educational and social science databases were searched using terms from the relevant thesauri (where these were available) in combination with free text searching. The keywords used in the database searches, together with a brief description of each of the databases searched, are outlined below. In all cases, the searches were for the period 1999-2004 and were performed on 22 April 2005. BRITISH EDUCATION INDEX (BEI) BEI provides bibliographic references to 350 British and selected European English-language periodicals in the field of education and training, plus developing coverage of national report and conference literature. #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 Research utilisation Use of research (ft) Research utilization (ft) Research use (ft) Evidence use (ft) Evidence-based practice (ft) Information utilisation Theory-practice relationships Research (ft) #8 and #9 (ft) Denotes free-text searching BRITISH EDUCATION INTERNET RESOURCE CATALOGUE This is a database of information about professionally evaluated and described internet sites which support educational research, policy and practice. #1 Research utilisation ERIC ERIC is sponsored by the United States Department of Education and is the largest education database in the world. It indexes over 725 periodicals and currently contains more than 7,000,000 records. Coverage includes research documents, journal articles, technical reports, program descriptions and evaluations and curricula material. #1 #2 #3 #4 Research utilization Research use (ft) Information utilization Research (ft) 41 #5 #6 #7 #8 #3 and #4 Theory-practice relationships Educational research #6 and #7 SWETSWISE SwetsWise is a searchable database of almost 5,000 full-text journal titles from a number of publishers. #1 #2 Research utilisation Research utilization 42