Bedfordshire and Hertfordshire Postgraduate Medical School University of Bedfordshire Clinical educators (dis) engagement with online learning JISC funded survey Report to the Higher Education Subject Centre for Medicine, Dentistry and Veterinary Medicine (MEDEV) 2 July 2009 1
Judy McKimm , Nick Theobald2 and Clare Morris3 Introduction We are reporting on a small scale study, carried out between February and July 2008, exploring the factors underpinning engagement of medical and healthcare educators as learners with on-­‐
line learning, specifically that related to the PGCert and Masters’ programme in Medical Education run by the Bedfordshire and Hertfordshire Medical School, University of Bedfordshire. The study aims to improve understanding of those features and support structures which facilitate optimal engagement with online learning environments and help to improve the student learning experience. The objectives of the study are to:  Explore whether there are specific factors influencing engagement that are common to all clinical teachers (as opposed to other groups of learners)  Determine whether certain groups or individuals experience barriers to engagement  Identify those features within the VLE that support or hinder engagement and learning The study is part funded by JISC through a small grant from the HEA MEDEV subject centre. Ethics approval was obtained from the University of Bedfordshire Faculty of Health and Social Sciences Ethics Committee. The study was carried out by the three authors. JM and CM developed the questionnaire, JM carried out the initial analysis of results, NT (as part of a masters dissertation) carried out the literature review and further data analysis, including interviews. All three authors contributed to the writing of the final report. Context of study Online learning has grown in the field of education over the past decade and has encouraged the development of distance and blended learning courses. The main aim of the study is to analyse features of student engagement in the University Virtual Learning Environment (VLE), 1
Visiting Professor and MA (MedEd) course tutor, University of Bedfordshire
Visiting course tutor PgCert (MedEd), University of Bedfordshire
3
Associate Dean and MA (MedEd) Programme Lead, Bedfordshire & Hertfordshire Postgraduate Medical
School, University of Bedfordshire
2
1
exploring commonalities and differences in the type, nature, frequency and motivation of engagement between different student groups. The VLE (BREO: Bedfordshire Resource for Education Online) is based on the Blackboard ™ system and is used both as a ‘repository’ for a range of resources relevant to the course (course materials, links to relevant journals and articles) and for more interactive elements aimed at developing a virtual learning community through discussion forums, moderated reading groups and reflective diary ‘blogs’ (visible to module tutors and the individual student but not to other students). The Masters course is a blended learning programme. Students are from a range of health professional backgrounds and have diverse experiences as educators in terms of experience and time spent working in the field of education. This study focuses primarily on students from two cohorts – the 2006 entry and the 2007 entry -­‐ undertaking the two teaching units – ‘Medical Education in Practice and ‘Contemporary Perspectives’ which form the PGCert stage of the Masters programme. The results suggest that there is significant variation in the general use of Internet tools among our students and their use of the VLE. Students report greater satisfaction with use of the facility to access materials and course information but less satisfaction in the area of communication and peer interactions. Background Throughout the educational sector there is increasing use of Information Technology (IT) to support learning – variously described as e-­‐learning, online learning, networked learning, computer assisted learning, Internet learning, distributed learning, or web-­‐based learning (Ally 2004; Ruiz et al 2006). For this review, the definition used by Ally will be used: The use of the Internet to access learning materials; to interact with the content, instructor, and other learners; and to obtain support during the learning process, in order to acquire knowledge, to construct personal meaning, and to grow from the learning experience. Outside the academic sector, the growth of online learning in the vocational and workplace settings is also significant in both the public and private sector. In some instances this is as an adjunct to face to face teaching and in others it is the principal method of interaction between students and educators in distance learning courses. From the early development of the Open University in the UK in 1969, the use of technology to support distance learning, particularly for adults moved from television to increasing use of computers during the 1980s (McKimm et al, 2003). A number of drivers are responsible for the change in design and delivery of educational programmes: pressure for greater accessibility, an increasing number of students (Harden 2008) and the increasing expectations of students who are already familiar with the potential of the computer and the Internet to deliver multimedia tools to support learning. Furthermore, the use of such technology to facilitate interactions through communication and networking tools such as Skype, Facebook and Second Life (Sandars, Homer et al. 2008) has increased in this century with the development of ‘Web 2.0’ (McGee, Begg 2008) and Wikis. The Internet has evolved from simply being a tool for accessing and downloading static readable resources which are 2
‘posted’ by whoever administers the website and experience and engagement becomes more interactive so the users can add content, comment and potentially enhance and share knowledge. The Internet provides huge potential for meeting the demands of students and educators but educators need to keep abreast of new technology as it changes and evolves. Government bodies have embraced online learning and its potential. For example, The Higher Education Funding Council for England (HEFCE) published a 10year strategy for Higher Education (HEFCE 2005) with the aim: ‘to support the HE sector as it moves towards embedding e-­‐learning appropriately, using technology to transform higher education into a more student focused and flexible system as part of lifelong learning for all who can benefit’ The learning strategy produced as part of the NHS Plan (Department of Health 2001) called for a well developed, competent and educated healthcare workforce and identified e-­‐learning as central to the delivery of the required learning. The Department of Health launched the NHS University (NHSU) in September 2003 following the publication of this report but it was short lived, being merged to form the NHS Institute for Learning, Skills and Innovation only a year later. The plan for a University was finally scrapped in July 2005. Nevertheless the aims and objectives were carried through to the new organisation and the DH continues to invest heavily in e-­‐learning for all levels of NHS staff. There is increasing use of web-­‐based learning in medicine for the provision of Continuing Medical Education (Academy of Medical Royal Colleges 2007) – through sites such as BMJ learning, doctors.net and others. These developments are replicated in other health professions CPD provision and in the recording of achievements and experiences through electronic professional development portfolios. The attraction of web-­‐based learning in the context of health professionals’ education is that resources can be available at any time and in any location (with Internet access) – often described as ‘just in time’ and ‘just for you’ (McKimm et al. 2003; Harden 2005; Ally 2004; Kim, Liu et al. 2005) and it is helpful if healthcare educators are familiar with the perspective and experiences as a learner as well as from the other side as designer and/or facilitator. In the context of increasing concern in the profession about commercial sponsorship of CME meetings (Moynihan 2008) and about the effectiveness of traditional CME meetings based upon a preponderance of didactic lecture format presentations (Bloom 2005, Davis, Thomson et al. 1995, Davis, O'Brien et al. 1999, Password 1998) and with concerns about the costs and environmental consequences of International conferences (JISC 2007b) it is likely that online CME (including online conferences) will continue to grow. Online learning is a different environment from the traditional classroom or lecture theatre based setting. The potential for learning at one’s individual pace, at a convenient time and without spending time travelling is welcome, as is the opportunity to facilitate reflection (Petrides 2002) but the loss of face-­‐to-­‐face networking opportunities needs to be balance against this. The ability to include a variety of material in several different formats, including multimedia, video, sound recordings as well as text can make the learning environment more stimulating and cater for a range of learning preferences (Anderson, Elloumi 2004; McKimm et 3
al. 2003). Despite the drawback of reducing face to face contact, the opportunity for dialogue through ‘asynchronous’ discussions and interactions are greater, i.e. transmission and receipt of information do not occur simultaneously (Ruiz 2006). However, the lack of an immediate response can be a source of frustration to some (Song et al. 2004). Advantages of Technology in Medical Education (adapted from AAMC, 2007) • Instruction can be tailored to individual or group needs • Uncoupling learning from place and time • Greater learner control of educational experience • Opportunities for repetition and practice • Economies of scale • Standardisation of instruction and assessment • Safe controlled environments and authentic contexts for learning (Virtual patients) • Documentation of learner behaviour Online learning also gives us the unique opportunity to collect data on student interaction and use in a way which was not previously possible. Since one of the challenges is to gain students interest and keep their attention, the ability to allow educators to more closely monitor what materials are being accessed can bring enormous benefits in a way not previously available to us. This does however raise some ethical considerations. Correlating such behavioural data with material obtained from interviews and questionnaires can then give a greater insight into the experiences of learners. Literature review The National Library for Health (www.library.nhs.uk) was used to access a range of search engines including CINAHL and PUBMED/MEDLINE. ERIC www.eric.ed.gov and Web of Science was searched together with general search engines such as Google Scholar® and Dogpile®. Search terms utilised: ‘e-­‐learning’ ‘online learning’ ‘e-­‐learning’ ‘online learning’ ‘computer assisted instruction’ ‘webbased learning’ Internet adj. learning ‘health’ ‘educators’ ‘clinical adj. educators’ Bibliographies and reference lists of selected articles were also used to look for relevant items. In searching through the published literature it became apparent that the majority of papers in the Medical Education literature are written about the design, technical and pedagogical aspects of e-­‐learning rather than the factors influencing engagement from the perspective of the student or learner. Many are anecdotal or even sometimes promotional (Boerema, Stanley et al. 2007). This was identified also in a review in the general Higher Education context (Ali, Hodson-­‐
Carlton et al. 2004). 4
Research about online education can be found in a wide variety of publications (in print as well as electronic) including those in the fields of communications, higher education and educational technology as well as in the subject disciplines (Wallace 2003). Further investigation into the general rather than medical/nursing literature on education did therefore reveal more studies. The specific area of focus for this study, exploring the factors influencing engagement with online learning for clinical educators (as learners), does not seem to have been addressed. A scoping study (Sharpe, Benfield 2005) concluded that there was a powerful argument for further exploration of student perspectives on e-­‐learning, and recommended more recognition of the holistic nature of the learners’ experiences, the use of a blend of methodologies to capture the motivations, beliefs and intentions of learners and identification of the characteristics of expert e-­‐learners. The later report produced by the Joint Information Systems Committee (JISC) of the Higher Education Funding Councils for England, Wales and Scotland (JISC 2007a) looked at learners narratives – thus enabling them to assess the varied and individual ways in which the students used the technology and produced a number of guidelines for institutions and designers. One of the principal recommendations was to use more than one source of data to enable triangulation between the outcomes. The authors comment that mixed mode studies are more likely to provide trustworthy results. Researchers in Hong Kong looked at positive and negative experiences of online learning among post registration nursing students (Sit, Chung et al. 2005) through the use of self administered questionnaire. The students were asked to answer questions by scoring on a 4 point Likert scale as well as three open ended questions. The development of this questionnaire was based on the findings of a small focus group followed by a pilot exercise undertaken beforehand. 198 students returned the questionnaires (49.5% response rate) and a significant number (43%) reported being dissatisfied with the online learning they had experienced. Despite this the majority of respondents reported positive emotions about being able to work at their own pace and at a time of their choosing, take responsibility for their own learning and navigating the resources. The principal reasons underlying the dissatisfaction were mostly around isolation and ‘inadequate opportunity for human contact’ and the authors recommended that a blended approach combining online learning with supplementary onsite classroom meetings might address this issue. They felt that establishing peer support and developing academic dialogue and socialisation were major challenges. Oliver and Herrington (2003) suggest that although an independent learner who is to engage effectively with an online course will need to have a relatively high level of technical skills to minimise barriers to success, it can be difficult to predict some of the technical problems which will be encountered by the learners. Nevertheless the impact of these problems can be significant. Barriers can be minimised if the system runs smoothly with minimum downtime and if scheduled maintenance can be planned with this in mind. Miller et al (2003) studied predictors of online engagement and participation using five propositions derived from a Technology Acceptance Model and a Computer Self Efficacy Model (Marakas, Yi et al. 1998; Agarwal, Sambamurthy et al. 2000; Joo, Bong et al. 2000). Sixty six students were surveyed to test these propositions using an independent variable of time spent by each student working on the online course modules. The data supported the propositions that Perceived Ease of Use and Perceived Usefulness were positively associated with the amount of time spent on the course. On the other hand, constructs of Subjective Norm and Computer Self Efficacy were not associated with engagement with the online course. The 5
authors suggest this may in part be due to the necessity of using the computer in order to complete the course regardless of their own feelings about their level of self efficacy. Harden and Laidlaw (1992) listed criteria for effective continuing medical education (CME). The CRISIS model proposed by these authors listed six attributes (Convenience, Reliance, Individualization, Self assessment, Independent Learning and Systematic Approach), all of which can be applied across to an online learning environment (Harden 2005). When originally proposed the technology to deliver many of these aspirations was not widely available, making it more difficult to deliver CME. Although the author suggests that the increasing use of web-­‐
based learning for CME meets all the criteria listed above – and advocates a global approach, he makes no allowance for the barriers to access present in many less developed countries or for the role of the learner in the process. Wallace (2003) reviewed the research on interactions between teachers and students in online leaning in a Higher Education context. He identifies the particular benefits in the HE context of students learning from each other and the way in which the online environment can facilitate this interaction as well as the need identified by students for discussion and feedback from the tutor or lead educator. He goes on to examine the literature on student participation and collaboration and concludes that characteristics vary according to the purpose and motivation as well as the duration of the online community. He speculates that the shortlived and involuntary nature of groups in formal learning environments is very different from the ideas of community as described by Wenger’s ‘communities of practice’ (Wenger 1998). He cites Palloff and Pratt (1999) who suggest that there is a need for social and emotional space as well as the academic and intellectual contact. However, Moule (2006) studied final year healthcare students studying an interprofessional module comprising nursing, radiography and radiotherapy students in 2004 and concluded that the essential elements of communities of practice could be developed when learning online, with some specific additional support. Childs et al (2005) looked at barriers in e-­‐learning for health professionals and students using a questionnaire survey of 149 students followed by semi structured telephone interviews of learning providers. They comment that with the rapid expansion of e-­‐learning in the NHS there is ‘a pressing need to understand’ the issues and the barriers and questioned whether the barriers might differ according to the profession or role of the individual within the healthcare environment. Although the authors acknowledged that the qualitative data which came from the questionnaire and interviews would not be generalisable, the findings could provide pointers to others. They identified a number of barriers which they classified as Organisational, Economic, Hardware, Software, Support, Pedagogical, Psychological and Skills issues. The overall views of the learners were very positive. However, once again the lack of face to face contact was frequently cited as a negative factor and the need for skills training and access to appropriate technology was highlighted. This group also brought out significant issues in time pressures in relation to needing their employer(s) to grant specific time to engage with online learning – a factor not described in earlier studies according to the authors. A team from Texas reported a study where they investigated the relationship between technological problems and the evaluations the students gave of their online teaching (Tallent-­‐
Runnels, Lan et al. 2005). Their earlier study 2 years earlier had suggested that in the undergraduate context, the greater the number of technological problems experienced by students, the lower would be the evaluation of the online course. This later study of 131 6
graduate students used an online questionnaire survey with Likert scales and concluded that the students who reported more issues with technological problems rated the course and the instructor more highly – which is not intuitively what might be anticipated. However, the authors suggest that the increased interaction with the tutor by those students having problems with the technology paradoxically increased their level of satisfaction. Seventy six graduate students from the University of Georgia were surveyed on their online learning experiences (Song, Singleton et al. 2004) through mixed research methodology comprising a questionnaire followed by face to face interviews with 9 participants. Notably 88 percent of the survey group were female; it is unclear whether this mirrors the gender balance of the students on this course. Once again lack of community was reported in questionnaire responses as well as the interviews together with technical problems and a lack of clarity of instructional goals. They concluded that effective design should focus on technological aspects as well as goals, objectives and expectations. Helping students with time management strategies and in establishing community and connection in an online context was also considered to be very important and they suggested further research into this area would be beneficial in advancing best practice in online collaborations. Ally (2004) lists successful attributes for online learning in the context of educational theory – and the Constructivist school in particular as he believes the shift towards constructivist learning should be reflected in the design of online course material. He lists successful attributes as below but only from the pedagogical perspective of the designer with no reference to the experiences or perspectives of the learner: • Learning should be an active process • Learners should construct their own knowledge • Collaborative and cooperative learning should be encouraged • Learners should be given control • Learners should be given time and opportunity to reflect • Learning should be made meaningful • Learning should be interactive to promote higher level learning He concludes by calling for the flexibility to be inclusive for behaviourist, cognitivist and constructivist theories and to appreciate their contribution to the design of online materials. This view is also supported by Ali, Hodson , Carlton et al. (2004) and Huang (2002). Dyke et al (2007) also examine the philosophical and theoretical approaches behind learning theories and their application to a variety of potential e-­‐learning applications, suggesting the three domains of Thinking/Reflection, Experience/Activity and Conversation/Interaction all needed to be considered to encompass the complex and multifaceted needs of the contemporary learner – and that e-­‐learning has the potential to nurture reflection, experience and interaction and “fuel the fire of learning”. Along with many authors they affirm the role of online learning in interaction – one of the themes to come out of the survey. Methods The sample for this study was a group of 59 students from the 2006 and 2007 entry cohort, each year cohort had participated in two teaching units, thus providing data from students studying four teaching units. These units (‘Medical Education in Practice’ and ‘Contemporary Perspectives in Medical Education’ were chosen as they were the units which comprised the Postgraduate 7
Certificate in Medical Education as well as the initial two units for the students continuing to study for the Postgraduate Diploma or the Masters degree. Two cohorts of students had completed these at the time of the survey allowing comparison between them. The study includes three key activities: 1. A questionnaire survey, exploring students experiences of on-­‐line activity generally and on-­‐
line learning specifically 2. Analysis of tracking data relating to individual engagement with the on-­‐line learning environment which supports the programme in order to identify: the most popular features/tools of the on-­‐line environment; patterns of engagement; those who are most / least engaged for purposeful sampling for focus group interviews 3. Pilot interview with a student in order to explore factors that influence engagement and/or disengagement, to identify ways in which learning experiences might be enhanced and to identify future directions for research. Excel was used to collate and analyse the quantitative data. Data arising from free text responses to the survey questionnaire and the interview was coded by the researchers to elicit key themes. These activities are described in more detail below. 1 Questionnaire survey A questionnaire survey was carried out between February and April 2008 of 59 students registered on the two postgraduate certificate level units of the medical education programmes in 2006 and 2007. The questionnaire (see Annex 1) was devised to include respondents’ perceptions of their general confidence and engagement in email and other online activities as well as explore specific aspects relating to the ‘BREO’ (Bedfordshire Resource for Education Online) University wide VLE used to support all undergraduate and postgraduate programmes. The questionnaire included both closed and open questions in order to explore the experiences of individuals from the accounts they provide. The questionnaire was distributed initially as a Word document via email in February 2008 by one of the researchers who was not directly engaged in the programme, two follow up emails were sent to capture non-­‐respondents in March and April 2008. In response to feedback that the Word version was time-­‐consuming to complete, the questionnaire was also included on an online survey website (Survey Monkey™) in March 2008 and the URL sent to all non-­‐respondents so they had the choice whether to complete this online or through the Word document. 2 Analysis of tracking data relating to individuals’ engagement with the VLE Tracking data was collected from the University webserver detailing the number of times individual students accessed a particular section of the VLE, the times of day and the days of the week whilst they were registered for the two units. The tracking information provided 8
quantitative data. Information relating to the length of time that students spent on BREO was not available, just the number and date/time of each ‘hit’. 3 Pilot interview Following analysis of the questionnaire survey data, some respondents were invited to participate in follow up interviews or focus groups as a second stage of the study. The responses to the request were very poor as this coincided with heavy assignment and other clinical workload commitments. It was therefore decided to carry out an exploratory pilot interview in order to identify areas for future research outside the scope of this small scale study. This interview was carried out in June 2008 with a nurse working as a training education manager who had entered the programme midway through the 2006/7 programme and therefore had engaged with students on both cohorts in this study. The interview schedule comprised a short series of open questions aimed to explore further the themes identified from the questionnaire survey analysis: general use of computers and IT; use of the VLE social networking tools, including discussion forums and blogs; navigation and reliability/access. Methodological issues A number of issues affected the study, these have been taken into account when interpreting the results. The original study design included a questionnaire survey of all students over two academic years, one or two focus groups of students and an analysis of the webserver usage data. However, due to practical constraints and staff movements, the original design had to be modified. Sample We used four groupings to categorise respondents by professional group: hospital doctors; GP/community doctors; nurses/midwives and ‘others’. This may not be meaningful in terms of their respective roles as educators, or their experience. We did not analyse by gender, proficiency in English language or age, which although possible, may not have relevance unless you control for other factors including educational role, experience and profession. Questionnaire survey The design of questionnaire was based on features of BREO, a literature review and descriptive data obtained informally from discussions with the students and staff on the course. We did not pilot the survey questionnaire as have other studies (eg. Sit, Chung et al., 2005) as we wanted to capture the first cohort of students before they finished their modules and left the university. It was not possible to make comparisons between the two cohorts surveyed. The questionnaire survey was sent to all students at the same time for logistical reasons. It surveyed one cohort who had been on the course for 18 months and had therefore undertaken around 5 teaching modules, whereas the second cohort had only been using the VLE for 6 months. The two cohorts also varied between each other considerably in their gender balance and in their professional makeup. Aggregated data was therefore used as the basis for analysis. Self reports and self assessment of any parameter are not always reliable indicators, particularly if there is any ambiguity in the scale items. ‘Frequently’ or ‘Seldom’ may be subject to different 9
interpretation if there is no accepted reference point. However, when comparing the webserver data, those who rated themselves as very low users were generally low users but the correlation was not very strong for those who believed their use was ‘frequent’ (some were below average). They could not know how they would compare with their peers. In future research, care must be taken to define the self-­‐reporting scales quantitatively and to align them with the way in which webserver data is reported. Focus group and interview We were unable to carry out a focus group as logistically trying to arrange a focus group proved too challenging. Being a blended course meant that contact days were few – focus group timings proved difficult to organise as the days were naturally already fully timetabled. The earlier cohort was already in the latter part of their course and no further contact days were planned. Individual interviews duplicate resources and limit interactions between students when compared with focus groups. We therefore carried out a pilot interview, the results of which are included, although data is taken as being one person’s views and not indicative of a wider group of representative students. Nevertheless useful information was gathered and it was consistent with both his access data and his questionnaire responses. Issues raised in the interview provide a basis for further exploration. Webserver data The description of areas in the questionnaire did not exactly match the categories in the report from the webserver – and the teaching units differed slightly in their online content so some sections were not used in all units. This made it difficult to make detailed comparisons. Again, aggregated data was used for comparison between similar aspects. Only being able to measure the number of times a student visits a particular page has serious limitations. A limitation of our study was that time spent online (and in particular areas) is not recorded, this would have provided more insight into the use of different areas of the VLE. The data suggests the second cohort were accessing the VLE less than the first cohort yet this is directly at odds with the impression of the course tutor (Morris, C., 2008, pers.comm., 23 September). Even if data were available relating to time spent online this would not necessarily reflect an accurate measure of the quality of that interaction – especially in the context of allowing learners to work at their own pace – the length of time may simply reflect a slower learner and they may have logged on and then not be actively learning. The apparent greater use during the weekdays rather than weekends may also not be truly representative – for example students may visit the site more often during weekdays but for shorter periods of time – and spend longer onsite during weekends. Without access to data on time spent online this is impossible to ascertain. Measuring only the number of ‘hits’ on the server distorts the measure of student engagement, for example it will include a student going through and back again as they navigate around the site. Less experienced or less technically astute students may paradoxically show higher ‘hit’ numbers as they stumble around the site rather than the more experienced navigators who find the relevant part of the course quickly but then stay longer and contribute more. For example, a student who appeared only to have visited BREO once during her second module had actually a higher than average hit rate for the first module she studied. This is very difficult to account for and it is very hard to believe a student would only access the VLE once during the entire teaching unit. Nevertheless, access to the Virtual Library can be made without passing through 10
the BREO VLE and much of the Internet based material can be obtained through other routes and/or the NHS Library, again making the data supplied insufficiently robust to allow sensitive and robust analysis. There were other anomalies. For example, two students accessed an area titled ‘student rosters’ many times more than others – one visited 246 times in the course of one teaching module – over 50% of her score. The student works for the same University so it is possible that the discrepancy relates to her dual roles. Another student accessed the area ‘drop box’ 200 times in one teaching module – significantly higher than his peers and it is hard to find an explanation as to what caused this. The number of times a student visits an area of the webserver may not reflect the usefulness of that particular section but may reflect popularity alone. Results 1 Questionnaire survey The questionnaire survey was carried out between February and April 2008 by one of the researcher and provided data for comparison with tracking data and the pilot interview. Here we report summaries of the main results from the survey. Table 1 – Characteristics of respondents and non-­‐respondents (all students) ALL STUDENTS Professional Group Respondents Non-­‐respondents Male Female Total Male Female Total Doctor (Hospital) 12 3 15 8 3 11 Doctor (GP/Community) 3 3 6 2 2 4 Nurse/Midwife 2 9 11 0 5 5 Other/Unknown 2 1 3 3 1 4 TOTAL 19 16 35 13 10 24 Response Rate (all students) = 59% Table 2 – Characteristics of respondents and non-­‐respondents (2006/2007 cohort) 2006/2007 COHORT Respondents Non-­‐respondents Professional Group Male Female Total Male Female Total Doctor (Hospital) 3 1 4 6 2 8 Doctor (GP/Community) 2 1 3 1 2 3 Nurse/Midwife 1 7 8 0 3 3 Other/Unknown 0 0 0 1 0 1 TOTAL 6 9 15 8 7 15 11
Response Rate (2006 cohort) = 50% Table 3 – Characteristics of respondents and non-­‐respondents (2007/2008 cohort) 2007/2008 COHORT Professional Group Respondents Non-­‐respondents Male Female Total Male Female Total Doctor (Hospital) 9 2 11 2 1 3 Doctor (GP/Community) 1 2 3 1 0 1 Nurse/Midwife 1 2 3 0 2 2 Other/Unknown 2 1 3 2 1 3 TOTAL 13 7 20 5 4 9 Response Rate (2007 cohort) = 66% The response rate was higher for the 2007 cohort and the 2006 cohort contained more females (and more who defined themselves as nurses or midwifes) than the 2007 cohort. a) General use of computers and the Internet 89% of respondents access the Internet on a daily basis and 83% describe their confidence in using the Internet as ‘Very’ or ‘Fairly’ confident. Familiarity with a Social networking and Internet tools varies, see Table 4. Table 4 – Use of Internet and related communication tools Tool Unaware of this Aware but Aware – use Aware and tool never used infrequently regularly use Email 0 0 0 100 Podcasts 11 40 31 17 Blogs 9 46 43 3 Wikis 20 20 49 11 Webcams 14 43 29 14 Video conferencing 6 57 29 9 RSS feeds 40 40 15 6 Facebook 0 43 37 20 Flickr 54 29 11 6 Skype 26 46 11 17 Second life 51 37 9 3 Note– percentages are rounded up/down so totals may not equal 100 Responses of 20% or above are highlighted 12
Only email was used regularly by all students. All the other listed tools were used regularly by fewer than 20% of respondents – in many cases by fewer than 10%. 51% had never used podcasts and over half of the respondents were unaware of Second Life®. 80%had never used RSS feeds and 40%had never used Wikis. b) Experience of online learning 40% of respondents had no previous experience of using an online environment as a learner. Those who reported previous engagement as a learner gave examples of online learning activities including (number reporting examples):  Royal College provision for CPD (6) o eg. RCPsych online CPD, Royal College of Surgeons e-­‐Step course, RCP online video mediacasts of renal association/RCP  Various online courses on specific topics (6)  BMJ online learning programmes (5)  Through courses at other Universities (5)  e-­‐learning modules for clinical and educational topics, medical websites (2)  Google search engines, online references (2)  Doctors.net e-­‐learning modules for CPD (2)  inductions online for new hospitals 66% of respondents had no previous experience of using an online environment as a teacher. Those who reported previous engagement as a teacher gave examples of online learning activities including (number reporting examples):  as part of their University role (7)  as part of their role in postgraduate education activity (5) o eg, Royal Colleges, Deaneries, professional journals Activities included developing case presentations, administering websites and developing materials for a VLE. c) Usage of BREO All respondents reported that they had used BREO, although three replied that they had done so only ‘very seldom’. The main reasons stated for using BREO were to: 



access e-­‐journals and library resources to access course and assessment information to submit assignments (including reflective diary) because it is a course requirement Only a minority of respondents reported using it for interactive tools and communication with other students. Respondents were asked to identify their main forms of general engagement with BREO, see Table 5: 13
Table 5: Main forms of engagement with BREO Form of engagement To find information about the course To access learning materials linked to study days To access readings and articles from e-­‐journals To submit assignments As a portal (access link) to other sites/resources To find information for my own learning To engage with on-­‐line learning tutorials linked to assessment activity To participate in online discussions and debates To capture my learning in the reflective diary space To communicate with other students As a ‘watcher’ ie to see what others are doing or saying To seek feedback on my writing (e.g. in the diary space) To communicate with tutors Percentages 87 87 80 64 49 49 49 46 44 41 38 36 26 Additional benefits respondents cite in the free text responses relating to engagement in the VLE include the ability to work at any time and at their own pace; easy access to materials and a way of gaining feedback from tutors on reflective writing through the diary space. We asked students to specify which particular features of BREO they had used and to indicate their usage, see Table 6. Table 6: Use of specific BREO features (% respondents)
BREO features Materials from contact days Preparatory materials for contact days Follow up materials for contact days Announcements, keeping up to date with course Information on assessments Link to e-­‐journals and search engines Admin info: handbooks etc Quick links to useful websites Programme, calendar and timetables Link to library Digital assignment drop box Wiki to develop shared resources Reflective diary/log To send emails to tutors and students Group spaces linked to learning sets/group activities Discussion boards Moderated discussions with ‘experts’ Other (please specify) None Frequently 88 88 85 79 62 59 56 47 47 44 Seldom 10 11 13 18 36 3 35 32 29 26 Once 2 1 1 3 1 2 7 15 8 13 Never 0 0 1 0 1 35 2 6 16 17 38 26 26 21 18 14 5 26 21 41 35 9 22 12 29 13 50 26 46 12 9 2 62 38 10 13 9 47 We asked which features of BREO have been most useful, see Table 7 14
Table 7: BREO feature specified as most useful (% respondents) BREO features Learning/course materials [always accessible from anywhere] Link to e-­‐journals and search engines Announcements, keeping up to date with course Preparatory materials for contact days Link to library Quick links to useful websites Digital assignment drop box [when it worked!] Reflective diary/log [tutor feedback good, helped to write assignment] Follow up materials for contact days Wiki to develop shared resources Admin info: handbooks etc Information on assessments Group spaces linked to learning sets/group activities Materials from contact days Ability to access support Discussion boards To send emails to tutors and students Online data repository % citing 50 41 26 26 26 24 24 21 18 18 15 15 9 9 6 6 3 3 When asked about reasons for not engaging with BREO a variety of responses were received (see Table 8). Table 8: Reasons for not engaging with BREO: themes derived from free text responses to open questions: Reasons % citing Lack of time I communicate in other ways with tutors and students I prefer to meet people face to face or use email or phone Too slow and ‘clunky’ Hard to access BREO from work (eg.NHS firewalls) Hard to use Lack of access to a computer I don’t like online learning Time spent on BREO could be better spent on other types of learning Information on there of little or no use Doesn’t seem relevant to my learning 51 41 31 23 21 18 10 10 8 3 0 We asked which features of BREO had been least useful, see Table 9 Table 9: BREO features cited as being least useful BREO features Numbers 15
cited 13 10 9 8 3 4 3 2 2 2 1 1 1 1 1 Discussion boards Lack of engagement with other students [in activities, link to *above] To send emails to tutors and students [easier to use own/work emails] Reflective diary/log [easier to do it on PC for assignment] Link to library [hard to use, didn’t know how] Wiki to develop shared resources Admin info: handbooks etc [used printed copies] Group spaces linked to learning sets/group activities, ALS General announcements, University home page [not relevant] Link to e-­‐journals and search engines Quick links to useful websites Digital assignment drop box [didn’t work] Too many accounts [email etc] Blogs Time/access through firewalls We asked two questions relating to possible improvements or enhancements to BREO: Q 19 asked: are there any other features you’d like to see in BREO? Q20 asked: what support, features or functionality would help you make more use of BREO? Responses to these questions are summarised in Table 10 Table 10: suggestions for improvements or enhancements to BREO Additional features/improvements BREO features Spell check Easier navigation, site layout [drop down menus, more intelligent interface, lists of databases] Improve links between pages/internal sites [especially library, searching, database access] Improve efficiency and speed of BREO More podcasts and ‘portable’ reading [eg for car or travel] Better discussion board Include student offers on hardware and software FAQ section [re BREO ‘how to’ so not reinventing the wheel] RSS feed [subscribed, that tells you when things are uploaded or gives new information] Email updates [to home/work, that give announcements etc] Easier access without multiple accounts/separate passwords More support for students via tutors/materials available through BREO *Hands on introductory session on BREO, more practical sessions on a PC with expert assistance and online help Links to more websites and e-­‐journals, especially full text Set times for online tutor support and groups discussions More specific course materials [to avoid searching] Tutorials on how to use webcams and videoconferencing More use of wikis Q19 Number citing Q20 Number citing 1 4 1 8 3 3 2 2 1 1 1 1 2 0 0 0 See below* 1 1 1 0 1 7 4 1 1 1 1 1 0 16
More use of blogs [as course activity to learn how to use it] Examples/extracts of previous assignments [to get us started] 1 1 0 0 There were inconsistencies in the responses. Most had found the available information and links the most useful part and discussion boards least useful but despite this, many stated that communication and interaction were the ‘main purposes’ of BREO when asked. In a similar way, although most students hadn’t engaged with the reflective diary, this was cited as a beneficial tool when asked about the main purposes of the VLE. When asked to specify specific needs or barriers to using BREO, 28% mention unreliable access and 23% mention poor functionality. However no students mention access or functionality as a barrier for general online learning, indicating this was perceived to be a local issue to the BREO experience rather than generalisable to all online learning. Analysis of data using Excel indicates that those reporting greater confidence with using social learning/internet tools also had the greatest familiarity and widest experience of such tools. Data analysis did not reveal any significant differences between professions or cohorts. A number of questions in the questionnaire survey asked students to reflect on the purpose, relevance and role of online learning to learning: both for themselves and on a more general basis. We asked: As a learner on the MA programme, what do you think are the main purposes of BREO? Table 11: Main purposes of BREO Main purposes of BREO To find information about the course To access learning materials linked to study days To access readings and articles from e-­‐
journals To submit assignments As a portal (access link) to other sites/resources To find information for my own learning Quotes from free text responses course information; online repository of course documents as I often find with time for my own learning that I am in the wrong place at the wrong time, and yet there is always (usually) a computer to log onto and access resources; Course guide; information about the course; access to course information; Mainly it comes across as conveying material for the assessments, contact days etc; access to online learning; as a communication tool re contact days; access to learning material; Access to journals, library facilities; depository for e-­‐resources; portal for journals; data repository submission of assignments; e-­‐library fantastic; a tool for accessing information signposting of resources; give us avenues to explore and expand learning; an excellent resource for information; Providing access to other resources not immediately otherwise available to the learner. To engage with on-­‐line learning tutorials linked to assessment activity To participate in online discussions and group discussions; attempt to use as discussion board; Enable group debates discussions; point of contact with teaching staff (especially via Blog which allows reassurance that you're barking up the right tree! 17
To capture my learning in the reflective diary space Repository for my own learning materials To communicate with other students As a ‘watcher’ ie to see what others are doing or saying To seek feedback on my writing (e.g. in the diary space) To communicate with tutors A one stop shop for the course, communications and associated learning materials/resources To engage students in a an educational experience and learning environment To engage students in the experience of online learning as part of their educator skills development Support reflective diary collation To keep individual material (blogs, e-­‐portfolios, and so on) so they can be accessed from wherever in a flexible way and at any time Interaction; provide a common forum for access to other group members; to facilitate group activities; to encourage interaction between the students; to provide a forum that students from all backgrounds could exchange ideas; It did allow you to make connection with students who naturally had links locally with study groups. The idea of interactive BREO components were ideal, to explore and debate in the group at a distance. You do need a critical mass of involvement to make it viable as I found a few early adopters tailed off as did the discussion. it may have been a reflection on the group's professional background and learning style feedback; formative assessment; ; vehicle for tutor feedback; communication with tutors; keeping in touch with course tutors; constant means of contact between tutors and students without the need to meet face to face central communications hub; to have a regular means of communication and updates. BREO makes me oriented for what I need to learn, what to do and how to do it. It makes me feel I am in a learning environment in relation to the courses of medical education; to facilitate a group learning environment; an aid to learning; as part of blended learning; Trying to get the learners to engage in discussions, supporting each other; to enhance/help with the learning experience; To allow students to continue to communicate and learn from each other outside of the teaching days. to encourage reflection; opportunity for collaborative learning, As a learning tool to get us more engaged with e-­‐learning as this will play a bigger and bigger role in medical education; new ways of learning; to be introduced to creative examples of teaching methods that we could consider using in our own teaching practice. To become familiar with IT; provide an online learning facility; to develop other on line skills eg wiki Support ; To support learning between contact days; To support learning according to the needs of the individual learner; support for learning; to inform and educate learners based away from the university site; to make life easier for the student to gather resources, gain feedback and remain in touch with other members of the group with discussions, group work and email -­‐ it also saves time when you get the hang of it Disseminate information and link to resources particularly useful for part time students with geographical barriers. 18
We also asked: As a learner and teacher, what do you think is the potential or actual value of BREO? The majority of responses to this free text question were very positive in identifying potential benefits and value of BREO. Responses included the following aspects: (number of respondents citing aspect in brackets) Actual value For the learner:  communication and connectedness, reduce isolation, especially for distant learners (12)  access to repository of resources, including course information (8)  to facilitate group, collaborative and shared learning, trading of knowledge (8)  providing information to help prepare and extend learning (5)  learners can learn in own time and pace, open access (3)  to help prepare educators for ‘modern life’ as a teacher and keep pace with educational technology (2)  essential to the course, reliable ‘one stop’ information source (2)  to submit assessments quickly and easily (2) For the teachers:  for teachers to monitor learning (1)  less printing for tutors/university (1)  more time efficient for the tutor (1) Potential value for learners:  As a portal (2)  As an interactive learning tool (5)  To encourage ‘non technophiles’ to engage with e-­‐learning (1)  To have access after the course has ended (1)  For social networking (4) We asked: in what ways do you think that online learning is appropriate and necessary for clinical educators as learners? Respondents identify the following:  62% -­‐ it is useful, necessary, appropriate, essential [but in conjunction with other forms of learning]  24% -­‐ it is important that educators have the skills [to keep up with students etc]  21% -­‐ it enables engagement with tutors, discussion, debate, access to experts  18% -­‐ ease of access: 24 hour access, expectations from students  18% -­‐ positive aspects [general] We asked two questions of respondents relating to needs or barriers to online learning: Q16 – please specify needs or barriers to your using BREO and online learning 19
Q22 -­‐ please specify specific needs or barriers to clinical educators and students using online learning Responses to both questions are summarised in Table 12 below Table 12: Needs and barriers to online learning Needs/barriers How to see relevance of online learning to future work, learning [eg. technical ‘know how’, engagement of organisations, communities, educators] Lack of time Lack of experience/familiarity with technology/BREO/VLEs Personal learning preferences re online learning [does lack of engagement mean that some are disadvantaged?] No barriers/needs Limited or intermittent access to BREO [site unavailable, multiple passwords] Poor functionality of BREO [slow, difficult navigation, ] Lack of or limited access to internet [eg from work, NHS firewalls etc, slow speed] Lack of or limited access to a computer [eg at work, clinical settings] Lack of engagement of other students Lack of variety of online learning activities/tools in course Q16 Q22 Numbers cited Numbers cited 1 8 6 4 3 6 6 5 N/A 11 5 9 2 3 1 3 3 3 0 0 2 Analysis of tracking data relating to individuals’ engagement with the VLE Analysis of actual engagement with the online learning environment was carried out by accessing data (in the form of ‘hits’) from the webserver for two teaching units for the 2006 and 2007 cohorts. The gender and professional group for each of the four teaching units are presented in Table 6. Seven students who responded to the questionnaire survey did not participate in any of the four modules surveyed in the webserver access part of the study but joined the course later. It was not possible to compare the responses these students gave to their recorded use of the VLE. Both the questionnaire and the webserver access data agree that students are accessing the content and materials sections of the VLE actively but less so for the areas relating to communication and forums. However the questions asked categorised areas of the site slightly differently to the breakdown given for the webserver data so it was not possible to completely match these. There was a huge difference in the number of hits between individual students on the same course. For one course, one student only accessed the announcements area once and appears not to have made any further interaction with the site whatsoever. Another student on the same module scored 598 hits. 20
Table 13: Gender and professional group for each of the four teaching units Medical Education in Practice 2006/2007 2007/2008 Professional Group Male Female Total Male Female Total Doctor (Hospital) 7 2 9 8 4 12 Doctor (GP/Community) 3 2 5 2 3 5 Nurse/Midwife 1 10 11 0 4 4 Other 1 0 1 2 1 3 TOTAL 12 14 26 12 12 24 Contemporary Perspectives in Medical Education Professional Group 2006/2007 Male Female Total Male 2007/2008 Female Total Doctor (Hospital) 6 3 9 7 3 10 Doctor (GP/Community) 3 2 5 2 2 4 Nurse/Midwife 1 10 11 0 2 2 Other/Unknown 0 1 1 3 2 5 TOTAL 10 16 26 12 9 21 Table 14: Student access by Cohort and area of VLE (number of ‘hits’) Medical Education in Practice 2006/2007 [26 students] Announcements 764 2007/2008 [24 students] 824 Communication 1049 362 Discussion Board 1364 448 Groups and Group forum 757 29 Total 3934 1663 Mean (for all areas) 238 112 Median (for all areas) 201 80 11 571 12 473 2006/2007 [26 students] 935 2007/2008 [21 students] 977 Range Contemporary Perspectives in Medical Education Announcements 21
Communication 878 69 Discussion Board 1033 324 Groups and Group forum 1478 595 Total 4324 1965 Mean (for all areas) 223 139 Median (for all areas) 199 127 1 598 42 325 Range Table 15: Student access by Gender (mean number of hits per student) There is no consistent pattern – one teaching unit attracts more hits from males, the other more from female students. Table 9: Student access by Professional Group (mean number of hits per student) Medical Education in Practice Professional Group 2006/2007 2007/2008 Doctor (Hospital) 168 82 Doctor (GP/Community) 288 119 Nurse/Midwife 302 103 Other 37 62 Contemporary Perspectives in Medical Education 22
Professional Group 2006/2007 2007/2008 Doctor (Hospital) 256 121 Doctor (GP/Community) 348 203 Nurse/Midwife 127 87 Other/Unknown 443 152 All Four modules Average number of hits per student Doctor (Hospital) 153 Doctor (GP/Community) 232 Nurse/Midwife 189 Other/Unknown 141 Analysis of access data by day of week Figures 1 to 5 (see Annex 2) display graphically the access data (number of ‘hits’) presented by day of week. The pattern varies between teaching modules and cohorts with a general trend to less use at weekends – more apparent in Fig. 5 when all four sets of data are combined. Analysis of access data by time of day Figures 6 – 10 (see Annex 2) display graphically the access data (number of ‘hits’) presented by time of day. The pattern again varies between teaching units and cohorts. The combined data suggests that usage is mostly between 0800 and 0100 with a slight peak in the evening between 1900 – 2200. 3 Pilot interview th The interview was conducted on 16 June and recorded digitally. Two students had been expected but only one arrived – a nurse working as a training education manager who had joined the course midway through the 2006/2007 academic year and had therefore taken part in teaching units with both cohorts of students. The interview was structured with open questions and lasted 39 minutes and 8 seconds (39’08”) in total. The short summary which follows is divided into 5 categories based on themes drawn from the recording of the interview. i)
General Use The student described his general use of computers for communication purposes as ‘a lot’ with the computer online all day at work and 12 times each evening after returning home. He stated that he would like to be more able to access email while away from the office but then went on to describe how negatively he felt about the use of email for communication – demanding and easily overwhelming – and how the increasing use of online communication had led to a ‘decay’ 23
in literacy and a failure to communicate in other ways – face to face or by telephone. “I hate them really” (6’14”) “A base level of communication” (10’29”). ii)
Social networking, Forums, Blogs Social networking and blogs were an area he had explored – initially with enthusiasm but “soon tired of it” because of the loss of nonverbal cues to communication and being unsure about who he was communicating with. He described himself as “totally allergic” (8’12”) to the concept of online social networking such as Facebook® and, despite being part of the second cohort, who had instigated a group (‘Medigogy’) on Facebook®, he stated that “no matter how much encouragement” (8’32”) he would not join such a group. This related to his experience of the Bedfordshire VLE (BREO) and specifically to the communication between students and between students and tutor. He had felt unable to keep up and very frustrated and intimidated by the greater technical ability – and the jargon and abbreviations – used by those who were more enthusiastic about the online discussion forums. He felt there was a “secret language” at times and that it was “a long winded way to get relatively little information” (13’27”). He felt that students were let down by those who didn’t participate as fully and that contributions by those who did could be “inarticulate, making poor assumptions” and “dumbing down knowledge” (35’28”). He expressed his concerns about the reliability and accuracy of the statements people were making and felt that being available online meant they had to be more authoritative than merely opinions. Some form of moderation to correct factual inaccuracies was identified as a need for him. Despite these largely negative comments about his experience as a learner on this course, he admitted to being an enthusiastic user of other ‘Wikis’ in general learning and enquiry and had even often gone back to revisit the group ‘Wiki’ to see if it had been updated even after the course had completed. He felt there was enormous potential for such a tool to be useful in the future and this was one of the few areas of BREO he described positively. iii)
Navigation Issues Local issues specific to the VLE used on this course attracted a good deal of comments both on the theme of navigation as well as reliability. He felt that the layout led to information becoming disjointed and felt that the lack of a “voice” or “presenter” was a problem for him. It should be mentioned here that the student mentioned a degree of dyslexia which might have been a contributory factor in this particular comment. He felt less constrained by the freedom of self directed information seeking and synthesis than by the rigid construction of a VLE and several times expressed his preference for online learning ‘freestyle’. He stated that the difficulty in navigation and negotiating the site led him to use BREO less as he went through the course and despite having undergone two training sessions he still had difficulties finding material. He felt impatient about his need to find his way quickly and not to go through “long winded and confusing routes” (22’49”) and “a long winded way to get relatively little information” (13’27”). He concluded that “I’m surviving the course very well without needing to get into that hassle” (25’01”). Browser compatibility issues had also been a significant problem for him. He later again emphasised his need for the VLE to be “well designed” (29’38”) and to have “a clear, intuitive navigation system” (37’12”) iv)
Reliability (Access) 24
Frequent references were made throughout the interview to frustration due to unreliable access – software and server issues including those mentioned above. “It must be reliable – this is hugely important” (29’45”) v) Trust Because of his concerns about reliability (iv) and the content added by fellow students (ii) he felt – as he forcefully stated at the termination of the interview “I don’t trust BREO” (27’27”) The tenor of the responses was largely negative – indeed at times extremely negative – to the use of a structured online learning package, yet it was clear that he used the Internet extensively to retrieve information in a self directed fashion. Overall however he stated that he was “hard pushed” to list any useful features of the BREO VLE. Although a user of the Internet for online learning resources, he preferred alternatives if possible. He disliked the increasing use of distance learning and the loss of face to face interaction. He stated that he would avoid any teaching modules/units which were designed to be dependent on online learning. Findings Findings are reported in alignment with the study objectives, namely to: 
Explore whether there are specific factors influencing engagement that are common to all clinical teachers (as opposed to other groups of learners) 
Determine whether certain groups or individuals experience barriers to engagement 
Identify those features within the VLE that support or hinder engagement and learning 1
Factors influencing clinical educators’ engagement These students are mostly senior level professionals working full time, either in clinical positions in the NHS, in universities or other educational establishments or in joint clinical and academic appointments. Data from the questionnaire about familiarity and use of a variety of Internet tools suggest that the overwhelming majority are using online computer features primarily for email and general web browsing. Lack of time (not specific to engaging with e-­‐learning) was cited by 51% of respondents as one of the reasons for not engaging fully with BREO. Most respondents were able to access BREO from their workplace as well as from home (although 21% cited problems with work access eg. NHS firewalls as an issue), pattern of access showed usage throughout the day with a slight peak in the evening and with a slightly lower use at weekends. We were unable to find out how long people were engaging at any one time or what specific activities they were engaging in other than knowing which part of the site they were on. This data would be useful to explore the nature of engagement at different times of day and between work and home situations. Although respondents reported their general use of the internet (89% use daily) and email (100% regularly use) and confidence as ‘very high’, when we explored more specific aspects of internet use, we found that a key factor influencing engagement with the VLE was familiarity with or previous experience of e-­‐learning, Web 2.0 tools and other communication media. 40% of respondents had no experience of engaging in an online environment as a learner, and the need for specific training and induction was cited as a reason for non-­‐engagement by some respondents. 25
Those who had engaged in online learning for themselves reported a range of activities, primarily related to supporting their own professional development and CPD. Doctors reported more use of these tools than other groups. 66% of respondents reported no previous engagement with online learning as a teacher. This raises issues for those developing and delivering blended or e-­‐learning programmes for health professional educators. All respondents reported that they had accessed BREO, this was supported by the access data, although the number of hits varied tremendously. For example, in one module, the number of hits ranged from 1 – 598 for 2 different students. Access also varied between the 2006/7 and the 2007/8 cohorts with the 2006/7 cohort accessing the VLE with over 50% more hits than the 2007/8 cohort. The primary motivators cited for accessing the VLE were to: 
Access course information 
Access learning materials 
Access e-­‐journals and library resources 
Meet course requirements: online submission of assignments; participation in assessed discussion groups and tutorials The online learning environment was seen as supportive of learning through bringing materials together in one place that can be accessed at any time and from anywhere. Some respondents used BREO for communicating with tutors (eg. to obtain feedback on reflective diary entries or in discussion groups) and with other students. 41% of respondents noted that they communicated in other ways with tutors and students and 31% said that they preferred to meet people face to face, use email or the telephone. A number of respondents noted that they would have liked to have engaged more with the online discussion groups, social networking and blog options. Barriers to engagement with social networking tools and discussion forums included a lack of confidence or technical knowledge of using the tools, not knowing the respondents very well; being intimidated by other students seeming to have a greater familiarity with both educational and technical jargon and ‘know how’; the non-­‐participation of some students; communications from some students as being inaccurate, inarticulate, over-­‐
authoritative, and a lack of group moderation. Only 21% of respondents used BREO as a means of emailing staff and students and 35% of respondents had never used the VLE for access to e-­‐journals and search engines. This was primarily because they already had other means of communicating and access to e-­‐resources. For example, one respondent noted that: “[this course] is not the same as an UG or longer course or full time course as most users will have access to other resources. I have never (and probably will never) use it to access e-­‐journals as I have more than adequate access from NHS and my own place of work”. Respondents cited some of their needs from online learning as being able to see the relevance of online learning to their future work and learning, including technical ‘know how’ and how to engage organisations, communities and educators in online learning. 26
We wanted to explore how students on a medical education programme perceived e-­‐learning, as well as the nature of their engagement. We found an interesting mismatch (cognitive dissonance) between respondents’ stated understanding of the benefits and main purposes of an online learning environment and actual patterns of engagement. Many respondents stated that communication and interaction were the ‘main purposes’ of e-­‐learning when asked, and the reflective diary was cited as a particularly beneficial tool, however many students had not engaged with the reflective diary or discussion boards unless as a course requirement (usually motivated by this being linked to assessment). 62% of respondents noted that online learning is useful, necessary, appropriate and essential for clinical educators as learners (but in conjunction with other forms of learning); that it is important that educators have the skills to keep up with students and that it enables access to experts, discussions and debates about medical education. There was a significant variety in the qualitative responses from the students – often contradictory in their statements of what worked well for them and what they saw as a problem or barrier. Themes emerged from the comments as described here, but the study also revealed a wide range of positive aspects, concerns and needs as expressed by respondents. This raises the importance of undertaking exercises to clarify the student perspective and voice – and to consider individual needs at the design stage – possibly involving offering a range of materials and styles to suit diverse needs. 2
Barriers to engagement for certain groups or individuals 21% of respondents reported difficulty in accessing the VLE from work, this was cited as due to lack of time, lack of opportunity to work on a PC or NHS firewalls. GPs and Nurses self assessed on the questionnaire survey that they used BREO more frequently when compared with the self-­‐rating of Hospital Doctors and other students. This was supported by the data from the webserver. However there was not a consistent pattern across all teaching modules nor were there significant gender differences. Those who self assessed as being low users of the VLE also had lower than average number of hits on the webserver data. For those who rated their use as being ‘frequent’ there was less consistent agreement. However, the discrepancy may be explained by the rating terms in that a student who uses BREO rarely will know this but there is no accepted ‘norm’ against which to measure or define ‘frequent’. Some individuals note little confidence or inclination to engage in on-­‐line learning. 30% of respondents cited lack of experience/familiarity with technology/BREO/VLEs as being a barrier to engagement whereas 25% of respondents noted that their own personal learning preferences were a barrier to engagement with online learning. As commented above, this may be due to earlier negative experiences or non-­‐engagement with e-­‐learning. Some respondents noted their lack of technical ability in using computers in general as well as not having familiarity with social networking and internet tools such as podcasts; blogs; wikis; webcams; RSS feeds or Second life. The student interviewed expressed largely negative experiences of the BREO VLE – citing unreliability, difficulty in navigation and lack of engagement in communication forums. Whilst probably atypical of this group, these comments demonstrate that for some students e-­‐learning provokes a very strong response which can be a huge barrier to learning. He self-­‐rated in the 27
questionnaire when asked how often he accessed the VLE as ‘seldom’ ‘once’ or ‘never’ and free text comments in his questionnaire responses were consistent with the responses he gave in the interview. The behaviour of this individual student in accessing the site as measured by the data from the webserver agreed with his self-­‐rating as he measured significantly below average for his use of the site, especially for the discussion boards and group forums. 38% of respondents cited their main forms of engagement as a ‘watcher’ to see what others are doing or saying, Webserver data also confirmed that there were a large number of ‘lurkers’. It is not possible to determine whether or how ‘watching’ facilitates learning and whether this reflects a certain learning style, or a lack of competence or a lack of confidence in using online tools. 3
VLE features that support and hinder engagement and learning Features that support engagement and learning Some of the specific features of the VLE that support engagement (as cited as ‘most useful’ by over 20% of respondents) are:  Learning/course materials [always accessible from anywhere]  Link to e-­‐journals and search engines  Announcements, keeping up to date with the course  Preparatory materials for contact days  Link to library  Quick links to useful websites  Digital assignment drop box (when it worked!)  Reflective diary/log plus tutor feedback These are predominantly around the VLE being a repository for resources or a portal to other resources and information. Being able to access the VLE course information is vitally important for many students as they work in different locations, travel and work unsocial hours without easy access to university facilities. “I often find with time for my own learning that I am in the wrong place at the wrong time, and yet there is usually a computer to log onto and access resources”. Learners can learn in their own time and at their own pace. In some ways, this might seem to be a less important use of the VLE than engagement in more interactive learning activities. However, many of these students had not engaged with e-­‐
learning as a student or as a teacher. Also, for many this is the first time that they have encountered educational literature, theories and concepts. For some, it is the first time they have studied formally since graduating, most have never studied at Masters level, and for the vast majority (particularly doctors) it is the first time they have studied within social science (as opposed to science) paradigms. As one respondent noted: “[the portal to other sites/resources] provide access to other resources not immediately otherwise available to the learner”. The capacity of a VLE to provide a ‘one stop shop’ for materials and resources that have been carefully reviewed, selected and endorsed by the course team is invaluable for enabling busy professionals to engage in reading and learning in a new field of study without having to sift 28
through articles and other resources from a position of relative naivety. This dissemination of resources and links was seen as “particularly useful for part time students with geographical barriers” to coming onto the University campus and library facilities. When we asked respondents to think about the relationship between the VLE (e-­‐learning) and their own educator development, an even richer picture emerged. Respondents commented that the VLE provided a ‘group learning environment’ that not only complemented but extended face to face contact time. This social networking learning environment enabled students to “continue to communicate and learn from each other outside the teaching days” but it also “makes me feel I am in a learning environment in relation to the courses of medical education”. The VLE also helped level the playing field between students: “provides a forum that students from all backgrounds could exchange ideas”. It also helped to reduce isolation, enabling connection, contact, collaboration and support between geographically dispersed, ‘distant’ learners: “the idea of interactive BREO components were ideal, to explore and debate in the group at a distance”, “to facilitate a group learning environment”, “supporting each other” and “to encourage reflection, opportunity for collaborative learning” and shared learning, to “trade knowledge”. Respondents also saw engaging with BREO as useful for developing online learning skills as part of educator skills development, helping them keep pace with ‘modern life’ as a teacher by keeping pace with educational technology. BREO is useful “as a learning tool to get us more engaged with e-­‐learning as this will play a bigger and bigger role in medical education”; “to be introduced to new ways of learning”; “to [provide] creative examples of teaching methods that we could consider using in our own teaching practice” and ”to develop other online skills eg wiki”. Communication with tutors was also seen as important, for enabling individual, ‘private’ comments on the reflective diaries (bogs) but also in engaging in discussion groups: “point of contact with teaching staff (especially via blog which allows reassurance that you are barking up the right tree!)”. Respondents noted that the value of the VLE for course teachers included that they would monitor learning, there was less printing for the tutors and university and it was more time efficient for the tutor. Features that hinder engagement and learning We have described above some of the general reasons for a lack of engagement in online learning. For other respondents, some of the specific features of the VLE itself hindered engagement and learning. 34% mention limited or intermittent access (site unavailable, multiple passwords), 28% mention unreliable access and 23% mention poor functionality. Some respondents cited unreliability as an issue which led to a lack of trust in the system: “issues of reliability and trust – system losing materials”, “system shutting students out for periods” and “system unable to accept assignments.” No respondents mention access or functionality as being a barrier for general online learning, thus indicating that this is perceived to be a local issue to BREO. 23% of students see BREO as “too slow and clunky”; 18% see BREO as too hard to use. The interviewee noted it was “cumbersome and unreliable, simply doesn't work for me beyond going in and picking up reading matter, info update and dropping assignments and I don't trust it for that without a backup phone call. Needs to be much more intuitive, clearer signposting and less fussy.” 29
Navigation difficulties were also noted as a barrier. Improvements suggested by respondents included easier navigation, site layout (drop down menus, more intelligent interface, lists of databases) (34% of respondents); improving links between pages/internal sites (17%) and improving efficiency and speed. The least useful features of BREO in terms of learning and learning support activities, as cited by over 20% of respondents, were:  Discussion boards (38%)  Email facility to staff and students – “easier to use own/work emails” (26%)  Reflective diary – “easier to do it on PC for assignment” (24%) However, engagement in specific learning activities relies partly on students feeling competent and confident in using BREO and also in students and tutors actually participating in activities regularly and meaningfully. Lack of engagement of other students in activities was cited by 29% of respondents as being a barrier to engagement. 34% of respondents noted that they needed more support from tutors in terms of practical help, an introductory session with expert help, online support and a ‘how to’ guide to the various BREO features. As one respondent noted “it makes life easier for the student to gather resources, gain feedback and remain in touch with other members of the group … it also saves time when you get the hang of it”. Discussion Our study has found that clinical educators’ engagement in online learning is based on a number of interacting factors based around their individual learning needs as a learner in a new field of study (here medical and health professions’ education) as well as learning needs relating to online learning, and their needs to become familiar with online learning to support their practice as a teacher or clinical educator. Factors influencing engagement Time and access We found that, although some students cited that lack of time (in general) was a factor in engaging with online learning, and some respondents had difficulties in accessing from work, most respondents were able to access the VLE and used some features of it. This correlates with Childs et al (2005) study of NHS workers. Salmon (2000) describes five steps as essential in the e-­‐learning process: access and motivation; online socialisation; information exchange; knowledge construction and development. However, If learners cannot access the VLE, or are unable to use key features (either because of a lack of technical ability or because the site itself is unreliable or hard to navigate) then they will become demotivated at a very early stage. Two factors are closely related in influencing engagement in online learning: general confidence and competence in using computers and reliability of a VLE. Confidence and competence in using computer based technologies We found that all students used email and the internet on a regular basis and most had good access to a networked computer at home and at work. However a number also noted that they had little experience of using computers in a learning environment and some students described 30
themselves as ‘technophobes’. These students required more support in order to gain competence and confidence: from tutors, in the form of online ‘technical’ help, from other students and via handbooks and guides. In practice, if we are to address the wide variation in IT skills and experience for students entering postgraduate blended learning courses, assessment of an individual’s learning needs at the outset of the course could be undertaken. We have not carried out a study relating engagement in online learning with assessment results, but if there was such a correlation, there may be value in requiring a basic competency qualification such as the ‘European Computer Driving Licence’ (ECDL) prior to study. Similarly, if a student were to rate their beliefs in the usefulness of and online learning environment negatively, then they might reconsider their choice of course (Miller, Ranier et al. 2003). Reliability and functionality of the VLE Technical aspects and issues with accessing the system appear to have been a source of frustration for many students. A University more geared to undergraduate students and structured around term dates may schedule maintenance downtime at times which are less convenient for the postgraduate students who are working full time and studying at different times. Other authors have reported that one of the principal barriers to effective use of the materials in an online environment is the technology (McKimm, Jollie et al. 2003) and our experiences agree with this. It can often be hard to separate the problems related to technological issues from those related to design and navigation (Ally 2004). The first cohort experienced more technical and access issues and this may have been ironed out for those who followed a year later and this might explain the unexpectedly higher access rate of the first cohort when compared to the second. Our study reflects and supports the findings of many other studies which indicate the importance of having an online environment that is easy to access, reliable and easy to use and navigate through. This particularly affects two different groups of learners. Our study showed that students with low confidence and competence in using learning technologies were much less likely to persevere with the VLE and instead sought other learning strategies. However, students who are highly experienced in using learning technologies, particularly those who have access to alternative online learning environments, also become easily disengaged and turn to their more familiar and reliable VLEs or web-­‐based resources. Previous experience of using online learning as a learner or teacher We saw that students who had little or no previous experience of online learning felt disadvantaged and lacking in confidence, particularly at the start of the course. Students who also lacked (or perceived that they lacked) technical computing abilities or familiarity felt even more disadvantaged and this ‘double disadvantage’ led to a further lack of engagement. Although previous experience generally led to students engaging more quickly and thoroughly with online learning, with some acting as guides and providing support for other students, this also led some learners to have higher expectations of the VLE (especially relating to navigation, features and speed of use) than were available. This then turned into a barrier to engagement, with these students becoming frustrated and disenchanted, this was exacerbated when other students did not engage in discussion groups or other online activities. 31
The three factors described above (if not compensated for) can adversely affect the performance and dynamics of an online learning group. We saw that non-­‐engagement in discussion groups or other online activities led to further non-­‐engagement even by motivated students. Timmis and Cook (2002) note five points concerning motivational strategies specific to online learning: 1. Virtual learning needs to provide opportunities not available elsewhere 2. Tangible external motivators need to be built in 3. Learners must have clear expectations in a virtual environment 4. Specific guidance is needed to exploit opportunities 5. The level of threat must be managed through support and peer group induction For clinical educators, it is also important that motivation for online learning is balanced with overall motivation for the programme. Many of these students struggle with studying within new paradigms and concepts, students are busy working professionals with demanding jobs and therefore have many pressures competing for their time. Online learning therefore needs to support and enhance the learning experience on the programme (which can in itself be seen as threatening) and not be a barrier. This leads onto a consideration of the underpinning pedagogical approach and the learning activities that might be undertaken online for clinical educators. Pedagogical approach and nature of learning activities The MA (MedEd) programme is underpinned by social constructivist views around learning in which “learning is an active process of trying to "make sense" of new experiences. As we integrate new ideas and information into our existing knowledge structures the new knowledge becomes unique to our own thinking” (Kilpatrick, Hart, Najee-­‐Ullah, & Mitchem, 1997, p. 1227). In other words, constructivist learning is about bringing learners to see the significance of the content for their own particular needs and purposes so that they might connect with the content in a meaningful way. This is highly consistent with current thinking on online learning design (Rourke, Anderson, Archer, & Garrison, 1999)which suggests that online learning should lead to three types of presence: cognitive presence, teacher presence and social presence and that “it is pedagogical methods, not technology per se, that most directly influence learning” (Reeves, Herrington, & Oliver, 2005, p105). However, regardless of the pedagogic and design principles underpinning the development of online learning, if the experiences of the user – the learner – are not fully considered, the potential of VLEs cannot be realised if students do not use it. In common with all groups of learners, students in this study are not an homogenous group. We have seen that learners have varied levels of experience, expectation, confidence, competence and learning needs relating to medical education as a discipline and online learning as a way of learning (a learning mode), as a social networking and communication mode, for accessing information and resources and as an examplar of learning methods that students might wish to use in their own teaching practice. We also need to take into account that students on a part time, blended learning, postgraduate, medical education programme are typically mature students who are working full time in senior positions in health and/or education. Many of these students have never studied education or social sciences before, having taken professional qualifications in medicine, nursing, dentistry and other health professions which 32
have predominantly been taught within a scientific, positivist paradigm. This means that when planning learning interventions and programmes for clinical teachers, layers of complexity are added which need to be taken into consideration. These layers of complexity may not exist in quite the same way for those planning programmes for undergraduates studying a full time degree programmes in non-­‐vocational topics such as English Literature or Economics. The nature of the subject matter and learning needs of the students have to be taken into account when planning any learning programme, so that online learning can be used to support and enhance learning and not act as another barrier to engagement. Taking each of the four aspects of online learning as distinct yet interlocking aspects helps us to plan, design, deliver and evaluate our pedagogical approach, learning outcomes, learning activities and assessments: 1
online learning as a way of learning (a learning modality) 2
as a social networking and communication mode 3
online accessing of resources, materials and information 4
as an exemplar of learning and teaching methods that students might wish to use in their own teaching practice Articulating online learning in this way enables a mapping of learning for both staff and students, some activities might meet all four aspects, others might meet only one or two. Activities that do not meet any of the aspects might need to be reviewed and delivered in a different modality. Requiring students to engage in an online environment without a learning gain will eventually demotivate learners, particularly busy, working, high level professionals who cite lack of time as one of the barriers to all learning, not just the online environment. As Cross et al note, one of the keys to success is to ask the question, “What can this technology do that will enhance the learner's experience that my current learning technology portfolio cannot?” (Cross, O'Driscoll, & Trondsen, 2007). This four aspect model also helps us to determine the skills that learners might need before starting on the programme, we can then plan more tailored individualised learning programmes for different students. Devising an instrument to assess clinical educators’ readiness for online learning using the four aspects would be a next step. This would enable students and staff to see where learners might need more help and support and might wish to be extended or gain more experience. The instrument would also enable the ‘profiling’ of a learning group to see how discussion groups or activities might best be structured to cater for different learning needs and expectations. Let us look at each of the four aspects in relation to our study. Online learning as a way of learning (a learning modality) It might be expected that those with specific roles and responsibilities in Medical Education would demonstrate increased capabilities in reflection and seek to engage as much as possible in the learning process – both as learners and educators (Sharpe, Benfield 2005). However the three parts to this study all appear to suggest that use of the VLE is highly variable. In general it is being used to access resources predominantly and few are exploiting the potential for communication. 33
Students use technology extensively and in a varied, personalised, flexible and interactive way (Conole, De Laat et al. 2005). Many of our learners have grown up in an environment in which they have been constantly exposed to computer based technology (Sandars and Morrison 2007) and incoming students at an undergraduate level already have considerable experience of the use of technology as a means to enhance communication and information sharing. Authors such as Sandars and Morrison (in considering the impact on undergraduate education) comment that the main challenge is for the institutions and the educators rather than for the students. In the postgraduate setting it may not be the same – learners who are returning to HE after a break may find the environment dramatically different, especially in medicine where their earlier studies may have been more didactic transfer of information from the ‘expert teacher’ with less emphasis on interaction and reflection. Add to this the wide variation in their experience in the use of technology which poses an additional set of challenges. Younger students are considered to be more familiar with the use of the Internet and other applications (particularly blogs, wikis and social networking) compared with older learners (JISC, 2009; Morris and McKimm, 2009) and this should be a factor in the design of such courses. The inflexibility of a structured VLE was mentioned by some students as an issue for them. The HEA review of VLEs in the undergraduate medical setting (Cook 2005) found that although more schools were using commercial VLEs mainly due to institutional pressures -­‐
there was greater satisfaction among developers of bespoke systems (although we cannot assume that this was necessarily transferable to the student experience). Many students mentioned that the VLE did not suit their learning style and the interviewee echoed this view. There are many theories of learning and none can claim to be exclusively applicable to online learning. However, strategies should be chosen which will facilitate deep and contextual learning, encourage interaction (with feedback), cater for individual variations in learning preferences and provide support during this (Ally 2004). Ally goes on to describe a variety of strategies which can motivate the students – he writes that regardless of the effectiveness of the online materials, if the motivation is not right, the students will not engage. Motivating Students to learn online (Ally, 2004) Attention An initial activity to capture the learners’ attention Relevance How will they benefit? Contextualise the learning and make it meaningful Confidence Design for success and provide ongoing feedback and encouragement Satisfaction Feedback on performance and how they can apply this on completing the learning activity Online learning as social networking and a mode of communication 34
Goodyear (2005, p85) suggest that effective pedagogical strategies used in online learning is to ensure that we achieve the three types of educational presence already mentioned: •
cognitive presence which occurs though frequent and effective interaction with content •
teaching presence which occurs through frequent and effective interaction with the instructor •
social presence which occurs through frequent and effective interaction between students (Hutchins, 2003). Acknowledging these ‘presences’ requires implementing a number of relevant pedagogical tactics – “the detailed methods we use to set tasks for students, encourage their participation, offer guidance and feedback, etc” (Goodyear, 2005, p. 3) which will engage students. Paloff and Pratt (1999) suggest also that there is a need to develop social/emotional space as well as the academic and intellectual contact. This is a greater challenge on a course with few formal contact days. The second cohort of students studied did set up a group in Facebook® but the interactions declined rapidly after the inception and interest waned as postings became infrequent. One area for possible study would be the relationship between VLE experience and linguistic skills. Would the online environment make it easier for student who do not speak English as their first language – they can learn and reflect at their own pace – or might the insecurity about their writing skills mean they would be less willing to share their postings online? Online learning is highly relevant to a global market, being able to understand the impact and influence of language ability on social networking in relation to learning would add to our understanding of engagement in online learning. Wallace (2003) suggests that students are more willing to share ideas but less willing to challenge each other’s thinking or move towards deeper learning. Measuring the quality of online interaction rather than simply the hits or time spent online is potentially more useful (Miller, Ranier et al. 2003). This might include the quantity and quality of the contributions to forums and Wiki® exercises. The relationship between final grades and engagement in specific features such as blogs (reflective diaries), wikis or discussion forums is also be an area for future research if more detailed analysis of what constitutes ‘engagement’ were possible. We were unable to find any studies in the literature correlating levels of participation with learning outcomes and achievement in assessment. This area would be worthy of further exploration. Asynchronous discussions and learning at a time of choice may have benefits for the individual but if students logon relatively infrequently and use their time mostly to seek out course materials and information the immediacy is lost and the time needed for deeper reflection may not be available. The need for more interaction in online discussion was an issue – students felt that there weren’t enough of their peers using the discussion forums and this in turn led to them visiting less often. Several authors (Hewitt Taylor 2003, Murphy 2004, O'Grady 2001) 35
emphasise the important role of the facilitator in actively promoting interactions in order to maximise the success of the dialogue and that the facilitator should be directly involved rather than merely observing proceedings. The student interviewed commented that he felt he could not trust comments made online by his peers and wanted more moderation and verification by the tutor. This was also reported by Petrides (2002) where she reports scepticism about the supposed expertise of peers as opposed to confidence in relying on that of the instructor. This resonates with the need for ‘teaching presence’ to support online learning. Overcoming this lack of trust in fellow students (or empowering students more to challenge and debate) is essential if collaborative learning is to be delivered. McKendree (2006) describes how students themselves can often provide faster and equally satisfactory responses in such an environment. The tutor will need to be mindful of the need to develop a sense of trust and safety within the electronic environment – without this, learners can feel uncomfortable and more reluctant to post their thoughts and comments (Anderson, Elloumi 2004). Students may feel that posting online is different to contributing to a class based tutorial/workshop. Chizmar and Walbert (1999) described how learners felt they had to take more care in posting comments because of the public display of these. Petrides (2002) found that participants felt they needed to think more carefully when responding in writing rather than giving verbal responses and suggests that this may lead to a greater degree of reflection than would otherwise be the case. So are our students afraid, don’t have time to (or cannot) reflect or don’t want to share? It may be that students require more ‘cognitive presence’ than they were receiving. Students report feeling isolated (Sit, J.W.H. 2005; McKimm et al., 2003; Kim, Liu et al. 2005) as well as being less willing to share thoughts and opinions with people they know less well and this was reflected in the responses of some of our students. Even in a blended learning environment this can be an issue but more so in an entirely online course. Song et al. (2004) reported that one of the students interviewed had been involved in a course where the instructor took a picture of each of the learners and posted them onto the VLE. This simple tactic is straightforward technically and makes the online environment more welcoming and more personal. Online accessing of resources, materials and information In common with other studies (Sharpe, Benfield 2005), both the questionnaire and the webserver access data suggest that students are accessing the content and materials sections of the VLE actively. The findings from this study of medical educators suggest a reluctance to engage with group discussions and forums yet a willingness to access material for information. We have discussed some of the possible reasons for this above, it may be particular to clinical educators or reflecting national characteristics. Some studies at institutions based in the US seem to suggest more enthusiasm for interaction than we have demonstrated, despite the added complications of students being in different time zones (Kim, Liu et al. 2005). 36
Online learning as an exemplar of learning and teaching methods students might wish to use in their own teaching practice This aspect is specifically relevant to those who are learning to teach, be they clinical educators or student teachers. Some respondents clearly felt that this was a motivating factor, that they might feel ‘left behind’ if they were not aware of or had not engaged in a range of learning technologies. There were expectations from some respondents that the MA programme would provide opportunities for learning about creative and ‘modern’ ways of learning and this was supported by comments that indicated respondents were very aware of the relevance of online learning to contemporary medical and health professionals’ education. Respondents had very different prior experiences of online learning, some were highly experienced both as teachers and learners, whereas others had very little experience of online learning or using computers for more than emailing and web browsing. This is one aspect where planning individual learning programmes would be of use, so that previous experience can be linked to learning needs and a learning programme. Some students may well want to simply ‘talk the talk’ more fluently and have engaged in a small number of activities for their own learning and limited teaching practice, others may want to engage in a wide range of very new technologies and explore how these might relate to their own educator practice. This poses potential challenges for those planning and delivering health professions’ education programmes, challenges that may well be met by collaborative learning initiatives with online learning specialists, educational technologists as well as education specialists. Ethical issues Online learning enables us to collect data on student use and interaction in a way that is not possible with face to face learning. The ability to allow educators to closely monitor access of materials and the quality of interactions could provide huge benefits in relation to enhancing our understanding of the nuances and ‘rich pictures’ of online engagement. However this also raises issues about what are public and private ‘learning spaces’ and who owns the products of recorded online ‘conversations’. Ethics approval was agreed for the study (in terms of the questionnaire, access to the BREO webserver data and the interview) however students were not necessarily aware that their use of a VLE is logged and available to tutors on a named basis. Some students might feel this is an unwelcome intrusion and that such a high level of ‘snooping’ is inappropriate. While it seems justifiable that course designers and tutors should have data to be able to analyse which parts of a VLE are seen as being more useful than others perhaps this might be more appropriate if data was anonymised for general (rather than research) use. Conclusions and next steps The study considered a group of learners who might be expected to be highly motivated to engage as educators who will be increasingly involved with online learning in one form or another – if not in design or organisation then in moderating, facilitating and mentoring and in assessments. However, our findings suggest that the issues and experiences of clinical educators using a VLE are not significantly different from learners in other studies and learning environments: technological, navigational and communication issues are the principal themes, even though such issues may not be within the power of the course team themselves (Conole, 37
De Laat et al. 2005). It is clear that our respondents are using the VLE effectively to access course material and links to related sites and have differing experiences of engaging in social networking and collaboration. The success of a blended learning programme ultimately depends on developing the collaboration between students (and tutor) and the online learning environment has much potential to enhance communication and collaboration. Although the findings of the study are not generalisable to all online courses – or even to all blended learning courses – some of the themes identified, in common with much of the existing literature, may have application to similar courses. In the future, increasing learner autonomy (JISC 2007a) associated with the increase in online learning will demand that these issues are acknowledged and addressed. It will become increasingly important for clinical educators to be able to guide, support and stimulate their own students in engaging in online learning, but we have seen that many of our respondents are reluctant users. The increasing use of the Internet to deliver education challenges the traditional providers of medical education, especially in the field of CME (Harden 2005). Learners expect their tutors to be as fully engaged as they are and aware of the relevant related technological resources (JISC 2007a). Our study found that clinical educators’ engagement or disengagement in online learning is primarily related to the inter-­‐relationship between three learning needs, which varied between individuals depending on their previous experience, current position, motivation for study and future plans. The three individual learning needs are:  Those related to learning about subject content (in this case medical and health professions’ education)  Those related to learning to learn and study in an online environment  Those related to learning about online learning to support their current or future practice as a teacher or clinical educator The way in which clinical educators engage with online learning to meet these needs are affected by a number of factors, many of which are similar to those identified in studies in other learning environments. These factors are:  Time and access  Confidence and competence in using computer based technologies  Reliability and functionality of the VLE  Previous experience of using online learning as a learner or teacher  The pedagogical approach and nature of learning activities Our study indicated that the subject material and specific student body (mature, senior health professionals or academics who mainly work full time and who have not studied this learning material or engaged with these conceptual frameworks before) was also highly influential on the level and nature of engagement with online learning. Issues such as time, motivation, learning styles and expectations all influenced engagement. 38
We suggest that when planning, designing, delivering and evaluating online learning for clinical educators, four distinct yet related aspects need to be taken into consideration when thinking about a pedagogical approach, learning outcomes, learning activities and assessments. The aspects to be considered are: 1 Online learning as a way of learning (a learning modality) 2 As a social networking and communication mode 3 Online accessing of resources, materials and information 4 As an exemplar of learning and teaching methods that students might wish to use in their own teaching practice Using these aspects as a framework or blueprint for planning online learning (as a discrete modality or within a blended learning programme) will enable both staff and students to identify learning needs, learning outcomes and potential gains for learners. It also provides a way of conceptualising online learning for educators in a way that moves beyond students’ own learning, and takes it towards their own developing educator practice. The four aspect model also provides a framework for determining student learning needs prior to starting a programme, tailored learning programmes can then be planned around individual student needs in relation to subject content, studying in an online learning environment and future educator practice, see Planning matrix below. Planning matrix for an individual student learning programme or planning learning activities for a programme (with examples of the latter) STUDENT LEARNING NEEDS FOUR ASPECTS Those related to Those related to Those related to learning about subject learning to learn and learning about online content (in this case study in an online learning to support medical and health environment their current or future professions’ education) practice as a teacher or clinical educator  Assessment See other cells Online learning as a See below  Review and requirement to  Accessing articles way of learning (a answering q
uestions write u
p r
eflections from other learning modality) As a social networking and communication mode 
Online accessing of resources, materials 
about online video clips about clinical teaching and supervision Posting a thread on a group discussion board on an educational ‘big thinker’ Signposting to articles and podcasts in the reflective diary space educational contexts on online learning 
Posting a photograph and introductory bio to the group in the first week of the course 

Formative assessment to 
Sharing the articles identified above with online learning set and discussing key points in relation to own practice on discussion board Glossary of online learning terms with 39
and information As an exemplar of learning and teaching methods that students might wish to use in their own teaching practice on educational theory search for allocated links to examples of articles and other Web 2.0 tools resources on educational paradigms to bring to a contact day All of the above All of the above All of the above  Presentation and  Discussion thread  Working in pairs demonstration by on challenges, with someone from medical education opportunities and different expert on T&L strategies for professional methods from engaging effectively background and another HEI to a face in online learning learning context to to face contact group share experiences and ideas re planning e-­‐learning activities for students Next steps Curriculum change Since this study commenced in 2007, a number of changes have been made to provide more support and help for students engaging in online learning and to further integrate online learning with face to face and individual learning activities. Specific changes that have been made in relation to the findings from the study include: 1. Identifying and responding to students on-­‐line learning needs at an earlier stage An extended ‘hands on’ orientation to BREO, in small groups (of around 12) provides an opportunity to establish who are the most / least confident and/or competent users of technology early on. Students work through a ‘treasure trail’ guide to find key resources, to contribute to discussion boards, post reflections etc at their own pace. Tutors (subject tutors as well as learning technologists) provide more support for those who appear to be ‘struggling’ and pair students to foster peer support and guidance. This hands-­‐on orientation is accompanied by a print ‘step by step’ guide, with tutorials including screen shots and arrows containing instructions. 2. Being more explicit about the on-­‐line tools we are using and the rationale for their use. Many students who said they did not know what a ‘blog’ was had been regularly using a blog in BREO to record reflections. We still ‘label’ tools and content areas in relation to their function but also provide the technical ‘labels’ to increase awareness in order to develop a shared e-­‐
learning vocabulary. This extends into opportunities to critique on-­‐line learning experiences and relate these to the challenges of supporting the on-­‐line learning of others. 3. More careful scaffolding of e-­‐learning experiences to gradually develop familiarity and 40
confidence with their use. Students explore ‘depository’ and ‘gateway’ functions before moving onto the use of discussion boards, blogs, wikis and voice tools. We have introduced more careful signposting to specific on-­‐
line resources linked to face to face work and a gradual broadening of the range of sign-­‐posted learning materials used. 4. More actively engaging students in the development of the on-­‐line learning curriculum. The use of a wiki to support students peer learning work around educational theories and perspectives arose from a student desire to capture and share the research work they undertook in the small groups. This has evolved into an ‘educational big thinkers’ wiki’ where students work collaboratively to develop a resource that supports much of their Masters level work. The reflective diaries and assignments provide many examples where students note the value of this activity in enhancing their e-­‐learning capability whilst providing an opportunity to engage in and reflect on online group activity. 5. Making the links between on-­‐line learning and assessment more explicit. Engagement in e-­‐learning is presented as a key way to engage in formative discussions with course tutors. For example, students who record reflections in the blog are able to access formative feedback on these from members of the team. The wiki provides opportunities to develop ability to critique different perspectives on education in readiness for writing an academic paper. Voice e-­‐mails provide rapid, personalised formative feedback on assessed presentations and group work. 6. Materials generated by students are shared on-­‐line A range of media are used to capture aspects of the face to face learning opportunities. E.g. students build ‘curricula models’ out of a range of modelling materials and a photo gallery is created and shared on-­‐line. Flip chart ‘posters’ generated during small group work are also photographed and posted with other learning materials. 7. Identifying the ‘value added’ aspects of on-­‐line learning BREO provides a space for one-­‐to-­‐one learning with students. The use of moderated discussion boards (e.g. on-­‐line reading groups) and opportunities to seek formative feedback on reflective diary entries are highlighted as ways to ensure more personalisation of learning experiences. Proposed research activities We have identified some future research areas stemming from the findings from this study. Although much has been written about the design of courses and the technology underpinning this, more needs to be understood about the complex factors which surround interaction and motivation to contribute to online learning activities for clinical educators. Ethical issues also need to be considered in relation to viewing students ‘online public and private spaces’ and using these in research. 41
We would like to develop a pre-­‐course and post-­‐course questionnaire for students around e-­‐
learning which would enable us to test out and develop the planning matrix further, particularly in terms of identifying student learning needs (in relation to the three areas) and planning learning activities (in relation to the four aspects). We feel that this, coupled with a qualitative textual analysis of engagement in online learning activities, would provide new findings in relation to exploring the relationship between engaging in online learning as a learning modality and for educator practice, and engaging with the complex and often unfamiliar subject material of medical education. We would like to explore further our learners’ experiences through interviews and focus groups, choosing the subjects through purposive sampling, designed to represent the range of experiences relevant to the research question (Kuper, Reeves et al. 2008) and one of the key elements of using grounded theory research (Lingard, Kennedy 2007). Specific groups on which we would like to focus include those for whom English is a second language to explore whether there are issues relating to engagement in social networking activities. We would also like to test out the validity of the findings from the questionnaire in relation to the factors that inhibit or promote engagement, looking at those who use the VLE a great deal as well as those who appear not to have done so, or who have expressed issues relating to interacting in an electronic forum. The focus group questions, used to guide the discussions, would be part of an iterative process derived from the findings of the earlier stages of the study (Kennedy, Lingard 2006). We would also like to incorporate the methodology of the LEX project (Learner’s Experience of E-­‐
learning) (Mayes 2006) which includes ‘interview’ plus artefacts which might be a learning log, course materials or a reflective blog specifically about their online experience. Finally, we were not able to correlate online learning engagement (in terms of quality, nature or quantity) with assessment results. We feel that it would be worth exploring whether engagement with different features (such as the reflective diary or wiki) enhances assessment performance. 42
Acknowledgments We would like to record our gratitude to the students who took the time to participate in the questionnaire survey and the individual student who gave us his time for the pilot interview. We would also like to thank Nick Collis for finding and collating archived assess data from the BREO webserver and MEDEV for funding this study. References AAMC, 2007. Effective Use of Educational Technology in Medical Education. Washington: AAMC. ACADEMY OF MEDICAL ROYAL COLLEGES, 2007. Development of E-­‐learning for Doctors. London: Academy of Medical Royal Colleges. AGARWAL, R., SAMBAMURTHY, V. and STAIR, R.M., 2000. Research Report: The Evolving Relationship Between General and Specific Computer Self Efficacy An Empirical Assessment. Information Systems Research, 11(4), pp. 418430. ALI, N.S., HODSONCARLTON, K. and RYAN, M., 2004. Students' Perceptions of Online Learning: Implications for Teaching. Nurse Educator, 29(3), pp. 111. ALLY, M., 2004. Foundations of educational theory for online learning. Theory and practice of online learning, , pp. 331. ANDERSON, T. and ELLOUMI, F., 2004. Theory and practice of online learning. Athabasca, Canada: Athabasca University. BLOOM, B.S., 2005. Effects of continuing medical education on improving physician clinical care and patient health: A review of systematic reviews. International Journal of Technology Assessment in Health Care, 21(03), pp. 380385. BOEREMA, C., STANLEY, M. and WESTHORP, P., 2007. Educators' perspective of online course design and delivery. Medical Teacher, 29(8), pp. 758765. CHILDS, S., BLENKINSOPP, E., HALL, A. and WALTON, G., 2005. Effective elearning for health professionals and studentsbarriers and their solutions. A systematic review of the literature-­‐
findings from the HeXL project. Health Information & Libraries Journal, 22(s2), pp. 2032. CHIZMAR, J.F. and WALBERT, M.S., 1999. WebBased Learning Environments Guided by Principles of Good Teaching Practice. Journal of Economic Education, 30, pp. 248259. CONOLE, G., DE LAAT, M., DILLON, T. and DARBY, J., 2005. Student experiences of technologies Final Report. JISC. COOK, J., 2005. Review of Virtual Learning Environments in UK Medical, Dental and Veterinary Education. Newcastle, UK: Higher Education Academy. 43
CROSS, J., O'DRISCOLL, T., & TRONSDEN, E. (2007). Another life: Virtual worlds as tools for learning [Electronic Version]. eLearn Magazine. Retrieved 12th December 2008, from http://www.elearnmag.org/subpage.cfm?section=articles&article=44-­‐1 DAVIS, D., O'BRIEN, M.A.T., FREEMANTLE, N., WOLF, F.M., MAZMANIAN, P. and TAYLORVAISEY, A., 1999. Impact of Formal Continuing Medical Education Do Conferences, Workshops, Rounds, and Other Traditional Continuing Education Activities Change Physician Behavior or Health Care Outcomes? JAMA, 282(9), pp. 867874. DAVIS, D.A., THOMSON, M.A., OXMAN, A.D. and HAYNES, R.B., 1995. Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA, 274(9), pp. 700705. DEPARTMENT OF HEALTH, 2001. Working together, learning together: a framework for lifelong learning for the NHS. London: Department of Health. DYKE, M., CONOLE, G., RAVENSCROFT, A. and DE FREITAS, S., 2007. Learning theories and their application to elearning. Contemporary perspectives in elearning research: themes, methods and impact on practice’, part of the Open and Distance Learning Series, F.Lockwood,(ed), RoutledgeFalmer, . GOODYEAR, P. (2005). Educational design and networked learning: Patterns, pattern languages and design practice. Australasian Journal of Educational Technology, 21(1), 82-­‐101. HARDEN, R. and LAIDLAW, J., 1992. Association for medical education in europe (AMEE) medical education guide no. 4: effective continuing education: the CRISIS criteria. Medical Teacher, 26, pp. 408422. HARDEN, R.M., 2005. A new vision for distance learning and continuing medical education. Journal of Continuing Education in the Health Professions, 25(1), pp. 4351. HARDEN, R.M., 2008. Elearningcaged bird or soaring eagle? Medical Teacher, 30(1), pp. 14. HEFCE, 2005. HEFCE strategy for elearning. HEFCE, Bristol, UK available at http://www.hefce.ac.uk/pubs/hefce/2005/05_12/ (Accessed 24 September 2008) HEWITT-­‐
TAYLOR, J., 2003. Technologyassisted Learning. Journal of Further and Higher Education, 27(4), pp. 457464. HUANG, H.M., 2002. Toward constructivism for adult learners in online learning environments. British Journal of Educational Technology, 33(1), pp. 2737. HUTCHINS, H. (2003). Instructional immediacy and the seven principles: Strategies for facilitating online courses. Online Journal of Distance Learning Administration, 6(3). JISC, 2009. Higher Education in a Web 2.0 world, available at www.jisc.ac.uk/publications/documents/heweb2.aspx 44
JISC, 2007a. In Their Own Words. Exploring the learner's perspective on elearning. JISC/HEFCE. available at http://www.jisc.ac.uk/media/documents/programmes/elearningpedagogy/iow final.pdf (accessed 8 September 2008 JISC, 2007b. Podcast 22: Online Conferencing tips from the JISC eLearning team (17 December 2007). available at http://www.jisc.ac.uk/news/stories/2007/12/podcast22judithmolegeoffminsh ull.aspx (accessed 19 August 2008) JOO, Y.J., BONG, M. and CHOI, H.J., 2000. Selfefficacy for selfregulated learning, academic selfefficacy, and internet selfefficacy in webbased instruction. Educational Technology Research and Development, 48(2), pp. 5 17. KENNEDY, T.J.T. and LINGARD, L.A., 2006. Making sense of grounded theory in medical education. Medical Education, 40(2), pp. 101108. KILPATRICK, C., HART, L., NAJEE-­‐ULLAH, D., & MITCHEM, P. (1997). Reflective teaching practice by university faculty: Rationale and case study in computer science. Paper presented at the Frontiers in Education Conference, 1997. 27th Annual Conference. 'Teaching and Learning in an Era of Change'. Proceedings. KIM, K., LIU, S. and BONK, C.J., 2005. Online MBA Students' Perceptions of Online Learning: Benefits, Challenges, and Suggestions. Internet and Higher Education, 8(4), pp. 335344. KUPER, A., REEVES, S. and LEVINSON, W., 2008. An introduction to reading and appraising qualitative research. British Medical Journal, 337(aug07 3), pp. a288. LINGARD, L. and KENNEDY, T.J., 2007. Qualitative Research in Medical Education, 2007, ASME. MARAKAS, G.M., YI, M.Y. and JOHNSON, R.D., 1998. The Multilevel and Multifaceted Character of Computer SelfEfficacy: Toward Clarification of the Construct and An Integrative Framework for Research. Information Systems Research, 9(2), pp. 126163. MAYES, T., 2006. LEX The Learner Experience of eLearning Methodology Report. JISC. available at http://www.jisc.ac.uk/media/documents/programmes/elearningpedagogy/lex _method_final.pdf (accessed 8 September 2008) MCGEE, J.B. and BEGG, M., 2008. What medical educators need to know about "Web 2.0". Medical Teacher, 30(2), pp. 164169. MCKENDREE, J., 2006. eLearning. Edinburgh: ASME. 45
MORRIS and MCKIMM, 2009. Becoming a digital tourist: a guide for clinical teachers. The Clinical Teacher, 6 (1) pp51 – 55 MCKIMM, J., JOLLIE, C. and CANTILLON, P., 2003. Web based learning. British Medical Journal, 326(7394), pp. 870-­‐873. MILLER, M.D., RANIER, R.K. and CORLEY, J.K., 2003. Predictors of Engagement and Participation in an OnLine Course. Online Journal of Distance Learning Administration, 6(1),. MOULE, P., 2006. E-­‐learning for healthcare students: developing the communities of practice framework. Journal of Advanced Nursing, 54(3), pp. 370380. MOYNIHAN, R., 2008. Is the relationship between pharma and medical education on the rocks? British Medical Journal, 337(aug14 1), pp. a925. MURPHY, E., 2004. Recognising and promoting collaboration in an online asynchronous discussion. British Journal of Educational Technology, 35(4), pp. 421431. O'GRADY, G., 2001. Maximising the potential of computer mediated discussion: Guidelines for facilitation. CDTL Brief, 4(4), pp. 1–7. OLIVER, R. and HERRINGTON, J., 2003. Factors influencing quality online learning experiences. In (G. Davies & E. Stacey Eds.) Quality education@ a distance. London: Kluwer Academic Publishers, . OPEN UNIVERSITY, About the OU: history of the OU. Available: http://www.open.ac.uk/about/ou/p3.shtml [September 22, 2008]. PALOFF, R. and PRATT, K., 1999. Promoting collaborative learning, Building learning communities in cyberspace (pp. 110128). San Fransisco: Jossey Bass, . PASSWORD, F., 1998. Comparison of problem based learning and lecture format in continuing medical education on headache diagnosis and management. Medical Education, 32(6), pp. 590-­‐
596. PETRIDES, L.A., 2002. Web Based Technologies for Distributed (or Distance) Learning: Creating Learning Centered Educational Experiences in the Higher Education Classroom. International Journal of Instructional Media, 29(1), pp. 6977. REEVES, S., ALBERT, M., KUPER, A. and HODGES, B.D., 2008. Why use theories in qualitative research? British Medical Journal, 337, pp. a949. REEVES, T. C., HERRINGTON, J., & OLIVER, R. (2005). Design research: A socially responsible approach to instructional technology research in higher education. Journal of Computing in Higher Education, 16, 96-­‐115. 46
ROURKE, L., ANDERSON, T., ARCHER, W., & GARRISON, D. R. (1999). Assessing social presence in asynchronous, text-­‐based computer conferences. Journal of Distance Education, 14(3), 51-­‐70. RUIZ, J.G., MINTZER, M.D., LEIPZIG, R.M., 2006. The Impact of ELearning in Medical Education. Academic Medicine, 81(3), pp. 207212 SALMON. G.2000. E-­‐Moderating: the Key to Teaching and learning online. London. Kogan Page. SANDARS, J., HOMER, M., PELL, G. and CROKER, T., 2008. Web 2.0 and social software: the medical student way of elearning. Medical Teacher, , pp. 15. SANDARS, J. and MORRISON, C., 2007. What is the Net Generation? The challenge for future medical education. Medical Teacher, 29(23), pp. 8588. SHARPE, R. and BENFIELD, G., 2005. The Student Experience of Elearning in Higher Education: A Review of the Literature. Brookes eJournal of Learning and Teaching, 3(1), pp. 19. SIT, J.W.H., CHUNG, J.W.Y., CHOW, M.C.M. and WONG, T.K.S., 2005. Experiences of online learning: students’ perspective. Nurse Education Today, 25(2), pp. 140147. SONG, L., SINGLETON, E.S., HILL, J,R. KOH, M.H. 2004. Improving Online Learning: Students Perceptions of Useful and Challenging Characteristics. The Internet and Higher Education, 7 (1), pp. 5970 TALLENTRUNNELS, M.K., LAN, W.Y., FRYER, W., THOMAS, J.A., COOPER, S. and WANG, K., 2005. The relationship between problems with technology and graduate students' evaluations of online teaching. The Internet and Higher Education, 8(2), pp. 167174. THEOBALD, N., 2008. SYSTEMATIC REVIEW: The experience of using eportfolios in medical and nursing education. (unpublished) WALLACE, R.M., 2003. Online Learning in Higher Education: a review of research on interactions among teachers and students. Education, Communication and Information, 3(2), pp. 241280. TIMMIS, S and COOK, J. 2002. Motivating students towards online learning: Institutional strategies and imperatives, in Kallenberg, AJ and Ven, MJ (eds) The new educational benefits of ICT in Higher Education: Proceedings. Rotterdam: Erasmus Plus. www.oecr.nl/conference WENGER, E., 1998. Communities of Practice: learning, meaning and identity. Cambridge: Cambridge University Press. 47
ANNEX 1 Bedfordshire and Hertfordshire Postgraduate Medical School
University of Bedfordshire
“Clinical educators (dis) engagement in on-line learning”
Survey questionnaire
Introduction and context of study
We are engaged in a small scale study exploring the factors underpinning engagement of
medical and healthcare educators with on-line learning, specifically that related to the
PGCert and Masters programme in Medical Education run by BHPMS, UoB.
The study is part funded by JISC through a small grant from the HEA MEDEV subject
centre.
The main aim of the study is to analyse features of engagement in the on-line
environment (VLE), exploring commonalities and differences in the type, nature,
frequency and motivation of engagement between different student groups.
As part of this study we would very much welcome your participation in the data
gathering process, initially through this survey questionnaire. Some respondents will be
invited to participate in follow up interviews or focus groups.
The data gathered will be anonymised and no one individual’s responses will be
identifiable. Data will be held securely and only the research team will have access to
personalised data. This study is not linked to any formal assessment process and we very
much welcome your open and frank engagement with the study. The findings will be
disseminated back to course participants in the form of a report as well as being
published and disseminated in a wider education arena.
On the basis of this information, by completing this questionnaire and returning it, we are
assuming that you have consented to participate in this study. If you require any further
information prior to completing the questionnaire, please don’t hesitate to contact us.
We very much welcome your participation in this study which will improve our
understanding of those features and support structures which facilitate optimal
engagement with online learning environments and will help to improve the student
learning experience.
Clare Morris and Judy McKimm
February 2008
48
Background information
Name
Professional or subject discipline
Job Role
Year of entry to programme
1. Thinking about your general engagement with computers and online
environments (specifically the use of the internet for home or work purposes), do
you go online
Daily
Weekly
Monthly
Seldom
Never
2. In general; how would you best describe your confidence in using the internet for
general purposes
Very confident Fairly confident Confident in some aspects
Lacking confidence
3. Please tell us about your familiarity with (and use of) the following social learning
/ internet tools for work or leisure reasons
Tool
Unaware of
this tool
Aware but
never used
Aware – use
infrequently
Aware and
regularly use
E- mail
Podcasts
Blogs
Wikis
Webcams
Video conferencing
Rss feeds
Facebook
Flicker
Skype
Second life
Other (please
specify)
4. Before you came on the PGCert/Masters course, had you had any experience of
online learning environments / e-learning as a learner?
49
Yes
Only a little
No
If yes, or a little can you briefly describe this
5. Before you came on the PGCert/Masters course, had you had any experience of
online learning environments / e-learning as a teacher?
Yes
Only a little
No
If yes, or a little can you briefly describe this
6. Have you used BREO?
Yes
No
Very seldom
If ‘yes’, go to Q 9
7. If ‘no’ or ‘very seldom’
Can you please tell us some of the reasons why you have not engaged with BREO
(please tick as many of the boxes below as apply and also add other reasons of your
own)
Reasons
Tick if
this
applies
to you
Comments
Lack of time
Lack of access to a
computer
Hard to access
BREO from work
50
(eg.NHS firewalls)
Doesn’t seem
relevant to my
learning
I communicate in
other ways with
tutors and students
Hard to use
Too slow and
‘clunky’
Time spent on
BREO could be
better spent on other
types of learning
Information on there
of little or no use
I don’t like online
learning
I prefer to meet
people face to face
or use email or
phone
Other (please tell us
more)
8. Can you tell us what support, features or functionality might help or encourage
you to use BREO
If YES
9. What are your main reasons for using BREO
51
52
10. Below is a list of reasons often given for NOT engaging in BREO.
As a BREO user, we are interested to know if you feel any of the following apply to
you, even though you do engage.
Reasons
Tick if
this
applies
to you
Comments
Lack of time
Lack of access to a
computer
Hard to access
BREO from work
(eg.NHS firewalls)
Doesn’t seem
relevant to my
learning
I communicate in
other ways with
tutors and students
Hard to use
Too slow and
‘clunky’
Time spent on
BREO could be
better spent on other
types of learning
Information on there
of little or no use
I don’t like online
learning
I prefer to meet
people face to face
or use email or
phone
Other (please tell us
more)
53
11. How would you describe your main forms of engagement with BREO
Form of engagement
Tick those
which you feel
apply to you
Comment
It would be helpful to know how valuable you
feel these forms of engagement are to you and
your learning
As a ‘watcher’
ie to see what others are doing or saying
To capture my learning in the reflective
diary space
To seek feedback on my writing (e.g. in
the diary space)
To find information about the course
To access learning materials linked to
study days
To access readings and articles from ejournals
To find information for my own learning
To communicate with tutors
To communicate with other students
To participate in online discussions and
debates
To engage with on-line learning tutorials
linked to assessment activity
As a portal (access link) to other
sites/resources
To submit assignments
Other
12. As a learner on the Masters programme, what do you think are the main purposes
of BREO
13. As a learner and teacher what do you think is the potential or actual value or
BREO?
54
14. What particular features of BREO have you used, please indicate your usage
BREO features
Programme,
calendar and
timetables
Admin info:
handbooks etc
Announcements,
keeping up to date
with course
Information on
assessments
Digital assignment
drop box
Materials from
contact days
Frequently
Seldom
Once
Never
Preparatory
materials for contact
days
Follow up materials
for contact days
Reflective diary/log
Quick links to
useful websites
Link to library
Link to e-journals
and search engines
To send emails to
tutors and students
Moderated
discussions with
‘experts’
Discussion boards
Group spaces linked
to learning
sets/group activities
Wiki to develop
shared resources
Other (please
specify)
55
15 Which three features of BREO have been of most use to you (1 is the most),
please indicate why.
Feature
1
Reason for usefulness
2
3
16 What features of BREO are the least useful to you, please indicate why
Feature
1
Reason for lack of usefulness
2
3
17 What features of BREO would you like to see made available, please indicate
why
Feature
1
Reason why
2
3
18 Please tell us about any barriers or frustrations to your using BREO as an
effective online learning environment
19 Can you tell us what support, features or functionality might help or encourage
you to make more use of BREO?
20 What do you think of online learning being viewed as appropriate and
necessary for clinical educators as learners?
21 Are there any specific needs or barriers to clinical educators and students
making use of online learning
22 Finally, have you any further comments
Thank you very much for your time
ANNEX 2
BREO TRACKING DATA: FIGURES
Analysis of access data by day of week Figures 1 to 5 display graphically the access data (number of ‘hits’) presented by day of week. The pattern varies between teaching modules and cohorts with a general trend to less use at weekends – more apparent in Fig. 5 when all four sets of data are combined. Figure 1: Access Data by Day of Week (Medical Education in Practice 2006/2007) Figure 2: Access Data by Day of Week (Contemporary Perspectives 2006/2007) Figure 3: Access Data by Day of Week (Medical Education in Practice 2007/2008) Figure 4: Access Data by Day of Week (Contemporary Perspectives 2007/2008) Figure 5 Access Data by Day of Week (All cohorts/units data combined) 37