May be copied for educational use provided that the source is acknowledged. Review of educational research on virtual learning environments [VLE] - implications for the improvement of teaching and learning and access to formal learning in Europe Paper presented at the European Conference on Educational Research, University of Ham burg, 17-20 September 2003 Abstract The importance of VLEs in European Higher Education has grown in the last five years under the influence of national developments and European policy initiatives. These changes have been accompanied by, and extended thr ough the initiatives of multinational corporations based in North America, although a number of national products, including open source VLEs are available. This situation is described in the EU e - learning strategy ‘Virtual campuses for all students’. “By end 2005, Member States, supported by the eLearning and eTEN programmes should ensure that all universities offer online access 1 for students and researchers to maximise the quality and efficiency of learning processes and activities." 2 The cost struc ture of the products of the major commercial suppliers tends to encourage central provision of VLE and, it is argued, a degree of inflexibility in delivery especially when linked to student management information systems. [Also known as a Managed Learning Environment – MLE] It is also argued that this approach promotes a degree of pedagogical inflexibility. In this situation, it is argued that the evolution of international specifications such as IMS 3 and SCORM 4 are likely to reflect market power rather than educational needs. The paper will review the main features of VLE and the principal approaches to their implementation and will analyse their contribution to learning with reference to two models "Cognitive Apprenticeship” 5, and "Conversational Framew ork for the Learning Process 6" A range of approaches to the specification of user requirements will be discussed and conclusions proposed as to the implications for universities in Europe. © Jo h n K on r ad 03 j o h n . k o n r a d @ d s l . p i p e x . c o m EC E R 200 3 N et wo r k 22 1 May be copied for educational use provided that the source is acknowledged. Although the process described above is constructed around general theoretical perspectives and policies, there is limited evidence that this process has been regarded as problematic from the perspective of the users, the students or participants in formal learning. Policy framework In the UK and other Anglophone count ries, there is now a process of systematic review of the use and impact of Virtual Learning Environments [VLE] 7. The benefits of this approach are cited as lower costs per student for Universities (economies of scale) as well as guaranteed high quality le arning for students who wish to learn ‘anywhere, anytime’ 8. The nature of the issues are well summarised by Paul Bailey of the UK Joint Information Systems Committee [JISC] 9 who identifies the policydriven nature of the use of VLEs as a means of increasi ng participation in Higher Education of people 18 -30 to 50%, principally by progression from Further Education; Sue Timmis and Julian Cook 10 address the issue of student motivation from an objectivist perspective concluding that VLEs need to provide learnin g opportunities not available elsewhere, clear expectations, guidance and support to enable students to respond to the potential of the VLE; Mark Stiles 11 identifies some debates about the learning process and advocates that we should be adventurous in our choice of pedagogies. Implementation of Virtual Learning Environments An Analysis of Users’ Needs is a frequently used strategy for any major software implementation 12. The JISC PORTOLE project 13 has developed a useful approach which involves a three -stage model: 1. “Identify possible objectives . The first step involved the collection of general set of potential objectives. These were identified from experts in the field of e learning and resource discovery, (academics, librarians, and IT developers) and f rom the project team and collated by the project manager. 2. Decide on important objectives . Focus groups were conducted in order to discuss the importance of different potential features for the tool. 3. Select a final set of objectives which will lead to the evaluation analysis.” 14 This approach assumes that an organisation would take a corporate approach to the implementation of a VLE. In © Jo h n K on r ad 03 j o h n . k o n r a d @ d s l . p i p e x . c o m EC E R 200 3 N et wo r k 22 2 May be copied for educational use provided that the source is acknowledged. one UK University, the Senior Management identified a systems requirement in terms of creating links between such major Information Systems such as Student Information [Registration, Enrolment onto Modules, Recording assessment, and Publishing results], Learning Resources management, Access to the Campus through a single portal. A representative group of teaching staff was consulted on the implementation of the strategy and three VLE products were identified as meeting the system requirement. Three course teams were given limited support to pilot one of these products for a semester. At the end of the semester, a group of B achelors’ students conducted their final dissertation in order to evaluate the pilots. The reports were submitted to Senior Management who decided on the final product. Mark Stiles has summarised an alternative view which identifies the importance of go od course design, the need for a planned, pedagogical approach and the importance of staff development when selecting and implementing a VLE . 15 The area of “appropriate pedagogy” has recently been widely debated and this issue will be returned to in the next Section. A major commercial influence on these developments is the software platform WebCT [Web Course Tools] which currently dominates the UK market 16 and is a major force in the rest of Europe 17. “WebCT's vision is to work in concert with leading insti tutions to go beyond online course delivery and actually transform the educational experience. To that end, we will deliver state -of-the-art educational technology that supports a full range of teaching and learning styles and optimizes intellectual and technical resources.” 18 “Characteristics of successful online students - not all students are successful in online courses. Some students have trouble with time management, need the structure of a classroom, or miss the face -to-face interaction with other students and the instructor. The following are some characteristics students should have to be successful learning online. Discipline to complete projects by deadlines instead of waiting until the end of the semester. Motivations to read, write, and part icipate fully in class activities. © Jo h n K on r ad 03 j o h n . k o n r a d @ d s l . p i p e x . c o m EC E R 200 3 N et wo r k 22 3 May be copied for educational use provided that the source is acknowledged. Time to devote approximately 12 hours a week to a 3-credit course. [equivalent to 10 ECTS] Ability to work independently and in teams. Flexibility in dealing with technology problems. Self-starter does not procrastina te. Able to learn from the printed word. Majority of material and communication happens through reading and writing. Set aside specific times on a routine basis to 'participate' in the course. Can ask questions when they do not understand. Access to a current computer and the Internet. Good, basic computer skills. Possess time management skills.” 19 Let us reflect for a moment on ourselves and our students. Are these criteria fully realisable in practice? Do they take enough account of the competing pressures of everyday life? An influential approach to teaching and learning in Higher Education has been the “ Seven Principles for Good Practice in Undergraduate Education.” This study suggests that “Good practice in undergraduate education: 1. Encourages contacts between students and faculty. 2. Develops reciprocity and cooperation among students. 3. Uses active learning techniques. 4. Gives prompt feedback. 5. Emphasizes time on task. 6. Communicates high expectations. 7. Respects diverse talents and ways of learning.20” This framework was updated more recently recognising that the multi -functional nature of VLE implies the need to be more selective to ensure fitness for purpose 21. For example, asynchronous staff -student and student-student communication by email (ins ide or outside a VLE) can improve understanding if all use it effectively (especially reading their Inbox regularly). However teachers’ expectations of a regular pattern of “office hours may not be congruent with expectation of students of ‘support at any time’”. © Jo h n K on r ad 03 j o h n . k o n r a d @ d s l . p i p e x . c o m EC E R 200 3 N et wo r k 22 4 May be copied for educational use provided that the source is acknowledged. The writer’s experience with first year full -time students aged 18-20 is that their motivation tends to be instrumental and defined as much in terms of University as a social process and possibly a rite of passage . In particular, their level of competence and confidence in using ICT applications is often too low to enable engagement with a VLE. However, the minority of the students who did not have these handicaps were very positive about the possibilities of the technology in extending the range of their learning. The experience with part-time postgraduate professional students is quite the reverse and having built an initial peer group through face to face Induction at the beginning of a Semester, the group becomes more self-sustaining. This experience supports the Model that “Learning is built around learning communities and interaction, extending access beyond the bounds of time and space, but offering the promise of efficiency and widening access. 22” The following view is fully supported: “A successful online learning community has many of the same characteristics as a 'real' community. It offers individual support to its members, so that they can feel safe to communicate openly, which in turn allows them to develop the shared vision that they need in order to learn together. 23” Pedagogical issues The principle and dominant model of e -learning in the Anglophone commercial world is ‘blended learning’. “The term blended learning is used to describe a solution that combines several different delivery methods, such as collaboration software, Web -based courses, EPSS, and knowledge management practices. Blended learning also is used to describe learning that mixes various event based activities, including face -to-face classrooms, live e learning, and self-paced learning. Unfortunately, there's no single formula that guarantees learning, but here are some guidelines from NIIT on how to order your learning activities. 24” As indicated in the critiques referenced in the last endnote, there appears to be a range of responses to this clear approach. In the writer’s experience with post graduate part time professionals, this concept often proves useful as an entry point to the conceptual “jungle”. “Almost anyone who has taught online would argue that the demands on online tutors are different from those on face-to-face tutors, although the general issues and © Jo h n K on r ad 03 j o h n . k o n r a d @ d s l . p i p e x . c o m EC E R 200 3 N et wo r k 22 5 May be copied for educational use provided that the source is acknowledged. situations with which they must deal are, in essence, the same. The online tutor must manage a course, guide students throughout the learning ex perience, motivate them, interact with them, assess them and deal with any conflicts or difficulties. The differences in tutor role result from the characteristics identified above: the absence of non -verbal clues, the use of text as the main means of co mmunications, and the constraints imposed by technology. However, any list of roles that can be produced should only be regarded as a general framework. 25” The COSE project at the University of Staffordshire 26, led by Mark Stiles, has made a major contribu tion to developments in the pedagogy of online learning. “eLearning presents a challenge to the roles and responsibilities of academics and “support” professionals. Apart from the expected “quality processes” most traditional course development is done in an ill-formed and often quite ad-hoc way. … Involvement of support professionals … is very rarely integrated in any real way into the course design/development process. eLearning, by its very nature, demands considerably more planning. However, there is little evidence that pedagogy is much considered in this process, with far too many staff seeking to model traditional practice onto e -delivery. Support, at all levels, is often either overlooked or not effectively used. Indeed, it is very often the c ase that developers have not thought through the reason why they are going to use eLearning in the first place. (Sometimes the reason for doing it is solely that funding exists!). Overall, the essential amateurism of HE course design and development is thrown into sharp relief by eLearning. 27” An earlier paper put these issues in a sharp focus: “Whilst the ultimate potential of VLEs is unlikely to be realised until the standards and technologies are in place to facilitate their interoperation with each othe r and wider organisational and information systems, the view of the developers of COSE is that their successful deployment depends first and foremost on addressing the pedagogic issues associated with effective learning and ultimately on the overall qualit y of course design and learner support. The author contends that over -attention on the "features" provided by VLEs can lead to a "check-list" approach to VLE selection, which, coupled with inattention to the educational issues, can result in mere transpos ition of © Jo h n K on r ad 03 j o h n . k o n r a d @ d s l . p i p e x . c o m EC E R 200 3 N et wo r k 22 6 May be copied for educational use provided that the source is acknowledged. traditional teaching approaches to the computer, and result in a poor learning experience which is ineffective. 28” “There is a need for institutions to address within their curriculum the developmental needs of ‘non -traditional learners’ and most i nstitutions have taken steps in this direction. Provision for this can be “separate” – where provision is specifically targeted at non -traditional learners or “integrated” where provision is aimed at developing requisite capabilities in all learners (Warr en, 2002). Warren argues strongly for the integrated approach as the best means of meeting both the widening participation and “skills” agendas. This view is in concurrence with that of the author, who has previously argued that the development of the us e of meaningmaking, or semiotic, tools which are specific to given disciplines or professions and the reapplication of the tools and practices of the given culture, coupled with recognition of the relatedness of tacit and codified knowledge in the context of tasks or problems, is fundamental to the development of expertise (Stiles 2000a). This supports a view that skills are more likely to be ‘transferable’ when developed in an authentic context, rather than treated in a separate, “generic” way. The author proposes that unless there is a national focus on the aspects of eLearning concerned with “form” (pedagogy and assessment) which is aimed at bringing these into balance with the current focuses on “content” (curriculum and resources) and issues of techno logy, students will be increasingly provided with isolating and passive learning experiences which, in turn, will impact most negatively on those very learners which the government is concerned to involve in FE/HE participation. This negative impact could well disillusion those staff who are currently becoming involved in the use of technology for the first time, and may result in a “backlash” among teaching staff and university managers, resulting in a set back to the achievement of national goals and competitiveness. 29” These critiques and similar views on a crude transference of conventional approaches to the VLE context have identified a relationship between successful online learning and the use of constructivist approaches. Issues in Evaluation of VL E use As indicated in the select bibliography, there is no shortage of recent work in this field. From a business © Jo h n K on r ad 03 j o h n . k o n r a d @ d s l . p i p e x . c o m EC E R 200 3 N et wo r k 22 7 May be copied for educational use provided that the source is acknowledged. perspective, it is generally agreed that “… in the US less than 25% of businesses evaluated the impact of training on business results. In th e UK, we suspect that it is a lot lower.” (Clark 2003:3) Apart from the projects cited below, the dominant approach is that of Donald Kirkpatrick 1959 four-level model 30. This puts forward the following structure: Level Target Evaluation goal 1. Reaction Training Initial endorsement by participants of the training measured by reaction questionnaires. 2. Learning Learner on the course That learning occurred because of the training, normally assessed by performance tests. 3. Behaviour Learner on the job That learning affected behaviour, or performance on the job assessed by observation and productivity data. 4. Results Organisation That the training had the desired results in the organisation, assessed by cost data, quality indicators and return on investment. The Institute of Employment Studies [IES] has recently published a study of Kirkpatrick’s approach and approaches developed from the original model. It forms an excellent starting point for a review of approaches to the evaluation of training and learning as well as reviewing alternative models which focus more on the purpose of evaluation and on alternative measures. (Tamkin P et al, 2002) Main findings include the lack of an established causal relationship between ascending levels; little corre lation between learner reactions and measures of learning or subsequent measures of changed behaviour (Levels 1 & 2); a failure to transfer training to the workplace (Level 3) which related to issues of how difficult the learner perceived the course (negat ive) and how much line managers were supportive (positive) (Level 4). (Tamkin P et al, 2002: xi -xiii) Previous IES Research (Pollard and Hillage 2001) is cited which raises the key issue about e -Learning “How can you tell if it is working?” in view of th e high costs of implementing the technology; the need to demonstrate the efficiency and effectiveness of this new learning solution; and the need to show organisational value of this (as yet) © Jo h n K on r ad 03 j o h n . k o n r a d @ d s l . p i p e x . c o m EC E R 200 3 N et wo r k 22 8 May be copied for educational use provided that the source is acknowledged. unproven process. The main approach identified was the measurement of performance, competences and intellectual capital. (Tamkin P et al, 2002: 18 -22) ASTD has conducted a major study on the evaluation of effectiveness and return on investment of e -learning and this makes clear that the availability of rich data within VLEs offers an important evaluation opportunity 31. A recent seminar paper concluded that: “What is lacking is a theoretical basis and a coherent research framework. There is little systematic research into broad based issues and concepts, or the generation of transferable models and processes of evaluating e-learning or into the design of tools for analysing, rather than collecting, data. Furthermore, there are few papers written which collate the results of the existing research and classify it in an accessible way. Nor is there substantial evidence of work that extrapolates and tests generalisable principles arising from the case studies and surveys or which comments on the implications or application of these in a European VET arena.” (Hughes and Attwell 2002/03) The writer argues that with the increasing use of competence based national qualification frameworks 32; such highly functional approaches should form a significant point of departure for teaching and learning in Higher Education, especially in the light of the earlier discussion on “Blended Learning.” In particular, there are significant implications for the working conditions of teachers, support staff, and learners of a move towards a “24/7” approach. Towards evaluative frameworks The bibliography of this Review indicates some important resources that are available for the evaluation of VLEs. The IHEP Benchmark study (Phipps & Merisotis 2000) has provided the basis for a pilot investigation of its appropriateness for a European cultural context o n the JISCMAIL list “Evaluation of Online Learning” [ w w w . j i s c m a i l . a c . u k ]. A copy of this document will be published as an Appendix to this paper. Another important resource is the “Evaluation Cookbook” (Harvey 1998) which makes effective use of a “recipe” approach to provide a range of practical approaches to the evaluation of learning technology (not just VLEs) for a wide variety of stakeholders. At a more specialist level and building on the above work, the JISC Evaluation Toolkit (Conole 2000) provide s a © Jo h n K on r ad 03 j o h n . k o n r a d @ d s l . p i p e x . c o m EC E R 200 3 N et wo r k 22 9 May be copied for educational use provided that the source is acknowledged. structured resource for effective evaluation. The writer has not been able to discover published evidence of its use outside the development context. In summary, the state of the art in evaluation of the use of VLEs in Higher Education is a matter of “under-use” rather than under-theorising! As this review has shown, there is a wide variety of theoretical and practical approaches available for use. In my view, it is unlikely that teaching and learning using VLEs in Higher Education will become more ef ficient and effective without significant applied research and evaluation. John Konrad John.konrad@dsl.pipex.com © Jo h n K on r ad 03 j o h n . k o n r a d @ d s l . p i p e x . c o m EC E R 200 3 N et wo r k 22 10 May be copied for educational use provided that the source is acknowledged. A select bibliography of Educational Research on virtual learning environments [VLEs] Britain, S & Liber, O, (1999), “A framework for pedagogical evaluation of virtual learning environments”. A report presented to the Joint Information Systems Committee's Technology Applications Programme (JTAP). http://www.leeds.ac.uk/educol/documents/00001237.htm Clark D (2003), “Evaluation of Learning”, An Epic White Paper, Epic Group plc, Brighton UK [ h t t p : / / w w w . e p i c . c o . u k ] Connole G (2000), “An Evaluation Toolkit for Practitioners”, Institute for Learning and Research Technology Bristol, w w w . l t s s . b r i s . a c . u k / i n t e r a c t 2 1 / i n 2 1 p 0 6 . h t m Cousin G, & Deepwell, F. (1998). “Virtual Focus Group Techniques in the Evaluation of an E lectronic Learning Environment”, In Oliver, M. (Ed.) ELT 98: Innovation in the Evaluation of Learning Technology Conference Proceedings, London: University of North London, 4 -7, http://www.unl.ac.uk/tl tc/elt/elt98.pdf Harvey J (ed.) (1998), “The Evaluation Cookbook”, Learning Technology Dissemination Initiative, Edinburgh, www.icbl.hw.ac.uk/itdl Hughes J & Attwell G (2002/03), “A Framework for the Evaluation of E-Learning”, Paper presented to a seminar series on Exploring models and partnerships for eLearning in SMEs, held in Stirling, Scotland and Brussels, Belgium, in Nov 2002 and Feb 2003, http://www.theknownet.com/ICT_SMEs_seminars/papers.html Joyes G (2000), “An evaluation model for supporting higher education lecturers in the integration of new learning technologies”, Educational Technology & Society 3(4), h t t p : / / i f e t s . i e e e . o r g / p e r i o d i c a l / v o l _ 4 _ 2 0 0 0 / j o y e s . h t m l McFadzean, E (2001a), “Supporting virtual learning groups: Part 1: a pedagogical perspective” Team Performance Management: 7, 3/4. 53-62. McFadzean, E (2001b), “Supporting virtual learning groups: Part 2: an integrated approach” Team Performance Management, 7, 5, 77-93 Milligan, C (1998), “The role of virtual learning environments in the online delivery of staff development. Phipps R & Merisotis J (20 00), “Quality on the Line: Benchmarks for success in Internet -based Distance Education”, The Institute for Higher Education Policy, Washington DC, April, w w w . i h e p . c o m © Jo h n K on r ad 03 j o h n . k o n r a d @ d s l . p i p e x . c o m EC E R 200 3 N et wo r k 22 11 May be copied for educational use provided that the source is acknowledged. Pollard E & Hillage J (2001), “Exploring e -Learning” IES Report 376, Brighton UK, w w w . e m p l o y m e n t - s t u d i e s . c o . u k Report 1: Review of experiences of delivering TALiSMAN online courses”. A report presented to the Joint Information Systems Committee's Technol ogy Applications Programme (JTAP). http://www.leeds.ac.uk/educol/documents/00001230.htm Oliver, M. (1999). “The ELT Toolkit” , http://www.unl.ac.uk/tltc/elt/toolkit.pdf Oliver M (2000), “An introduction to the Evaluation of Learning Technology”, Educational Technology & Society 3(4), h t t p : / / i f e t s . i e e e . o r g / p e r i o d i c a l / v o l _ 4 _ 2 0 0 0 / i n t r o . h t m l Richardson, Julie A (2001), “Changes and challenges of academic lives through the introduction of virtual learning environments”, Paper presented at SCUTREA, 31st Annual Conference, 3 -5 July 2001, University of East London. http://www.leeds.ac.uk/educol/documents/00002517.htm Richardson, Julie A & Turner A, (2000), “A Large -scale ‘local’ evaluation of students’ learning experiences using virtual learning e nvironments”, Educational Technology & Society 3(4), h t t p : / / i f e t s . i e e e . o r g / p e r i o d i c a l / v o l _ 4 _ 2 0 0 0 / r i c h a r d s o n . h t m l Scanlon E, et al (2000), “Evaluating information and communicatio n technologies for learning”, Educational Technology & Society 3(4), http://ifets.ieee.org/periodical/vol_4_2000/scanlon.html Taylor J, Woodman M, Sumner T, Blake C T, (2000), “Peering Through a Glass Darkly: Integrative evaluation of an on-line course”, Educational Technology & Society 3(4 ), http://ifets.ieee.org/periodical/vol_4_2000/taylor.html Tamkin P et al (2002), “Kirkpatrick and Beyond”, IES Report 392, Brighton UK, w w w . e m p l o y m e n t - s t u d i e s . c o . u k van der Veen J, & de Boer W, “W3LS: Evaluation framework for World Wide Web learning”, Educational Technology & Society 3(4 ), http://ifets.ieee.org/periodical/vol_4_2000/veen.html A m o r e g en e ral t e r m i s “ on l i n e l e a rn i n g” s o m eti me s m o r e gen e r al l y kn own a s “ e- l ea rn i n g”. T h e l att e r t e rm i s u s e d i n a c om m e r ci al 1 © Jo h n K on r ad 03 j o h n . k o n r a d @ d s l . p i p e x . c o m EC E R 200 3 N et wo r k 22 12 May be copied for educational use provided that the source is acknowledged. c on t e xt an d al s o i n c l u des th e u s e o f su c h “ o ff - l i n e” mat e ri al a s CD RO M an d D VD . In F r en ch , th e equ i val e n t t e rm i s “l ’a pp r en t i ssag e en li gn e”- « M o d e d ' ap pr en ti s sa g e ba s é su r l 'u ti li sati on d es n o u v el l es te ch n ol ogi es , p e rm e ttan t l 'a cc è s à d e s f o rm ati on s en l i gn e, i n t er ac ti ve s et pa r f o i s p e rs on n al i sé e s, d i ffu s é e s pa r l 'i n te rm édi ai r e d' In t e rn et, d 'u n i n tr an e t ou au t r e m é di a él e ct r on i qu e , afi n d e dé v el op pe r l e s c o mp ét en c e s, t ou t en r en dan t l e pr o c e s su s d'a pp r en ti s sag e i n d ép en dan t d e l 'h eu r e et d e l ' en dr oi t. » http://www.olf.gouv.qc.ca/ressources/bibliotheque/dictionnaires/Internet/fiches/8 872865.html C om mi s si on o f th e Eu r op e an C omm u n i ti es (20 02) , e E u r o p e 20 05: An in f o rma ti on s o ci et y f o r all , C O M ( 20 02) 263 fi n al , B ru s s el s , Pag e 13. 2 http://europa.eu.int/information_society/eeurope/news_library/d ocuments/eeurop e2005/eeurope2005_en.pdf 3 The IMS Global Learning Consortium develops and promotes the adoption of open technical specifications for interoperable learning technology. Several IMS specifications have become worldwide de facto standards for delivering learning products and services. IMS specifications and related publications are made available to the public at no charge from www.imsglobal.org. No fee is required to implement the specifications. IMS is a worldwide non-profit organization that includes more than 50 Contributing Members and affiliates. These members come from every sector of the global e-learning community. They include hardware and software vendors, educational institutions, publishers, government agencies, systems integrators, multimedia content providers, and other consortia. The Consortium provides a neutral forum in which members with competing business interests and different decision-making criteria collaborate to satisfy real-world requirements for interoperability and re-use. (my emphasis) http://www.imsproject.org/aboutims.cfm Sharable Content Object Reference Model [SCORM] provides a technical architecture for learning objects to be easily shared across multiple learning delivery environments. The SCORM specification describes an architecture built on specifications designed to address the US Department of Defence's (DOD) frustration with not being able to share distance learning courses among its different learning delivery systems. This set of specifications when applied to course content, produces small, reusable learning objects. These can be easily merged with other compliant elements to produce a highly modular repository of learning material. 4 http://www.cetis.ac.uk/encyclopedia/entries/20011129121727/view “ In tra di ti on al app r en ti c e sh i p, th e e xp e rt s h o w s th e app r en ti ce h o w t o d o a ta s k, watch e s a s th e app r e n ti ce p ra cti c e s p o rti on s o f th e ta s k, a n d th en t u rn s ove r mo r e an d mo r e r es p on si bi l ity u n ti l th e app r en ti c e i s p r o fi ci en t en ou gh t o a cc o mpl i sh th e ta sk i n dep en d en tl y. Th a t i s th e ba si c n oti on of ap p r en ti c e sh i p: sh owi n g th e a pp r en ti c e h o w to d o a ta s k an d h el pi n g th e app r en ti c e to d o i t. Th e r e a r e f ou r i mp o rta n t as p ec ts o f t ra di ti on al app r en ti c e s h i p: mo d el l i n g, sc af f ol di n g, f adi n g, an d c oa c h i n g. … Ou r di s cu s si on o f c ogn i ti ve app r en ti c e sh i p rai s es n u m er ou s p eda go gi cal an d t h e o r eti c al i ssu e s th at w e b el i eve a r e i m po rt an t t o th e d e si g n of l ear n i n g en vi r on m en ts g en eral l y. To fa ci li tate con si d erati on o f th e s e i ssu e s, w e h ave d evel op e d a fram e w or k c on si s ti n g of f ou r di men si on s th at c on sti tu t e any l e a rn i n g en vi r on m en t: c on t e n t, me th od, s equ en c e , an d s o ci ol og y. Rel e van t t o ea ch of th e s e di men si on s i s a s et o f ch a ra ct e ri sti c s th at w e b el i eve sh ou l d b e 5 © Jo h n K on r ad 03 j o h n . k o n r a d @ d s l . p i p e x . c o m EC E R 200 3 N et wo r k 22 13 May be copied for educational use provided that the source is acknowledged. c on si d e r ed i n c on st r u cti n g o r e val u ati n g l ea rn i n g e nvi r on m e n ts . … Co gn i ti ve a pp r en ti c e sh i p i s n o t a m od el of t ea ch i n g th at gi ve s te ach e r s a pa c kag e d f o rm u l a f o r i n st ru ct i on . In s te ad , i t i s a n i n stru cti on al pa radi gm f o r t ea ch i n g. C ogn i ti ve app r en ti c e s h i p i s n ot a r el e va n t m od el f o r al l asp e ct s of t ea ch i n g. It d o e s n ot m a ke s en s e t o u s e i t to t e ach th e ru l e s o f c on j u gati on i n F r en ch o r t o te ach th e el em en ts o f th e p e ri o di c t abl e . If th e ta rg et e d g o al o f l ear n i n g i s a r ot e ta sk , c o gn i ti ve ap p r en ti ce sh i p i s n ot an app r op ri at e m od el o f i n st ru cti o n . C ogn i ti ve a pp r en ti c e sh i p i s a u s ef u l i n str u cti on al para di gm wh en a t e ach e r n e ed s to t e ac h a f ai rl y c ompl ex ta s k to stu den t s .” http://www.21learn.org/arch/articles/brown_seely.html Th e m ai n pu rp o s e o f t h e C on v e r sati on al M od el f o r K n o wl ed g e App r en ti c e sh i p i s to r ev e al th e c on v e rs a ti on s ta ki n g pl a c e b et w e en an e xp e rt an d a n o v i ce . Fu rth e r , i t i s al s o t o i l l u strat e th e bu i l di n g o f ca pabi l i ti es i n b o th a c on t ex t - sp e ci fi c an d i n a g en e ri c s en s e. Th e CM K A i s a m o d e l th at bu i l ds u p on t wo w el l - esta bl i sh ed te ach i n g an d l ea rn i n g th e o ri e s: th e c on v e rs ati on al f ra m ew o r k ( La u ri l l ard, 1993 P g. 1 02 - 1 04) an d c ogn i ti v e app r e n ti ce sh i p ( C ol li n s, B r own , an d N e wman , 19 89) . Di an a Lau ri l l ard' s c on v e r sati on al m od el att emp ts t o r ev e al t h e ph ysi c al c on v e r s ati on s t aki n g pl ac e bet w e en an i n st ru ct o r an d a stu d en t wh en i n v ol v ed i n th e t ea ch i n g a n d l ear n i n g a r en a . Th es e c on ve r s ati on s a r e th e " c on s ci ou s pr o c e ss e s a c c es si bl e t o th e l ear n er t o co n si d e r an d m odi fy " ( Lau ri l lar d, 19 93 P g. 10 2) . In c on c e r t wi th th e s e c on v e rs ati on s, th e La u ri l l ardi an f ra m ew o r k al s o r e v eal s th e ac ti vi ti es th at a r e n e c es sa r y f o r a c o mp r eh en si v e l ear n i n g p r o c es s . T h e c on v e r sati on s an d th e a cti vi ti es ta k e pl ac e wi th i n tw o di f f e r en t stat e s of kn owl e dg e c on st ru cti on : kn owl edg e bu i l di n g an d kn o wl e dge appl i cati on . Wi th i n kn owl e dg e b u i l di n g, th e i n s t ru c t o r an d th e s tu d en t i n te r act t o c r eat e th e c on t ext a n d t o bu i l d th e 'i n tr odu ct o r y' u n d e r st an di n g o f th e m at e ri al o r su bj e ct mat t e r. Wi th i n kn owl e dg e a ppl i cati on , th e i n st r u ct o r an d th e s tu d e n t i n ter ac t t o a ppl y th e kn o wl edg e th at w as bu i l t (i n th e kn o wl edg e bu i l di n g acti vi ti e s), wi th i n an i n te r ac ti ve c on t ext . In t ot al , Lau ri l l ard s pe ci fi e s tw el v e a cti v i ti es th at mu st o c cu r i n o rd e r t o h av e a w el l - r ou n d e d l e a rn i n g e n vi r on m en t th at c on si st s o f both kn owl e dg e bu i l di n g an d k n o wl ed g e ap pl i cati on 6 http://watserv1.uwaterloo.ca/~acpalmer/purpose.html F o r a g oo d i n t r odu c ti on t o th e fi el d an d s o m e M od el s of d el i ve ry , s e e “D el i v eri n g L e a r n i n g On l i n e: A c ou r s e f o r f aci l i tato rs ” , 7 http://www.tafe.net/learnscope/dlo/learning8.html (Last Visited 14/09/2003) “Wi th i n on l i n e l ea rn i n g th e r e a r e a n u m be r of d el i ve r y pa ra di gms th at t ea ch e rs u s e , d ep en d en t on th ei r b el i ef s ab ou t h ow p e opl e l ear n ( w e c al l th i s p ed ag og y) an d th ei r ac c e ss t o an d a bi li ty t o u s e th e t e ch n ol og y . T h e s e ca n b e c on d en s ed i n to th e f ou r m o del s bel o w . Wh i l e t ea ch e r s wi l l mo v e i n an d ou t o f th e m od el s a n d s om e wi ll pr og r e ss f r om o n e m od el t o an oth e r a s th ei r u n d e rs tan d i n gs, bel i ef s an d ski l l s ch an g e, th e y d es c ri b e th e p ra cti c e s th at w e s e e i n th e V E T s e ct o r n ow to v a ryi n g d eg r e e s. ” F o r e xa mpl e , “ Hi gh e r e du cati on t od ay f ac e s ri si n g en r ol m e n ts an d c os ts , a d em og r aph i c sh i ft t ow a rd ol de r stu d en t s an d l i fel on g l ear n i n g, an d a g r ea te r d em an d f o r an yt i me/an ywh e r e l e ar n i n g an d on l i n e s e rvi c e s .” Ir vi n e M( 200 3), Th e E me rg in g e - Edu cat io n 8 © Jo h n K on r ad 03 j o h n . k o n r a d @ d s l . p i p e x . c o m EC E R 200 3 N et wo r k 22 14 May be copied for educational use provided that the source is acknowledged. Lan d s ca p e: A Bla ck bo ar d St rat eg ic Wh i te P ap e r , p ag e 5 http://products.blackboard.com/cp/release6/CIOSeriesWhitePaper.pdf http://www.ukeu.com/aboutelearning.shtml s e e al s o Bai l e y P (20 02) , “ In teg r at ed M an ag e d L ea rn i n g Sy st em s fo r Li f el on g L e a rn i n g: F act o r Fi cti on ? Th e Wa y F or wa r d with V L E s: L es s on s L e a rn ed , 2 4 Ju n e 20 02 , U n i ve r s i ty of B ri st ol h t t p : / / w w w . l t s s . b r i s . a c . u k / e v e n t s _ v l e 6 0 2 . h t m ( L a s t V i s i t e d 1 4 / 0 9 / 2 0 0 3 ) N O T E: “Th e c on c e pt o f a M an ag ed L ea rn i n g En vi r on m en t ( ML E ) r e f e r s t o th e wh ol e ra n g e o f i n f o rmati on s ys t em s an d p ro c e s s es o f a c ol l eg e o r u n i ve r si ty th a t c o n tri bu t e di r e ctl y, o r i n di re ctl y , t o l e a rn i n g an d th e m an ag e m en t of th at l ea rn i n g. Th i s al so i n cl u de s th e i n sti tu ti on ’s Vi rtu al L ea rn i n g En vi r on m e n t (V L E) i f i t h a s on e . M L E s ar e al s o r ef e r r ed t o as i n teg r at ed en vi r o n me n ts f o r l e a rn e rs be cau s e th e y p r ovi d e an o pp o rtu n i ty f o r c ompl e t e i n teg r ati on of al l edu ca ti on al a cti vi ti e s. Di f f e r en t o r gan i s ati on s ma y h av e di f f e r en t typ e s o f M L E s b ec au s e th ey h a v e di ff e r e n t edu ca ti on al an d bu si n e s s n e ed s . F e w c ol l eg e s an d u n i v e rsi ti e s c u r r en tl y h a v e fu l l y de v el op ed ML E sy st e ms i n pl a c e bu t m an y a r e p r og r e ssi n g t ow ar d th i s go al .” 9 http://www.jisc.ac.uk/index.cfm?name=about_mle (Last Visited 14/09/2003) Th e r an g e of r es e a r ch an d d e v el opm e n t i n th i s a r ea a r e su mma ri s ed at h t t p : / / w w w . j i s c . a c . u k / i n d e x . c f m ? n a m e = p r o g r a m m e _ m l e _ h e Ti mmi s S & C o ok J (2002 ), “ ’ Th e N egl e ct ed H ea rt ’ o f On l i ne L ea rn i n g? M oti v ati n g stu d en t s t o w o rk e ff e cti v el y wi th On l i n e L ea rn i n g En vi r on m e n ts ”, Th e Wa y F o rw ar d with V L E s: L es s on s L e a rn ed , 2 4 Ju n e 20 02 , U n i ve r s i ty of B ri st ol 10 http://www.ltss.bris.ac.uk/1 11 http://www.ltss.bris.ac.uk/vle02_Mark.pdf In th e bu si n e ss w or l d an an al y si s o f R e tu rn on In v es tm en t [RO I] i s al s o i mp o rt an t . 12 13 14 S e e h ttp: //w ww .l e e ds .a c. u k/ p or t ol e/ P OR T O L E P r oj ec t: Us e r N e ed s An al ysi s Pa r a 1. 1 http://www.leeds.ac.uk/portole/userneeds.doc Sti l es M , P e dag og y an d Vi rtu al L ea rn i n g En vi r on m en t (V L E) E val u ati on an d S el e cti on (2 000) , “ R et e n ti on a n d a ch i ev e m en t a r e j u st a s i mp o rt an t i n e - l ea rn i n g a s i n i ts t radi ti on al c ou n t e r pa rt. F o r l ear n i n g t o b e e f f ect i ve i t mu st b e an a c ti ve p r oc e s s wh i ch t ak e s th e f ol l owi n g i n to c on si de r ati on : 15 • L e arn i n g i s a s o ci al pr o c e s s an d d ev e l opm en t i s l i n ke d t o th e sp e ci fi c ‘ su bj e ct ’ cu l tu r e i n wh i ch ‘au th e n ti c’ (an d u su al l y c ol l abo rati v e) l ea rn i n g a cti vi ti es i n v ol vi n g sp e ci fi c kn owl e d ge , gen e r al kn o wl edg e an d s ki l l s a r e sh a r e d; • L e arn i n g a cti vi ty , r e s ou rc e s an d a s s e s sm en t n e ed t o b e cl ea rl y r el at ed t o th e s yl l abu s an d th e l ea rn e r ' s cu r r en t l e v el of kn owl e dg e .” h t t p : / / w w w . n i l t a . o r g . u k / N I L T A / R e s o u r c e s / M L E / b p 5 . p d f S e e “ W eb CT , p r ovi de r of th e w o rl d' s m o st fl e xi bl e an d wi d el y u s e d h i gh er edu c ati on e - l ea rn i n g s ol u ti on s, a n n ou n c ed t od ay th a t l ea di n g UK h i gh e r edu c ati on ( H E) an d fu rth e r e d u cati on ( F E) i n s ti tu ti on s a r e ra pi dl y ch o o si n g W e bCT Cam pu s Edi ti on a s th ei r c ampu s s ta n da rd f or e- l e a rn i n g. Th i s an n ou n c em en t n ot on l y ma r k s th e g r o w i n g de man d f or edu c ati on al i n sti tu ti on s t o stan da rdi s e t o on e Vi rtu al L ea rn i n g En vi r on m e n t (V L E) , bu t al s o h i gh l i gh ts W e bC T 's on goi n g l ead e rs h i p i n th e UK H E an d F E ma r k e t s i n del i ve ri n g t h e mo st 16 © Jo h n K on r ad 03 j o h n . k o n r a d @ d s l . p i p e x . c o m EC E R 200 3 N et wo r k 22 15 May be copied for educational use provided that the source is acknowledged. adv an ce d e du c ati on al te ch n ol o gy a vai l abl e t o su pp o rt al l te ach i n g an d l ea rn i n g st yl e s. ‘ En h an ci n g t h e qu al i ty o f edu cati on i s vi tal l y i mpo rta n t an d i s s om eth i n g w e a r e h ea vi l y c o mmi tt ed t o at th e Un i v e r si ty o f Edi n bu rgh . Hi gh st a n da rd s an d ch an gi n g stu d en t d e man d s n e ed t o be m et su cc e s s fu l l y,’ c om m en t ed J e ff Ha yw o od , di r e ct o r, mu l ti medi a an d l ea rn i n g t e ch n ol og y s e r vi c es , Th e U n i ve r si ty o f E di n bu r gh . ‘U si n g a Vi r tu al L ea r n i n g En vi r on m e n t th at c on fo r m s t o i n te r n ati on al stan da rd s h e l ps u s t o en su r e th at a c on t i n u al l evel o f h i gh q u al i ty l ear n i n g an d t e ach i n g i s e xp e ri e n c e d th r ou gh ou t th e i n sti tu t i on an d th at th e o v e ral l man ag em en t of c ou rs e s i s c a r ri ed ou t i n t h e m o st e ffi ci en t an d ef f e cti v e wa y. ’ " http://www.webct.com/service/ViewContent?contentID=17540607 W e bC T i s a vai l abl e i n Du tc h , Fi n n i sh , F r en ch , G e rm an , S pa n i sh , UK En gl i sh , Ital i an , P or tu gu e s e, Ru s si an , Sw edi sh . S e e 17 http://www.webct.com/europe#1 18 W e bC T 's Vi si on : T r an s f o rmi n g th e Edu c ati on al E xp e ri en c e http://www.webct.com/transform/viewpage?name=transform_webct_vision 19 C al i forn i a St at e Un i v e rsi ty , Sa c ra m en t o http://www.csus.edu/uccs/webct/student/character.htm Ch i c k e ri n g A W & G am so n Z F ( 198 7), “S ev en Pri n ci pl e s f o r G o o d Pr a cti c e i n Un d e rg ra du at e Edu cati on ” AA H E Bu l l etin M ar ch 20 http://www.aahebulletin.com/public/archive/sevenprinciples1987.asp Ch i c k e ri n g, A W & Eh rm an n S C (19 96 ) , " Impl e me n ti n g th e S e v en Pri n ci p l es: T e ch n ol o gy a s L e v e r, " AA H E Bu ll etin , Oc to b e r, p p. 3 6. a vai l abl e at h t t p : / / w w w . t l t g r o u p . o r g / p r o g r a m s / s e v e n . h t m l 21 22 “A ppli c ati on s an d s ys te m s in c lu d e: pe e r- t o - p e e r t e ch n o l ogy t o al l ow i mm ed i ate an d s ati s fyi n g i n ter ac ti on b etw e en s tu d en t s an d stu d e n ts , an d b et w e en e mo d er at o r s an d st u den t s k ey t e ch n ol og y i s th e d e vel op ed , en t er t ai n i n g, e f f ec ti ve In t e rn et as yn ch r on o u s an d s yn c h r on ou s g r ou p s ys t em s su p p o rti n g a wi de v a ri et y o f en vi r on m en ts fo r wo r ki n g an d l e a rn i n g tog e th er o f k ey i mp o rtan c e a r e b oth c o - a n d r e m ot el y - l o ca t ed l ea rn i n g c omm u n i ti es (cl i ck s an d m o rta r) l ear n er s co n n ec t th r ou gh b oth l o w an d h i gh ban d wi dth d e vi c e s an d s y st em s th e V L E s a c t as m e d i ati n g de vi c e s, p r om oti n g c r eati vi ty an d c ol l abo rati on . As s oc iat ed p e dag og y: l ear n er c en t r ed an d c on s t ru c ti vi st i n n at u r e k ey a cti vi ty f o r l ea r n e r s i s fi n di n g l i ke - mi n ded i n di vi du al s an y wh e r e ( e .g . by g en d e r , by i n te r e st g r ou p , by p r o f es si on ) l ear n er s b ei n g i n t el l e ctu al l y e xt en d ed b y di al ogu e an d ch al l en g e f r om oth e r s l ear n er s ex p r es s th e ms el v e s f r e el y th r o u gh sp e e ch an d t e xt © Jo h n K on r ad 03 j o h n . k o n r a d @ d s l . p i p e x . c o m EC E R 200 3 N et wo r k 22 16 May be copied for educational use provided that the source is acknowledged. th e r ol e s o f r e fl e cti on (an e ss en ti al to o l of e xp e rt l ea rn e r s) , pr o f es si on al d e vel o pm en t an d th e s h a ri n g o f t aci t kn o wl edg e ar e of c ri ti cal i mpo r tan c e as s e ss m en t i s b a s ed on c om pl ex p r obl e m s ol vi n g an d kn owl e dg e c on st ru ct i on ski l l s. Vir tu al L ea rn in g En v ir on m en t (V L E) r eq u ir e me n ts : mu st su pp o rt th e p e dag og y bu i l t -i n r e fl ec ti on t o ol s , s yn ch r on ou s c h at an d a - s yn ch r on o u s di scu s si on b oa rd s, p e e r - t o- p e er c om mu n i cati on s t o ol s th at i n cl u de a wi de v a ri e ty o f c ol l abo r ati v e t o ol s el e ct r on i c an d st ru c tu r ed i n f o rm ati on s u pp or t sti mu l ate th e l ea rn i n g g r ou p rat h er th a n r epl ac e th e a cti v e , pa rti ci pati v e l e a rn i n g e xp e ri en c e s oph i sti ca t ed d oc u m en t sh a ri n g d e vi c e s rat h er th an as s e ss m en t t o ol s an i n t er a cti v e an d p arti ci pati v e en vi r on men t mu st su pp o rt m an y bu i l t -i n kn owl e dg e man ag e m en t t o ol s a n d mu ch i mp r ov ed g r ou p w o r ki n g t o ol s . Th i s i s th e m od el t o wa rd s wh i ch I b el i ev e t e a ch e rs d el i ve ri n g on l i n e l ear n i n g a r e s tri vi n g . Th e r e i s evi d en c e of a m o v em en t t o w ar ds th i s mo d el i n Hi gh e r E du cati o n an d th e V ET s ec t or , du e i n l arg e pa rt to acti v e r es e a rc h an d pr acti c e th at i s c on f i rmi n g th e s ou n d ped ag ogi c al ba si s o f th e m od el .” D el iv e r in g L ea rn i n g on l in e: m od el s o f d e liv e r y . M od el 4 . h t t p : / / w w w . t a f e . n e t / l e a r n s c o p e / d l o / l e a r n i n g 8 . h t m l Zi mm e r B et al , (2 000) , “B u i l di n g an O n li n e L ea rn i n g C om mu n i ty” On l i n e Tu t o ri n g e - b o ok # 3 h t t p : / / o t i s . s c o t c i t . a c . u k / o n l i n e b o o k / o t i s - t 3 . h t m 23 Val i th i an P (20 02) , “ Bl en d ed L ea rn i n g Mo d el s” , L e a rn in g C ir cu it s , ASTD ’ s on lin e mag a z in e , Au gu st , h t t p : / / w w w . l e a r n i n g c i r c u i t s . c o m / 2 0 0 2 / a u g 2 0 0 2 / v a l i a t h a n . h t m l F o r ev a l u ati on s o f th i s a pp r oa ch s e e B on k C ( et al ) (2 00 2), “ L ea rn i n g F r om Fo cu s G r ou p s: An Ex ami n a ti on o f Bl en d ed L ea r n i n g”, J ou rn al of di s tan c e edu ca ti on / R evu e d e l ’ édu cati on à di st a n c e, 2 002 , v ol . 17 , n o 3 , tl •n c e s p eci al edi ti o n /édi ti on s p éci al e rc e •ta , 97 - 118 ( ab st r act s i n En gl i sh an d F r en ch ) h t t p : / / w w w . p u b l i c a t i o n s h a r e . c o m / d o c s / b l e n d . p d f ; “ T e n 24 key Web-based instruction considerations or issues mentioned across participants related to feedback; meaningfulness of content; content size; course development and organization; the role of the on -line instructor; structuring small groups; flexible and active learning; use of technology; assessment practices; and general skills such as on-line communication, problem -solving, and teamwork. Participants offered many relevant recommendations for fine -tuning this program as well as building similar programs.” « Les dix considérations -clés de la formation en ligne mentionnés par les participants touchent la rétroaction, la cohérence du contenu, la taille du contenu, le développement du cours et l’organisation, le rôle du formateur en ligne, l’organisation des pe tits groupes, l’apprentissage souple et dynamique, l’utilisation de la technologie, les pratiques d’évaluation et les habiletés générales telles que la communication en ligne, la résolution de problèmes et le travail en équipe. Les participants ont fait pl usieurs recommandations pertinentes pour améliorer ce programme ainsi que pour construire des programmes similaires. » One of the benefits of the recent shakeout in ICT is that most people are now t a l k i n g o f b l e n d e d l e a rn i n g – q u i t e s i m p l y : u s i n g t h e m e d i a t h a t a r e a v a i l a b l e , and no longer trying to squeeze everything through a not -very-broad-band Internet, for instance. So, Web CD’s are OK, paper has its role, face to face training and learning is valuable, and the trick is to get the “blend” right. And © Jo h n K on r ad 03 j o h n . k o n r a d @ d s l . p i p e x . c o m EC E R 200 3 N et wo r k 22 17 May be copied for educational use provided that the source is acknowledged. there is still plenty that digitalisation will bring – in mobile/wireless technologies and broadband particularly. All of this is most welcome, and very healthy for the IT sector as well as for learning. Research in e-learning is now starting to provide a systematic critique of what might be called the first phase in the development of e -learning. Up to now, much of the activity has been to get it up and running, to establish the three or four VLE platforms, at least one Open Source VLE – Bodington Common – (http://bodington.org/index.html ) and to deliver the goods. That has been done. The second phase will be to develop the next generation of platforms, and provide more user-friendly environments for learning, as opposed to just ensuring the delivery of courses.” Editorial Electronic Journal of e -Learning, Volume 1 Issue 1 (2003) I-ii, http://www.ejel.org/issue -1/issue1 -editorial.pdf “A stu dy b y Pet e r D ean a n d h i s c ol l eag u e s f ou n d th at p r ovi di n g s e veral l i n ked o pti o n s f o r l ea rn e r s, i n a ddi ti on t o cl a ss r o om trai n i n g, i n cr ea s ed w h at th e y l ea rn ed . In 20 02 , H ar va rd Bu si n e s s S ch o ol fa cu l ty D eL ac e y an d L e on a rd r e p or t ed t h at stu d en t s n ot on l y l e a rn ed mo r e wh en on l i n e s e ssi on s w er e a dd ed to tradi ti on al co u r s e s, bu t stu d en t i n t era cti on an d sa ti sf ac ti on i mp r oved a s w el l . Th om s on an d N E Tg r el ea s ed a 2 0 0 3 w h i t e p a p e r th at r ep o rt ed sp e edi e r p e r fo rm an ce on r eal w o rl d t as k s by p eo pl e wh o l ea rn ed th r ou gh a bl e n d e d strat eg y—f a st e r t h a n th o s e stu d yi n g t h r ou gh e - l e a rn i n g al o n e .” Ro s s ett A ( et al ) (2 003) , “ St rat e gi e s f o r Bu i l di n g Bl en d ed L ea rn i n g ”, L ea rn in g Ci rc u it s , A STD ’ s on l in e mag az i n e , Ju n e. http://www.learningcircuits.org/2003/jul2003/rossett.htm C o rn el i u s S & Hi ggi s on C (20 00) , On li n e Tu t or in g e - B o ok: 2 - Th e Tu t o r' s R ol e , H e ri o t - Wat t Un i v er si ty an d Th e Ro b e rt G o rd o n Un i ve r si ty , h t t p : / / o t i s . s c o t c i t . a c . u k / o n l i n e b o o k / o t i s T 2 0 2 . h t m 25 26 See http://www.staffs.ac.uk/COSE/ Sti l es M & Y o r k e J ( 2003) , “ D e si gn i n g a n d Impl e m en ti n g L e arn i n g T ech n ol og y P r oj ect s – A Pl an n ed Ap pr o a ch ” , EFF E CT S/ Em b e ddi n g L ea rn i n g T e ch n ol ogi e s S emi n a r – 8 t h A p r il 2003 27 http://www.staffs.ac.uk/COSE/cosenew/eltfinal.doc Sti l es M J (2 000 ), “ Ef f ec t i v e Le ar n in g a n d t he V irt u al Le ar n in g E n vi ro nm en t” , Th i s p a pe r w as o ri gi n all y d el i ve r ed a s a K eyn ot e at th e 2000 Eu r op ea n U n i ve r si ti es In f o rmati on S ys te m s C on g r e ss - EU N IS 2000 - " T owa r ds Vi rtu al Un i v e rsi ti e s" i n P oz n an , P ol an d on Ap ri l 2000 . It h a s b e en pu bl i sh ed i n P r o c e e din gs : E UN IS 20 00 - To wa rd s Vir tu al Un i v e r siti e s , In st ytu t In f o rma ty k i Pol i te ch n i ki P oz n a n ski ej , P oz n an Ap ri l , ISB N 83 913 639 1 0 . al s o f r om 28 http://www.staffs.ac.uk/COSE/cose10/posnan.html Sti l es M J (2 003 ), “ Str at egi c an d P eda g ogi c R equ i r e m en t s f o r Vi rtu al L ea rn i n g i n t h e C on t e xt of Wi den i n g Pa rti ci pati on ” 29 http://www.inter-disciplinary.net/Stiles%20Paper.pdf K i rk pat ri c k ( 19 96) , “ G r ea t i d ea s r e vi si ted: r e vi si ti n g K i rkp atri c k ’ s f ou r- l e v el m od el ”, T rai n in g an d D e v el op men t, 5 0, 1 , Jan u ar y, pp . 19- 25 30 31 See ASTD (2000), “There are many reasons for evaluating any type of learning solution, and several of them apply specifically to e-learning. Many individuals are interested in information about e-learning programs. The developers, coordinators, and even the participants are often interested in receiving data on these programs. Managers of participants and top management often want specific types of information. Finally, those who support and advocate the use of technology need certain types of data about e-learning.” www.astd.org/virtual_community/research/What_Works/e-learning/reason_1.html © Jo h n K on r ad 03 j o h n . k o n r a d @ d s l . p i p e x . c o m EC E R 200 3 N et wo r k 22 18 May be copied for educational use provided that the source is acknowledged. S e e f o r a fu rth e r di s cu s si on , B eh ri n ge r F & C ol e s M (20 02) , Th e Rol e o f Nati on al Qu al i fi cati on s Sy st e m s i n P r om oti n g Li f el o n g L ea rn i n g - Li n ks b et w ee n Qu al i fi cati on s Sy st em s an d Li f el on g L ea rn i n g: A C on c ept u al Fr am e w o rk f o r F u tu r e Wo r k , O E CD P ari s . 32 http://www.oecd.org/home/ © Jo h n K on r ad 03 j o h n . k o n r a d @ d s l . p i p e x . c o m EC E R 200 3 N et wo r k 22 19