Do WWW-based Presentations Support Better (Constructivistic) Learning in the Basics of Informatics? Pekka Makkonen University of Jyväskylä Department of Computer Science and Information Systems P.O. Box 35, Fin-40350 Jyväskylä, Finland E-mail pmakkone@jyu.fi Abstract Hypertext and the WWW appear to have a positive effect on learning. Both techniques permit a constructivist learning environment to support the student's knowledge construction and permit a more active student role. However, the problems associated with the aforementioned may endanger the benefit to learning. Thus, the role and the forms of hypertext and the WWW must be discussed. Due to the common problems of "information overload" and being "lost-in hyperspace" we suggest guided tours in the form of a slideshow presentation as a solution to organizing a hypermedia presentation on the WWW for learning the basics of informatics. These slides include links to the appropriate supporting web sites. In addition to the idea of a guided tour, based on the cognitive flexibility theory, we also suggest that the use of search engines and directories must be promoted. This paper describes the use of guided tours as a complementary addition to conventional lectures in the learning of the basics of informatics. We claim that learning can be promoted in the spirit of constructivism, situated action and cognitive flexibility when organizing coursework based on guided tours as well as search engines and directories. We analyze the benefit of optional coursework including the use of guided tours and in addition the use of search engines and directories on the WWW. At the beginning and end of the course we analyzed how useful and interesting the students regarded six themes of the course. In addition, the students were expected to analyze their own development of computer use skills, Internet use skills, and their knowledge of basic concepts in informatics. The study found that WWW-based coursework affects both external and internal motivation equally in most cases. The effect on Internet skills and especially on the knowledge of basic concepts in informatics was beneficial. 1. Introduction During the 90s the reform of teaching has also reached the universities. For example, Isaacs [11] and Rosenthal [25] have reported various problems in regard to traditional lecture-based teaching. These problems include ineffectiveness, an increase in passivity, and isolation in students. Especially in the context of technology and related sciences, some revisions have been suggested to improve lecturing as a teaching method by activating students using, for example, co-operative learning in small groups and essay-writing assignments about technical topics [11]. From this perspective lecturing is not without potential if the previously mentioned problems can be corrected, but other learning methods must also be considered. For example, in informatics coursework utilizing information technology and its new possibilities may be a good and natural alternative to conventional lectures. Hypertext reflects a human's way of thinking and provides an opportunity to process information and the interrelatedness between information cues [14]. To learn meaningfully, individuals must choose to relate new knowledge to relevant concepts and propositions with which they are already familiar [22]. Hypertext enables learning as a knowledge construction process, because it allows information and concepts to be represented as a user or a learner has adopted them. This helps meaningful learning and may improve the motivation for learning. Thus, it is important that we comprehend hypertext as a cognitive tool, in other words, as a tool for knowledge construction. Hypertext and hypermedia permit an alternative view of learning by supporting knowledge construction and individual learning. Thus, they may be alternatives to conventional lectures or textbooks in many ways. Hypermedia enables the possibility to store, interconnect, and provide access to a wide range of knowledge presented in different ways [12]. Thus, hypertext and hypermedia provide an ideal environment for learning based on a constructivist approach. One alternative to conventional lectures is a hypermedia presentation on the World Wide Web (WWW), because vast information resources are available to teachers and students via the WWW. A presentation on the WWW can be supported by coursework emphasizing learning as knowledge construction. Jonassen [13] claims that cognitive tools actively engage learners in the creation of knowledge that reflects their comprehension and conception of the information rather than focusing on the presentation of objective knowledge. These tools are learner controlled, not teacher or technology driven. The use of a cognitive tool changes the role of the student into that of an active learner. Figure 1 shows cognitive tools in the general framework for computer-based learning, which includes three dimensions [13]. These dimensions are generativity, control, and engagement. Active Cognitive Tools Engagement Teacher/System Control Passive Presentation Creation Student the WWW we suggest a solution consisting of three layers. These layers are (a) the support of guided tours as a slideshow on the WWW, (b) the support of appropriate links, and (c) the support of search engines and directories. This approach may provide a real basis for successful WWW-based coursework. This paper has two purposes. First, it introduces our WWW-based coursework as a way to apply the WWW in the learning of basic concepts in informatics. Second, it includes an evaluation of how dedicated the students who completed the optional WWW-based coursework were while learning different areas of informatics. Additionally, the paper contains an evaluation of how three different main areas, which are (1) skills in using computers, (2) skills in using the Internet, and (3) knowing the basic concepts, were developed during the course. We made all the evaluations by comparing the students who completed the coursework to the students who did not participate in the coursework. The paper is organized as follows: in section two we describe the meaning of engagement and motivation in learning. Section three describes constructivism as an alternative view of learning. In section four we discuss the opportunities of the WWW in learning. The fifth section presents our research objectives based on the theoretical discussion conducted earlier in this paper. The sixth section presents our web-based course and evaluation of the same from the perspective of engagement and motivation as well as skills and learning basic concepts. In section seven we summarize the results of the study and draw conclusions. Generativity Figure 1. Cognitive tools in the general framework for computer-based learning. However, hypertext and hypermedia have their own problems. They do not typically offer an explicit mechanism to help learners better interpret and assimilate information, the context surrounding its creation and use, or the perspectives on the information of the author or other learners [29]. Simply improving information access without supporting learning leads directly to the problems of "information overload" and being "lost-in-hyperspace". Thus, students need some degree of guidance. Additionally, the form and structure of hypermedia presentations must be discussed. This is more important in the era of the WWW. The problems inherent in any information system such as disorientation, navigation inefficiency and cognitive overload are multiplied on the Internet [5]. The WWW provides both the possibility to organize information in a strict form (e. g. using trails and guided tours) and also opportunities for free "surfing" with its advantages and disadvantages. To realize the benefits of 2. Engagement and motivation The general framework for computer-based learning presented in section one shows the importance of evaluating motivation. Jonassen [13] claims that by using cognitive tools in learning, a student's engagement in learning is better. Thus, it is important to evaluate motivation to show the depth of engagement in this way. From the perspective of our research context many problems in regard to learning the basics of informatics make evaluating motivation important. Romano and Balthazard [24] have summarized the problems. These include limited individualized attention, cyberphobia, in other words a fear of computers, a lack of interest in the subject matter, limited or no access to interactive learning software, and information overload leading to failure to make associations. Most commonly in learning from text, motivation is understood both internally and externally [3, 4, 9, 17]. Internal motivation (or intrinsic motivation) reflects a student's own interest in regard to espousing new knowledge. It is associated with a human's high-level needs such as self-actualization. External motivation (or extrinsic motivation) reflects the need to reach goals set by others. This is connected to a human's low-level needs such as security and survival. Motivation in learning from text can be evaluated as shown in figure 2. Pre-motivation is the sum of preinterest and pre-benefit. Post-motivation is the sum of post-interest and post-benefit. Internal motivation is the sum of pre-interest and post-interest. External motivation is the sum of pre-benefit and post-benefit. Pre-interest Pre-motivation Internal motivation Pre-benefit Post-interest Post-motivation External motivation Post-benefit Figure 2. Motivation of learning from text. 3. Constructivism as an alternative view of learning In this section we introduce constructivism as an alternative view of learning especially from the perspective of the use of the WWW. Widely known and discussed views associated with (computer-supported) learning include behaviorism and its opposite constructivism. Behaviorism is interested in a student's behavior (reactions) in relation to teaching (stimulus), while constructivism is interested in the mental processes which affect the behavior of a student [23]. A traditional lecture is mainly based on the behaviorist approach while coursework and projects are typical constructivist learning [27]. Jonassen [15] summarizes what he refers to as "the implications of constructivism for instructional design". The following principles illustrate how knowledge construction can be facilitated by: providing multiple representations of reality, representing the natural complexity of the real world, focusing on knowledge construction, not reproduction, presenting authentic tasks (contextualizing rather than abstracting instruction), providing real-world, case-based learning environments, rather than pre-determined instructional sequences, fostering reflective practice, enabling context-and content-dependent knowledge construction, and supporting collaborative construction of knowledge through social negotiation. According to Brandt [6], constructivism asserts that learners construct knowledge by making sense of experiences in terms of what is already known. In constructivist learning the concept of a mental model is essential. Learning is comprehended as the development of a learner's mental models (or declarative and structural knowledge). Brandt [6] emphasizes that constructivism is an essential basis when applying the WWW for teaching and learning. While the goal of constructivism is to recognize and help to facilitate a learner's ability to construct knowledge when applied to teaching information retrieval on the Internet, it also provides the teacher with a structure for teaching. By focusing on concepts and connecting them to mental models, instructors and teachers can gain both confidence and control over the amount of material they cover in the small blocks of time usually allotted to teaching and training. Integrated with experiences that learners use to alter and strengthen mental models, the constructivist approach to teaching information retrieval also gives users the structure needed to get the most out of the Internet. Despite the promise of constructivism several researchers emphasize the meaning of guidance. For example, Silverman [26] points out that by providing the right amount of traditional instruction, students seem to favor constructivist environments. Additionally, he suggests different tools (e.g. a multimedia authoring environment, better communication media, and easily integrated microworld simulators) to support lessons based on the constructivist approach. 4. The WWW in learning Vast information resources are available to teachers and students via the WWW. However, the problems inherent in any information system such as disorientation, navigation inefficiency and cognitive overload are multiplied on the Internet [6]. On the other hand, these problems can be overcome using a suitable pedagogical approach and/or appropriate tools. Constructivism, which is mentioned in section three, is the basic approach for Internet solutions in learning. Additionally, it is useful to discuss the use of the WWW from other perspectives. In this section we introduce trails and guided tours as a way of improving the usefulness of the WWW. In addition, we introduce the concepts of situated action and cognitive flexibility, since these concepts are useful while applying the WWW for learning. One way to organize a presentation on the WWW is to provide trails and guided tours. Trails connect a chain of links through information spaces [2]. These can include multiple "recommended" trails through a network. Guided tours restrict users to the trail, prohibiting detours. While trails lower the cognitive overhead (or overload) by recommending the next logical link to take, guided tours reduce the overhead further by removing all other choices. Thus, they can also prevent the phenomenon of ‘lost-inhyperspace’. Chen and Rada [8] stress the term "situated action", which emphasizes the interrelationship between an action and its context of performance. Additionally, according to Bruckman et al. [7] and Koschmann [16], the success of a computer-supported learning environment depends not just on the software, but on the context in which that software is used. Situated action theory emphasizes a person's responsiveness to the environment and focuses on the improvisory nature of human activity [21] and the local management of activity as mediated by relevant environmental cues [1, 28]. The implications for learning are that appropriate actions are generated from recognition of appropriate opportunities given by the context. In addition to situated action theory, Jacobson et al. [12] also emphasize the meaning of cognitive flexibility theory affecting hypertext-based learning. This theory was initially formulated to address factors contributing to failures to learn complex knowledge at advanced instructional levels. It proposes that complex knowledge may be better learned for flexible application in new contexts by employing case-based learning environments that include features such as: (a) the use of multiple knowledge representations, (b) linking abstract concepts in cases to depict knowledge-in-use, (c) demonstrating the conceptual interconnectedness or web-like nature of complex knowledge, (d) emphasizing knowledge assembly rather than reproductive memory, (e) introducing both conceptual complexity and domain complexity early, and (f) promoting active student learning. These features have been used prescriptively to specify design elements for complex, multidimensional, and nonlinear environments such as hypertext and hypermedia. Based on the above it is important to comprehend views of learning while outlining courses as well as comprehending ways of using the WWW in learning. We stress three issues. First, we must discuss what the right amount of traditional (behaviorist) teaching is. Second, we must analyze what the right way to use the WWW is. Active learning must be promoted and the pitfalls of the WWW must be avoided. Third, scaffolding support is needed to support constructivist learning based on the WWW. 5. Research objectives of this paper Because constructivism is the leading approach to learning of our time, we evaluate learning based on this approach. Our main focus is to show how successful the learning process has been. We conducted this evaluation principally on the basis of two views: 1) the meaningfulness of learning, 2) understanding of the basic concepts. Secondarily, we are interested in use skills, because these indirectly show how the concepts are learned. 6. A solution for WWW-based learning: coursework and its evaluation We pursued a study concerning WWW coursework based on the idea of trails and guided tours supported by the use of search engines and directories. In this section we describe our experiment, sample, measures and tests, followed by the results of our study. 6.1. Experiment At the University of Jyväskylä the themes of an introductory course in automatic data processing are (1) introduction (including such themes as the meaning of automatic data processing, the information society and problems in utilizing computers), (2) presentation of data in the PC environment, (3) computer software, (4) hardware technology, (5) data communications, and (6) the meaning of information technology in business. The course usually lasts 10 weeks including lectures (8 hours), compulsory practical exercises in basic skills with personal computers and the Internet (18 hours) as well as a final examination. The course given in spring 1998 also lasted for this length of time and included the abovementioned activities and in addition material and activities on the WWW to support the lectures in the constructivist fashion. Our approach to using the WWW for teaching and learning on this course was to combine the idea of trails and guided tours on the WWW, both behaviorist teaching/learning and constructivist learning, situated action, and cognitive flexibility theory. In the WWW-based learning the basic point is situated action. The constructivist approach is the commonly accepted principle for learning. Since the structural form of knowledge (or knowledge structures) is typical for the basic concepts of informatics [20], it is also natural to approach the learning and teaching of it from the perspective of constructivism. However, in our context, in the basics of informatics, students may need guidance at the beginning of learning. Thus, the traditional learning methods in the behaviorist manner must be accepted as a part of teaching and learning methods. The students were introduced to our approach and the pre-questionnaire administered at the beginning of the course. The results of this questionnaire [18] support our approach, since over 58% of the students considered both the lectures and printed lecture notes and the material on the WWW significant or strongly significant (based on a 5-point Likert scale ranging from strongly insignificant to strongly significant). Additionally, less than 9% of the students considered both the lectures and printed lecture notes and the material on the WWW insignificant and none of them strongly insignificant. We organized our basic course lectures on the basis of the above-mentioned approach. Thus, the lectures consisted of printed lecture notes, which all students obtained at the beginning of the course, conventional lectures (8 hours), lecture notes on the WWW including links to supporting sites, and optional coursework using the WWW in regard to lecture notes on the WWW. Due to our context, we claim that conventional lectures and printed lecture notes are needed as a behaviorist part of the course, but lecture notes on the WWW provide an opportunity for a constructivist approach. The lecture notes on the WWW were organized in the form of a slideshow using Microsoft Powerpoint 97 and its Internet assistant. These tools permit the slides in each lecture to be organized in the strict form of a guided tour on the WWW providing support for a student who is at the beginning of learning in informatics and not familiar with computers and the WWW. Each slide in the slideshow may include a set of links to interesting WWW sites and in this manner a slideshow can also be comprehended as a trail. Our slides included links concerning the critical concepts to the appropriate link pages, which were evaluated as supporting the learning of these concepts best. The selection of the links on the slides was based on the list of critical concepts for learning produced by a group of teachers (n=12) of informatics at our university. Since the slides are on the WWW, a student can also support his or her learning using search engines and directories. Thus, the form of a lecture is flexible and it can be seen as a trail or a guided tour depending on the situation and information needs. This allows different views and brings a real constructivist way of learning, since the role of instructional media shifts from one which seeks to maximize the communication of fixed content and/or skills to one in which students engage in the knowledge construction process: constructing interpretations, appreciating multiple perspectives, developing and defending their own positions while recognizing other views, and becoming aware of and able to manipulate the knowledge construction process itself. In this sense our approach is also consistent with cognitive flexibility theory. To realize the benefit of the lecture notes on the WWW we organized coursework in which students were expected to enter their findings in their personal diaries. These findings included their opinion about (a) the general form of presentation, (b) the links provided by the teacher, and (c) the links found by the students themselves using search engines and directories. Additionally, the students were expected to give various examples of what they had learned during the coursework. To promote the students' participation in the optional coursework, the students got credits by completing the coursework for the final examination. Although the coursework is a constructivistic part of the course, the teacher's office hours were available as an additional resource to promote their work. The students had six and a half weeks for the coursework before the final examination. The work was expected to be conducted as an individual task or in groups of two or three students. 6.2. Sample One hundred and three students, 78 females and 26 males, whose mean age was 23 years (range 19-42 years), entered the course and completed both pre- and posttreatments. All the students were familiar with university lecturing. They familiarized themselves with the use of a WWW browser and basic search engines and directories (i. e. Altavista and Yahoo) before the introduction of the optional coursework. All the students studied informatics as a minor. During the course they were required to participate in practicals concerning the use of personal computers and the Internet (18 hours) and all the students had the same exercises concerning these matters. Participating in the lectures (8 hours) and the coursework was optional. Forty-six of the students, 38 females and 8 males, whose mean age was 22 years (range 19-40 years), participated in the optional coursework. 17 of them completed the coursework individually, 23 in groups of two students and 6 in groups of three students. The students spent 13 hours (range 5-30 hours) on the coursework on average. They attended 6.5 hours of lectures and spent 43 hours on the whole course on average. We call this group the WWW group in this paper. Fifty-seven students, 39 females and 18 males, whose mean age was 23 years (range 19-42 years) did not complete the optional coursework. These students attended 4.9 hours of lectures and spent 31 hours on the whole course on average. We call this group the nonWWW group in this paper. 6.3. Collecting data The data for this study was collected by administering a questionnaire both at the beginning and end of the course to both types of groups. The respondents rated each theme of the course with regard to (a) how interesting they considered the themes of the course (where 1=very uninteresting and 5=interesting), and (b) how beneficial they considered the themes of the course (where 1=very useless and 5=very useful). Additionally, in the questionnaires both at the beginning and the end of the course the respondents were expected to analyze their: computer use skills, Internet use skills, and knowledge of the basic concepts in informatics. The respondents ranked these skills or knowledge on a 5 point Likert scale (where 1=very poor and 5=very good). 6.4. Results The Kolmogorov test showed that the data based on the responses of the students concerning the themes of the course agreed with the normal distribution. Thus, the oneway ANOVA test was appropriate for statistical analysis of the data. However, the Kolmogorov test showed that the data based on the students' rankings concerning their skills and knowledge disagreed with the normal distribution. Thus, the Mann-Whitney test as a nonparametric test was appropriate for statistical analysis of the data. We calculated the scores for pre-motivation, postmotivation, internal and external motivation of each theme. This was based on the framework presented in section two. The one-way ANOVA test did not show significant differences in pre-motivation between the NonWWW group and the WWW group in regard to any theme of our course (p varying between .061 and .513). Additionally, the one-way ANOVA test did not show significant differences in post-motivation between the Non-WWW group and the WWW group in regard to any theme of our course (p varying between .094 and .974). Based on the one-way ANOVA test the study found that the WWW-based coursework affected both external and internal motivation equally in most cases (in these cases p varying between .131 and .973 in regard to internal motivation and p varying between .043 and .753 in regard to external motivation). External motivation was significantly higher in the group of students without the optional coursework in learning the basic concepts of information technology in business (the mean of the NonWWW group was 8.54, that of the WWW group 7.91, p=.043). Concerning skills and knowledge, and based on the Mann-Whitney test, the study found that the optional WWW-based coursework was useful in the learning of Internet skills and the basic concepts of informatics. However, the coursework did not improve general computer use skills significantly. Statistical analysis showed that the coursework was most beneficial in the learning of the basic concepts. The difference between the groups was extremely significant at the beginning and at the end the difference was highly equivalent. The results show that WWW-based coursework can help students to learn the basic concepts of informatics. If a student is a real beginner (a novice) concerning both skills in using a computer and knowledge of basic concepts WWW-based coursework can be especially useful. The details of the analysis concerning skills and knowledge are shown in table 1 (see next page). For this analysis the students were expected to evaluate their skills and knowledge based on a 5-point Likert scale in the questionnaires. The non-WWW group refers to the students who did not complete the WWW-based coursework and the WWW group to the students who completed the optional WWW-based coursework. Table 1. Analyzing the students' skills and knowledge. Mean at the beginning Mean at the end of of the course the course NonNonWWW WWW WWW WWW group group p group group p Computer use 2.66 2.17 .004 3.38 3.00 .003 skill Internet use 2.54 2.11 .039 3.47 3.57 .553 skill Knowledge of 2.14 1.65 .003 2.71 2.74 .888 the basic concepts in informatics 7. Discussion The study found that the main effects of WWW-based coursework are skills and knowledge rather than the meaningfulness of learning (or motivation). Thus, considering the theoretical discussion the results presented in this paper are contradictory. In the sense of the cube of computer-supported learning presented in the introduction to this paper, the WWW coursework does not support constructivistic learning. However, the positive result concerning the learning of the basic concepts shows that our coursework supports constructivistic learning in the sense of the development of students' mental models. Additionally, the results of our previous papers [19, 20] confirm the results concerning the learning of basic concepts. Especially in our most recent paper [20] we showed that WWW-based coursework improves learning outcomes concerning the learning of the basic concepts. In the analysis section of that paper we compared the students who had completed the optional coursework to the students who had not participated in the optional coursework. For the evaluation we utilized a SOLO (Structure of Observed Learning Outcome) taxonomybased measure to clarify learning outcomes and their quality, because this reflects the quality and development of a student's knowledge structures [5]. Additionally, in the analysis section of this paper we found that WWWbased coursework was useful for Internet skills, which is however a very natural effect. The results contained in this paper leave open questions. First, what are the reasons for the success of the WWW-based coursework concerning the development of conceptual knowledge? Second, why does WWW-based coursework not improve the meaningfulness of learning? To both questions answers will be found by analyzing the coursework reports and questionnaires in detail. Our preliminary analysis of the students' coursework [19] also supports the results outlined in this paper and partly provides an answer to the first question presented in this paragraph. We found that especially the slideshow format, links and appropriate link pages are beneficial to learning. Search engines and directories are less effective in learning than guided tours in the form of slideshows or links and link pages (destinations) [19]. However, we claim that search engines and directories may be more effective subsequent to the basic course in informatics when students are more familiar with these tools. Based on this preliminary analysis of the coursework we can claim that in our context, the basics of informatics, the least apt part of the coursework was search engines and directories and it is worth studying why. In the same way it is worth studying why our slideshow format and links on slides were more successful in promoting learning. In the coursework students were required to give reasons for their ratings and analyzing this information will clarify what the reasons for success or failure were. Nevertheless, the results contained in this paper and in our previous papers [18, 19, 20] show that based on many reasons it is useful to combine behavioristic teaching and constructivistic learning utilizing the WWW in our context. The WWW-based coursework, designed in a constructivistic manner, especially helped the students who were real beginners (novices) in the field of informatics. In this way everyone had the opportunity to learn. This study was conducted in Finland, where the rate of Internet use is one of the highest in the world [10]. Finns are very technology-oriented people and the penetration rates for many other technological solutions, such as cellular phones, are at a very high level compared to many other countries. Thus, the coursework presented in this paper is a good solution for web-based teaching and learning in countries highly advanced in IT such as the other countries of the European Union and in the USA. 8. Author note Special thanks are reserved for Dr. Seppo Puuronen, M. A. Armi Rintala, M. A. Markku J. Suihkonen, and M. A. Tony Melville from Jyväskylä in Finland for their ideas and assistance in the preparation of this paper. Additionally, the author is grateful to Dr. Urban Nuldén and M. Sc. (Econ.) Christian Hardless from Gothenburg in Sweden for their helpful comments and assistance. 9. References [1] Agre, P. E., Chapman, D. (1987). Pengi: An Implementation of a Theory of Activity. Proceedings of AAAI-87, Los Altos, CA, pp. 196-201. [2] Bieber, M., Vitali, F., Ashman, H., Balasubramanian, V., Oinas-Kukkonen, H. (1997). Some Hypermedia Ideas for the WWW. Proceedings of the 30th HICSS, Hawaii International Conference of Systems Science (Jan. 1997), Vol. 4, IEEE Computer Society Press, pp. 309-319. [3] Biggs, J. B. (1984) Learning Strategies, Student Motivation Patterns and Subjectively Perceived Success. In Kirby, J. R. (Ed.), Cognitive Strategies and Educational Performance. Orlando, Florida: Academic Press, pp. 111-134. [4] Biggs, J. B. (1985). The Role of Metalearning in Study Processes. British Journal of Educational Psychology, 55, pp. 185-212. [5] Biggs, J. B., Collis, K. F. (1982). Evaluating the Quality of Learning. The SOLO Taxonomy (Structure of the Observed Learning Outcome). New York: Academic Press. [6] Brandt, D. A. (1997). Constructivism: Teaching for Understanding of the Internet. Communications of ACM, Vol. 40, No. 10, pp. 112-117. [7] Bruckman, A., De Bonte, A. (1997). MOOSE Goes to School: A Comparison of Three Classrooms Using a CSCL Environment. Proceedings of CSCL 97. Toronto, Canada, December 10-14, 1997, http://www.oise.utoronto.ca/cscl/. [8] Chen, C., Rada, R. (1996). Modelling Situated Actions in Collaborative Hypertext Databases. Journal of Computer Mediated-Communication, Vol. 2, No.3, http://www.ascusc.org/jcmc/vol2/issue3/chen.html. [9] Entwistle, N. J., Ramsden P. (1983) Understanding Student Learning. London: Croom Helm. [10] European Commission (1997). Evolution of The Internet and the WWW in Europe. [11] Isaacs, G. (1994). Lecturing Practices and Note-Taking Purposes. Studies in Higher Education, Vol. 19, No. 2, 1994, pp. 203-216. [12] Jacobson, M. J., Maouri, C. , Mishra, P., Kolar, C. (1996). Learning with Hypertext Learning Environments: Theory, Design, and Research. Journal of Educational Multimedia and Hypermedia, Vol. 5, No. 3/4, pp. 239-281. [13] Jonassen, D. H. (1992). What are Cognitive Tools? In Kommers, P. A. M., Jonassen, D. H., Mayes, J. T. (Eds.), Cognitive Tools for Learning (pp. 1-6), Berlin: Springer-Verlag (NATO ASI Series). [14] Jonassen, D. H. (1993). Effects of Semantically Structured Hypertext Knowledge Bases on Users' Knowledge Structures. In McKnight, C., Dillon, A., Richardson, J. (Eds.), Hypertext. A Psychological Perspective (pp. 153-166), New York: Ellis Horwood. [15] Jonassen, D. H. (1994). Thinking technology. Educational Technology, Vol. 34, No. 4, pp. 34-37. [16] Koschmann, T. (1996). Paradigm Shifts and Instructional Technology: An Introduction. In Koschmann T. (Ed.) CSCL: Theory and Practice of an Emerging Paradigm (pp. 1-23), Mahwah, NJ: Lawrence Erlbaum. [17] Linnakylä, P. (1988). Miten opitaan tekstistä? Ammattiopiskelijoiden tekstistä oppimisen arvioimisen taustaa.University of Jyväskylä: Institute for Educational Research. Research report 17. In Finnish. [18] Makkonen, P. (1998a). The WWW-based Presentations as a Complementary Part of Conventional Lectures in the Basics of Informatics. In proceedings of the ACM Conference on Integrating Technology into Computer Science Education (18th- 21st August 1998): Dublin City University and the ACM, pp. 162-165. [19] Makkonen, P. (1998b). Benefit of the WWW-based Presentations as a Complementary Part of Conventional Lectures in the Basics of Informatics. In proceedings of the 21st IRIS Information Systems Research Seminar in Scandinavia (8th- 11th August 1998), Vol. 2: University of Aalborg and the IRIS Association, pp. 583-591. [20] Makkonen, P. (1999). The WWW-based Presentations as a Complementary Part of Conventional Lectures in the Basics of Informatics: Is It Worth it? In proceedings of the IRMA (Information Resources Management Association) 1999 Conference. Hershey, PA: Information Resources Management Association, pp. 365-371. [21] Nardi, B. (1996). Studying Context: A Comparison of Activity Theory, Situated Action Models, and Distributed Cognition. In Nardi, B. (Ed.), Context and Consciousness (pp. 69-102), Cambridge, MA: MIT Press. [22] Novak, J. D., Gowin, D. G. (1984). Learning How to Learn. Cambridge: Cambridge University Press. [23] Risku, P. (1996). A Computer-Based Mathematics Learning Environment in Engineering Education. Jyväskylä: University of Jyväskylä, Department of Mathematics, Report 71. [24] Romano, N. C. Jr., Balthazard P. A. (1993). Computer Literacy Instruction Using Nonlinear Environments. Journal of Education for MIS, 1(1), pp. 11-16. [25] Rosenthal J. (1995). Active Learning Strategies in Advanced Mathematics Classes. Studies in Higher Education, Vol. 19, No. 2, 1994, pp. 223-228. [26] Silverman, B. G. (1995) Computer Supported Collaborative Learning (CSCL). Computers in Education, Vol. 25, No. 3, pp. 81-91. [27] Solomon, C. (1986). Computer Environments for Children Learning: a Reflection on Theories of Learning and Education. Cambridge, MA: MIT Press. [28] Suchman, L. (1987). Plans and Situated Action: The Problem of Human-Machine Communication. Cambridge, UK: Cambridge University Press. [29] Wan, D., Johnson, P. M. (1994). Computer Supported Collaborative Learning Using CLARE: The Approach and Experimental Findings. In Proceedings of the ACM Conference of Computer-Supported Cooperative Work, October, 1994, pp. 187-198.