DEVELOPMENT AND EVALUATION OF A WEB-BASED LEARNING SYSTEM
BASED ON LEARNING OBJECT DESIGN AND GENERATIVE LEARNING TO
IMPROVE HIGHER-ORDER THINKING SKILLS AND LEARNING
TAN WEE CHUEN
UNIVERSITI TEKNOLOGI MALAYSIA
DEVELOPMENT AND EVALUATION OF A WEB-BASED LEARNING SYSTEM
BASED ON LEARNING OBJECT DESIGN AND GENERATIVE LEARNING TO
IMPROVE HIGHER-ORDER THINKING SKILLS AND LEARNING
TAN WEE CHUEN
A thesis submitted in fulfilment of the
requirements for the award of the degree of
Doctor of Philosophy
Faculty of Education
Universiti Teknologi Malaysia
MAY 2006
iii
To my beloved parents, sisters, brothers and husband
iv
ACKNOWLEDGEMENT
In preparing this thesis, I was in contact with many people, researchers,
academicians, and practitioners. They have contributed towards my understanding and
thoughts. In particular, I wish to express my deepest appreciation to my thesis
supervisors, Associate Professor Dr. Baharuddin Aris and Professor Dr. Mohd Salleh
Abu for guidance, encouragement, critics and friendship. My sincere appreciation also
extends to the experts and researchers whom I was contacted during my research. Their
views and suggestions are useful indeed.
I am also indebted to Southern College and Lee Foundations for funding my
Ph.D. study.
I am also very thankful the librarians at UTM, National Institute of
Education Singapore and National University of Singapore for their assistance in
supplying the relevant literatures.
I am also very thankful to my colleagues in Southern College for their
suggestions, assistance and motivation. Last but not least, I am grateful to all my family
members.
v
ABSTRACT
This research aims to design, develop and evaluate the effectiveness of a Webbased learning system prototype called Generative Object Oriented Design (GOOD)
learning system. Result from the preliminary study conducted showed most of the
students were at lower order thinking skills (LOTS) compared to higher order thinking
skills (HOTS) based on Bloom’s Taxonomy. Based on such concern, GOOD learning
system was designed and developed based on learning object design and generative
learning to improve HOTS and learning. A conceptual model design of GOOD learning
system, called Generative Learning Object Organizer and Thinking Tasks (GLOOTT)
model, has been proposed from the theoretical framework of this research. The topic
selected for this research was Computer System (CS) which focused on the hardware
concepts from the first year Diploma of Computer Science subjects. GOOD learning
system acts as a mindtool to improve HOTS and learning in CS. A pre-experimental
research design of one group pretest and posttest was used in this research. The samples
of this research were 30 students and 12 lecturers. Data was collected from the pretest,
posttest, portfolio, interview and Web-based learning system evaluation form. The
paired-samples T test analysis was used to analyze the achievement of the pretest and
posttest and the result showed that there was significance difference between the mean
scores of pretest and posttest at the significant level α = 0.05 (p=0.000). In addition, the
paired-samples T test analysis of the cognitive operations from Bloom’s Taxonomy
showed that there was significance difference for each of the cognitive operation of the
students before and after using GOOD learning system. Results from the study showed
improvement of HOTS and learning among the students. Besides, analysis of portfolio
showed that the students engaged HOTS during the use of the system. Most of the
students and lecturers gave positive comments about the effectiveness of the system in
improving HOTS and learning in CS.
From the findings in this research, GOOD
learning system has the potential to improve students’ HOTS and learning.
vi
ABSTRAK
Kajian ini bertujuan untuk merekabentuk, membina dan menilai keberkesanan
prototaip sistem pembelajaran melalui web, iaitu sistem pembelajaran Generative Object
Oriented Design (GOOD). Hasil daripada kajian awal yang dijalankan menunjukan
bahawa kebanyakkan pelajar mempunyai aras kemahiran rendah (LOTS) berbanding
dengan aras kemahiran tinggi (HOTS) berdasarkan taksonomi Bloom. Sistem
pembelajaran GOOD dibinakan berasaskan reka bentuk objek pembelajaran (learning
object design) dan pembelajaran generatif untuk meningkatkan HOTS dan
pembelajaran. Model GLOOTT, iaitu model reka bentuk konsep dicadangkan
berdasarkan kerangka teori dalam kajian ini. Tajuk pembelajaran adalah subjek untuk
Diploma Sains Komputer iaitu Sistem Komputer (SK) dengan tumpuan kepada topik
perkakasan komputer. Sistem pembelajaran GOOD bertindak sebagai mindtool untuk
meningkatkan HOTS dan pembelajaran dalam SK. Pendekatan kajian pra-eksperimen
dengan reka bentuk satu kumpulan ujian pra- ujian pos digunakan dalam kajian ini.
Sampel kajian merupakan 30 orang pelajar dan 12 orang pensyarah. Data diperolehi
menerusi ujian pra dan ujian pos, portfolio, temu bual dan borang penilaian
pembelajaran berasaskan Web. Hasil analisis ujian paired-samples T test menunjukkan
wujudnya perbezaan signifikan di antara min ujian pra dan min ujian pos dalam ujian
pada aras signifikan α = 0.05 (p=0.000). Bagi setiap aras operasi kognitif taksonomi
Bloom, analisis ujian paired-samples T test juga menunjukkan wujudnya perbezaan
signifikan di antara setiap aras operasi kognitif pelajar sebelum dan selepas
menggunakan sistem pembelajaran GOOD. Hasil kajian telah menunjukan peningkatan
HOTS dan pembelajaran di kalangan pelajar. Hasil analisis dalam portfolio
menunjukkan penglibatan pelajar dalam HOTS semasa menggunakan system tersebut.
Kebanyakkan pelajar dan pensyarah memberi komen yang positif terhadap keberkesanan
sistem tersebut dalam peningkatan HOTS dan pembelajaran dalam SK. Hasil kajian
telah menunjukkan bahawa sistem tersebut berpotensi untuk meningkatkan HOTS dan
pembelajaran.
vii
TABLE OF CONTENTS
CHAPTER
1
TITLE
PAGE
DECLARATION
i
DEDICATION
iii
ACKNOWLEDGEMENTS
iv
ABSTRACT
v
ABSTRAK
vi
TABLE OF CONTENTS
vii
LIST OF TABLES
ix
LIST OF FIGURES
xviii
LIST OF ABBREVIATION
xx
LIST OF APPENDICES
xxi
INTRODUCTION
1.0
Introduction
1
1.1
Research Background
2
1.1.1
Higher Order Thinking Skills
3
1.1.2 HOTS and Computer Science
4
1.1.3 Generative Learning and HOTS
6
1.1.4 Learning Object Design
9
1.1.5 Generative Learning, HOTS, Learning Object
11
Design and Web Technology
1.1.6 Instructional Design Model
13
1.2
Problem Statements
14
1.3
Research Rationale
16
viii
2
1.4
Research Objectives
20
1.5
Research Questions
20
1.6
Research Theoretical Framework
21
1.7
The Framework of Instructional Design Model
27
1.8
Research Importance
29
1.9
Research Scope and Limitation
30
1.10
Operational Definition
31
1.11
Summary
35
LITERATURE REVIEW
2.0
Introduction
36
2.1
Higher Order Thinking Skills Learning Object
37
2.1.1 Higher-Order Thinking Skills Teaching
38
Programs and Practice
2.1.2
Definitions of HOTS Attributes of Learning
42
Object
2.1.3 Instructional Strategies for HOTS
48
Granularity of Learning Object
2.1.4 Technology and HOTS Metadata of Learning
51
Object
2.1.5 Research Studies on HOTS
53
2.1.6 Research Studies of Using Technology to
54
Improve HOTS
2.2
2.1.7 HOTS in Computer Science Learning
57
Learning Object
59
2.2.1 What is Learning Object Design?
60
2.2.2 Attributes of Learning Object
62
2.2.3 Granularity of Learning Object
63
2.2.4 Metadata of Learning Object
65
2.2.5 Learning Object Design in Learning
67
ix
2.2.6 Research Studies on Learning Object Design
67
2.3
Generative Learning and Learning Object Design
69
2.4
Generative Learning, HOTS and Learning Object
75
Design
2.5
Web-Based Learning
76
2.5.1 Web-Based Learning and Learning Object
78
Design
2.6
3
Learning of Computer System
80
2.6.1
80
Learning Problems of Computer System
2.6.2 How to Teach and Learn Computer System
83
2.7
Instructional Design Model
86
2.8
Summary
87
RESEARCH METHODOLOGY
3.0
Introduction
89
3.1
An Overview of the HOTS Assessment
89
3.2
Research Framework
95
3.2.1 Phase I: Analysis
97
3.2.2 Phase II: Design
100
3.2.3 Phase III: Development
101
3.2.4 Phase IV: Implementation
102
3.2.5 Phase V: Evaluation
102
3.3
3.4
3.2.5.1 Formative Evaluation
103
3.2.5.2 Summative Evaluation
106
Sampling
109
3.3.1 Learners Sampling
109
3.3.2
Expert Sampling
111
3.3.3
Lecturers Sampling
112
Research Instruments
112
3.4.1 Evaluation Form
112
x
3.5
3.4.2 Pre-test and Post-test
114
3.4.3 Rubric
115
3.4.4 Electronic Portfolio
116
3.4.5 Interview
117
Data Analysis
117
3.5.1 Analysis of Students’ Current Level of HOTS
118
From the Conventional Teaching and
Learning of Computer System (CS)
3.5.2 Analysis of the Effectiveness of the Web-
119
based Learning System in the Improvement of
Students’ Learning
3.5.3 Analysis of the Effectiveness of the Web-
120
based Learning System in the Improvement of
HOTS
3.5.4 Analysis of the Effectiveness of the Web-
121
based Learning System in HOTS Engagement
3.5.5 Analysis of the Effectiveness of the Web-
121
based learning system as Perceived by the
Lecturers and Students
3.6
4
Summary
122
SYSTEM DESIGN AND DEVELOPMENT
4.0
Introduction
123
4.1
Web-Based Learning in Southern College
123
4.2
Design and Development of the Web-based Learning
125
System
4.3
A Pedagogical Design Learning Conceptual Model:
127
GLOOTT
4.4
GOOD Learning System
131
4.5
Structure of Learning Object Design
132
xi
4.6
4.5.1 The Metadata Elements
136
The Design and Development of GOOD Learning
138
System
4.6.1
Students Log In
141
4.6.1.1 Upload Learning Objects
142
4.6.1.2 Design Learning
143
4.6.1.2.1 Implementation of
143
GLOOTT Model in
Learning Tasks of
GOOD Learning System
4.6.1.3
Forum
154
4.6.1.4 Message from the Instructor
154
4.6.1.5 Others Feature That Support
155
Learning
4.6.2 Instructor Log In
4.6.2.1
4.7
Uploading Tasks
GOOD Learning System as Mindtool to Engage
156
157
159
Learners in HOTS
4.8
5
Summary
161
DATA ANALYSIS AND RESULTS
5.0
Introduction
162
5.1
Data Analysis of Students’ Current Level of HOTS
162
From the Conventional Teaching and Learning of
Computer System
5.2
Analysis of the Effectiveness of GOOD Learning
165
System in the Improvement of Students’ Learning
5.3
Analysis of the Effectiveness of GOOD Learning
167
System in the Improvement of HOTS
5.4
Analysis of the Effectiveness of GOOD Learning
169
xii
System in the Students’ HOTS Engagement
5.5
Analysis of the Effectiveness of GOOD Learning
177
System as Perceived by Instructors
5.6
Analysis of the Effectiveness of the Web-based
186
Learning System as Perceived by Students
5.7
6
Summary
198
DISCUSSIONS AND CONCLUSION
6.0
Introduction
199
6.1
Research Summary
199
6.2
Discussion
201
6.2.1 Current Level of HOTS of Students from the
202
Conventional Teaching and Learning of
Computer System (CS)
6.2.2 Effectiveness of GOOD Learning System in
202
the Improvement of Students’ Learning
6.2.3 Effectiveness of GOOD Learning System in
203
Improving HOTS
6.2.4 Effectiveness of GOOD Learning System in
204
HOTS Engagement
6.2.5 Effectiveness of GOOD Learning System as
205
Perceived by the Lecturers
6.2.6 Effectiveness of GOOD Learning System as
206
Perceived by the Students
6.3
Implications of the Research
208
6.4
Limitations of the Research
210
6.5
Suggestions for Further Research
211
6.6
Conclusion
213
xiii
6.7
Summary
214
REFERENCES
215
Appendices A - R
255-295
xiv
LIST OF TABLES
NUMBER OF
TABLE
TITLE
PAGE
1.1
Bloom Taxonomy of Thinking
32
2.1
Transitioning Learning Design
67
3.1
Data to be Collected and Research Design
96
3.2
The System Implementation Plan
102
3.3
Evaluation Plan
103
3.4
Number of Students and the Number of Lesson Maps
They Had Designed
110
3.5
The Sections and Items in the WEF for Lecturer or
Expert
113
3.6
The Sections and Items in the WEF for Students
114
3.7
Table for Record of Mean Score of Cognitive
Operations of HOTS for Each Question
118
3.8
Table for Record of the Sum of the Students’ Scores
and Percentage of Cognitive Operations for All
Questions
119
3.9
Table for the Comparison of Mean Score of Each
Cognitive Operation between Pre-Test and Post-Test
120
4.1
The Metadata Elements
136
5.1
Division of Questions According to the Bloom
Taxonomy of Thinking
163
xv
5.2
Mean Score of Each Taxonomy of Thinking for Each
Question
163
5.3
The Students’ Scores and Percentage in the Taxonomy
of Thinking of All Questions
164
5.4
Score of Pre-Test and Post-Test
166
5.5
Mean Scores of Pre-Test and Post-Test
166
5.6
T-Test Analysis for Mean Scores of Pre-Test and PostTest
167
5.7
T-Test Analysis of Mean Scores for Each Cognitive
Operation in Pre-Test and Post-Test
168
5.8
The HOTS Engagement of Students from Different
Groups
170
5.9
Mean and Standard Deviation of Each Item in Section
A
178
5.10
Mean and Standard Deviation of Each Item in Section
B
178
5.11
Mean and Standard Deviation of Each Item in Section
C
179
5.12
Mean and Standard Deviation of Each Item in Section
D
179
5.13
Mean and Standard Deviation of Each Item in Section
E
180
5.14
Mean and Standard Deviation of Each Item in Section
E
180
5.15
Means and Standard Deviations of Each Item in WEF
for the Lecturers
181
5.16
Aspects They Like in GOOD Learning System
183
5.17
Aspects They Don’t Like GOOD Learning System
183
5.18
Suggestions to Improve GOOD Learning System
184
5.19
The Suitability of GOOD Learning System in
184
xvi
Improving HOTS
5.20
The Suitability of GOOD Learning System in the
Teaching and Learning of Computer System
185
5.21
The Suitability of GOOD Learning System the
Teaching and Learning of Other Subjects
185
5.22
Mean and Standard Deviation of Each Item in Section
A
186
5.23
Mean and Standard Deviation of Each Item in Section
B
187
5.24
Mean and Standard Deviation of Each Item for WEF
Student in
Section C
187
5.25
Mean and Standard Deviation of Each Item in Section
D
188
5.26
Mean and Standard Deviation of Each Item in Section
E
188
5.27
Means and Standard Deviations of Each Item in WEF
for the Students
189
5.28
Comments from the Students About GOOD Learning
System
190
5.29
Suggestions from the Students About GOOD Learning
System
191
5.30
GOOD Learning System in Improving HOTS
192
5.31
GOOD Learning System in Developing Problem
Solving Skills
192
5.32
Parts of GOOD Learning System that Engages Students
in HOTS
193
5.33
Cognitive Operations of HOTS They Aware of in
GOOD Learning System
194
5.34
The Effectiveness of GOOD Learning System in
Improving CS
194
xvii
5.35
The Effectiveness of GOOD Learning System in the
Understanding of the Vocabulary Used In Computer
195
5.36
The Effectiveness of GOOD Learning System in
Developing the Concept of Computer
195
5.37
The Effectiveness of GOOD Learning System in
Improving Your Problem Solving Skills in CS
196
5.38
Feedbacks from the Students about the Importance of
HOTS in Learning CS
196
5.39
The Importance of HOTS Improvement in Learning
Others Computer Science Subject
197
5.40
The Use of HOTS in the Posttest
197
5.41
Outcome of the Use of HOTS in the Posttest
197
5.42
GOOD Learning System in Helping Students in
Learning CS
198
xviii
LIST OF FIGURES
FIGURE NO
TITLE
PAGE
1.1
Conceptual Model Derived from the Research Theoretical
Framework
25
1.2
The Framework of ID Model Modified from ISDMELO
(Baruque and Melo, 2003)
28
3.1
The Design Framework of HOTS Assessment (Adapted from
Costa and Kallick, 2000)
92
3.2
Pre-Experimental Design, One-Group Pretest-Posttest Design
106
3.3
The Distribution of Lesson Maps of the Students
111
4.1
An Example of Interactive Learning Objects
133
4.2
An Example of Web Page Learning Objects
134
4.3
An Example of Learning Objects Designed as Table
134
4.4
An Example of Learning Objects Designed as Graphic
135
4.5
The Suggested Flow of Learning Activities in GOOD
Learning System
139
4.6
The Main Page of GOOD Learning System
140
4.7
The Introduction to GOOD Learning System
140
4.8
Home Page of Student Log In
141
4.9
Mr. TQ
141
4.10
Uploading Learning Objects of Students
142
4.11
Subject and Subtopic Selection in Learning Tasks
143
xix
4.12
GLOO Design in GOOD Learning system
144
4.13
Search Engine and Learning Objects in Library
145
4.14
Lesson Mapping in Learning Object Organizer
146
4.15
Course Map
149
4.16
Try It Out
150
4.17
Apply It
151
4.18
“How Am I Doing” Checklist
152
4.19
Chart of How Am I Doing Checklist
152
4.20
Reflection Worksheet
153
4.21
Forum
154
4.22
The Messages from the Instructor
155
4.23
Mr. TQ, the Information Agent in GOOD Learning System
156
4.24
Mr. TQ Helps Students to Reflect Their Learning Tasks
156
4.25
Upload Subject and Subtopics
157
4.26
Message to the Student
158
4.27
Thinking Task Generator
159
5.1
Chart of the Mean Score in the Taxonomy of Thinking for
Each Question
164
5.2
Percentage of the Total Score for the Taxonomy of Thinking
of All Questions
165
5.3
Mean Scores of Cognitive Operations for Pre-Test and PostTest
169
5.4
The Engagement of HOTS from Each Chapter for Student
P12
172
5.5
The Engagement of HOTS from Each Chapter for Student
173
xx
P29
5.6
The Engagement of HOTS from Each Chapter for Student P2
173
5.7
The Engagement of HOTS from Each Chapter for Student P9
174
5.8
The Engagement of HOTS from Each Chapter for Student
P21
174
5.9
The Engagement of HOTS from Each Chapter for Student
P24
175
5.10
The Engagement of HOTS from Each Chapter for Student
P27
175
5.11
The Engagement of HOTS from Each Chapter for Student
P18
176
5.12
The Engagement of HOTS from Each Chapter for Student
P19
176
5.13
The Engagement of HOTS from Each Chapter for Student
P30
177
xxi
LIST OF ABBREVIATION
HOTS
-
Higher Order Thinking Skills
LOTS
-
Lower Order Thinking Skills
CS
-
Computer System
ICT
-
Information Communication Technology
MOE
-
Ministry of Education
GLOOTT
-
Generative Learning Organizer and Thinking Task
GLOO
-
Generative Learning Organizer
TT
-
Thinking Tasks
LOO
-
Learning Object Organizer
LOR
-
Learning Object Repository
LOs
-
Learning Objects
LMS
-
Learning Management System
LORI
-
Learning Object Review Instrument
WEF
-
Web-based Learning System Evaluation Form
ISDMELO
-
ADDIE
-
Instructional System Design Methodology based on eLearning Object
Analysis, Design, Development, Implementation, Evaluation
DFD
-
Data Flow Diagram
ID
-
Instructional Design
ISD
-
Instructional Systems Development
IMS
-
Instructional Management System
MOODLE
-
Modular Object-Oriented Dynamic Learning Environment
SCORM
-
Sharable Content Object Reference Model
GOOD
-
Generative Object-Oriented Design
xxii
LIST OF APPENDICES
APPENDIX
TITLE
PAGE
A
Rubric of Higher Order Thinking Skills Evaluation
255
B
“How Am I Doing” Checklist
257
C
System Data Flow (DFD) diagram
258
D1
Web-Based Learning Evaluation Form :
Expert/Lecturer
266
D2
Web-Based Learning Evaluation Form: Student
272
E
Pre-Test And Post-Test Questions
275
F
Interview
278
G
Research Instrument Validation (GOOD Learning
System Evaluation Form)
279
H
Research Instrument Validation (GOOD Learning
System Evaluation Form for Students)
281
I
Research Instrument Validation (Pre-Test and PostTest)
283
J
Research Instrument Validation (Pre-Test and PostTest)
284
K
Research Instrument Validation (Rubric of HOTS)
285
L
Research Instrument Validation (“How am I Doing”
Checklist)
286
M
Expert Validation: Content of Learning Object
287
xxiii
N
Expert Validation: Design of the Web-based Learning
System
288
O
List of Papers Publication
289
P
Content of Computer Hardware
290
Q
Score of LOTS And HOTS For Pretest And Posttest
292
R
Analysis of The Distribution of Students’ Scores In
Pretest and Posttest
294
CHAPTER ONE
INTRODUCTION
1.0
Introduction
The emergence of World Wide Web has caused a lot of changes and innovations
in the way people communicate, work, and learn. It has mesmerized educators for over
a decade with its potential of distributed learning and universal educational resources
delivery. An educational revolution is gradually taking place, which includes changes in
the development and delivery of instruction. The changes provide an opportunity to
improve the learning with the appropriate use of learning theories that are coupled with
technologies.
With the rapid development of Information Communication Technology (ICT) in
teaching and learning, the school is no longer essential as an information supplier. The
amount of information channeled through the Internet has outstripped people’s abilities
to process and utilize the information. They are not only required to learn, but also need
to analyze and evaluate the validity and reliability of the information received. The
education system now should emphasize the students as producers but not simply
consumers of information. Hence, it is increasingly important for students have to
possess higher order thinking skills in order to process and to apply the information.
2
While a lot of people are eagerly developing the Web-based learning
environment, there are question marks on how to keep the online learners captivated and
self-motivated to achieve the learning objectives and able to use higher order thinking
skills (HOTS). One of the solutions is to identify the learners’ needs, and economically
customize the individual learning in order to promote the successful learning (Wiley,
2000).
This has brought to the transition from the one-size-fits-all approach to
customization with the growing use of the learning object design (LTSC, 2000).
Learning object is an instructional technology currently used by the educational
technologists and instructional designers for the choice of the instructional design,
development and delivery (Hodgins, 2001; Wiley, 2000).
This chapter provides a background study for the research project by providing
an overview of learning objects and generative learning, higher order thinking skills
(HOTS) and the instructional design model.
Besides, the statements of problem,
suggestions of problem solving, objectives of research, questions of research, suggested
framework of instructional design model, framework of research theory, rational of
research, importance of research, scope and limitation of research, and the definition of
terminology used in research are discussed in this chapter.
1.1
Research Background
Modern life requires people to face various experiences and environments (Tal
and Hochberg, 2003). However, for many years, contemporary education is paying
more focus on the “transmission of information” from teachers and books. In reality,
students should be equipped with higher order thinking skills (HOTS) in their learning
process (Dunlap and Grabinger, 1996a; Osborne and Wittrock, 1983; Hollingworth and
McLoughlin, 2003; Jonassen et al., 1993). The learning process requires the students to
construct their understanding meaningfully and to search for innovative solution in
problem solving.
3
1.1.1 Higher Order Thinking Skills
Higher order thinking skills (HOTS) represent multi-faceted and complex
cognitive processes that develop and improve the processing and construction of
information (Resnick, 1987; Swartz, 2001). The term HOTS used in this research refers
to the analysis, synthesis and evaluation according to Bloom’s Taxonomy of thinking
(Bloom et al., 1956). Thus, the recall of knowledge, comprehension and application are
classified as lower order thinking skills (LOTS) (Dori, Tal and Tsaushu, 2003; Bloom et
al., 1956; Morgan, 1996).
Skills such as analyzing, synthesizing and evaluating
information in the learning process are important in order to develop HOTS (Bloom et
al., 1956; Bloom, Hasting and Madaus, 1971; Ennis, 1987; Zohar, Weinberger and
Tamir, 1994; Jonassen, 1992; Tal and Hochberg, 2003; Morgan, 1996).
A lot of
researchers have pointed out the increasing importance of HOTS in the teaching and
learning process.
Problem solving requires the use of HOTS such as analysis, synthesis and
evaluation. This is in line with the argument from Kallick (2001a) that the cognitive
operations such as analyzing, inferring and evaluating are necessary in problem solving.
According to Jonassen (1992), argumentation is an appropriate outcome for problem
solving where students generate arguments and make reasoning to defend their
solutions. This encourages them in using HOTS. Besides, reflective thinking is also
often related to HOTS (Quellmalz, 1987; Vockell, and van Deusen, 1989; Fogarty and
McTighe, 1993; Wai and Hirakawa, 2001; Fogarty, 2002; Harrigan and Vincenti, 2004).
Reflective thinking helps students to be aware of their thinking as they perform tasks or
learning and this engages them in HOTS. Hence, the reflective thinking was used as an
important cognitive operation for scaffolding the encouragement of HOTS in this
research.
From the above description, it is apparent that HOTS requires students to
manipulate information and ideas in ways that transform their meaning and implication.
This occurs when students combine facts and ideas in order to analyze, synthesize, and
4
evaluate in generating knowledge. The manipulation of information and ideas through
these processes allows students to solve problem, generate knowledge and promote
understanding.
1.1.2 HOTS and Computer Science
As technology changes at an ever-increasing speed, the students must have the
ability to adapt to changes and become lifelong learners. This is especially true in the
computing field. The students have to be good in both thinking and problem solving
skills. However, most of the colleges focus more on rote lecturing, assignments and
tests (Tan Wee Chuen, Baharuddin Aris and Mohd Salleh Abu, 2005). They rarely
promote HOTS among the students in order to understand and apply problem solving
and logical reasoning skills in the learning of Computer Science (Parham, 2003; Arup,
2004).
Harrigan and Vincenti (2004) noted that HOTS are important in college teaching
and learning. A lot of studies have been conducted to study the teaching and learning
process in Computer Science domain in higher education. Empirical results from the
studies show that many students can not demonstrate HOTS in their learning (Chmura,
1998; Henderson, 1986; Arup, 2004). Most of the students resort to trial and error, and
memorizing facts from their learning, rather than learning problem solving skills.
However, these HOTS such as analysis, synthesis and evaluation thinking skills are
found important in the learning of Computer Science.
This was demonstrated by
Parham (2003) in which there is a direct correlation between the students’ HOTS and
their performance in their learning.
Hadjerrouit (1999) noted that the conventional predominant teaching model
viewed learning as the passive transmission on knowledge and this cause serious
misconception and lack of conceptual understanding in Computer Science learning.
This is further supported by Arup (2004) that the existing learning in computer system
5
tends to regurgitate what the instructors have taught and does not imply the ability to use
HOTS among the students. This is also proven by the studies from Maj, Veal, and
Charlesworth (2000); Holmboe (1999) and Mirmotahari, Holmboe and Kaasboll (2003)
that the college students are lacking of knowledge of computer technology and the basic
skills to operate computer systems.
Another main problem of Computer Science students is the lack of deep
understanding of the relationships in the facts they have learned (Scragg, Baldwin and
Koomen, 1994; Mirmotahari, Holmboe and Kaasboll, 2003). Students are better in the
practical skills than theoretical questions. In computer education, the prior knowledge
of students is the foundation for further knowledge construction (Holmboe, 1999; White,
2001; Mirmotahari, Holmboe and Kaasboll, 2003; Scragg, 1991). New information must
be linked to information already understood (Rosenberg, 1976; Hamza, Alhalabi and
Marcovitz, 2000). Learners would generate and test ideas that either have been created
from their prior knowledge.
The content of the computer has to stay abreast of the rapidly developing
computer technology. HOTS are essential to the students in this rapidly changing
technological society (Morgan, 1996). The growth of knowledge in computer needs
more timeliness in teaching resources, expertise and preparation time (Wolffe et al.,
2002). This leads to a large amount of information being drained to the students.
Instructors and students have been burdened with the task of communicating massive
and rapidly changing computer content. Consequently, the overemphasis on content has
resulted in the lack of attention on the HOTS that are necessary for the students to
successfully solve the complex scenarios (Arup, 2004).
Researchers in the education field are progressing toward the teaching and
learning with technology to develop HOTS. Studies of HOTS program from Pogrow
(1988a, 1988b), Herrington and Oliver (1999), Tay (2002) and Tal and Hochberg (2003)
showed encouraging results. Technology can be used as a mindtool for conceptual
development (Reeves, 1998; Jonassen, 2000) and to enable higher order learning (Ting,
6
2003; Reeves, 1998; Pogrow, 1988a, 1988b). In this context, the learner acts as a
designer in the learning process (Jonassen, 1994; Jonassen and Reeves, 1996). Jonassen,
Mayes and McAleese (1993) found that individual learns the most from the design of
instructional materials. Therefore, if the students were given opportunity to construct or
design their own learning, it creates an active learning environment. This process
requires the students to think more meaningfully and therefore helps to develop their
HOTS.
1.1.3 Generative Learning and HOTS
A lot of instructional strategies have been proposed to develop the HOTS. One
of the most frequent proposals is generative learning.
Generative learning is an
important constructivist learning environment (Bannan-Ritland, Dabbagh and Murphy,
2000; Dunlap and Grabinger, 1996a; Duffy and Jonassen, 1992; Morrison and Collins,
1996; Grabowski, 1996; Bonn, and Grabowski, 2001; Jonassen, Mayes and McAleese,
1993; McLoughlin, 1998).
According to Bonn, and Grabowski (2001), generative
learning provides the necessary theoretical framework for research in a constructivist
perspective.
As described by CTGV (1993), the generative learning is the first key
element of constructivism learning environment. In generative learning environment,
learning is generative; learners focus on the construction of their own learning. In this
research, the Web-based learning system used the generative learning to design the
constructive learning environment.
Originally, generative learning is conceived under the cognitive information
processing proposed by Wittrock (1974). The focus of generative learning model is that,
learner is an active participant who works to construct meaningful understanding by
generating relationships between the information. The cognitive psychologists and
educationists usually refer the skills associated with this kind of thinking activities as
HOTS. These activities are completely in contrast to those which simply copy down
information and memorize them, where the students passively receive information and
7
respond to the exercises or examinations that require only facts recalling and simple
understanding. Dunlap and Grabinger (1996b) pointed out that generative learning is a
higher-level thinking activity. HOTS depict the dynamic role of learners in which they
act as thinkers actively participate in constructing knowledge. Such a view of learning
fits well with the empirical evidence from the studies of technology and Computer
Science learning and teaching (refer Chapter 2 for further discussion).
Learning that emphasizes on the connection between the new and old concepts,
and among the concepts is important to enhance understanding (Nickerson, 1995). The
connection among the concepts is also important in learning Computer Science
(Rosenberg, 1976; Hamza, Alhalabi and Marcovitz, 2000). However, the conventional
teaching models in Computer Science often view learning as a passive transmission on
knowledge. This results in misconception, lack of conceptual understanding and the poor
understanding of the relationships in the concepts that the students learned (Hadjerrouit,
1999; Maj, Veal, and Charlesworth, 2000; Holmboe, 1999; Mirmotahari, Holmboe and
Kaasboll, 2003; Scragg, Baldwin and Koomen, 1994). In contrast, generative learning
provides a learning environment that enhances the learning through actively construct
meaningful understanding by generating relationships among the concepts.
According to Dunlap and Grabinger (1996a), content is often presented to the
learners in the format that promotes memorization rather than higher order thinking.
Most of the schools are still examination-oriented. The teaching and learning often
focus in answering specific questions in the examinations. With the generative learning,
it promotes active processing in the linkage of the concepts and supportive environment
that encourages them to think and construct knowledge from their understanding.
Higher education institute is the most appropriate venue for this learning approach
because their goals are to promote advanced knowledge acquisition and HOTS
(Jonassen, Mayes and McAleese, 1993).
Concept map provides an important tool in generative learning environment
(Osborne and Wittrock, 1983; Bannan-Ritland, Dabbagh and Murphy, 2000). Concept
8
map functions as a tool to engage learners to generate and to organize the ideas in the
content being studied. According to Jonassen (2000), concept map engages learners in
the reorganization of knowledge, explicit description of concept and their
interrelationships, deep processing of knowledge that promotes better remembering,
retrieval and application of knowledge; and relating new concepts to existing ones that
improves understanding. This is consistent with the theoretical perspective of generative
learning.
The concept map used in this research is called as lesson map. It is an outline
form of concept map as suggested by Alpert and Grueneberg (2000), and Dabbagh
(2001). The lesson map used in this research enables the construction of concepts
require HOTS when students organize the lesson map, select important and relevant
concepts to add to the map, search the crosslink and indicate the relationships between
concepts. These activities engage students in HOTS that are analysis thinking while they
are organizing the concepts in hierarchical structure, synthesis thinking while they are
searching crosslink and indicating the relationships between the concepts and evaluating
while they are searching and judging the important and relevant of the concepts
(Jonassen, 2000; Dabbagh, 2001; Alpert and Grueneberg, 2000).
Jonassen, Mayes and McAleese (1993) pointed out that the generative learning
results deeper levels of knowledge processing and construction, and these necessitate the
HOTS. They further pointed out that constructivist learning environments aim to engage
students in higher order and meaningful learning.
Besides that, Jonassen (1992),
Jonassen, Mayes and McAleese (1993) noted that the outcomes of constructivist
learning environments should assess HOTS in order to reflect the intellectual processes
of knowledge construction.
Studies show that generative learning and teaching
provoked learners’ thinking skills and developed their understanding (Laney, 1990;
Schaverien, 2000; McLoughlin, 1998; Dunlap and Grabinger, 1996a, 1996b; McGriff,
2002; Osborne and Wittrock, 1983; McLoughlin, 1998). The process of generative
learning engages student in HOTS (Grabowski, 1995; Grabowski, 1996; McLoughlin,
1998). From this perspective, the generative learning is strongly related to the HOTS.
9
Gao and Lehman (2003) noted that most of the researches in generative learning
emphasize in the facts and concepts-level learning and deal little with HOTS. Therefore,
further research on generative learning that focuses on HOTS is necessary.
1.1.4 Learning Object Design
Nowadays, most of the instructional designers understood the importance of
pedagogical perspectives in the design and development of Web learning environments.
According to Snow (1989), instructions differ in structure and completeness and highly
structured instructions (linear sequence, restricted and high external control) seem to
help those with low ability but hinder those with high ability. The concept of one-sizefits-all design is no longer suitable in the design and development in e-learning. The
learning environment should be highly flexible in structures and hands control out of the
hands of the systems or instructors to the learners. Therefore, the concept of learning
object design fits this very well as it provides flexible paths to the users’ exploration in
the teaching and learning process. The non-linearity of the learning object design allows
students to access information in different patterns and to take control in their own
action and learning.
A learning object is a small, reusable digital component that can be selectively
applied alone or in combination by computer software, learning facilitators or learners
themselves, to meet individual needs for learning or performance support (Shepherd,
2000). There are three interdependent components in the learning object design model:
the learning object itself; metatagging (a standardized way to describe the content in
code); a Learning Content Management System (LCMS) that stores, tracks, and delivers
content.
Learning object design is the design of instructions into small learning contents
that can be reused in different context and combined to form learning that are
appropriate to the individual (Wiley, 2000; Hodgins, 2001; Wagner, 2002; Mills, 2002;
10
Longmire, 2000a, 2000b; Robbins, 2002; Lau, 2002; Gibbons, Nelson and Richards,
2000; Hanaffin, Hill and McCarthy, 2000; South and Monson, 2000; Collis and Strijker,
2003). The design of instructions into learning objects can be deployed into multiple
setting and learning goals. It is a current trend of computer-based instructions and
learning that are grounded in the object-oriented paradigm of Computer Science.
The idea of information in small chunks which are reusable and flexible in a
learning environment has received a lot of compliments from the educators and
instructional designers of e-learning environment. According to Reigeluth and Nelson
(1997), when teachers first gain access to instructional materials, they often break the
materials down into their constituting parts and then reassemble these parts in ways that
support their instructional goals. Thus, the notion of small and reusable units of learning
content, learning components, and learning object design have the potential to provide
the flexibility and reusability by simplifying the assembly and disassembly of
instructional design and development.
E-learning industry has anticipated the day where learners could personalize,
assemble, and access e-learning on demand for years (Mortimer, 2002). Most electronic
learning content is currently developed for specific purposes. How does a learner select
only a small part of content that suit their learning needs? The educational software
development is an extremely expensive process in terms of cost and time (Wiley, 2000;
Downes, 2000; Longmire, 2000a). With the learning object design, the learning objects
can be reused and shared. Molenda and Sullivan (2002) noted that there is a critical
need for more efficient design and production of the digital learning materials. Thus,
learning object design had become more practical now especially with the essential
features of the World Wide Web.