DEVELOPMENT AND EVALUATION OF A WEB-BASED LEARNING SYSTEM
BASED ON LEARNING OBJECT DESIGN AND GENERATIVE LEARNING TO
IMPROVE HIGHER-ORDER THINKING SKILLS AND LEARNING
TAN WEE CHUEN
UNIVERSITI TEKNOLOGI MALAYSIA
DEVELOPMENT AND EVALUATION OF A WEB-BASED LEARNING SYSTEM
BASED ON LEARNING OBJECT DESIGN AND GENERATIVE LEARNING TO
IMPROVE HIGHER-ORDER THINKING SKILLS AND LEARNING
TAN WEE CHUEN
A thesis submitted in fulfilment of the
requirements for the award of the degree of
Doctor of Philosophy
Faculty of Education
Universiti Teknologi Malaysia
MAY 2006
iii
To my beloved parents, sisters, brothers and husband
iv
ACKNOWLEDGEMENT
In preparing this thesis, I was in contact with many people, researchers,
academicians, and practitioners. They have contributed towards my understanding and
thoughts. In particular, I wish to express my deepest appreciation to my thesis
supervisors, Associate Professor Dr. Baharuddin Aris and Professor Dr. Mohd Salleh
Abu for guidance, encouragement, critics and friendship. My sincere appreciation also
extends to the experts and researchers whom I was contacted during my research. Their
views and suggestions are useful indeed.
I am also indebted to Southern College and Lee Foundations for funding my
Ph.D. study. I am also very thankful the librarians at UTM, National Institute of
Education Singapore and National University of Singapore for their assistance in
supplying the relevant literatures.
I am also very thankful to my colleagues in Southern College for their
suggestions, assistance and motivation. Last but not least, I am grateful to all my family
members.
v
ABSTRACT
This research aims to design, develop and evaluate the effectiveness of a Webbased learning system prototype called Generative Object Oriented Design (GOOD)
learning system. Result from the preliminary study conducted showed most of the
students were at lower order thinking skills (LOTS) compared to higher order thinking
skills (HOTS) based on Bloom’s Taxonomy. Based on such concern, GOOD learning
system was designed and developed based on learning object design and generative
learning to improve HOTS and learning. A conceptual model design of GOOD learning
system, called Generative Learning Object Organizer and Thinking Tasks (GLOOTT)
model, has been proposed from the theoretical framework of this research. The topic
selected for this research was Computer System (CS) which focused on the hardware
concepts from the first year Diploma of Computer Science subjects. GOOD learning
system acts as a mindtool to improve HOTS and learning in CS. A pre-experimental
research design of one group pretest and posttest was used in this research. The samples
of this research were 30 students and 12 lecturers. Data was collected from the pretest,
posttest, portfolio, interview and Web-based learning system evaluation form. The
paired-samples T test analysis was used to analyze the achievement of the pretest and
posttest and the result showed that there was significance difference between the mean
scores of pretest and posttest at the significant level α = 0.05 (p=0.000). In addition, the
paired-samples T test analysis of the cognitive operations from Bloom’s Taxonomy
showed that there was significance difference for each of the cognitive operation of the
students before and after using GOOD learning system. Results from the study showed
improvement of HOTS and learning among the students. Besides, analysis of portfolio
showed that the students engaged HOTS during the use of the system. Most of the
students and lecturers gave positive comments about the effectiveness of the system in
improving HOTS and learning in CS.
From the findings in this research, GOOD
learning system has the potential to improve students’ HOTS and learning.
vi
ABSTRAK
Kajian ini bertujuan untuk merekabentuk, membina dan menilai keberkesanan
prototaip sistem pembelajaran melalui web, iaitu sistem pembelajaran Generative Object
Oriented Design (GOOD). Hasil daripada kajian awal yang dijalankan menunjukan
bahawa kebanyakkan pelajar mempunyai aras kemahiran rendah (LOTS) berbanding
dengan aras kemahiran tinggi (HOTS) berdasarkan taksonomi Bloom. Sistem
pembelajaran GOOD dibinakan berasaskan reka bentuk objek pembelajaran (learning
object design) dan pembelajaran generatif untuk meningkatkan HOTS dan
pembelajaran. Model GLOOTT, iaitu model reka bentuk konsep dicadangkan
berdasarkan kerangka teori dalam kajian ini. Tajuk pembelajaran adalah subjek untuk
Diploma Sains Komputer iaitu Sistem Komputer (SK) dengan tumpuan kepada topik
perkakasan komputer. Sistem pembelajaran GOOD bertindak sebagai mindtool untuk
meningkatkan HOTS dan pembelajaran dalam SK. Pendekatan kajian pra-eksperimen
dengan reka bentuk satu kumpulan ujian pra- ujian pos digunakan dalam kajian ini.
Sampel kajian merupakan 30 orang pelajar dan 12 orang pensyarah. Data diperolehi
menerusi ujian pra dan ujian pos, portfolio, temu bual dan borang penilaian
pembelajaran berasaskan Web. Hasil analisis ujian paired-samples T test menunjukkan
wujudnya perbezaan signifikan di antara min ujian pra dan min ujian pos dalam ujian
pada aras signifikan α = 0.05 (p=0.000). Bagi setiap aras operasi kognitif taksonomi
Bloom, analisis ujian paired-samples T test juga menunjukkan wujudnya perbezaan
signifikan di antara setiap aras operasi kognitif pelajar sebelum dan selepas
menggunakan sistem pembelajaran GOOD. Hasil kajian telah menunjukan peningkatan
HOTS dan pembelajaran di kalangan pelajar. Hasil analisis dalam portfolio
menunjukkan penglibatan pelajar dalam HOTS semasa menggunakan system tersebut.
Kebanyakkan pelajar dan pensyarah memberi komen yang positif terhadap keberkesanan
sistem tersebut dalam peningkatan HOTS dan pembelajaran dalam SK. Hasil kajian
telah menunjukkan bahawa sistem tersebut berpotensi untuk meningkatkan HOTS dan
pembelajaran.
vii
TABLE OF CONTENTS
CHAPTER
1
TITLE
PAGE
DECLARATION
i
DEDICATION
iii
ACKNOWLEDGEMENTS
iv
ABSTRACT
v
ABSTRAK
vi
TABLE OF CONTENTS
vii
LIST OF TABLES
ix
LIST OF FIGURES
xviii
LIST OF ABBREVIATION
xx
LIST OF APPENDICES
xxi
INTRODUCTION
1.0
Introduction
1
1.1
Research Background
2
1.1.1
Higher Order Thinking Skills
3
1.1.2
HOTS and Computer Science
4
1.1.3
Generative Learning and HOTS
6
1.1.4
Learning Object Design
9
1.1.5
Generative Learning, HOTS, Learning Object
11
Design and Web Technology
1.1.6
Instructional Design Model
13
1.2
Problem Statements
14
1.3
Research Rationale
16
viii
2
1.4
Research Objectives
20
1.5
Research Questions
20
1.6
Research Theoretical Framework
21
1.7
The Framework of Instructional Design Model
27
1.8
Research Importance
29
1.9
Research Scope and Limitation
30
1.10
Operational Definition
31
1.11
Summary
35
LITERATURE REVIEW
2.0
Introduction
36
2.1
Higher Order Thinking Skills Learning Object
37
2.1.1
38
Higher-Order Thinking Skills Teaching
Programs and Practice
2.1.2
Definitions of HOTS Attributes of Learning
42
Object
2.1.3
Instructional Strategies for HOTS
48
Granularity of Learning Object
2.1.4
Technology and HOTS Metadata of Learning
51
Object
2.1.5
Research Studies on HOTS
53
2.1.6
Research Studies of Using Technology to
54
Improve HOTS
2.1.7
2.2
HOTS in Computer Science Learning
57
Learning Object
59
2.2.1
What is Learning Object Design?
60
2.2.2
Attributes of Learning Object
62
2.2.3
Granularity of Learning Object
63
2.2.4
Metadata of Learning Object
65
2.2.5
Learning Object Design in Learning
67
ix
2.2.6
Research Studies on Learning Object Design
67
2.3
Generative Learning and Learning Object Design
69
2.4
Generative Learning, HOTS and Learning Object
75
Design
2.5
Web-Based Learning
76
2.5.1
78
Web-Based Learning and Learning Object
Design
2.6
3
Learning of Computer System
80
2.6.1
Learning Problems of Computer System
80
2.6.2
How to Teach and Learn Computer System
83
2.7
Instructional Design Model
86
2.8
Summary
87
RESEARCH METHODOLOGY
3.0
Introduction
89
3.1
An Overview of the HOTS Assessment
89
3.2
Research Framework
95
3.2.1
Phase I: Analysis
97
3.2.2
Phase II: Design
100
3.2.3
Phase III: Development
101
3.2.4
Phase IV: Implementation
102
3.2.5
Phase V: Evaluation
102
3.2.5.1
Formative Evaluation
103
3.2.5.2
Summative Evaluation
106
3.3
3.4
Sampling
109
3.3.1
Learners Sampling
109
3.3.2
Expert Sampling
111
3.3.3
Lecturers Sampling
112
Research Instruments
112
3.4.1
112
Evaluation Form
x
3.5
3.4.2
Pre-test and Post-test
114
3.4.3
Rubric
115
3.4.4
Electronic Portfolio
116
3.4.5
Interview
117
Data Analysis
117
3.5.1
118
Analysis of Students’ Current Level of HOTS
From the Conventional Teaching and
Learning of Computer System (CS)
3.5.2
Analysis of the Effectiveness of the Web-
119
based Learning System in the Improvement of
Students’ Learning
3.5.3
Analysis of the Effectiveness of the Web-
120
based Learning System in the Improvement of
HOTS
3.5.4
Analysis of the Effectiveness of the Web-
121
based Learning System in HOTS Engagement
3.5.5
Analysis of the Effectiveness of the Web-
121
based learning system as Perceived by the
Lecturers and Students
3.6
4
Summary
122
SYSTEM DESIGN AND DEVELOPMENT
4.0
Introduction
123
4.1
Web-Based Learning in Southern College
123
4.2
Design and Development of the Web-based Learning
125
System
4.3
A Pedagogical Design Learning Conceptual Model:
127
GLOOTT
4.4
GOOD Learning System
131
4.5
Structure of Learning Object Design
132
xi
4.5.1
4.6
The Metadata Elements
The Design and Development of GOOD Learning
136
138
System
4.6.1
Students Log In
141
4.6.1.1
Upload Learning Objects
142
4.6.1.2
Design Learning
143
4.6.1.2.1
143
Implementation of
GLOOTT Model in
Learning Tasks of
GOOD Learning System
4.6.1.3
Forum
154
4.6.1.4
Message from the Instructor
154
4.6.1.5
Others Feature That Support
155
Learning
4.6.2
4.7
Instructor Log In
156
4.6.2.1
157
Uploading Tasks
GOOD Learning System as Mindtool to Engage
159
Learners in HOTS
4.8
5
Summary
161
DATA ANALYSIS AND RESULTS
5.0
Introduction
162
5.1
Data Analysis of Students’ Current Level of HOTS
162
From the Conventional Teaching and Learning of
Computer System
5.2
Analysis of the Effectiveness of GOOD Learning
165
System in the Improvement of Students’ Learning
5.3
Analysis of the Effectiveness of GOOD Learning
167
System in the Improvement of HOTS
5.4
Analysis of the Effectiveness of GOOD Learning
169
xii
System in the Students’ HOTS Engagement
5.5
Analysis of the Effectiveness of GOOD Learning
177
System as Perceived by Instructors
5.6
Analysis of the Effectiveness of the Web-based
186
Learning System as Perceived by Students
5.7
6
Summary
198
DISCUSSIONS AND CONCLUSION
6.0
Introduction
199
6.1
Research Summary
199
6.2
Discussion
201
6.2.1
202
Current Level of HOTS of Students from the
Conventional Teaching and Learning of
Computer System (CS)
6.2.2
Effectiveness of GOOD Learning System in
202
the Improvement of Students’ Learning
6.2.3
Effectiveness of GOOD Learning System in
203
Improving HOTS
6.2.4
Effectiveness of GOOD Learning System in
204
HOTS Engagement
6.2.5
Effectiveness of GOOD Learning System as
205
Perceived by the Lecturers
6.2.6
Effectiveness of GOOD Learning System as
206
Perceived by the Students
6.3
Implications of the Research
208
6.4
Limitations of the Research
210
6.5
Suggestions for Further Research
211
6.6
Conclusion
213
xiii
6.7
Summary
214
REFERENCES
215
Appendices A - R
255-295
xiv
LIST OF TABLES
NUMBER OF
TABLE
TITLE
PAGE
1.1
Bloom Taxonomy of Thinking
32
2.1
Transitioning Learning Design
67
3.1
Data to be Collected and Research Design
96
3.2
The System Implementation Plan
102
3.3
Evaluation Plan
103
3.4
Number of Students and the Number of Lesson Maps
They Had Designed
110
3.5
The Sections and Items in the WEF for Lecturer or
Expert
113
3.6
The Sections and Items in the WEF for Students
114
3.7
Table for Record of Mean Score of Cognitive
Operations of HOTS for Each Question
118
3.8
Table for Record of the Sum of the Students’ Scores
and Percentage of Cognitive Operations for All
Questions
119
3.9
Table for the Comparison of Mean Score of Each
Cognitive Operation between Pre-Test and Post-Test
120
4.1
The Metadata Elements
136
5.1
Division of Questions According to the Bloom
Taxonomy of Thinking
163
xv
5.2
Mean Score of Each Taxonomy of Thinking for Each
Question
163
5.3
The Students’ Scores and Percentage in the Taxonomy
of Thinking of All Questions
164
5.4
Score of Pre-Test and Post-Test
166
5.5
Mean Scores of Pre-Test and Post-Test
166
5.6
T-Test Analysis for Mean Scores of Pre-Test and PostTest
167
5.7
T-Test Analysis of Mean Scores for Each Cognitive
Operation in Pre-Test and Post-Test
168
5.8
The HOTS Engagement of Students from Different
Groups
170
5.9
Mean and Standard Deviation of Each Item in Section
A
178
5.10
Mean and Standard Deviation of Each Item in Section
B
178
5.11
Mean and Standard Deviation of Each Item in Section
C
179
5.12
Mean and Standard Deviation of Each Item in Section
D
179
5.13
Mean and Standard Deviation of Each Item in Section
E
180
5.14
Mean and Standard Deviation of Each Item in Section
E
180
5.15
Means and Standard Deviations of Each Item in WEF
for the Lecturers
181
5.16
Aspects They Like in GOOD Learning System
183
5.17
Aspects They Don’t Like GOOD Learning System
183
5.18
Suggestions to Improve GOOD Learning System
184
5.19
The Suitability of GOOD Learning System in
184
xvi
Improving HOTS
5.20
The Suitability of GOOD Learning System in the
Teaching and Learning of Computer System
185
5.21
The Suitability of GOOD Learning System the
Teaching and Learning of Other Subjects
185
5.22
Mean and Standard Deviation of Each Item in Section
A
186
5.23
Mean and Standard Deviation of Each Item in Section
B
187
5.24
Mean and Standard Deviation of Each Item for WEF
Student in
Section C
187
5.25
Mean and Standard Deviation of Each Item in Section
D
188
5.26
Mean and Standard Deviation of Each Item in Section
E
188
5.27
Means and Standard Deviations of Each Item in WEF
for the Students
189
5.28
Comments from the Students About GOOD Learning
System
190
5.29
Suggestions from the Students About GOOD Learning
System
191
5.30
GOOD Learning System in Improving HOTS
192
5.31
GOOD Learning System in Developing Problem
Solving Skills
192
5.32
Parts of GOOD Learning System that Engages Students
in HOTS
193
5.33
Cognitive Operations of HOTS They Aware of in
GOOD Learning System
194
5.34
The Effectiveness of GOOD Learning System in
Improving CS
194
xvii
5.35
The Effectiveness of GOOD Learning System in the
Understanding of the Vocabulary Used In Computer
195
5.36
The Effectiveness of GOOD Learning System in
Developing the Concept of Computer
195
5.37
The Effectiveness of GOOD Learning System in
Improving Your Problem Solving Skills in CS
196
5.38
Feedbacks from the Students about the Importance of
HOTS in Learning CS
196
5.39
The Importance of HOTS Improvement in Learning
Others Computer Science Subject
197
5.40
The Use of HOTS in the Posttest
197
5.41
Outcome of the Use of HOTS in the Posttest
197
5.42
GOOD Learning System in Helping Students in
Learning CS
198
xviii
LIST OF FIGURES
FIGURE NO
TITLE
PAGE
1.1
Conceptual Model Derived from the Research Theoretical
Framework
25
1.2
The Framework of ID Model Modified from ISDMELO
(Baruque and Melo, 2003)
28
3.1
The Design Framework of HOTS Assessment (Adapted from
Costa and Kallick, 2000)
92
3.2
Pre-Experimental Design, One-Group Pretest-Posttest Design
106
3.3
The Distribution of Lesson Maps of the Students
111
4.1
An Example of Interactive Learning Objects
133
4.2
An Example of Web Page Learning Objects
134
4.3
An Example of Learning Objects Designed as Table
134
4.4
An Example of Learning Objects Designed as Graphic
135
4.5
The Suggested Flow of Learning Activities in GOOD
Learning System
139
4.6
The Main Page of GOOD Learning System
140
4.7
The Introduction to GOOD Learning System
140
4.8
Home Page of Student Log In
141
4.9
Mr. TQ
141
4.10
Uploading Learning Objects of Students
142
4.11
Subject and Subtopic Selection in Learning Tasks
143
xix
4.12
GLOO Design in GOOD Learning system
144
4.13
Search Engine and Learning Objects in Library
145
4.14
Lesson Mapping in Learning Object Organizer
146
4.15
Course Map
149
4.16
Try It Out
150
4.17
Apply It
151
4.18
“How Am I Doing” Checklist
152
4.19
Chart of How Am I Doing Checklist
152
4.20
Reflection Worksheet
153
4.21
Forum
154
4.22
The Messages from the Instructor
155
4.23
Mr. TQ, the Information Agent in GOOD Learning System
156
4.24
Mr. TQ Helps Students to Reflect Their Learning Tasks
156
4.25
Upload Subject and Subtopics
157
4.26
Message to the Student
158
4.27
Thinking Task Generator
159
5.1
Chart of the Mean Score in the Taxonomy of Thinking for
Each Question
164
5.2
Percentage of the Total Score for the Taxonomy of Thinking
of All Questions
165
5.3
Mean Scores of Cognitive Operations for Pre-Test and PostTest
169
5.4
The Engagement of HOTS from Each Chapter for Student
P12
172
5.5
The Engagement of HOTS from Each Chapter for Student
173
xx
P29
5.6
The Engagement of HOTS from Each Chapter for Student P2
173
5.7
The Engagement of HOTS from Each Chapter for Student P9
174
5.8
The Engagement of HOTS from Each Chapter for Student
P21
174
5.9
The Engagement of HOTS from Each Chapter for Student
P24
175
5.10
The Engagement of HOTS from Each Chapter for Student
P27
175
5.11
The Engagement of HOTS from Each Chapter for Student
P18
176
5.12
The Engagement of HOTS from Each Chapter for Student
P19
176
5.13
The Engagement of HOTS from Each Chapter for Student
P30
177
xxi
LIST OF ABBREVIATION
HOTS
-
Higher Order Thinking Skills
LOTS
-
Lower Order Thinking Skills
CS
-
Computer System
ICT
-
Information Communication Technology
MOE
-
Ministry of Education
GLOOTT
-
Generative Learning Organizer and Thinking Task
GLOO
-
Generative Learning Organizer
TT
-
Thinking Tasks
LOO
-
Learning Object Organizer
LOR
-
Learning Object Repository
LOs
-
Learning Objects
LMS
-
Learning Management System
LORI
-
Learning Object Review Instrument
WEF
-
Web-based Learning System Evaluation Form
ISDMELO
-
ADDIE
-
Instructional System Design Methodology based on eLearning Object
Analysis, Design, Development, Implementation, Evaluation
DFD
-
Data Flow Diagram
ID
-
Instructional Design
ISD
-
Instructional Systems Development
IMS
-
Instructional Management System
MOODLE
-
Modular Object-Oriented Dynamic Learning Environment
SCORM
-
Sharable Content Object Reference Model
GOOD
-
Generative Object-Oriented Design
xxii
LIST OF APPENDICES
APPENDIX
TITLE
PAGE
A
Rubric of Higher Order Thinking Skills Evaluation
255
B
“How Am I Doing” Checklist
257
C
System Data Flow (DFD) diagram
258
D1
Web-Based Learning Evaluation Form :
Expert/Lecturer
266
D2
Web-Based Learning Evaluation Form: Student
272
E
Pre-Test And Post-Test Questions
275
F
Interview
278
G
Research Instrument Validation (GOOD Learning
System Evaluation Form)
279
H
Research Instrument Validation (GOOD Learning
System Evaluation Form for Students)
281
I
Research Instrument Validation (Pre-Test and PostTest)
283
J
Research Instrument Validation (Pre-Test and PostTest)
284
K
Research Instrument Validation (Rubric of HOTS)
285
L
Research Instrument Validation (“How am I Doing”
Checklist)
286
M
Expert Validation: Content of Learning Object
287
xxiii
N
Expert Validation: Design of the Web-based Learning
System
288
O
List of Papers Publication
289
P
Content of Computer Hardware
290
Q
Score of LOTS And HOTS For Pretest And Posttest
292
R
Analysis of The Distribution of Students’ Scores In
Pretest and Posttest
294
CHAPTER ONE
INTRODUCTION
1.0
Introduction
The emergence of World Wide Web has caused a lot of changes and innovations
in the way people communicate, work, and learn. It has mesmerized educators for over
a decade with its potential of distributed learning and universal educational resources
delivery. An educational revolution is gradually taking place, which includes changes in
the development and delivery of instruction. The changes provide an opportunity to
improve the learning with the appropriate use of learning theories that are coupled with
technologies.
With the rapid development of Information Communication Technology (ICT) in
teaching and learning, the school is no longer essential as an information supplier. The
amount of information channeled through the Internet has outstripped people’s abilities
to process and utilize the information. They are not only required to learn, but also need
to analyze and evaluate the validity and reliability of the information received. The
education system now should emphasize the students as producers but not simply
consumers of information. Hence, it is increasingly important for students have to
possess higher order thinking skills in order to process and to apply the information.
2
While a lot of people are eagerly developing the Web-based learning
environment, there are question marks on how to keep the online learners captivated and
self-motivated to achieve the learning objectives and able to use higher order thinking
skills (HOTS). One of the solutions is to identify the learners’ needs, and economically
customize the individual learning in order to promote the successful learning (Wiley,
2000).
This has brought to the transition from the one-size-fits-all approach to
customization with the growing use of the learning object design (LTSC, 2000).
Learning object is an instructional technology currently used by the educational
technologists and instructional designers for the choice of the instructional design,
development and delivery (Hodgins, 2001; Wiley, 2000).
This chapter provides a background study for the research project by providing
an overview of learning objects and generative learning, higher order thinking skills
(HOTS) and the instructional design model.
Besides, the statements of problem,
suggestions of problem solving, objectives of research, questions of research, suggested
framework of instructional design model, framework of research theory, rational of
research, importance of research, scope and limitation of research, and the definition of
terminology used in research are discussed in this chapter.
1.1
Research Background
Modern life requires people to face various experiences and environments (Tal
and Hochberg, 2003). However, for many years, contemporary education is paying
more focus on the “transmission of information” from teachers and books. In reality,
students should be equipped with higher order thinking skills (HOTS) in their learning
process (Dunlap and Grabinger, 1996a; Osborne and Wittrock, 1983; Hollingworth and
McLoughlin, 2003; Jonassen et al., 1993). The learning process requires the students to
construct their understanding meaningfully and to search for innovative solution in
problem solving.
3
1.1.1 Higher Order Thinking Skills
Higher order thinking skills (HOTS) represent multi-faceted and complex
cognitive processes that develop and improve the processing and construction of
information (Resnick, 1987; Swartz, 2001). The term HOTS used in this research refers
to the analysis, synthesis and evaluation according to Bloom’s Taxonomy of thinking
(Bloom et al., 1956). Thus, the recall of knowledge, comprehension and application are
classified as lower order thinking skills (LOTS) (Dori, Tal and Tsaushu, 2003; Bloom et
al., 1956; Morgan, 1996).
Skills such as analyzing, synthesizing and evaluating
information in the learning process are important in order to develop HOTS (Bloom et
al., 1956; Bloom, Hasting and Madaus, 1971; Ennis, 1987; Zohar, Weinberger and
Tamir, 1994; Jonassen, 1992; Tal and Hochberg, 2003; Morgan, 1996).
A lot of
researchers have pointed out the increasing importance of HOTS in the teaching and
learning process.
Problem solving requires the use of HOTS such as analysis, synthesis and
evaluation. This is in line with the argument from Kallick (2001a) that the cognitive
operations such as analyzing, inferring and evaluating are necessary in problem solving.
According to Jonassen (1992), argumentation is an appropriate outcome for problem
solving where students generate arguments and make reasoning to defend their
solutions. This encourages them in using HOTS. Besides, reflective thinking is also
often related to HOTS (Quellmalz, 1987; Vockell, and van Deusen, 1989; Fogarty and
McTighe, 1993; Wai and Hirakawa, 2001; Fogarty, 2002; Harrigan and Vincenti, 2004).
Reflective thinking helps students to be aware of their thinking as they perform tasks or
learning and this engages them in HOTS. Hence, the reflective thinking was used as an
important cognitive operation for scaffolding the encouragement of HOTS in this
research.
From the above description, it is apparent that HOTS requires students to
manipulate information and ideas in ways that transform their meaning and implication.
This occurs when students combine facts and ideas in order to analyze, synthesize, and
4
evaluate in generating knowledge. The manipulation of information and ideas through
these processes allows students to solve problem, generate knowledge and promote
understanding.
1.1.2 HOTS and Computer Science
As technology changes at an ever-increasing speed, the students must have the
ability to adapt to changes and become lifelong learners. This is especially true in the
computing field. The students have to be good in both thinking and problem solving
skills. However, most of the colleges focus more on rote lecturing, assignments and
tests (Tan Wee Chuen, Baharuddin Aris and Mohd Salleh Abu, 2005). They rarely
promote HOTS among the students in order to understand and apply problem solving
and logical reasoning skills in the learning of Computer Science (Parham, 2003; Arup,
2004).
Harrigan and Vincenti (2004) noted that HOTS are important in college teaching
and learning. A lot of studies have been conducted to study the teaching and learning
process in Computer Science domain in higher education. Empirical results from the
studies show that many students can not demonstrate HOTS in their learning (Chmura,
1998; Henderson, 1986; Arup, 2004). Most of the students resort to trial and error, and
memorizing facts from their learning, rather than learning problem solving skills.
However, these HOTS such as analysis, synthesis and evaluation thinking skills are
found important in the learning of Computer Science.
This was demonstrated by
Parham (2003) in which there is a direct correlation between the students’ HOTS and
their performance in their learning.
Hadjerrouit (1999) noted that the conventional predominant teaching model
viewed learning as the passive transmission on knowledge and this cause serious
misconception and lack of conceptual understanding in Computer Science learning.
This is further supported by Arup (2004) that the existing learning in computer system
5
tends to regurgitate what the instructors have taught and does not imply the ability to use
HOTS among the students. This is also proven by the studies from Maj, Veal, and
Charlesworth (2000); Holmboe (1999) and Mirmotahari, Holmboe and Kaasboll (2003)
that the college students are lacking of knowledge of computer technology and the basic
skills to operate computer systems.
Another main problem of Computer Science students is the lack of deep
understanding of the relationships in the facts they have learned (Scragg, Baldwin and
Koomen, 1994; Mirmotahari, Holmboe and Kaasboll, 2003). Students are better in the
practical skills than theoretical questions. In computer education, the prior knowledge
of students is the foundation for further knowledge construction (Holmboe, 1999; White,
2001; Mirmotahari, Holmboe and Kaasboll, 2003; Scragg, 1991). New information must
be linked to information already understood (Rosenberg, 1976; Hamza, Alhalabi and
Marcovitz, 2000). Learners would generate and test ideas that either have been created
from their prior knowledge.
The content of the computer has to stay abreast of the rapidly developing
computer technology. HOTS are essential to the students in this rapidly changing
technological society (Morgan, 1996). The growth of knowledge in computer needs
more timeliness in teaching resources, expertise and preparation time (Wolffe et al.,
2002). This leads to a large amount of information being drained to the students.
Instructors and students have been burdened with the task of communicating massive
and rapidly changing computer content. Consequently, the overemphasis on content has
resulted in the lack of attention on the HOTS that are necessary for the students to
successfully solve the complex scenarios (Arup, 2004).
Researchers in the education field are progressing toward the teaching and
learning with technology to develop HOTS. Studies of HOTS program from Pogrow
(1988a, 1988b), Herrington and Oliver (1999), Tay (2002) and Tal and Hochberg (2003)
showed encouraging results. Technology can be used as a mindtool for conceptual
development (Reeves, 1998; Jonassen, 2000) and to enable higher order learning (Ting,
6
2003; Reeves, 1998; Pogrow, 1988a, 1988b). In this context, the learner acts as a
designer in the learning process (Jonassen, 1994; Jonassen and Reeves, 1996). Jonassen,
Mayes and McAleese (1993) found that individual learns the most from the design of
instructional materials. Therefore, if the students were given opportunity to construct or
design their own learning, it creates an active learning environment. This process
requires the students to think more meaningfully and therefore helps to develop their
HOTS.
1.1.3 Generative Learning and HOTS
A lot of instructional strategies have been proposed to develop the HOTS. One
of the most frequent proposals is generative learning.
Generative learning is an
important constructivist learning environment (Bannan-Ritland, Dabbagh and Murphy,
2000; Dunlap and Grabinger, 1996a; Duffy and Jonassen, 1992; Morrison and Collins,
1996; Grabowski, 1996; Bonn, and Grabowski, 2001; Jonassen, Mayes and McAleese,
1993; McLoughlin, 1998).
According to Bonn, and Grabowski (2001), generative
learning provides the necessary theoretical framework for research in a constructivist
perspective.
As described by CTGV (1993), the generative learning is the first key
element of constructivism learning environment. In generative learning environment,
learning is generative; learners focus on the construction of their own learning. In this
research, the Web-based learning system used the generative learning to design the
constructive learning environment.
Originally, generative learning is conceived under the cognitive information
processing proposed by Wittrock (1974). The focus of generative learning model is that,
learner is an active participant who works to construct meaningful understanding by
generating relationships between the information. The cognitive psychologists and
educationists usually refer the skills associated with this kind of thinking activities as
HOTS. These activities are completely in contrast to those which simply copy down
information and memorize them, where the students passively receive information and
7
respond to the exercises or examinations that require only facts recalling and simple
understanding. Dunlap and Grabinger (1996b) pointed out that generative learning is a
higher-level thinking activity. HOTS depict the dynamic role of learners in which they
act as thinkers actively participate in constructing knowledge. Such a view of learning
fits well with the empirical evidence from the studies of technology and Computer
Science learning and teaching (refer Chapter 2 for further discussion).
Learning that emphasizes on the connection between the new and old concepts,
and among the concepts is important to enhance understanding (Nickerson, 1995). The
connection among the concepts is also important in learning Computer Science
(Rosenberg, 1976; Hamza, Alhalabi and Marcovitz, 2000). However, the conventional
teaching models in Computer Science often view learning as a passive transmission on
knowledge. This results in misconception, lack of conceptual understanding and the poor
understanding of the relationships in the concepts that the students learned (Hadjerrouit,
1999; Maj, Veal, and Charlesworth, 2000; Holmboe, 1999; Mirmotahari, Holmboe and
Kaasboll, 2003; Scragg, Baldwin and Koomen, 1994). In contrast, generative learning
provides a learning environment that enhances the learning through actively construct
meaningful understanding by generating relationships among the concepts.
According to Dunlap and Grabinger (1996a), content is often presented to the
learners in the format that promotes memorization rather than higher order thinking.
Most of the schools are still examination-oriented. The teaching and learning often
focus in answering specific questions in the examinations. With the generative learning,
it promotes active processing in the linkage of the concepts and supportive environment
that encourages them to think and construct knowledge from their understanding.
Higher education institute is the most appropriate venue for this learning approach
because their goals are to promote advanced knowledge acquisition and HOTS
(Jonassen, Mayes and McAleese, 1993).
Concept map provides an important tool in generative learning environment
(Osborne and Wittrock, 1983; Bannan-Ritland, Dabbagh and Murphy, 2000). Concept
8
map functions as a tool to engage learners to generate and to organize the ideas in the
content being studied. According to Jonassen (2000), concept map engages learners in
the reorganization of knowledge, explicit description of concept and their
interrelationships, deep processing of knowledge that promotes better remembering,
retrieval and application of knowledge; and relating new concepts to existing ones that
improves understanding. This is consistent with the theoretical perspective of generative
learning.
The concept map used in this research is called as lesson map. It is an outline
form of concept map as suggested by Alpert and Grueneberg (2000), and Dabbagh
(2001). The lesson map used in this research enables the construction of concepts
require HOTS when students organize the lesson map, select important and relevant
concepts to add to the map, search the crosslink and indicate the relationships between
concepts. These activities engage students in HOTS that are analysis thinking while they
are organizing the concepts in hierarchical structure, synthesis thinking while they are
searching crosslink and indicating the relationships between the concepts and evaluating
while they are searching and judging the important and relevant of the concepts
(Jonassen, 2000; Dabbagh, 2001; Alpert and Grueneberg, 2000).
Jonassen, Mayes and McAleese (1993) pointed out that the generative learning
results deeper levels of knowledge processing and construction, and these necessitate the
HOTS. They further pointed out that constructivist learning environments aim to engage
students in higher order and meaningful learning.
Besides that, Jonassen (1992),
Jonassen, Mayes and McAleese (1993) noted that the outcomes of constructivist
learning environments should assess HOTS in order to reflect the intellectual processes
of knowledge construction.
Studies show that generative learning and teaching
provoked learners’ thinking skills and developed their understanding (Laney, 1990;
Schaverien, 2000; McLoughlin, 1998; Dunlap and Grabinger, 1996a, 1996b; McGriff,
2002; Osborne and Wittrock, 1983; McLoughlin, 1998). The process of generative
learning engages student in HOTS (Grabowski, 1995; Grabowski, 1996; McLoughlin,
1998). From this perspective, the generative learning is strongly related to the HOTS.
9
Gao and Lehman (2003) noted that most of the researches in generative learning
emphasize in the facts and concepts-level learning and deal little with HOTS. Therefore,
further research on generative learning that focuses on HOTS is necessary.
1.1.4 Learning Object Design
Nowadays, most of the instructional designers understood the importance of
pedagogical perspectives in the design and development of Web learning environments.
According to Snow (1989), instructions differ in structure and completeness and highly
structured instructions (linear sequence, restricted and high external control) seem to
help those with low ability but hinder those with high ability. The concept of one-sizefits-all design is no longer suitable in the design and development in e-learning. The
learning environment should be highly flexible in structures and hands control out of the
hands of the systems or instructors to the learners. Therefore, the concept of learning
object design fits this very well as it provides flexible paths to the users’ exploration in
the teaching and learning process. The non-linearity of the learning object design allows
students to access information in different patterns and to take control in their own
action and learning.
A learning object is a small, reusable digital component that can be selectively
applied alone or in combination by computer software, learning facilitators or learners
themselves, to meet individual needs for learning or performance support (Shepherd,
2000). There are three interdependent components in the learning object design model:
the learning object itself; metatagging (a standardized way to describe the content in
code); a Learning Content Management System (LCMS) that stores, tracks, and delivers
content.
Learning object design is the design of instructions into small learning contents
that can be reused in different context and combined to form learning that are
appropriate to the individual (Wiley, 2000; Hodgins, 2001; Wagner, 2002; Mills, 2002;
10
Longmire, 2000a, 2000b; Robbins, 2002; Lau, 2002; Gibbons, Nelson and Richards,
2000; Hanaffin, Hill and McCarthy, 2000; South and Monson, 2000; Collis and Strijker,
2003). The design of instructions into learning objects can be deployed into multiple
setting and learning goals. It is a current trend of computer-based instructions and
learning that are grounded in the object-oriented paradigm of Computer Science.
The idea of information in small chunks which are reusable and flexible in a
learning environment has received a lot of compliments from the educators and
instructional designers of e-learning environment. According to Reigeluth and Nelson
(1997), when teachers first gain access to instructional materials, they often break the
materials down into their constituting parts and then reassemble these parts in ways that
support their instructional goals. Thus, the notion of small and reusable units of learning
content, learning components, and learning object design have the potential to provide
the flexibility and reusability by simplifying the assembly and disassembly of
instructional design and development.
E-learning industry has anticipated the day where learners could personalize,
assemble, and access e-learning on demand for years (Mortimer, 2002). Most electronic
learning content is currently developed for specific purposes. How does a learner select
only a small part of content that suit their learning needs? The educational software
development is an extremely expensive process in terms of cost and time (Wiley, 2000;
Downes, 2000; Longmire, 2000a). With the learning object design, the learning objects
can be reused and shared. Molenda and Sullivan (2002) noted that there is a critical
need for more efficient design and production of the digital learning materials. Thus,
learning object design had become more practical now especially with the essential
features of the World Wide Web.
11
1.1.5 Relationships between Generative Learning, HOTS, Learning Object Design
and Web Technology
The current design and development of learning objects has overlooked the use
of learning objects in supporting learning (Bannan-Ritland, Dabbagh and Murphy, 2000;
Shi et al., 2004). Most of the discussions and researches in learning objects concentrate
on the standards, metadata and others technical issues related to the development of the
learning object system design. The evidence of the dynamic attributes of learning object
design in learning is still not well addressed (Shi et al., 2004). The unique attributes of
the learning objects lies in providing a customized, individualized and flexible learning
environment. The required approach can be grounded in constructivist principles of
learner centered, learner-controlled and learner-constructed learning. Thus, there is a
need for research and development works to study the pedagogical issues of the learning
object.
According to Wagner (2002), the development of learning objects involves a
significant shift from behavioral to cognitive perspectives and from objectivist to
constructivist perspectives. One of the principles of constructivism is that learners are
active participants in the learning process (Jonassen, 1994; Reeves, 1998; Friesen, 2001;
Bannan-Ritland, Dabbagh and Murphy, 2000). In addition, Collis and Strijker (2003)
mentioned that the learning object design makes a pedagogical shift from the emphasis
on learning as acquisition of predetermined content, towards the emphasis of learning as
participating and contributing to the learning experience. Therefore, learners construct
their own understanding from experiencing objects, activities and processes by
exploring, analyzing, synthesizing and evaluating knowledge in self-directed or
collaborative fashions rather than in a predetermined structure. These processes involve
learners in HOTS.
From the theoretical perspective of the generative learning, learning object
design can be configured as generative learning environments (Bannan-Ritland,
Dabbagh and Murphy, 2000). The attributes and nature of learning object design match
12
well with a generative learning. Learning object design offers the flexible, reusable and
generative learning environment by allowing learners to participate more actively in the
construction of knowledge and understanding. Learners are able to generate the
relationships between the learning objects that are flexible and reusable, and this
engages them in HOTS.
Toh (2004) indicated that the learning object design has the potential to deal with
the expanding growth of knowledge and skills. The attributes of the learning object that
allow learner-centered, generative-oriented activities have not yet been fully explored
and may reveal significant implications for the development of the educational
technology. As the amount of information about the computer system is growing
parallelly with the fast changing technology, learning object design can help to reduce
the cost and time of the e-learning system development where it allows the reusable
content between the courses that teaching in the same concept.
It is apparent that the learning object design with generative learning
environment engages students in HOTS. This learning environment encourages and
requires students to manipulate the content which is designed as small chunks of
learning object. The HOTS occur when students analyze, synthesize, and evaluate to
design their learning by connecting and generating the relationships between the
learning objects with the use of concept mapping. This enables students to generate, to
evaluate their ideas and to construct their learning actively.
The Web provides an excellent environment for generative learning, especially
with the use of learning object design. The advent of the WWW technology tools and
features, and the growing of learner –centered instruction have provoked the Web-based
learning (Bonk and Reynolds, 1997).
Web-based learning environment is able to
support student-centered learning and learning by doing (Lim, 2000; Jonassen and
Reeves, 1996). The Web-based learning designed with appropriate instructional
theoretical models can act as mindtool to promote HOTS (Jonassen and Reeves, 1996;
Reeves, 1997; Bonk and Reynolds, 1997). The Web-based learning design that based on
13
generative learning can provoke learners’ thinking skills and developed their
understanding (Schaverian and Cosgrove, 2000; Shepherd, Clendinning and Schaverian,
2002). The dynamic attributes of learning object design support the reuse of resources
on the Web (Mohan and Brooks, 2003). In addition, the use of hypermedia that allows
extensive links between learning objects supports learning (Dodds and Fletcher, 2003;
Zhu, 1999). Hence, Web-based technologies are able to support the use of learning
object design in learning. These reveal the great potential of the development of Webbased learning objects that incorporates with the generative learning to improve HOTS
and learning.
1.1.6
Instructional Design Model
Instructional design (ID) theories are very important for the development of high
quality instructional program that meets the users’ needs.
According to Reigeluth
(1996), instructional design is concerned with differentiating the methods of instruction
that are suitable for different situations. ID plays an important role in the application of
learning object design if it is to succeed (Wiley, 2000). The ID model of the design and
development of this research is modified from the ISDMELO (Instructional System
Design Methodology based on e-Learning Object) which is based on ADDIE model
(Baruque and Melo, 2003).
The ISDMELO methodology which is built on the
fundamental of learning object-based instructional design has been developed for the
design and development of Web-based educational content.
From the research background outlined above, it is thus necessary to concretize a
conceptual framework by designing and developing suitable learning models for
computer-based learning environments, which ultimately lead to effective learning.
14
1.2
Problem Statements
In the information age, HOTS are important to facilitate people to cope with
rapidly changing world. Learning to think is necessary in promoting life-long learning.
The education system should nurture the productive growth by paying more emphasis on
teaching for HOTS (Onosko, 1990). In addition, Morrison and Lowther (2001) pointed
out that the school can no longer focus on a body of knowledge that a student needs to
master. The emphasis of helping students to master in content should be shifted to a
focus on thinking. When students develop their HOTS, they are more equipped to
control their learning and to develop deep understanding of the content. Students need
to have the ability to think so that they can learn instead of pure memorization of facts.
HOTS are emphasized by Resnick (1987) as a different learning process as compared to
rote learning and information withdrawal. This is of utmost important in view of the
massive growth of knowledge in the ICT world.
Researchers in Computer Science education have noted that the predominant
model of instruction that views learning the passive transmission has caused the lack of
conceptual understanding in Computer Science (Arup, 2004; Scragg, Baldwin and
Koomen, 1994; Tan Wee Chuen, Baharuddin Aris, and Mohd Salleh Abu, 2005). Some
of the researchers even demonstrated that the problems are due to the inability of HOTS
(Parham, 2003; Arup, 2004; Tan Wee Chuen, Baharuddin Aris, and Mohd Salleh Abu,
2005). Details about the problems of the learning of Computer System will be discussed
in chapter 2.
As the learning object design is new in the instructional design, it is challenging
to design and develop a Web-based learning environment that is based on this design. It
is difficult to implement the learning object design in the traditional learning
environment. The inherent strength of World Wide Web technology is the distribution
and sharing of information in hyper-space. However, most of the Web-based content
materials nowadays are actually similar to the approach of linear mode delivery of the
learning materials found in traditional lecture presentations. This conventional “one-
15
size-fits-all” learning environment is no longer suitable and satisfactory for the needs of
the learners. Converting these to a digital deliverables through the Internet would not
make any change to these passive learning materials and does not promise in fostering
understanding as well as HOTS. Furthermore, Beaver and Moore (2004) noted that there
is a wide range of educational software but most of them are not designed to encourage
HOTS.
Numerous studies have documented the effectiveness of the incorporation of drill
and practice computer programs into teaching and learning. However, Morgan (1996)
highlighted that many drill and practice computer programs engage students only at
lower levels of Bloom’s Taxonomy (Knowledge, Comprehension, Application). Morgan
(1996) also pointed out that the use of technology in education must ensure that the
technology is being used to engage students to HOTS. Thus, the shift of teaching and
learning now is not to be a process of regurgitating and reproducing information but a
process of constructing knowledge and learning environment that involves learners in
HOTS.
It is essential to understand that the design of e-learning is a design for
promoting HOTS and not a design for teaching or delivering information.
As reviewed in literature study, limited research has been done on the learning
object design and it’s effectiveness in learning. Current research and development of the
learning object design are primarily focusing on establishment of technical issues
(Bannan-Ritland, Dabbagh and Murphy, 2000; Tan Wee Chuen, Baharuddin Aris and
Mohd Salleh Abu, 2004). There is little research on the pedagogical based learning
objects in the design of Web-based learning, especially in Malaysia. To improve
learning, the learning object design and the generative learning in instructional design
based on ISDMELO was adapted as the elements of design and development of the
Web-based learning system in this research.
The prototype of Web-based learning system focused on one of the subjects
offered in Diploma of Computer Science, which is selected in conjunction with the
16
implementation of the subject in the first year of Diploma in Computer Science. This
research aims to design and develop a prototype of Web-based learning system in
Computer Science that incorporates the learning object design and generative learning to
improve the HOTS as well as the understanding of the students in their learning.
1.3
Research Rationale
As citizens in the information age, students need to have strong problem solving
skill and thinking skill (Morgan, 1996). Hence, experiences that encourage and improve
students in HOTS should become a common practice in education. This is important as
the development of information technology has become ubiquitous in schools and
colleges. The Malaysian Education Ministry has taken appropriate steps to ensure the
students to be good thinkers. The curriculum design has focused in the development of
HOTS.
The Malaysian Education Ministry has introduced Information Technology (IT)
(Teknologi Maklumat) from the Form 1 to Form 6 in the secondary schools. One of the
purposes is to develop HOTS such as analysis, synthesis and evaluation. Besides, it also
aims to promote the problem solving skill that involves thinking skills as mentioned
earlier (Zanariah Abdullah and Rosmayuzie Ab. Satar, 2001). This reveals the effort of
the Malaysian Education Ministry in promoting HOTS through the teaching and learning
of IT. It also shows the importance of HOTS in learning IT.
The attention to thinking skills is explicit with the extensive research in this field.
Peck and Dorricot (1994) noted that the students must be able to access, evaluate,
communicate information and solve complex problems. According to Kallick (2001b),
when the computer is used with full potential, it is able to enhance thinking skills and
create new knowledge. In this context, technology can be harnessed to support and
encourage the students learning and HOTS (Morgan, 1996; Peck and Dorricot, 1994).
17
A lot of learning problems and issues have been identified and discussed from
the research conducted in Computer Science domains (see Scragg, Baldwin and
Koomen, 1994; Miron, O’Sullivan, and McLoughlin, 2000; Parham, 2003; Henderson,
1986; Maj, Veal and Charlesworth, 2000; Yurcik and Osborne, 2001; Holmboe, 1999;
Magagnosc, 1994; Yehezkel et al., 2002; Skrien, 2001; Ivanov, 2003; Makkonen, 1997;
Mirmotahari, Holmboe and Kaasboll, 2003). The learning problems are mainly due to
the inability of students in HOTS and the lack of inter-relatedness among the concepts
they have learned. Empirical evidence obtained by Parham (2003) demonstrated that the
inability of students’ HOTS will affect their performance in Computer Science. Details
of the learning problems were discussed in 2.6.
The students’ problem solving skills are essential in computer subject such as
Computer System, in which they are required to analyze, synthesize and evaluate the
complex scenarios. All these activities involve HOTS. Timely resources in the content
of Computer System are needed due to the high pace of computer technology
development. Students have been burdened with the task of communicating a large
amount of the fast changing content. This has brought to the overemphasis on the
content and has resulted in the lack of emphasis on the HOTS that is necessary for
students to successfully deal with complex scenarios (Arup, 2004).
The literature and research findings clearly show the need to promote HOTS
among the Computer Science students. In this research, Computer System subject has
been chosen as a topic of study for the effectiveness of the Web-based learning system
based on the result found in the preliminary study.
This subject is taken by the
Computer Science students as fundamental knowledge of computer technology.
The issues of the flexibility and pedagogical perspective in the development of elearning have brought to the concept of learning object design in the development of
educational software.
The traditional courseware that comprises the instructional
content and a navigation scheme to move around the content no longer meets the
expanding growth of knowledge (Toh, 2004). The learning object design features are
18
engaged with the design and development of a more flexible and generative learning
environment. The delivery of the learning materials in the form of chunks of lesson,
organization and customization of the materials based on the learning objectives can
now be realized with the use of learning object design.
However, the current
development on the learning object design in e-learning tends to overlook the use of
learning object design in supporting learning (Bannan-Ritland, Dabbagh and Murphy,
2000; Shi et al., 2004). The pedagogical perspective in the design and development of
learning object has been left behind and put in a less important place compared to the
standard, metadata and technical issues.
Van Zele et al. (2003) pointed out that very little is known about the educational
pitfalls or benefits of the learning object design, and the reports on its implementation
and evaluation in higher education are lacking. In addition, Agostinho et al. (2003)
noted that there is little research on how learning object design should be incorporated
into constructivist and learner-centered approaches to learning.
At present, the
discussion of learning object design is commonly related to the concerns content, its
values and management (Tan Wee Chuen, Baharuddin Aris and Mohd Salleh Abu,
2004). Currently most of the studies conducted in the use of learning object design
emphasize more on the technical issues and the design for the use of instructors and
trainers. Besides, the learning object design is still focusing on the potential in gaining
profit and incentive from the e-learning industry, leaving behind the emphasis on the
impact of the learning object design to learning. Thus, the pedagogical intent in learning
design has to be addressed as the important issue in supporting and enhancing the
learning process (Ramsay et al., 2004; Bannan-Ritland, Dabbagh and Murphy, 2000; Shi
et al., 2004; Toh, 2004; Bradley and Boyle, 2004; Agostinho et al., 2003).
As reviewed earlier, it is important to conceptualize and design the Web learning
based on pedagogical perspectives. However, most of the educational software tends to
emphasize the sophisticated multimedia display (Cohen, 1983; Campoy, 1992; Koper,
1998). According to Jonassen (1991), instructional designers should focus more on the
thinking technologies rather than developing a sophisticated multimedia delivery
19
technology. Mere multimedia does not turn students into active participants during the
lectures (Van Zele et al., 2003). The learning system should be designed towards a
more student-driven and student-oriented interactive learning. Merely providing predetermined structure of content will not aid significantly in learning. The one-size-fitsall traditional courseware no longer meets the requirement of personal knowledge
construction. Learning object design and generative learning provide the environment
that allows students to construct their own learning. This learning environment enables
the students to be active participant in their learning and most importantly, engages them
in HOTS.
Currently, common Web-based learning systems are more to enrich access to
course materials, search course materials, post project or assignment, provide tutorials
and learning support, and enable the Web discussion. There is lack of integration of Web
technologies into actual teaching and learning (Reeve, 1996). The promise of the Web
technologies must be accompanied with pedagogical perspective (McLoughlin, 1998).
The use of communication technologies to support learners’ centered learning is well
documented in the literature and research (eg. McLoughlin, 1998; Reeve, 1996).
A lot of literatures highlight the need for learner-controlled learning in the design
of technological learning environment (eg. McLoughlin, 1998; Jonassen and Reeves,
1996). Web technology is conceived as enabling the students-centered learning. Webbased technologies are able and suitable to support the use of learning object design in
learning (Hawryszkiewyez, 2002). The interactivity and flexibility of the Web enable the
design of a Web-based learning tool that leverages the learning object design and
generative learning. It provides an environment that enables students to explore and
manipulate the learning objects so that students can continuously reconfigure to
construct their knowledge.
For the reasons discussed above, this research focuses on the development and
design of a Web-based learning system, using the learning objects in the design
approach of learning content and generative learning in the design of learning strategy to
20
assist the learning in Computer System as well as to improve the HOTS. A conceptual
model is suggested, namely Generative Learning Object Organizer and Thinking Tasks
(GLOOTT).
This model incorporates the three important components, namely the
learning object design, generative learning and HOTS in a technologically-supported
learning environment. The model aims to facilitate the students to engage themselves in
HOTS as well as to promote understanding in the Computer System.
A comprehensive study was conducted in this research to evaluate the
effectiveness of GLOOTT model in improving learning and HOTS. Besides, the
researcher hopes that this study will contribute to the framework of instructional design
and development based on the learning object design and generative learning in the
Web-based learning environment to improve the students’ HOTS.
1.4 Research Objectives
This research aims to achieve two main objectives:
(i)
To design and develop a prototype of Web-based learning system that based
on the learning object design and generative learning.
(ii)
To evaluate the Web-based learning system in the aspects of:
(a) The improvement of learning through test.
(b) The improvement of HOTS based on Bloom’s taxonomy.
(c) The engagement of HOTS.
(d) The effectiveness of the Web-based learning system as perceived by
the instructors and students.
1.5
Research Questions
Based on the research objectives discussed earlier, the research is carried out to
answer the following questions:
21
(i)
What levels of HOTS are exhibited by the Computer Science students after the
conventional teaching and learning of Computer System in the first year of
Diploma of Computer Science course?
(ii)
Is there any significant difference between the students’ score in the test before
and after the use of the Web-based learning system?
(iii) Is there any significant difference between the students’ HOTS before and after
the use of the Web-based learning system?
(iv) How do the students’ HOTS engagement change when they use the Web-based
learning system?
(v) How effective is the Web-based learning system as perceived by the
instructors?
(vi) How effective is the Web-based learning system as perceived by the students?
1.6
Research Theoretical Framework
The theoretical framework of this research incorporates a few important
components from different perspectives. Learning object design was adapted for the
instructional design structure, while the pedagogical perspective, the generative learning
and HOTS were incorporated into the design of the Web-based learning system. The
Web was used as a delivery medium for the system. All these aspects had been studied
in detailed in order to meet the objectives of this research.
The Web-based learning environment in the system design is based on the
generative learning from constructivism learning from Bonn and Grabowski (2001),
Grabowski
(1996), Bannan-Ritland, Dabbagh and Murphy (2000), Dunlap and
Grabinger (1996a, 1996b), Duffy and Jonassen (1992), Morrison and Collins (1996), and
Wittrock (1974; 1991; 1986). The features of the generative learning include:
(i)
Provide a learning environment that enables the active process of
knowledge construction.
22
(ii)
Provide a learning environment that supports the construction of
knowledge.
(iii)
Learners are active participants in constructing their knowledge.
(iv)
Design a learning environment that emphasizes on the construction of
knowledge and allows learners to interpret their learning and build the
mental model to represent the knowledge.
(v)
Provide a learning environment that requires students to participate actively
in the learning process and construct the knowledge meaningfully rather
than in a predetermined structure.
(vi)
Provide a generative learning environment that enables learners to
construct and design their own learning.
(vii)
Design learning activities that engage learners in HOTS.
(viii)
Design the learning environment that allows students to generate
organizational relationships between different components of the
knowledge through learning aids such as concept mapping that engages
students in HOTS. The generative learning environment also includes the
activities for knowledge integration and elaboration such as problem
solving.
(ix)
Design activities to encourage students to actively participate in
constructing meaningful understanding by generating relationships among
the information received and apply it to support problem solving.
From the descriptions above, it is apparent that generative learning requires
students to construct their learning in a meaningful way and this will engage them in
HOTS.
The generative learning environment encourages students to analyze,
synthesize, and evaluate facts and ideas in the process of knowledge generation. Such
learning environment engages students in HOTS.
The cognitive operations of HOTS emphasized in the system are analysis,
synthesis and evaluation. These cognitive operations are based on the works from the
taxonomy of Bloom et al. (1956), Bloom, Hasting and Madaus (1971), Tal and
23
Hochberg (2003), Parham (2003) and Swatrz (2001) with consideration on the
curriculum of the Computer Science course for college students.
According to Dede (1990), learning environment that contains a highly
developed information-gathering tool to stimulate the learners to actively construct
knowledge will engage students in HOTS. Besides, the organization of information into
an integrated system to show relationships among the information through concept
mapping will encourage and assist students in HOTS (Ivie, 1998; Hollingworth and
McLoughlin, 2002; Hobgood, 2002).
In addition to the HOTS activities mentioned above, reflection and thinking tasks
were integrated into the system to provide a more comprehensive generative learning
environment. Reflection was designed to scaffold the students so that they are conscious
in applying the HOTS and aware of their learning. Quellmalz (1987), Fogarty (2002)
and Harrigan and Vincenti (2004) pointed out that reflection will foster HOTS because it
enables students to reflect on their learning. In addition, finding from Harrigan and
Vincenti (2004) demonstrated the reflection engages students in HOTS.
According to Costa and Kallick (2001), thinking tasks such as problem solving,
scenario generation and exercise are important to engage students in HOTS. In this
research, the thinking tasks are scenario-based problem solving and multiple-choice
question exercise to reinforce the students’ HOTS as well as to test their understanding.
This aligns with the generative learning that advocates the inclusion of concept mapping
and scenarios-based problem solving as generative learning activities.
The strategy of learning environment in the system that based on the generative
learning and HOTS aligns with the features of learning object design. According to Ip
and Morrison (2001), learning object has the potential to be integrated into different
learning paradigms. This is further elaborated by Bannan-Ritland, Dabbagh and Murphy
(2000). They pointed out that the premise underlying the features of a learning object
that support flexibility and reusability is aligned and heavily related to generative
24
learning from constructivism learning. This is further supported by Agostinho et al.
(2003) that research should be conducted about the incorporation of the learning object
design with the constructivism learning environment.
Figure 1.1 depicts the theoretical framework about the incorporation of the
generative learning, learning object design and HOTS in the research. Based on this, a
conceptual model of Web-based learning system, GLOOTT is proposed and applied in
the
design
of
the
learning
environment
in
the
system
development.
25
LO Design
Reflection
Generative Learning
Object Organizer (GLOO)
Analysis
Synthesis
Evaluation
Thinking Tasks (TT)
TT
Try it Out
Apply It
Reflection
Generative
Learning
GLOOTT Model
HOTS (Bloom
et al., 1956)
(Grabowski, 1996)
Figure 1.1: Conceptual Model of the System Design and Development
26
Jonassen and Reeves (1996) stated that technology can be used as a mindtool that
enables learners to act as designers to design and construct their learning, rather than as
passive recipients in the learning process. Computer and information technology can
stimulate students to become active learners and provide tools to manipulate their
learning (Morgan, 1996).
The GLOOTT model incorporates three important
components, namely the learning object design, the generative learning based on
Grabowski (1996) and HOTS based on Bloom Taxonomy of thinking (Bloom et al.,
1956) in technology-supported learning environment. The Web-based environment is a
promising technology that enables the designers to create flexible and powerful learning
systems that support the design of GLOOTT model.
The GLOOTT model provides a pedagogically-enriched learning environment to
engage students in HOTS and to promote their understanding. The GLOOTT model
consists of Generative Learning Object Organizer (GLOO) and Thinking Task (TT) as
depicted in Figure 1.1. TT consists of Try it Out that contains multiple-choice questions
and Apply it that contains scenario-based problems. Details about the design and
development of Web learning system would be discussed in Chapter 4.
To the best of the researcher’s literature study, a learning object design system
that is based on theoretical learning approaches which pervades in constructivism and
focuses on learner-centered learning and HOTS has not yet been developed. Most of the
learning object design systems focus mainly in the designs of teaching materials for
trainer and instructor. Besides, minimal research has been done to demonstrate the
effects of learning object design on learning (Bannan-Ritland, Dabbagh and Murphy,
2000), and the researcher has not found substantial research showing the effects of the
learning object design with pedagogical design on academic achievement and HOTS.
The suggested conceptual model (GLOOTT model) incorporates the theoretical,
pedagogical and technological perspectives from generative learning, learning object
and essential cognitive operations of HOTS in the Web-based learning environment.
27
1.7
The Framework of Instructional Design Model
The instructional design (ID) model used in the research is modified from the
ISDMELO (Instructional System Design Methodology based on e-Learning Object)
(Baruque and Melo, 2003).
Figure 1.2 illustrates the proposed framework of the
instructional design model used in this research.
The ID model of the research incorporates learning object design principles,
generative learning design principles, and Web-based learning design principles in order
to promote understanding and improve HOTS of the students in the learning process.
The prototype of the Web-based learning system is designed, developed and evaluated to
determine its effectiveness in a college. The modified ISDMELO model is used because
it was developed for the design and development of Web-based educational content that
was built on the fundamental of instruction design based on learning object.
The
ISDMELO model is modified from the ADDIE model (Molenda, Pershing and
Reigeluth, 1996) that provides systematically instruction plan.
In addition, the
ISDMELO is grounded on pedagogical principles and supports the adoption of learning
theories such as constructivism, cognitive and behaviourism.
phase of the ISDMELO were discussed in Chapter 3.
The details about each
28
Phase I: Analysis
a) Subject selection
b) Problem Analysis
c) Content and Task Analysis.
Phase II: Design
a) Learning activities design
b) Learning objects
c) Data Flow Diagram (DFD) and Storyboards Design
Phase III: Develop
Develop the Web-based learning system, digital learning
objects, repository, learning objects organizer, thinking
tasks, reflection corner and information agent that acts as
pedagogical assistant.
Phase IV: Implementation
a) Implementation of the Web-based learning system.
No
Is the prototype
stable?
Yes
Phase V: Evaluation
a) Effectiveness evaluation of the Web-based learning
system based on research questions.
Figure 1.2: The Framework of ID Model
(Modified from ISDMELO, Baruque and Melo, 2003)
29
1.8
Research Importance
This research proposes a theoretical framework of Web-based learning system
that provides the instructor with the flexible and reusable learning content in Computer
System (CS). The Web-based learning system helps the instructor to identify students’
engagement of HOTS and their understanding. The findings will help the instructors,
especially Computer Science instructors in planning the teaching and learning of CS
using the Web.
This research also proposes a unique framework of instructional design model
that provides an alternative of instructional design based on the learning object design
and generative learning.
The findings from this research would demonstrate the
effectiveness of technology in improving students’ learning and HOTS.
In addition,
it also contributes in the design and development of the Web-based learning especially
with the use of learning objects. Results from this study are important to show the
effectiveness of the learning objects in learning as most of the current researches in
learning object design mainly focus in technical and standard issues. Besides, the
findings from this study would contribute to the existing evaluations of Web-based
learning and learning object design. The results would also suggest that an empowered
learning can be achieved by putting more emphasis on the pedagogical design learning
environment rather than the technological aspects in order to develop a system that is
able to encourage students to learn actively and improve their HOTS.
The proposed system conceptual model from the research theoretical framework,
namely GLOOTT model, the Web-based instructional design framework, research
methodology and findings may be used as guide or reference besides provoking ideas for
other researchers who are interested in learning object design, generative learning,
HOTS and Web-based learning. On the other hand, it also can be used as a guide in
helping the higher education institutes, educational technology and other relevant parties
in the design and development of e-learning system to engage students in HOTS.
30
1.9
Research Scope and Limitation
This research aims to design and develop a Web-based learning system that
incorporates the generative learning strategies and learning object design to provide a
learning environment that engage students in HOTS.
The cognitive operations of
HOTS in this research are Analysis, Synthesis and Evaluation.
The features and
rationales of generative learning, learning object design and HOTS have been discussed
previously in the rationale and theoretical framework of study.
The activities of generative learning used in this research are concept mapping
and scenarios-based problems solving. These generative learning activities are well
documented in the literature as discussed in the research background and research
rationale. The use of learning object design in teaching and learning has received
increasing attention in the recent years. The main advantages of the learning objects are
flexibility and reusability. However, it is a widely belief that the learning object design
does not add significant value to the learning if there is an overemphasis on the technical
aspect rather than supporting learning. In this research, the learning object design was
focused on how its application to support learning. The HOTS are widely discussed in
the literature and research. The Bloom’s Taxonomy of Thinking is used to identify the
students’ HOTS in this research because it is well documented in the literature and
research in determining the level of HOTS.
This research focuses on learning of Computer System for the college students
from Computer Science Department in Southern College. The subtopics of the subject
studied in this research were Introduction to Computer System, System Unit, Input,
Output and Storage. The Web-based learning system designed in this research is a
learning tool that can be used for other subjects. However, in studying its effectiveness
in learning Computer System, the research was limited to Computer Science students in
a college. The content of the learning had been validated by the lecturers who had
taught the subject. The effectiveness of the system was studied from the aspects of
engagement of HOTS, improvement of HOTS and learning of the students.
This
31
research did not consider the interest and learning styles of the students that could affect
their performance and learning. Moreover, the findings of this research should not be
generalized to other students. It is important to point out that the main purpose of this
research is to conceptualize and to design Web-based learning objects based on the
pedagogical perspectives. The emphasis of this research is to study the effectiveness of
the proposed design in learning rather than the technical issues relating to the learning
object design.
1.10
Operational Definition
The following are definitions of some terminologies used in this dissertation for
clearer comprehension of the issues in this research.
(i)
Learning Object
A learning object is an object or set of resources that can be used to
facilitate the learning of certain subject (Mills, 2002). It is flexible and
reusable. It is stored and accessed using meta-data attributes. A learning
object is a self-contained, reusable chunk of instruction that can be
assembled with other objects.
A learning object can teach facts,
concepts, principles, procedures and processes.
(ii)
Granularity
The meaning of granularity in this research is the size (content) of the
learning objects (Wiley, 2002a; Wiley, 2002b).
It is the amount of
information and content to be included into a learning object.
(iii)
HOTS
HOTS is the abbreviation of Higher Order Thinking Skills. The cognitive
operations of HOTS in this research are Analysis, Synthesis and
Evaluation (see Johnson, 1999; Jonassen, 1992; Parham, 2003; Swatrz,
2001; Marzano, et al. 1988; Bloom et al., 1956; Bloom, Hasting and
32
Madaus, 1971) with the consideration of the curriculum in learning
Computer System.
(iv)
Bloom’s Taxonomy of Thinking
Table 1 describes the features of the Bloom Taxonomy of Thinking used
in this research (Bloom et al., 1956).
Table 1.1: Bloom Taxonomy of Thinking (from Bloom, et al., 1956; Bloom, Hasting
and Madaus, 1971)
Bloom
Taxonomy of Features
Thinking
Knowledge
Knowledge is defined as the remembering of previously learned
material. This involves the recall of specific elements in a subject
matter.
Knowledge represents the lowest level of learning
outcomes.
Comprehension Comprehension is the ability to grasp the meaning of material. It is
described in three different operations: translating material from
one form to another, interpreting material and estimating future
trends.
These learning outcomes represent the lowest level of
understanding.
Application
Application is the ability to use learned material to new problems
and situations. For examples, the application of rules, methods,
principles and theories. The learning outcomes represent the higher
level of understanding than knowledge and comprehension.
Analysis
Analysis is the ability to break down material into its constituent
parts into the relative hierarchy of ideas with the relations between
the ideas.
This includes the identification of parts and the
hierarchical organization, and analysis of the relationships between
the parts.
Learning outcomes are higher than knowledge,
comprehension and application.
element in HOTS.
Analysis is recognized as an
33
Synthesis
Synthesis is the ability to put parts together to form a whole. This
involves the process of arranging, combining and working with
parts them in such a way as to constitute a new pattern or structure.
The learning outcomes emphasize on the formation of new patterns
or structures and creative behavior. Synthesis is recognized as an
element in HOTS.
Evaluation
Evaluation is defined as the ability to judge the values of materials
for some purposes or solutions.
The judgments are based on
definite criteria either those determined by the students or those
given to them. The learning outcomes are at the highest cognitive
hierarchy. Evaluation is recognized as a cognitive operation in
HOTS.
(v)
Learning improvement
In this study, the improvement of learning is defined as the improvement
of the score in the test that was designed based on the learning goals of
Computer System.
(vi)
Effectiveness
In this study, the evaluation of the Web-based system effectiveness is
focused on the improvement of students’ learning and HOTS before and
after the use of the system through the one group pretest and posttest.
(vii)
Generative Learning
Constructivist design provides learning environment that enables students
to synthesize, analyze and evaluate as well as to create and contribute
resources (McLoughlin, 1998).
Generative learning is a type of
instruction developed by constructivists that is widely documented. The
generative learning activities involve the creation of relationships and
meanings of the learning. In the generative learning, students are active
in the knowledge construction. Experts and researchers advocate that
concept mapping and problem solving are activities of generative
learning. Concept mapping and problem solving will engage students in
34
analysis, synthesis, and evaluation skills. Thus, it is important to integrate
these skills into learning in order to promote HOTS. In the generative
learning environment, students are active in constructing meaningful
understanding of information found and generating relationships among
the information.
(viii) GLOOTT Model
GLOOTT refers to Generative Learning Object and Thinking Tasks. It is
a pedagogically-enriched conceptual model that was designed based on
learning object, generative learning and HOTS.
(ix)
GOOD Learning System
GOOD learning system refers to Generative Object-Oriented Design
Learning System. It is the Web-based learning system designed based on
the system conceptual model, namely GLOOTT Model in this research.
(x)
Computer System (CS)
CS is a core subject of the first year Diploma in Computer Science course
in Southern College.
(xi)
Lesson Mapping
Lesson mapping is the mapping of concepts in the learning of CS based
on the design of concept mapping. It is the generative learning activity
designed in the Web-based learning system that aims to engage students
in HOTS. It is an outline form of concept map suggested by Alpert and
Grueneberg (2000), and Dabbagh (2001).
(xii)
Electronic Portfolio
Electronic portfolio is the portfolio that is saved in electronic format
(Lankes, 1995). The electronic portfolio used in this research contains
only the record of “How am I doing” checklist list in the Web-based
system. The checklist is used to record the students’ engagement of
HOTS when they use the Web-based learning system.
(xiii) Learning Object Design
Learning object design is an application of object-oriented thinking to the
world of learning. It is a term used to describe the design of leaning into
35
flexible pieces of learning content that could be assembled and
reassembled as needed. Learning objects are small reusable components
such as video, tutorials, procedures, stories, animations, simulations and
so on.
1.11
Summary
This chapter presents an overview of the background and rationale for this
research. Chapter 2 will present a detailed analysis of the literature relevant to this
research, which is a key part of the theoretical framework and the framework of
instructional design model used in this research.
Chapter 2 will also present the
instructional design and the learning object design, generative learning, HOTS, the
learning of computer and literature that pertinent to this research.
CHAPTER TWO
LITERATURE REVIEW
2.0
Introduction
This chapter presents an analysis of the literature regarding the use of technology
as the focus for an information age instructional environment results in the identification
of five broad topics. First, a discussion describes HOTS and the adoption for the design
and development of the Web-base learning. Second, the generative learning will be
reviewed in order to provide context for the overview of current instructional design
theories with learning object design. Third, the relatively new field of learning objects
design will be discussed. Fourth, a discussion about the principles of Web learning
design that are relevant to the models used in this study. The last part discusses the
subject content (Computer System) in this research.
2.1
Higher Order Thinking Skills
In most teaching and learning, HOTS received little or no attention (Ivie, 1998).
It seems that most of the conventional education system predominate in lecturing, with
students passively receiving information or responding to the exercise or examination
that require only recall and simple understanding of learning. The school is seen as a
37
landscape of mindlessness (Onosko, 1990). The students are unable to think effectively.
Quellmalz (1987) made this clear by saying:
“We have mountains of test data to document that most
students of all ages do not perform well on higher order
tasks”
(Quellmalz, 1987: 95)
Seen in this light, schools should be reformed in order to educate students with HOTS.
The development of high performance computing and communication is creating
new media in education. Nevertheless, the development has brought to the transition of
society that needs ability in thinking skills.
Ramirez and Bell (1994) noted that
educators must recognize that students require an education to enable them to master
HOTS because those skills are important in the workplace.
The advent of the
information age has made the development of HOTS crucial in the workplace.
In the discussion of the traditional practice of education system, Crawford (1996)
pointed out that there are significant tensions between traditional forms of educational
practice and the new forms of cultural activity associated with the new technological
systems. The necessary of the transition in the information age emphasis on the learner
that is capable to handle a large volume of information into meaningful knowledge
without lost in a quagmire of information.
From the phenomenon discussed above, what would make the technology truly
valuable in the use to encourage and develop learners in HOTS? Privateer (1997)
concluded that a major consequence of ignoring the pedagogical aspect of educational
technology is that failing to prepare student to become “knowledge workers”. It should
be a movement of the simple drill and practice and the overemphasis of sophisticated
multimedia design in computer learning system into a system that require learners to
delve into ways that promote HOTS and understanding.
38
Privateer (1997) further pointed out that educator should focus on the use of
technology to incorporate with pedagogical theories rather than strictly on the delivery
of the materials. Failure to do so will result an education system that is failing to meet
the needs of workforce in the information age. Besides that, learning in the overinformation context and the Internet environment in particular involves the higher level
of thinking structuring of the information (Van Zele et al., 2003). Ministry Eductaion of
Malaysia has shown the effort in promoting students’ HOTS by implementing the
curriculum that focuses on the development of creative and critical thinking.
The
teaching and learning activities are planned which promote critical, analytical and
creative thinking. The education system is gradually transforming from one that is
predominantly regurgitation of what the instructors taught to one that is thoughtful.
2.1.1
Higher-Order Thinking Skills Teaching Programs and Practice
The education system is rife with issues about thinking skills. The importance of
teaching thinking skills are relatively clear in the information age that engulfs in a
quagmire of data and information. Findings from the research show that nearly all of the
thinking skills programs and practice were found to have a positive difference in the
learning achievement. Reports with such findings include Barba and Merchant (1990),
Bass and Perkins (1984), Bransford et al. (1986), Crump, Schlichter, and Palk (1988),
Haller, Child, and Walberg (1988), Hudgins and Edelman (1986), Progrow (1988a,
1988b), Ristow (1988), Herrington, and Oliver (1999), Tay (2002), Tal, and Hochberg
(2003), and Hopson, Simms and Knezek (2001).
Considering there are many available thinking skills practice and programs,
Sternberg (1985), Sholseth and Watanabe (1991) and Cradler (1985) suggested some
guidelines that would assist in the searching of the most suitable program. Sternberg
(1985) pointed out the thinking program not only teach student how to perform tasks but
also when to use the thinking skills and how to implement or transfer them in new
39
situations. In addition, Baum (1990) has outlined the ten top programs which have
proven effective and reflects a strong theoretical base to teach thinking.
Among the available thinking skills programs that promote HOTS are HigherOrder Thinking Skills (HOTS) program, Project Improving Minimal Proficiencies by
Activating Critical Thinking (IMPACT), Structure of Intellect (SOI), Thinking to Write,
Expand Your Thinking, Connections, and Sage. The following discussion will focus on
these programs that contribute ideas to this research.
HOTS is a famous program that was designed specifically for at-risk students in
Chapter 1 and Learning Disabilities program by Stanley Pogrow. The intended audience
of HOTS is the Chapter 1 and learning disabled students in grades 4-7. The goal of this
program is to develop the HOTS to improve basic skill achievement, problem solving
ability, and social confidence (Pogrow, 1985, 1987, 1991). The ultimate goal is to
prepare learners to learn in the conditions where they can apply and integrate thinking to
learning content (Pogrow, 1988b). The design of HOTS focuses on the curriculum and
pedagogical approaches. The concepts from the information processing theories of
cognition were used as the theoretical base for the HOTS program (1987). It combines
the use of drama, Socratic dialogue, a detail curriculum and computers to promote
higher order thinking processes (Pogrow, 1991, 1996).
According to Pogrow (1985), there is no evidence that the students must master
the basic skills before they engage in HOTS. The thinking skills chosen in the HOTS are
(Pogrow, 1999):
(i)
Metacognition – Consciously applying strategies to solve problems
(ii) Inference from context – Figuring out unknown words and information
from the surrounding information.
(iii) Decontextualization – Generalizing ideas across content area from one
context to another context.
40
(iv) Information synthesis – Combining information from various sources
and identifying the key component of information needed for the
solutions to problem.
The HOTS design deploys computers to create a learning environment that
engages learners in HOTS. The program uses the Socratic teaching strategies to probe
students’ answers and act as coaches who guide students to construct and test their
understanding in solving problems posed by instructor.
The computer provides a
continuous flow of information for students to process as they develop their problem
solving strategies.
A number of researches findings demonstrated that the HOTS
program resulted in the improvement of performance and HOTS (Pogrow, 1988b, 1990,
1994, 1995, 1996; Crump, Schlichter, and Palk, 1988).
Project IMPACT is a thinking skill program to improve students’ performance in
mathematics and language arts by facilitating their acquisition of HOTS (Winocour,
1985, 1991). In seeks to improve students’ performance by infusing critical thinking
instruction into the subject content.
The cognitive operations of the HOTS are
classifying and categorizing, ordering, identifying relevant and irrelevant information,
formulating valid inductive and deductive arguments, and rendering judgments. The
intended audience of the program is all students. Students’ basic thinking skills are
improved through learning activities design in features that
includes skills clearly
identified in sequential to help students reason, skills are presented in a lesson-format
that infuses thinking into content areas, and ten teaching behaviors that promote critical
thinking are identified and reinforce students use of thinking in an interactive
environment.
The SOI was developed based on the Guilford’s theory of intelligence. The goal
of SOI is to equip students with the abilities that lead to higher-level critical thinking
abilities (Meeker, 1985, 1991). The program identifies about 90 different thinking kills,
ranging from basic foundation to the higher-level thinking skills. It is designed for any
age and ability level. The program emphasizes the learning process that focuses on
41
reasoning and higher-level critical thinking skills.
The SOI assessment allows the
administrators to evaluate the effectiveness of the change of curriculum by providing
information before and after the change has been made through the pretest and posttest
where students take the SOI-LA (Structure of Intellect-Learning Abilities) diagnostics
test. The SOI-LA assessments determine 26 intellectual abilities of students. Students
use materials prescribed for them from a computer software analysis based on a
diagnostics test.
With the SOI test result, it helps teachers to address students’
individual abilities, weakness and learning styles in order to promote their higher-level
critical thinking skills.
Thinking to Write was developed to provide a school-based evaluation system
that focuses on teaching thinking, writing, and problem solving that addresses operations
of higher-order thinking (Link, 1991b).
The higher-order thinking cognition and
operations includes are planning, strategies for problem solving, decoding and encoding
information and symbols, setting priorities, logical reasoning and concept development.
It is targeted for nine years old through college students. It is used to assess higher-order
thinking skills and abilities.
In the Thinking to Write, it contained documentary
evidence in the form of student journals, teacher journals, classroom videotapes, preand post-profiles of change in student cognitive behavior, and test essay. It focuses on
student-composed writing as an important assessment method providing insightful
information of higher order thinking.
Expand Your Thinking is a program that uses the graphic representations as tools
for applying thinking skills to content learning through working in cooperative pairs
(Hyerle, 1991). The use of graphic representations as thinking skills map is based on the
theoretical view of knowledge and thinking as the active making of mental connections.
It is targeted for students from grades five to six. There are six fundamental thinking
processes in the program such as thing-making, qualification, classification, structure
analysis, operation analysis, and seeing analogies. All these thinking processes help
students to display and apply connected ways of thinking and understanding, and finally
42
in greater depth of understanding within a specific content area activity. The program
improves students’ ability to use the six fundamental thinking for higher order thinking.
The Connections (Tishman, 1991) is a program that infuses the teaching of
HOTS into the regular curriculum. The materials are written for students from grades
three to six, and the strategies are appropriate for students from any grade level. The
higher-order thinking is taught in the context of real problem situations in the standard
school subject areas. The three thinking strategies are taught to solve with the real
problems are decision making, deep understanding and inventive thinking.
Each
strategy is presented in a 10-lesson module. Sage is a program designed for gifted
elementary students in order to develop HOTS through extending the regular curriculum
(Baum, 1990). The thinking skills development, independent study, and mini study
units are incorporated in the program to promote the students’ HOTS.
From the discussion of the existing programs that promote and develop HOTS, it
is clear that they are not only used to teach student how to perform tasks but also how to
use and implement HOTS. The programs’ activities focus on the cognitive operations
that would develop and promote students’ HOTS. Most importantly, the programs have
incorporated the theoretical based design of learning that enhances the development of
HOTS.
2.1.2
Definitions of HOTS
Many educators and researchers have discussed the important to engage students
in learning environment that develops HOTS. This issue is important because many
schools are at a crossroads of trying to recreate the learning environment as well as
curriculum to promote HOTS. To decide whether HOTS should be emphasized in
teaching and learning, it is important to understand the cognitive operations or
characteristics of HOTS, ways to implementing HOTS, and perceiving how computer
plays a role with HOTS. A large number of definitions on HOTS have been proposed.
43
Discussion in this part will focus on the definitions of HOTS that are relevant to this
research.
HOTS are emphasized by Resnick (1987) as the difference between rote-learning
and information withdrawal. Resnick also points out that the HOTS:
(i)
is nonalgorithmic,
(ii)
tends to be complex,
(iii) often yields multiple solutions,
(iv)
involves nuanced judgment,
(v)
involves the application of multiple criteria,
(vi)
involves uncertainty
(vii) involves self-regulation,
(viii) involves imposing meaning
(ix)
is effortful.
The definition of HOTS from Resnick reveals that developing HOTS require complex
cognitive processes. According to Resnick (1987), it is hard to differentiate between the
various HOTS. These skills are overlapping and integrative by nature.
Swartz (2001) noted that HOTS represents a multi-faceted and complex network
process that develops and improves the processing and construction of information.
Besides, Swartz also points out that HOTS are composed of seven interrelated processes:
concept formation, problem solving, rule use, reasoning or logical thinking, critical
thinking, creativity/brainstorming, and mental representation. Link (1991b), Marzano et
al. (1988) also noted that the concept development involves HOTS. Besides, Osborne
and Wittrock (1983) pointed out that HOTS are needed for conceptual change because
the processes of students’ conceptual change will engage them into HOTS. In addition,
Osborne and Wittrock (1983) and Hewson and Hewson (1988) noted that analysis,
synthesis, evaluation and reflection are essential if students accommodate information
44
into new mental structures. In addition, Ivie (1998) highlighted that HOTS tend to
reflect three related criteria:
(i)
The utilization of abstract structures for thinking. Ivie (1998) also
noted that the structure of knowledge gives form to think in abstract
terms.
(ii)
The organization of information into an integrated system. Hence,
HOTS organizes information into an integrated system. This is inline
with Sexton and Poling (1973) that the high performing students
mental world is organized, and they are able to see things as classes,
systems and relationships.
(iii)
The application of logic and judgment. It is thinking about thinking
or the engagement of the thinking skills.
Ivie (1998) further mentioned that it is important to address analysis; evaluation
and synthesis as thinking skills that develop students’ HOTS. Newmann and Wehlage
(1993) defined higher order thinking as skills which require students to manipulate
information and ideas in ways that transform their meaning and implications. The
thinking skills are such as synthesis, generalize, explain and testing hypothesis. These
skills are related with the analysis, synthesis and evaluation skills. Seen in this light,
strengthening cognitive operations of HOTS is important to help students to build new
concepts by providing learning environment that engages them in making analysis,
evaluation and synthesis.
According to the HOTS program developer, Stanley Pogrow (1987), there are
specific thinking skills that are considered as HOTS. These skills are metacognition,
inferring information from context, information decontextualization , and information
synthesis. When students develop these HOTS, they are more equipped to develop
deeper understanding of learning content areas.
45
Vockell and van Deusen (1989) defined that the HOTS fall into three main
categories.
(i)
Metacognitive skills
The metacognitive skills enable students to be aware of their thinking
when they perform tasks. Students can use the skills to improve their
performance on task.
(ii)
Critical and creative thinking
According to Ennis (1987), critical thinking is “reasonable, reflective
thinking that is focused on deciding what to believe or do” (p.10).
Creative thinking has been defined as the ability to take facts and
concepts out of their original context and apply them in new situation.
(iii)
Thinking processes
Thinking processes is defined as the dimension of thinking the
includes set of skills that are global strategies that involve several
thinking skills such as organizing, analyzing, evaluating, generating,
integrating skills.
The thinking processes are concept formation,
problem solving, comprehension, decision making and so on.
Kean (1985) noted that there is lack of consensus among educators on the
definition of HOTS. However, Kean (1985) has defined the HOTS in more specific
attributes of skills. Some of the elements of HOTS definition identified are such as:
(i)
Comparing and contrasting
(ii)
Making inferences
(iii)
Analyzing event
(iv)
Synthesizing information
(v)
Judging credibility of sources
(vi)
Organizing
(vii)
Classifying
(viii) Assessing the reliability, relevance, sufficiency, validity, and meaning
of data.
46
From the outline of the Kean’s definition, it is clear that the attributes of the HOTS
constitute analysis, synthesis, and evaluative skills.
Quellmalz (1987) identified the processes of HOTS consist of cognitive and
metacognitive. The cognitive operations of HOTS are analysis, comparison, inference,
and evaluation whereas metacognitive processes constitute planning, monitoring and
reviewing.
Quite often the definitions of HOTS include metacognition reflection
(Quellmalz, 1987; Vockell, and van Deusen, 1989; Fogarty and McTighe, 1993; Wai
and Hirakawa, 2001; Fogarty, 2002). Besides, according to Fogarty (2002), reflective
thinking is the foundation of HOTS and it begins when one’s aware of his/her cognitive
strategies. Students are self-aware and they plan, monitor, and evaluate their thinking
and learning when they engage in reflective thinking. It is utmost important where the
mind makes meaning of the learning and is then able to transfer and implement the
learning.
Quite often the HOTS are related to problem solving.
Quellmalz (1987)
proposed that the goal of HOTS is to enable students to engage in purposeful thought in
which they use the problem-solving strategies and become skillful in monitoring,
improving and evaluating the strategies. Buschman (1994) noted that when faced with
problem-solving, activities, it will encourage students to apply HOTS. Besides, Kallick
(2001a) pointed out that the cognitive operations such as analyzing, inferring and
evaluating are necessary in problem solving. This is further supported by Jonassen
(1992) that argumentation would be an appropriate outcome for problem solving which
engage students to in HOTS. Therefore, it is important to integrate skills such as
analysis and problem solving into learning content (Ennis, 1987; Zohar, Weinberger and
Tamir, 1994; Jonassen, 1992; Tal and Hochberg, 2003).
Bloom’s Taxonomy category of thinking skills is often related to HOTS by most
of the researches and reports. The taxonomy is a popular instructional model developed
by the prominent educator Benjamin Bloom et al. (1956). It categorizes thinking skills
47
from concrete to the abstract that contains skills such as knowledge, comprehension,
application, analysis, synthesis, and evaluation.
(i)
Knowledge
Knowledge is defined as the remembering of previously learned material.
This may involve the recall of specific elements in a subject matter.
Knowledge represents the lowest level of learning outcomes.
(ii)
Comprehension
Comprehension is the ability to grasp the meaning of material. It is
described in three different operations, the translating material from one
form to another, interpreting material and estimating future trends. These
learning outcomes represent the lowest level of understanding.
(iii)
Application
Application is the ability to use learned material to new problems and
situations. For examples, the application of rules, methods, principles
and theories. The learning outcomes represent the higher level of
understanding than knowledge and comprehension.
(iv)
Analysis
Analysis is the ability to break down material into its constituent parts
into the relative hierarchy of ideas with the relations between the ideas.
This may include the identification of parts and the hierarchical
organization, and analysis of the relationships between the parts. The
learning outcomes are higher than knowledge, comprehension and
application. Analysis is recognized as the cognitive operation of HOTS.
(v)
Synthesis
Synthesis is the ability to put parts together to form a whole.
This
involves the process of working with parts and arranging and combining
them in such a way as to constitute a new pattern or structure. The
learning outcomes emphasis on the formation of new patterns or structure
and creative behavior. Synthesis is recognized as the cognitive operation
of HOTS.
48
(vi)
Evaluation
Evaluation is defined as the ability to judge the value of material for some
purposes, of ideas, solutions etc. The judgments are to be based on
definite criteria either those determined by the students or those given to
them. The learning outcomes are the highest cognitive hierarchy and it is
recognized as the cognitive operation of HOTS.
The last three are widely considered as HOTS in the studies related to HOTS (see
Jamalludin Harun, 2005; Cradler et al., 2002; Tal and Hochberg, 2003, Swartz, 2001;
Ivie, 1998; Quellmalz,1987; Yuretich, 2004; Johnson, 1999; Eken, 2002; Hopson,
Simms, and Knezek, 2001; Hopson, 1998; Newmann and Wehlage, 1993; Kean, 1985).
On the other hand the first three are considered as LOTS (Dori, Tal and Tsaushu, 2003;
Bloom et al., 1956; Morgan, 1996).
There are a lot of definitions and elements of HOTS proposed by educators.
However, the description about the definition of HOTS as discussed makes it apparent
that HOTS is represents a multi-faceted and complex thinking process. In promoting
HOTS, it is important to integrate skills such as analysis, synthesis and evaluation
thinking into the learning content. The learning activities of HOTS are such as concept
formation and problem solving.
2.1.3 Instructional Strategies of HOTS
Many individual practices and reports look at the different instruction in various
clusters of HOTS. The discussion in this part will focus on the strategies of instruction
that are relevant to this research.
The advocate of teaching thinking skills in the context of education is divided
into two groups.
First, those who adopt a direct method to teach thinking skills
explicitly which are independent of subject matter and second, those who are in favor of
49
an infusion method that teaching thinking skills with the subject matter while
transforming content into instruction that will stimulate thinking (Maclure, 1991).
Among the experts that support the direct method are De Bono (1991), Edwards (1991),
Link (1991a), Debray (1991) and Beyer (1988). The experts that support the infusion
method are Greene (1991), Adey (1991), Caillot (1991), Lipman (1991) and Voutilainen
(1991).
Presseisen (1988) noted that in the movement to teach thinking skills, the
emphasis should be focused on presenting knowledge so that it is usable and useful to
the learners. Thus, the arguments on the best method of teaching thinking skills should
be avoided as the ultimate goal of educating thinking is to make the learners to be able to
think effectively. Both methods aim to meet the same goal no matter in what ways or
strategies. Maclure (1991) concluded that nothing suggests that there is any single
effective method and strategy to teach thinking skills. Presseisen (1988) made this
clearer by saying:
“Let us avoid the battle; call a truce. Let’s move from
the “either-or” notion and start a dialogue about
content and process in the curriculum”
(Presseisen,1988:8)
Seeing the phenomenon above, it is utmost important to emphasize on the thinking skills
development rather than debating on the methods of direct or infusion.
Schools that choose to nurture students’ HOTS have different options for
developing HOTS in the curriculum. One option is to use the thinking skill programs as
discussed in 2.1.1. The HOTS program from Stanley Pogrow (1987) is the most general
program use to develop the HOTS. The HOTS program has demonstrated its potential in
developing HOTS.
50
To foster the HOTS in teaching and learning process, Ivie (1998) suggested that
HOTS can be taught by erecting students’ meaningful cognitive operations in the
acquisition of new information. The idea is based on the learning theory suggested by
Ausubel (1963). This approach suggests that providing students in meaningful learning
enable students to grasp the interrelationship between the new and old ideas. According
to Ivie (1998), it is part and parcel to HOTS. Johnson (1999) made this point clearer by
stating that the cognitive operations such as analyzing, evaluating, synthesizing, and
organizing engage students in HOTS.
In addition, the HOTS instruction strategies suggested by Quellmalz (1987)
involve analyze, comparison, inference, evaluation, and metacognitive skills. Some
suggest that creating lesson using problem solving engages students in HOTS (Jonassen,
1992; Quellmalz, 1987; Tal and Hochberg, 2003). Students strengthen and practice their
HOTS when they are working to solve a problem.
Dede (1990) suggested that HOTS are best acquired in the learning environment
where:
(i)
a highly developed information-gathering tool is used to stimulate the
learners on hypotheses testing.
(ii)
learners actively construct knowledge
(iii) evaluation that measure HOTS rather than simple recall of facts.
Another method to foster HOTS is to enable the students to reflect on their
learning. This was also highlighted by Quellmalz (1987), Fogarty (2002) and Harrigan
and Vincenti (2004). The method for student to make reflection is the use of portfolio.
Portfolio can be used by students to collect and reflect on their work, this gives them an
opportunity to evaluate their learning development (Wolf, 1989). In portfolio, students
can monitor how their skills are progressing, and plan for their learning. This gives a
visual proof on their growth of learning as well as thinking.
51
In addition, concept map is useful for summarizing units of study or as an
advance organizer. When students are asked to create a concept map to explain a
concept and/or to show relationships between domains of knowledge, they are using the
HOTS of synthesis and evaluation (Hobgood, 2002).
Beaver and Moore (2004)
indicated that concept mapping can be used as a tool to encourage HOTS. The concept
mapping engages students in HOTS and deeper exploration in the learning. Concept
map reflects a student’s way of thinking and provides an opportunity to process
information and interrelatedness between information.
It is an effective tool for
presenting information visually. The wealth of research shows that it is easier to make
meaning of new information when it is represented visually. Research used the concept
map as tool as well as tool to promote and evaluate HOTS in the instruction; learning
and concept understanding (see Hollingworth and McLoughlin, 2002).
Another method is the use of computer to help student in promoting HOTS. This
is supported by findings from a number of researches in the use of computer to improve
HOTS. The details of this are discussed in 2.1.4.
2.1.4 Technology and HOTS
Research findings in the effectiveness of computer in learning and teaching have
demonstrated the potential for significant curriculum and instruction reform is becoming
more a reality through the use of technology. The use of technologies to support
students’ centered learning is well documented in the literature and research. Computer
has considerable potential to promote HOTS. According to Vockell and van Deusen
(1989), computer can be used to promote HOTS in
(i)
helping students to perform activities such as collecting, analyzing,
categorizing, and synthesizing more efficiently.
(ii) permitting students to spend more time on higher-order tasks rather
than on trivial activities.
52
(iii) helping students to practice skills tailored to their needs, proceed to
their pace, receive immediate or corrective feedback, repeat process as
necessary.
(iv) programming computer that involves HOTS.
Computer provides powerful tools to use in creating learner-centered and bring
vast resources of information to students (Morgan, 1996). Besides that, technology can
be used as cognitive tool allowing students to use interpret and organize their knowledge
and design their learning (Jonassen and Reeves (1996)).
This is aligned with the
proposed instructional design of the system in this research where the learner acts as
designer to design and construct their learning. In this situation, technology is used as a
mindtool for problem solving and conceptual development as noted by Reeves (1998)
and Jonessen (2000). It also acts as an intellectual partner to enable higher order learning
(Ting, 2003; Reeves, 1998).
Learners act as designers in the learning process when they use the mindtool
(Jonassen, 1994; Jonassen and Reeves, 1996). Jonassen, Mayes and McAleese (1993)
found that individuals seem to learn the most from the design of instructional materials.
They further pointed out that when students are given chance to construct and design
their learning, it becomes a powerful learning environment that engage them in HOTS.
Peck and Dorricot (1994) noted that the use of technology applications will allow
students to organize, analyze, interpret, and evaluate their work. This process requires
them to think in more meaningful and complex ways and therefore developing HOTS.
Baylor and Ritchie (2002) found that the impact of technology on HOTS is based
on the level of constructivist modes of the technology use. It is apparently that the
computer technology itself does not promote HOTS. Computer technology needs to be
used with constructivist activities that encourage the development of HOTS where
students are required to evaluate and manipulate resources, then construct artifacts of
their knowledge.
53
2.1.5 Research Studies on HOTS
A lot of researches have been conducted in the development and improvement of
HOTS in the teaching and learning process. In the following part, the researcher will
discuss several researches regarding the instruction and development of HOTS.
Zoller (1999) presented an action-oriented research in scaling-up of higher-order
cognitive skills (HOCS)-oriented college chemistry teaching to freshman and sophomore
science majors. The teaching strategies employed in the study are teaching via asking
question to be discussed and analyzed rather than via statements to be memorized, self
and/or group out of class study and active involvement of students in the learning
process through the HOCS-oriented homework assignments.
The performance of
students was assessed through the HOCS-oriented examination in midterms, term, and
final examination. Result shows the strategies improved students’ HOCS.
Yuretich (2004) had conducted a study to assess the HOTS in a large
introductory science classes. The methods of learning in the study are active learning,
and online quizzes or homework with rapid evaluation and feedback. In addition, the
multiple-choice examinations that include questions involving HOTS in analysis,
synthesis and evaluation were designed and modified to assess the students’ HOTS. The
evidence of the effectiveness in the methods was displayed through the analysis of
student performance in examination, surveys and interview.
Result from student
performance, surveys, and interview confirmed the efficacy of the methods in
developing HOTS.
Olwell (2002) had conducted a study in developing HOTS in a college survey.
The HOTS focuses on developing the historical thinking and critical thinking in the
historical analysis. In the survey, students improved their critical thinking through the
ill-structured problems and reflective judgment. The result demonstrated that if students
are introduced to this higher-order historical thinking skills, the students have a deeper
understanding of history.
In addition, research from Harrigan and Vincenti (2004)
54
showed the importance of HOTS in college teaching and learning. They found that
reflection engages students in HOTS. Result of the research shows the use of reflections
gleaned evidence of HOTS.
Eken (2002) had conducted a study to investigate the development of HOTS in a
film studies class. The study indicates that developing students’ film literacy and HOTS
can help think in numerous dimensions and identify significant details while viewing a
film. The HOTS in the study are analysis, synthesis, and evaluation skills. Students
were encouraged to actively made use of details and view situations from different
aspects in the film. The symbolic system in the film facilitated judgment making when
students tried to interpret the meanings. In addition, student assignments indicated that
films enable students to make analysis about the films they watched. In the study,
interviews between the students revealed that acquiring HOTS enabled students to think
more independently, skillfully and efficiently.
The study demonstrated that the
development of HOTS enable students to develop skills to effectively read other media
products.
2.1.6
Research Studies of Using Technology to Improve HOTS
There is a consensus about the intervention of ICT on students’ HOTS towards
learning. A lot of researchers advocate the integration of ICT to engage students in
HOTS (see. Cher and David Hang, 2003; Baylor and Ritchie, 2002). Sinclair, Renshaw
and Taylor (2004) stated that the use of Computer-assisted instruction (CAI) can
improve HOTS.
Research from Hopson, Simms, and Knezek (2001) demonstrated that the
technology-enriched classroom environment appears to have had a positive effect on
students’ acquisition of HOTS. Students can use the technological resources to manage
their learning. Computer can be used as tool to improve HOTS. Besides, finding of the
study also demonstrated that the technology-enriched classroom environment has a
significant effect on the student’s attitudes towards computer. In the study, students
55
were taught to use spreadsheet, database and word processing program. They were
required to use these tools to take notes, produce assignments, and projects. They were
also required to access the Internet and were taught to use a scanner, QuickTake Camera
and the multimedia presentation software. The students were tested using the Ross Test
of Higher Cognitive processes in the analysis, synthesis and evaluation thinking skills.
The research design used a posttest and a quasi-experimental design. From the study,
the use of technology shows its potential as a tool that allowed students to move beyond
knowledge acquisition to knowledge application.
In addition, Tay (2002) had conducted a study using ICT tools to engage students
in higher order thinking. The study aimed to examine and analyze where and how ICT
is integrated in Singapore schools to engage students in higher order thinking. The study
also demonstrated positive findings where the ICT tools can engage students in higher
order thinking. Besides, in the study conducted by Cousins and Ross (1993) on the use
of computer for improving HOTS, they concluded that studies should be designed to
measure change in student performance, specifically in HOTS.
The use of interactive multimedia in promoting HOTS has been studied by
Stoney and Oliver (1999). In the study, a multimedia microworld was developed for use
in a university course in business field. The study found that using the interactive
microworld led learners to engage in cognitive and higher-order thinking. The learning
environment in the program was designed to engage students to analyze, clarify, and
synthesize information. This has led students towards HOTS which helps them to
achieve the goals set by the program. Their findings suggested that the multimedia
program can provide a learning environment capable of supporting and maintaining
significant levels of higher-level thinking.
Results from the research conducted by Juliana (2001) revealed how useful the
technology support and enhance students’ HOTS. The study integrated technology to
support and enhance HOTS. In addition, Sarapuu and Adojaan (1999) had conducted a
study in the usage of educational Web pages to develop students’ HOTS. The study
56
concerned with enhancing students’ abilities to analyze and synthesize tasks on
worksheets. Results from the study demonstrated the use of educational Web pages with
appropriate teaching method can develop students’ HOTS.
Galambos (2001) had conducted a study in transforming online discussion to
develop HOTS. The study described a successful approach to promoting HOTS in
online discussion. The study focused on HOTS from Bloom’s Taxonomy, in which
students are invited to understand and apply HOTS to their contribution to asynchronous
discussions. A grading rubric matrix based on Bloom’s Taxonomy was used to analyze
the result in the study. Result from the study demonstrated that the participation in the
asynchronous discussions has improved. The asynchronous discussion combined with
instruction about HOTS seems to promote deep connected education.
Hollingworth and McLoughlin (2003) conducted a study in the development of
HOTS in teaching of fist-year chemistry. The aim of the study is to present students
with a range of learning activities, relating chemistry to the real world and developing
HOTS. A number of approaches to teaching practice, with the focus on pedagogical and
design principles were incorporated to develop the HOTS. Particular attention has been
paid to the design of a distance learning that use active learning, challenging assessment
task and self-directed learning. The HOTS targeted in the study are analysis, synthesis
and evaluation. Students use a textbook/CD ROM, access to a multimedia library and
Web site. Result demonstrated that the development of the HOTS has enhanced
students’ learning.
Tal and Hochberg (2003) applied a Web-based learning environment that
involves HOTS, and allowed group and class discussion. They used the Web-based
Inquiry Science Environment (WISE) to encourage students to engage in HOTS. The
study used WISE to introduce assignments that required various level of thinking skills
and offered teaching HOTS in a gradual manner. It aimed to assess the HOTS in the
variety of assessment assignments. The Web-based learning helped students to collect,
organize and evaluate information through the use of inquiry assignment. The HOTS
57
assessed in the study are students’ progress in their argumentation skills, reflective
thinking, and problem solving. Data was collected in the pre and post questionnaires
and portfolio. Data was analyzed based on the reasoning complexity rubric, dimension
of reflective thinking, and the complexity of stages in problems solving. Findings show
the evidence of HOTS and problem solving improvement.
The findings suggested that technology is the ideal mechanism for promoting
HOTS, and a lot of researchers are beginning to validate this claim and it is well
documented in the literature.
2.1.7 HOTS in Computer Science Learning
The role of Computer Science education is not only producing competent
programmer, but also good thinker and problem solver (Reed, 2002). The computer
technology changed rapidly and in high speed. Computer Science students must be able
to stay abreast with rapidly changing trends and technology as most of the content in
Computer Science is based on the latest trend of the technology. This will need them to
be good in thinking.
Scragg, Baldwin and Koomen (1994) pointed out that the problem of Computer
Science students is lack of the deep understanding of the relationships between the facts
they learned. The students were unable to create the relationship between their prior
knowledge and new knowledge.
In fact, the Computer Science discipline is
hierarchically in structure (Henderson, 1986). It is necessary to develop and obtain a
balance and related concept between the subjects. Scragg, Baldwin and Koomen (1994)
further explained this by pointing out the evidence of the lack of insight understanding:
(i)
Most of the Computer Science graduates are over emphasize on the
minutiae of particular than understanding a problems and creation of an
appropriate solution.
58
(ii)
Employers complain that students have limited ability to apply their
knowledge to a new environment.
From the earlier description, it is clearly demonstrated that the source of the problem is
that students are unable to think effectively. This is supported by the argument from
Scragg, Baldwin and Koomen (1994) where students are lack of analytic and problem
solving skills.
According to Scragg, Baldwin and Koomen (1994), the problem of Computer
Science learning is lack of the emphasis on the problem solving and analytic skills.
Consequently, students focus more on a technical design process and the less on the
conceptual understanding activities. The phenomenon is the students are generally hard
to explain, describe and relate the terms in Computer Science such as volatile memory in
computer memory, its relationships with other hardware in the computer system and so
on.
Computer Science is a fundamentally creative endeavor that needs students to
have a deep insight into the relationships between the facts (Scragg, Baldwin and
Koomen, 1994). In order to handle the rapid changing of new computer technology,
students should be taught to the thinking skills development aspects rather than teaching
exclusively to the content. Miron, O’Sullivan, and McLoughlin (2000) noted that it is
important to design into a curriculum aspects that will “teach learners to learn” in a
research report on teaching first-year Computer Science students. Students should be
developed with both the technical and conceptual knowledge of Computer Science
subject.
In addition, Scragg, Baldwin and Koomen (1994) proposed the teaching of
Computer Science should focus on pedagogical issues such as:
(i) Integration of analysis and empirical skills it should start in the
introductory courses and built upon through out the course.
59
(ii) Students should have the commitment in the learning.
(iii)Design, analysis and experimentation are tools with which all subtopics
in Computer Science are studied.
These pedagogical issues will engage student in the learning environment to enable them
to learn independently and be prepared for the rapidly change of technology in
Computer Science. This clearly shows the necessity of HOTS in Computer Science
learning. This is supported by the study from Parham (2003). In the study, Parham
found that the ability of students’ HOTS would affect their performance in Computer
Science. Besides, Hamza, Alhalabi and Marcovitz (2000) suggested that to promote
HOTS, student must be acknowledged with learning goals and cognitive skills. In this
case, concept map can be used to enhance thinking, generate multiple ideas, and
organize associations between new and prior knowledge that elevate HOTS in Computer
Science learning.
2.2
Learning Object Design
The educational software development is an extremely high in terms of cost and
time. Learning object design is an issue in the e-learning that facilitating the usage and
manipulation of the highly similar learning sources. Technology is able to distributed
information, however much of the Web-based content materials that we can see now is
leveled down to the same approach of segmented mode of delivery of passive learning
material such as the use of the Web as information retrieval and electronic book. The
teaching and learning should not to be a process of reproducing and memorizing
information but a process constructing knowledge that involves learner in changing
information to knowledge. Hence, it is essential to understand that design of e-learning
is a design of learning and not a design of teaching.
According to Longmire (2000a), most electronic learning content is currently
developed for a specific purpose such as a course. How would designer wish to add a
60
layer of complexity to their work by including object capability in their design? How
would learner wish to select only a small part of content? In most cases this will spend
many times over the costs, development time, and learning effectiveness. In fact, the
dynamic features of learning object design reduce the problems.
2.2.1
What is Learning Object Design?
There is vast amount of literature about the learning object design. There has
been an increased usage of the learning object in the instructional technology over the
past few years (Anderson, 2003). Learning object design is an application of objectoriented thinking to the world of learning. Like Lego bricks, learning objects are small
reusable components- video demonstrations, tutorials, procedures, stories, assessments,
simulations and so on. Based on this idea, learning design efforts might benefit from
plugs and plays interoperable pieces of learning content that could be assembled and
reassembled as needed.
The idea of learning object design is grounded in the object-oriented paradigm
from Computer Science. This approach asserts that the learning objects are selfcontained, though they may contain references to other objects, they may also be
combined or sequenced to form larger educational interactions (Quinn, 2000). Learning
objects are central to instructional theories that support breaking down content to meet
specific learning goals as offered by most of the instructional design models.
A number of definitions of learning object have been offered.
From the
description of learning object found in literature, the definition of learning object
primarily depends upon the context of the definition and background of authors.
However, the learning objects definition generally cover the following categories
(Mortimer, 2002):
61
(i)
Content:
The content and learning activities supporting the learning
objective.
(ii)
Size and seat time: A chunk of learning that takes no longer than 15
minutes to complete.
(iii) Context and capabilities: A nugget of learning that exists in stand-alone
and be delivered to a learner on an as-needed, just-enough bases.
There are a few terms about the learning object with same meaning such as
Reusable Learning Object (RLO), Reusable Instructional Object (RIO), sharable content
object, educational object, modular building blocks, and chunk. International Electrical
and Electronics Engineers (IEEE) Learning Technology Standards Committee (LTSC,
2000) described “learning objects as entity, digital or non-digital, which can be used, reused or referenced during technology-supported learning. The LTSC provides an
examples of these objects, including “multimedia content, instructional content, learning
objectives, instructional software and software tools, and persons, organizations or
events referenced during technology supported learning”
Wiley (2000, 2002a, 2002b) defined learning objects as any digital resource that
can be reused to support learning. A learning object as an instructionally sound content,
combined with opportunities for practice, simulation, collaborative interaction and
assessment that directly relate to a learning objective or outcome. Hodgins (2001)
suggested that there is no set absolute size to a learning object, since the size of the
object will be relative to the needs of learners and the requirements of given learning
tasks. Meanwhile, Wagner (2002) noted that the object is a stand-alone data element
holding “content”, “learning” and “knowledge” to provide a highly personalized
learning programs, easily updated courses and performance support tools.
Wagner
(2002) further pointed out that it is commonly viewed as the smallest element of standalone information required for an individual to achieve a performance objective or
outcome and is used on online instruction.
62
Robson (1999) pointed out that the learning object is an approach to learning
content in which content is broken down into “bite size” chunks. These chunks can be
reused, independently created and maintained, pulled apart and sucked together like so
many Legos. From this view, it offers the learner-centered and active learning solutions
and tools.
The description above makes it apparent that the learning objects are selfcontained, reusable chunks of instruction that can be assembled with other objects to
provide some larger instructional sequences. The learning object can be used to make
up lesson objects or courses. The learning objects can be taken out from the certain
context and plug them into a new context, or deliver them through a new medium (print
or display). Learning object design can teach facts, concepts, principles, procedures and
processes. They can be in form of simulations, games, drill-and-practice or tutorials.
According to Mortimer (2002), there are three interdependent components in the
learning object design: the learning object itself; metatagging (a standardize way to
describe the content in code); a Learning Content Management System (LCMS) that
stores, tracks, and delivers content.
2.2.2
Attributes of Learning Object
Most often that the attributes of learning object are referred to modularity,
interoperability and discoverability (see Singh, 2000; Longmire, 2000b; Roschelle et al.,
1998).
(i)
Modularity.
The learning object is free standing, non-sequential, coherent and unitary
(Longmire, 2000a, 2000b). It is flexible and reusable to be used in multiple
contexts. The modular learning objects maximize the potential of software
that personalizes content by permitting the delivery and recombination of
material at the level of granularity desired (Longmire, 2000a).
63
(ii)
Interoperable
The object approach allows organizations to set specification regarding the
design, development, and presentation of learning objects based on the
organizational needs (Longmire, 2000a).
(iii) Discoverable
The learning object contains metadata (structure data about data) used to
describe and categorize it. It provides searchable, standardized information
about digital objects such as authorship, subject classification, size, format,
delivery requirements or interactivity level (Quinn, 2000). Quinn (2000)
further pointed out that for these objects to be used intelligently, they
should be labeled with certain criteria such as what they contain, what they
teach and what requirements exist for using them.
2.2.3
Granularity of Learning Object
Learning object can be a relatively small learning material, a unit of lesson; a
subject contains some lessons or even larger learning material. The granularity of
learning object concerns in the combining learning objects in terms of sequencing
(Wiley, 2000, 2001, 2002a). According to Wiley (2002a, 2002b), granularity refers to
the size of a learning object. On the other hand, combination of the learning object
refers to the manner how the learning objects are assembled into a larger learning
material.
There are different ideas and suggestion found in literature about the granularity
of learning objects depends upon the different uses. Wiley (2002a, 2002b) noted that the
decision of the size in a learning object is viewed as scope and the decision must be
made in an instructionally principled manner. Wiley (2000, 2001) also suggested a
framework of the learning object granularity that incorporated and developed
accompanied with instructional design theories.
In his framework, a taxonomy of
learning object has been presented. The taxonomy contains five learning object types
64
that lead to the sequence, scope and structured decision in learning object.
The
following differentiates between the five objects types.
(i)
Fundamental
An individual digital resource uncombined with any other.
It is
designed in such a way to enable the greatest numbers of combination.
For example, a JPEG computer monitor picture.
(ii)
Combined-closed
A small number of digital resources combined at design time by the
designer. The combined-closed learning object is designed to present a
single, stand alone, whole piece of information. For example, a video
about computer hardware with audio.
(iii) Combined-open
A larger number of digital resources combined by a computer in realtime when a request for the object is made.
The combined-open
learning object is the combination of other learning objects.
The
purpose of the combined-open is to be instructional. For example, a
Web page combining the pictures and video together with textual
material.
(iv) Generative-presentation
Logic and structure for combining or generating and combining the
fundamental and combined-closed learning object.
The generative
presentation learning object generates the objects and combines them to
create presentation for instructional.
It has high intra-contextual
reusability but relatively low inter-contextual reusability.
(v)
Generative-instructional
Logic and structure for combining the fundamental and combinedclosed learning objects and evaluating students’ interactions with the
combinations.
It is created to support the abstract instructional
strategies such as understand and perform a series of procedure. It is
high in both intra and inter-contextual reusability.
65
Lau (2002) noted there are three ways to utilize and create Web-based learning
objects. There are Unique Learning Objects (ULO), Dynamic Learning Object (DLO)
and Customized Learning Object (CLO).
(i) ULO
The individual learning object that is tagged and stored in learning object
repository. It can be used to create other unique structures like DLO and
CLO. It can be used as an independent learning object to enhance certain
concept.
(ii) DLO
It is a lesson created from the ULOs. It is a template needed to format
ULOs for varying presentation and delivery media. The purpose of DLO
is to be a complete instructional experience. However, the DLOs are less
inter-context reusable.
(iii)CLO
The CLO is a customized lesson based on the ULOs. The CLOs have
better look and feel and it eliminates the unnecessary information or links
inside the original reference page.
2.2.4
Metadata of Learning Object
As discussed earlier, learning object is highly flexible and it can be a small self-
contained learning material. An associated requirement for learning objects to be linked
and contextualized is important. Description about learning objects is utmost important
so that they can be used intelligently. They must be labeled as to what they contain, what
they teach, what requirements exist for using them. Thus, they need a reliable and valid
scheme for tagging (Quinn, 2000).
Metadata tag is a way of details and richly describing the content through the use
of tags created in X-tensible Markup Language (XML). It enables authors to describe
the learning objects in a database or learning object repository. This helps students to
66
retrieve and search for the specific learning objects. The metadata of learning objects
comprise technical issues about the learning object such as format, size and delivery,
authorship, instructional styles, learning styles and so on are important to define the
learning object.
Currently, there are a few standard scheme of metadata that utilizes a standard
vocabulary for the fields within a metadata files such as:
(i)
Instructional Management System (IMS) Global Consortium is a
consortium of members that focuses in developing metadata XML
bindings and content packaging.
(ii)
Learning Technology Standards Committee (LTSC) of the Institute of
Electrical and Electronic Engineers (IEEE).
The LTSC produces
specifications and standards of the learning object metatag.
(iii) Sharable Content Object Reference Model (SCORM) from Advanced
Distributed Learning (ADL). SCORM has produced a few versions of
the standards and specifications of learning object metadata.
(iv) The Dublin Core Metadata Initiative is an organization dedicated to
promoting the metadata standards and developing specialized metadata
vocabularies for describing the learning object.
There are many metadata descriptions in literature depending upon on the project and
research focus. El Saddik et al. (2000, 2001) proposed a metadata for multimedia
learning object for the reusability of interactive resources in Web-based educational
systems. The categories of metadata are the information concerning the learning object,
presentation information, topic information and explanation information.
Besides,
Mahadevan (2002) used metadata from IMS for the design and development of the
learning objects in an e-learning.
According to Hamel and Ryan-Jones (2002), the metadata is usually written by
instructional designers to describe the learning object they have created. The high
67
quality of the metadata should be adapted to the course materials and learners’ needs.
Hamel and Ryan-Jones further pointed out that each organization should adopt a
metadata scheme and tagging rules that are appropriate for the kinds of information that
it uses. The metadata tag used in this research will be based on the modification of
metadata from the Mahadevan (2002). Details of this will be discussed in Chapter 4.
2.2.5
Learning Object Design in Learning
Wiley (2000) noted that the learning object design has brought to the shift of the
conventional e-learning design. Table 2.1 illustrates the transitioning learning design
from traditional to learning object design as noted by Wiley.
Table 2.1: Transitioning Learning Design
Traditional Learning Design
Content drive
Knowledge transfer
Instructor as expert
Linear
Instructor determination
Learning Object Design
Thinking task, scenario or problem solving
Knowledge construction and generation
Instructor as facilitator, designer and supporter
Hyper-linked and learner directed
Learner decision and choices
The shift of learning design clearly showed the emphasis on the active role of learners in
their learning. In the learning object design learning environment, the learners involve
in the construction and also the design and evaluation of its usefulness in their learning.
This learning design encourages and promotes the HOTS as learners construct
knowledge. Hence, a lot of researchers and authors advocate the knowledge design and
construction as a suitable metaphor of the learning object design.
2.2.6 Research Studies on Learning Object Design
Van Zele et al. (2003) had conducted a study in the implementation and
evaluation of a course concept based on learning object. In their study, students have
68
accessed to a Web-delivered component and an identical printed material as two sources
of information additional to the altered lectures. Both pre-test and post-test surveys
were conducted to investigate the impact and effects of the learning. The result of the
study reveals the impact of the learning object design based teaching and learning
approach where the students’ satisfaction was relatively high, mainly because it was
perceived to be useful. In addition, almost half of the students perceived to have learned
more by using the system.
Lau (2002) developed a reusable object-based lesson for mathematics. In the
system, the object-based lesson has been divided into ULO, DLO and CLO. From the
experiment conducted in the study, most of the learners felt the reusable learning object
lessons, as a supplement to the traditional lecture and textbook provided substantial
benefits.
Juell et al. (2002) conducted a project called “Learning Objects”. The project
provides a pedagogy based outline structure that is visible to instructor and student. It is
used to provide structure to on-line course material and to provide consistent and useful
indexing of the course material. In their preliminary study, a pre-test, post-test and an
attitude survey have been conducted on the use of the Web pages in the project. The
results indicate that the students thought the outline parts were very useful to useful.
Boyle (2003) delineated a framework for the authoring of repurposable learning
object design based on a synthesis idea from pedagogy and software engineering. The
study has outlined a series of design principles for the design and authoring of reusable
and repurposable learning objects for a project to improve the learning of Java. A case
study has been conducted to carry out a detailed evaluation of the impact on learning.
This study found the positive acceptance and preliminary results from the students of the
learning design.
Cochrane (2005) had conducted a study to identify the application of learning
object design in supporting the teaching and learning of Audio Engineering subject. The
69
result demonstrated the creating of multimedia learning object can enhance the teaching
and potentially provide online learning environment. The research also found that the
learning object design supports learners’ interactivity and interest in the learning. Result
demonstrated that the learning object design enhances the concept understanding. In
addition, the learning object design provides pedagogically rich learning environments
that engaged and motivated learners. The findings show the learning object design has
the potential to enhance learning.
Findings from the study conducted by Bradley and Boyle (2004) also showed
that the use of learning object design has demonstrated dramatically improved of results
in the learning of computer programming. The study indicates learning object design
has a significant impact in improving the teaching and learning process. Students have
more control over their learning in the learning process. They pointed out that the
learning object design provides online materials that support learning and they should be
designed with the incorporation of pedagogic principles.
2.3
Generative Learning and Learning Object Design
The premise underlying the concept of learning object design is that it is reusable
and flexible. The discussions of learning object design are now commonly related
directly to the concerns for content and its modification, utility, value and management.
Current research and development are primarily focus on establishing technical issues
about the learning object design. To this point, most of the applications and literature
related to learning object are mainly focused on technological attributes, metadata
standards, issues such as granularity, sequencing and interoperability (Singh, 2000;
Wiley, 2002a).
Ip and Morrison (2001) pointed out that despite of the issues regarding the
technical and standard format, learning object design can be adapted into different
pedagogical paradigms and it has the potential to match the requirements of different
70
learning paradigms.
They further pointed out that it is important to integrate the
pedagogies into the learning object design to amplify the attributes of learning object in
providing reusable and flexible learning environment. As stated by Bannan-Ritland,
Dabbagh and Murphy (2000):
“Only through sound pedagogical grounding will learning
object system have the potential to be used as effective
learning environments.”
(Bannan-Ritland, Dabbagh and Murphy, 2000:10)
Toh (2004) highlighted that the features of learning object design fits
constructivist learning well where students actively construct their own knowledge.
However, Agostinho et al. (2003) noted that there is little research being conducted
about the incorporation of the learning object with the constructivism learning
environment and learner centered approach learning. The researcher would like to quote
from Wiley (2002a) to conclude this situation:
“if this out-of-balance research and development thrust is
not righted soon, we will find ourselves with digital libraries
full of easy-to-find learning objects we don’t know how to
use.”
(Wiley, 2002a: 2)
Constructivism is an educational philosophy that encompasses a wide variety of
views, theories and instructional models (Bonn and Grabowski, 2001; Bannan-Ritland,
Dabbagh and Murphy, 2000). These views include the two principles: (1) learning is an
active process of constructing knowledge; (2) the instruction is a process of supporting
the construction of knowledge according to Duffy and Cunningham (1996), Duffy and
Jonassen (1992), Jonassen and Reeves (1996). Based on these views, students construct
their knowledge by interpreting their experience and building mental model to represent
the knowledge. Generally, the constructivism learning environment requires students to
71
be active in the learning process to construct the knowledge meaningfully rather than in
a predetermined sequence of study. However, the constructivist learning requires some
design for a true learning experience to occur. As pointed out by Wilson (1997),
constructivist learning activities do not indicate a lack of structure, instead, some
structures and disciplines are needed to provide goal-oriented opportunities that allow
and help students in constructing their learning.
This is further supported by the
research conducted by Dalgarno (1998) where the specific learning outcomes that based
on Bloom taxonomy of educational objectives (Bloom et al., 1956) are important for
constructivist computer assisted learning design.
According to Jonassen, Mayes, McAleese (1993), nearly every definition of
constructivism refers to knowledge construction rather than reproduction and learners
are actively engaging in constructing knowledge. Therefore, assumptions about learning
based on a constructivist approach are knowledge construction, generative processing
and active learning.
In the knowledge construction, students construct knowledge
through creating meaning from their experience. The knowledge construction relies on
active mental processing of students’ experience.
There is a vast literature about the constructivism learning environment. In order
to further understand how the learning object design grounded in constructivism, this
research worked based on the application of constructivist theories to the learning object
design from Bannan-Ritland, Dabbagh and Murphy (2000). They propose a few models
which are aligned and heavily related to constructivism and implications when applied
to the learning object model. One of the models which is related to this research is
generative learning.
According to Bannan-Ritland, Dabbagh and Murphy (2000), Dunlap and
Grabinger (1996a, 1996b), Duffy and Jonassen (1992), Morrison and Collins (1996),
Grabowski (1996), Bonn and Grabowski (2001),
Jonassen, Mayes and McAleese
(1993), and McLoughlin (1998), generative learning is an important constructivist
learning. Besides, Bonn and Grabowski (2001) pointed out that generative learning
72
provides the necessary theoretical framework for researches in a constructivist
perspective. Reigeluth (1996) noted that generative tasks and learner exploration are
useful constructivism instructional strategies. In addition, the generative learning
strategy has been applied in development of technology-based constructivist learning
environment (Cognition and Technology Group at Vanderbilt (CTGV), 1993; Grabinger,
1996).
As described by CTGV (1993), the generative learning is the first required
element of constructivism learning environment.
Generative learning is the process of constructing links between new and old
knowledge, or a personal how new ideas fits into an individual’s web of known concepts
(Wittrock, 1974; Wittrock, 1991; Wittrock ,1986; Osborne and Wittrock, 1983). Two
types of relationships from the generation are: i) Generating relations between concept,
ii) Generating relations between experience or prior knowledge and new information.
Wittrock (1974, 1991) proposed the idea of generative learning with the assumption that
active mental participation of learner is required for learning to occur. The generative
learning involves the creation and refinement of learners’ mental constructions about the
world (Ritchie and Volk, 2000). The focus of the generative learning is that students are
active participants in the instructional process where they construct knowledge through
information in the instructional environment to their prior knowledge and previous
experience (Grabowski, 1996). From these views, the knowledge construction is a
generative learning process.
In generative learning, students are active participants in working to construct
meaningful understanding of information found in the learning environment by
generating relationships between the information and use it to support problem solving.
Dunlap and Grabinger (1996b) pointed out that this is a higher-level thinking activity, as
opposed to simply copying down information and memorizing. A lot of research have
been conducted and showed the evidences of the generative learning to enhance learning
and demonstrated meaningful learning (see Barba and Merchant, 1990; Laney, 1990;
Wittrock and Alesandrini, 1990; Linden and Wittrock, 1981; Gao and Lehman, 2003).
73
These studies on generative learning yield positive results and have shown that in most
cases, active and generative learning produced significant gains of learning.
Schaverien and Cosgrove (2000), Shepherd, Clendinning and Schaverien (2002)
presented a learning model derived from this generative learning view. The generative
learning model provided six acts, exploring, designing, making and operating,
explaining, and understanding. The six acts follow idiosyncratic pathways in learning.
The model has provided a set of principles to support Web-based learning design
process in generative learning environment. Finding from the study shows that the
generative learning strategy with relevant teaching model provoked learners’ thinking
skills and developed their understanding (Laney, 1990; Schaverien, 2000).
According to Grabowski (1996), there are two basic families of generative
learning strategies.
One is used to generate organizational relationships between
different components of the environment that helps learner to understand the relationship
between the components. Example of this generative learning activity is concept map.
Another one includes integration and elaboration. Examples are constructing examples
or scenarios, metaphors, applications, analogies, problem solving and so on. These
generative learning activities require deeper processing of learning and result in HOTS.
Concept map is a type of knowledge representation used in education that is a
graphical node representing illustrating the relationship among concepts (Noval, 1998;
Novak and Gowin, 1984). Concept map has been widely used in educational settings.
Concept map allows students to demonstrate and illustrate their knowledge; encourages
them to reflect and elaborate their knowledge graphically (Novak and Gowin, 1984;
Alpert and Grueneberg, 2000). Novak and Gowin (1984) indicated that this technique is
appropriate for all levels of students and they further pointed out that concept mapping
encourages students to think. Concept maps can be in various forms such as network
map, chain map, and hierarchy outline map (Alpert and Grueneberg, 2000; Dabbagh,
2001).
74
According to Jonassen (2000), concept map engages learners in the
reorganization of knowledge, clear description of concept and their interrelationships,
deep processing of knowledge that promote students better in the learning and
application of knowledge; and relating new concepts to existing one and thus improves
understanding and thinking. Alpert and Grueneberg (2000) noted the concept map is a
tool to aid comprehension of knowledge and support the organization, elaboration,
justification and sharing of ideas. These activities engage learners in HOTS. This is
well documented in literature (Dabbagh, 2001; Alpert and Grueneberg, 2000).
According to Alpert and Grueneberg (2000) and Dabbagh (2001), concept map can be
designed in hierarchical outline form. The hierarchical outline form serves the same
purpose with the network concept map that contains the relationships between the
concepts (Alpert and Grueneberg, 2000). According to Alpert (2004), many students are
comfortable with seeing their thoughts arranged in outline form. Thus, an outline form
of concept map provides an alternative representation of concepts.
Concept map is an important tool in generative learning (Grabowski, 1996;
Osborne and Wittrock, 1983; Bannan-Ritland, Dabbagh and Murphy, 2000). Ritchie
and Volk (2000) pointed out that concept mapping is found to be valuable and effective
generative learning strategy and it is proven by their findings on students score in their
achievement tests. Dabbagh (2001) even said that computer-based concept mapping can
be used to provide generative learning environment. Confirming this idea, it has been
shown from the positive result gained from the research conducted by O’Reilly and
Samarawickrema (2003) about the significant use of multimedia concept map in
enhancing the learning. The concept map is proved to be a successful tool in providing
engaging and constructive learning environment.
Concept mapping makes students to think meaningfully about the content by
identifying and judging important concepts, classifying concepts, identifying the
relationships and generating the connection between the concepts, depicting the
hierarchical structure of knowledge in a way to show their understanding. All these
cognitive activities are HOTS.
75
2.4
Generative Learning, HOTS and Learning Object Design
With every new design of learning and teaching process, it is critical to consider
learning environment and strategy that grounded on a theoretical framework.
As
discussed in generative learning, learning is generative and active. Generative learning
requires students to actively engage mental processes to examine the new information
and to construct a new interpretation of the information. In this case, students must
work with information, manipulate it, change it, relate it, and use it to support problem
solving (Dunlap and Grabinger, 1996b). This is a higher-level thinking activity and it
helps in assisting the HOTs. McLoughlin (1998) noted that the learning environment
that enables learners to design, create and explore learning materials is important in
fostering HOTS. Therefore, the generative learning environment is conducive to engage
and promote HOTS.
With the features of learning object discussed earlier, the learning object design
is flexible and highly engaging technology-based environment. This dynamic nature of
learning object has great potential to capitalize on the learning process as well as
permitting students to associate instructional content with their prior knowledge as found
in the generative learning.
Bannan-Ritland, Dabbagh and Murphy (2000) took this
position further by saying that the learning object design should be configured as
generative learning environments because its nature aligns well with a generative
learning. The generative learning environment allows students to generatively construct,
manipulate and organize learning objects. Hence, this provides a meaningful learning
experience that promotes HOTS. There is evidence for this perspective in that allowing
students to design the hypermedia to show the interrelatedness of content improved the
learning of students and HOTS (Liu and Pedersen, 1998). Besides, study from Jonassen,
Peck and Wilson (1999) demonstrated the student-produced multimedia or hypermedia
is a powerful learning strategy.
Bannan-Ritland, Dabbagh and Murphy (2000) noted that certain granularity of
learning object holds promise for generative learning. Students will be engaged in the
76
learning environment which will need them to organize the learning objects in their own
way based on their own understanding. This provides the structure that encourages the
construction and reconstruction of information. That is the powerful of generative
principle that reveals the capabilities of learning object for learning. Students design the
learning content during the learning and this is similar to the “learners as designers”
posited by Jonassen and Reeves (1996). The learning object design that incorporates
generative learning provides a learning environment for students to support them
effectively in constructing learning and engaging in HOTS.
2.5
Web-Based Learning
The development of World Wide Web (WWW) has made us to be able to access
to the vast information.
The WWW is widely used as a vehicle for instructional
purpose. The integration of WWW into the process of teaching and learning in school
setting is becoming prevalent phenomena.
The use of Web-based learning in the
colleges and universities in Malaysia are progressing with the implementation of their
own learning delivery system. The Web-based is gaining attention as a tool of course
and content delivery especially at the tertiary level in Malaysia (Philips, 2000; Penman
and Lai, 2003). Study from Phillips (2000), Penman and Lai (2003) and Hong, Lai, and
Holton (2001) demonstrated the positive results of the use of Web in learning.
Result from the study conducted by Lim (2000) demonstrated the positive
feedback from both students and teacher in the use of Web-based learning environment
in supporting student centered learning and learning by doing. The advent of the WWW
technology tools and features, and the growing of learner –centered instruction have
provoked the Web-based learning (Bonk and Reynolds, 1997). The WWW tools and
features offer users with various online resources such as simulation, instructor notes,
reports, resources updates, interactive learning and so on. This learning environment
exhibits the potential of the WWW in the learner-centered instruction.
77
According to Jonassen, and Reeves (1996) and Reeves (1997), WWW can be
used as a cognitive tool that enhances the cognitive powers during thinking, problem
solving and learning when it is properly implemented in the instructional process.
However, Reeves further pointed out that simply putting instruction on the Web does not
guarantee better learning. Instead, WWW is just a method and it should be designed
according to the appropriate instructional theoretical models to support and enhance
learning. Different technologies should be employed and incorporated with the WWW
to design and develop a Web-based learning that is grounded on pedagogical aspect.
Reeve (1997) noted that:
“…the Web is an excellent vehicle for facilitating group
work, but it is a lousy vehicle for academic reading.”
(Reeve, 1997: 3)
The Web-based learning can help students to make connection, analyze ideas,
and develop conceptual frameworks for thinking and problem solving (Rohaida and
Kamariah, 2000). The Web with additional software tool permits students to analyze,
organize, synthesize, and share their knowledge with others. These are important skills
to engage students in their knowledge construction. As stated by Reeves (1997), Web as
a cognitive tool that can enhance students’ thinking. In addition, findings from the study
conducted by Yuretich (2004) demonstrated that the Internet or on-line activities can be
effective means of developing HOTS outside of class. Finding from Penman and Lai
(2003) show that the synchronous communication on Web can facilitate higher-order
thinking.
Bonk and Reynolds (1997) noted that the Web instruction offers learning that
provides learner-centered learning environment for HOTS. The Web learning
environment assists learning by scaffolding, encouraging students to articulate ideas,
foster students to engage in thinking such as reflection, self-awareness, creative and
critical thinking. Bonk and Reynolds (1997) further pointed out that the Web enables
78
the design of instructional strategies in promoting creative, critical and cooperative
learning.
Currently, there are a lot of researches that looking for an effective way to
combine online learning and face to face meetings learning system (Cole, 2005).
Learning Management System (LMS) is a Web-based learning system used that
currently used to provide the Web-based learning system that organize and provide
access to online learning for students, instructors and administrators (Paulsen, 2002;
Cole, 2005). The learning activities usually include access control, provision of learning
content, communication tools, organization of user groups and that tracks student
progress in a course as well as record scores of quizzes and tests (Paulsen, 2002).
Universities and higher education institutions are increasingly using the LMS.
There are a lot of commercial and open source LMS available. Examples of commercial
LMS are such as Blackboard and ANGEL learning LMS and ePortfolio. Examples of
open source LMS are Modular Object-Oriented Dynamic Learning Environment
(MOODLE) and Sakai. The open source LMS are increasingly popular and can be
easily installed and customized with little programming knowledge.
Cole (2005)
pointed out that most of the commercial LMS systems are tool-centered and only some
of the open source LMS are designed based on certain educational philosophy such as
MOODLE. MOODLE’s design philosophy was based on social constructionism where
people learn best when they are engaged in a social process of constructing knowledge
(Cole, 2005). It focuses on tools for discussion and sharing artifacts that engaged
students in the construction of knowledge. Hence, it is important to design the LMS that
incorporates the pedagogical aspects in order to promote learning.
2.5.1
Web-Based Learning and Learning Object Design
Khan (1997) defined Web-based instruction as a hypermedia-based instructional
program that utilizes the resources and attributes of the WWW to create meaningful
learning to support and foster learning. In addition, Khan also noted that the features
79
associated with the Web-based learning environment such as hypertext and hypermedia
are important in supporting learning. Both can be designed by students and this engages
them in generative learning environment where they actively construct their knowledge.
This increases the interaction of students with the learning. As noted by Atkins (1993),
the richer and more comprehensive the interactions between learners and materials, the
more is learned.
The hypermedia and hypertext enable information to be delivered as linear or
non-linear in more interactive form, it is thus possible to provide flexible learning which
promote active, learner controlled, and self-paced learning.
The hypermedia
instructional design offers greater opportunities for learner control and interactivity in
the learning (Barab, Young and Wang, 1999). Thus, the hypermedia and hypertext can
be used to create generative learning opportunities. Study from Jonassen and Wang
(1993) showed that the use of hypertext system in depicting information structure result
in comprehension improvement.
The hypertext also found to be appropriate in
providing the generative and constructive of learning process. In addition, study from
Schaverien and Cosgrove (2000) found that the generative learning environment design
in the Web-based learning provoked learners’ thinking skills and developed their
understanding.
More recently, the reusability of the content and learning object design becomes
an important issue in e-learning.
As the content development is time and cost
consuming, the content can be designed as a self-contained learning object to be
reusable. Besides, the learning object design is found suitable for the design of learning
from the various learning theoretical models discussed earlier. Further more, according
to Robson (1999), the hyperlink should be object-oriented instead of procedural as we
observed in most of the Web design. The use of learning object design will empower
online learners in unprecedented ways by enabling them to participate more actively in
the contextualization of information (Longmire, 2000a).
Therefore, students will be
engaged in a generative learning environment that encourages them in constructing the
context and meaning of their learning with the hyperlinks of the learning objects.
80
Mohan and Brooks (2003) noted that learning object design provides an
important foundation for the effective reuse of resources on the Web. In addition,
Dodds and Fletcher (2003) pointed out that the Web allows extensive links between
learning objects for use in instruction and the increasing technologies of Web also
support systems that generate instruction on demand. In addition, Zhu (1999) had
conducted a study that used the hypermedia in the design of the granularity of nodes in
information searching.
Hawryszkiewyez (2002) highlighted that Web-based
technologies are able and suitable to support the use of learning object design in
learning. These reveal the great potential of the development of Web-based learning
objects incorporates with the generative learning. In addition, interactive program such
as Java applets and Flash are now growing rapidly to create instructional materials on
Internet. Hence, this will move the trends of online learning from computer-based
learning such as CD-ROM to the Web-based system.
2.6
Learning of Computer System
Computer System (CS) is an important topic in the introductory to Computer
Science that provides knowledge of the vocabulary, knowledge of fundamental concepts
and information sources (Rosenberg, 1976). It is a fundamental Computer Science
concept in order to progress towards other Computer Science subjects.
2.6.1
Learning Problems of Computer System
Computer Science students come from various background, skills and
prerequisite knowledge. According to Henderson (1986), they are simply looking for a
career opportunity and do not really understand what Computer Science is about. Thus,
it is quite often to find the students complain about the difficulty of perceiving how a
computer works in their learning. Results from study conducted by Maj, Veal and
81
Charlesworth (2000) demonstrated that students lacked of knowledge about personal
computer (PC) technology and the basic skills needed to operate on computer in a
commercial environment.
Yurcik and Osborne (2001) and Arup (2004) noted that nowadays, students are
crowded with the marketing information about computers. Nevertheless, they are not
informed the architecture of computer. Consequently, they have a false sense of
competence because of the familiarity with the computer marketing hype and overlook
the useful facts about computer architecture to the computer system. In addition, before
the formal instruction, knowledge developed from daily experience about the computer
may be imprecise or totally mistaken and such misconceptions is referred as “folk
wisdoms” by Holmboe (1999). Most of the students develop their knowledge about
computer based on daily experience such as from the advertisement or brochure about
computer.
Their knowledge of computer system is insufficient when dealing with
concept on a more formal basis. They only know the common name of the computer
specification without the knowledge of the exact meaning. For example, the meaning of
speed in MHz (Mega Hertz), MB (Mega Byte) and so on.
The learning of Computer System is facing problem of supplementing lectures
and readings with hands-on experience especially in learning computer hardware
(Magagnosc, 1994; Yehezkel et al., 2001; Skrien, 2001; Ivanov, 2003). Computer
hardware and software have been brought into schools as a part of traditional teaching
emphasizing relatively passive absorption of information (Makkonen, 1997). In learning
computer system, the computer architecture is a difficult topic to teach because the
system may contain characteristics that are unique to the manufacturer and complex to
understand (Yurcik and Osborne, 2001). Besides that, most of the students are found
reluctant to open the computer cover for discovery (Yurcik and Osborne, 2001). As a
result, the computer concepts are sometimes too abstract for them.
Further more, Yehezkel et al. (2001) noted that students of Computer Science
acquire a cognitive model of the internal computer operation and this model is normally
82
found incomplete. Subsequently, the incomplete of the concept cause the lack of the
deep insight understanding of the relationships between the facts they learn as this is
important in Computer Science. They further pointed out that this has provided no
cognitive hooks that might enable students to relate new learning to their previous
learning. The students are unable to relate to their learning. However, the ability to see
the relationships between the concepts learned is important in learning Computer
System.
Research conducted by Mirmotahari, Holmboe and Kaasboll (2003) found that
students were good in the practical skills than on more theoretical questions. According
to Holmboe (1999), this indicates a type of understanding calls “Practical knowledge”.
The students in learning the computer architecture need more theoretical knowledge in
order to reach the holistic knowledge (Mirmotahari, Holmboe and Kaasboll, 2003).
They further pointed out that the problem of learning seems to correspond with their
understanding on basic computer system
Krishnaprasad (2002) pointed out that the Computer Science course less
emphasizes on topics in computer hardware. In fact, it is important to have a balanced
treatment of hardware and software concepts to prepare students to the workplace.
Computer Science graduates should have basic understanding of hardware and software
issues. They should develop knowledge in the impact of hardware techniques and
choices in a computing environment in relation to its performance, efficiency, cost and
reliability. These will need them to think rather than the regurgitation of what the
instructors taught. Despite of this, a solid background in hardware topics will benefit to
the software choice, design and development.
Arup (2004) found that the existing courses about computer system fail to
provide the environment for the development of HOTS. It tends to the remembering of
what the instructors taught and does not imply the ability to think. Students are exposed
to a large amount of fast changing information about computer system. The amount of
information about the computer system has grown as technology changed. As a result,
83
instructors and students tend to focus on the content. This overemphasis on content has
resulted in lack of emphasis on the HOTS that is necessary to successfully deal with
solving complex scenarios.
2.6.2
How to Teach and Learn Computer System
The development of knowledge and concepts in Computer Science is important,
particularly in a rapidly changing technology. The concept of computer is generally
being taught as an introductory in the first year of Computer Science course or in the
computer literacy for non-Computer Science students. According to Rosenberg (1976),
the introductory to Computer Science course should be designed that includes the
following characteristics:
(i)
It is concept-based.
(ii)
It structured modularly that the depth one is able to proceed down the
branches of the concepts and hierarchically that progressing down a
branch gradually uncovers more and more detail.
Because this
structure helps students build new knowledge on acquired knowledge.
(iii) Use significant media to support the communication of concept and
permits some self-instruction.
Scragg (1991) suggested that every course should build upon the existing
knowledge that students bring to the course. Scragg further pointed out that student
should learn the subject based on their previously achieved competence in relevant
subjects that are important to the new subject. Therefore, the learning of computer
system provides a “hook” to which students may attach new material as it is learned.
This hook allows them to comprehend and see the relevance of new concepts as it is a
fundament in learning of other Computer Science courses.
Hamza, Alhalabi and
Marcovitz (2000) suggested that the instruction should pave the way for students to
meaningfully bridge their prior knowledge with new knowledge.
84
Seeing the problem of the overemphasis of practical skills in learning computer,
Holmboe (1999) suggested that both of practical skills and conceptual understanding are
necessary and should be interconnected to relate the learning to the real world. There
should be a continuous interaction between the development of conceptual
understanding and practical knowledge. Holmboe further pointed out that teaching of
Computer Science should emphasize in:
(i)
Formal definition to avoid folk wisdoms on early stages in the
learning process (as mention in 2.6.1).
(ii)
Application to real life situation to understanding the essential
relationship between real world and the learning.
Frequent claims have seen in the past few years that introductory Computer
Science students should learn analytic skills (Henderson and Romero, 1989; Scragg,
Baldwin and Koomen, 1994). This is because most of the students have weaknesses
related to thinking skills such as logical reasoning, analytical thinking, synthesis
thinking and problem solving (Henderson, 1986). The skills are considered as HOTS.
This is further supported by Arup (2004) that the courses about computer system should
provide a learning environment for the development of HOTS.
These skills are
important in the learning of other Computer Science subject such as programming. So
the introductory courses such as computer system should promote students’ HOTS.
Studies in the Computer Science learning show that constructivist defines
learning as an active process of construction that offers a potential and powerful way to
the pedagogical practice in teaching Computer Science (see Hadjerrouit 1999; Ben-Ari,
1998; Brandt, 1997; Gotschi, Sanders and Galpin, 2003; Parker and Becker, 2003). It
provides a structure of teaching and learning by focusing on concepts and connecting
those to mental models in an active learning environment. In addition, Henderson
(1986) noted that concept map can be used as a mean for conveying information and
their relationships in Computer Science discipline.
The approach is important for
85
students to understand and develop their HOTS. Research finding from Jakovljevic
(2003) supports the use of concept mapping as a mind tool in learning computer
programming. The concept mapping enables students to represent their knowledge,
concept and procedures as an interrelated idea.
Students learn well in Computer Science when they actively engage in
exploration, interpretation and construction of ideas with multiple resources (Hawkins,
1993; McConnell, 1996). Computers can be used as cognitive tools to enable students to
synthesize, analyze and process meaningful information in computer learning (Hamza,
Alhalabi and Marcovitz, 2000).
In learning computer system, students should
investigate, explore, share and construct knowledge individually or together.
The
learning environment should engage them in analyzing and synthesizing computer
related theories, problems and challenging tasks.
At the beginning, most of the students do not have cognitive structure that they
can use to make viable constructions of knowledge based on sensory experiences of
computer (Ben-Ari, 1998). Hence, suggesting that actively engagement in visualization
technology is important in Computer Science education (Naps et al., 2002).
Visualization technology such as simulation can be used to visualize concept that gives
students a true insight about various concepts in Computer Science. Findings in the
study on using computer simulation and multimedia in learning computer showed its’
effectiveness in helping students in learning the concept especially in hardware concept
(see Yehezkel et al., 2001; Skrien, 2001; Ivanov, 2003; Knox, 1997; Butler and
Brockman, 2001; Naps et al., 2002; Ibrahim Ahmad, 2005).
Visualization allow
students to view different computer hardware components and its’ architectures. It
represents a visual metaphor to graphically illustrate how a computer system works and
its’ technology (Yehezkel et al, 2001; Skrien, 2001; Bem, 2002; Magagnosc, 1994).
This is important in providing the hook between students’ conceptual and practical
knowledge in learning computer system.
86
2.7
Instructional Design Model
Instructional design (ID) is very important for the development of high quality
instructional program that meets the users’ needs.
According to Reigeluth (1996),
instructional design is concerned with differentiating the methods of instruction that are
suitable for different situations. ID is an important component of any instructional
software development. Good ID can significantly increase the effectiveness of a
courseware (Baharuddin Aris, 1999). Wilson (1997) suggested that the ID should be
based on pedagogically design one in order to improve learning.
This is further
supported by Baruque and Melo (2003) that a solid foundation in learning theory is an
essential element in the application of ID.
There is a wide range of ID models that show the systematically planning and
evaluation process of instruction (Dick and Carey, 1996). Examples of the ID models
are Dick and Carey model, Gagne and Briggs model and Gerlach and Ely model
(Shambaugh and Magliaro, 1997).
Although there are many different types of
instructional design models, one model that has been particularly widely adopted is the
instructional systems development (ISD) model. ISD is a set of procedures for
systematically designing and developing instruction (Baruque and Melo, 2003; Molenda,
Pershing and Reigeluth, 1996).
The main elements of the ISD model are analysis,
design, development, implementation and evaluation (Baruque and Melo, 2003;
Baharuddin Aris, 1999). This model is commonly called as ADDIE model (Baharuddin
Aris, 1999). The ADDIE model has been widely used as for the process in educational
system production (Molenda, Pershing and Reigeluth, 1996).
In the object-orientation paradigm of e-learning system, the ISD that rooted in
learning theory is important for learning object design e-learning system (Baruque and
Melo, 2003). Instructional System Design Methodology based on e-Learning Object
(ISDMELO) is an ISD model that aims to design and develop the Web-based learning
system that based on learning object. This model is grounded in pedagogical principles.
87
The procedures in ISDMELO are based on the ADDIE model. Details of each phase
will be discussed in Chapter 4.
2.8
Summary
In this chapter, the five main categories have been discussed and analyzed to
support this research. The analysis reviews that the learning object has great potential to
be use as an instructional design structure for active and distributed learning due to its
flexibility and reusability in terms of user control. However, the learning object alone is
not adequate in supporting the learning. Thus, It is therefore reasonable to consider the
incorporation of the learning object with proper learning theoretical framework,
particularly generative learning.
Studies show the importance of HOTS especially for the students in higher
education who will soon join their careers. Several researches findings demonstrated the
positive effect of the relationships between HOTS and students’ achievement. Besides, it
may also be useful to look into elements which are essential for an effective computerbased learning to design the learning objects which are incorporated with generative
learning in order to improve HOTS. Thus, the elements of Web-based learning have
been discussed in details. All these need to be carefully considered in the formulation
and design of an instructional design model of a Web-based support learning tool. This
was discussed in further in the following chapter.
Several studies and reports show evidences of the learning and teaching problems
in the learning of CS in Computer Science. These provide the rationale for the choice of
topic in the learning of CS for this research. Most of the studies support the use of
computer in the teaching and learning of CS. In addition, a few studies show the
structures of the teaching and learning of CS in order to improve the HOTS. Besides,
some studies demonstrated the necessity of HOTS in the learning of Computer Science.
88
Particularly, this is further supported by the study from Parham (2003) that the ability of
students’ HOTS will affect their performance in Computer Science.
Based on the discussion in this chapter, the researcher believes that there is a need
to study the possibilities of developing a Web-based learning system which incorporates
the design of learning object with generative learning to improve the HOTS in the
learning of CS. In the next chapter, a detailed discussion about the methodology of this
research was presented.
CHAPTER THREE
RESEARCH METHODOLOGY
3.0
Introduction
This chapter presents the methodology which was conducted to determine the
effectiveness of the Web-based learning system.
The discussion has included an
overview of the assessment of HOTS, the rationale of the research design used in this
study, the research design and procedures, a description of the formative testing, the
selection of research samples and the methods of data analysis used to accomplish the
questions of research as identified in Chapter 1. The research methodology is based on
the systematic procedure modified from the ISDMELO.
3.1
An Overview of the HOTS Assessment
Since the attention to HOTS is made explicit with extensive researches,
researchers and educators have recognized that thinking can be taught, improved and
assessed. There are many experts and educators who advocate testing HOTS in two
main methods, either using the standardized commercial test instruments or selfdeveloped instruments. Besides, issues concerning whether to infuse thinking skills into
curriculum and the test or to teach and assess thinking skills separately (Baron, 1985)
90
affected the choice of the instrument. However, Ennis (1987) made his position clear
that thinking skills should be assessed in both ways.
As noted by Baron (1985),
Bloom’s taxonomy could be very useful in designing test items for infusing HOTS into
the assessment of subject area.
One of the commercial tests for HOTS instruments is Ross Test of Higher
Cognitive Processes (Hopson, Simms, and Knezek, 2001). The test consists of 105
items. The purpose of the test is to judge the effectiveness of curricular or instructional
methodology design to teach HOTS such as analysis, synthesis and evaluation as noted
by Bloom et al. (1956). The self-developed instruments designed to assess HOTS
include rubric, electronic portfolio, checklist, problem solving, concept map and
multiple-choice test. An example of the self-designed instrument for HOTS assessment
is the Rubric of Higher Order Thinking Evaluation by Bell, Allen and Brennan (2001)
where HOTS are assessed based on the Bloom taxonomy of thinking. Besides, Hogan,
Nastasi and Pressley (2000) have proposed a rubric to assess the HOTS in their study,
while Tal and Hochberg (2003) adopted and modified their instruments for HOTS
assessment.
Moreover, Zoller (1999) has designed and developed higher-order
cognitive-oriented instrument to assess the students’ HOTS, while Herrington and Oliver
(1999) have proposed the corroboration of characteristics of higher order thinking from
Resnick (1987) and the indicators for classification of higher order thinking in their
research. All of these instruments are found useful in assessing HOTS.
There are a number of tests for HOTS, but most of them are not tied specifically
to the purpose and outcome of the Web-based learning system. Many educators admit
that true thinking involves not only the ability to think but also the disposition to do so
(Boron, 1986; Norris, 1989; Costa and Kallick, 2001; Tishman, 2001). Fremer and
Daniel (1985) noted that the disposition is essential to HOTS because the desire and
willingness to think in HOTS is important to improve the thinking. The main goal of the
instruction and assessment of HOTS is to create the engagement of the thinking as noted
by Costa (2001), Asp (2001) and Costa and Kallick (2001). In addition, thinking skills
91
can be assessed by engaging students in some form of metacognition in which they
describe their thinking processes (Asp, 2001).
According to Asp (2001), thinking is a cognitive process that can not be directly
observed.
We need to make an inference about students’ thinking based on their
behavior in a particular situation. Asp (2001) further pointed out that thinking is content
bound and a student can not think about nothing. Marzano and Arredondo (1986)
pointed out that the students’ cognitive abilities can be improved by incorporating
content areas, the skills for learning, understanding and reasoning. In addition, the
manner in which thinking is assessed will vary with subject areas (Asp, 2001). Thus, the
best assessment of thinking will require students to apply their knowledge and skills
such as solving a problem or drawing concept maps. To obtain an accurate result, the
assessment of thinking should represent the specification from a particular content area.
Multiple assessment techniques should be employed in the assessment of HOTS
to increase the validity and reliability of the instruments used (Fremer and Daniel, 1985;
Baron, 1985; Asp, 2001; Costa and Kallick, 2001). Fredericksen (1984) has highlighted
the inherent bias in relying solely on the multiple choice test. According to Bell, Allen
and Brennan (2001), an over-dependence on traditional modes of assessment such as
tests can work against the assessment of HOTS. This is supported by Wiggins (1992)
who has argued that
“Typical tests, even demanding ones, tend to over assess
student ‘knowledge’ and under assess student ‘know-how
with knowledge’…”
(Wiggins,1992: 27)
Marzano and Arredondo (1986) pointed out that the assessment and evaluation of
thinking skills should also use qualitative data gathering methods because many of the
thinking skills can not be assessed via objective methods. On the other hand, Boron
(1986) noted that both the qualitative and quantitative methods are important in the
92
evaluation of thinking skills. Through the quantitative methods, the educators will know
how much the students are improving thinking skills whereas the qualitative methods
allow them to understand how the students attain those skills.
This research employed both the quantitative and qualitative methods in the
evaluation of the effectiveness of the Web-based learning system. Multiple assessment
techniques were used in the assessment of HOTS. The assessment of HOTS in this
research was modified from the thinking skills assessment design framework by Costa
and Kallick (2001). Triangulation method was used in the design of framework in order
to provide balanced assessment strategies in acquiring data about the students’
improvement in HOTS. Figure 3.1 depicts the design framework of HOTS assessment
in this research.
Assessing the cognitive
operations of HOTS:
- Scenario-based problem
solving test.
Assessing improvement of
learning:
-
Scenario-based
problem solving test
Assessing the
engagement of HOTS:
- Electronic Portfolio (How
am I doing checklist)
Figure 3.1: The Design Framework of HOTS Assessment (Adapted from Costa
and Kallick, 2000)
The framework contains:
(i) Assessment of the cognitive operations of HOTS
To be able to think effectively, one has to acquire and perform certain basic
thinking skills (Beyer, 1988 and 2001). In this research, the assessment of the
93
cognitive operations of HOTS was based on the cognitive operations from the
Bloom’s Taxonomy of thinking and the learning outcomes of CS. Tests were
designed to assess the cognitive operations of HOTS. Questions of the tests were
designed to engage students in HOTS.
A rubric of HOTS evaluation (see
Appendix A) modified with the permission from Hansen (2001) and it was used
to evaluate the cognitive operations of HOTS. The rubric was designed by
Hansen (2001) based on the Bloom’s Taxonomy (1956).
(ii) Assessment of the improvement of learning
The test was used as the thinking task that consisted of scenario-based problems
to assess students’ improvement of learning. Test was also used by Lim (2000)
to evaluate the effectiveness of a self-designed Web-based learning to improve
students’ learning. The problems in the test were designed based on the
characteristics of problems design that engage HOTS proposed by Weiss (2003):
a) Must be appropriate for students’ current content knowledge and the
problems are designed slightly beyond their knowledge to avoid the
regurgitation of knowledge.
b) Ill-structured problems that possess several solutions or no solution.
c) Authentic in which the problems are designed based on students’
experience or related to their expected career.
d) Promotes life-long and self-directed learning through the authentic
problems that require students to further analyze their solution or to
seek for alternative solutions to the problems.
The problems for HOTS in the tests required students to give their
argumentation.
This is because according to Jonassen (1992), giving
argumentation will make the students to utilize their HOTS. In addition,
Jonassen (1992) further pointed out that the process of giving
argumentation in the problem solving engages the students with HOTS. In
94
addition, Asp (2001) also noted that questions that require students to apply
their knowledge and skills will engage them with HOTS.
(iii)Engagement of HOTS
The assessment of the HOTS engagement aims to determine if the students
are aware of their own thinking and to identify their thinking during the
learning process. According to Chapman (2003), the students’ engagement is
indicator of their willingness to participate in learning activities. Chapman
(2003) further pointed out that analysis of the students’ engagement can show
their willingness to persist with cognitive tasks by regulating their learning
behavior and their inclination of using the cognitive. This can indicate the
active engagement of students in certain cognitive tasks.
The analysis of the engagement of HOTS was based on the electronic
portfolios of the students that contain the record of “How am I doing”
checklists. Portfolio is a folder owned by a student that contains a collection
of the students’ work to demonstrate their skill level and growth over time
and it is owned by the students (Haladyna, 1997; Grace, 1992; Klenowski,
2002; Paulson, Paulson and Meyer, 1991; Costa and Kallick, 2001; Ash,
2000). It offers a way of assessing the students’ learning (Paulson, Paulson
and Meyer, 1991; Moersch and Fisher III, 1995; Haladyna, 1997; Melograno,
1994; Engel, 1994).
Costa and Kallick (2001) noted that portfolio is a
suitable instrument to study the improvement of thinking. Furthermore, Liu,
Zhuo and Yuan (2004) noted that portfolio is an assessment tool that
provides information about the development of the students’ HOTS over
time. The electronic portfolio used in this research only contains “How am I
doing” checklists. This research only used the small part of the electronic
portfolio to analyze the students’ engagement of HOTS
“How am I doing” checklist was designed to examine the engagement of
HOTS when learning with Web-based learning system. It is a self-assess
95
checklist that needs to be filled up by the students. Self-assess checklist is a
method to assess the students’ engagement of certain cognitive tasks
(Chapman, 2003). The checklist was adapted from the cognitive operations
of analysis, synthesis and evaluation thinking by Bloom et al. (1956), Bloom,
Hasting and Madaus (1971), Jonassen (2000) and Beyer (1988).
The
checklist aims to facilitate the self-assessment of the students for the
improvement of HOTS. It is used to measure whether the students are
engaged with HOTS in the cognitive activities (refer to Appendix B).
3.2
Research Framework
It appears that currently, most of the studies conducted in the use of the
computer-based learning environments are aimed at investigating the effectiveness by
measuring learning outcomes quantitatively. There are relatively few studies which look
into the learning process in developing thinking skills. Thus, this research combined
both the quantitative and qualitative research in order to gain a complete picture.
According to Neuman (2000) and Boron (1986), the combination of quantitative and
qualitative methods provides a better research design.
In this research, a preliminary study was conducted to identify the current
problems of learning a particular Computer Science subject and the students’ level of
HOTS in that subject. The research design was based on the pre-experimental design,
one group pretest-posttest design (Campbell and Stanley, 1963). The qualitative method
was used to identify the conventional teaching and learning of the subject. The results
gave an overview to this research about the conventional teaching and learning methods
and their consequences. The quantitative method was employed to study the level of
HOTS of the Computer Science students. A combination of qualitative and quantitative
method was also employed to study the effectiveness of the Web-based learning system.
Table 3.1 shows the research design and the data to be collected in this research.
96
Table 3.1: Data to be Collected and Research Design
Data to be Collected
Research Design
Conventional methods of teaching and Qualitative method through interview with the
learning in a particular subject.
HOTS
of
the
Computer
lecturers.
Science Quantitative method through the analysis of
students from the conventional teaching the answers from the past examinations of the
and learning method.
Effectiveness
of
subject.
the
Web-based (i)
Quantitative method where the pre-
learning system based on the students’
experimental design of one group
score in the test, improvement and
pretest and posttest was used to study
engagement of HOTS.
the improvement of learning and the
improvement of cognitive operations of
HOTS.
(ii)
Quantitative
method
through
the
analysis of the HOTS engagement in
the portfolio.
(iii)
Quantitative method through the Webbased Evaluation Form.
(iv)
Qualitative data through interviews.
As discussed in section 3.1, multiple assessment techniques are most appropriate in the
assessment of HOTS (see Fremer and Daniel, 1985; Baron, 1985; Asp, 2001; Costa and
Kallick, 2001). The combination of qualitative and quantitative evaluation is important
in the evaluation of thinking skills (Boron, 1986; Jamalludin Harun, 2005). Hence, both
the quantitative and qualitative methods were used in this research.
This research was divided into a few phases according to ISDMELO
(Instructional System Design Methodology based on e-Learning Object) which is based
on the ADDIE model.
ISDMELO was developed on the fundamental of learning
theories for the design of instruction based on learning object design. Generally, the
model can be divided into five main phases. The first phase is preliminary study
97
constitutes of analysis, the second phase is design, the third phase is development, the
fourth phase is implementation and the last phase is evaluation (Baruque and Melo,
2003).
3.2.1
Phase I: Analysis
This phase aims at analyzing and determining the conventional learning
problems and subject content before the design and development of the Web-based
learning system. The activities in this phase are:
(i) Subject Selection
The subject selection process involved the curriculum analysis for the Computer
Science subjects offered in Diploma of Computer Science in Southern College.
An informal discussion was conducted with 3 experienced lecturers from the
Computer Science Department. The selection also considered the relationship of
the subject with others subjects from the aspects of early development and
improvement of HOTS among the students. The importance of establishing the
cognitive hook of the early concepts learned and HOTS development to other
subjects in Computer Science has been well documented in the literature (see
Chapter 2). Based on these consideration and the preliminary study conducted,
Computer System (CS) was selected as the target subject in this research with
computer hardware as the focus. The rationales of the selection are as follow:
(a) It is an important subject in the introductory to Computer Science that
provides fundamental Computer Science concept in order to progress
towards other Computer Science subjects (Rosenberg, 1976). The subject
is a first year core subject for Computer Science students in Southern
College. Many researchers advocate the promotion of HOTS in the early
learning stage of Computer Science students.
As a result, the
introductory courses such as Computer System should promote students’
98
HOTS.
Therefore, this subject is important to develop the students’
HOTS and problem solving skills.
(b) The content of this subject grows rapidly with the change of technology.
Most of the conventional teaching and learning tend to focus on the
content rather than the HOTS that are necessary to successfully deal with
problem solving (Arup, 2004; Hadjerrouit 1999). This causes the
misconceptions, the lack of conceptual understanding and the
understanding of the relationships in the concepts they learned (Scragg,
Baldwin and Koomen, 1994; Mirmotahari, Holmboe and Kaasboll, 2003;
Hadjerrouit 1999).
Besides, problem of the overemphasis on the
regurgitation of the knowledge cause the inability of students in HOTS
(Arup, 2004). Thus, HOTS are useful for the students to handle the large
amount of rapidly changing information of new computer technology in
this subject.
(c) It is a subject to prepare the students with the understanding of computer
hardware. Thus, the students should develop knowledge in hardware
techniques and the choices of the hardware in a computing environment
in relation to its performance, efficiency, cost and reliability
(Krishnaprasad, 2002; Arup, 2004). These require them to think rather
than mere regurgitation of what the instructors have taught.
(d) The learning outcome of this subject is parallel with the objectives of the
Information Technology (IT) (Teknologi Maklumat) in Form 1 to Form 6
of secondary school in developing the HOTS. This reveals the effort of
the Malaysian Education Ministry in promoting the HOTS through the
teaching and learning of this subject.
(e) The high pace of computer technology development causes the content of
this subject must stay abreast of the rapidly changing technology. The
growth of the knowledge in computer requires more timeliness in
teaching resources, expertise and preparation time (Wolffe et al., 2002).
Thus, the learning object design used in the design of the Web-based
99
learning system in this research has the potential to deal with this
expanding growth of knowledge and skills.
(ii) In computer education, the prior knowledge of the students is the fundament for
the further knowledge construction and it can interfere with new concepts
(Holmboe, 1999; White, 2001; Mirmotahari, Holmboe and Kaasboll, 2003;
Scragg, 1991). Quite often that this knowledge is the subject of an introductory
to the Computer Science such as Computer System. According to Rosenberg
(1976), the introductory subject to Computer Science should be designed in the
structure such that one is able to proceed down the branches of the concepts
hierarchically and progressing down into a branch and gradually in more detail.
This provides the fundamental knowledge to other Computer Science subjects.
Thus, the learning of this subject should pave the way for the students to
meaningfully bridge their prior knowledge with new knowledge (Hamza,
Alhalabi and Marcovitz , 2000). These views fit well with the generative learning
used in the design and development of the Web-based learning system.
(ii)
Problems Analysis
A preliminary study was conducted on the conventional teaching and learning
methods used in CS and the level of the cognitive operations of HOTS. The
results from this study provided an overview of the current methods used in
teaching and learning of CS and their level of cognitive operations of HOTS.
Discussion with lecturers and analysis of CS examinations were carried out to
identify the current problems. The findings of the analysis will be reported in
chapter 5.
A discussion had been conducted with 3 experienced lecturers of the subject to
identify the current teaching methods and the problems of learning. From the
discussion, it was found that the method used by the lecturers was lecturing and
giving a predetermined structure of notes which were designed using Microsoft
Power Point. Besides, the lecturers also provided some extra materials from the
100
Internet and give assignments such as preparing a report about the CS. According
to the lecturers, the main problem of the learning in this subject was that the
learners could hardly see the relationships between the subtopics and the
concepts they had learned. Consequently, the learners encountered difficulties in
remembering and applying the concepts they learned. Further more, many of
them were poor in answering problems or questions that required them to use
HOTS. This is consistent with the findings from Scragg, Baldwin and Koomen,
(1994), and Mirmotahari, Holmboe and Kaasboll (2003). They also suggested to
use the Web-based learning tool to develop students’ HOTS.
(iii) Content and Task Analysis
The learning outcomes and major learning objectives of CS were identified and
decomposed it into sub-objectives in a way to show sequences of the prerequisites to be followed in the learning objects. The tasks to be performed by
the students in learning CS were identified based on the discussion with the 3
experienced lecturers of the subject. The analysis of learning objectives, content
of learning objects and the tasks involved the consideration of the research
objectives and the theoretical framework in this research.
3.2.2
Phase II: Design
This phase aims at designing the interface design and functions of the Web-based
learning system. It also determines the design of learning objects. The activities in this
phase are:
(i) Learning activities design of the Web-based learning system
The learning activities designed in the Web-based learning system was based on
the theoretical framework as discussed in Chapter 1.
presented in Chapter 4.
The details will be
101
(ii) Learning objects
The learning objects were designed based on the suitability of the learning
objectives to be accomplished from the content analysis and the dynamic features
of learning object design in supporting the generative learning.
Multimedia
elements were used to enhance the content of the learning objects. Based on the
task and content analysis, the learning contents were designed and organized in
small chunks. The structure of the learning objects was based on the learning
object sequencing model from Wiley (2000).
The details of this will be
discussed in Chapter 4.
(iii) Data Flow Diagram (DFD) and Storyboards Design
Analysis in phase I and phase II was used to determine the profile of the user
interface and suitable metaphor. The DFD of the learning system was depicted
based on the theoretical framework in this research (see Appendix C). The
design of storyboards in this research was constituted of a number of nunfunctional Web pages. It contained the navigation, instructions and metaphor
used in the learning system.
3.2.3
Phase III: Development
This phase aims to develop the Web-based learning system and learning objects.
It encompassed the following procedures:
(i)
The development of learning objects repository
(ii)
The development of search engine in the system
(iii) The development of learning objects organizer
(iv) The development of thinking tasks
(v)
The development of reflection corner
(vi) The development of information agent
(vii) The development of forum
102
3.2.4
Phase IV: Implementation
In this phase, the system was installed into the local area networking system of
Southern College and a plan for the implementation was prepared in order to ensure the
accomplishment of the research objectives. The system was implemented for about 9
weeks in the college. Table 3.2 displays the implementation plan of the system.
Table 3.2: The System Implementation Plan
Week
Activities
1-2
Implementation of the system on the intranet of the college.
Researcher conducts a workshop to explain the Web-based
learning system. Students and lecturers test and familiarize
with the system.
3-9
Implementation of the system on the intranet. Students use the
system.
Before the evaluation, students and lecturers were briefed about how the research will be
conducted. This will ensure that the students were able to design concept map. A
workshop was conducted by the researcher before the implementation of the system.
Each of the students involved in the research were given a user name and password. A
learning and trial section of the Web-based learning system was conducted for the
participants to familiarize with the system.
3.2.5
Phase V: Evaluation
This phase aims at measuring the effectiveness of the Web-based learning system
and it consists of formative evaluation and summative evaluation.
The summative
evaluation was conducted in order to meet the objectives of the research.
Both
103
formative and summative evaluations of the system took about 16 weeks to complete.
Table 3.3 displays the evaluation plan of this research.
Table 3.3: Evaluation Plan
Week
1-2
Activities
Formative evaluation
3
System refinement
4
Pretest
5-13
System implementation in the college.
14
Posttest
14
Students and lecturers fill up the Web-based learning system
evaluation form
15-16
Interview with students.
From table 3.3, the time interval between the pretest and posttest is 10 weeks.
The planning of the time interval was discussed with an expert who has experienced in
conducting this type of research and he has agreed with the time interval due to the same
set of questions administered in the pretest and posttest. The discussion with the expert
has also considered the internal validity such as maturation that would affect the results.
According to McMillan and Schumacher (1997), maturation problem is probably not a
threat if there is less a year in between the pretest and posttest.
3.2.5.1 Formative Evaluation
The formative evaluation was conducted before the summative evaluation took
place, and adjustments as well as refinement were made accordingly. The formative
evaluation involved 2 experts and 10 students. The experts are lecturers who have
experienced in Web-based learning system design and development and using Web-
104
based learning system in teaching. The number of expert and student is based on the
research conducted by Noraffandy Yahaya (2000) and Jamalludin Harun (2005).
During the formative evaluation, interview and observation techniques were
used to gather feedbacks from the students and experts. The formative evaluation of the
students was focused on the learning experience and usability design of the system.
Besides, the time required for the sample to answer the instruments such as Web-based
Evaluation form (WEF), pretest and posttest and interview was also collected in the
formative evaluation. The formative evaluation of the experts was focused on the
learning strategies and content of the system. Feedbacks of the formative evaluation are
as follow:
(i) Feedbacks from the students
(a) The students suggested that the system to provide user manual for them
to use the system.
(b) The uploading learning objects took time and there were no guidance on
the format of self-designed learning objects.
(c) The system should provide an electronic user manual to be downloaded.
(d) The forum couldn’t display some of the special characters and symbols.
(e) The learning activities in the system engaged them to use HOTS.
(f) The students only realized that the learning of CS can be so interactive
after using the system.
(g) The students found that the system is interesting because they can read
their friends’ solutions and refer their concept maps through the system.
Besides, they were happy to be given the chance to commend their
learning in the system.
(h) The students were happy because they could submit their learning tasks
through the system on Internet without any delay.
(i) The students found that the system enables them to contribute their
knowledge.
105
(j) The students found that they were actively involved in the learning
process with the system compared to the conventional teaching and
learning of CS.
(k) The students found that the system is interesting with the reflection
corner in the system because this was the first time they reflected their
learning and this helped them to recall and reflect their learning.
Besides that, the time required by the students to answer the WEF was about 25
minutes, the pretest or posttest was 2.5 hours and the interview was about 1.5 hours.
The information is important to enable the researcher in the time planning for the
summative evaluation. In the study, researcher also found that the students have the
problems in designing outline form lesson map. Hence, the researcher will conduct a
workshop about designing lesson map for the students.
(ii) Findings from the experts
(a) The learning process designed in the system provides generative learning
environment.
(b) The students are active in their learning.
(c) The students must think when they use the system.
(d) The learning activities encourage students to use HOTS.
(e) The thinking tasks in the system promote the students in problem solving
and HOTS.
(f) The activities in the reflection corner engage students in self-reflection
and they can monitor and gain better understanding of the students’
learning.
(g) The content of the learning objects are suitable for the students.
(h) More multimedia learning objects should be provided to visualize some
of the concepts.
(i) Function should be designed to enable lecturers to validate and remove
the irrelevant or incorrect learning objects for the subject.
106
(j) More instructions and guidance about the system should be provided for
users.
Besides that, the researcher found that the time required for the experts to answer
the WEF was about 40 minutes. Results of the formative evaluation from the experts and
students were used to improve the Web-based learning system.
3.2.5.2 Summative Evaluation
The summative evaluation was conducted in order to accomplish the objectives
of the research. The research design was based on the pre-experimental design, one
group pretest-posttest design (Campbell and Stanley, 1963) with the combination of
qualitative and quantitative approaches. According to Campbell and Stanley (1963), this
design is widely used in educational research. In the one-group pretest-posttest design, a
single group of the subjects is given a pretest (O), then the treatment (X), and then the
posttest (O). Figure 3.2 depicts the design of the research.
Group
Pretest
Methods
Posttest
A
O
X
O
O
X
=
=
Represents the measurement or scores.
Represents the treatment instrument of which the effects are to be
measured.
Figure 3.2: Pre-Experimental Design, One-Group Pretest-Posttest Design
Copeland (1988) pointed out that the comparative research model between two
methods has become less attractive. One group pretest-posttest design was used because
this research was aimed to study the outcomes of the system on the treatment group
rather than knowing something other than the treatment. The rationale is supported by
Neuman (2003).
This design is widely used by educational researchers such as
Baharrudin Aris (1999), Lim Tick Meng (2000), Mohd Salleh Abu and Tan Wee Chuen
107
(2002), Jamalludin Harun (2005) in the evaluation of the effectiveness of the selfdeveloped educational software.
According to Campbell and Stanley (1963), and McMillan and Schumacher
(1997), the serious threats of internal validity for the one group pretest-posttest design
are history, maturation, pretesting and instrumentation. However, the comparison group
can not be conducted as the sample size allowed by the college in this research is small
(30 students). Anyway, the researcher has taken some actions to minimize the threats as
follow:
(i)
History
Because there was no comparison group, the researcher is aware with the
other events that might occur between the pretest and posttest and that
would cause the change in attitude. In order to minimize the threat, the
researcher kept in touch with the students and tracked closely the
students’ development between the pretest and posttest.
(ii)
Maturation
The maturation that would probably happen in this research is being
bored as the time interval of the pretest and posttest increased (McMillan
and Schumacher, 1997). The researcher tried to make the learning system
as interesting as possible to stimulate the students’ interest. Forum and
reward in term of certificate of participation are part of the effort.
(iii)
Pretesting
Pretesting is not a threat in this research as the attitude of the students is
not a concern, as noted by Campbell and Stanley (1963), and McMillan
and Schumacher (1997). However, effort was put in this research to
minimize the threat by providing enough time between the pretest and
posttest based on the discussion with an expert.
(iv)
Instrumentation
Instrumentation is not considered as a threat in this research as
observation was not used in this research, as noted by Campbell and
108
Stanley (1963), and McMillan and Schumacher (1997).
In order to
minimize the threat, however, the instruments as well as the answers
collected in the pretest and posttest assessments have been checked by an
expert.
Besides the actions taken to minimize the major threats in this research, the researcher
has used reliable instruments as suggested by McMillan and Schumacher (1997) and the
instruments have been validated by experts.
The external validity refers to the generalizability of the results (McMillan and
Schumacher, 1997; Hoepfl, 1997; Campbell and Stanley, 1963). This research aims to
propose a conceptual model based on learning object design with pedagogical aspect to
improve HOTS and learning. Thus the results found in this research couldn’t be
generalized if the degree of similarity between the environments where the experiments
are conducted is different.
The threats for the external validity of the one group pretest-posttest design are
interaction of testing and X, and interaction of selection and X (Campbell and Stanley,
1963). The interaction of testing and X is not considered a threat in this research as the
research did not focus on testing the attitudes, as noted by Campbell and Stanley (1963).
The test used in this research is regular examination that does not involve attitudes and
thus the threat could be minimized. In order to minimize the threat of interaction of
selection and X, the researcher has highlighted a few requirements for successful
implementation of the learning system such as the facilities and the knowledge of
concept map design and hyperlink. The students do not need other specialized skills
such as programming to use the system. However, in order to minimize the threat, the
researcher has prepared an electronic manual in the Web-based learning system and
conducted a workshop before the summative evaluation.
109
3.3
Sampling
There were three sampling groups in this study, namely the students, lecturers
and experts in the theoretical perspective, content and technique used in this study.
3.3.1
Students Sampling
Generally, the sample in this research represents the students from Southern
College. Thus, the findings of the research can not be generalized to all the students
majoring in Computer Science in other colleges. The college was selected because the
college emphasizes the teaching and learning processes that involve students in HOTS
and problem solving skills. Besides, the college also plans to implement the Web-based
learning and has the facilities such as multimedia computers, server and network to carry
out the research.
The sample was randomly selected from the Computer Science
department to address the objectives of the research. The selected students were divided
into four groups.
(i)
The First Sample Group
The first sample group represented those who had taken the CS. This group
was used to identify the current level of cognitive operations of HOTS based
on the Bloom’s Taxonomy among the Computer Science students in CS
resulted from the conventional teaching and learning method. The number of
the population was 64 students and the sample size was 30 students. Their
final examination papers in the second semester year 2003 for CS were
analyzed to identify their level of HOTS.
(ii)
The Second Sample Group
From the population of the first sample group, 10 students were randomly
selected as the second sample group. The second sample group was used in
the formative evaluation to help the author to test the Web-based learning
system and determine the reliability of the instruments used in the research.
110
(iii) The Third Sample Group
The third sample group was the sample for the evaluation of the effectiveness
of Web-based learning system. The sample consisted of 30 students from a
class who had taken CS. The sampling was based on the random cluster
sampling by choosing a class of students arranged by the college.
(iv) The fourth Sample Group
The fourth sample group was the sample for the evaluation of the
engagement of HOTS when using the Web-based learning system.
The
stratified sampling was used to select the sample from the third sample
group. The third sample group was divided into three groups, namely the less
active, the active and the very active group based on the number of lesson
maps designed by the students in their learning with the Web-based learning
system.
Table 3.4 tabulates the number of students with the number of
lesson maps they had designed.
Table 3.4: Number of Students and the Number of Lesson Maps They Had
Designed
No. of Lesson Maps (LM)
7
8
9
10
11
12
14
15
16
18
19
20
25
Number of Students
1
1
2
3
3
4
5
3
2
3
1
2
1
Figure 3.3 illustrates the distribution of the number of lesson maps designed by
the students from the three different groups. The division of the groups was
based on the calculation of standard deviation (σ) of the distribution. The active
111
group was classified as ±0.5σ of the mean, very active group as above 0.5σ and
less active as below 0.5σ of the mean. The samples were then drawn randomly
based on proportional stratified sampling from each group. One third of the
students from respective group were selected for the analysis of the engagement
of HOTS which would be discussed in detail in Chapter 5. The number of the
sample was 10 students.
6
Less Active
Active
Very Active
No. of Student
5
4
3
2
1
0
LM7
LM8
LM9 LM10 LM11 LM12 LM14 LM15 LM16 LM18 LM19 LM20 LM25
No. of Lesson Map
Figure 3.3: The Distribution of Lesson Maps of the Students
3.3.2
Expert Sampling
The selection of expert sample was based on purposive sampling. They were 2
experts involved in the formative evaluation. The selection of the expert sampling was
based on their experience and knowledge on the design and development of Web-based
learning, and the experience in teaching computer subjects. They have the experience or
112
knowledge of the learning object design, generative learning, design and development of
Web-based learning system and the experience of using the Web-based learning system.
3.3.3
Lecturers Sampling
The selection of lecturer sample was based on purposive sampling. Since the
Web-based learning system used in the research is a tool for the learning of Computer
System, it is necessary for the lecturers to have adequate knowledge of the topic. The
selection of lecturers was based on their experience in teaching, knowledge of computer
and the experience in using or designing the digital teaching materials from various
departments. There were 12 lecturers involved in the system summative evaluation.
The 12 lecturers consisted of 2 lecturers from the Computer Science department, 3
lecturers from the Commerce department, 2 lecturers from the Engineering department
and 5 lecturers from the Language (English Language, Malay Language and Mandarin
Language) department.
3.4
Research Instruments
This research employed multiple forms of instruments which included Webbased Evaluation Form, pretest and posttest, rubric, checklist, portfolio and interview.
3.4.1
Evaluation Form
The evaluation form was used in this research to evaluate the Web-based
learning system. Two evaluation forms were used in this research, namely the Webbased Evaluation Form for the lecturers and experts, and the students respectively. Both
evaluation forms used Likert scale measurement. The evaluation forms were modified
from the reliable sources in learning object design and relevant to the theoretical
113
framework used in this research. The Web-based evaluation form was designed by the
researcher based on the instrument for hypermedia courseware from Elissavet and
Economides (2003), Learning Object Review Instrument (LORI version 1.5) from
Nesbit, Belfer and Leacock (2003), Baharuddin Aris (1999), Harmon and Reeves
(1998), and Jamalludin Harun (2005).
The Web-based Evaluation Form (WEF) for the lecturers or experts (Appendix
D1) consists of six main sections with six items in each of the section and a section of
comments as shown in Table 3.5. For each item in section A to F, the quality is
evaluated on a rating scale consisting of five points, range from 1 (strongly disagree) to
5 (strongly agree). If the item is judged not relevant to the system, or if the sample is not
qualified to judge the criterion, then the sample may select the NA (not applicable).
There is also a section (Section G) that contains 7 open-ended questions.
Table 3.5 shows the sections and items of WEF for the lecturers or experts. Two
experts were asked to fill up the Web-based evaluation form during the formative
evaluation. The alpha of the reliability coefficient for the WEF for the lecturers or
experts is 0.84. Besides, the form was validated by an expert who has experience in the
research and designing the Web-based learning evaluation form (see Appendix G).
Table 3.5: The Sections and Items in the WEF for Lecturer or Expert
Section
A: Learning object content quality
B: Organization of content
C: Presentation design
D: Pedagogical parameters
E: Motivation and user control
F: Interaction usability.
No. of Item
6
6
6
6
6
6
Alfa Value
0.80
0.80
0.80
0.90
0.85
0.88
Table 3.6 shows the sections and items of WEF for the students. The WEF for
the students (Appendix D2) consists of five main sections with six items in each of the
section and the quality is evaluated on a rating scale consisting of five points, range from
1 (strongly disagree) to 5 (strongly agree) and N/A for not applicable. There is also a
114
section (Section F) of open-ended question that requests for comments and suggestions.
10 students were asked to fill up the Web-based evaluation form during the formative
evaluation. The alpha of the reliability coefficient is 0.86. The form was validated by an
expert who has experience in the research and designing the Web-based learning
evaluation form (see Appendix H).
Table 3.6: The Sections and Items in the WEF for Students
Section
A: Technology and technical factors
B: Presentation design
C: Learning strategy
D: Motivation and user control
E: Interaction usability
3.4.2
No. of Item
6
6
6
6
6
Alfa Value
0.753
0.908
0.831
0.715
0.756
Pretest and Posttest
Pretest and posttest were used to identify the improvement of learning through
the score and the level of the cognitive operations of HOTS. Pretest was conducted
before the use of the Web-based learning system. Posttest was conducted after the use
of the Web-based learning system. Posttest was conducted ten weeks after the pretest.
Both tests contain same questions.
The questions were designed based on the
characteristics of problems design to engage students in HOTS proposed by Weiss
(2003) according to the cognitive operations of Bloom’s Taxonomy. There are three
main questions. Each question consists of a few sub-questions that represent different
cognitive operations from Bloom’s Taxonomy as shown in Appendix E. The questions
were checked and validated by experts as shown in Appendix I and Appendix J. The
expert is Diane L. Hansen who is the instrument designer of the Rubric of HOTS
Evaluation used in this research. The suitability of the questions to test the HOTS was
checked and validated by Diane L. Hansen (Appendix I). The test was also checked and
validated by an expert who has experience in teaching the subject in Diploma of
115
Computer Science (Appendix J). Besides that, the researcher and an expert who is an
experienced lecturer in the subject will check the students’ answer of the pretest and
posttest.
3.4.3
Rubric
Andrade (1997) and Goodrich (1997) defined rubric as a scoring tool that lists
the criteria for a work that articulates the quality of each criterion, from excellent to poor
for example. Jonassen, Peck and Wilson (1999) defined rubric as a code, or a set of
codes that is/are designed to govern actions. It is an assessment tool to assess complex
performance and process in learning (Jonassen, Peck and Wilson, 1999; Jonassen, 2000;
Andrade, 1997; Jensen, 1995). It makes assessing work quick and efficient (Andrade,
2000). Rubric has been used as assessment tool to study HOTS in educational research
conducted by Tal and Hochberg (2003), Hogan, Nastasi, and Pressley (2000), Zoller
(1999) and Christopher, Thomas and Tallent-Runnels (2004). An example of the selfdesigned HOTS assessment instrument is the Rubric of Higher Order Thinking
Evaluation from designed Bell, Allen and Brennan (2001) where the higher order
thinking is assessed based on the Bloom’s Taxonomy of thinking. In this research,
rubrics will be used to evaluate level of HOTS and achievement of students in pretest
and posttest (see Appendix A).
The rubric of HOTS evaluation was modified with permission from Hansen
(2001), the instrument designer.
The modification was based on the taxonomy of
thinking from Bloom et al. (1956) and Bloom, Hasting and Madaus (1971). The rubric
was validated by Hansen for its validity in the HOTS assessment (see Appendix K).
There are five scores in the rubric that represent different criteria of the assessment
answers. The maximum score is 4 and the minimum is 0 for Knowledge (K),
Comprehension (C), Application (App), Analysis (Ana), Synthesis (S) and Evaluation
(E).
116
3.4.4
Electronic Portfolio
Costa and Kallick (2001) noted that portfolio is a suitable instrument to study the
improvement of thinking. Besides, portfolio can be used for assessment that aligns with
current assessment and learning theory which focus in the development of thinking skills
and constructivist learning theory (Klenowski, 2002; Paulson and Paulson, 1996).
Portfolio assessment emphasizes the process of change and improvement. Grace (1992)
used the portfolio as assessment tool in their study. Finding from Tillema (1998)
demonstrated that portfolio instrument is a predictor of successful performance and
supports the assertion that a portfolio gives better insight about the attained level of
competence.
Study from Tillema and Smith (2000) reported that the portfolio
assessment gives people control over their own learning. In addition, Tal and Hochberg
(2003) have also used portfolio in their study to assess the students’ HOTS.
Electronic portfolio was used in this research. The term electronic portfolio is
used to describe portfolio that is saved in electronic format (Lankes, 1995). The
electronic portfolio is able to demonstrate the students’ progress and their thinking skills
(Lankes, 1995; Haladyna, 1997). The portfolio used in this research is a Web-based
checklist to record the progress of HOTS engagement during the learning with the Webbased learning system.
Checklist is a useful to help students to assess themselves (Costa and Kallick,
2001). The checklist used in this research is called as “How am I doing” checklist
(Appendix B).
The checklist aims to facilitate self-assessment of the students’
engagement of HOTS when they use the Web-based learning system. According to
Resnick (1987), the disposition shaping of thinking is essential to develop HOTS.
Hence, the checklist acted as a guide to identify the students’ engagement of HOTS as
well as to measure the progress of the HOTS engagement. The checklist used in this
research was based on the attributes of analysis, synthesis and evaluation thinking from
Bloom et al. (1956), Jonassen (2000) and Beyer (1988). The checklist was checked and
validated by an expert who has experienced in conducting HOTS research (Appendix L).
117
3.4.5
Interview
According to Costa and Kallick (2001), interview is an effective way to obtain
the information about the students’ reflection and attainment of the HOTS. Interview
was conducted to collect qualitative data in order to obtain in depth of understanding
about the findings. In this research, an interview was conducted after the posttest to
support the findings from other instruments. Structured interview with 13 open-ended
questions was used in this study (Appendix F). The interview was conducted based on
one to one. The total number of the students involved in the interview was 30. Each
interview took about 1.5 hours and the total was about 45 hours. The interview took
about 5 days. The interview was conducted by the researcher. It was started with a brief
and the questions then addressed to the students. The recording of the interview was
done by written notes. The examples of the interview questions:
(i)
Do you think the Web-based learning help you to improve HOTS? How?
(ii)
Was the Web-based learning effective in developing your problem solving skills?
How?
(iii) Which part of the software do you find engage you the most in HOTS? Explain
and give an example.
(iv) As you reflected on your work in the Web-based learning, which of the HOTS did
you most aware of? Why?
3.5
Data Analysis
Analysis of quantitative data in this research was based on descriptive and
inferential statistics methods. The qualitative data collected was used as additional
explanation to the quantitative result. The pre-experimental design was employed to
analyze on the effectiveness of the software. The data was analyzed based on the
descriptive and inferential statistics. Application software such as Statistical Package
for the Social Sciences (SPSS) and Microsoft Excel were used to analyze the data.
118
3.5.1
Analysis of Students’ Current Level of HOTS From the Conventional
Teaching and Learning of Computer System (CS)
The current level of HOTS of the Computer Science students from the
conventional teaching and learning of CS was analyzed and identified. Quantitative data
was collected through the analysis of the answers of the past CS examination.
Descriptive statistic methods such as mean and percentage were used in the data
analysis. A modified rubric of HOTS evaluation from Hansen (2001) was used to
analyze the cognitive operations of HOTS of students. There are five scores in the
rubric that represent different criteria of the assessment answers. The maximum score is
4 and the minimum is 0. The analysis of the HOTS level was based on the scores of
each cognitive operation from the Bloom’s Taxonomy of thinking, which include
Knowledge (K), Comprehension (C), Application (App), Analysis (Ana), Synthesis (S)
and Evaluation (E). The mean score of the taxonomy of thinking was tabulated using
Table 3.7. This method was also employed by Tal and Hochberg (2003) in the analysis
of HOTS.
Table 3.7: Table for Record of Mean Score of Cognitive Operations of HOTS for
Each Question
Taxonomy of Thinking K
C
App
Ana
S
E
No. of Question
1
2
3
The sum of the students’ scores and percentage of each thinking skill for all the
questions will be tabulated using Table 3.8.
119
Table 3.8: Table for Record of the Sum of the Students’ Scores and Percentage of
Cognitive Operations for All Questions
Taxonomy of Thinking
K
C
App
Ana
S
E
No. of Question
1
2
3
Total
Percentage (%)
A chart representing the percentage of the total score of each thinking skill for all
questions was depicted to demonstrate the overall percentage of the six thinking skills.
A discussion was done based on the data and results to examine the level of thinking and
HOTS from the conventional teaching and learning process of CS. A conclusion was
drawn from the discussion.
3.5.2
Analysis of the Effectiveness of the Web-based Learning System in the
Improvement of Students’ Learning
The effectiveness of the Web-based learning system in the improvement of
students’ learning was attained through the analysis of the students’ scores in the pretest
and posttest. The scores were compared by using the paired-samples T test to check for
significant differences between the scores of pretest and posttest. The significant value
used in the paired-samples T test to check for significant differences between the scores
of pretest and posttest at 95% of confidence interval is also being used in other research
such as the research conducted by Jamalludin Harun (2005) in the evaluation of the
effectiveness of a Web-based learning system in higher education level.
120
3.5.3
Analysis of the Effectiveness of the Web-based Learning System in the
Improvement of HOTS
The analysis of the level of HOTS improvement was based on the Rubric of
HOTS Evaluation (Hansen, 2001) in pretest and posttest. The score of each cognitive
operation in the Bloom’s Taxonomy was analyzed using paired-samples T test. The
mean score of each cognitive operation between the pretest and posttest was compared
to check for significant differences (see Table 3.9). This method was also used by
Jamalludin Harun (2005) in the evaluation of the effectiveness of a Web-based learning
in the development of the cognitive operation based on Bloom’s Taxonomy of Thinking.
Table 3.9: Table for the Comparison of Mean Score of Each Cognitive Operation
between Pretest and Posttest
Mean level of
knowledge in pretest
Mean level of comprehension in pretest
Mean level of application in pretest
is compared to
Mean level of
knowledge in posttest
Mean level of
comprehension in posttest
Mean level of
application in posttest
Mean level of
analysis in pretest
Mean level of
analysis in posttest
Mean level of
synthesis in pretest
Mean level of
synthesis in posttest
Mean level of
evaluation in pretest
Mean level of
evaluation in posttest
Besides, the mean score of each cognitive operation was compared, categorized
and displayed in chart to display the improvement of the level of HOTS before and after
the use of the Web-based learning system. This method has been used in the study
conducted by Tal and Hochberg (2003) and Yuretich (2004). A discussion about the
improvement of the level of thinking and HOTS is presented in Chapter 5. In addition,
data collected from interview was transcribed and discussed in which the results were
used to support and to cross-check the findings. The analysis of the data from the
interview was organized and then identified to the themes that presented the repeating
ideas as suggested by Hoepfl (1997) and checked by an experienced researcher. This
method was used in the research conducted by Jamalludin Harun (2005) in the
evaluation of the effectiveness of a Web-based learning system.
121
3.5.4
Analysis of the Effectiveness of the Web-based Learning System in HOTS
Engagement
The changes of the students’ engagement of HOTS over the time when they used
the Web-based learning system were analyzed using “How am I Doing” checklist in the
portfolio. The students were required to fill up the checklist for each topic of their
learning in the Web-based learning system. Their responds of “yes” and “no” were
collected and calculated in percentage for each topic to show the progressive
engagement of HOTS. Histogram was used to display the findings of each topic for
each student. This method was also used in a research conducted by Lim Teck Meng
(2000) to assess the understanding of learning through the achievement comparison in
the progressive test. In addition, data collected from interview was transcribed and
discussed in which the results were used to support and to cross-check the findings.
Conclusion about the findings was presented to demonstrate the engagement of HOTS
during the use of the Web-based learning system.
3.5.5
Analysis of the Effectiveness of the Web-based learning system as Perceived
by the Lecturers and Students
Descriptive statistics was used to analyze data from the Web-based learning
system Evaluation Form (WEF) to demonstrate the effectiveness of the prototype in
improving HOTS and learning of CS as perceived by the lecturers and students. There
were WEFs for the lecturers and students respectively. The analysis of the WEFs was
based on the various sections in the forms. Total ratings were calculated to show the
degree of satisfaction towards the prototype. To be more specific, the total score for
each section was also calculated to obtain a more comprehensive view on the degree of
satisfaction for each section. Table was used to display the result. Besides, the means
and standard deviations of each section were analyzed and tabulated for the lecturers and
students respectively. This method had been used by Baharuddin Aris (1999) and Lim
122
Teck Meng (2000) for the analysis of the usability of their self-designed educational
software.
The answers in section G (comments) for WEF for the lecturers and section F of
WEF for the students were summarized based on the categories of the responds and the
findings were discussed in detail in Chapter 5. In addition, findings from the interview
were transcribed and discussed in detail to support the results from the WEFs. The
analysis of the data from the interview was organized and then identified to the themes
that presented the repeating ideas.
3.6
Summary
This chapter outlines the research methodologies employed to study the
effectiveness and usability the Web-based learning system in this research. A mix of
methodologies, quantitative and qualitative methods was used to acquire different sets of
data to allow investigation of various components. In addition, method of assessment of
the proposed Web-based learning system has been discussed. The chapter also outlines
a comprehensive HOTS assessment framework in order to accomplish the objectives of
the study.
CHAPTER FOUR
SYSTEM DESIGN AND DEVELOPMENT
4.0
Introduction
Based on the discussion and analysis in the previous chapters, a conceptual
model of a Web-based learning system was proposed in this research. This system
adapts the learning object design and generative learning to provide an instructional
system design to improve the HOTS and learning. The proposed conceptual model in
the system design framework was employed to achieve this objective. The system was
designed specifically for the learning of Computer System (CS). The conceptual model
to be synthesized for the design and development of the system will be discussed in
detail in this chapter. Chapter 4 outlines necessary principles and features underlying the
conceptual model in the design and development of the Web-based learning system.
4.1
Web-Based Learning in Southern College
The Department of Computer Science at Southern College is moving towards an
instructional setting that emphasizes student-centered and active learning environments.
In line with this, the department plans to design and develop a Web-based learning
system that enables the students to learn from the Web and allows the instructors and the
students to upload as well as to share the learning materials on the Web. The key
124
strategy in this move is the design and development of Web-based learning system based
on learning object design. In addition, the appropriate pedagogical aspect that grounds
on the generative learning was pervaded into the design and development of the system.
The learning object design and generative learning which emphasize on active learning
and learner-controlled learning environment to enable students to construct and organize
their own learning, and most importantly is to improve HOTS and understanding.
With conventional teaching, the students learn through lecturing, reading, doing
assignment and group discussion. An alternative to this conventional teaching is
available at Southern College with the use of the Web-based learning system. The
subject area chosen by the researcher was Computer System, which is a first year subject
for the Diploma of Computer Science.
The learning outcomes of the subject can be described in four areas: knowledge,
understanding, application and HOTS. Upon completion of the subject, students should
be able to:
(i)
Describe the principles, concepts and techniques associated with the
computer technology.
(ii)
Be aware of the broad range of contexts in which computer systems are
used;
(iii)
Be aware of the major trends in information technology.
(iv)
Apply the principles, concepts and techniques of computer in various
contexts;
(v)
Use HOTS to:
(a) compare, contrast and critically assess the differences of the computer
hardware.
(b) analyze and refine specifications and implementations of small-scale
hardware;
(c) retrieve, critically assess and effectively use the computer technology;
(d) suggest various computer hardware to support your work;
125
The content of learning objects designed in the Web-based learning system was
validated by an expert as shown in Appendix M. The content of the subject focuses on
the computer hardware that consists of a few topics and the details of each topic is
illustrated in Appendix P:
4.2
(i)
Introduction to Computer System.
(ii)
System Unit
(iii)
Input
(iv)
Output
(v)
Storage
Design and Development of the Web-based Learning System
As discussed in Chapter 1, the objective of the research is to investigate the
effectiveness of the Web-based learning environment which incorporates the learning
object design and generative learning in the improvement of HOTS. The Web-based
learning system has the following attributes:
(i)
It encourages active, meaningful and thoughtful learning.
(ii)
It aims to improve and engage HOTS such as analysis, synthesis and
evaluation.
(iii) It has the flexibility of learning design that carries self-paced, self-directed
and self-controlled learning.
(iv) It allows reuse of the learning materials.
(v)
It encourages student-centered activities.
(vi) It aims to make the students as designers in the learning process.
(vii) It encourages students to reflect their learning.
As discussed earlier, the learning object design and Web-based learning have
great potential in e-learning. It has also been reviewed that most of the research on
126
learning object design and Web-based learning has been emphasized more on the
technological issues. Based on the discussion and analysis in the Chapter 1 and Chapter
2, this research would like to propose a conceptual model which utilizes the full
potential of the learning object design and Web-based learning, besides incorporating
generative learning from constructivism learning and emphasis on HOTS. The Webbased learning system was validated by an expert as shown in Appendix N.
A pedagogical design and development framework was proposed in this
research, namely Generative Learning Object Organizer and Thinking Tasks
(GLOOTT) model. A Web-based learning system, called as Generative Object Oriented
Design (GOOD) learning system was developed to probe the effectiveness of this
conceptual model.
GOOD learning system was designed and developed with multimedia and Webbased development tools. Multimedia software such as Macromedia Flash MX and
Adobe Photoshop were used to design and develop the learning objects as well as the
multimedia in the system.
Macromedia Flash was used to design and develop
multimedia and interactive learning objects. The information agent Mr. TQ in the
system was designed with Macromedia Flash as well. Macromedia Flash is a popular
authoring program in animations, interactive applications and applications that are crossplatform and cross browser (London and London, 2001; Muller, 2003).
Adobe
Photoshop was used to edit the graphics in the system. Adobe Photoshop is a powerful
image-editing program in creating, modifying, manipulating digital images for Web
(Reding, 2004). Both software were used to design and enhance the interface in the
system.
The Web pages in GOOD learning system were developed with Macromedia
Dreamweaver MX. Dreamweaver MX is a Web page authoring program that allows
Web site designers and developers to create, maintain and manage Web sites (Shelly,
Cashman and Vermaat, 2004).
It was used because of its ability in supporting
Macromedia Flash MX, Javascript, PHP, XML and MySQL which were used in the
127
system design and development. JavaScript was used to add the interactivity of the
system such as highlight of the navigation buttons and design message box. JavaScript
is a popular client-based Web scripting language and it can run in a browser without
additional tools on both Netscape Navigator and Internet Explorer (Murdock, 2000). It
is generally used to design dynamic Web pages (Valade, 2002).
GOOD learning system contains a Web database to store learning objects, the
users’ learning profiles and records and so on. Hence, Apache server was used as the
server for the system. It is the most widely used Web server platform, free available on
the Internet and can run on a lot of Websites (Laurie and Laurie, 1997). MySQL was
used for database management in the system. MySQL is a popular database of its speed
and small size (Valade, 2002). It is easy to use and can run on many operating systems.
PHP is the scripting language used to connect and communicate the databases in the
system. PHP is strong in its ability to interact with databases (Valade, 2002). Besides, it
can be embedded with HTML code. PHP and MySQL are frequently used together and
they are perfect to provide and support the Web database application (Valade, 2002).
XML was used in the system to design the metadata of the learning objects. XML was
used to design the metadata of learning objects due to its flexibility. XML is a format
for data structuring that allows Web developers to create one version of a Web page that
can then be displayed on various devices (Shelly, Cashman and Vermaat, 2004). The
application of XML-based metadata system can support learning and instructional
technology (Ogbuji, 2003). It is used to define the structure format of learning objects
as well as the metadata of the learning objects.
4.3
A Pedagogical Design Learning Conceptual Model: GLOOTT
As mentioned in previous chapters, the computer-based tool which is used for
effective learning should have the essential elements to promote learning.
In this
research, the proposed model was based on the attributes as mentioned in section 4.2.
The GLOOTT model was designed based on the attributes of learning object design, the
128
generative learning and HOTS. The model is not only a knowledge acquisition tool but
is also a mind tool that improves HOTS. According to Jonassen (1996), mindtools are
used by learners to represent their knowledge and engage them in HOTS. The following
points describe the prominent properties of GLOOTT:
(i)
It is a knowledge base that contains chunks of learning (learning objects) and
allows linking between the learning objects as designed in Learning Object
Repository (LOR)
(ii)
It contains a knowledge domain which is broken into small parts (learning
objects) that are flexible and reusable and stored in LOR.
(iii) It provides an environment in which the learning objects can be meta-tagged
to describe the learning materials as designed in the uploading of the learning
objects to the LOR in the system.
(iv) It provides a learning environment with multiple representation modes by
utilizing essential tools in computer as designed in the GLOO.
(v)
It provides a collaborative learning environment for knowledge sharing as
shown in GLOO and forum.
(vi) It provides a learning environment which can be controlled, assessed and
directed by the students as designed in GLOO and Reflection.
(vii) It provides an environment to generate and organize learning as shown in the
GLOO.
(viii) It provides an environment for concept construction and design as designed
in GLOO and TT.
(ix) It provides an environment to scaffold learning process that facilitates
students’ reflection and HOTS as designed in Reflection and TT.
(x)
It provides an environment for testing and reflection of the students’ own
concepts as designed in GLOO, Reflection and TT.
(xi) It provides a dynamic environment that is conducive for proactive interaction
as designed in GLOO, Reflection, TT and forum.
(xii) It provides an environment for practicing HOTS as designed in TT.
129
It is also important to note that the learning activities are learner-centered while
the learning environment is generative-oriented.
Thus, various means have been
considered in the design and development of the system to engage students in active
learning. It is commonly agreed that an active learner will integrate new knowledge
more readily than a passive learner (Lim, 2000). The students act as designers in the
active learning process. Based on these views, the GLOOTT model (refer Figure 1.1)
was proposed to be the conceptual model for the Web-based learning design and
development in this research. The suggested model is equipped with the learning object
design as its stem and essential elements in generative learning as its pedagogical
perspective to improve the HOTS as well as understanding in the subject of Computer
System.
The GLOOTT model consists of two main parts. The first part is Generative
Learning Object Organizer (GLOO), and the second part is Thinking Tasks (TT). The
design of this model was based on the generative learning, which consists of:
(i) generation of organizational relationships between different components through
concept mapping;
(ii) integration and elaboration of
knowledge through solving scenario-based
problems;
Besides, the design of this model focuses on instructional planning framework for HOTS
according to the Bloom’s Taxonomy. The cognitive operations of the HOTS in this
research are analysis, synthesis and evaluation. As discussed earlier, these thinking
skills are of utmost important to prepare the students with HOTS in learning Computer
Science. The model represents a multi-faceted, overlapping and integrative tool for
knowledge construction. This model aims to engage students in HOTS in learning
Computer System.
GLOO specifies the development of concepts and the engagement of HOTS.
Students work with learning objects that engage them actively in generating or
130
constructing the organizational relationships between the learning objects. To facilitate
generative learning, GLOO offers the students the opportunity to construct, or reconstruct their knowledge by assimilating and accommodating new knowledge schemata
with their existing frameworks. They analyze, organize, synthesize, evaluate and reflect
their learning in the learning environment. These activities follow idiosyncratic
pathways in learning and they are complementary to each other. In this context, the
students act as designers by constructing and designing their own learning through
analyzing, synthesizing, evaluating and organizing the learning objects in the Learning
Object Repository (LOR). The LOR is a computer database that contains the content of
CS that was designed as learning objects.
When the students participate in designing their learning by adapting and
organizing the learning objects, they engage themselves with HOTS. According to
Wilson (1997), constructivist learning activities do not indicate a lack of structure,
instead, some structures and disciplines are needed to provide goal-oriented
opportunities that allow and help students in constructing their learning. Hence, the
design of the learning is based on the learning objectives of CS.
Meanwhile, an
information agent will scaffold the learning process to facilitate the students in their
reflection and HOTS. A tool named Learning Object Organizer is designed to enable
and help the students to include, adapt, manipulate and organize the learning objects in
designing the hierarchical outline of the concept map, which is called as lesson map in
the system. It is based on the design of concept mapping in hierarchical outline form.
According to Alpert and Grueneberg (2000) and Dabbagh (2001), concept map can be
designed in outline form.
Lesson mapping encourages students to actively and
generatively construct, relate and organize their concepts. This allows the students to
have control over the selection of learning objects and design of learning. Besides,
lesson map allows the students to share their own conceptual understanding with
students. Stoyanova and Kommers (2002) employed the concept mapping as a medium
of shared cognition in their study. Result from the study demonstrated that it is an
effective tool for mediating computer-supported collaboration.
131
It can be seen that the main function of the GLOO is to provide the knowledge
base that engages students with HOTS through the generative learning environment,
whereas the Thinking Tasks (TT) part serves as an environment for students to test their
understandings as well as to reinforce and practice HOTS as mentioned by Costa and
Kallick (2001). There are two parts in TT, namely Try It Out and Apply It. Try It Out
contains multiple-choice questions uploaded by the instructor to assess the students’
understanding and reflect the lesson maps they have designed in GLOO. Apply It
consists of scenario-based problems that engage students with HOTS. Students need a
deeper processing of content and use of HOTS in solving the problem. It aims to assist
the students to implement what they have learned, to reflect on the learning content and
to incorporate the content into related areas.
A lot of literature and research have highlighted that reflection engages students
with HOTS. According to Fogarty (2002), reflection involves awareness and control
over one’s learning. Students will think back on what they have done and what they
need to do.
This is important to assist students in monitoring their learning and
engaging them with HOTS. In short, the GLOOTT model that is framed within the
learning object design, generative learning strategies and the emphasis on HOTS, is a
conceptual framework to improve HOTS and learning in Computer System.
4.4
GOOD Learning System
The concept of one-size-fits-all design in the Web-based learning is no longer
suitable to support learning. The learning environment should be highly flexible in
structures and promote active learning and HOTS (Deubel, 2003). Most of the LMS
systems have been built around tools sets, technical aspects and not pedagogy (Cole,
2005). Cole (2005) further pointed out that most of the commercial LMS systems are
tool-centered. The Web-based learning system should be designed based on pedagogical
aspect in order to promote higher order learning (Reeve, 1997; Bonk and Reynolds
1997; Deubel, 2003). Hence, it is important to design the Web-based learning system
132
that incorporates the pedagogical aspects in order to promote learning which focuses on
HOTS. Seen in this light, this research has designed a Web-based learning system that
includes the important components such as pedagogy aspect, instructional design and
HOTS.
The Web-based learning system designed in this research that based on
GLOOTT model is called as GOOD learning system. The learning activities in the
GOOD learning system are learner-centered whereas the learning environment is
generative-oriented. The system is not only a knowledge acquisition tool but also a
mind tool that promotes HOTS. It also guides learners to be ‘learning designers’. The
system engages learners in the construction of knowledge as they actively generate
knowledge in the form of lesson maps that are hyperlinked to the learning objects. They
not only use but also contribute to knowledge sharing by designing and uploading their
learning objects.
4.5
Structure of Learning Object Design
A learning object is defined as an electronic resource that has two main
components: metadata and resources. The metadata facilitates the search in learning
object.
From technical perspective, the learning object design comprises of the
granularity and metadata of the learning object. The learning objects in the context of
Web-based learning take form as Web pages, pdf documents, animations, graphics and
documents (Oliver, 2001).
The granularity of learning object in the Web-based learning was adapted from
the framework proposed by Wiley (2000, 2001) and Lau (2002). The types of learning
object designed in the learning are combined-closed, combined-open and Unique
Learning Objects (ULO). As discussed in Chapter 2, these kinds of learning object are
flexible and the size of the learning material allows students to generate and design their
lesson maps. The GOOD learning system supports various formats of learning objects
133
such as Web pages, documents, pdf documents, animation, graphics, audio and video.
The learning content is chunked into small units of learning object that contain one to
three learning objectives. The learning object is designed to present a single, stand
alone, whole piece of information. The learning objects in the GOOD learning system
take form as Web pages, animations and graphics. The learning objects in the system
can be the self-designed learning objects or downloaded from others repository on
Internet. Figure 4.1 to 4.2 show various formats of learning object in GOOD learning
system.
Figure 4.1: An Example of Interactive Learning Objects
134
Figure 4.2: An Example of Web Page Learning Objects
Figure 4.3: An Example of Learning Objects Designed as Table
135
Figure 4.4: An Example of Learning Objects Designed as Graphic
Meta-tagging is a way of richly describing and identifying the content of a
document through definition of the tags. The metadata enables the learning object
creators to describe their learning objects and it assists the students to search and retrieve
the specific learning material that they are looking for from the LOR. As discussed in
Chapter 2, there are various standards of metadata in learning object. The metadata used
in this research is modified from IMS Learning Resources Meta-Data (IMS, 2001). The
metadata from IMS has been used for the development of the learning object in an elearning system developed by Mahadevan (2002). According to Mahadevan (2002),
IMS has many advantages over other schemes as it comes with metadata elements for
educational information. The focus of the metadata used in the GOOD learning system
is to describe the attributes of learning objects in the system. Thus, the metadata will not
indicate the index of how the objects will be linked.
136
4.5.1
The Metadata Elements
The metadata elements used in the description of learning objects in the Webbased learning system consists of two main categories, namely information about the
authors and information about the content. Table 4.1 shows the details of the metadata
elements.
(i)
Information about the authors
The creators or designers of the learning object will be recorded. They
need to define their user type either as student or instructor.
(ii)
Information about the content
There are two parts in the information about the content. Here is where the
creators are expected to enter the basic information about the learning
objects to be uploaded into the system. The first part includes the elements
that are essential for searching the learning objects. The second part deals
with the pedagogical information. Details of the elements are:
Table 4.1: The Metadata Elements
Category
Information
author
Information
content
about
about
Metadata Element
the
• Author’s name
• E-mail address
• Department
• Organization
• User type
the Part I:
• Title
• Language
• Description
• Keywords
• Date of creation
Part II:
• Objectives
• Pre-requisites
• Interactivity type
• Learning Resource type
137
•
•
•
•
•
•
•
•
Intended user
Context of use
Sequence
Semantic density
Learning duration
Classification
Copyrights
Other remarks
(a) Objectives
The objectives of the learning object tell what the content is trying to
accomplish.
(b) Pre-requisites
Pre-requisites imply the background knowledge the students must have
in order to understand the learning objects. It measures the educational
level of the students who are trying to use the learning objects.
(c) Interactivity type
The interactivity type is an indication of the interactivity level of the
learning object. The suggested types are passive, active, mixed and
undefined.
(d) Learning resource type
Learning resources type defines the intended use of the learning object.
Examples of the learning resource type are simulation, Web page,
assessment, narrative text, animation, experiment, slide, thinking task,
exercise and graphic.
(e) Intended user
The intended user defines the main user of the learning object. They
are: students and instructor.
(f) Context of use
The context of use defines the educational level at which the intended
use of the learning object takes place. The elements are secondary
education, higher education, certificate education, diploma education,
university first cycle (I and II year), university second cycle (III and IV
year), technical school, continuing education and so on. The designers
138
of learning objects use these elements to inform the searchers of the
targetted learning objects.
(g) Sequence
The sequence defines the logical arrangement of the learning objects.
The designers of learning objects use this element to show the correct
sequence of the learning objects.
(h) Semantic density
The semantic density defines the learning usefulness as well as its high
interactivity and demonstrative nature of a learning object. There are 3
levels of semantic density, namely high, medium and low level.
(i) Learning duration
It defines the minimum time required to work with the learning object.
(j) Classification
The classification defines the content categories of the learning objects,
namely content field, subject area and its sub-topics. The content field
defines the field of the learning object such as Computer Science,
commerce, engineering and so on.
The subject area contains
information such as computer system, computer programming, system
design and development. The information under this element is based
on the selection of the content field.
(k) Copyrights
It contains information about the copyrights of the learning object.
4.6
The Design and Development of GOOD Learning System
GOOD learning system aims to stimulate the students to become active learners
and to provide tools to construct their learning as well as to improve HOTS. It contains
vast learning resources designed in learning objects to provide generative learning
environment. Morgan (1996) suggested four checkpoints to enhance learning using
technology. The checkpoints are:
139
(i)
Provide multiple exposures to variation of concepts.
(ii)
Get students actively involved.
(iii) Increase student productivity.
(iv) Use higher order thinking of Bloom’s Taxonomy.
The design and development of GOOD learning system has adhered to the checkpoints
that emphasize on the active role of students in their learning. The GOOD learning
system emphasizes on the unstructured learning process as noted in generative learning.
In order to facilitate the learning in GOOD learning system, figure 4.3 shows the
suggested flow of learning activities in system.
Student log in
Check message
from instructor
Upload self-designed
learning object
Learning
Forum
Select subject
and subtopic
Search learning
objects
Design
learning
Design lesson map
in LOO
View completed
learning
GLOO
View uploaded
lesson map
Try it Out
TT
Apply it
Reflection Corner
How am I doing checklist
Reflection worksheet
Figure 4.5: The Suggested Flow of Learning Activities in GOOD Learning System
140
GOOD learning system begins with a main page that contains the user log in and
introduction of the system (see Figure 4.6 and 4.7). The user login profiles are students
and instructors. Section 4.6.1 discusses the learning process of the students log in in
detail while section 4.6.2 will discuss the instructor log in process in GOOD learning
system.
Introduction to GOOD learning
system
User login
Figure 4.6: The Main Page of GOOD Learning System
Figure 4.7: The Introduction to GOOD Learning System
141
4.6.1
Students Log In
When the student has successfully login with their user name and password, the
system will display the home page as shown in Figure 4.8. GOOD learning system
enables students upload their self-designed learning objects (see Figure 4.10). All the
learning activities in the system is assisted and guided by an information agent that is
called Mr. TQ (see Figure 4.9).
Upload self-designed
learning object
Learning
Student name
Forum
Mr. TQ
Check message from
instructor
Figure 4.8: Home Page of Student Log In
Figure 4.9: Mr. TQ
142
The learning activities consist of uploading learning object, designing learning,
participating in the forum and checking the message from the instructor.
4.6.1.1 Upload Learning Objects
GOOD learning system enables students to upload their self-designed learning
objects. Each learning object must be tagged with the metadata and then uploaded to the
Learning Object Repository (LOR) as shown in Figure 4.10. The LOR acts as a
knowledgebase that enables students to store and expand their understanding of the
learned concept.
Describe the learning object in
metadata.
Upload learning objects.
Figure 4.10: Uploading Learning Objects of Students
143
4.6.1.2 Design Learning
The learning tasks in the design learning of GOOD learning system are designed
based on the proposed conceptual model, GLOOTT. There are three activities in the
learning tasks, namely Generative Learning Object Organizer, Thinking Tasks and
Reflection Corner. To begin the learning tasks, the users need to choose the subject and
subtopic. Then they design their learning or view completed lesson maps of the subtopic
as shown in Figure 4.11.
User can choose design
lesson or view completed
lesson.
Figure 4.11: Subject and Subtopic Selection in Learning Tasks
4.6.1.2.1 Implementation of GLOOTT Model in Learning Tasks of GOOD
Learning System
The design of learning tasks of GOOD learning system is based on the
GLOOTT model that is framed within the learning object design, generative learning
strategies and emphasis of HOTS.
The GLOOTT model consists of Generative
Learning Object Organizer (GLOO) and Thinking Tasks (TT). GLOOTT specifies the
144
development of concepts to be learnt and aims to improve HOTS among the students. In
GLOO, learners work with learning objects that engage them actively in generating or
constructing the organizational relationships among the learning objects. A learning
object is defined as a small chunk of lesson that contains one to three learning objectives
in this research (see Figure 4.1 to Figure 4.4).
To facilitate generative learning, GLOO is designed to offer learners the
opportunity to construct, or re-construct their knowledge by assimilating and
accommodating new knowledge with their existing one through concept mapping.
Confirming this idea, Bannan-Ritland, Dabbagh and Murphy (2000), Grabowski (1996),
Jonassen (2000), and Hollingworth and McLoughlin (2003) noted that concept mapping
is a generative learning strategy that facilitates the promotion of HOTS among the
learners. GLOO is a tool that is capable of representing a student’s knowledge more
comprehensively and allows the learners to learn through design.
GLOO contains
search engine, Learning Object Organizer and published lesson map as shown in Figure
4.12.
Three functions in GLOO.
Figure 4.12: GLOO Design in GOOD Learning system
145
In the learning process, the users participate in searching learning objects,
organizing learning objects and designing lesson maps. The users search the learning
objects from the LOR and the searched result will be displayed in a table that contains
description of the learning objects for the users’ preview. The learning objects selected
for the lesson will be added into the learning object library where they can be organized
as shown in Figure 4.13.
Student searches
the learning
objects from the
LOR and adds
them into the
learning object
library.
Student organizes the
learning objects in the
learning object
library.
Figure 4.13: Search Engine and Learning Objects in Library
146
The Learning Object Organizer (LOO) is a concept mapping tool that is capable
of representing the students’ knowledge more comprehensively and allows the students
to learn by designing their knowledge (see Figure 4.14). The LOO provides the
representation of lesson organization in an outline form that is called as lesson map in
this research. As mentioned by Alpert and Grueneberg (2000), and Dabbagh (2001),
concept map could be designed in an outline form. Lesson map is the cognitive chunk
of concept designed by the students.
LO organizer tool.
The
relationships
in lesson
map.
The linked
to the LO.
The crosslink shows
relationships in lesson
map.
Figure 4.14: Lesson Mapping in Learning Object Organizer
The design of LOO capitalizes the Web-based learning where students design
their lesson trough the lesson map design that contains proposition and concepts link to
various learning objects stored in the database (LOR). LOO provides a dynamic feature
where students can create the lesson map trough the use of hyperlink. The mapping of
lesson map onto the structure of a hypertext contributes to the development of the
students’ knowledge structure. Hypertext is gaining more attention in the use and
147
research of the e-learning design and development (Barab, Young and Wang, 1999).
Hypertext supports the integration of information into the students’ knowledge
structure (Jonassen and Wang, 1993).
In the hypertext design, the information is
organized into a network that contains links to various multimedia nodes. It offers
sequencing and pace control for the students and engages themselves with generative
activities (Barab, Young and Wang, 1999). Barab, Young and Wang (1999) highlighted
that hypertext is a generative learning strategy. This concept fits well with the design of
lesson map in LOO. This learning environment enables the students to decide which,
and in what sequence the learning objects should be accessed. When using the hypertext
design approach in the design of lesson maps, learners are able to control the navigation
of the lesson maps. As this occurs, the students are elevated from passive learners to
authors or designers in their learning. It appears that learners control their learning rather
than program-controlled instruction. Research form Barab, Young and Wang (1999)
showed that learners with a high degree of learning control will be more self-determined
and perform better on problem solving than the learners with less control.
Liu and Pedersen (1998) had explored whether being hypermedia designers have
an effect on the students’ motivation and their learning of knowledge.
They
demonstrated that engaging students in the design of their learning could support the
development of knowledge construction and HOTS as well as enhance their motivation.
In addition, Jonassen and Reeves (1996) noted that people who learn the most from the
design of instructional materials is the designer themselves. They further pointed out
that when students try to represent what they know, they will engage in HOTS. The
LOO which is in hypertext design fits well with the pedagogical design principle in
promoting HOTS.
Lesson mapping involves the creation of hyperlinks to the learning objects.
Learners design the lesson map that contains hyperlinks to certain learning objects to
represent relationships among the ideas. The outlines form of concept maps used in the
lesson mapping provide a hierarchical structure of concept maps based on the
148
relationships of links to various learning objects. As the learning objects and links
become interrelated in the lesson map, a structural knowledge representation depicts the
understanding of the lesson. The process of lesson mapping engages the learners to
identify the key concepts and relate them in a more meaningful way. The learners
actively construct knowledge as they form the lesson maps that contain hyperlinks to
various learning objects.
The LOO allows the students to design their lesson maps in multiple forms of
learning objects. The students can portray concrete instances of concepts by adding the
learning objects that are designed in textual or visual images. They can hyperlink the
concepts to the learning objects in the LOR or self-designed learning objects, or to other
Websites. It has been confirmed that the process of mapping the concepts in multiple
forms is more easily understood and learned (Alpert and Grueneberg, 2000). Besides,
this will also enrich their lesson maps and engage them with HOTS. This is supported
by empirical results that demonstrate the engagement of HOTS through concept
mapping activities (Dabbagh, 2001; Alpert and Grueneberg, 2000).
When the students design the lesson map, they organize the learning objects,
generate the relationship among the learning objects and assimilate the new learning
objects into their existing lesson map as shown in Figure 4.14. This process involves
generating links, relating learning objects, adapting the existing learning objects to the
new learning objects, and correcting any misconceptions in the existing lesson map.
Besides, as the students progress in the learning process, they can modify their lesson
maps by adding new nodes, refining and reorganizing the relationships among the nodes.
This assists the students to learn the concepts in a meaningful way and engages
themselves with HOTS. In this case, the learners act as designers that construct and
design their own learning through analyzing, synthesizing and evaluating the learning
objects.
149
The lesson maps of each student are then grouped in a Web page called as course
map. Course map is the collection of lesson maps for the subject as shown in Figure
4.15.
Figure 4.15: Course Map
In short, GLOO involves learners in the construction of knowledge as they
actively generate knowledge in the form of lesson maps that hyperlink to the multiple
forms of learning objects design. They not only use but also contribute to the shared
knowledge by designing and uploading their learning objects. This engages the learners
with HOTS as they socially interact contribute and shared their learning and thus giving
them ownership of their learning. GLOO can engage the students with HOTS and
enhance their learning by:
(i)
Assisting in relating and consolidating the learning objects.
(ii)
Providing an approach to note-taking which is different from the
conventional note-taking.
(iii)
Providing a way of relating, reviewing and understanding the lesson.
150
(iv)
Providing a tool for construction of existing information by developing
the new linkages among the learning objects.
(v)
Providing an approach to share and link the students’ self-designed
learning objects.
(vi)
Assisting in flexible, constructive and self-directed learning.
(vii)
Providing a multiple representation of knowledge by linking a variety
multimedia design of learning objects.
Thinking Tasks (TT) allow learners to test their understanding as well as to
reinforce and practice HOTS. There are two parts in the TT, namely Try it out (see
Figure 4.16) and Apply It (see Fig. 4.17). In the Try it out, the learners need to solve the
multiple choice questions to test their understanding whereas Apply it contains scenariobased problem solving questions that will engage the students with HOTS. The students
are required to solve the problems and upload their solutions to the system.
Figure 4.16: Try It Out
151
Figure 4.17: Apply It
The Reflection Corner acts as a self-assessment tool to help the learners in
monitoring their engagement with HOTS and reflecting their learning. Students will fill
up the “How am I doing” checklist to reflect their engagement with HOTS (see Figure
4.18).
The checklist was modified from the attributes of analysis, synthesis and
evaluation thinking from Bloom et al. (1956), Bloom, Hasting and Madaus (1971),
Jonassen (2000) and Beyer (1988). A chart showing the percentage of the use of HOTS
will be displayed in the system (see Figure 4.19).
152
Figure 4.18: “How Am I Doing” Checklist
Figure 4.19: Chart of How Am I Doing Checklist
153
In addition, the students need to fill up the self-assessment questions in the
reflection worksheet (see Fig. 4.20). The reflection worksheet, which has been modified
from Parry and Gregory (2003), allows students to do self-assessment of their progress.
Students need to answer the questions in the reflection worksheet to reflect on what they
have done and learned in order to project and decide how the learning can be
incorporated for the next learning. These questions are as follow:
(i)
What did I learn?
(ii)
What are my strengths and weaknesses?
(iii)
What did I improve?
(iv)
What do I need to consider after this lesson?
(v)
What do I need to focus on in the next lesson?
(vi)
What do I need to improve?
Fill in the reflection
worksheet.
Indicate the
condition of each
question.
Report of reflection
worksheet.
Figure 4.20: Reflection Worksheet
154
4.6.1.3 Forum
A forum was designed to enhance the interactions between the instructors and
the learners as shown in Figure 4.21. They can discuss their learning in the forum.
Learners are encouraged to post comments and suggestions about the lesson maps in the
system.
Figure 4.21: Forum
4.6.1.4 Message from the Instructor
Students can check the messages from the instructor once they log in to the home
page as shown in Figure 4.22. They can read the messages and delete them from their
message box. The message could be the comments from the instructor about their selfdesigned learning objects, lesson map, thinking tasks and others learning activities in the
system.
155
List of messages
Content of a message
Figure 4.22: The Messages from the Instructor
4.6.1.5 Others Feature That Support Learning
The learning activities in the system are assisted by an information agent called
Mr. TQ as shown in Figure 4.23. The information agent will scaffold the learning
process to facilitate and help the students to reflect their learning as shown in Figure
4.24. Besides, it also provides guidance for the users in using the system.
156
Figure 4.23: Mr. TQ, the Information Agent in GOOD Learning System
Figure 4.24: Mr. TQ Helps Students to Reflect Their Learning Tasks
4.6.2
Instructor Log In
GOOD learning system allows more functions to be accessed by instructor such
as uploading tasks, design learning and participate in the forum. The instructors act as a
facilitator in the teaching and learning process with GOOD learning system. They can
upload their self-designed learning objects or learning objects downloaded from Internet.
They can interact with students by giving comments, sending messages and monitoring
students’ learning. It is important for the instructor to keep track the students’ learning
as this system acts as a supplementary tool in the teaching and learning process.
157
4.6.2.1 Uploading Tasks
GOOD learning system enables the instructors to upload subjects with the
learning outcomes, subtopics with learning objectives, learning objects and thinking
tasks generator for each subtopic of the subject. Instructors are allowed to perform three
uploading tasks such as subject and subtopics, learning objects and thinking tasks as
shown in Figure 4.25.
Instructor login with
three upload tasks.
Add new subject and
subtopics of the subject.
Figure 4.25: Upload Subject and Subtopics
158
For learning objects upload, each learning object must be tagged with the metadata and
then uploaded to the Learning Object Repository (LOR) as shown in Figure. 4.7.
However, instructor is allowed to check and delete the irrelevant learning objects
uploaded by the students. Instructor can message the student to inform them about the
deleted learning objects or some comments on the learning objects as shown in Figure.
4.26.
Instructor can send messages to the
student about his/her self-designed
learning objects.
Figure 4.26: Message to the Student
The thinking task generator has two functions. One is a template to assist
instructors to design the multiple-choice questions for Try It Out and another enables
instructor to upload the scenario-based problem solving questions in various formats for
Apply It (see Figure 4.27).
159
Multiplechoice
questions
generator of
Try It Out.
Upload
questions of
Apply It.
Figure 4.27: Thinking Task Generator
Instructor has the same access to the design learning and forum with the students as
illustrated from Figure 4.11 to 4.21.
4.7
GOOD Learning System as Mindtool to Engage Learners in HOTS
As discussed in previous sections, GOOD learning system acts as a mindtool to
engage and improve HOTS.
Jonassen (1996) noted that computers are used as
mindtools by employing application software for knowledge construction and
160
representation which facilitates meaningful learning and encourage HOTS. According
to Jonassen (1996), and Jonassen and Reeve (1996), technology can be used as
mindtools by providing learning environment that contains the features as follow:
(i)
Learners as designers
Mindtool requires the learners to design their learning by thinking and
generating ideas for the subject they are learning. They need to employ
HOTS as they construct their own ideas and design their own knowledge
bases.
(ii)
Knowledge construction
Mindtool is a constructivist use of technology that enables the learners to
construct their knowledge.
They develop their knowledge bases by
constructing they own conceptualization of the organization of the subject.
This engages them with generative learning environment where they need
to construct their knowledge based on their prior knowledge. They are
actively participating in the learning environment.
(iii)
Learning with technology
Learning with technology indicates the intellectual partnership with the
learners in their learning. In this situation, the capabilities of technology
are used to enhance thinking and learning.
(iv)
Unintelligent tools
Mindtool is an unintelligent tool that relies on the learners to provide the
intelligence but not the computer. In other words, planning, designing,
reflection and self-regulation of learning are responsibilities of the learners.
Technology merely serves as a tool to facilitate these skills.
(v)
Distributed cognitive processing
Computer acts as a tool to share the cognitive load of the learners such as
calculation and information storage, access and retrieval. Whereas the
learners are responsible in employing thinking skills to design and
construct their learning.
161
From the descriptions mentioned by Jonassen (1996), and Jonassen and Reeve (1996), it
is clear that the design of GOOD learning system fits well with the concept in which
computer technology is used as a mindtool to engage learners with HOTS in order to
encourage meaningful learning and HOTS.
4.8
Summary
This chapter discussed the design and development of GOOD learning system
that was based on the learning objects, generative learning strategy and the emphasis on
HOTS. The conceptual model, GLOOTT model, provides a pedagogically-enriched
model to the learning environment of GOOD learning system. The discussion has
included the implementation of GLOOTT model in GOOD learning system and how it
was designed in the Web-based learning system to improve HOTS and learning. The
design and development of GOOD learning system were aimed to provide a learning
environment that promotes the students as designers in their learning process.
Moreover, it provides an opportunity for the students to extend their learning time and to
share their learning. The effectiveness of GOOD learning system was described in
Chapter 5.
CHAPTER FIVE
DATA ANALYSIS AND RESULTS
5.0
Introduction
This chapter presents the data analysis and results of the research that aims to
answer the research questions stated in Chapter 1. The results can be categorized into
two major parts, namely the preliminary study before the design and development of
GOOD learning system and the study after the design and development of GOOD
learning system.
5.1
Data Analysis of Students’ Current Level of HOTS From the Conventional
Teaching and Learning of Computer System
An analysis had been conducted to analyze 64 students’ answers in the
examination of Computer System (CS) for Diploma in Computer Science. The rubric of
HOTS evaluation which was based on Bloom’s taxonomy, Knowledge (K),
Comprehension (C), Application (App), Analysis (Ana), Synthesis (S) and Evaluation
(E) was used to assess the level of HOTS (see Appendix A). The rubric was modified
with permission from Hansen (2001). Table 5.1 shows the division of the questions
based on Bloom’s taxonomy.
163
Table 5.1: Division of Questions According to the Bloom Taxonomy of Thinking
Taxonomy of Thinking
K
C
App
Ana
S
E
Total
1
1
1
1
1
1
1
6
2
1
1
1
1
1
1
6
3
1
1
1
1
1
1
6
Total of questions
3
3
3
3
3
3
18
Percentage (%)
16.67
16.67
16.67
16.67
16.67
16.67
100
No. of Question
There were three essay questions in the examination. Six questions under each essay
question and each question represented different cognitive operation from Bloom’s
taxonomy. There were three questions of LOTS (K, C and App) and three questions of
HOTS (Ana, S and E) for each essay question.
Table 5.2 shows the mean score of students for each essay question of each
cognitive operation.
The mean scores show the level of each cognitive operation of
Bloom’s Taxonomy for each essay question.
Figure 5.1 depicts the level of each
cognitive operation based on Bloom’s Taxonomy.
Table 5.2: Mean Score of Each Taxonomy of Thinking for Each Question
Taxonomy of Thinking K
C
App
Ana
S
E
No. of Question
1
2.69
2.14
1.67
1.13
0.86
0.48
2
1.83
1.34
0.94
0.39
0.20
0.08
3
1.31
0.83
1.84
1.59
1.41
0.56
164
Mean Score of Each Cognitive Operation for Each Question
4.00
mean score
3.00
Q1
Q1
Q2
2.00
Q3
Q1
Q2
Q3
Q3
Q1
Q2
Q3
1.00
Q3
Q1
Q3
Q1
Q2
Q2
Q2
0.00
K
C
App
Ana
S
E
Q1
Cognitive Operation from Bloom's Taxonomy
Q2
Q3
Figure 5.1: Chart of the Mean Score in the Taxonomy of Thinking for Each
Question
From Table 5.2 and Figure 5.1, it was found that most of the students had higher mean
scores of the LOTS compare to HOTS for each essay question.
Table 5.3 shows the students’ scores and percentages of cognitive operations for
all the questions analyzed and checked by the researcher. Besides, the analysis was also
rechecked by a researcher who has experience in using rubric. Figure 5.2 depicts the
chart for the data in Table 5.3.
Table 5.3: The Students’ Scores and Percentage in the Taxonomy of Thinking of
All Questions
Taxonomy of Thinking K
C
App
Ana
S
E
No. of Question
1
172
137
107
72
55
31
2
117
86
60
25
13
5
3
84
373
49
53
276
36
118
285
37
102
199
26
90
158
21
36
72
9
Total
Percentage (%)
165
Percentage of the Total Score of Each Cognitive Operation
for All Questions
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
K
C
App
Ana
S
E
Percentage
Figure 5.2: Percentage of the Total Score for the Taxonomy of Thinking of All
Questions
From the data shown in Table 5.3 and Figure 5.2, the cognitive operation with
the highest percentage is knowledge which is 49 % and the lowest is Evaluation which is
only 9%. The score of the three cognitive operations of HOTS are lower than LOTS.
5.2
Analysis of the Effectiveness of GOOD Learning System in the
Improvement of Students’ Learning
The pretest and posttest were conducted to identify the effectiveness of the Webbased learning system in improving the students’ learning. There were 30 students in
the pretest and posttest. Table 5.4 displays the score of pretest and posttest for each
student.
166
Table 5.4: Score of Pretest and Posttest
Student
Pretest
Posttest
Student
Pretest
Posttest
P1
22
58
P16
18
58
P2
60
82
P17
51
65
P3
21
60
P18
39
55
P4
13
49
P19
36
67
P5
32
69
P20
39
67
P6
50
74
P21
15
53
P7
19
56
P22
44
65
P8
40
71
P23
53
72
P9
47
74
P24
47
69
P10
54
71
P25
43
67
P11
25
54
P26
39
64
P12
64
81
P27
44
65
P13
43
60
P28
40
69
P14
17
53
P29
51
74
P15
26
58
P30
40
64
With reference to Table 5.4, the lowest scores of pretest and posttest are 13 and
49 respectively.
The highest score of pretest is 64 whereas the posttest is 82. The
scores of posttest are higher than pretest for all students. The table about the scores of
LOTS and HOTS for each student is illustrated in Appendix Q. Comparison of the
results of pretest and posttest shows that all students’ scores had been improved in
learning CS. Table 5.5 shows the difference of mean scores for pretest and posttest.
Mean score of pretest (38) is lower than posttest (65).
Table 5.5: Mean Scores of Pretest and Posttest
Test
N
Mean
Pre
30
38
Post
30
65
167
The normality of the scores distribution was evaluated using software called
JMP as demonstrated in Appendix R. The analysis shows that the distribution of the
scores fits the normal distribution quite well. JMP is statistics software provides statistic
analysis to address scientific needs of students especially for the research students (JMP,
2006). It can be used to analyze the normal distribution of sample (McMurry, 1992). The
paired-samples T test was employed to examine the significant difference between the
mean scores of pretest and posttest. Table 5.6 shows the analysis from the T-test. The
results indicate that there are significant differences (sig. value = 0.000) between the
mean scores of pretest and posttest in 95% of confidence interval.
Table 5.6: T-Test Analysis for Mean Scores of Pretest and Posttest
Test
N
Sig.
Pair Pretest – Posttest
30
0.000
Confidence Interval = 95%
5.3
Analysis of the Effectiveness of GOOD Learning System in the
Improvement of HOTS
Analysis of paired-samples T test was used to study the differences of HOTS
between pretest and posttest. The scores of both tests were analyzed based on each
cognitive operation of Bloom’s Taxonomy. There were three essay questions containing
the six sub-questions and each represents the six cognitive operations. The analysis of
students’ answers in pretest and posttest were done based on the Rubric of HOTS
Evaluation. Table 5.7 shows the results of the analysis.
168
Table 5.7: T-Test Analysis of Mean Scores for Each Cognitive Operation in Pretest
and Posttest.
Cognitive operation K
C
App
Ana
S
E
Test
Pretest
2.50
1.99
1.46
1.31
1.17
0.66
Posttest
3.43
3.04
2.76
2.54
2.13
1.71
Significance
0.000
0.000
0.000
0.000
0.000
0.000
N = 30; Confidence Interval = 95%
Figure 5.3 shows the histogram for the results from Table 5.7. Comparison of
the mean score of each cognitive operation for pretest and posttest showed the
improvement of students in both LOTS and HOTS. The mean score of each cognitive
operation for the posttest was significantly higher than the pretest which indicated
positive improvement in HOTS and LOTS. The figure shows the improvement of
LOTS and HOTS in pretest and posttest is quite consistent through the pattern of the
figure. The highest mean score is K for the pretest (2.50) as well as posttest (3.43).
The lowest mean score is E for the pretest (0.66) as well as posttest (1.71). The highest
improvement between the pretest and posttest is App (1.30) and the lowest improvement
is K (0.93).
169
4.00
3.50
Mean Score
3.00
2.50
Pre-Test
2.00
Post-Test
1.50
1.00
0.50
0.00
K
C
App
Ana
S
E
Cognitive Operation of Bloom's Taxonomy
Figure 5.3: Mean Scores of Cognitive Operations for Pretest and Posttest
5.4
Analysis of the Effectiveness of GOOD Learning System in the Students’
HOTS Engagement
The analysis of the students’ engagement of HOTS in using the system was
based on the fourth sample group which consisted of 2 students from the very active
group, 5 students from the active group and 3 students from the less active group.
Analysis of the HOTS engagement was based on the progress of HOTS engagement in
the use of GOOD learning system. Data had been collected from the students’ portfolios
that contained their responds of “How am I Doing” checklist. The frequency of the
responds to the “Yes” and “No” was collected and the frequency of HOTS was
calculated in percentage for each topic. The data analysis is tabulated in Table 5.8.
170
Table 5.8: The HOTS Engagement of Students from Different Groups
Group Student
Very
Active P12
Chapter
One
Two
Three
Four
Five
P29
One
Two
Three
Four
Five
Active
P2
One
Two
Three
Four
Five
P9
One
Two
Three
Four
Five
P21
One
Two
Three
No.
of
Checklist Analysis
Synthesis Evaluation
First
Second
Third
First
Second
First
First
First
100%
100%
100%
100%
100%
100%
100%
100%
80%
100%
100%
100%
100%
100%
100%
100%
60%
80%
100%
80%
100%
100%
100%
100%
First
Second
Third
Fourth
First
Second
First
First
First
100%
100%
100%
100%
100%
100%
100%
100%
100%
40%
80%
100%
100%
80%
100%
100%
100%
100%
40%
80%
100%
100%
60%
100%
100%
100%
100%
First
Second
First
Second
First
First
First
100%
100%
100%
100%
100%
100%
100%
80%
100%
100%
100%
100%
100%
100%
60%
80%
80%
100%
100%
100%
100%
First
Second
First
Second
First
First
First
80%
100%
80%
100%
100%
100%
100%
40%
100%
80%
100%
100%
100%
100%
60%
80%
60%
100%
100%
100%
100%
First
Second
First
Second
First
80%
100%
80%
100%
100%
80%
100%
80%
100%
100%
60%
80%
80%
100%
100%
171
Four
Five
First
First
100%
100%
100%
100%
100%
100%
One
First
Second
Third
First
Second
First
First
First
100%
100%
100%
100%
100%
100%
100%
100%
80%
80%
100%
80%
100%
100%
100%
100%
80%
100%
100%
100%
100%
100%
100%
100%
First
Second
First
Second
Third
First
First
100%
100%
100%
100%
100%
100%
100%
80%
100%
100%
100%
100%
100%
100%
60%
80%
60%
80%
100%
100%
100%
First
100%
100%
100%
Three
Four
Five
First
Second
First
Second
First
First
First
80%
100%
100%
100%
100%
100%
100%
60%
80%
80%
100%
80%
100%
100%
40%
60%
60%
80%
100%
100%
100%
P19
One
Two
Three
Four
Five
First
First
First
First
First
100%
100%
100%
100%
100%
80%
80%
100%
100%
100%
60%
80%
80%
80%
100%
P30
One
First
Second
Third
First
Second
First
First
First
80%
100%
100%
100%
100%
100%
100%
100%
60%
80%
100%
80%
100%
100%
80%
100%
60%
60%
80%
60%
80%
80%
80%
100%
P24
Two
Three
Four
Five
P27
One
Two
Three
Four
Chapter
Five
Less
Active
P18
One
Two
Two
Three
Four
Five
172
There were totally 74 records of checklist from the 10 students. The number of
the records varied among the students. The maximum number of record was 9 whereas
the minimum was 5.
There were 5 students who had 7 records, 4 students with 8
records and one student with 5 records of checklist in their portfolios. Comparison of the
results of each checklist from each student shows that all students had improved in the
engagement of HOTS after the use of GOOD learning system. The results also showed
that all the students engaged in HOTS after a few entries of the checklist. Similar
indication was reflected in each chapter. Figures 5.4 and 5.5 depict the progressive
engagement of HOTS for each student in the very active group. Figures 5.6 to 5.10
depict the progressive engagement of HOTS for each student in the active group
whereas figures 5.11 to 5.13 for each student in the less active group.
100%
Ana
Ana S
Ana S
E
Ana S
Ana S
E
Ana S
E
Ana S
E
Ana S
E
90%
80%
S
E
E
70%
E
60%
50%
40%
30%
20%
10%
0%
First
Second
Chapter One
Third
First
Second
Chapter Two
First
First
First
Chapter Three
Chapter Four
Chapter Five
P12
Ana
S
E
Figure 5.4: The Engagement of HOTS from Each Chapter for Student P12
173
100%
Ana
Ana
Ana S E
Ana S E
Ana
Ana S E
Ana S E
Ana S E
Ana S E
90%
S E
80%
S
70%
E
60%
50%
S E
40%
30%
20%
10%
0%
First
Second
Third
Fourth
First
Chapter One
Second
Chapter Two
First
First
First
Chapter Three
Chapter Four
Chapter Five
P29
Ana
S
E
Figure 5.5: The Engagement of HOTS from Each Chapter for Student P29
100%
Ana
Ana S
Ana S
Ana S
E
Ana S
E
Ana S
E
Ana S
E
90%
80%
S
E
E
70%
E
60%
Ana
S
E
50%
40%
30%
20%
10%
0%
First
Second
Chapter One
First
Second
Chapter Two
First
First
First
Chapter Three
Chapter Four
Chapter Five
P2
Figure 5.6: The Engagement of HOTS from Each Chapter for Student P2
174
Ana S
100%
Ana S
E
Ana S
E
Ana S
E
Ana S
E
90%
80%
Ana
E
Ana S
70%
E
60%
E
50%
40%
S
30%
20%
10%
0%
First
Second
First
Chapter One
Second
Chapter Two
First
First
First
Chapter Three
Chapter Four
Chapter Five
P9
Ana
S
E
Figure 5.7: The Engagement of HOTS from Each Chapter for Student P9
Ana S
100%
Ana S
E
Ana S
E
Ana S
E
Ana S
E
90%
80%
Ana S
E
Ana S
E
70%
E
60%
50%
40%
30%
20%
10%
0%
First
Second
Chapter One
First
Second
Chapter Two
First
First
First
Chapter Three
Chapter Four
Chapter Five
P21
Ana
S
E
Figure 5.8: The Engagement of HOTS from Each Chapter for Student P21
175
100%
Ana
Ana
E
Ana S
E
Ana
E
Ana S
E
Ana S
E
Ana S
E
Ana S
E
90%
S
80%
E
S
S
70%
60%
50%
40%
30%
20%
10%
0%
First
Second
Third
First
Second
Chapter One
Chapter Two
First
First
First
Chapter Three
Chapter Four
Chapter Five
P24
Ana
S
E
Figure 5.9: The Engagement of HOTS from Each Chapter for Student P24
100%
Ana
Ana S
Ana S
Ana S
Ana S
E
Ana S
E
Ana S
E
Ana S
E
90%
80%
S
E
E
70%
E
60%
E
50%
40%
30%
20%
10%
0%
First
Second
Chapter One
First
Second
Third
Chapter Two
First
First
First
Chapter Three
Chapter Four
Chapter Five
P27
Ana
S
E
Figure 5.10: The Engagement of HOTS from Each Chapter for Student P27
176
Ana
100%
Ana
Ana S
Ana
E
Ana S
E
Ana S
E
90%
80%
Ana
S
S
E
S
70%
60%
S
E
E
50%
E
40%
30%
20%
10%
0%
First
Second
First
Chapter One
Second
Chapter Two
First
First
First
Chapter Three
Chapter Four
Chapter Five
P18
Ana
S
E
Figure 5.11: The Engagement of HOTS from Each Chapter for Student P18
100%
Ana
Ana
Ana
S
Ana
S
Ana
S
E
90%
80%
S
S
E
E
E
70%
E
60%
50%
40%
30%
20%
10%
0%
First
First
First
First
First
Chapter One
Chapter Two
Chapter Three
Chapter Four
Chapter Five
P19
Ana
S
E
Figure 5.12: The Engagement of HOTS from Each Chapter for Student P19
177
Ana
100%
Ana S
Ana
Ana S
Ana S
Ana
Ana S
E
90%
80%
Ana
S
E
S
E
E
S
E
70%
60%
S
E
E
E
50%
40%
30%
20%
10%
0%
First
Second
Third
Chapter One
First
Second
Chapter Two
First
First
First
Chapter Three
Chapter Four
Chapter Five
P30
Ana
S
E
Figure 5.13: The Engagement of HOTS from Each Chapter for Student P30
5.5
Analysis of the Effectiveness of GOOD Learning System as Perceived by
Lecturers
GOOD learning system was also evaluated by 12 lecturers who have experience
in using computer in their teaching from various departments (Computer Science,
Electrical and Electronic Engineering, English Language, Malay Language and
Mandarin Language, Commerce) of Southern College. Among the 12 lecturers, they are
3 lecturers who had the experience in teaching CS. The Web-based Evaluation Form
(WEF) consists of 6 sections that contained 36 items. The different items were rated
from scale 1 that represents “strongly disagree” to scale 5 that represents “strongly
agree”. An analysis of mean and standard deviation of each item and each section is
tabulated from Table 5.9 to Table 5.14.
178
Table 5.9: Mean and Standard Deviation of Each Item in Section A
A. Evaluation of the Learning Object (LO)
content quality
Mean
Standard Deviation
4.08
0.79
4.00
0.95
detail.
4.17
0.83
4. Content relevant to age group curriculum.
4.25
0.45
5. Content of sufficient scope and depth.
3.83
0.72
4.25
0.62
1. The content is free of error.
2. The content is presented without bias that
could mislead learners.
3. Presentation emphasizes key points and
significant ideas with appropriate level of
6. Variety of information, with options for
increasing complexity.
Table 5.10: Mean and Standard Deviation of Each Item in Section B
B. Organization of content
Mean
Standard Deviation
4.50
0.52
4.50
0.52
4.50
0.52
4.42
0.51
4.25
0.62
4.75
0.45
1. Learning objectives are declared, either
within content accessed by the learner or in
available metadata.
2. Learning objectives are appropriate for the
intended learners.
3. The learning content from LOs align with
the declared objectives.
4. The LO is sufficient in to enable learners
to achieve the learning objectives.
5. The LO is able to be used in varying
learning context and within learners from
different background.
6. The LO allows learners to generate and
design their learning.
179
Table 5.11: Mean and Standard Deviation of Each Item in Section C
C: Presentation Design
Mean
Standard Deviation
4.42
0.51
4.50
0.52
4.08
0.79
4.33
0.65
4.25
0.45
4.17
0.58
1. The design of visual information efficient
mental processing.
2. Text is legible and graphics are
abelled.
3. The multimedia design aids learning and
aesthetically pleasing.
4. The design does not overload learners’
memory.
5. There is consistency in the functional use
of colors, text format and layout.
6. GOOD learning system includes the site
map.
Table 5.12: Mean and Standard Deviation of Each Item in Section D
D: Pedagogical Parameters
Mean
Standard Deviation
4.33
0.49
4.42
0.51
4.58
0.51
4.67
0.65
0.49
0.49
1. GOOD learning system allows learners to
generate learning through active exploration.
2. GOOD learning system uses appropriate
tools during the learning to get students to
think and reflect.
3. GOOD learning system facilitates learning
by designing and doing.
4. GOOD learning system stimulates and
encourages learners in analysis, synthesis and
evaluation thinking (HOTS).
5. The design of GOOD learning system
incorporates the activities of analysis,
synthesis and evaluation (HOTS).
180
6. GOOD learning system provides tasks that
enable learners to improve and practice their
HOTS.
4.58
0.51
Table 5.13: Mean and Standard Deviation of Each Item in Section E
E: Motivation and User Control
Mean
Standard Deviation
4.25
0.45
learning activities.
4.58
0.51
3. Feedback shows learners’ performance.
4.50
0.67
4.42
0.67
4.58
0.51
4.58
0.51
1. The LO is highly motivating and its
content is relevant the intended learners.
2. GOOD learning system offers active
4. GOOD learning system can be used by
learner alone.
5. GOOD learning system allows learners to
control and manage their learning.
6. GOOD learning system permits learners to
share their learning.
Table 5.14: Mean and Standard Deviation of Each Item in Section E
F: Evaluation on the Interaction Usability
Mean
Standard Deviation
4.17
0.39
4.00
0.85
4.42
0.51
4.25
0.45
1. The user interface design implicitly
informs learners how to interact with GOOD
learning system.
2. There are clear instructions guiding use
from Mr. TQ.
3. Navigation through GOOD learning
system is easy and intuitive.
4. The behavior of the user interface is
consistent.
181
5. The interface design is easy to remember.
4.33
0.65
4.50
0.52
6. GOOD learning system is flexible and
allows learners to access all its content.
Based on Table 5.9 to Table 5.14, an analysis of mean and standard deviation of
each section is tabulated as shown in Table 5.15.
Table 5.15: Means and Standard Deviations of Each Item in WEF for the Lecturers
Section
Mean
Standard Deviation
A: Learning Object Content Quality
4.10
0.73
B: Organization of Content
4.49
0.53
C: Presentation Design
4.29
0.59
D: Pedagogical Parameters
4.49
0.53
E: Motivation and User Control
4.49
0.56
F: Interaction Usability
4.28
0.59
Average
4.35
0.59
N= 12 lecturers
Table 5.15 shows encouraging results as all the means were higher than 4. The
highest mean was 4.49 for pedagogical parameters, organization of content and
motivation and user control. The lowest mean was 4.10 for learning object content
quality.
Generally, the responses for every section were highly positive. This is
indicated by the average means of 4.35 and average of standard deviation of 0.59. It is
obvious that there were positive responses from lecturers.
Most lecturers seem to
strongly agree that GOOD learning system has the following attributes:
(i)
Organization of Content (Mean = 4.49)
(ii)
Pedagogical Parameters (Mean = 4.49)
(iii)
Motivation and User Control (Mean = 4.49)
182
The Web-based Evaluation Form also included seven open-ended questions. These
questions were administered so as to get detailed feedback on the GOOD learning
system. There were seven questions:
(i)
Explain some of the aspects in the Web-based learning system that you
like.
(ii)
Explain some of the aspects in the Web-based learning system that you do
not like.
(iii) What are your suggestions (if any) to improve the Web-based learning?
(iv) Do
you
think
the
Web-based
learning
system
is
suitable
in
developing/improving the HOTS? Why?
(v)
Do you think the Web-based learning system is suitable in teaching and
learning for Computer System? Why?
(vi) Do you think the Web-based learning system is suitable in teaching and
learning for other subjects? What are the subjects (if any)? Why?
(vii) Would you recommend others (instructors/students) to use this Web-based
learning system? Why?
Analysis of the responses is demonstrated through the identification of the theme from
Table 5.16 to Table 5.22. Table 5.16, 5.17, 5.18, 5.19, 5.20, 5.21, and 5.22 show the
findings of each open-ended question in the WEF.
Table 5.16 shows most of the lecturer like the aspects of the GOOD learning
system such as it provides active learning environment, collaborative learning
environment, self-assessment, lesson map design and learning by designing. Table 5.17
shows the aspect that most of the lecturers dislike such as simple and poor interface
design in the system.
183
Table 5.16: Aspects They Like in GOOD Learning System.
Item
1
Theme
Active learning environment
Example of Feedback
(i)
Learning through active learning
activities.
(ii)
The learning system follows the
constructivism learning where the
learning is student-centered.
2
Collaborative learning
(i)
environment
3
Self-assessment
Learner can share their learning
among themselves.
(ii)
Learners can discuss in the forum.
(i)
Learners
can
self-assess
their
monitor
their
learning.
(ii)
Learners
can
progress.
4
5
Lesson map design
Support learning by designing
(i)
Lesson map design in the system.
(ii)
Self-organizing lesson map.
(i)
Students can design their own notes
in the lesson map.
(ii)
Students are encouraged to design
their own learning.
Table 5.17: Aspects They Don’t Like GOOD Learning System.
Item
1
2
Theme
Simple Interface Design
Poor Interface Design
Example of Feedback
(i)
The interface is too simple.
(ii)
The design is not very attractive.
(i)
The font is too small.
(ii)
No clear instruction for users.
184
Table 5.18 shows the suggestions from the lecturer in the system improvement.
The suggestions are interface improvement and provide instruction in the system. Table
5.19 shows the positive comments of the lecturers about the suitability of GOOD
learning system in HOTS improvement.
Most of the lecturers agreed that GOOD
learning system can improve the HOTS because it supports thinking activities in the
learning process.
Table 5.18: Suggestions to Improve GOOD Learning System.
Item
1
Theme
Improve interface
Example of Feedback
(i)
Improve the GUI.
(ii)
More
attractive
and
flexible
navigation.
2
Provide instruction
(i)
Provide more interactive tutorial.
(ii)
Provide the keywords in lesson map
Table 5.19: The Suitability of GOOD Learning System in Improving HOTS.
Item
1
Theme
Yes
Example of Feedback
(i)
Why?
(i) Support thinking activities
It helps students to actualize their
learning and thinking.
(iii)
Thinking and problem solving skills
are required in the system.
Table 5.20 shows the opinions of the lecturers about the suitability of GOOD
learning system in the teaching and learning of CS. Some of the lecturers agreed but
some are not sure about that because they are not familiar with the subject. Table 5.21
shows the positive feedback from the lecturers about the suitability of GOOD learning
system the teaching and learning of other subjects. All the lecturers agreed that the
system can be used in other subjects because the system supports self-learning and
promote conceptual understanding. Most of them agreed that the system can be used in
185
teaching and learning of other domains such as languages, commerce and electronic and
electrical engineering. Table 5.22 shows that all the lecturers agreed to recommend the
system to others instructor and students. Most of their reasons are because the system
supports active learning, supports collaborative learning, provides learner-centered
learning, encourage thinking and improve conceptual understanding.
Table 5.20: The Suitability of GOOD Learning System in the Teaching and
Learning of Computer System.
Item
1
Theme
Example of Feedback
Yes
(i)
It involves a lot of computer skills from the beginning
stage to the last stage.
(ii)
The CS is a subject with rapid knowledge updates.
Learner may share the latest information with the
others.
2
Not Sure
(i)
I’m not sure.
(ii)
Probably yes.
Table 5.21: The Suitability of GOOD Learning System the Teaching and Learning
of Other Subjects.
Item
1
Theme
Yes
Example of Feedback
(ii)
Some English subjects that need
What subject?
HOTS. The system can be used in
(i)
learning
Most of the subjects.
English
writing.
The
students are responsible for their own
learning and the lecturers can check
Why?
(i)
Support self learning
what they have learned.
(ii)
Promote conceptual (iii)
The system should be suitable for
understanding
most
of
the
subjects
such
as
commercial, social science, languages
and art subjects.
186
It is apparent that the lecturers were impressed by the features in GOOD learning
system such as active learning, flexibility lesson mapping, resources sharing and made
students to think.
5.6
Analysis of the Effectiveness of the Web-based Learning System as
Perceived by Students
At the end of the learning session using GOOD learning system, Web-based
evaluation was conducted to obtain the feedback of the students. The Web-based
Evaluation Form consisted of 5 sections that contained 30 items. The different items
were rated from scale 1 “strongly disagree” to scale 5 represents “strongly agree”. The
analysis of mean and standard deviation of each item for each section is tabulated in
Table 5.22 to Table 5.26.
Table 5.22: Mean and Standard Deviation of Each Item in Section A
A. Technology and Technical Factors
Mean
Standard Deviation
3.80
0.81
4.03
0.67
4.10
0.76
3.97
0.61
3.87
0.57
1. The Web-based learning system is free
of programming error that affects my
navigation.
2. The Web-based learning system can be
used without a very high-end computer.
3. The Web-based learning system allows
me to store my works.
4. The multimedia used in the Web-based
learning system does not affect the
download time of the system.
5. I am satisfied with the quality of
multimedia used in the Web-based learning
system.
187
6. The Web-based learning system allows
me to upload and store my works.
4.33
0.66
Table 5.23: Mean and Standard Deviation of Each Item in Section B
B: Presentation Design
Mean
Standard Deviation
assists the learning processing.
4.03
0.76
2. Text is legible.
3.97
0.67
3. Graphics are labelled.
4.20
0.48
3.83
0.75
4.10
0.66
4.03
0.76
1. The design of multimedia information
4. The design does not overload my
memory.
5. There is consistency in the text format
and layout.
6. The Web-based learning system
includes the site map.
Table 5.24: Mean and Standard Deviation of Each Item for WEF Student in
Section C
C: Learning Strategy
Mean
Standard Deviation
4.23
0.63
3.90
0.66
4.20
0.61
4.27
0.64
1. The Web-based learning system allows
me to learn through active exploration.
2. The Web-based learning system uses
appropriate tools during the learning to get
me to reflect.
3. The Web-based learning system
facilitates learning by doing.
4. The Web-based learning system
stimulates and encourages me in analysis,
synthesis and evaluation thinking (HOTS).
188
5. The Web-based learning system
provides thinking tasks that enable me to
improve HOTS.
4.17
0.65
3.97
0.76
6. Learning strategies used in the Webbased learning system enable better
understanding on the concepts I learned.
Table 5.25: Mean and Standard Deviation of Each Item in Section D
D: Motivation and User Control
Mean
Standard Deviation
3.63
0.61
active learning activities.
4.00
0.79
3. Feedback shows my performance.
4.03
0.61
learning.
4.13
0.63
5. I can control and manage my learning.
4.03
0.67
4.37
0.61
1. Learning activities in the Web-based
learning system are highly motivating.
2. The Web-based learning system offers
4. Design of the Web-based learning
system promotes the ownerships of my
6. The Web-based learning system allows
me to share my learning.
Table 5.26: Mean and Standard Deviation of Each Item in Section E
E: Interaction Usability
Mean
Standard Deviation
3.80
0.76
information agent (Mr. TQ).
3.80
0.71
3. Navigation through the Web-based
4.03
0.72
1. The user interface design implicitly
guides me how to interact with the Webbased learning system.
2. There are clear instructions from the
189
learning system is easy and intuitive.
4. The user interface is consistent.
4.00
0.64
4.07
0.74
4.07
0.74
5. The interface design is easy to
remember.
6. The Web-based learning system is
flexible and allows me to access all its
content.
Based on the Table 5.22 to Table 5.26, the analysis of mean and standard
deviation of each section is tabulated in Table 5.27.
Table 5.27: Means and Standard Deviations of Each Item in WEF for the Students
Section
Mean
Standard Deviation
A: Technology and Technical Factors
4.02
0.70
B: Presentation Design
4.03
0.69
C: Learning Strategy
4.12
0.67
D: Motivation and User Control
4.03
0.68
E: Interaction Usability
3.96
0.72
Average
4.03
0.69
N= 30 students
The highest mean is 4.12 for learning strategy in the GOOD learning system.
The lowest mean is 3.96 for interaction usability design in the GOOD learning system.
The feedback from the students on the system was highly positive. This is indicated by
the average mean of 4.03 and average of standard deviation of 0.69. Most of the
students seemed to agree that the GOOD learning system has good design attributes in
the five sections.
The Web-based Evaluation Form also contained open-ended questions. These
questions were administered so as to get comments from the students on the GOOD
190
learning system. Besides, qualitative data was collected from the comments of the
students in WEF. The data was analyzed based on the identification of theme as
demonstrated in Table 5.28 to 5.29
Table 5.28: Comments from the Students About GOOD Learning System.
Item
1
Theme
The system provides active
Example of Feedback
(iii)
learning
2
The system encourages HOTS
The system provides a lot of
activities and we need to be active.
(iii)
The system reminds me about the
use of HOTS in my learning.
(iv)
I always check my HOTS when I
use the system.
3
The system promotes
(iii)
understanding
The
system
improves
my
knowledge of CS.
(iv)
The system improves my learning
of CS.
4
The system has technical error
(iii)
The system contains programming
errors in lesson map.
5
The system supports learning
(iv)
The network is very slow.
(iii)
I can design my note in lesson map
by doing.
in hyperlink format.
(iv)
I need to do a lot of activities in the
system when I learn SC.
6
The system is less motivated
(i)
I have to finish a lot of tasks in the
system and I feel bored.
(ii)
It is quite boring
Table 5.28 shows the positive comments from most of the students. Most of
their comments focus on the design of GOOD learning system in supporting their
learning such as the system provides active learning, encourages HOTS, promotes
191
understanding and supports learning by doing.
Besides, there are some negative
comments about the system such as the system contains technical error and it is lack of
motivation because the design was quite boring and the students are required to
complete a lot of tasks. Besides, the students also put a few suggestions to improve the
interface of the system as shown in Table 5.29.
Table 5.29: Suggestions from the Students About GOOD Learning System.
Item
1
Theme
Improve Interface Design
Example of Feedback
(iii)
Provide undo and redo functions.
(iv)
Add attractive functions to attract
students.
A structured interview was conducted in order to gain further insight about the
effectives of the GOOD learning system in learning CS as well as in improving and
engaging HOTS. There were thirteen structured questions in the interview.
The
interview was conducted after the posttest. The qualitative data from the interview was
analyzed through the identification of the themes as demonstrated in Table 5.30 to Table
5.42. The tables present the themes from the students’ feedbacks based on the interview
protocol transcripts.
Table 5.30 shows that most of the students agreed that GOOD learning system
improved HOTS through lesson mapping, the design of learning objects, solving
problems in the thinking tasks and reflecting their learning in reflection corner. Table
5.31 shows the positive feedback from the students about the development of problem
solving with the use of GOOD learning system. Most of them found that the thinking
tasks, learning tasks, lesson map and the use of HOTS have developed their problem
solving skills.
192
Table 5.30: GOOD Learning System in Improving HOTS
Item
1
Theme
Example of Feedback
Yes.
(i)
How?
(i)
learning and this engage us in
Through
lesson
mapping
(ii)
The lesson mapping reflects our
Through
HOTS.
(ii)
designing
We can improve our understanding
and HOTS of CS by linking the
learning objects.
learning objects
(iii)
We can upload our learning objects.
reflection (iv)
It encourages us to think when
(iii)
Through thinking tasks
(iv)
Through
corner
completing the learning tasks.
(v)
We need to solve thinking tasks in
the system.
(vi)
The reflection corner reminded me
to use HOTS.
Table 5.31: GOOD Learning System in Developing Problem Solving Skills
Item
1
Theme
Yes.
Example of Feedback
(i)
How?
there are a lot of learning tasks
designed in problem-based in the
(i)
Through thinking tasks
(ii)
Through learning tasks
(iii)
Design Lesson map
(iv)
Use HOTS
system such as Apply It.
(ii)
The thinking tasks exposed me to
various scenarios problems.
(iii)
We faced a lot of problems in order
to complete the learning tasks in the
system; this helped and trained us
in problem solving.
(iv)
The process of lesson mapping
trained us in problem solving.
(v)
The
system
improved
our
193
conceptual
HOTS.
understanding
and
These helps in problem
solving.
Table 5.32 shows that most of the students found that the search engine, lesson
map, thinking tasks and forum are the parts in the system which engaged them in HOTS.
Table 5.33 shows that most of the students aware of all the cognitive operations of
HOTS in the system because of the reflection in “How am I doing” checklist, lesson
mapping and problem solving. Table 5.34 shows that all the students agreed that the
system can improve their learning of CS through lesson mapping, solving problems in
thinking tasks, designing learning object and discussion in the forum.
Table 5.32: Parts of GOOD Learning System that Engages Students in HOTS
Item
1
Theme
Search engine
Example of Feedback
(i)
We need to analyze and evaluate in searching
learning objects.
(ii)
Search learning objects in learning object
library because we have to analyze, synthesize
and evaluate the learning objects in lesson
mapping.
2
Lesson map
(i)
The knowledge arrangement and relationships
in lesson mapping.
(ii)
Viewing others’ lesson maps because we need
to evaluate and think the accuracy of the lesson
maps.
3
Thinking tasks
(i)
I think solve problems in thinking tasks. We
used HOTS to solve most of the problems.
(ii)
Of course the difficult problems scenariobased problems.
4
Forum
(i)
I think forum.
Miss Tan always ask me
194
question about my lesson map.
(ii)
Forum. Because I need to explain my lesson
map design.
Table 5.33: Cognitive Operations of HOTS They Aware of in GOOD Learning
System
Item
1
Theme
Example of Feedback
Analysis, Synthesis and
(i)
Analysis, synthesis and evaluation
Evaluation.
because the checklist in the system
Why?
alerted me to use them.
(i)
How
am
I
doing
reflection checklist reminded us
checklist
(ii)
Lesson mapping
(iii)
Problem solving
The
what we need to improve.
(ii)
Analysis, synthesis and evaluation
because lesson mapping required all
these skills.
(iii)
Analysis, synthesis and evaluation
because i used these to solve
problems in the system.
Table 5.34: The Effectiveness of GOOD Learning System in Improving CS
Item
1
Theme
Yes.
Example of Feedback
(i)
How?
The design of lesson maps acted as
our note and it captured the main
points.
(i)
Lesson mapping
(ii)
Thinking tasks
(ii)
Solving questions of “Apply It”.
(iii)
Design learning object
(iii)
I think design our learning object
(iv)
Forum
and upload it.
(iv)
Discuss with lecturer and friend in
forum.
195
Table 5.35 shows the positive feedback from the students about the effectiveness
of the system in the understanding of the vocabulary used in computer through the
searching of learning objects, activities in thinking tasks and lesson mapping. Besides,
most of them agreed that the lesson map, learning tasks and learning objects in the
system has developed their concept of computer as illustrated in Table 5.36. Most of
them also agreed that the lesson mapping, the use of HOTS and thinking tasks in the
system improved their problem solving skills in CS as shown in Table 5.37.
Table 5.35: The Effectiveness of GOOD Learning System in the Understanding of
the Vocabulary Used In Computer
Item
1
Theme
Example of Feedback
Yes.
(i)
How?
(i)
I need to type a keyword when I
search the learning objects.
Searching
learning (ii)
object
(ii)
Thinking tasks
(iii)
Lesson map
Solving multiple-choice questions
in “Try It Out”.
(iii)
Learning object organizer because
we can produce our digital notes.
Table 5.36: The Effectiveness of GOOD Learning System in Developing the
Concept of Computer
Item
1
Theme
Yes.
Example of Feedback
(i) The design of lesson map depicted the
How?
concepts from general to more details.
(i)
Lesson mapping
(ii) The sharing of knowledge where we
(ii)
Thinking tasks
could refer others’ lesson maps and
(iii)
Learning objects
their solutions of thinking tasks.
(iii) Designing our own learning objects.
(iv) Solving problems in thinking task.
196
Table 5.37: The Effectiveness of GOOD Learning System in Improving Your
Problem Solving Skills in CS
Item
1
Theme
Example of Feedback
Yes.
(i)
I used my lesson map.
How?
(ii)
I learn HOTS in the system and I
(i)
Lesson mapping
(ii)
Use HOTS
(iii)
Thinking tasks
tried to use it.
(iii)
I think solving problems in “Apply
It”.
Table 5.38 shows that all the student agreed that HOTS is important in learning
CS because CS required HOTS and problem solving skills. They also agreed that HOTS
is important in learning other Computer Science subjects that required HOTS and
problem solving skills as shown in Table 5.39. Table 5.40 shows that all the students
used HOTS in the posttest. Most of them designed lesson map and applied the cognitive
operations of HOTS in the posttest and the outcomes of using HOTS in the posttest are
they are able to answer the question and have better understanding about the questions
as illustrated in Table 5.41. Table 5.42 shows that most of the students agreed or quite
agreed that GOOD learning system helped them in the learning of CS.
Table 5.38: Feedbacks from the Students about the Importance of HOTS in
Learning CS
Item
1
Theme
Yes.
Example of Feedback
(i)
Why?
We need HOTS to understand the
subject.
(i)
It required HOTS
(ii)
For problem solving
(ii)
We need HOTS to understand the
application of CS in our real life..
(iii)
We need HOTS to solve problems.
197
Table 5.39: The Importance of HOTS Improvement in Learning Others Computer
Science Subject
Item
1
Theme
Example of Feedback
Yes.
(i)
Why?
I think almost most of the subjects
because they need thinking skills to
(i)
It required HOTS
(iii)
For problem solving
understand it.
(ii)
Some of the subjects especially the
subject we need to solve complex
scenarios problems.
Table 5.40: The Use of HOTS in the Posttest
Item
1
Theme
Example of Feedback
Yes.
(i)
I draft the solutions in the map first.
How?
(ii)
I list out the main points, seeing
(i)
Drawing lesson map
their relationships and accuracy.
(ii)
Applying
Then elaborate my answer.
cognitive
operations of HOTS.
(iii)
I analyze the problems first and
then evaluate my answer.
Table 5.41: Outcome of the Use of HOTS in the Posttest
Item
1
Theme
Able to answer the question
Example of Feedback
(i)
I know how to explain more of my
answer
and
give
supporting
examples.
(ii)
I’m able to plan my solutions;
analyze and evaluate the logic of
solutions. See whether it is correct
or not.
2
Better understanding about the (i)
I can list the main points of the
198
questions
questions.
(ii)
I found myself better to know the
meaning of the questions.
Table 5.42: GOOD Learning System in Helping Students in Learning CS
Item
1
2
5.7
Theme
Yes
A Bit
Example of Feedback
(i)
I think yes.
(ii)
Of course yes.
(i)
I think only a bit.
(ii)
May be not but a bit..
Summary
This chapter discussed the results of data analysis in this research. The data was
analyzed based on the research questions in Chapter 1. The quantitative data had been
analyzed through the use of mean, standard deviation, percentage, paired-samples T test
and charts. On the other hand, qualitative data was analyzed by the transcription of the
feedback and the identification of the themes. Chapter 6 will discuss the results derived
from the data analysis.
CHAPTER SIX
DISCUSSIONS AND CONCLUSION
6.0
Introduction
This chapter presents a discussion of the key findings and conclusion of the
research. There are six parts in this chapter. It starts with the summary of research. The
second part discusses the effectiveness of GOOD learning system based on the research
questions. Part three discusses the implications of implementation of GOOD learning
system in learning. The limitations of the research are then discussed in part four. The
fifth part presents some recommendations for further research. It ends with a conclusion
and summary.
6.1
Research Summary
This research aims to design and develop a Web-based learning system prototype
called Generative Object Oriented Design (GOOD) learning system. GOOD learning
system acts as a mind tool that provides a generative learning environment that is based
on learning object design to improve HOTS and learning. The research was conducted
in five main phases: Analysis, Design, Development, Implementation and Evaluation.
200
The first phase was aimed to analyze and determine the conventional learning
problems and learning content before the design and development of GOOD learning
system. This involved the process of subject selection and curriculum analysis for
Computer Science subjects offered in the Diploma of Computer Science in Southern
College. A preliminary study had been conducted to identify the current level of HOTS
among the Computer Science students in learning Computer System (CS), a first year
subject taught in Diploma of Computer Science. Besides, an informal discussion had
been conducted with three experienced lecturers about the conventional teaching and
learning methods used in CS. The content and the learning tasks of CS were then
analyzed to fit the theoretical framework and conceptual model called Generative
Learning Object Organizer and Thinking Tasks (GLOOTT) model, which was proposed
in this research.
The second phase was the design phase that involved the design of the interface
and functions of GOOD learning system. It also determined the design of learning
objects, learning activities that were based on the theoretical framework, data flow
diagram and storyboards design that contained the navigation as well as instructions and
metaphor used in the learning system.
The third phase aimed to develop GOOD learning system and learning objects. It
involved the development of learning objects repository, search engine in the system,
learning objects organizer, thinking tasks, reflection corner, forum and information
agent. The development tools used in the system were Macromedia Flash MX, Adobe
Photoshop, Macromedia Dreamweaver MX, Javascript, PHP, XML and MySQL.
The fourth phase was the implementation of the system. The system was
installed in the local area networking system of Southern College and a plan for the
implementation was prepared in order to meet the objectives of the research. Briefing,
workshop, learning and trial sections had been conducted so that the participants were
familiarized with the system.
201
The last phase aimed at measuring the effectiveness of GOOD learning system in
improving HOTS and learning of CS.
summative evaluation.
It consisted of formative evaluation and
The formative evaluation was conducted before the summative
evaluation, and adjustments as well as refinement were made accordingly from the
findings. The summative evaluation was then conducted in order to accomplish the
objectives of the research. The research design was pre-experimental design, one group
pretest-posttest design (Campbell and Stanley, 1963) with the combination of qualitative
and quantitative approaches.
30 students and 12 lecturers had participated in this
research. The instruments used in the evaluation were pretest and posttest to identify the
students’ improvement of learning, rubric of HOTS evaluation to identify the level of
each cognitive operation, portfolio that contained checklist to evaluate the engagement
of HOTS during the use of GOOD learning system,
interview and Web-based
Evaluation Form to identify the effectiveness of GOOD learning system from the
students and lecturers.
6.2
Discussion
Two main objectives of the research were stated in Chapter 1. First, the
researcher aimed to design and develop a Web-based learning system based on learning
object design and generative learning. In order to achieve this objective, a conceptual
model based on the theoretical framework, called GLOOTT model, was proposed and
subsequently a Web-based learning system prototype based on the model was designed
and developed. The Web-based learning system is called Generative Object Oriented
Design (GOOD) learning system. Second, the researcher conducted a series of studies
to evaluate the effectiveness of the system. The following sections present a detailed
discussion of the research findings.
202
6.2.1
Current Level of HOTS of Students from the Conventional Teaching and
Learning of Computer System (CS)
Result from the analysis of current level of HOTS of the students indicated that
most of the students were unable to answer questions that required them to use HOTS
(analysis, synthesis and evaluation). However, most of them were able to answer the
questions of LOTS (knowledge, comprehension and application).
These were
demonstrated by their mean scores of cognitive operations of Bloom’s taxonomy as
shown in Table 5.2, Figure 5.1 and Figure 5.2.
The data analysis obtained in Figure 5.1 and Figure 5.2 shows that the students
performed better in questions that test LOTS than HOTS. In other word, most of the
students are poor in HOTS as a result of the conventional teaching and learning of CS
that focus more on rote lecturing, assignment and test. This is consistent with the
findings from Parham (2003), Chmura (1998), Henderson (1986), Arup (2004) and
Jamalludin Harun (2005) that rote lecturing prevents students from demonstrating HOTS
in their learning. This is also agreed by discussion with the three lecturers as reported in
Chapter 3. As noted by Harrigan and Vincenti (2004), HOTS are important in college
teaching and learning. Students require HOTS in order to successfully deal with a large
amount of fast changing information about the computer system.
6.2.2
Effectiveness of GOOD Learning System in the Improvement of Students’
Learning
The analysis of the students’ scores in pretest and posttest showed the
improvement of their learning. Table 5.4 shows the scores of posttest for all the students
were higher than pretest. In addition, result from the paired-samples T test as shown in
Table 5.6 indicated the significance difference between the mean scores of pretest and
posttest.
Hence, it can be concluded that the students’ learning of CS might be
significantly improved after the use of GOOD learning system that was designed and
203
developed based on learning object design, generative learning and the emphasis on
HOTS engagement. This is consistent with the results obtained by other researchers (see
Phillips, 2000; Penman and Lai, 2003; Schaverian and Cosgrove, 2000; Hong, Lai and
Holton, 2001) who had used the Web-based learning systems to improve learning.
Students showed their learning improvement in CS through the test that was designed
based on HOTS. Thus, it can be said that the ability of HOTS affect the learning of the
subject. The finding is consistent with study conducted by Parham (2003).
The finding from the interview with students shows that most of the students
found that GOOD learning system improved their learning of CS, developed their
computer concepts, improved their problem solving skills in CS and vocabulary of
computer. The examples of the students feedbacks are demonstrated in Table 5.34, 5.35,
5.36 and 5.37.
6.2.3
Effectiveness of GOOD Learning System in Improving HOTS
From the mean scores of cognitive operations based on Bloom’s taxonomy for
the pretest and posttest as shown in Table 5.7 and Figure 5.3, there was positive
improvement of LOTS and HOTS among the students. In addition, the results in pairedsamples T test also showed the significant differences between the mean scores of LOTS
and HOTS in the pretest and posttest at α=0.05.
It was found that most of the students were unable to answer the questions that
were designed to evaluate their HOTS before the GOOD learning system was introduced
to them. However, after the students went through the learning of CS with GOOD
learning system, they were able to solve the HOTS questions as well as the LOTS
questions. This is indicated by the improvement of the mean scores of the HOTS level
in the posttest. Similar finding was reported by Tal and Hochberg (2003) in which there
was an improvement in the scores of HOTS for the students who used Web-based
learning system that engaged them in HOTS. The results from the research were also
204
consistent with those reported by other researchers who used the Web-based learning to
improve HOTS such as Yuretich (2004), Penman and Lai (2003), Sarapuu and Adojaan
(1999), and Hollingworth and McLoughlin (2003). Hence, it can be concluded that
GOOD learning system that was designed based on the learning object design and
generative learning with Web technology is capable to improve HOTS.
Finding from the interview with students also shows that GOOD learning system
improved their HOTS. Most of them found that the lesson mapping, the design of
learning objects, solving problems in the thinking tasks and reflecting their learning in
reflection corner helped them to improve HOTS.
Details about the finding are
illustrated in Table 5.30.
6.2.4
Effectiveness of GOOD Learning System in HOTS Engagement
GOOD learning system is a unique system that was designed based on the
learning object design and generative learning. It is unique because there are limited
Web-based learning systems that are based on learning object design with pedagogical
aspect to improve learning as discussed in Chapter 1 and 2. Most of the Web-based
learning environments that are designed based on learning object only serve as a
management tool in the learning process.
Moreover, the technical issue has been
overemphasized in the design and development process of those Web-based learning
tools.
GOOD learning system has been designed to provide generative learning
environment that actively engage students in HOTS. The improvement of the students’
HOTS engagement is demonstrated by the analysis of the progressively change of the
HOTS engagement throughout learning process with GOOD learning system. The data
was collected from the students’ portfolios that contained their responds in “How am I
Doing” checklist from the fourth group of sample in the learning with GOOD learning
system.
205
The progressive engagement of the students in HOTS in the learning with
GOOD learning system was tabulated in Table 5.8. This is further illustrated in Figure
5.4 and Figure 5.5 for the very active group, Figure 5.6 to Figure 5.10 for the active
group and Figure 5.11 to Figure 5.13 for the less active group respectively. The results
showed that each individual’s HOTS engagement significantly change over in the
learning with GOOD learning system. Besides, finding from the interview with students
in Table 5.32 shows that the tools such as search engine, lesson map, thinking tasks and
forum in GOOD Learning System engaged them in HOTS. It is clear that GOOD
learning system has great potential in improving HOTS engagement for students from
all groups. In short, the results are consistent with the reported findings in which Webbased learning systems that are grounded on appropriate instructional theoretical models
have the potential to develop and engage students in HOTS (Jonassen and Reeves ,1996;
Reeves, 1997; Bonk and Reynolds, 1997; Khan, 1997).
6.2.5
Effectiveness of GOOD Learning System as Perceived by the Lecturers
The GOOD learning system was meant to investigate the possibility of
developing a Web based learning system which incorporated the learning object design
and generative learning as a mindtool to improve HOTS and learning. The positive
responses of the 12 lecturers were clearly reflected in their feedbacks and suggestions
given during the Web-based Evaluation session.
Table 5.15 presents the findings of the effectiveness of GOOD learning system
as perceived by the lecturers. The results showed that the responses for the learning
object content quality, organization of content, presentation design, pedagogical
parameters, motivation and user control, and interaction usability in GOOD learning
system were highly positive. Most of the lecturers agreed that GOOD learning system
has the attributes of the good organization of content, good pedagogical parameters and
good motivation and user control.
206
In addition, the analysis of the qualitative data of the open-ended questions
demonstrated the positive impression on GOOD learning system. A few themes have
been identified through the data analysis. Details of the data analysis are demonstrated
in Table 5.16 to Table 5.21. Generally, the feedbacks from the lecturers revealed that:
(i)
The system supports collaborative learning, active learning, lesson mapping
and self-assessment.
(ii)
The system facilitates learner-centered learning and learning by designing.
(iii) The system enables students to monitor the learning process.
(iv) The system can improve students in HOTS and promote conceptual
understanding.
(v)
The system can be used in teaching and learning of Computer System and
other subjects that required conceptual understanding and self-learning.
(vi) They are willing to recommend the system to other instructors and students
because of the good aspects of design in the system in supporting learning.
In short, most of the lecturers were impressed by the design of GOOD learning
system that contains learning object design, lesson map, thinking tasks, self-assessment,
self-reflection and resources sharing. Most importantly, the design of GOOD learning
system engages students with HOTS and develops their HOTS. The result is found
consistent from the research conducted by in the design of Web-based learning with
appropriate pedagogy design can promote students’ HOTS from Sarapuu and Adojaan
(1999) and Tal and Hochberg (2003).
6.2.6
Effectiveness of GOOD Learning System as Perceived by the Students
As discussed in chapter 1, the studies on the learning object design have seldom
examine its’ effectiveness in learning.
This research attempted to evaluate the
effectiveness of learning object design with pedagogical aspect in the Web-based to
207
improve HOTS and learning. The analysis of the effectiveness of GOOD learning
system as perceived by the students was conducted in the Web-based Evaluation Form
and interview sections.
The feedback obtained through Web-based Evaluation Form as shown in Table
5.27 indicated that most of the students agreed with the aspects of technology and
technical factors, presentation design, learning strategy, motivation and user control as
well as interaction usability in GOOD learning system. The feedback about the system
from the students was highly positive especially on the learning strategy design in the
GOOD learning system that allows active exploration, self-reflection, learning by doing,
engagement of HOTS, improvement of HOTS and understanding. Besides, comments
from the students in the open-ended question in WEF for the students revealed that the
system:
(i)
supports active learning and learning by doing.
(ii)
promotes understanding of CS and encourages HOTS.
The details about the finding are demonstrated in Table 5.28 and Table 5.29.
The analysis of interview was conducted to support the findings from the Webbased Evaluation Form and the findings discussed in other sections. The interview has
gained the insights about the effectiveness of GOOD learning system in learning CS as
well as in improving and engaging HOTS.
The analysis of interview transcript through the identification of themes
demonstrated that most of the students found the design of learning tasks engaged and
improved their HOTS, especially the lesson maps, designing and uploading learning
objects, reflection corner, learning object design, learning objects, searching learning
objects, solving problems in thinking tasks and forum.
Furthermore, most of the
students agreed that GOOD learning system improved their knowledge, development of
concepts and problem solving skills in CS. They also agreed that the system can be used
208
in other subjects. Besides, they applied the activities of HOTS in learning with GOOD
learning system in the posttest. These improved their HOTS and their learning of CS in
the posttest. Details of the interview finding are demonstrated in Table 5.30 to Table
5.42. The finding is consistent with the results reported by Tal and Hochberg (2003),
Hollingworth and McLoughlin (2003) and Parham (2003).
6.3
Implications of the Research
The findings from this research have shown the potential of GOOD learning
system as a Web based learning system for higher education, where the teaching and
learning processes should focus on HOTS.
The encouraging results give positive
implication to the advocate of learning object design in e-learning. Most importantly,
the effectiveness of GOOD learning system has demonstrated the potential of learning
object design in supporting learning and improving HOTS.
The research findings show that the proposed conceptual model, GLOOTT
model that is based on learning object design, generative learning strategy and activities
that engage students in HOTS has positive influence on the students’ improvement of
HOTS as well as achievement in the test. This suggests a theoretical framework and
instructional design to improve HOTS and learning.
Besides the assessment of the
system’s effectiveness, this study also proposes the specific attributes of the conceptual
design model, GLOOTT model, which can be incorporated into the future Web-based
learning systems to increase effectiveness in developing HOTS. The model also implies
the application of pedagogy with learning object design and Web for the reference of the
other researchers and educational technologists.
Mere technological design does not guarantee effective transformation of
learners into active learners. Therefore this research aims to provide a technologysupported learning system that is designed towards a more learner-driven and learner-
209
oriented interactive learning environment. The learning object design provides the
structure that allows learners to actively participate in the learning. It is clear that
learning experiences, which improve the HOTS of the students will soon become a
common practice in a rapidly changing technological society.
This is of utmost
importance as the development of information technology has become ubiquitous in
higher education. This little attempt would be resourceful in offering an alternative for
technology-supported learning, especially for those who intend to improve their HOTS.
The instructors play an important role in the conventional teaching and learning
process in colleges. Most of the time, they are in full control of the teaching and learning
situation and act as the knowledge provider. However, the instructor’s role will change
in GOOD learning system. The instructor will become a facilitator in the learning
process. Students play an active role in their learning process. They construct and share
their learning. At the same time, they also contribute in the learning by uploading their
self-designed learning objects or creating hyperlinks to other websites in the lesson
mapping. In the earlier discussion, most of the students indicated that they could control
and monitor their learning. In addition, most of the instructors found that the system
encourages students to design their learning and supports active learning.
GOOD learning environment encourages higher-order learning through
generative learning and learning by doing. It also encourages students in HOTS. The
conventional mode of learning which focuses on regurgitation of what the instructors
have taught does not imply the ability to use HOTS. Therefore, it is necessary to change
the emphasis of learning especially for Computer Science learning that must stay abreast
of the rapidly changing technology. Otherwise, it may be difficult to equip the students
with HOTS that enable them to produce but not simply consume the information.
With the advancement of the World Wide Web (WWW), the role of computer in
education has become more important. GOOD learning system was intended to be
introduced as a Web based learning system for higher education. The system was
designed to introduce a unique learning environment that encourages constructivist and
210
higher-order learning. Since GOOD learning system is completely Web based, this is
particularly useful in the implementation of e-learning that has become a main focus in
most of the higher education institutes. The success in wide implementation of such a
system may contribute towards e-learning that aims to improve and support teaching and
learning process.
6.4
Limitations of the Research
The research findings had demonstrated the improvement of HOTS and
achievement of the students in learning Computer System (CS).
The results also
demonstrated that they engaged themselves in HOTS through the learning activities in
GOOD learning system. The summative evaluation showed that effective learning did
take place with the use of GOOD learning system. However, this research did not
conduct the evaluation to compare the effectiveness of GOOD learning system with the
other instructions. GOOD learning system has been shown to be an alternative Webbased learning system, but may not be the most effective one. The study was conducted
in one of the college which offers Diploma in Computer Science in Johor Bahru. As a
result, the results could not be generalized to other colleges and higher institutes.
Since the GOOD learning system was developed as a prototype and the research
scope focused only in learning hardware of CS for Computer Science, the positive
results from the research do not imply that the system is effective in learning all of the
domain subjects. However, most of the lecturers and students found the system is
suitable for other subjects.
The features of learning object design focused in this research is the flexibility
and reusability in supporting the active and generative learning environment in GOOD
learning system. The research did not look into the interoperability of learning object
design in other Web based learning systems. Therefore, the metadata was aimed to
211
support the technical design and development of the databases design in GOOD learning
system only.
In the concept of Web-based learning, learning should not be confined within the
boundary of a campus. Thus it is important to determine the efficacy of a Web-based
learning system as a learning tool that can be can be accessed remotely from home. In
the research carried out, however the students were arranged to work with GOOD
learning system within the local area network of the college. Therefore, the effectiveness
of remote access of GOOD learning system as a Web-based learning tool, was not able
to be studied.
6.5
Suggestions for Further Research
The Web-based learning system that was developed in this research was confined
to the learning of CS. The research results showed that it has the potential to improve
HOTS and the learning of CS with the use of GOOD learning system. However, it does
not mean that GOOD learning system will be effective in learning all subject domains. It
would be useful to study the effectiveness of the system in various Computer Science
topics and subject domains. In addition, studies could also be conducted to determine
which topics or subject domains are suitable to be learned with GOOD learning system.
The study conducted in this research involved only a small number of
participants from a college and the duration of the study was rather short. Hence, it may
not be appropriate to generalize the results obtained from the research. The research
could be extended to include a larger sample of participants from various higher
institutions for a longer period of time. Quantitative studies could also be conducted to
compare the learning outcomes of the experimental groups with the controlled groups
that follow the conventional instructions or other Web based learning system.
212
The design of GOOD learning system could be further improved extended to
make it a more effective learning tool for most of the subject domains. The feature of
interoperability for learning object design could be included into the system to study its
effectiveness in supporting learning. As mentioned by some of the student participants,
the design of GOOD learning is less motivated. Hence, the interface design and the role
of information agent in the system could be improved to provide a more motivated
learning environment. It would also be useful for various domain specialists and
researchers to study the design elements that should be included in GOOD learning
system to make it a more effective and motivated learning environment. Besides, a
video screen capture about lesson map design and system use could be used to provide
clearer instructions to the students.
The conceptual model design for GOOD learning system, the GLOOTT model
represents a multi-facet theoretical design that incorporates a few important components,
namely the learning object design, generative learning, the elements that engage students
in HOTS and Web-supported learning environment. The generative learning is chosen to
provide pedagogical platform for the learning. The research results showed that the
features of generative learning and learning object design fit well in enhancing learning.
This supports the argument from Bannan-Ritland, Dabbagh and Murphy (2000) that the
unique attributes of the learning object design could be incorporated with generative
learning from constructivism.
Hence, it might be useful for other researchers to
investigate various instructional designs of constructivism with the learning object
design in promoting learning.
The lesson map designed in the system was confined to outline form. Thus, the
lesson mapping design tool could be extended to network form. This provides options
for the students with various learning styles to design their lesson maps. The data
tracking in GOOD learning system revealed that the students employed various
strategies in exploring the system. Further research could be conducted to determine
what and which strategies would yield better learning outcomes.
Such a study is useful
213
for instructors to determine how to guide the students. It is also important for the
students to know their learning strategies so that they are more alert in their learning.
This research focused on GOOD learning system as an individual learning tool.
The advent of the technology and features of WWW has spurred to the growing of
collaborative learning. Hence, GOOD learning system could be extended to
collaborative learning tool. Such a study would be useful to examine the effectiveness of
the system in supporting collaborative learning to achieve the desired learning outcomes.
6.6
Conclusion
Most of the conventional teaching and learning in colleges emphasize on the
development of knowledge but not HOTS. Literature and research show that HOTS are
important to educate people to cope with the rapidly changing world. Various
researchers claim about the potential of the Web-based instructions to provide learnercentered learning environment for HOTS (Bonk and Reynolds, 1997), which could be
further enhanced by the unique features of the learning object design.
However,
literature shows that there is a lack of research in the development of computer-based
learning models based on learning object design that is grounded on pedagogical aspects
to improve and support learning. Nevertheless, with the advent of the World Wide Web
technologies, the potential of the learning object design with pedagogy design has been
greatly extended to improve HOTS and learning.
The design and development of GOOD learning system has taken into
consideration of the pedagogical design which is the generative learning, the principle of
instructional design which is learning object design and the learning activities that
encourage HOTS. In the conventional methods, students play the passive role and they
do not have any opportunity of constructing and reflecting their learning. These actions
of constructing and reflecting their learning are via HOTS. The learning environment of
GOOD learning system acts to engage higher order activity, to encourage learners to
214
construct their learning, and to reflect on the consequence of their own thinking. It is
hoped that the findings of this research would be resourceful in offering an alternative
for learning object design that incorporates the pedagogical design in the Web
environment, especially for those learners who intend to improve their HOTS.
6.7
Summary
This chapter discussed the discussion on the research findings and the
conclusion. The discussion has included the research summary, a detailed discussion on
the results of the research, implications of the research, limitations of the research and
the suggestions of future research. A conclusion of the research is presented at the end
of this chapter.
215
Reference:
Adey, P. (1991). Cognitive Acceleration through Science Education. In: Marcure, S,
and Davies, P. eds. Learning to Think, Thinking to Learn. The Proceedings of the
1989 OECD Conference. Oxford: Pergamon. 79-94.
Agostinho, S., Bennett, S., Lockyer, L., and Harper, B. (2003). Integrating Learning
Objects with Learning Designs. In: Crisp, G., Thiele, D., Scholten, I., Barker, S.,
Baron, J. eds. Interact, Integrate, Impact: Proceedings of the 20th Annual
Conference of the Australasian Society for Computers in Learning in Tertiary
Education (ASCILITE). December 7-10. Adelaide, Australia: Ascilite.
Alpert, S. R., and Grueneberg, K. (2000). Concept Mapping with Multimedia on the
Web. Journal of Educational Multimedia and Hypermedia. 9(4): 313-331.
Anderson, T. A. (2003).
I Object! Moving Beyond Learning Objects to Learning
Components. Educational Technology. 43(4): 19-24.
Andrade, H. G. (2000). Using Rubrics to Promote Thinking and Learning. Educational
Leadership. 57(5): 13-18.
Arup, M. (2004). Promoting Higher Order Thinking in MIS/CIS Students Using Class
Exercise. Journal of Information Systems Education. Summer 2004. [On-line]
Available:
http://www.findarticles.com/p/articles/mi_qa4041/is_200407/ai_n9437401/pg_1
Retrieved on: 9 March 2005
Ash, L. E. (2000). Electronic Student Portfolios. Illinois: Skylight.
Asp, E (2001). To Think or Not to Think: Thinking as Measured on State and National
Assessments. In: Costa, A. L. ed. Developing Minds: A Resource Book for
216
Teaching Thinking, 3rd Edition. Alexandria, Virginia: Association for Supervision
and Curriculum Development. 497-510
Atkins, M. J. (1993). Evaluating Interactive Technologies for Learning. Journal of
Curriculum Studies. 26(4): 333-342.
Ausubel, D. (1963). The Psychology of Meaningful Verbal Learning. New York: Gruna
& Stratton.
Baharuddin Aris. (1999). The Use Of Information Technology In Education:
Using An Interactive Multimedia Courseware Package To Upgrade Teachers’
Knowledge And Change Their Attitudes. The Robert Gordon University: Ph.D.
Thesis.
Bannan-Ritland, B., Dabbagh, N., and Murphy, K. (2000). Learning Object Systems as
Constructivist Learning Environments:
Related Assumptions, Theories and
Application. In: Wiley, D. A. ed. The Instructional Use of Learning Objects:
Online Version. [On-line] Available: http://reusability.org/read/ Retrieved on: 3
October 2002
Barab, S. A., Young, M. F., and Wang, Jian Juan (1999). The Effects of Navigational
and Generative Activities in Hypertext Learning on Problem Solving and
Comprehension. International Journal of Instructional Media, [On-line] Available:
http://www.findarticles.com/p/articles/mi_qa3756/is_1999901/ai_n8839530/pg_1
Retrieved on: 15 January 2005
Barba, R. H., and Merchant, L. J. (1990).
The Effects of Embedding Generative
Cognitive Strategies in Science Software. Journal of Computers in Mathematics
and Science Teaching. 10(1): 59-65.
217
Baruque, L. B., and Melo, R. N. (2003). Learning Theory and Instructional Design
Using
Learning
Object.
[On-line]
Available:
http://www.cs.kuleuven.ac.be/~erikd/PRES/2003/Baruque.pdf Retrieved on:
10
Jan 2004.
Baron, J. (1985). Assessing Higher Order Thinking Skills in Connecticut: Lesson from
Connecticut.
In: Kearney, C. P., Kean, M. H., Roeber,E. D., Stevens, B. L.,
Boron, J. B., Fremer, J. and M. Daniel eds. Assessing Higher Order Thinking
Skills.
NJ: ERIC Clearinghouse on Tests, Measurement and Evaluation.
Educational Testing Service. 31-45
Bass, G. M., and Perkins, H. W. (1984). Teaching Critical Thinking Skills with CAI.
Electronic Learning. 14(2): 32- 34.
Baum, R. (1990). Finishing Touches. Educational Leadership. 18(6). 51-55.
Baylor, A. L., and Ritchie, D. (2002). What Factors Facilitate Teacher Skill, Teacher
Morale, and Perceived Student Learning In Technology-Using Classrooms?
Computers and Education. 39 (2002): 395–414
Beaver, R., and Moore, J. (2004). Curriculum Design and Technology Integration: A
Model To Use Technology In Support Of Knowledge Generation And HigherOrder Thinking Skills. Learning & Leading with Technology. 32(1): 42-45.
Bell, E., Allen, R., and Brennan, P. (2001). Assessment of Higher Order Thinking Skills:
A Discussion of the Data from the 2001 Random Sampling Exercise and a
Workshop for Teachers. Queensland: Queensland Board of Senior Secondary
School Studies.
218
Bem, E. Z. (2002). Experiment-Based Project in Undergraduate Computer Architecture.
Proceedings of the 33rd SIGSCE Technical Symposium on Computer Science
Education. 34(1): 171-175.
Ben-Ari, M. (1998). Constructivism in Computer Science Education. Proceedings of
the 29th SIGSCE Technical Symposium on Computer Science. 257-261.
Beyer, B. K. (1988). Developing a thinking Skills Program. Boston: Allyn and Bacon.
Beyer, B. K. (2001). A Format for Assessing Thinking. In: Costa, A. L. ed. Developing
Minds: A Resource Book for Teaching Thinking, 3rd Edition. Alexandria, Virginia:
Association for Supervision and Curriculum Development. 533- 540
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., and Krathwohl, D. R. (1956).
Taxonomy of Educational Objectives: the Classification of Educational Goals.
New York: David McKay Company.
Bloom, B. S., Hasting, J. T., and Madaus, G. F. (1971). Handbook on Formative and
Summative Evaluation of Student Learning. New York: David McKay Company.
Bonk, C. J., and Reynolds, T, H. (1997). Learner-Centered Web Instruction for HigherOrder Thinking, Teamwork, and Apprenticeship. In: Khan, B. H. ed. Web-based
Instruction. New Jersey: Educational Technology. 167-178
Bonn, K. L., Grabowski, B. L. (2001, January). Generative Learning Theory: A
Practical Cousin to Constructivism.
Paper Presented at the Joint Meeting of
Mathematics. New Orleans, LA.
Boron, J. B. (1986).
Evaluating Thinking in the Classroom. In: Boron, J. B. and
Sternberg, R. J. eds. Teaching Thinking Skills: Theory and Practice. NY: W. H.
Freeman. Inc. 221-248.
219
Boyle, T. (2003).
Design Principles for Authoring Dynamic, Reusable Learning
Objects. Australian Journal of Educational Technology. 19(1): 46-58.
Bradley, C., and Boyle, T. (2004). Students’ Use of Learning Objects. Interactive
Multimedia Electronic Journal of Computer-Enhanced Learning. 6(2): [Online]
Available: http://imej.wfu.edu/articles/2004/2/index.asp Retrieved on: 11 April
2005
Brandt, D. S. (1997). Constructivism: Teaching for Understanding of the Internet.
Communications of the ACM. 40(10): 112-117.
Bransford, J. D., Burns, M. S., Delclos, V. R., and Vye, N. J. (1986).
Teaching
Thinking: Evaluating Evaluations and Broadening the Data Base. Educational
Leadership. 44(2): 68-70.
Buschman, L. (1994). Sometimes Less is More. Arithmetic Teacher. 41(7): 378-381.
Butler, J., and Brockman, J. B., (2001). A Web-Based Learning Tool that Simulates a
Simple Computer Architecture. ACM SIGCSE Bulletin. 33(2): 47-50.
Caillot, M. (1991). Learning Thinking Through the New Technologies. In: Marcure, S.
and Davies, P. eds. Learning to Think, Thinking to Learn. The Proceedings of the
1989 OECD Conference. Oxford: Pergamon. 95-102
Campbell, D. T. and Stanley, J. C. (1963).
Experimental and Quasi-Experimental
Designs for Research. Boston: Houghton
Campoy, R. (1992).
The Role of Technology in the School Reform Movement.
Educational Technology. 32(8): 17-22.
220
Chapman, E. (2003). Assessing Student Engagement Rates. ERIC Digest. (ED482269).
[On-line] Available: http://www.ericdigests.org/2005-2/engagement.htm Retrieved
on: 3 Oct 2005.
Cher, Ping Lim and David Hang (2003). An Activity Theory Approach To Research Of
ICT Integration In Singapore Schools. Computers & Education. 41(2003): 49–63.
Chmura, G. A., (1998).
What Abilities Are Necessary for Success in Computer
Science? ACM SIGCSE. 30(4): 55-59.
Christopher, M. M., Thomas, J. A., and Tallent-Runnels, M. K. Raising the Bar:
Encouraging High Level Thinking in Online Discussion Forums. Roeper Review.
26(3):166-172
Cole, J. (2005). Using Moodle. Cambridge: O’Reilly Media.
Cognitive Technology Group at Vanderbilt (CTGV), (1993).
Designing Learning
Environments that Support Thinking. In: Lowyck, J., Jonassen, D. H. and Duffy,
T. M. eds. Designing Environments for Constructive Learning.
New York:
Springer.
Cohen, V. B (1983).
Criteria for the Evaluation of Microcomputer Courseware.
Educational Technology. 23(1): 9-14.
Collis, B., and Strijker, A. (2003). Re-usable Learning Objects in Context. International
Journal on E-Learning. 2(4): 5-17.
Copeland, P. (1988). Interactive Video – What the Research Says. Media in Education
and Development. June: 60-63.
221
Costa, A. L. (2001). Habits of Mind. In: Costa, A. L. ed. Developing Minds: A
Resource Book for Teaching Thinking, 3rd Edition.
Alexandria, Virginia:
Association for Supervision and Curriculum Development. 80-86
Costa, A. L., and Kallick, B. (2001). Building a System for Assessing Thinking. In:
Costa, A. L. ed. Developing Minds: A Resource Book for Teaching Thinking, 3rd
Edition.
Alexandria, Virginia: Association for Supervision and Curriculum
Development. 517- 527
Cousins, J. B., and Ross, J. A. (1993). Improving Higher Order Thinking Skill by
Teaching with the Computer: A Comparative Study. Journal of Research on
Computing in Education. 26(1): 94-115.
Cradler, J. (1985). CompuThink and Criteria for Selecting Software. In: Costa, A. L.
ed. Developing Minds: A Resource Book for Teaching Thinking. Alexandria, VA:
Association for Supervision and Curriculum Development. 256-265
Cradler, J., McNabb, M., Freeman, M., and Burche, R. (2002). How Does Technology
Influence Student Learning? Learning and Leading with Technology. 29(8): 4649.
Crawford, K. (1996). Vygotskian Approaches to Human Development in the
Information Era. Educational Studies in Mathematics. 31(1-2): 43-62.
Cochrane, T. (2005). Interactive QuickTime: Developing and Evaluating Multimedia
Learning Objects to Enhance Both Face-to-Face and Distance E-Learning
Environments. Interdisciplinary Journal of Knowledge and Learning Objects.
1:33-54. [Online] Available: http://ijklo.org/Volume1/v1p033-054Cochrane.pdf
Retrieved on: 5 January 2004.
222
Crump, W. D. Schlichter, C. L. and Palk, B. E. (1988). Teaching HOTS in the Middle
and High School: A District-Level Initiative in Developing higher Order Thinking
Skills. Roeper Review. 10(4): 205-211.
Dabbagh, N. (2001). Concept Mapping as a Mindtool for Critical Thinking. Journal of
Computing in Teacher Education. 17(2): 16-24.
Dalgarno, B (1998). Choosing learner Activities for specific Learning outcomes: A
Tool for Constructivist Computer Assisted Learning Design. In McBeath, C. and
Atkinson, R. eds. Planning for Progress, Partnership and Profit. Proceedings
EdTech’98. Perth: Australian Society for Educational Technology. [On-line]
Available: www.aset.org.ayu/confs/edtech98/pubs/articles/dalgarno.html
Retrieved on: 27 April 2006
De Bono, E (1991). The Direct Teaching of Thinking in Education and the CoRT
Method. In: Marcure, S. and Davies, P. eds. Learning to Think, Thinking to Learn.
The Proceedings of the 1989 OECD Conference. Oxford: Pergamon, 3-14
Debray, R. (1991). Reviving Thought Processes in Pre-Adolescents. In: Marcure, S.
and Davies, P. eds. Learning to Think, Thinking to Learn. The Proceedings of the
1989 OECD Conference. Oxford: Pergamon, 49-61
Dede, C. (1990).
Imaging Technology’s Role in Restructuring for Learning.
In:
Sheingold, K. and Tucker, M. S.ed. Restructuring for Learning with Technology.
New York: Center for Technology in Education. 49-72
Deubel, P. (2003). Learning from Reflections – Issues in Building Quality Online
Courses. Online Journal of Distance Leanring Administration. 6(3). [On-line]
Available: http://www.westga.edu/~distance/ojdla/fall63/deubel63.htm Retrieved
on: 1 May 2006
223
Dick, W and Carey, L. (1996). The Systematic Design of Instruction. 4th. edition. New
York: HarperCollins College.
Dodds, P. and Fletcher, J. D. (2003).
Opportunities for New Smart Learning
Environments Enabled by Next Generation Web Capabilities. Learning Object
2003 Symposium:
Lesson Learned, Questions Asked. [On-line] Available:
http://www.cs.kuleuven.ac.be/~erikd/PRES/2003/LO2003/Kabel.pdf Retrieved on:
15 Jan 2004
Dori, Y. J., Tal, R. T., and Tsaushu, M. (2003). Teaching Biotechnology Through Case
Studies- Can We Improved Higher Order Thinking Skills of Nonscience Majors?
Science Education. 87(6): 767
Downes, S. (2000). Learning Objects.
University of Alberta. [On-line]. Available:
http://www.atl.ualberta.ca/downes/naweb/Learning_Objects.doc Retrieved on: 9
December 2002.
Duffy, T. M., and Jonassen, D. H. (1992).
Instructional Technology.
Constructivism: New Implications for
In: Duffy, T.M. and Jonassen, D. H. eds.
Constructivism and the Technology of Instruction: A Conversation. NJ: Lawrence
Erlbaum. 1-16.
Duffy, T. M., and Cunningham, D. J. (1996). Constructivism: Implications for the
Design and Delivery of Instruction.
In: Jonassen, D. H. ed.
Handbook of
Educational Communications and Technology. New York: Simon & Schuster
Macmillan. 170-198
Dunlap, J. C., and Grabinger, R. S., (1996a). Rich Environment for Active Learning in
the higher Education Classroom. In: Wilson, B. G. ed. Constructivist Learning
Environments: Case Studies in Instructional Design.
Technology. 65- 82
NJ:
Educational
224
Dunlap, J. C., and Grabinger, S. (1996b). Make Learning Meaningful. In: Kommers, P.
A. M., Grabinger, S. and Dunlap, J. C. Hypermedia Learning Environments:
Instructional Design and Integration.
New Jersey:
Lawrence Erlbaum
Association. 227- 254
Edwards, J. (1991). Research Work on the CoRT Method. In: Marcure, S. and Davies,
P. eds. Learning to Think, Thinking to Learn. The Proceedings of the 1989 OECD
Conference. Oxford: Pergamon, 19-30
Eken, Ali Nihat (2002). The Third Eye: Critical Literacy and Higher Order Thinking
Skills are Improved Through a Film Studies Class. Journal of Adolescent & Adult
Literacy. 46(3): 220-231.
Elissavet, G., and Economides, A. A (2003). An Evaluation Instrument for Hypermedia
Courseware. Educational Technology & Society. 6(2): 31-44.
El Saddik, A., Fischer, S., Ipsi, G., and Steinmetz, R. (2001).
Reusability and
Adaptability of Interactive Resources In Web-Based Educational System. ACM
Journal of Educational Resources in Computing. 1(1): 1-19.
El Saddik, A., Ghavam, A, Fischer, S., and Steinmetz, R. (2000). Metadata for Smart
Multimedia Learning Object. Proceedings of the Australasian Conference on
Computing Education. New York: ACM Press, 87-94.
Engel, B. (1994). Portfolio Assessment and the New Paradigm: New Instruments and
New Places. The Educational Forum. 59(1): 22-27.
Ennis, R. H. (1987). A Taxonomy of Critical Thinking Dispositions and Abilities. In:
Baron, J. B. and Sternberg, R. J. eds. Teaching Thinking Skills- Theory and
Practice. New York: W. H. Freeman. 9-26.
225
Fredericksen, N. (1984). The Real Test Bias. American Psychologist. 39(1): 1-10.
Fremer, J., and Daniel, M. (1985). The Assessment of Higher-Order Thinking Skills:
Recent Development. In: Kearney, C. P., Kean, M. H., Roeber, E. D., Stevens, B.
L., Boron, J. B., Fremer, J., and Daniel, M. eds. Assessing Higher Order Thinking
Skills.
NJ: ERIC Clearinghouse on Tests, Measurement and Evaluation,
Educational Testing Service. 48-61
Friesen, N. (2001). What are Educational Objects? Interactive Learning Environments.
9(3).
[On-line]
Available:
http://www.careo.org/documents/objects.html.
Retrieved on 9 Dec 2002.
Fogarty, R. (2002). Brain Compatible Classroom, 2nd Edition. IL: Skylight Professional
Development.
Fogarty, R., and McTighe, J. (1993). Educating Teachers for Higher Order Thinking:
The Three-Story Intellect. Theory into Practice. 32(3): 161-167.
Galambos, D. (2001). Transforming Online Discussions: Getting the H.O.T.S.! Paper
presented at the 2001 WebCT Users Conference, June 23-27, 2001. [On-line]
Available: http://boobooo.webct.com/2001/papers/Galambos.pdf Retrieved on: 31
Dec 2003
Gao, T., and Lehman, J. D. (2003). The Effects of Different Levels of Interaction on the
Achievement and Motivational Perceptions of College Students in a Web-Based
Learning Environment. Journal of Interactive Learning Research. 14(4): 367-387.
Gibbons, A. S., Nelson, J., and Richards, R. (2000).
The Nature and Origin of
Instructional Objects. In: Wiley, D. A. ed. The Instructional Use of Learning
Objects:
Online
Version.
[On-line]
Available:
http://reusability.org/read/chapters/gibbons.doc Retrieved on: 9 Dec 2002.
226
Goodrich, H. (1997). Understanding Rubrics. Educational Leadership. 54(5): 14-17.
Gotschi, T., Sanders, I., and Galpin, V. (2003).
Mental Models of Recursion.
Proceedings of the 34th SIGSCE Technical Symposium on Computer Science
Education. 35(1), 346-350.
Grabinger, R. S. (1996). Rich Environments for Active Learning. In: Jonassen, D. H.
ed. Handbook of Research for Educational Communications and Technology.
New York: Simon Schuster McMillan.
Grabowski, B. L. (1995). Mathemagenic and Generative Learning: A Comparison and
Implications for Designers. In: Rmszowski, A. J. and Dills, C. eds. Instructional
Developments: State of the Art. Vol. 3: The Paradigm.
NJ: Educational
Technology.
Grabowski, B. L. (1996). Generative Learning: Past, Present, and Future. In Jonassen,
D. H. ed.
Handbook of Research for Educational Communication and
Technology. NY: Simon and Schuster Macmillan. 897- 918.
Grace, C. (1992). The Portfolio and Its Use: Developmentally Appropraite Assessment
of Young Children.
ERIC Digest. (ED351150).
[On-line] Available:
http://ericae.net/db/edo/ED351150.htm Retrieved on: 28 Dec 2003.
Greene, M. (1991). The Passion of Thoughtfulness: Arts, Humanities and the Life of the
Mind. In: Marcure, S. and Davies, P. eds. Learning to Think, Thinking to Learn.
The Proceedings of the 1989 OECD Conference. Oxford: Pergamon. 65-78
Hadjerrouit, S. (1999). A Constructivist Approach to Objwct-Oriented DSesign and
Programming.
Proceedings of the 4th Annual SIGSCE/SIGCUE ITiCSE
Conference on Innovation and Technology in Computer Science Education. 31(3),
171-174.
227
Haladyna, T. M. (1997).
Writing Test Items to Evaluate Higher Order Thinking.
Boston: Allyn an Bacon.
Haller, E. P., Child, D. A., and Walberg, H. J. (1988). Can Comprehension to be
Taught? A Quantitative Synthesis of Metacognitive Studies.
Educational
Researcher. 17(9): 5-8.
Hamel, C. J., and Ryan-Jones, D. (2002). Designing Instruction with Learning Objects.
IJET.
3(1).
[On-line]
Available:
http://www.ao.uiuc.edu/ijet/v3n1/hamel/index.html Retrieved on 24 February
2004.
Hamza M. Khalid, Alhalabi Bassem, and Marcovitz, D. M. (2000). Creative Pedagogy
for Computer Learning: Eight Effective Tactics. SIGCSE Bulletin. 32(4), 70-73.
Hannafin, M. J., Hill, J. R., and McCarthy, J. E. (2000). Designing Resource-Based
Learning and Performance Support Systems.
In: Wiley, D. A. ed. The
Instructional Use of Learning Objects: Online Version. [On-line] Available:
http://reusability.org/read/chapters/hannafin.doc Retrieved on: 9 Dec 2002
Hansen, D. (2001). Effective Practices for Gifted Education in Kansas. [Online]
Available: http://www.kansped.org/ Retrieved on: October 1, 2001.
Harrigan, A., and Vincenti, V. (2004). Developing Higher-Order Thinking through an
Intercultural Assignment. College Teaching. 52(3): 113-120.
Harmon, S. W., and Reeves, T. C. (1998).
Instrument.
Educational WWW Sites Evaluation
[On-line]
Available:
http://it.coe.uga.edu/~treeves/edit8350/wwweval.html Retrieved on: 21 November
2003.
228
Hawkins, J. (1993). Technology and The Organization of Schooling. Communications
of the ACM. 36(5): 30-35.
Hawryszkiewyez, I. T. (2002). Designing Learning Activities from Learning Objects.
ASCILITE
2002
Conference
Proceedings.
[Online]
Available:
http://www.ascilite.org.au/conferences/auckland02/proceedings/papers/067.pdf
Retrieved on: 5 January 2004.
Henderson, P. B. (1986). Anatomy of An Introductory to Computer Science Course.
Proceedings of the 17th SIGCSE Technical Symposium on Computer Science
Education. 18(1), 257-263.
Henderson, P., and Romero, F. (1989). Teaching Recursion as a Problem-Solving Tool
Using Standard ML. Proceedings of the 20th SigCSE Technical Symposium, 27-31.
Herrington, J., and Oliver, R. (1999). Using Situated Learning and Multimedia to
Investigate Higher-Order Thinking.
Journal of Educational Multimedia and
Hypermedia. 8(4): 401-421.
Hewson, P. W., and Hewson, M. G. A. B. (1988). An Appropriate Conception of
Teaching Science: A View from Studies of Science Learning. Science Education.
72(5): 597-614.
Hobgood,
B.
(2002).
Concept
http://www.learnnc.org/index.nsf/
Maps.
[On-line]
Available:
Retrieved on: 31 Dec 2003.
Hodgins, H. W. (2001). The Future of Learning Objects. In: Wiley, D. A. ed. The
Instructional Use of Learning Objects: Online Version. [On-line] Available:
http://reusability.org/read/chapters/gibbons.doc Retrieved on: 9 Dec 2002.
229
Hoepfl, M. C. (1997). Choosing Qualitative Research: A Primer for Technology
Education Researchers.
Journal of Technology Education. 9(1). [On-line]
Available: http://scholar.lib.vt.edu/ejournals/JTE/v9n1/hoepfl.htm
Retrieved on:
27 April 2006.
Hogan, K., Nastasi, B. K., and Pressley, M. (2000).
Discourse Patterns and
Collaborative Scientific Reasoning in Peer and Teacher-Guided Discussions.
Cognition and Instructional. 17(4): 379-432.
Hollingworth, R. W., and McLoughlin, C. (2003). Even Foundation Level Students Can
Get the HOTS for Science! 7th Pacific Rim, First Year in Higher Education
Conference, 9-11 July 2003. Australia: Queensland University of Technology.
[On-line]
Available:
www.qut.edu.au/talss/fye/papers03/Refereed%20Papers/Full%20papers/McLoughl
in&Hollingworth_final.doc Retrieved on: 12 Feb 2004.
Holmboe, C. (1999). A Cognitive Framework for Knowledge in Informatics: The Case
of Object-Orientation. ITiCSE’ 0. 31(3): 17-20.
Hong, Kian-Sam, Lai Kwok-Wing, and Holton, D. (2001).
Web Based Learning
Environments: Observation from a Web Based Course in a Malaysian Context.
Australian Journal of Educational Technology. 17(3): 223-243.
Hopson, M. H. (1998).
Effects of Technology Enriched Learning Environment on
Student Development of Higher Order Thinking Skills. University of North Texas:
Doctoral Dissertation
Hopson, M. H., Simms, R. L., and Knezek, G. A. (2001). Using a Technology-Enriched
Learning Environment to Improve Higher Order Thinking Skills.
Research on Technology in Education. 34(2): 109-119.
Journal of
230
Hudgins, B., and Edelman, S. (1986). Teaching critical Thinking Skills to Fourth and
Fifth Graders Through Teacher-Led Small Group Discussions.
Journal of
Educational Research. 79(6): 333-342.
Hyerle, D. (1991). Expand Your Thinking. In: Costa, A. L. ed. Developing Minds: A
Programs for Teaching Thinking. Alexandria, VA: Association for Supervision
and Curriculum Development. 2: 16-26
Ibrahim Ahmad (2005). Pembangunan Prototaip Perisian Multimedia Interktif System
Komputer dan Aplikasinya Berasaskan Pendekatan Pembelajaran Bersaskan
Masalah. University Technology Malaysia: Master Dissertation
IMS (2001).
IMS Learning Resource Meta-Data Inofmration Model.
[On-line]
Available: http://www.imsglobal.org/metadata/imsmd_infovlp2pl.html Retrieved
on: 13 March 2004
Ip, A., and Morrison, I. (2001). Learning Objects in Different Pedagogical Paradigms.
Conference Proceedings of ASCILITE 2001. 289-298.
[On-line].
Available:
http://www.medfac.unimelb.edu.au/ascilite2001/pdf/papers Retrieved on: 15 Jan
2004
Ivie, S. D. (1998). Ausubel’s Learning Theory: An Approach to Teaching Higher Order
Thinking Skills. High School Journal. 82(1): 35-47.
Ivanov, L. (2003). A Hardware Lab for the Computer Organization Course at Small
Colleges. The Journal of Computing in Small Colleges. 19(2): 185-190.
JMP (2006). JMP Academic.
Retrieved on: 4 May 2006
[On-line]
Available: http://www.jmp.com/academic/
231
Jakovljevic, M. (2003). Concept Mapping and Appropriate Instructional Strategies in
Promoting Programming Skills of Holistic Learners. Proceedings of SAICSIT
2003. New York: ACM Press, 308-315.
Jamalludin Harun (2005). Pembinaan dan Penilaian Keberkesanan Sistem Pembelajaran
Berasaskan Masalah Menerusi Web. University Technology of Malaysia: PhD.
Dissertation.
Jensen, K. (1995).
Effective Rubric Design: Making the Most of This Powerful
Assessment Tool. The Science Teacher. 62(5): 34-37.
Johnson, C. R. (1999). Synthesizing Information: A Model That Works for Everyone,
Presentation, ISETA (International Society for Exploring Teaching Alternatives),
Tempe,
Arizona,
Oct
1999.
[On-line]
http://www.west.asu.edu/johnso/synthesis/background.html#objective.
Available:
Retrieved
on: 20 January 2004
Jonassen, D., H. (1991). What are Cognitive Tools. In Kommers, P.A.M., Jonassen, D.
H., and Mates, J. T. eds. Cognitive Tools for Learning. Germany: Springer-Verlag
Berlin Heidelberg. 81: 1-6.
Jonassen, D., H. (1992).
Jonassen, D. H. eds.
Evaluating Constructivist Learning. In Duffy, T. M. and
Constructivism and the Technology of Instruction: A
Conversation. New Jersey: Lawrence Erlbaum Associates. 137-148.
Jonassen, D. H. (1994). Technology as Cognitive Tools: Learners as Designer. IT
Forum, 1994. #1.
Jonassen, D. H. (1996). Computers in the Classroom: Mind Tools for Critical Thinking.
Columbus: Merrill/Prentice Hall.
232
Jonassen, D. H. (2000). Computers as Mindtools for Schools. 2nd ed. New Jersey:
Prentice Hall.
Jonassen, D., Mayes, T., and McAleese, R. (1993). A Manifesto for a Constructivist
Approach to Uses of Technology in Higher Education. In Duffy, T. M., Lowyck,
J., Jonassen, D. H. and Welsh, T. M. eds. Designing Environments for
Constructive Learning. Berlin: Springer-Verlag. 231-247
Jonassen, D. H., Peck, K. L., and Wilson, B. G. (1999). Learning with Technology: A
Constructivist Perspective. Upper Saddle, NJ: Merrill, Prentice Hall.
Jonassen, D. H., and Reeves, T. C. (1996).
Learning with Technology: Using
Computers as Cognitive Tools. In Jonassen, D. H. ed. Handbook of Research for
Educational Communications and Technology. New York: Macmillan. 693-719
Jonassen, D. H., Reeves, T. C., Hong, N., Harvey, D., and Peters, K. (1997). Concept
Map as Cognitive Learning and Assessment Tools.
Journal of Interactive
Learning Research. 8(3/4): 289-308.
Jonassen, D. H., and Wang, S (1993).
Semantically Structured Hypertext.
Acquiring Structural Knowledge from
Journal of Computer-Based Instruction.
20(1): 1-8.
Jonassen, D. H., Wilson, B. G., Wang, S., and Grabinger, R. S. (1993). Constructivist
Uses of Expert Systems to Support Learning.
Journal of Computer-Based
Instruction. 20(3): 86-94.
Juell, P., Smith, E., Daniels, L., and Shanmugasundaram, V. (2002). Learning Objects –
Pedagogy Based Structuring of Course Materials. Conference on Advances in
Infrastructure for Electronic Business, Science, and Education on the Internet,
[On-line]
Available:
233
http://webdb.cs.ndsu.nodak.edu/~shanmuga/vijay’sListofPub.html. Retrieved on:
15 March 2003.
Juliana, M. (2001). Science Concept, Technology, and Higher Order Thinking SkillsWhat’s the Connection? Society for Information Teaching and Teacher Education
International Conference, 2001(1), 2344-2344.
Kallick, B (2001a). Introduction. In: Costa, A. L. ed. Developing Minds: A Resource
Book for Teaching Thinking 3rd Edition. Alexandria, Virginia: Association for
Supervision and Curriculum Development. 472-473
Kallick, B (2001b). Introduction. In: Costa, A. L. ed. Developing Minds: A Resource
Book for Teaching Thinking 3rd Edition. Alexandria, Virginia: Association for
Supervision and Curriculum Development. 496
Kean, M. H. (1985). Assessing Higher Order Thinking Skills: An Overview of the
Issues. In: Kearney, C. P., Kean,M. H., Roeber, E. D., Stevens,B. L., Baron, J.
B., Fremer, J. and Daniel, M. Assessing Higher Order Thinking Skills. Eric/TME
Report 90. Princeton, NJ: OERI. 6-17
Khan, B. H. ed. (1997). Web-based Instruction. New Jersey: Educational Technology.
Klenowski, V. (2002). Developing Portfolios for Learning and Assessment: Processes
and Principles. London: Routledge Falmer.
Knox, D. L. (1997). Integrating Design and Simulation into a Computer Architecture
Course.
Proceedings of the 2nd Conference on Integrating Technology into
Computer Science Education. 29(3), 42- 44.
Koper, E. J. R. (1998). A Method and Tool for the Design of Educational Multimedia
Material. Journal of Computer Assisted Learning. 14(1): 19-30.
234
Krishnaprasad, S. (2002).
Relecance of Computer Topics in Computer Science
Curriculum. The Journal of Computing in Small Colleges. 18(2): 328-336.
Lankes, A. M. D. (1995). Electronic Portfolios: A New Idea in Assessment. ERIC
Digest. NY: ERIC Clearinghouse on Information & Technology. (ED390377),
[On-line]
Available:
http://www.ericdacility.net/ericdigests/ed390377.html
Retrieved on: 31 Dec 2003.
Laney, J. D. (1990). Generative Teaching and Learning of Cost-Benefit Analysis: An
Empirical Investigation. Journal of Research and Development in Education.
23(3): 136-144.
Laurie, B., and Laurie, P. (1997). Apache The Definitive Guide. Cambridge: O’Reilly.
Lau, Siong Hoe (2002). Designing Reusable Object-based Lesson for Mathematics. In
W. C. Yang, Tilak de Alwis, S. C. Chu., and Faqir M Bhatti, Proceedings of the 7th
Asian Technology Conference in Mathematics , December 17-21. VA: ATCM,
135-144
Linden, M., and Wittrock, M. C. (1981). The Teaching of Reading Comprehension
According to the Model of Generative Learning. Reading Research Quarterly.
17(1): 44-57.
Link, F. R. (1991a). Instrumental Enrichment: A Strategy for Cognitive and Academic
Improvement. In Marcure, S. and Davies, P. eds. Learning to Think, Thinking to
Learn. The Proceedings of the 1989 OECD Conference. Oxford: Pergamon, 3148
Link, F. R. (1991b). Thinking to Write: Assessing Higher-Order Cognitive Skills and
Abilities.
In:
Costa, A. L. ed. Developing Minds: A Program for Teaching
235
Thinking. Alexandria, VA:
Association for Supervision and Curriculum
Development. 2, 12-15.
Lim, Teck Meng (2000). Prototyping Hypercal for Exploratory Learning of Quadrating
Function. University Technology of Malaysia: Ph.D. Thesis
Lipman, M. (1991). Strengthening Reasoning and Judgment through Philosophy. In:
Marcure, S. and Davies, P. eds. Learning to Think, Thinking to Learn.
The
Proceedings of the 1989 OECD Conference. Oxford: Pergamon, 103-114.
Liu, Eric Zhi-Feng, Zhuo, Yi-Chin and Yuan, Shyan-Ming (2004). Assessing HigherOrder Thinking Using A Networked Portfolio System With Peer Assessment,
International Journal of Instructional Media, 2004. [On-line] Available:
http://www.findarticles.com/p/articles/mi_qa3756/is_200401/ai_n9362924
Retrieved on: 3 Sept 2005
Liu, M, and Pedersen, S. (1998). The Effects of being Hypermedia Designers on
Elementary School Students Motivation and Learning of Design Knowledge.
Journal of Interactive Learning Research. 9(2): 155-182
London, S., and London, D. (2001). Flash 5 Visual Insight. Arizona: Coriolis.
Longmire, W. (2000a). A Primer on Learning Objects. ASTD’s Online Magazine All
About
E-Learning.
Mar,
2000.
[On-line]
Available:
http://www.learningcircuits.org/mar2000/primer.html
Longmire, W. (2000b).
Content and Context: Designing and Developing Learning
Objects. In Brightman, D. ed. Learning Without Limits. San Francisco:
Informania, Inc. 3, 21-30
236
LTSC. (2000). Learning Technology Standards Committee Website.
[On-line]
Available: http://ltsc.ieee.org/ Retrieved on: 20 Feb 2003.
Mahadevan, Shankar (2002).
A Learning Object Model for Electronic Learning.
Virginia Polytechnic Institute and State University: Master Degree Thesis
Magagnosc, D. (1994). Simulation in Computer Organization; A Goals Based Study.
Proceedings of the 25th SIGSCE Symposium on Computer Science Education,
26(1), 178-182.
Maj, S. P., Veal, D. and Charlesworth, P. (2000). Is Computer Technology Taught
Upside Down?
Proceedings of the 5th Annual IGCSE-SIGCUE ITiCSE
Conference on Innovation and Technology in Computer Science Education. 32(3),
140-143.
Makkonen, P. (1997). Does Collaborative Hypertext Support Better Engagement in
Learning of the Basics in Informatics? ACM SIGCSE Bulletin. 29(3), 130-132.
Maclure, S. (1991). Introduction: An Overview. In Maclure, S. and Davies, P. eds.
Learning to Think, Thinking to Learn.
The Proceedings of the 1989 OECD
Conference. Oxford: Pergamon. viii-xviii
Marzano, R. J., and Arredoddo, D. E. (1986).
Restructuring Schools Through the
Teaching of Thinking Skills. Educational Leadership. 43(8): 20-25.
Marzano, R. J., Brandth, R. S., Hughes, C. S., Jones, B. F., Presseisen, Z., Rankin, R.,
and Suthor, C. (1988). Dimension of Thinking: A Framework for Curriculum and
Instruction.
Development
Alexandria, VA: Association for Supervision and Curriculum
237
McConnell, J. J. (1996). Active Learning ad its Use in Computer Science. Proceeding
of the 1st Conference on Integrating Tech. into C.S.E. 28(1), 52-54.
McGriff, S. J. (2002). Using Written Summaries as Generative Learning Strategy to
Increase
Comprehension
of
Science
Text.
Online
Available:
http://www.personal.psu.edu/faculty/s/j/sjm256/portfolio/research/Quantitative
Research.pdf Retrieved on: 27 Jan 2003.
McLoughlin, C. (1998).
Participation, Cooperation and Autonomy: Students’
Perceptions of Learning at Distance Using Technology. In McBeath, C. and
Atkinson, R. eds. Planning for Progress, Partnership and Profit. Proceedings
EdTech’98. Perth: Australian Society for Educational Technology.
McMillan, J. H., and Schumacher, S. (1997). Research in Education: A Conceptual
Introduction. 4th ed. New York: Longman
McMurry, S. J. (1992). JMP-An Introductory User’s Guide. Online Available:
http://www.personal.psu.edu/faculty/s/j/sjm256/portfolio/research/Quantitative
Research.pdf Retrieved on: 4 May 2006.
Meeker, M. N. (1985). SOI. In Costa, A. L. ed. Developing Minds: A Resource Book
for Teaching Thinking. Alexandria, VA:
Association for Supervision and
curriculum Development. 187-192
Meeker, M. N. (1991). Structure of Intellect (SOI). In Costa, A. L. ed. Developing
Minds: A Programs for Teaching Thinking. Alexandria, VA: Association for
Supervision and curriculum Development. 2, 3-8
Melograno, V. J. (1994).
Portfolio Assessment: Documenting Authentic Student
Learning. Journal of Physical Education, Recreation, and Dance. 65(8): 50-55,
58-61
238
Miron, D., O’Sullivan, M., and McLoughlim, C. (2000). Authentic Student Learning:
Assisting Students to Become Constructive Learners in a First Year Computer
Science Unit. Proceedings of 4th Pacific Rim, First Year in a Higher Education
Conference
University
of
Wollongong
(2001),
[On-line]
Avaliable:
http://cedir.uow.edu.au/programs/literacies/Tertiary_Literacies.pdf
Mills, S. (2002). Learning about Learning Objects with Learning Objects. [On-line]
Available: http://www.alivetek.com/learningobjects/site_paper.htm. Retrieved on
14 Jan 2003
Mirmotahari, O., Holmboe, C., and Kaasboll, J. (2003). Difficulties Learning Computer
Architecture. ITiCSE’ 03. 35(3): 247.
Moersch, C., and Fisher III, L. M. (1995). Electronic Portfolios. Learning and Leading
with Technology. 23(2). 10-15.
Mohan, P., and Brooks, C. (2003). Engineering a Future for Web-based Learning
Objects.
ICWE.
[On-line]
Available:
http://www.cs.usask.ca/research/research_groups/aries/papers/icwe2003_mohan_b
rooks.pdf Retrieved on: 18 Feb 2004
Mohd Salleh Abu, and Tan, Wee Chuen (2002). VATrans: A Model of Self-Designed
Learning Software That Helps Learners to Promote and Engage Themselves with
the Visualization and Analysis Thinking. In: W. C. Yang, Tilak de Alwis, S. C.
Chu., and Faqir M Bhatti, Proceedings of the 7th Asian Technology Conference in
Mathematics. December 17-21. VA: ATCM. 253-262
239
Molenda, M., Pershing, J. A. and Reigeluth, C. M. (1996). Designing Instructional
Systems. In: Craig, S. L. ed. The ASTD Training and Development Handbook - A
Guide to Human Resource Development. London : McGraw-Hill.
Molenda, M., and Sullivan, M. (2002). Issues and Trends in Instructional Technology.
Trends and Issues. 27: 3-18.
Morgan, T. (1996). Using Technology to Enhance Learning: Changing the Chunks.
Learning and Leading with Technology. 23(5): 49-51.
Morrison, D., and Collins, A. (1996). Epistemic Fluency and Constructivist Learning
Environment. In: Wilson, B. G. ed. Constructivist Learning Environments: Case
Studies in Instructional Design. NJ: Educational Technology, 107- 119.
Morrison, G. R., and Lowther, D. L. (2001). Thinking in the Inofrmation Age. In:
Costa, A. L. ed. Developing Minds: A Resource Book for Teaching Thinking 3rd
Edition. Alexandria, Virginia: Association for Supervision and Curriculum
Development. 479-483
Mortimer, L. (2002). Learning Objects of Desire: Promise and Practicality. [On-line]
Available: http://www.learningcircuits.org/2002/apr2002/mortimer.html Retrieved
on: 5 Nov 2002.
Muller, R. (2003). Developing Web Sites with Macromedia Flash MX. MA: Thomson
Course Technology
Murdock, K. L. (2000). JavaSciptTM. Ontario: IDG Books Worldwide
Naps, T. L., Robling, G., Almstrum, V., Dann, W., Fleischer, R., McNally, M.,
Hundhausen, C., Korhonen, A., Malmi, L., McNally, M., Rodger, S., and Angel
Velazquez-Iturbide, J. (2002). Annual Joint Conference Integrating Technology
240
into Computer Science Education Working Group Reports from ITiCSE on
Innovation and Technology in Computer Science Education. 35(2). 131-152.
Nesbit, J., Belfer, K., and Leacock, T. (2003). Learning Object Review Instrument
(LORI). User Manual. E-Learning Research and Assessment Network. [On-line]
Available:
http://elera.matchbox.surrey.sfu.ca/eLera/Home?Articles/LORI%201.5.pdf
Retrieved on: 12 Jan 2004.
Newmann, F. M., and Wehlage, G. G. (1993). Five Standards of Authentic Instruction.
Educational Leadership. 50(7): 8-13.
Neuman, W. L. (2000).
Social Research Methods:
Qualitative and Quantitative
Approaches. Needham Heights, MA: Allyn & Bacon
Neuman, W. L. (2003).
Social Research Methods: Qualitative and Quantitative
th
Approaches 5 Edition. Boston: Pearson Education.
Nickerson, R. S. (1995). Can Technology Help Teach for Understanding. In: Perkins,
D. N., Schwartz, J. L., West, M. M. and Wiske, M. S. eds. Software Goes to
School.
Teaching for Understanding with New Technologies.
NY: Oxford
University Press. 3-20.
Noraffandy Yahaya (2000). Pembangunan dan Penilaian Perisian Alat Pengajaran dan
Pembelajaran Melalui Web.
Universiti Teknologi Malaysia : Master Degree
Thesis.
Norris, S. P. (1989).
Can We Test Validly for Critical Thinking?
Researcher. 18(9): 21-26.
Educational
241
Novak, J. D. (1998). Learning, Creating, and Using Knowledge: Concept MapTM as
Facilitative Tools in Schools and Corporations. New Jersey: Lawrence Erlbaum
Associates.
Novak, J. D., and Gowin, D. B. (1984). Learning How to Learn. London: Cambridge
University Press.
Ogbuji, U. (2003). Thinking XML: Learning Objects Metadata. [On-line] Available:
http://www-106.ibm.com/developerworks/library/x-think21.html Retrieved on: 14
Dec 2003
Oliver, R. (2001). Learning Objects: Supporting Flexible Delivery of Online Learning.
In: Kennedy, G., Keppell, M., McNaught, C. and Petrovic, T. eds. Meeting at the
Crossroads: Proceedings of ASCILITE 2001. Melbourne: The University of
Melbourne. 453-460
Olwell, R. (2002). Building Higher-order Historical Thinking Skills in a College survey
Class. Teaching History: A Journal of Methods. 27(1): 22-33.
Onosko, J. J. (1990). Comparing Teacher’s Instruction to Promote Students’ Thinking.
Journal of Curriculum Studies. 22(5): 443-461.
O’Reilly, J., and Samarawickrema, G. (2003). Using Concept Maps to Enhance the
Learner Experience in Business Law. EET Conference Proceeding 2003. July 16
– 17. Monash University, Clayton Campus. 38-46. Online Available:
http://www.monash.edu.au/groups/fit/eet/full_papers/oreilly.pdf
15 November 2004
Osborne, R. J., and Wittrock, M. C. (1983). Learning Science: A Generative Process.
Science Education. 67(4): 489-508.
242
Parham, J. R. (2003). An Assessment and Evaluation of Computer Science Education.
The Journal of Computing in Small Colleges. 19(2). 115-127.
Parker, J. R.., Becker, K. (2003).
.Measuring Effectiveness of Constructivist and
Behaviorist Assignments in CS102.. Proceedings of the 8th Annual Conference on
Innovation and Technology in Computer Science Education. 35(3): 40-44.
Parry, T., and Gregory, G. (2003). Designing Brain Compatible Learning, 2nd Edition.
Illinois: Skylight Professional Development.
Paulsen, M. F. (2002). Online Education Systems: Discussion and Definition Terms.
[On-line]
Available: http://web.mit.edu/oki/learn/gloss.html.
Retrieved on: 27
April 2006.
Paulson, F. L., and Paulson, P. R. (1996). Assessing Portfolios Using the Constructivist
Paradigm. In: Fogarty, R. Student Portfolios: A Collection of Articles. Illinois:
Skylight Training and Publishing. 27-45
Paulson, F. L., Paulson, P. R., and Meyer, C. A. (1991). What Makes a Portfolio a
Portfolio. Educational Leadership. 48(5). 60-63.
Peck, K. L. and Dorricot, D. (1994). Why Use Technology? Educational Leadership.
51(7): 11-14.
Penman, M., and Lai, K. W. (2003). Synchronous communication and Higher-Order
Thinking in a Tertiary Course in Occupational Therapy. Journal of Interactive
Learning Research. 14(4): 387-405.
Philips, J. A. (2000).
Teaching A Graduate Course Using The Web to Support
Classroom-Based Teaching and Learning: A Case Study.
International
243
Conference Education & Information Communication Technology in the New
Millennium, 27th-28th Oct, 44-53.
Pogrow, S. (1985). HOTS:
A Computer-Based Approach.
In:
Costa, A. L. ed.
Developing Minds: A Resource Book for Teaching Thinking. Alexandria, VA:
Association for Supervision and curriculum Development. 239-240
Pogrow, S. (1987). The HOTS Program: The Role of Computers in Developing
Thinking Skills. TechTrend. 32(2): 10-13.
Pogrow, S. (1988a). Teaching Thinking to At-Risk Elementary Students. Educational
Leadership. 45(7): 79-85.
Pogrow, S. (1988b). HOTS: A Thinking Skills Programs for At-Risk Students.
Principal. 67(4): 21-24.
Pogrow, S. (1990). Challenging At-Risk Students: Findings from the HOTS Program.
Phi Delta Kappan. 71(5): 389-397. [On-line] Available: http://www.hots.org.
Retrieved on: 16 January 2004.
Pogrow, S. (1991). HOTS. In: Costa, A. L. ed. Developing Minds: A Programs for
Teaching Thinking. Alexandria, VA: Association for Supervision and curriculum
Development. 2, 62-64
Pogrow, S. (1994). Helping Students Who “Just Don’t Understand’.
Educational
Leadership. 52(3): 62-67.
Pogrow, S. (1995). Making Reform Work for the Educationally Disadvantaged.
Educational Leadership. 52(5): 20-25.
244
Pogrow, S. (1996). HOTS:
November, 1996.
Helping Low Achievers in Grades 4-8.
[On-line] Avalable: http://www.hots.org.
Principal.
Retrieved on: 16
January 2004.
Pogrow, S. (1999). Systematically Using Powerful Learning Environments to Accelerate
the Learning of Disadvantaged Students in Grades 4-8.
In: Reigluth, C. ed.
Instructional Design Theories and Models Vol II: A New Paradigm of
Instructional Theory. New Jersey: Lawrence Erlbaum. [On-line] Avalable:
http://www.hots.org. Retrieved on: 16 January 2004.
Presseisen, B. Z. (1988). “Avoiding Battle at Curriculum Gulch: Teaching Thinking
and Content. Educational Leadership. 45(7): 7-8.
Privateer, P. (1997). The Future of Learning in the Post-Gutenberg/Post-Industrial Era.
British Journal of Educational Technology. 28(2): 83-86.
Quellmalz, E. S. (1987). Developing Reasoning Skills. In: Baron, J. B. and Sternberg,
R. J. eds. Teaching Thinking Skills: Theory into Practice. New York: W. H.
Freeman. 86-105
Quinn, C. (2000). International Forum of Educational Technology & Society, Formal
Discussion Initiation: Learning Objects and Instruction Components. [On-line]
Available: http://ifets.ieee.org/discussions/discuss_feb2000.html Retrieved on: 13
December 2002.
Ramirez, R., and Bell, R. (1994).
Technology in Education.
Byting Back:
Policies to Support the Use of
Illinois: North Central Regional Educational
Laboratory.
Ramsay, J., McAteer, E., Harris, R., Allan, M., and Henderson, J. (2004). Flexible,
Structured Support for the Reuse of Online Learning Objects. Networked Learning
245
Conference 2004. April 5-7. Lancaster University, UK. [Online] Available:
http://www.shef.ac.uk/nlc2004/Proceedings/Individual_Papers/Ramsay_et_at.htm
Retrieved on: 11 April 2005
Reding, E. E. (2004). Adobe Photoshop CS Design Professional. MA: Thomson Course
Technology
Reed, D. (2002). The Use of Ill-Defined Problems for Developing Problem-Solving and
Empirical Skills in CS1. The Journal of Computin in Small College. 18(1): 121133.
Reeves, T. C. (1997). Using the WWW as a Cognitive Tool in Higher Education. In:
Zahran Halim, Thomas Ottmann, and Zaidah Razak eds. International Conference
on Computers in Education. Malaysia: UNIMAS. 1-8
Reeves, T. C. (1998, February). The Impact of Media and Technology in Schools. A
Research Report Prepared for The Bertelsmann Foundation. [On-line] Available:
http://www.athensacademy.org/instruct/media_tech/reeves0.html Retrieved on: 31
Dec 2003.
Reigeluth, C. M. (1996). Instructional Design Theories. In: Plomp, T. and Ely, D. P.
eds.
International Encyclopedia of Educational Technology 2nd ed.
Oxford:
Pergamon. 163-169
Reigeluth, C. M., and Nelson, L. M. (1997). A New Paradigm of ISD? In: Branch, R. C.
and Minor, B. B. eds. Educational Media and Technology Yearbook. Englewood:
Libraries Unlimited. 22: 24-35
Resnick, L. (1987). Educating and Learning to Think.
Academy Press.
Washington, DC: National
246
Ristow, R. S. (1988). The Teaching of Thinking Skills: Does It Improve Creativity?
Gifted Child Today. 11(2): 44-46.
Ritchie, D. and Volkl, C. (2000). Effectiveness of Two Generative Learning Strategies
in the Science Classroom. School Science and Mathematics. 100(2): 83-93.
Robbins, S. R. (2002). The Evolution of the Learning Content Management System.
ASTD’s Online Magazine All About E-Learning. April, 2002.
[On-line]
Available: http://www.learningcircuits.org/apr2002/robbins.html Retrieved on: 13
Dec 2002.
Robson, R. (1999). Object-Oriented Instructional Design and Applications to the Web.
World
Conference
on
Educational
Multimedia,
Hypermedia
and
Telecommunication. VA: AACE. 1999(1): 698-702
Rohaida Mohd. Saat, and Kamariah Abu Bakar (2000). A Development of a Web-Based
Instruction for Primary School: SPICE. International Conference Education &
Information Communication Technology in the New Millennium, 27th-28th Oct,
164-185.
Roschelle, J., Kaput, J., Stroup, W., and Kahn, T. M. (1998). Scaleable Integration of
Educational Software: Exploring the Promise of Component Architectures.
Journal of Interactive Media in Education. 98(6): 1-31.
Rosenberg, I. M. (1976). Introductory Computer Science Courses A Modular Design.
Proceedings of the ACM SIGCSE-SIGCUE Technical Symposium on Computer
Science. 2(1): 49-57.
247
Sarapuu, T., and Adojaan, K. (1999). Usage of Educational Web Pages to Develop
Students’ Higher Order Thinking Skills. Society for Information Technology and
Teacher Eductaional International Conference. 1999(1): 257-262.
Schaverien, L. (2000).
Towards Research Based Designing for Understanding
Fundamental concepts: The case of the Web Delivered Generative Virtual
Classroom for Teacher Education. ASET. 16(1): 1-12.
Schaverien, L., and Cosgrove, M. (2000). A Biological Basis for Generative Learning in
Technology-and-Science Part II: Implications for Technology-and-Science
Education. International Journal of Science Education. 22(1): 13-35.
Scragg, G. (1991). Most Computer Organization Courses are Built Upside Down.
SIGCSE Bulletin. 23(1): 341-346.
Scragg, G., Baldwin, D., and Koomen, H. (1994). Computer Science Needs an InsightBased Curriculum. SIGCSE Bulletin. 26(1): 150-154.
Sexton, T. G., and Poling, D. R. (1973). Can Intelligent be Taught? Bloomington: Phi
Delta Kappa Educational Foundation.
Shambaugh, R. N. and Magliaro, S. G. (1997). Mastering the Possibilities: A Process
Approach to Instructional Design. Boston: Allyn and Bacon.
Shepherd, C. (2000). Objects of Interest. TACTIX, December 2000, [On-line] Available:
http://www.fastrakconsulting.co.uk/tactix/features/objects/objects.htm#Objects%20defined Retrieved
on 9 Dec 2002.
248
Shelly, G. B., Cashman, T. J., and Vermaat, M. E. (2004). Discovering Computers
2005:
A Gateway to Information, Web Enhanced.
MA:
Thomson Course
Technology
Shepherd, J., Clendinning, J, and Schaverien, L. (2002). Rethinking E-Learning Design
on Generative Learning Principles. ASCILITE. Auckland: UNITEC. [On-line]
Available: http://www.unitec.ac.nz/ascilite/proceedings/papers/166.pdf. Retrieved
on: 12 Jan 2004.
Shi, H., Rodriguez, O., Chen, S., and Shang, Y. (2004). Open Learning Objects as an
Intelligent Way of Organizing Educational Material. International Journal on ELearning. 3(2): 51-63.
Sholseth, R. D., and Watanabe, D. Y. (1991). How Do You Choose a Thinking Skills
Program That Is Right for You? In: Costa, A. L. ed. Developing Minds: Programs
for Teaching Thinking. Alexandria, VA:
Association for Supervision and
curriculum Development. 2: 14-120
Sinclair, K. J., Renshaw, C. E., and Taylor, H. A (2004). Improving Computer-Assisted
Instruction in Teaching Higher-Order Skills. Computers & Education. 42(2004):
169–180.
Singh, H. (2000). Achieving Interoperability in e-Learning. ASTD’s Online Magazine
All
About
E-Learning
Mar,
2000.
[On-line]
Available:
http://www.learningcircuits.org/mar2000/singh.html Retrieved on: 13 Dec 2002.
Skrien, D. (2001). CPU Sim 3.1: A Tool for Simulating Computer Architectures for
Computer Organization Classes. Journal of Educational Resources in Computing.
1(4): 46-59.
249
Snow, R. (1989). Toward Assessment of Cognitive and Conative Structures in Learning.
Educational Researcher. 18(9). 8-14.
South, J. B., and Monson, D. W. (2000). A University-Wide System for Creating,
Capturing, and Delivering Learning Objects. In: Wiley, D. A. ed. The Instructional
Use of Learning Objects. [On-line] Available: http://reusability.org/read/
Sternberg, R. J. (1985). Choosing the Right Program. In: Costa, A. L. ed. Developing
Minds: A Resource Book for Teaching Thinking. Alexandria, VA: Association for
Supervision and Curriculum Development. 185-186
Stoney, S., and Oliver, R. (1999).
Can Higher Order Thinking and Cognitive
Engagement Be Enhanced with Multimedia? Interactive Multimedia Electronic
Journal
of
Computer-Enhance
Learning.
1(2).
[On-line]
Available:
http://imej.wfu.edu/articles/1999/2/07/printver.asp Retrieved on: 13 Jan 2004.
Stoyanova, N., and Kommers, P. (2002). Concept Mapping as a Medium of Shared
Cognition in Computer-Supported Collaborative Problem Solving.
Journal of
Interactive Learning Research. 13(1/2): 111-133.
Swartz, C. W. (2001). Concept Formation and Problem Solving: Understanding and
Managing Two Key Components of Higher Order Cognition. [On-line] Available:
http://www.allkindsofminds.org/articleDisplay.aspx?articleID=18&categoryID=10
Retrieved on: 31 Dec 2003.
Tal, R., and Hochberg, N (2003).
Assessing High Order Thinking of Students
Participating in the “WISE” Project in Israel. Studies in Educational Evaluation.
29(2). 69-89.
Tan Wee Chuen, Baharrudin Aris, and Mohd Salleh Abu (2004). A Proposed Design
and Development Framework of GLOOTT: A Pedagogically-Enriched Web-Based
250
Learning Environment Designed to Inculcate Higher Order Thinking Skills.
International Conference in University Teaching and Learning, 2004. University
Technology Mara, Shah Alam, Malaysia.
Tan Wee Chuen, Baharrudin Aris, and Mohd Salleh Abu (2005). GLOOTT Model: A
Conceptual Model of Web-Based Learning System Designed Framework to
Improve Higher Order Thinking (HOT). Southern College Academic Journal. 1(1):
69-92.
Tay, Lee Yong (2002). Using Information and Communication Technological Tools to
Engage Students in Higher-Order Thinking: A Case Study. Singapore National
Institute of Education, Nanyang Technological University: Master Thesis.
Tillema, H. H. (1998). Design and Validity of a Portfolio Instrument for Professional
Training. Studies in Educational Evaluation. 24(3): 263-278.
Tillema, H. H., and Smith, K. (2000). Learning From Portfolios: Differential Use of
Feedback in Portfolio Construction. Studies in Educational Evaluation. 26(3): 263278.
Ting, Choo Yee (2003). Enhancing Conceptual Change through Cognitive Tools: An
Animated Pedagogical Agent Approach. The 3rd IEEE International Conference on
Advanced Learning Technologies (ICALT), July 09-11, 314-315.
Tishman, S. (2001). Added Value: A Dispositional Perspective on Thinking. In: Costa,
A. L. ed. Developing Minds: A Resource Book for Teaching Thinking, 3rd Edition.
Alexandria, Virginia: Association for Supervision and Curriculum Development.
72- 79.
251
Tishman, S. (1991). Connections. In: Costa, A. L. ed. Developing Minds: A Program
for Teaching Thinking. Alexandria, VA:
Association for Supervision and
curriculum Development. 2: 69-72
Toh, Seong Chong (2004). Creating Reusable Learning Objects: Instructional Design
and
Development
Considerations.
Malaysian
Journal
of
Educational
Technology: 4(2): 63-74.
Valade, J. (2002). PHP & MySQL for Dummies. NY: Hungry Minds.
Van Zele, E., Vandaele, P., Bottledooren, D., and Lenaerts, J. (2003). Implementation
and Evaluation of a Course Concept Based on Reusable Learning Objects. Journal
of Computing Research. 28(4): 355-372.
Vockell, E., and van Deusen, R. M. (1989). The Computer and Higher-Order Thinking
Skills. California: Mitchell Publishing.
Voutilainen, T. (1991). The Project FACE (Formal Aims of Cognitive Education. In:
Marcure, S. and Davies, P. eds. Learning to Think, Thinking to Learn. The
Proceedings of the 1989 OECD Conference. Oxford: Pergamon, 19-30
Wagner, E. D. (2002.) The New Frontier of Learning Object Design. The eLearning
Developers’
Journal.
18
June
2002.
http://www.wlearningguild.com/pdf/2/061802DST-H.pdf
[On-line]
Available:
Retrieved on 27 Dec
2002
Wai, N. N., and Hirakawa, Y. (2001). Teachers’ Conceptualization and Actual Practice
in the Student Evaluation Process at the Upper Secondary School Level in Japan,
Focusing on Problem Solving Skills. Studies in Educational Evaluation. 27(2001):
175-198.
252
Walker, J. M. T., and King, P. H. (2002).
Assessment and Instruction.
Concept Map as a Form of Student
Proceedings of the 2002 American Society for
Engineering Education Annual Conference & Exposition. [On-line] Available:
http://www.vanth.org/docs/010_2002.pdf
Weiss, R. E. (2003). Designing Problems to Promote Higher-Order Thinking. In New
Direction for Teaching and Leaning. 95: 25-31.
White, G. (2001). Misconceptions in CIS Education. The Journal of Computing in
Small Colleges. 16(3): 149-152.
Wiggins, G. (1992). Creating Tests Worth Taking. Educational Leaderships. 49(8): 2633.
Wiley, D. A. (2000). Learning Object Design and Sequencing Theory. Brigham Young
University: Thesis Doctor of Philosophy.
Wiley, D. A. (2001). Connecting Learning Objects to Instructional Design Theory: A
Definition, A Metaphor, and A Taxonomy. In Wiley, D. A. ed. The Instructional
Use of Learning Objects.
[On-line]
Available: http://reusability.org/read/
Retrieved on: 3 October 2002
Wiley, D. A. (2002a). Learning Objects. In Kovolchick, A. and Dawson, K. eds.
Educational Technology: An Encyclopedia. Santa Barbara: ABC-CLIO. [On-line]
Available: http://wiley.ed.usu.edu/docs/encyc.pdf Retrieved on: 15 April 2003.
Wiley, D. A. (2002b). Learning Objects Need Instructional Design Theory. In: Rossett,
A. ed. The ASTD E-Learning Handbook. NY: McGraw-Hill. 115-126
253
Wilson, B. G. (1997). Reflections on Constructivism and Instructional Design. In: Dills,
C. R. and Romiszowski, A. J. eds. Instructional Development Paradigms, Engle
Cliffs, NJ: Educational Technology Publication.
Winocour, S. L. (1985). Project IMPACT. In: Costa, A. L. ed. Developing Minds: A
Resource Book for Teaching Thinking. Alexandria, VA: Association for
Supervision and Curriculum Development. 210-211
Winocour, S. L. (1991). IMPACT. In: Costa, A. L. ed. Developing Minds: A Program
for Teaching Thinking. Alexandria, VA:
Association for Supervision and
curriculum Development. 2: 33-34
Wittrock, M. C. (1974). Learning as a Generative Process. Educational Psychologist.
11(2): 87-95.
Wittrock, M. C. (1986). Students’ Thought Process. In: Wittrock, M. C. ed. Handbook
of Research on Teaching, 3rd Edition. NY: Macmillan. 297-314
Wittrock, M. C. (1991). Generative Teaching of Comprehension. The Elementary
School Journal. 92(2): 169-184.
Wittrock, M. C., and Alesandrini, K. (1990). Generation of Summaries and Analogies
and Analytics and Holistic Abilities. American Educational Research Journal.
27(3): 489-502.
Wolf, D. P. (1989).
Portfolio Assessment: Sampling Student Work.
Educational
Leadership. 46(7): 35-39.
Wolffe, G. S., Yurcik, W., Osborne, U., and Holliday, M. A., (2002).
Teaching
Computer Organization with Limited Resources Using Simulators. Proceedings of
254
the 20th SIGCSE Technical Symposium on Computer Science Education. 34(1):
176-180.
Yehezkel, C., Yurcik, W., Person, M., and Armstrong, D. (2002). Three Simulator
Tools for Teaching Computer Architecture: EasyCPU, Little Man Computer, and
RTLSim. Journal of Educational Resources in Computing. 1(4): 60-80.
Yurcik, W., Osborne, H. (2001). A Crowd of Little Man Computers: Visual Computer
Simulator Teaching Tools. In: Peters, B. A., Smith, J. S., Medeiros, D. J. and
Rohrer, M. W. eds. Proceedings of the 2001 Winter Simulation Conference, 16321639.
Yuretich, R. F. (2004). Measuring Skills in Large Introductory Science Classes. Journal
of College Science Teaching. 33(3): 40-45.
Zanariah Abdullah and Rosmatyuzie Ab. Satar (2001). Kajian Teknologi Maklumat dan
Komunikasi di Sekolah.
Available:
Kota Putra: Maktab Perguruan Besut. [On-line].
http://www.angelfire.com/me4/zana/kajianTMK.html#silibusTM
Retrieved on: 1 April 2004
Zhu, E. (1999). Hypermedia Interface Design: The Effects of Number of Links and
Granularity of Nodes. Journal of Educational Multimedia and Hypermedia. 8(3):
331-358.
Zohar, A., Weinberger, Y., and Tamir, P. (1994).
The Effect of Biology Critical
Thinking Project on the Development of Critical Thinking. Journal of Research in
Science Teaching. 39(1): 183-196.
Zoller, U. (1999).
Scaling-up of Higher-Order Cognitive Skills-Oriented College
Chemistry Teaching:
An Action-Oriented Research.
Science Teaching. 36(5): 583-596.
Journal of Research in
255
APPENDIX A
Rubric for Higher Order Thinking Skills Evaluation (Modified with permission
granted and validated by Diane Hansen, KS Academic Decathlon, Kansas)
Thinking/Score
Knowledge
1
Very little
(20% from
the answer)
amount of
None information
is recalled;
answer is
incorrect.
2
Limited
amount (50%
from the
answer) of
information is
recalled;
answer is
incomplete
Comprehension
Very brief
explanation
of content
(20% from
the answer);
no evidence
to support
Brief
explanation of
content (50%
from the
answer); little
evidence to
support
Application
Analysis
0
3
Sufficient
amount (80%
from the
answer) of
facts are
recalled;
answer is
complete and
acceptable
Overall
understanding
of content
(80% from the
answer);
implied
content/issues
not addressed
4
Numerous
facts (100%
from the
answer) and
details are
recalled;
answer is
thorough
An interrelated, holistic
interpretation
of literal and
implied
content given
(100% from
the answer);
uses examples
and
illustrations to
support
Solution has Solution has a Workable
Solution has a
none or a
limited
solution is
"new slant";
very limited number (40% supported by
supports
number
from the
an adequate
solution with
(10% from
answer) of
number (70% an abundant
the answer)
elements to
from the
amount (100%
of elements
support;
answer) of
from the
to support;
solution is not generalizations answer) of
solution is
workable
and principles facts and
not workable
details
Solution
Solution
Solution
Solution
shows no or shows
demonstrates
classifies
a very
minimal
the relation
elements
limited
classification
and structure
(100% from
number
(40% from the between
the answer),
(10% from
answer) of
elements (70% their
the answer)
elements; no
from the
relationship to
of
relation
answer);
each other
classification between
recognizes
while
of elements; elements and
patterns;
identifying the
no relation
their relation
rationally
arrangement
between
and structure
supported
and structure
elements and to each other
connecting
256
Synthesis
Evaluation
their relation
and structure
to each other
Solution
lacks selfexpression; a
very limited
number
(10% from
the answer)
of important
elements
included;
solution not
workable
Judgements
have no or a
very limited
number
(10% from
the answer)
of supports
them in a
rational and
persuasive way
Solution lacks Workable
Workable
selfsolution is
solution which
new and
is new and
expression;
some (40%
includes
includes all
from the
sufficient
parts (100%
amount (70%
from the
answer)
important
from the
answer);
elements
answer) of
demonstrates
essential
unique selfincluded;
solution not
elements;
expression;
workable; not adequately
communication
clearly
communicated is directed to a
communicated solution to
specific
appropriate
audience in a
audience;
unique and
demonstrates
highly
self-expression effective
manner
Judgements
Judgements
Judgements
have limited
have sufficient have abundant
amount (30% amount (70%
amount (100%
from the
from the
from the
answer) of
answer) of
answer) of
supports
supports.
supports.
Judgements
Judgements are
are on both
based on a
cognitive and
variety of
effective
facets at both
levels; based
the cognitive
on given
and effective
criteria or
levels
selected
remembered
criteria
257
APPENDIX B
“How Am I Doing” Checklist
Instruction: Click the relevant column to show your engagement of the thinking
activities during your learning.
Thinking Activities
Analysis thinking
I compared the similarities of the content in learning objects.
I broke down the concepts of my learning into categories.
I identified the core ideas and differentiated other main ideas.
I identified causal relationships between the learning objects.
I found sequences in the learning objects when I designed my lesson
map.
Synthesis thinking
I combined separate learning objects to produce a coherent whole
lesson map.
I generated the relationships between the learning objects.
I perceived various ways in which the learning objects may be
organized to form a more understandable structure of lesson map.
I integrated the knowledge gained from my thinking tasks to enrich
my lesson maps.
I integrated the self-designed learning objects or/and related Website
address into my lesson maps.
Evaluation thinking
I assessed the content of the learning objects for their usefulness with
the stated learning objectives.
I distinguished relevant learning objects with the stated learning
objectives.
I used the thinking tasks to judge the completeness of my learning.
I assessed others’ lesson maps and gave my comments or suggestions.
I gave my rationales to support the design of my lesson maps.
Yes
No
258
APPENDIX C
SYSTEM DATA FLOW (DFD) DIAGRAM
LOGIN
259
UPLOAD TASK_INSTRUCTOR
Instructor
Uploading Task
2
User’s Selection
2.1
User’s selection
Upload
Task
User’s Selection
Subjects &
Subtopics
Subject Information
2.2
2.3
Thinking
Task
Learning
Objects
LO Information
2.1.1
Add New &
Edit
Subject
Subject Record
2.3
2.2.1
Add & Edit
Learning
Objects
Subject Record
LO Record
Upload Try
It Out
2.1.1
Subject Record
Subtopic Record
D tbl_keywords
Try It Out Record
Keywords for searching
D tbl_assessment
Try It Out Choice
Subtopic Record
D tbl_assesschoice
D tbl_subtopic
Subtopic Record
2.2.2
Upload
Learning
Object Files
Upload
Apply It
2.3
D tbl_lo
Add New &
Edit
Subtopic
Apply It
Information
Try It Out
Information
D tbl_maincontent
Subtopic Information
User’s Selection
LO files
D tbl_file
Apply It Record
D tbl_assessscenarioset
260
UPLOAD TASK_STUDENT
Student
Uploading Task
2
Upload
Task
User’s Selection
2.2
Subject Record
Learning
Objects
D tbl_maincontent
LO Information
2.2.1
LO Keywords
Add & Edit
Learning
Objects
D tbl_keywords
D tbl_lo
LO Record
2.2.2
Upload
Learning
Object Files
LO files
D tbl_file
261
LEARNING
262
LEARNING DESIGN 1 GLOO
Student
3.1.1
GLOO
D tbl_keywords
D tbl_user_subject_lo
Search based on
Keywords
Content of LO & LO
3.1.1.2
3.1.1.1
Search
LO
Add
LO Record
Read
LO Record
LO Record
User’s Selection of
Organized LO
3.1.1.3
LO
Organizer
User’s LO Organized
Record
Lo Record
Subject record
D tbl_lo
D tbl_maincontent
D tbl_user_subject_lo
Read and
Update LO
library
3.1.1.1.1
LO
Library
Read LO
Information
D tbl_user_organize
View
Published
Lesson
263
LEARNING DESIGN 2
3.1.2
TTs
Choice / History Selection
D tbl_result
Apply It
3.1.2.1
Try It
Out
Students’ Answers
of
User Try It Out
3.1.2.2
3.1.2.3
Apply It
Upload
Solution of
Apply It
Time Record
D tbl_resulttime
Try It Out Record
Choice of
Try It Out Record
Apply It/Solution File
D tbl_assessment
Assessment Scenario Set
Record
Apply It Solution Record
D tbl_scenarioSolution
D tbl_assessScenarioSet
D tbl_assessment
3.1.3
Analyzing Record
D tbl_stuanalyzingchecklist
D tbl_stuevaluatingchecklist
Reflection
Corner
D tbl_reflectimportant
D tbl_reflectionquestion
D tbl_reflectionserious
D tbl_stusynthesizingchecklist
Serious Indication Record
Synthesizing Record
Choice / History Selection
Choice / History Selection
Question Indication Record
3.1.3.1
3.1.3.2
Important Indication Record
Evaluating Record
Printing
Record
How am I doing
check list
D tbl_userevaluationprint
Time Record
D tbl_stuchecklisttime
Reflection
Worksheet
Student Reflection Record
D tbl_sturefletion
Printing Record
D tbl_userreflectionprint
Time Record
D tbl_stureflectiontime
264
DESIGN LEARNING VIEW
3.2.1
D tbl_user
List Of
Students
User’s Information
D tbl_user_design_status
tbl_maincontent
User’s record
User’s Selection
User’s Selection
3.2.1.1
3.2.1.2
User Organized Map Record
Lesson
Map
Course
Map
D tbl_user_organize
D tbl_subtopic
Subtopic Record
User Organized Map Record
Subject Record
3.2.1.1.1
3.2.1.1.1.1
GLOO
Lesson
Map
User’s Selection
User’s Selection
D tbl_assesschoice
Choice of Try It Out Record
D tbl_maincontent
D tbl_assessscenarioset
D tbl_assessment
User’s Selection
3.2.1.1.2
3.2.1.1.2.2
3.2.1.1.2.1
Try It Out Record
User’s Selection
TTs
Try It
Out
Time Record
D tbl_resulttime
Apply It
D tbl_result
Students’ Answers of Try It Out
D tbl_stureflectiontime
User’s Selection /History Selection
Time Record
D tbl_sturefletion
Student Reflection Record
User’s Selection
3.2.1.1.3
3.2.1.1.3.1
3.2.1.1.3.2
Reflection
Corner
How am I
doing
check list
Reflection
Worksheet
Analyzing Records
Student
Reflection Record
Time Record
D tbl_stuanalyzingchecklist
Printing
Record
D tbl_stuchecklisttime
Evaluating Records
D tbl_stuevaluatingchecklist
Synthesizing Records
D tbl_stusynthesizingchecklist
D tbl_userevaluationprint
Important Indication Record
D tbl_reflectimportant
Question Indication Record
D tbl_reflectionquestion
Serious Indication Record
D tbl_reflectionserious
265
FORUM
266
APPENDIX D1
WEB-BASED LEARNING EVALUATION FORM: EXPERT/LECTURER
I.
Particulars of the evaluator:
Name
:____________________________________________________
Date
:____________________________________________________
Occupation
:____________________________________________________
Experience
:____________________________________________________
_____________________________________________________
Time started
:____________________________________________________
Time completed:___________________________________________________
II:
Evaluation of the Web-based learning (section A to G).
Instructions:
i.
Not
Applicable
N/A
ii.
Rate the scale according to the number assigned.
Strongly
Disagree (SD)
1
Disagree
(D)
2
Moderately
Agree (MA)
3
Agree Strongly
(A)
Agree (SA)
4
5
Circle your response.
A. Evaluation of the Learning Object
(LO) content quality
1. The content is free of error.
2. The content is presented without bias
that could mislead learners.
3. Presentation emphasizes key points and
significant ideas with appropriate level of
detail.
4. Content relevant to age group
curriculum.
SD
D
MA
A
SA
N/A
N/A
1
1
2
2
3
3
4
4
5
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
267
5. Content of sufficient scope and depth.
6. Variety of information, with options
for increasing complexity.
B. Organization of content
1. Learning objectives are declared, either
within content accessed by the learner or
in available metadata.
2. Learning objectives are appropriate for
the intended learners.
3. The learning content from LOs align
with the declared objectives.
4. The LO is sufficient in to enable
learners to achieve the learning objectives.
5. The LO is able to be used in varying
learning context and within learners from
different background.
6. The LO allows learners to generate and
design their learning.
C: Presentation Design
1. The design of visual information
efficient mental processing.
2. Text is legible and graphics are
labeled.
3. The multimedia design aids learning
and aesthetically pleasing.
4. The design does not overload learners’
memory.
5. There is consistency in the functional
use of colors, text format and layout.
6. GOOD learning system includes the
site map.
D: Pedagogical Parameters
1. GOOD learning system allows learners
to generate learning through active
exploration.
2. GOOD learning system uses
appropriate tools during the learning to
get students to think and reflect.
3. GOOD learning system facilitates
learning by designing and doing.
4. GOOD learning system stimulates and
encourages learners in analysis, synthesis
and evaluation thinking (HOT).
5. The design of GOOD learning system
incorporates the activities of analysis,
synthesis and evaluation (HOT).
6. GOOD learning system provides tasks
N/A
N/A
1
1
2
2
3
3
4
4
5
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
268
that enable learners to improve and
practice their HOT.
E: Motivation and User Control
1. The LO is highly motivating and its
content is relevant the intended learners.
2. GOOD learning system offers active
learning activities.
3. Feedback shows learners’
performance.
4. GOOD learning system can be used by
learner alone.
5. GOOD learning system allows learners
to control and manage their learning.
6. GOOD learning system permits
learners to share their learning.
F: Evaluation on the Interaction
Usability
1. The user interface design implicitly
informs learners how to interact with
GOOD learning system.
2. There are clear instructions guiding use
from Mr. TQ.
3. Navigation through GOOD learning
system is easy and intuitive.
4. The behavior of the user interface is
consistent.
5. The interface design is easy to
remember.
6. GOOD learning system is flexible and
allows learners to access all its content.
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
Section G: Overall Evaluation
Instruction: Answer the following question
1.
Explain some of the aspects in the Web-based learning system that you like?
269
2.
Explain some of the aspects in the Web-based learning system that you do not
like?
3.
What is your suggestion (if any) to improve the Web-based learning?
4.
Do you think the Web-based learning system suitable in
developing/improving the HOTS? Why?
270
5.
Do you think the Web-based learning system suitable in teaching and
learning for Computer System? Why?
6.
Do you think the Web-based learning system suitable in teaching and
learning for other subjects? What are the subjects (if any)? Why?
7.
Would you recommend other (instructors/students) to use this Web-based
learning system?
If Yes, why?
271
If No, why?
----------------------------------------------Thank you ----------------------------------------
272
APPENDIX D2
WEB-BASED LEARNING EVALUATION FORM: STUDENT
II.
Fill in your particular:
Name
II:
:____________________________________________________
Rate the scale according to the number assigned and circle your response.
Not
Applicable
N/A
Strongly
Disagree (SD)
1
Disagree
(D)
2
Moderately
Agree (MA)
3
A. Technology and Technical Factors
1. The Web-based learning system is free of
programming error that affects my navigation.
2. The Web-based learning system can be used
without a very high-end computer.
3. The Web-based learning system allows me to
store my works.
4. The multimedia used in the Web-based
learning system does not affect the download
time of the system.
5. I am satisfied with the quality of multimedia
used in the Web-based learning system.
6. The Web-based learning system allows me to
upload and store my works.
B: Presentation Design
1. The design of multimedia information assists
the learning processing.
2. Text is legible.
3. Graphics are labeled.
4. The design does not overload my memory.
5. There is consistency in the text format and
layout.
6. The Web-based learning system includes the
site map.
C: Learning Strategy
1. The Web-based learning system allows me to
learn through active exploration.
2. The Web-based learning system uses
Agree Strongly
(A)
Agree (SA)
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
N/A
N/A
N/A
1
1
1
1
2
2
2
2
3
3
3
3
4
4
4
4
5
5
5
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
273
appropriate tools during the learning to get me to
reflect.
3. The Web-based learning system facilitates
learning by doing.
4. The Web-based learning system stimulates
and encourages me in analysis, synthesis and
evaluation thinking (HOTS).
5. The Web-based learning system provides
thinking tasks that enable me to improve HOTS.
6. Learning strategies used in the Web-based
learning system enable better understanding on
the concepts I learned.
D: Motivation and User Control
1. Learning activities in the Web-based learning
system are highly motivating.
2. The Web-based learning system offers active
learning activities.
3. Feedback shows my performance.
4. Design of the Web-based learning system
promotes the ownerships of my learning.
5. I can control and manage my learning.
6. The Web-based learning system allows me to
share my learning.
E: Interaction Usability
1. The user interface design implicitly guides me
how to interact with the Web-based learning
system.
2. There are clear instructions from the
information agent (Mr. TQ).
3. Navigation through the Web-based learning
system is easy and intuitive.
4. The user interface is consistent.
5. The interface design is easy to remember.
6. The Web-based learning system is flexible and
allows me to access all its content.
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
N/A
1
1
2
2
3
3
4
4
5
5
N/A
N/A
1
1
2
2
3
3
4
4
5
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
1
2
3
4
5
N/A
N/A
N/A
1
1
1
2
2
2
3
3
3
4
4
4
5
5
5
274
Section F: Comments
My comments about the Web-based learning system:
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Thank You.
275
APPENDIX E
PRE-TEST AND POST-TEST QUESTIONS
Southern College
Kolej Selatan
TEST
Year 2005
COURSE
COURSE CODE
TIME
DEPARTMENT
CLASS
LECTURER
: COMPUTER SYSTEM
: CSIS 1003
: 2 1/2 HOURS
: COMPUTER SYSTEM
: CS & IT
: TAN WEE CHUEN
Student’s Name
Student’s ID
:
:
Notes to candidates:
1) The question paper consists of 3 pages and 3 questions.
2) Answer all questions.
3) Return the question paper with your answer booklet.
276
Question 1:
You are currently taking a computer course in a college. You have read some
magazines about the specification of a computer system. Your course mate would
like to but a personal computer (PC). Give him some information about the following
aspects.
i)
What is RAM and microprocessor? (LOTS: Knowledge)
ii)
Discuss the importance of RAM and the microprocessor in a PC (LOTS:
Comprehension).
iii)
What would he need to buy for the RAM and microprocessor in a
computer shop if he will use the computer to do his coursework such as
designing multimedia website? Give some examples to him (LOTS:
Application).
iv)
Explain to him why the clock speed of the microprocessor and the size of
the RAM critically affect the software that can be used in the PC (HOTS:
Analysis).
v)
Suggest what he needs to consider for the selection of RAM and
microprocessor if he would like to buy a PC (HOTS: Synthesis).
vi)
Give your rational for each of your suggestion (HOTS: Evaluation).
Question 2:
The PPD Insurance Company is information intensive, meaning they must keep track
of and manipulate large amounts of important and confidential data. So choosing a
suitable storage medium is a crucial decision.
i)
List types of computer storage media (LOTS: Knowledge).
ii)
Explain the features of each the storage media (LOTS: Comprehension).
iii)
What are the storage requirements for PPD Insurance Company (LOTS:
277
Application)?
iv)
Determine the factors of the storage media selections for PPD Insurance
Company (HOTS: Analysis).
v)
If you were the chief executive in the MIS department what would be
your suggestion of the storage media selection? How would you justify it
(HOTS: Synthesis and Evaluation)?
Question 3:
Mary has recently taken control of the accounts office of a small business. She plans
to purchase a personal computer (PC) for general accounts, spreadsheets and printing
services from AGE Computer Centre. She has no ideas about the computer. Assume
that you are the staff of the centre and you were asked to explain the following
aspects to her.
i)
What are the basic components of a PC (LOTS: Knowledge)?
ii)
Describe the functions of each component of the PC (LOTS:
Comprehension).
iii)
Explain to her to show how a PC can help in doing her work. Give some
examples to her (LOTS: Application).
iv)
Analyze the advantages and disadvantages of using a computer (HOTS:
Analysis).
v)
Generate a list of the PC hardware specification that is suitable for her use
(HOTS: Synthesis).
vi)
Write a note to convince her of the list you have suggested in (v) (HOTS:
Evaluation).
278
APPENDIX F
INTERVIEW
1.
Do you think the Web-based learning help you to improve HOTS? How?
2.
Was the Web-based learning effective in developing your problem solving
skills? How?
3.
Which part of the software do you find engage you the most in HOTS? Explain
and give an example.
4.
As you reflected on your work in the Web-based learning, which of the HOTS
did you most aware of? Why?
5.
Was the Web-based learning effective in improving your knowledge about
computer and its applications? How?
6.
Did the Web-based learning help you in better understanding of the vocabulary
used in computer? How?
7.
Did the Web-based learning help you in developing the concept of computer?
How?
8.
Was the Web-based learning effective in improving your problem solving skills
in CS? How?
9.
Do you think the HOTS are important in learning CS? Why?
10.
Do you think the improvement of HOTS will help you in learning other
subjects in Computer Science? Why?
11.
As you reflected on your work in the post-test, did you employ the HOTS?
How?
12.
What have you gained as a result of employing the HOTS in the post-test?
13.
Overall, did the Web-based learning help you in learning the CS?
279
280
281
282
283
284
285
286
287
288
289
APPENDIX O
LIST OF PAPERS PUBLICATION
International Journal Paper
Tan Wee Chuen, Baharuddin Aris and Mohd Salleh Abu (2006). GLOOTT Model: A
Pedagogically-Enriched Design Framework of Learning Environment to
Improve Higher Order Thinking Skills. AACE Journal. 14(2):158-171
National Journal Paper
Tan Wee Chuen, Baharuddin Aris and Mohd Salleh Abu (2005). GLOOTT Model: A
Conceptual Model of Web-based Learning System Design Framework to
Improve Higher Order Thinking. Southern College Academic Journal. 1(1):
69:92
International Conference Paper
Tan Wee Chuen, Baharuddin Aris and Mohd Salleh Abu (2004). A Proposed Design
and Development Framework of GLOOTT: A Pedagogically-Enriched Webbased Learning Environment Design to Improve Higher Order Thinking
Skills. International Conference in University Teaching and Learning 2004.
Shah Alam: Uitm.
290
APPENDIX P
CONTENT OF COMPUTER HARDWARE
Content of the Computer Hardware based on the syllabus of CS subject in the
Computer Science Department of Southern College:
Chapter 1: Introduction to Computer System
(i)
(ii)
(iii)
(iv)
What is a computer?
The components of a computer.
Categories of computer.
Computer application.
Chapter 2: System Unit
(i)
(ii)
(iii)
(iv)
(v)
(vi)
(vii)
(viii)
(ix)
(x)
The system unit
Processor
Data representation
Memory
Expansion cards
Ports and connectors
Buses
Bays
Power supply
Mobile computer devices
Chapter 3: Input
(i)
(ii)
(iii)
(iv)
(v)
(vi)
(vii)
(viii)
(ix)
(x)
What is input?
Input devices
Pointing devices
Voice input
Input for mobile computers
Digital cameras
Video input
Scanners and reading devices
Biometric input
Input for physically challenged users
291
Chapter 4: Output
(i)
(ii)
(iii)
(iv)
(v)
(vi)
(vii)
(viii)
What is output?
Display devices
Flat-panel display
CRT monitor
Printers
Speakers and headset
Other output devices
Output for physically challenged users
Chapter 5: Storage
(i)
(ii)
(iii)
(iv)
(v)
(vi)
(vii)
What is storage?
Magnetic disks
Optical disks
Tape
PC cards
Miniature mobile storage media
Microfilm and microfiche
292
APPENDIX Q
SCORE OF LOTS AND HOTS FOR PRETEST AND POSTTEST
Student
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
P16
P17
P18
P19
P20
P21
P22
P23
P24
P25
P26
P27
P28
P29
P30
Table 1: Score of LOTS and HOTS for Pretest
LOTS
HOTS
17
5.56
36
24
13
8
8.33
4.17
26
6
37.5
12.5
15
4
26
14
27.78
19.44
27.78
15.28
16.67
8.33
40.28
24
28
15
10
7
16.67
9.72
18
0
36
15
26
13
25
11
25
14
11
4
25
19
35
18
29
18
29
14
26
13
30.56
13.89
29
11
30.56
20.83
20.83
19.44
Total
22
60
21
13
32
50
19
40
47
43
25
64
43
17
26
18
51
39
36
39
15
44
53
47
43
39
44
40
51
40
293
Table 2: Score of LOTS and HOTS for Posttest
Student
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
P16
P17
P18
P19
P20
P21
P22
P23
P24
P25
P26
P27
P28
P29
P30
LOTS
31.94
47.22
34.72
30.56
43.06
44.44
33.33
41.67
40.28
40.28
34.72
44.44
36.11
31.94
37.50
37.50
38.89
37.50
37.50
41.67
31.94
37.50
41.67
41.67
40.28
37.50
40.28
38.89
43.06
36.11
HOTS
26.39
34.72
25.00
18.06
26.39
29.17
22.22
29.17
33.33
30.56
19.44
36.11
23.61
20.83
20.83
20.83
26.39
27.78
29.17
25.00
20.83
27.78
30.56
27.78
26.39
26.39
25.00
30.56
30.56
27.78
Total
58
82
60
49
69
74
56
71
74
71
54
81
60
53
58
58
65
65
67
67
53
65
72
69
67
64
65
69
74
64
294
APPENDIX R
ANALYSIS OF THE DISTRIBUTION OF STUDENTS’ SCORES IN
PRETEST AND POSTTEST
Table 1: Students’ Scores in Pretest
Range of Score
Frequency
0 to 10
0
11 to 20
5
21 to 30
4
31 to 40
8
41 to 50
7
51to 60
5
61 to 70
1
71 to 80
0
81 to 90
0
91 to 100
0
80
70
60
50
40
30
20
10
Figure R1: The Distribution of Students’ Scores in Pretest
Mean
Std Dev
Std Err Mean
upper 95% Mean
lower 95% Mean
N
42
14.239334
2.5997347
47.317055
36.682945
30
Fitted Normal Parameter Estimates
Type
Parameter
Estimate
Location
Mu (mean)
42.00000
Dispersion
Sigma (Std
14.23933
Dev)
Lower 95%
36.68295
11.34030
Upper 95%
47.31705
19.14215
295
Table 2: Students’ Scores in Posttest
Range of Score
Frequency
0 to 10
0
11 to 20
0
21 to 30
0
31 to 40
0
41 to 50
1
51to 60
10
61 to 70
11
71 to 80
6
81 to 90
2
91 to 100
0
4
5
6
7
0
100
90
80
70
60
50
40
Figure R2: The Distribution of Students’ Scores in Posttest
Mean
Std Dev
Std Err Mean
upper 95% Mean
lower 95% Mean
N
69.333333
9.8026504
1.7897109
72.993703
65.672964
30
Fitted Normal
Parameter Estimates
Type
Parameter
Location Mu
Dispersion Sigma
Estimate
69.33333
9.80265
Lower 95%
65.67296
7.80690
Upper 95%
72.99370
13.17785