Intelligent Question Bank and Examination System
A dissertation submitted to the
Faculty of Computer Science and Information Technology
University of Malaya
By
HU HENG SIEW
(WGC020032)
In Partial Fulfilment of the Requirement for the Degree of
Master Software Engineering (Worth 18/36 credit hours)
May 2006
Supervisor: Mr. Ang Tan Fong
Page 1 of 252
ABSTRACT
Question bank can be described as the databank that keeps all the examination questions
whether pre-existing or created by user. Web-based examination is an online assessment
tool that used to evaluate students’ performance. Examination will incorporate questions
with varying complexity from question bank. Students are required to response to those
examination questions. With this, examination serves as the assessment tool to track
student understanding of the classroom material.
This research is to develop an online Intelligent Question Bank and Examination System
(IQBAES), which will make use of open source technology. The technologies include
java server pages (jsp), servlet, jdbc thin driver, jfreechart, ant, xml, log4j and hibernate.
The system run on Tomcat 4.1 Application Server resides on a J2EE framework, which
makes use of Model View Controller (MVC) design pattern. Hibernate technology is
used to access the database.
The system that had been developed enable lecturer to author and store a bank of webbased questions. Furthermore, this system able to assemble questions and generate exam
based on lecturer’s specifications. The examination module allowed user to conduct
examination over the Internet and Intranet. Questions from various topics with different
level of complexity can be included and assigned to user as per the level. Upon
examination completed, the system able to grade student based on the questions with
varying complexity they have answered.
Page 2 of 252
ACKNOWLEDGEMENT
It is not a trivial task to complete a project of this magnitude, especially having to meet
the dateline of the project. Most of the acknowledgement I must give here must go to my
supervisor Mr. Ang Tan Fong. Throughout the project, he has provided me with much
helpful guidance and moral support towards the end when I needed it most! His guidance
on Question Bank and Examination System concept and explanation on the inner
working of examination were crucial to the understanding of how this project of this
nature should be developed. His guidance, insight, and confidence have inspired me to
complete this project within the time schedule. There are many people who had
contributed to the completion of the project. There are also too many people to mention,
who have provided feedback and inspiration in one way or another that contributed to the
success of this project. Last but not least, I would like to express my gratitude to all those
helpful people. Thank you very much.
Page 3 of 252
TABLE OF CONTENTS
ABSTRACT ......................................................................................................................... 2
ACKNOWLEDGEMENT................................................................................................... 3
1
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
2
INTRODUCTION ................................................................................................... 13
OVERVIEW OF THE SYSTEM ..................................................................................... 13
OBJECTIVE .............................................................................................................. 14
PROJECT SCOPE ....................................................................................................... 15
PROBLEM STATEMENT ............................................................................................. 16
SYSTEM LIMITATION ............................................................................................... 17
EXPECTED OUTCOME .............................................................................................. 17
SIGNIFICANCE OF SYSTEM ....................................................................................... 17
THESES ORGANIZATION ........................................................................................... 19
LITERATURE REVIEW ........................................................................................ 21
2.1 ANALYSIS STUDIES ................................................................................................. 21
2.1.1 Overview of current Examinations Systems ..................................................... 21
2.1.2 Current Process Flow (Manual) or Existing System.......................................... 22
2.1.3 Case Study 1: UTS Online Examination System............................................... 24
2.1.4 Case Study 2: MIND Online Evaluation System (OES) .................................... 26
2.1.5 Case Study 3: ExamManager ........................................................................... 28
2.1.6 Case Study 4: Online Courses Examination System (OCES) ............................ 31
2.1.7 Case Study 5: Web-based Adaptive Testing (WAT) ......................................... 33
2.1.8 Case Study 6: SIETTE ..................................................................................... 35
2.1.9 Case Study 7: Architecture Tutor ..................................................................... 39
2.1.10 Relationship to existing work ........................................................................... 40
2.1.11 IQBAES Approach .......................................................................................... 42
2.2 REVIEW ON LATEST TECHNOLOGIES ........................................................................ 52
2.2.1 Client-Server Architecture................................................................................ 52
2.2.2 Two-Tier Architecture...................................................................................... 52
2.2.3 Three-Tier Architecture.................................................................................... 52
2.2.4 J2EE technology .............................................................................................. 53
2.2.5 J2EE Framework.............................................................................................. 56
2.2.6 .NET Framework ............................................................................................. 57
2.2.7 Comparing J2EE and .NET Platform ................................................................ 58
2.2.8 Component Technology ................................................................................... 60
2.2.9 Application Server and Web Server.................................................................. 60
2.2.10 Operating System ............................................................................................. 62
2.2.11 Database Server ............................................................................................... 63
Page 4 of 252
2.2.12 Database Access Technology ........................................................................... 66
2.2.13 Open Source Technology ................................................................................. 69
2.2.14 Programming Languages .................................................................................. 71
3
METHODOLOGY .................................................................................................. 73
3.1 SYSTEM DEVELOPMENT FLOW ................................................................................. 73
3.2 SOFTWARE METHODOLOGY ..................................................................................... 74
3.2.1 Techniques Used to Define Requirements ........................................................ 76
3.2.2 Software Process Model ................................................................................... 81
3.2.3 Choice of Incremental Development Model as Methodology............................ 83
3.2.4 Advantages of Incremental Development Model .............................................. 88
4
SYSTEM ANALYSIS .............................................................................................. 90
4.1 REQUIREMENT ANALYSIS ........................................................................................ 90
4.2 FUNCTIONAL REQUIREMENT .................................................................................... 90
4.2.1 User Manager Module ...................................................................................... 92
4.2.2 Administrator Module ...................................................................................... 92
4.2.3 Lecturer Module............................................................................................... 95
4.2.4 Student Module ................................................................................................ 98
4.2.5 Examination Module ...................................................................................... 100
4.2.6 Question Module ............................................................................................ 113
4.2.7 Flows of Events ............................................................................................. 116
4.3 NON-FUNCTIONAL REQUIREMENT .......................................................................... 119
4.3.1 Product requirements ..................................................................................... 119
4.3.2 User Interface Requirements .......................................................................... 120
4.3.3 Organizational requirements........................................................................... 120
4.3.4 External requirements .................................................................................... 121
4.4 SYSTEM TOOLS REQUIREMENT .............................................................................. 122
4.4.1 Hardware Requirements ................................................................................. 122
4.4.2 Software Requirements .................................................................................. 124
4.4.3 Development Tools ........................................................................................ 126
5
SYSTEM DESIGN AND DETAILED DESIGN .................................................. 127
5.1 SYSTEM ARCHITECTURE ........................................................................................ 127
5.2 SYSTEM DESIGN .................................................................................................... 127
5.2.1 Struts Framework ........................................................................................... 128
5.2.2 IQBAES Framework Design .......................................................................... 130
5.2.3 IQBAES System Design ................................................................................ 137
5.3 IQBAES SECURITY DESIGN .................................................................................. 140
5.4 DETAILED DESIGN ................................................................................................. 142
Page 5 of 252
5.4.1 User Role Management Class Diagram .......................................................... 143
5.4.2 Examination System Collaboration Diagram .................................................. 146
5.4.3 Examination System Sequence Diagram ........................................................ 147
5.4.4 Examination System Class Diagram ............................................................... 148
5.4.5 Package Dependency Diagram ....................................................................... 149
5.5 ENTITY RELATIONSHIP DIAGRAM .......................................................................... 150
5.5.1 Entities ........................................................................................................... 150
5.5.2 Relationships.................................................................................................. 150
5.5.3 Optionally and Cardinality ............................................................................. 151
5.5.4 Bridge Entities ............................................................................................... 152
5.5.5 Recursive Relationships ................................................................................. 153
5.5.6 Multiple Relationships Between Entities ........................................................ 153
5.5.7 Entity Subtypes .............................................................................................. 153
5.5.8 Exclusive-Or Relationship.............................................................................. 153
5.5.9 User Role Management E-R Diagram ............................................................ 153
5.5.10 Examination Management E-R Diagram ........................................................ 155
5.5.11 Question Bank E-R Diagram .......................................................................... 156
5.5.12 Question System E-R Diagram ....................................................................... 157
5.6 DATA DICTIONARY................................................................................................ 158
6
SYSTEM IMPLEMENTATION AND TESTING ............................................... 176
6.1 INTRODUCTION...................................................................................................... 176
6.2 DEVELOPMENT ENVIRONMENT .............................................................................. 176
6.2.1 Hardware Environment .................................................................................. 177
6.2.2 Software Environment.................................................................................... 177
6.2.3 Development with Provision for Disaster Recovery and Mirroring ................. 179
6.2.4 Implementation Environment ......................................................................... 180
6.2.5 System Development ..................................................................................... 181
6.2.6 Classes and Components ................................................................................ 181
6.2.7 Question Difficulty Assessment Implementation ............................................ 186
6.2.8 Automatic Question Generator ....................................................................... 187
6.2.9 Intelligent Questioning System....................................................................... 188
6.2.10 Coding ........................................................................................................... 191
6.3 TESTING APPROACH .............................................................................................. 195
6.4 TYPE OF TESTING .................................................................................................. 197
6.4.1 Unit Testing ................................................................................................... 198
6.5 MODULE TESTING ................................................................................................. 199
6.6 INTERFACE TESTING .............................................................................................. 199
6.7 INTEGRATION TESTING .......................................................................................... 199
Page 6 of 252
6.8
SYSTEM TESTING .................................................................................................. 200
7
EVALUATION AND RESULTS .......................................................................... 201
7.1 EVALUATION METHODS ......................................................................................... 201
7.1.1 Evaluation in Term of User Interface.............................................................. 201
7.1.2 Evaluation in Term of Implementation ........................................................... 212
7.1.3 Finding .......................................................................................................... 213
7.1.4 Reliability Assessment ................................................................................... 218
7.1.5 Fairness .......................................................................................................... 221
8
CONCLUSION AND FUTURE WORK .............................................................. 223
8.1 SUMMARY OF PROJECT .......................................................................................... 223
8.2 RESEARCH GOALS AND CONTRIBUTIONS ................................................................ 224
8.3 PROBLEMS ENCOUNTERED AND ITS SOLUTIONS ...................................................... 226
8.3.1 Difficulties in configuring Jcrontab ................................................................ 226
8.3.2 Problem with user access rights ...................................................................... 227
8.3.3 Insufficient documentation for open-source plug-in component...................... 227
8.3.4 Problem in mirroring development environment............................................. 227
8.4 EVALUATION BY END USERS ................................................................................. 227
8.5 SYSTEM CONSTRAINTS AND FUTURE ENHANCEMENT ............................................. 228
8.5.1 System constraints.......................................................................................... 228
8.5.2 Future Enhancement....................................................................................... 228
8.6 KNOWLEDGE AND EXPERIENCE GAINED................................................................. 229
8.6.1 Hibernate ....................................................................................................... 229
8.6.2 Apache Ant .................................................................................................... 230
8.6.3 JfreeChart ...................................................................................................... 230
8.6.4 Log4j ............................................................................................................. 230
8.7 CONCLUSION......................................................................................................... 231
9
APPENDIX A – LIST OF REFERENCES ........................................................... 232
10
APPENDIX B - QUESTIONAIRE ....................................................................... 234
11
APPENDIX C – IQBAES SCREEN SHOT .......................................................... 236
12
APPENDIX D – USER REGISTRATION FLOW ............................................... 249
13
APPENDIX E – QUESTION BANK FLOW ........................................................ 250
14
APPENDIX F – EXAMINATION CREATION FLOW ...................................... 251
15
APPENDIX G – EXAMINATION FLOW ........................................................... 252
Page 7 of 252
LIST OF FIGURES
FIGURE 1 – FOUR STAGES IN THE EXAMINATION SYSTEM ................................................ 21
FIGURE 2 – CLASSIC EXAMINATION SYSTEM ................................................................... 23
FIGURE 3 – SEMI-COMPUTERIZED EXAMINATIONS MARKS SYSTEM ................................. 23
FIGURE 4 – EXAMMANAGER MAIN SCREEN ..................................................................... 29
FIGURE 5 – EXAMMANAGER LOGIN SCREEN .................................................................... 30
FIGURE 6 – SCREEN SHOT OF ARCHITECTURE TUTOR ....................................................... 40
FIGURE 7 – SCREEN SHOT OF SAMPLE QUESTION ON STAGE 1 .......................................... 47
FIGURE 8 – SCREEN SHOT OF SAMPLE QUESTION ON STAGE 2 .......................................... 48
FIGURE 9 – SCREEN SHOT OF SAMPLE QUESTION ON STAGE 3 .......................................... 49
FIGURE 10 – SCREEN SHOT OF SAMPLE QUESTION ON STAGE 4 ........................................ 50
FIGURE 11 – SCREEN SHOT OF QUESTIONING PROCESS ..................................................... 51
FIGURE 12 – J2EE ENVIRONMENT (FROM SUN MICRO-SYSTEM)....................................... 54
FIGURE 13 – THE MICROSOFT .NET PLATFORM............................................................... 57
FIGURE 14 – THE CONCEPTUAL JAVA ARCHITECTURE ..................................................... 59
FIGURE 15 – THE CONCEPTUAL .NET ARCHITECTURE..................................................... 59
FIGURE 16 – INCREMENTAL DEVELOPMENT MODEL ........................................................ 88
FIGURE 17 – ADMINISTRATOR MODULE USE CASE DIAGRAM .......................................... 93
FIGURE 18 – LECTURER MODULE USE CASE DIAGRAM ................................................... 96
FIGURE 19 – STUDENT MODULE USE CASE DIAGRAM ..................................................... 99
FIGURE 20 – CONCEPTUAL GRAPH ................................................................................ 102
FIGURE 21 – EXAMINATION MODULE USE CASE DIAGRAM............................................ 112
FIGURE 22 – INTELLIGENT QUESTIONING SYSTEM CONCEPT .......................................... 114
FIGURE 23 – QUESTIONING SYSTEM ARCHITECTURE ..................................................... 114
FIGURE 24 – STRUTS FRAMEWORK ARCHITECTURE ....................................................... 128
FIGURE 25 – STRUTS SEQUENCE DIAGRAM ................................................................... 129
FIGURE 26 – MVC DESIGN PATTERN ............................................................................. 131
FIGURE 27 – MODEL 1 DESIGN PATTERN (FROM SUN MICRO-SYSTEM)............................ 132
FIGURE 28 – MODEL 2 DESIGN PATTERN(FROM SUN MICRO-SYSTEM) ............................ 133
FIGURE 29 – IQBAES FRAMEWORK OVERVIEW ............................................................ 134
FIGURE 30 – IQBAES FRAMEWORK FLOW ................................................................... 135
FIGURE 31 – IQBAES FRAMEWORK ............................................................................. 136
FIGURE 32 – IQBAES SYSTEM ARCHITECTURE............................................................. 139
FIGURE 33 – IMPLEMENTATION OF IQBAES SYSTEM..................................................... 140
FIGURE 34 – OVERLAY BROWSER ................................................................................. 142
FIGURE 35 – HIERARCHY DIAGRAM OF BASEROLE ........................................................ 143
FIGURE 36 – USER ROLE MANAGEMENT CLASSES ......................................................... 144
FIGURE 37 – USER ROLE LISTING CLASSES ................................................................... 145
FIGURE 38 – COLLABORATION DIAGRAM OF EXAMINATION SYSTEM .............................. 146
FIGURE 39 – SEQUENCE DIAGRAM OF EXAMINATION SYSTEM ........................................ 147
FIGURE 40 – CLASS DIAGRAM OF EXAMINATION SYSTEM .............................................. 148
FIGURE 41 – PACKAGE DEPENDENCY OF IQBAES ........................................................ 149
FIGURE 42 - CROWS-FOOT NOTATION ........................................................................... 151
Page 8 of 252
FIGURE 43 – USER ROLE MANAGEMENT E-R DIAGRAM ................................................ 154
FIGURE 44 – EXAMINATION MANAGEMENT E-R DIAGRAM ............................................ 155
FIGURE 45 – QUESTION BANK MANAGEMENT E-R DIAGRAM ........................................ 156
FIGURE 46 – QUESTION SYSTEM E-R DIAGRAM ............................................................ 157
FIGURE 47 – ICM COMPUTER LABORATORY LAYOUT ................................................... 180
FIGURE 48 – TOP-DOWN DEVELOPMENT STRATEGY ...................................................... 181
FIGURE 49 – ERROR HANDLING IF IQBAES .................................................................. 193
FIGURE 50 – TESTING PROCESS..................................................................................... 197
FIGURE 51 – INTEGRATION THREAD TESTING ................................................................ 200
FIGURE 52 – COMPARISON OF IQBAES AND TRADITIONAL EXAMINATION .................... 222
FIGURE 53 – IQBAES LOGIN PAGE .............................................................................. 236
FIGURE 54 – SYSTEM ADMINISTRATOR MAIN PAGE ...................................................... 236
FIGURE 55 – USER ROLE LISTING PAGE ........................................................................ 237
FIGURE 56 – USER ROLE MAINTENANCE PAGE.............................................................. 237
FIGURE 57 – USER PROFILE LISTING PAGE .................................................................... 238
FIGURE 58 – GENERAL MAINTENANCE PAGE ................................................................ 238
FIGURE 59 – CHAPTER MAINTENANCE PAGE ................................................................. 239
FIGURE 60 – EXAMINATION LISTING PAGE .................................................................... 239
FIGURE 61 – EXAMINATION MAINTENANCE PAGE ......................................................... 240
FIGURE 62 – QUESTION BANK LISTING BY CHAPTER PAGE............................................ 240
FIGURE 63 – LIST OF EXAMINATION QUESTIONS IN QUESTION BANK ............................. 241
FIGURE 64 – EXAMINATION QUESTION DETAIL PAGE .................................................... 241
FIGURE 65 – EXAMINATION QUESTION UPDATE PAGE ................................................... 242
FIGURE 66 – STUDENT COURSE ENROLMENT PAGE ....................................................... 242
FIGURE 67 – STUDENT ACTION BULLETIN PAGE ............................................................ 243
FIGURE 68 – EXAMINATION MAIN PAGE ....................................................................... 243
FIGURE 69 – EXAMINATION PAGE ................................................................................. 244
FIGURE 70 – STUDENT EXAM SUMMARY PAGE.............................................................. 244
FIGURE 71 – STUDENT EXAM SUMMARY BAR CHART.................................................... 245
FIGURE 72 – STUDENT GRADE BOOK PAGE ................................................................... 245
FIGURE 73 – STUDENT DETAIL GRADE BOOK PAGE....................................................... 246
FIGURE 74 – STUDENT’S EXAMINATION HISTORY PAGE ................................................ 246
FIGURE 75 – STUDENT’S EXAM QUESTION HISTORY LISTING PAGE ............................... 247
FIGURE 76 – QUESTION ANALYSIS REPORT PAGE .......................................................... 247
FIGURE 77 – STUDENT’S EXAM QUESTION HISTORY DETAIL PAGE ................................ 248
FIGURE 78 – EXAMINATION SUMMARY REPORT PAGE ................................................... 248
Page 9 of 252
LIST OF TABLES
TABLE 1 – ADMINISTRATOR MODULE ACTOR DESCRIPTION ............................................ 94
TABLE 2 – ADMINISTRATOR MODULE USE CASE DESCRIPTION ........................................ 94
TABLE 3 – LECTURER MODULE ACTOR DESCRIPTION ...................................................... 96
TABLE 4 – LECTURER MODULE USE CASE DESCRIPTION ................................................. 97
TABLE 5 – STUDENT MODULE ACTOR DESCRIPTION........................................................ 99
TABLE 6 – STUDENT MODULE USE CASE DESCRIPTION ................................................... 99
TABLE 7 – EXAMINATION MODULE USE CASE DESCRIPTION ......................................... 112
TABLE 8 – CHAPTER ..................................................................................................... 158
TABLE 9 – CLASSSCHEDULE .......................................................................................... 158
TABLE 10 – COUNTRY................................................................................................... 159
TABLE 11 – COURSE ..................................................................................................... 159
TABLE 12 – COURSEENROLMENT................................................................................... 159
TABLE 13 – COURSETYPE .............................................................................................. 160
TABLE 14 – DEPARTMENT ............................................................................................. 160
TABLE 15 – EXAM ........................................................................................................ 160
TABLE 16 – EXAMMANAGER ......................................................................................... 161
TABLE 17 – EXAMSUMMARY......................................................................................... 161
TABLE 18 – FACULTY ................................................................................................... 162
TABLE 19 – GENDER ..................................................................................................... 162
TABLE 20 – GRADEDETAIL ............................................................................................ 162
TABLE 21 – GRADEMASTER........................................................................................... 163
TABLE 22 – INTAKE ...................................................................................................... 163
TABLE 23 – LECTURER.................................................................................................. 163
TABLE 24 – LEVELOFDIFFICULTY .................................................................................. 164
TABLE 25 – MAJORDETAIL ............................................................................................ 164
TABLE 26 – MAJORMASTER ........................................................................................... 165
TABLE 27 – MARITALSTATUS ........................................................................................ 165
TABLE 28 – NATIONALITY............................................................................................. 165
TABLE 29 – PLACEOFBIRTH ........................................................................................... 166
TABLE 30 – PROGRAM .................................................................................................. 166
TABLE 31 – PROGRAMTYPE ........................................................................................... 166
TABLE 32 – QUESTIONBANK ......................................................................................... 167
TABLE 33 – RACE ......................................................................................................... 168
TABLE 34 – RELIGION ................................................................................................... 168
TABLE 35 – RESOCCUPANCY ......................................................................................... 168
TABLE 36 – RESOURCE ................................................................................................. 169
TABLE 37 – RESOURCETYPE .......................................................................................... 169
TABLE 38 – SALUTATION .............................................................................................. 169
TABLE 39 – SEMESTER .................................................................................................. 170
TABLE 40 – SEMESTERDATE .......................................................................................... 170
TABLE 41 – STATE ........................................................................................................ 170
TABLE 42 – STUDENT ................................................................................................... 171
Page 10 of 252
TABLE 43 – STUDENTCOURSE ....................................................................................... 171
TABLE 44 – STUDENTCOURSEHISTORY .......................................................................... 171
TABLE 45 – STUDENTDETAIL ........................................................................................ 172
TABLE 46 – STUDENTPROGRAM .................................................................................... 173
TABLE 47 – STUDENTPROGRAMHISTORY ....................................................................... 173
TABLE 48 – STUDENTSEMESTER .................................................................................... 173
TABLE 49 – T_GRP_REG................................................................................................ 174
TABLE 50 – T_PROPERTY_HOLDER ................................................................................ 174
TABLE 51 – T_ROLE_REG.............................................................................................. 174
TABLE 52 – T_USER ...................................................................................................... 174
TABLE 53 – T_USER_GROUP.......................................................................................... 175
TABLE 54 – T_USER_GROUP_CHILD .............................................................................. 175
TABLE 55 – TOWN ........................................................................................................ 175
TABLE 56 – SOFTWARE TECHNOLOGY LISTING ............................................................. 178
TABLE 57 – RESULTS OF TALLYING THE QUESTIONNAIRE RESPONSES............................. 201
TABLE 58 – JOB TITLES ................................................................................................ 204
TABLE 59 – GENDER .................................................................................................... 204
TABLE 60 – AGE GROUPS OF RESPONDENTS .................................................................. 205
TABLE 61 – SUMMARY OF SECTION B, QUESTIONS 1 –3 ................................................. 205
TABLE 62 – SUMMARY OF SECTION B, QUESTIONS 4-6................................................... 206
TABLE 63 – SUMMARY OF SECTION B, QUESTIONS 7-17................................................. 206
TABLE 64 – SECTION C, USER INTERFACE COLOUR SCHEME.......................................... 207
TABLE 65 – SECTION C, USER INTERFACE LAYOUT ....................................................... 208
TABLE 66 – SECTION C, USER INTERFACE BUTTONS...................................................... 209
TABLE 67 – SECTION C, USER INTERFACE MENUS ......................................................... 210
TABLE 68 – SECTION D, QUESTION 1 ............................................................................ 210
TABLE 69 – SECTION D, QUESTION 2 ............................................................................ 211
TABLE 70 – SECTION D, QUESTION3 ............................................................................. 211
TABLE 71 – ENJOYABILITY OF IQBAES EXAM PROCESS ............................................... 214
TABLE 72 – PERCEIVED LEARNING IN IQBAES EXAM................................................... 215
TABLE 73 – PERCEIVED FAIRNESS IN GRADING IN IQBAES EXAM ................................ 215
TABLE 74 – OVERALL SATISFACTION WITH THE IQBAES EXAM PROCESS ..................... 216
TABLE 75 – OVERALL RECOMMENDATION OF USING THE IQBAES EXAM PROCESS ....... 216
TABLE 76 – COMPARISON BETWEEN TRADITIONAL EXAM AND IQBAES EXAM ............. 218
TABLE 77 – RELIABILITIES’ SUMMARY ......................................................................... 220
Page 11 of 252
Abbreviations
CAT
Computer Adaptive Testing
DR
Detailed Requirement
EIS
Enterprise Information Systems
EJB
Enterprise Java Bean
IQBAES
Intelligent Question Bank And Examination System
J2EE
Java 2 Enterprise Edition
JDBC
Java Database Connectivity
JDK
Java Development Kit
JSP
Java Server Page
JVM
Java Virtual Machine
MVC
Model View Controller
UM
University Malaya
Page 12 of 252
1
INTRODUCTION
1.1
Overview of the System
Nowadays, Internet-based education is supposed to greatly grow in number and in
quality. We expect many kinds of merits of IT-based education. It enables us to let
learners more eager to study the subject, to easily share the world-wide instructive
resources that have been created by other educational staff in the world, to give students
distance learning and conduct online examination at anywhere in the world. Online
examinations are a cost-effective and popular means of assessing student knowledge.
Bicanich, Slivinski and Hardwicke (1997) note that from a study of 400 vocational
learners, 75% preferred online testing to paper-based assessment. However, it is not so
easy to make a fair online evaluation of how well the students have understood the
contents. There are several disturbances for realizing fair grading such as mere
duplication of homework answers between the students, illegally pretending to be other
persons to answer the exam or something like those. Web-based exam presented in the
following sections is an example as a solution to this issue. We have developed an online
examination system based on a Browser/Server framework, which carries out the
examination and auto-grading for objective questions.
Online Question Bank and Examination System is a relatively new and rapidly expanding
system. The main idea of this system is to provide a system that will enable lecturer to
author and store a bank of examination questions. The system will then assemble the
questions to become an examination paper. Student can then access the system and take
part in the exam. Upon examination complete, the system will be able to do a grading
base submitted exam paper.
Page 13 of 252
1.2
Objective
This thesis undertakes a comprehensive study of the web-based examination system and
question bank. This research will focus on developing a web-based applications that can
assists in authoring and storing examination question items, assembling questions into
exam and as a centralized on-line system for mass education evaluation and auto-grading.
The objectives of the thesis can be summarized as follows:
To define a framework for web-based application development and develop a
question bank and examination system on top of the framework.
To develop a system that will define questions’ level of difficulty based on
functional concept graph (Conceptual Graph Structure).
To develop a system that serve as question bank for storing and creation of
examination questions dynamically.
To develop a system which will provide fair assessment method for online
examination system.
Page 14 of 252
1.3
Project Scope
In conjunction with the objectives of the thesis, the scope of the thesis is defined in order
to provide a basic guideline that enables the research to be conducted within a certain
range and depth. The following statements summarize the scope of the thesis in
accordance with the stated objectives:
The study and survey current manual examination and question bank.
The study and survey existing online examination and question bank.
The analysis of the flow involved in examination, grading algorithm and question
bank items creation.
The study and survey of all the components related to web-based examination
system.
The study and explore some of the possible factors that are directly and indirectly
influencing the accuracy of grading system and investigate the possible solutions
to overcome the issue.
To build a framework that is able to support web-based application and allow the
question bank and examination system builds on top of this framework.
To build Intelligent Question Bank and Examination System on top of the
framework.
The study of open source technology and utilize the technology in system
development.
This project will not include developing the student enrolment module, student
registration module, financial module and time tabling module.
Page 15 of 252
1.4
Problem Statement
Web-based education is supposed to greatly grow in number and in quality. We expect
many kinds of merits of web-based education. It enables us to let learners more eager to
study the subject, to easily share the world-wide instructive resources that have been
created by other educational staff in the world, to give students distance learning at
anywhere in the world, and many other things (Toshihiro, 2000).
Web-based examination system, which is part of the web-based education, is an effective
solution for mass education evaluation. However, there are some difficulties in this kind
of evaluation. The main difficulty is the use of appropriate assessment techniques to
support student learning and to record their achievement. Most of the present systems
were designed to grade students on how well they have done on their examination only
after they have answered all the questions in the exam that was assigned to them. The
only algorithm that used to evaluate student performance is based on how well they have
answered the exam and no of questions they have answered correctly. These systems
were designed with the concept of traditional paper based examination in mind. There is
a need to use a range of different assessment methods, in order to prevent assessment
being biased against students that have particular problems with one particular method.
Hence, a proposed system that can make fair evaluation of how each student understood
the content of the lecture play a paramount important role.
Another drawback of present systems is that there is no flexibility and there are very
limited options for the examination questions. Most of the systems were designed to
deliver and mark multiple-choice questions. Multiple-choice questions have many
disadvantages. Not least of which is different students will interpret the questions
differently. Hence, by using multiples choice question to evaluate students’
understanding, these systems will not precise enough to represent the knowledge of
individual users, and to select problem to extend the user’s current level of
understanding.
Page 16 of 252
On top of that, the definition of the level of difficulty for examination question often
creates an argument. We do not have a clear mechanism to define the level of difficulty
for each question. Currently, the level of difficulty for a question is solely defines by
setter of the questions. Hence, there is a need to come out with a system that can based on
question difficulty assessment algorithm to determine the level of difficulty for each
question.
1.5
System Limitation
Currently, this system is an independent system which independent from university
central database. Hence, some information likes administration, admission, course
management, enrolment, finance, resource management will not be integrated into this
system. However, this system should be scale enough to handle the possibility of
integration with any other academic module in university.
1.6
Expected Outcome
This system is designed for actual implementation of online examination and question
bank in university and schools. The system should support concurrent access by a pool of
lecturers as well as students. The useful statistical information from this system will be
the examination results, which can be used in assessment and projections of student
performance.
1.7
Significance of System
This is a system with real benefits to be gained from the development of the whole
system. In effect, it is an extension of the efforts done in university over the past few
years in the computerization of examination results and monitoring student performance.
The significance of the project can be realized in:
Page 17 of 252
Introduction of newer and better technologies.
The development of this system will introduce newer technologies such as J2EE
framework with MVC design pattern, hibernate technology etc. These
technologies will improve the scalability of the system.
A more scalable and user-friendly system.
Newer technologies promise to deliver presentation of results in better userinterfaces, including Web Page design, and possibility even scalable to
presentation on mobile devices. The results are user-friendlier, as they use
controls that is more familiar to the current generation of younger users who are
net-savvy.
Opens up new avenues in examination processing.
The design of this new system could result in developing newer ways of
administering and maintaining the examination system. This may lead to new
possibilities that are yet not thought of by anyone. Certainly, using the Web to
process results is a step in the new direction.
Techniques developed could be applied in other systems.
Developing this system will lead to a pattern of development that can be applied
in other and the other associated new technologies. The techniques can be applied
beyond the boundaries of the online examination system. The same techniques
could be used in various other industries, including commercial usage.
Page 18 of 252
1.8
Theses Organization
This report consists of 8 chapters and its organization is outlined as below:
Chapter 1: Introduction
This chapter gives an Overview of the system, system motivation, system
objectives, system scope and the project schedule.
Chapter 2: Literature Review
This chapter is the literature review of the thesis. In this chapter, reviews are
conducted on all the necessary areas that are relevant to the thesis. It provides
study on existing manual and web-based question bank and examination system.
The study includes all the development paths leading to the core technologies
used in the project. There is also a review of the development tools that were
considered for use in this system.
Chapter 3: Methodology
This chapter outlines the methodology used in arriving at this system, the
different ways and techniques used to develop the whole system.
Chapter 4: System Analysis
This chapter lays out the details of system requirements, including functional and
non-functional requirements, hardware and software requirements.
Chapter 5: System Design
This chapter describes in detail the system design, database design and user
interface design.
Page 19 of 252
Chapter 6: System Implementation And Testing
This chapter focuses on the development environment and strategy. On top of
that, it lays out the testing process, and the various testing techniques.
Chapter 7: System Evaluation And Conclusion
This chapter discuss the procedures used to obtain evaluation results regarding the
proposed IQBAES system in term of effectiveness and efficiency.
Chapter 8: Conclusion And Future Work
This chapter discuss problems encountered, system strength, system constraints,
future enhancements, and knowledge and experience gained.
Page 20 of 252
2
LITERATURE REVIEW
This chapter covers the study on existing systems available in the market. The purpose of
this study is to obtain all the features that are necessary to be included in the new system.
The study seeks to discover how the old systems perform their tasks, their strengths and
weaknesses. This chapter also reviews the latest open source technologies that have
arrived recently in the field of Information Technology. It seeks to discover the concepts
of the new technologies, and how these concepts came about, built upon earlier
pioneering technology. Besides, it covers tools and methods that will be useful for the
project development.
2.1
Analysis Studies
2.1.1
Overview of current Examinations Systems
An examination system in Malaysia consists of four main stages. Each of these
components represents a challenge in organization and implementation.
REGISTRATION
EXAMINATION
PROCESS
MARKS
REPORTING
• List of Students
• Set Questions
• Prepare
• Recording
•
•
•
•
in each class
• Subject
combinations
(packages)
Marking
Scheme
• Execute
Examination
• Mark Test
of Test
Marks
• Process
Results
Mark sheets
Result Slips
Analysis
Awards
Figure 1 – Four stages in the Examination System
The first stage in an examination system is the registration of students for an
Page 21 of 252
examination. In an education institution system, the registration involves obtaining
relevant particulars of each student, and entering the data in a form of Register. Some
institutions are still using a large book as a Register while others have implemented some
form of computerized data entry system. Most rural education institution is still using the
manual system, as they do not have the computer skills needed to implement a
computerized system. Even in most of the large town schools, they are using Microsoft
Excel spreadsheets to handle their examinations processing.
The second stage involves organizing the Examination Process. This includes Setting the
Questions, Preparing the Marking Scheme, Executing the examination (students sitting
for the examination), Marking the examination scripts, Issuing the examination results.
Each of these steps can be subjected to varying degrees of computerization, including
online tests where the students enter their answers in a computer.
The third stage is the recording of marks in marks sheets, processing the results,
including calculating grades, averages and class positioning.
The final stage of the Examination System is the presentation of the results. This can take
the form of displaying the results on screen, printing out the results as mark sheets,
generating result slips, analyzing the results for overall performance. It also includes
selection of students for various incentives, such as prize giving.
2.1.2
Current Process Flow (Manual) or Existing System
The current manual system of school examination systems is that marks are entered on
paper forms that are distributed to each teacher in a particular class. Some teachers would
transfer these marks into a large master mark sheet. All the master mark sheets of each
class are then bound in one huge master examination book. The process flow looks like
this.
Page 22 of 252
Mark sheets
for each
subject
All marks
for each
class
Transfer
Marks
Bind
Mark
Sheets
All marks
for all
classes
Figure 2 – Classic Examination System
Some teachers would enter marks in individual electronic spreadsheets to aid in the
process. Others would use database programs such as dBase IV and Microsoft Excel to
enter the marks in their home computers. As such, they use different file formats and
different file structures, using different types of programs. This makes it extremely
difficult or impossible to combine all the data in the various files to form a master
examination marks database.
Enter Marks into
Excel File or
Access Database
Process
Marks
Print
Marks
Sheet
Bind
Mark
sheets
Figure 3 – Semi-Computerized Examinations Marks System
Page 23 of 252
2.1.3
Case Study 1: UTS Online Examination System
Business, Law and Ethics provides the foundation for all law subjects in the Bachelor of
Business. It tries to achieve an appropriate blend of law and ethics for the 80% of
business students who will need to understand the underlying legal issues of being in
business in Australia (Michael Adam, 2001). For the 1,300 students who study the
subject each year this may be the only law subject they study and as a result their only
chance to receive information on legal compliance and professional responsibility.
The ethics component of the subject allows a broader look at contexts of social
responsibility than a straight legal subject. This lends itself to a case study approach and
the tutorials are based around case studies that identify business problems, explain legal
issues and demonstrate how students can support it with evidence. The evidence for these
legal issues is case law and legislation which is addressed in a two hour lecture each
week. The lecture presents a skeleton of the legal argument around which the students are
expected to apply their knowledge and skills in legal problem solving. Each problem has
a set structure that the students need to know.
The subject has a mid-semester examination to test that the students have a sufficient
understanding of the case law to operate in the Australian regulatory environment. The
mid-semester examination is worth 20% of the total assessment and deals with the
material taught in the first 6 lectures. The examination takes place on UTS Online which
is also used for subject announcements, any online discussion about the subject and the
distribution of resource materials. Prior to the examination practice questions are made
available on UTS Online for the students to become familiar with the assessment tool and
the format of the questions.
Each student who sits the examination undertakes a different set of multiple-choice
questions. The exam takes the questions from pools that have been re-written to change
Page 24 of 252
them into unique questions. The four of five topics covered in the exam each have their
own pool that generates a unique combination of questions for every student. The subject
does not have an extensive database of questions, just enough so that each student gets a
unique exam. The students do not receive any feedback other than their mark but they are
able to access their results immediately through UTS Online.
The students can do the examination at any time between 9.00 am on the Monday until
6.00 pm Friday on the week of the exam. Students have 25 minutes to complete all 40
questions with each question worth half a mark. There is no negative marking. A timer on
the screen helps the students monitor their time. They are warned when they have only
one minute to go. The students then need to submit their answers within in this final
minute.
Students are given two attempts at the mid-semester examination. There are two
examinations scheduled over a two-week period and students are allowed to have a
second attempt in the following week. This allays any fears that there is a drop out or
other technological problem. The best result of the students’ two attempts is used for the
final assessment.
Page 25 of 252
2.1.4
Case Study 2: MIND Online Evaluation System (OES)
MothersonSumi Infotech & Designs Ltd (MIND) has developed online Evaluation
System as a product for online examination and opinion polls, with key features like
question bank, time control, security etc. The needs of OES was raised due to the fact that
conducting exams, checking them and compilation of results take up lot of time in any
organization; also different types of exams have to be designed for different purposes.
Recording of exam results and controlling of examination procedure is also a time
consuming job. Organizations also conduct opinion polls from time and time for various
purposes and conducting these polls and then analyzing the results is very tedious job and
need lot of manpower and time. Thus a need exists which can make these tasks easy and
manageable with least amount of manpower and time.
MIND came out with the solution to design and build an Online Examination System
(OES) in order to provide a cost effective solution to organizations for conducting exams
and opinion polls. Its key features are:
Question Bank
Questions with varying complexity can be included in different exams from a
question bank and assigned to user as per the level.
User defined exams and polls.
Questions from various topics can be included
Control over time
Does not allow user to give exam after the defined time limit.
Stopwatch is displayed so that user can manage the time during the examination.
Page 26 of 252
Security
Robust security system takes care that no unauthorized user can have access.
User can only see his own result but an Administrator can see the result of all
candidates.
Exam & Grading
Exam questions were compiled based on pool of questions from question bank.
Questions were from varying complexity.
Grading was done when exam completed or student has answered all questions.
Other features
Report Cards are generated so that user can immediately view the result.
Results of opinion poll can be analyzed.
User having options to move back and forth while taking the exam.
MIND makes use of the following technical environment to build the OES:
Developed using Java, jsp and servlets
Uses SQL Server as database
Browser based client
The key benefits of OES are:
Saving of resources both manpower and time
Better organization and control
Better security options.
It can be deployed on web and used by organizations conducting online courses.
Page 27 of 252
OES has quite comprehensive features as an online examination system. It consists of a
question bank, which keeps questions with varying complexity from various topics.
However, users or lecturers who need to create an exam are required to manually compile
the questions based on pool of questions from question bank. Grading was done when
exam completed or student has answered all questions. The grading was solely based on
traditional marking strategy. Grading will not based on level of difficulty and student will
be graded based on how well they have performed in the exam or how many questions
they have answered correctly. Report Cards are generated so that user can immediately
view the result.
2.1.5
Case Study 3: ExamManager
ExamManager is a web-based exam assessment system. Exam Manager enables students
to review old and new exams setup by their lecturers, take a graded quiz to test
themselves, setup their own review exams and quizzes from a database of questions in
the subject category of their choice and review their own performance and scores.
On top of that, Exam Manager™ allows lecturer to create multiple-choice questions. User
can add pictures, case studies, and explanations to the questions. Faculty users can either
create their own questions or use the questions that already exist in the question bank.
Faculty users can determine how much time a student will have to complete the exam.
They also control how many times students are allowed to take an exam. Faculty users
have the option of hiding grades from students when they are done taking a quiz.
Once an exam is published, a student can use it as a study tool or an assessment tool.
Study tool: When reviewing an exam, the student will get immediate feedback on
whether the selected answer is correct. An explanation page can be linked to the question,
Page 28 of 252
providing students with study references or further information on the question. Students
can also generate random exams to quiz or review by selecting the category(ies),
difficulty level(s) and number of questions they want to be tested on. Scores of random
quizzes cannot be saved.
Assessment tool: When students take a timed exam, they are only allowed a specific
amount of time that has been set by the faculty. The students will not be able to see the
questions' explanations nor to get immediate feedback. When the exam is ready to be
graded, students will be given the option to go back to the exam to answer questions left
blank.
Once students have taken the exam, Exam Manager™ scores their tests immediately.
Faculty users can choose to hide the exam score from students. The following screenshots
shows a typical screen shot for ExamManager page.
Figure 4 – ExamManager main screen
Page 29 of 252
Figure 5 – ExamManager login screen
As compare to OES, ExamManager provide more flexibility to its users by providing
study tools and assessment tools to its students. Once students have taken the exam,
Exam Manager™ scores their tests immediately. Obviously, student will be graded only
when they have answered all the questions. The questions are not be properly categorized
into various level of difficulty. Student will be graded based on how well they have
performed in the exam or how many questions they have answered correctly.
Page 30 of 252
2.1.6
Case Study 4: Online Courses Examination System (OCES)
OCES is an Internet Based Testing (IBT), which provides extensibility of distance
Examination. This system enables registered students to attend online courses and
undergo training. The student can then measure their proficiency in such courses and
module themselves using the computer based tests. iSummation Technologies provides
such kind of solution in OCES. This application keeps information of registered courses
their modules as well as their progress report. It shows knowledge content it self in
module also. The users of this application can register and enrich their knowledge. They
can study the theories in depth and can assess themselves on the spot.
The application has a front-end that is acts as an interface to the users of the application
and a backend that aids in the management of the application. The features of both are
listed below.
Front-End
The Front End of the Application provides several functionalities to the users:
User can see the detail of the course without registering.
They can access Module information and theories after registering for a particular
Course.
Student will face multiple choice and textual questions.
They can view the progress report after going through in Examination but they
will see temporary report .The complete report will be showed after judgment of
textual answer.
Back-End
The Back End of the Application is used by administrator:
Administrator can add/delete/edit the courses.
Page 31 of 252
He can also change the launch status of Courses that will be affected in Front End.
Users, registered for a particular course can appear in that Examination and view
Reports also.
Administrator can update the visual Interface for every Course.
In different Courses there may be same Modules included.
He can add / edit / delete Modules
He can view report.
He can change the status of a User.
He has to check out the textual answer of users
Page 32 of 252
2.1.7
Case Study 5: Web-based Adaptive Testing (WAT)
Web-based Adaptive Testing is an online assessment system which has a module called
“Test Module” that can generates questions adaptively, based on the student’s ability and
also calculates the results. This system is able to asses the student’s knowledge based on
his/her abilities to answer the questions, given in the form of quizzes mainly multiplechoice and true or false.
The architecture of the system is based on simple client-server architecture. A Java
applet will be downloaded onto the client’s machine, where the questions will be given to
the students. Once the student answers the question, the applet sends the reply back to the
server to calculate the result and send the next relevant question. This is possible because
of Java’s powerful networking classes, JDBC capability and able to access any platforms.
The questions are selected from various topics set by the teacher/author, who is in charge
of the system. There are 3 difficulty levels and the teacher sets these difficulty levels,
based on their difficulty. Initially, the student is given a pre-test, which is nothing but
basic concepts about programming and computers. These questions itself will be in
random so as to eliminate cheating. The scores obtained from this pre-test will be used as
the starting estimated ability value. This pre-test is given to give equal opportunities to
students when they start the actual quiz session. The policy of adapting the question is
based on the logic of Computer Adaptive Testing (CAT), which is given below.
1. All the items that have not yet been administered are evaluated to determine
which will be the best one to administer next, given the currently estimated ability
level.
2. The “best” next item is administered and the student responds.
3. A new ability level is computed based on the responses of the student
4. Steps 1 to 3 are repeated until a stopping criterion is met.
Page 33 of 252
Questions would be given randomly to eliminate cheating. The new ability level is
computed based on the following equations. This has been used in MANIC system during
the quiz sessions to compute the new estimated ability level.
NewValue = OldValue + OldValue * (level of Question / 10)
NewValue = OldValue + OldValue * (5 - level of Question / 10)
The first equation is for correctly answered questions and the second one is for wrongly
answered questions. This is because a student should not be punished too much if he
could not answer a hard question. When the student starts the quiz, the OldValue will
have the value scored by the student during pre-test. After that, for every question, this
value will be updated, based on student’s response. The quiz grade is in the boundary 0 to
1.
Termination criteria are defined based on certain parameter. The parameters for ending
the test would be either half of the questions answered by the student are correct, in this
case, he/she does not have to continue the test since the student would have reached the
threshold value or the student has answered the maximum questions allowed by the
system.
Finally, at the end of quiz, the results will be displayed on the screen. It will contain the
student name, id, the total number of questions he/she has taken, number of right answers
and wrong answers, whether the student has to revise the topic or not will be shown on
the screen. These results will be stored in a user profile manager, which keeps track of all
users, their ids, question ids, test ids etc.
Page 34 of 252
2.1.8
Case Study 6: SIETTE
SIETTE is a web-based adaptive testing system which was developed in year 1998. It
implements computerized adaptive tests (CAT). These tests are tailor-made, theory-based
tests, where questions shown to students, finalization of the test, and student knowledge
estimation are accomplished adaptively. To construct these tests, SIETTE has an
authoring environment comprising a suite of tools that helps teachers/instructors create
questions and tests properly, and analyze students’ performance after taking a test.
SIETTE works as a student knowledge diagnosis tool. It is a multilingual system,
currently translated to Spanish and English. The components of the SIETTE system are
Question Knowledge Base which is a database of questions related to a test. All
these questions are calibrated with some parameters. All the contents are
structured in subjects. Each subject comprises a set of topics. Each topic can be
decomposed
into
a set
of subtopics
following
aggregation relations.
Questions/items are associated to topics in such a way that if a question is related
to a topic, it means it can be used to assess the topic and any parent of that topic.
Tests are defined based on the topics they assess, the rules for selecting the items
and some finalization criterions.
Test Edition Module, which is the tool used by teachers to define the tests and the
structure of the subject domain: topics, questions, relationships between them, and
relative weights of topics in the test. This information is stored in the question
knowledge base.
Student Model, which is created and updated by SIETTE for each student that
takes the test. Basically, the student model consists of a vector of K probabilities
(p0, p1, ..., pK-1), where pi represents the probability that the student has reached
certain knowledge level i about the domain, where 0 is the lowest level and K-1 is
the expert level. In addition, the student model stores information about which
questions have been asked by the system.
Page 35 of 252
Test Generator, is responsible for selecting the questions that will be posed to
each student. The generation process will be guided by the student model and the
test specifications.
Associated to each question, SIETTE have the following parameters:
•
General question data, which are the title and the number of possible answers.
•
Question stem, should be as simple as possible, and never confusing or
ambiguous.
•
ICC parameters, it will be considered that the student's knowledge can be
measured in terms of a discrete random variable q that takes integer values in [0,
K-1]. This means that an ICC (Item Characteristic Curve) is a set {pi} of K values,
that give the probability of a correct answer given that the student's knowledge is
i, i.e., pi = P(Ui=1/ q=i) for i=0,...,K-1. The discrete distribution is derived from
an underlying continuous three-parameter logistic distribution, so a, b, and c
values are also needed. The guessing factor c is determined automatically by
SIETTE according to the number n of different answers that are shown to the
student (c = 1/n). On the other hand, the teacher must define the discrimination
index a and the difficulty degree b. The former must be a number between 0.5 and
1.5. Concerning the latter, since too easy or too difficult questions are hardly
useful, only natural numbers between 0 and K-1 are allowed. (Ricardo Conejo,
2000)
•
Extreme values of the ICC are not very significant, so only the "central" fragment
has been discretized. Namely, and following some empirical adjustments, the
interval [(K-1)/2, (K+1)/2] has been selected for discretization. The interval has
been divided in K-1 subintervals of equal length and the separation points give the
values of the continuous variable q that are in correspondence with the discrete
variable; that is to say, pi = ICC(i - (K-1)/2) for i= 0, 1, ..., K-1. (Ricardo Conejo,
2000)
Page 36 of 252
In SIETTE, when a student takes a test, he will be administered question by question in
terms of his knowledge level current estimation. The question selected to be administered
is the one which will make the student’s knowledge estimation more accurate. After the
student answers the question, his knowledge level is estimated taking the response into
account. This process is carried out until his knowledge estimation is accurate enough.
The student model will be created and updated by the system for each student that takes
the test. This information will be used to provide the test generator module with adaptive
capabilities. To present an example of student model, SIETTE will assume that the
number of knowledge levels fixed by the teacher is K=11. Then, the student's knowledge
level would vary from Level 0 (novice) to Level 10 (expert). Initially, and in the absence
of any other information, the probability is uniformly distributed among the 11 levels
and, as the student takes the test, they are updated. The system keeps track of different
types of data regarding this particular student:
•
Date that the test was taken, knowledge level reached, and confidence interval
associated.
•
Probability that the student has reached each knowledge level.
•
Topic distribution in the test posed.
•
Questions that have been selected and presented to the student.
Once the test is finished, the student model will be updated with all the information it
contains until the next session.
The test generator will use the algorithm which consists of three main procedures: (a)
question selection, (b) knowledge level estimation, and (c) termination criterion.
Instructor/teacher can choose between three different question selection procedures:
a. Bayesian procedure, which selects the question that minimizes the posterior standard
deviation
b. Difficulty-based procedure, which selects the question that gives the minimum
distance between the mean of the ICC and the mean of the current student's
knowledge distribution
c. Random procedure which the question is selected randomly.
Page 37 of 252
With the selected procedure, the system will extend with the following features:
•
Random question selection. If the selection criterion does not allow differentiating
between two questions, the question is selected randomly between the possible
alternatives.
•
Content balancing. To assure content balanced tests, SIETTE uses the weights
specified by the instructor for each topic. These weights determine the desired
percentage of questions about each topic. SIETTE compares the empirical
percentages of the questions that have already been posed with the desired
percentages, and selects the topic with the biggest difference as the target topic.
Then, SIETTE selects the best next question belonging to that topic using the ICC
associated to each question.
Once the best question has been chosen, the system poses it to the student. The system
computes the student's new proficiency level once he/she has answered the question. The
test can be terminated once it reaches the termination criteria define by the instructor, for
instance, the system has already posed all the questions in a test or it has posed at least
the minimum number of questions
In order to use this system, student must provide a username and a password to access the
tests. Although there are tests restricted to predefined sets of students, others can be
freely accessed by simply supplying some optional personal information (e.g. name,
surname, email, etc). Students can be organized in groups defined by teachers. When the
student has been authenticated, his personalized list of subjects is displayed. Three types
of user’s profiles are managed in SIETTE: teachers, learners and examinees. Teachers are
responsible of test creation. They can have a look at their tests before any student takes
them. Consequently, when a teacher accesses to the test generator tool, all the tests of his
subjects (or the subjects on which he has privileges) are available. Learners are students
that take tests for self-assessment. In this kind of tests, question correction is shown
immediately after the student answers. These tests are administered with question hints
Page 38 of 252
and feedbacks. Examinees are students that take tests for academic grading. No question
hints or feedbacks are provided.
2.1.9
Case Study 7: Architecture Tutor
The Architecture Tutor is an intelligent tutor system for computer architecture developed
at Azusa Pacific University. This system applies functional concept graph as an
assessment approach in the system. Specifically, its approach is for course subjects that
are based on well-formulated concept hierarchies and applicable for tutoring subject in
science and engineering. In the functional conceptual graph, a set of nodes are being
defined. These nodes are corresponding to each question in the system. Each node of the
graph is associated with a value. In the implementation, this system uses nodes in the
graph to represent questions in the system. Each node is corresponding to a question with
an associated value. Typically, if a node consists of a number of child nodes, this node
will have a higher value than its child node. If a student is able to solve a problem
involving more concepts (more nodes in the graph) about a subject, he or she should have
a reasonably good understanding of the subject. This does help in assessing the student’s
understanding of a certain subject. The screen shot for this system is shown as below:
Page 39 of 252
Figure 6 – Screen shot of Architecture Tutor
2.1.10 Relationship to existing work
There are a few web-based education systems that being reviewed in this section. The
IQBAES that I propose clearly fits into the family of these web-based education/testing
systems which were designed earlier. SIETTE is one of the systems which influence my
software design the most. It is an example of a Web-based adaptive testing system. The
kind of learning material it possesses is questions and the only thing it can do is to
generate an adaptive sequence of questions to assess student’s knowledge. SIETTE is
incomplete by its nature and has to be used as components in distributed Web-based
Adaptive and Intelligent Educational System (AIES).
Question weights are values assigned to each question measuring the difficulty level and
it is relative importance of the material being tested. In such a case, the higher the weight
of a question, the greater the difficulty. However in SIETTE or any other web-based
Page 40 of 252
examination/testing system, there is no algorithm in defining the difficulty of question.
Assessing the difficulty of a question is a subjective process and is usually done by the
instructor/teacher based on experience. Hence an accurate assessment of the difficulty of
exam questions is important and will help to better allocate credits/scores/marks to
exams. I will propose and incorporate a question difficulty assessment algorithm into
IQBAES.
On top of this, SIETTE as well as most of the web-base educational systems use a limited
set of assessment questions generated by the instructors or teachers. On top of providing
a question/answer interface for instructors/teachers to create and add questions into
question bank, my propose system will define an approach which serve as an automatic
question generator.
An important feature of SIETTE is adaptivity. Speaking about adaptivity, SIETTE
attempts to be different for different students by taking into account information
accumulated in the student models. SIETTE is able to be aware of student’s behaviour so
that it can take into account the level of knowledge and provide the student with the right
kind of question. SIETTE keeps track of many aspects of the student. For instance, how
much and what students learned, what learning styles are successful for students and what
seem to be less successful and so on. Student modelling does keeping track of all these
aspects of a student’s learning. Without a student model, it is impossible for such a
learning environment to adapt to the needs of individual students.
Student model is used to predict the current state of knowledge of the student. The
function of the student model is to capture a student’s understanding of the domain
knowledge. A student model records a student’s progress towards a complete solution
(Kinshuk, 1996). Student modelling mainly occurs at runtime when the student uses the
system. It is through the evidence provided by the student's inputs to the system that the
student model is created. Although there is no consensus as to what should be included in
Page 41 of 252
a student model, a comprehensive student model would include the student’s prior
relevant learning, the students’ progress, and other types of student-related information.
However, implementing such a model is often a difficult task. The main problem that the
student model has is (Ericco, 1996) the lack of clear or unambiguous semantics for the
(epistemic) concepts deal with. Hence, without a robust student model, the web-based
examination/educational system will not be considered as “Intelligent”. In order to make
the system more adaptively, an educational system should not fully rely on student
model. Thus, I would like to propose a question model (intelligent questioning system)
into my propose system, IQBAES. It will consist of an adaptive question management
system that uses a hierarchical knowledge map to direct the assessment process based on
the students’ degree of understanding of concepts.
2.1.11 IQBAES Approach
Based on the disadvantages and the limitations of existing systems discussed above, a
more efficient approach had to be used. The IQBAES approach, which will be discussed
here, is aimed at solving the problems presented by the systems stated above. The
IQBAES will consist of 3 core components namely
a. Question difficulty assessment algorithm
b. Automatic question generator
c. Intelligent questioning system
2.1.11.1 Question difficulty assessment algorithm
IQBAES is defining an approach to access questions difficulties and this approach is
making use of functional concept graph. A functional conceptual graph G also refers to
Directed Acyclic Graph (DAG) and it consists of a set of nodes V and a set of directed
edges E (Li and Sambasivam, 2003). The edges connect the nodes of V. Each node of the
DAG is associated with a level. Node may have a set of incoming edges that connect to
lower level nodes. A node without any incoming edge is a source node. A node is also
Page 42 of 252
associated with a value. The value of a node is computed from the value of the input node
connected by edge.
By using the DAG, IQBAES defines the degree of difficulty as D = w1N + w2P + w3M,
where N is the number of conditions given in the questions (that is, the number of
terminal nodes), P is the number of downward edges in the paths traversed during
question generation, and M is the number of upward edges traversed during question
generation. Three weight factors w1, w2 and w3 is used to balance between the path
length and the number of conditions. Typically, w1<w2<w3 because upward edges
traversed represent more difficult concept association and downward edges, and the total
number of edges, which corresponds to the number of problem solving steps involved,
carries more information about the effort needed in problem solving. This definition of
degree of difficulty serves only as a rough guidance during automatic question
generation. It is by no means a perfect measurement. However, it does help in assessing
the student’s understanding of a certain subject. If a student is able to solve a problem
involving most concepts and equations about a subject, he or she should have a
reasonably good understanding of the subject.
2.1.11.2 Automatic question generator
IQBAES will build an automatic question generator. The questions to be generated
typically are the types of direct evaluation of a result from given conditions. The result
can be compared against a value or a range.
The algorithm is presented as below:
An integer named credit is used to represent complexity. The larger is this number, the
more complex the question is. It uses a queue OPEN to store those nodes that are not yet
expanded. A node is expanded if its descendants are generated. It is assumed that each
node Ni stores an estimated maximum difficulty Di that is obtained by downward
expanding the concept graph rooted at this node to the maximum. In addition, we assume
that we have two basic types of nodes in a concept graph: the array nodes and the regular
Page 43 of 252
nodes. An array node may have several descendants of the same class and a result is
computed from the descendent values. A regular node has descendants of different
classes.
1. Define a queue: Queue OPEN = new QueueList();
2. Select a node Nr from the entire concept graph and mark it as “closed”.
3. If Dr < credit, then
Traverse upward;
credit = credit–w3;
Push the parent of Nr to OPEN: OPEN.enqueue(parent of Nr);
else
Randomly decide traverse up or down to node N;
Push the descendants of N to OPEN: OPEN.enqueue(descendants of N);
credit = credit–w2* num_descendants;
3. Dequeue a node N from OPEN and mark it as closed: OPEN.dequeue();
4. If D >= credit, then
Select a random value in the interval specified in the node for each node in
OPEN queue;
Finish question generation;
else
Randomly decide traverse up or down to node Nd;
Push the descendants of Nd to OPEN: OPEN.enqueue(descendants of Nd);
credit = credit-w2*num_descendants;
5. Go to step 3.
Remark: the credit value in fact denotes the desired question difficulty. The difficulty of
the automatically generated question may exceed the desired difficulty by a small
percentage
Page 44 of 252
2.1.11.3 Intelligent questioning system
The Intelligent Questioning System is comprised of three basic components: the User, the
Knowledge System and the Question System. The Knowledge System presents the user
with his or her current level in the system. The Questioning System then identifies an
appropriate set of questions within the level. A question is then randomly selected and
presented to the user. When the user completes exam, the Questioning System provides
an assessment measure to the Knowledge System. The Knowledge System then updates
the user’s status in the system.
To implement this system, several databases are required. The question database contains
the complete set of questions. The student system database encapsulates a model of the
student’s progress, maintaining information about which questions the student has
attempted and the assessment measure for each question. The knowledge system database
maintains the structure of knowledge flow.
The assessment system uses the weight of the question to provide the incremental score
contributed by the answer to this question. The “knowledge key” information about each
question is used by the adaptive questioning system, together with student input and the
current state of knowledge, to determine which question to pose next. After each question
is answered, the question management system updates the knowledge map.
Questions are grouped into a Question Set based on level of difficulty. The questioning
process involves: randomly selecting a question from the set, completing the question,
gathering assessment information, and providing a measure of the degree of
understanding back to the knowledge model for use in knowledge assessment. Besides
selecting and asking a question, the Question Set also provides a mechanism to quantify
degree of understanding. This is used to assist in assessing the student’s level of
competence in each topic.
Page 45 of 252
Example:
When the student logins into IQBAES and starts the examination process, a question will
be displayed to the student based on his/her current level/profile. If this is the first time
the student has attempted question on this chapter, then a random question will be
selected by the system and displayed to student. For each question, the system maintains
a set of attribute in the database. For instance, for each question in the question bank, the
question database contains text of the question, figures if any, difficulty level or question
weight and parameter that need to be defined.
The question generator will pick the question and assemble the html code for the question
together with the images (if any). This information is then passed to the web server using
JSP script. If the answer is correct, the student record is updated and the option for
another harder question is given. If the answer is incorrect, the student record is updated
and the option for another easier question is given. Figures below presents the example of
what will actually happen in the system when a student manages to answer the question
correctly or fail to answer a particular question.
Page 46 of 252
Stage 1:
The student’s profile stores the student’s understanding level and is used for constructing
the first question.
Figure 7 – Screen shot of Sample Question on Stage 1
Page 47 of 252
Stage 2:
When the student fails to answer the first question correctly, an easier question will be
displayed to student. In the above example, the system firstly tests the student’s
understanding on the attribute of software object. The student fails to answer the question
correctly and hence he/she was directed to an easier question as below. The system will
choose an easier question based on the level of difficulty value as defined for each
question. The question to be selected will be chosen based on a topic/chapter that the
student is currently undergone. This question was intended to test the student’s
understanding on the attribute of all real world objects have and hopefully with this
similar concept in mind, the student can answer the question posted earlier where
software object should also have the same attribute as the real world object.
Figure 8 – Screen shot of Sample Question on Stage 2
Page 48 of 252
Stage 3:
Upon successfully answering the previous question, another harder question will be
posted to student. The system will choose a harder question based on the level of
difficulty value as defined for each question. The question to be selected will be chosen
based on a topic/chapter that the student is currently undergone.
Figure 9 – Screen shot of Sample Question on Stage 3
Page 49 of 252
Stage 4:
When the student manages to the above question correctly, a harder question will be
directed to the student. To answer the question “what are the static variables and methods
of a class” below, student needs to have a correct understanding on what is a class.
Hence, student will need to answer the question on stage 3 correctly before the question
on stage 4 will be posted to the student.
Figure 10 – Screen shot of Sample Question on Stage 4
Page 50 of 252
Typically the process can be depicting as in figure below.
Figure 11 – Screen shot of questioning process
Page 51 of 252
2.2
Review on Latest Technologies
2.2.1
Client-Server Architecture
In the early history of distributed computing, huge mainframe computers did most of the
processing of data and functionality, handling multiple users. Dumb terminals were
connected to access the resources on the mainframe. As desktop computers became
cheaper and increased greatly in processing power, some of the processing was done on
these intelligent terminals, later referred to as clients. The mainframe became the
“Server”.
This led to the Client-Server model of computing, that involves a central server handling
the database functionality. The ‘fat-client’ handles the user interface, and some of the
business logic, requesting data access from the server.
However, there are problems with this architecture, especially when maintenance of the
client software involves replicating new versions over a widely distributed environment.
With the advent of the World-Wide Web, ‘thin clients’ came into place, where the only
software required is an Internet browser. The bulk of the processing is now distributed
into a multi-tier architecture at the server level.
2.2.2
Two-Tier Architecture
Two-tier architecture refers to client/server architecture. The user interface runs on the
client side and the database is stored on the server. The actual application logic can run
on either the client or the server. There are only the architecturally tiered data server and
client.
2.2.3
Three-Tier Architecture
Three-tier architecture is a special type of client/server architecture consisting of three
well-defined and separate processes, each running on a different platform.
Page 52 of 252
The three tiers consist of:
Client-tier: Client tier runs on the user’s computer and provides user interface.
Application-server-tier: Application server tier is the functional module that actually
process data. This middle tier does not explicitly present in 2-tier architecture. This tier
protects the data from direct access by the clients.
Data-server-tier: Data-server-tier is a database management system (DBMS) that stores
the data required by the middle tier.
2.2.4
J2EE technology
The idea of J2EE was originate from the combination of client-server architecture and
component technology. It addresses architectural requirements for a distributed
computing environment. J2EE architecture is based on 3 tier-models which are:
Clients on the front-end for presentation (browser)
Middle tier where applications and business functions reside (Web application
servers)
Data resources and transaction application on the back-end
The J2EE architecture defines a client tier, a middle tier (consisting of one or more sub
tiers), and a back-end tier(Dr. Max Chi, 2001). The client tier supports a variety of client
types, both outside and inside of corporate firewalls. The middle tier supports client
services through Web containers in the Web tier and supports business logic component
services through Enterprise JavaBeans (EJB) containers in the EJB tier. On the back end,
the enterprise information systems in the EIS tier are accessible by way of standard APIs.
Figure below illustrates the various components and services that make up a typical J2EE
environment:
Page 53 of 252
Figure 12 – J2EE Environment (from Sun Micro-system)
Central to the J2EE component-based development model is the notion of containers.
Containers are standardized runtime environments that provide specific services to
components. Components can expect these services to be available on any J2EE platform
from any vendor. For example, all J2EE Web containers provide runtime support for
responding to client requests, performing request-time processing (such as invoking JSP
pages or servlet behavior), and returning results to the client. In addition, they provide
APIs to support user session management. All EJB containers provide automated support
for transaction and life cycle management of EJB components, as well as bean lookup
and other services. Containers also provide standardized access to enterprise information
systems for example, providing access to relational data through the JDBC API.
In addition, containers provide a mechanism for selecting application behaviors at
assembly or deployment time. With deployment descriptors (XML files that specify
component and container behavior), components can be configured to a specific
Page 54 of 252
container's environment when deployed, rather than in component code. Features that can
be configured at deployment time include security checks, transaction control, and other
management responsibilities.
While the J2EE specification defines the component containers that a platform
implementation must support, it doesn't specify or restrict the containers' configurations.
Thus, both container types can run on a single platform, Web containers can live on one
platform and EJB containers on another or a J2EE platform can be made up of multiple
containers on multiple platforms.
With features designed to expedite the process of developing distributed applications, the
J2EE platform offers several benefits:
•
Simplified architecture and development
•
Freedom of choice in servers, tools, and components
•
Integration with existing information systems
•
Scalability to meet demand variations
•
Flexible security model
Page 55 of 252
2.2.5
J2EE Framework
Frameworks are reusable, “semi-complete'' applications that can be applied to produce
custom applications (Johnson, 1988). Frameworks are tested, proven software designs
and implementations that reduce the cost, accelerate development speed and improve the
quality of software (Objectwave, 2004).
A typical framework will provide developers with modular, reusable and extensible
software components. On top of that, modular frameworks reduce the time, effort and
investment required to understand and maintain existing software. It contributes to the
improvement in productivity, as well as enhancing the quality, performance, reliability
and interoperability of software. Extensibility is essential to ensure timely customization
of new application services and features. Beside that, framework can reduces the cost of
building and maintaining software.
A typical open source J2EE framework is a free framework created for building J2EE
application. For instance, Struts is a JSP/Servlet open source framework created within
the Jakarta project by the Apache Software Foundation for building J2EE applications. It
accelerates application development based on the Model-View-Controller (MVC) design
pattern. MVC is an abstract design pattern for software applications that was first
introduced on the SmallTalk platform. The goal of the MVC design pattern is to separate
the application object (model) from the way it is represented to the user (view) from the
way in which the user controls it (controller).
The model provides a representation of the data to the components comprising the view.
It also knows about all the operations that can be applied to transform the data. It knows
nothing about how it is being viewed by the users. The view provides presentation
services in the application, such as a data entry screen or a screen rendering a query
result. The controller is responsible for linking the view with the model. The controller
can be thought of as the controlling influence on the state of the model. All paths of
navigation in the application are channeled through the controller.
Page 56 of 252
Normally view is represented by jsp. The controller is implemented by an ActionServlet.
Users can extend the controller’s behavior by creating Action classes to trigger the
execution of a user request.
2.2.6
.NET Framework
The .NET platform consists of the following parts:
The .NET framework
Visual Studio.NET, which creates .NET solutions using VB.NET, Managed C++,
and C#
Figure 13 – The Microsoft .NET platform
The .NET Framework is a Microsoft technology that consists of five concepts:
The Common Language Runtime (CLR) – hosts all .NET code and provides core
services. It is a managed environment that executes all .NET programs and
functions as the execution engine. This is similar to the Java Virtual Machine
(VM). Code that runs within a CLR is referred to as ‘managed’ code.
Page 57 of 252
The Common Type System (CTS) and the Common Language Specification
(CLS) – CTS fully defines all possible data types supported by the runtime. CLS
is a subset of CTS that must be supported by all .NET languages.
The Base Class Libraries – a comprehensive, object-oriented collection of
reusable classes. These include ASP.NET, ADO.NET, web applications and Web
Services classes.
Microsoft Intermediate Language (MSIL) – all code written in .NET languages
compiles to a platform-neutral code called MSIL. It is a binary intermediate form
for a .NET program. This is similar to Java byte code.
Assemblies – the fundamental unit of deployment, version control, reuse,
activation and security.
2.2.7
Comparing J2EE and .NET Platform
.NET shines in its tool and language support. J2EE is a winner in industry maturity and
interoperability support (Chad Vawter and Roman June 2001). Both platforms are good
in legacy and standards support. J2EE offers only Java as a language of development, and
consists of many disparate technologies.
J2EE and .NET are both based on the concept of 3-tier architecture. The aim is that the
construction and delivery of the user interface is separate from the construction of the
business logic, which in turn is separate from the back-end data or infrastructure being
accessed.
Page 58 of 252
The conceptual Architectural comparison between the J2EE platform and .NET platform
can be divided into Presentation and Access, Business Logic, Connectivity and Runtime,
as shown in Figure 14 – The Conceptual Java Architecture and Figure 15 – The
Conceptual .NET Architecture.
Figure 14 – The Conceptual Java Architecture
Figure 15 – The Conceptual .NET Architecture
Page 59 of 252
2.2.8
Component Technology
Code reuse began with sharing common functions and subroutines. The problem was that
these codes were not portable. When the object-oriented programming model was
introduced, it brought about the concept of hiding the implementation, while exposing
only the header. However, portability was still a problem. Then component technology
arrived, which uses interface-based programming. COM, DCOM and JavaBeans worked
well in an Intranet environment. However, the problem lies in distributing components
across the Internet.
2.2.9
Application Server and Web Server
An application server is a component-based product that resides in the middle-tier of a
server centric architecture. It provides middleware services for security and state
maintenance, along with data access and persistence.
2.2.9.1
Internet Information Service (IIS)
This is a Microsoft-specific Web server and runs only on Windows platforms. It is an
enterprise-level Web Server that is included with Windows 2000. This is the server of
choice when selecting the .NET platform. In order to use IIS as a Web Server for
development, it is necessary to have Visual Studio Developer rights. Also, to debug
applications, it is necessary to have Debugger rights. It supports the Windows GUI and
provides integration with other Windows mechanisms, such as Active Directory, MS
SQL Server and Windows Security services. IIS also supports the benefits of COM+.
2.2.9.2
Microsoft Personal Web Server (PWS)
PWS is a scaled-down version of IIS for personal computers. It is ideal for educational
institutions, small businesses and individuals who only have access to Windows 98.
Page 60 of 252
2.2.9.3
Apache Web Server
The Apache Software Foundation maintains Apache Web Server. It is currently the most
popular Web server because it is stable, reliable, efficient and portable. It is an open
source product and runs on UNIX, Linux and Windows platforms. The source code for
Apache is freely available. This has spurred the development of Apache functions from
many volunteer programmers (and recently from IBM). Apache is modular in structure,
so users can pick and choose modules to fit their requirements. However, there is no GUI
available yet from Apache to manage this server.
2.2.9.4
Tomcat
This is an open source application server bundle together with apache web server
developed by the Jakarta Project. It is a servlet engine that serves as the official
Reference Implementation of the Java Servlet and Java Server Pages technologies.
Tomcat is developed in an open and participatory environment and released under the
Apache Software License.
2.2.9.5
Other application server and web server encountered in the literature
include
Sun Application Server 7
Oracle 9i Application Server
IBM Websphere Application Server
Weblogic
Silver Stream Market Maker
JRun
JBoss
Page 61 of 252
2.2.10 Operating System
The following Operating Systems were reviewed:
2.2.10.1 Microsoft Windows Family
This is a set of commercial operating systems offered by Microsoft Corporation, and the
latest products include Windows 2003 and Windows XP. Windows is a menu-driven
operating system that uses a graphical user interface (GUI) as its main method of
communication with the user. Its user interface is very user friendly. It is the most
popular operating system.
2.2.10.2 UNIX
UNIX has three major advantages:
It is portable from large systems to small systems
It has powerful utilities
It is device independent
The UNIX operating system is written using the C programming language. However it
has some distinct disadvantages:
There is no single standardized version of the software.
Its commands are so terse that it is not user-friendly to novices
2.2.10.3 Linux
This is a free, open source operating system that is gaining widespread acceptance in the
industry. Its commands are similar to the UNIX operating system. This operating system
is more scalable than the Windows operating system. It consists of a Linux kernel that is
totally different from UNIX. The Linux system includes a multitude of components,
borrowed from other development projects.
Page 62 of 252
2.2.11 Database Server
A "database engine" is the core process that a database management system uses to store
and maintain data. It serves two functions. The first is to store information, and the
second is to process requests for stored information. The following Database Servers
were reviewed:
2.2.11.1 MySQL
We will make use of this database server in IQBAES system. MySQL is a multi-user,
multithreaded RDBMS server that uses SQL to interact with and manipulate data. This is
an open source relational database product.
It can be downloaded for free at
www.mysql.com. It is a high performance and scalable DBMS. It has an in-memory
query results cache that contributes to its performance. Most agree, however, that it
works best when managing content and not executing transactions. Its important features
are:
•
Multithreaded capability enables the database to perform multiple tasks
concurrently.
•
Support for various programming languages: C, C++, Java, Python, Perl, and
PHP.
•
It runs on just about any operating system. Specifically, it runs on Windows,
Linux and UNIX
•
The ability to handle large databases.
•
The ability to access tables from different databases by using a single query.
2.2.11.2 Microsoft SQL Server
SQL Server is a client-server database management system. Unlike file-based databases,
client-server database engines manage read-write operations to the database. Because of
this, client-server databases such as SQL Server can handle many more concurrent users
and vastly greater amounts of data.
Page 63 of 252
SQL Server and the MSDE are server-based database systems. Requests for information
are processed on the server and only the resultant data is sent back to the client
application. This scenario greatly reduces network traffic; it also allows users to access
data and experience acceptable application performance without a high-end client
workstation.
SQL Server runs only on the Windows platform, which severely restricts its portability.
SQL Server 2000 comes in seven editions: Enterprise, Standard, Personal, Developer,
Desktop Engine, and SQL Server CE. The first two editions are the choices if a developer
is building a database or analysis application that accesses SQL Server 2000. They can
only be installed on server operating systems, namely Microsoft Windows 2000 Server,
Windows 2000 Advanced Server, Windows 2000 Datacenter Server, and Windows NT
Server 4.0. Between these two editions, the features are as follows:
SQL Server 2000 Enterprise Edition
It includes the complete set of SQL Server database and analysis features. It
supports the largest Web sites and enterprise online transaction processing
(OLTP) and data warehousing systems. It supports up to 64 gigabytes of memory
and up to 32 processors. It can divide database workload across independent
servers. It supports System Area Networking (SAN) connection technologies
between servers. For availability, this edition includes features for mission critical
applications such as failover clustering to ensure that applications stay up and
running even when disaster strikes. In terms of performance, this edition speed up
an application by taking full advantage of symmetric multiprocessor (SMP)
computers, faster queries and transaction applications.
SQL Server 2000 Standard Edition
This is a more affordable option for small-sized and medium-sized organizations.
Standard Edition can be used on symmetric multiprocessing systems with up to 4
CPUs and 2 GB of RAM. It includes the core functionality needed for nonmission-critical e-commerce, data warehousing, and line-of-business solutions. It
Page 64 of 252
has all the data mining features and the core OLAP functionality. Apart from
these, it also includes Data Transformation Services (DTS), Replication, Full-Text
Search, English Query, Stored procedure development and debugging tools, SQL
Profiling and performance analysis tools.
SQL Server 2000 Personal Edition
SQL Server 2000 Personal Edition is ideal for mobile users who spend some of
their time disconnected from the network but run applications that require SQL
Server data storage, and for stand-alone applications that require local SQL Server
data storage on a client computer.
SQL Server 2000 Developer Edition
This edition allows developers to build any type of application on top of SQL
Server. It includes all of the functionality of Enterprise Edition but with a special
development and test end-user license agreement (EULA) that prohibits
production deployment.
Microsoft SQL Server 2000 Windows CE Edition
Microsoft SQL Server 2000 Windows CE Edition (SQL Server CE) is the
compact database for rapidly developing applications that extend enterprise data
management capabilities to devices.
SQL Server 2000 Desktop Engine
SQL Server 2000 Desktop Engine is the successor to Microsoft Data Engine
(MSDE) 1.0, which was based on SQL Server version 7.0. As such, SQL Server
2000 Desktop Engine is often referred to as MSDE 2000. The most significant
characteristic of the Desktop Engine is that it is a redistributable version of the
SQL Server relational database engine. The Desktop Engine has the smallest
Page 65 of 252
footprint of any edition of SQL Server 2000. The SQL Server 2000 Desktop
Engine does not include graphical management tools; the application distributing
the engine is usually coded to perform any needed database administration.
2.2.11.3 IBM DB2
This is a desktop version of IBM’s mainframe database product running on OS/390. It
can be programmed using the Call Language Interface (CLI). In terms of performance,
this is the top DBMS product. There are more administrative products on the market for
DB2 on OS/390 than for the others, making it easier to manage. However, there is little
difference on the desktop versions.
2.2.11.4 Oracle 9i Database
This is a very reliable database product from Oracle Corporation, which is the current
industry leader in database technology. There are more programmers who can program
Oracle DBMS than any other. Oracle 9i Database comes in three editions: Enterprise,
Standard and Personal. Using this database requires a lot of configurations, which are
beyond the scope of this dissertation.
2.2.12 Database Access Technology
Databases could be accessed using the following technologies:
2.2.12.1 JDBC
JDBC is short for Java Database Connectivity, a Java API that enables Java programs to
execute SQL statements. This allows Java programs to interact with any SQL-compliant
database. Since nearly all relational database management systems (DBMSs) support
SQL, and because Java itself runs on most platforms, JDBC makes it possible to write a
single database application that can run on different platforms and interact with different
DBMSs.
Page 66 of 252
JDBC is similar to ODBC, but is designed specifically for Java programs, whereas
ODBC is language-independent. With a JDBC technology-enabled driver, you can
connect all corporate data even in a heterogeneous environment.
2.2.12.2 Hibernate
Hibernate is an open source technology that provides an easy-to-use and powerful objectrelational persistence framework for Java applications. It is an object-relational mapping
technology that is an alternative to JDBC, EJB entity beans, and JDO. It allows
programmer to query and persist data in a relational database through object manipulation
instead of through SQL. Hibernate provides support for collections and object relations,
as well as composite types. In addition to persisting objects, Hibernate provides a rich
query language to retrieve objects from the database, as well as an efficient caching layer
and Java Management Extensions (JMX) support. User-defined data types and composite
primary keys give additional flexibility to support legacy applications. It supports
numerous databases, including Oracle and DB2, as well as popular open source databases
such as PostgreSQL and MySQL. Hibernate uses runtime reflection to determine the
persistent properties of a class. The objects to be persisted are defined in a mapping
document, which serves to describe the persistent fields and associations, as well as any
subclasses or proxies of the persistent object. The mapping documents are compiled at
application startup time and provide the framework with necessary information for a
class. Additionally, they are used in support operations, such as generating the database
schema or creating stub Java source files.
2.2.12.3 Open Database Connectivity (ODBC)
A consortium of companies designed ODBC, including Microsoft, Lotus, Oracle and
IBM. It was one of the first attempts to create a generic way of talking to different
database engines. However, it was built to access relational data from relational
databases. It is accessed via a straight-C API.
Page 67 of 252
2.2.12.4 Microsoft JET and Data Access Objects (DAO)
Jet and DAO were built to support file-based databases, and uses ODBC to access
relational databases such as Access. Thus it tends to be slow. It was built for small
databases used in local deployments and single-system applications.
2.2.12.5 Remote Data Objects (RDO)
RDO was created to allow faster access, via ODBC, to relational database engines, such
as SQL Server and Oracle. It provides the objects, properties, and methods needed to
access stored procedures and complex result sets.
2.2.12.6 OLE DB
OLE DB was designed to access virtually any kind of data, whether it is stored
relationally in tables, directories or free form. As a result, it is very large and
complicated.
2.2.12.7 ActiveX Data Objects (ADO)
ADO is a wrapper to OLE DB, but is focused on relational data. It is fairly fast, simple to
use and supports multi-tier architectures via the use of disconnected Recordsets.
Microsoft ActiveX Data Objects Multidimensional (ADO MD) was an extension to
include objects specific to multidimensional data from languages such as Microsoft
Visual Basic and Microsoft Visual C++. Microsoft ActiveX Data Objects Extensions for
Data Definition Language and Security (ADOX) includes objects for schema creation
and modification, as well as security.
Page 68 of 252
2.2.12.8 Remote Data Service (RDS)
RDS enables data to be moved from a server to a client application or Web page,
manipulate the data on the client, and return updates to the server in a single round trip.
2.2.12.9 Microsoft Data Access Components (MDAC)
MDAC is not an access mechanism; rather it includes a single version of ODBC, OLE
DB, and ADO, together with drivers for SQL Server, Access and Oracle.
2.2.12.10
ADO.NET
ADO.NET is the database access method of the .NET platform. It is multi-tier (code
could be easily moved from tier to tier), disconnected, XML-based, scalable and fast.
There is no Recordset. Instead, data is manipulated using DataSets. There are four classes
that are used to read and write data from data sources:
1. Connection - Connect to data source
2. Command - Execute stored procedures
3. DataAdapter - Connects DataSet to database
4. DataReader - Forward/only, read/only cursor
2.2.13 Open Source Technology
Open source refers to projects that are open to the public and which draw on other
projects that are freely available to the general public. Open source or open-source
software (OSS) is any computer software distributed under a license which allows users
to change and/or share the software freely. Open-source software is required to have its
source code freely available; end-users have the right to modify and redistribute the
software, as well as the right to package and sell the software. Software with source code
in the public domain meets these criteria, as does any software distributed under the
popular GNU General Public License (GPL). Open-source licenses may have additional
Page 69 of 252
restrictions, such as a requirement to preserve the authors' names and copyright statement
in the code.
2.2.13.1 Apache Ant
Apache Ant is a build tool with special supports for the Java programming language but
can be used for just about everything. Ant is platform-independent; it is written purely in
Java. Ant is particularly good at automating complicated repetitive tasks and thus is well
suited for automating standardised build processes. Ant accepts instructions in the form
of XML documents thus are extensible and easy to maintain.
2.2.13.2 Log4j
Log4j is an open source technology that offers a more systematic way of creating debug
statement during program development. Logging within the context of program
development constitutes inserting statements into the program that provide some kind of
output information that is useful to the developer. Examples of logging are trace
statements, dumping of structures and the familiar System.out.println or printf debug
statements. Log4j offers a hierarchical way to insert logging statements within a Java
program. Multiple output formats and multiple levels of logging information are
available.
By using a dedicated logging package, the overhead of maintaining thousands of
System.out.println statements is alleviated as the logging may be controlled at runtime
from configuration scripts. Log4j maintains the log statements in the shipped code.
2.2.13.3 JFreeChart
JFreeChart is a open-source Java class library for generating charts, including pie charts,
line charts and horizontal/vertical bar charts (regular and stacked, with optional 3Deffect), XY plots and scatter plots, time series, high/low/open/close charts and candle
Page 70 of 252
stick charts, combination charts, wind plots, meter charts and symbol charts, JFreeChart
includes many features: tooltips, zooming, printing and saving (PNG/JPEG/SVG), access
data from any source that supports the defined interfaces (a similar mechanism to Swing's
JTable).
2.2.14 Programming Languages
The following languages were reviewed:
2.2.14.1 C#
This is the language of choice for the .NET platform, as it is closely tied to the .NET
CLR. It is an object-oriented language that closely resembles Java and C++. C# was
designed in response to the strengths and weaknesses of other languages, in particular
Java and C++. There are a few new syntax elements in C# that are not available in C++
or Java, namely properties, indexers and attributes. Properties are syntactic support for
get() and set() methods. Indexers are a way to treat an object as an array. Attributes
support declarative programming.
Key features of C# are:
Component orientation. This makes component building easy, with language
constructs such as properties, events and attributes.
Automatic and manual memory management. The programmer does not have to
dispose objects, eliminating memory leaks and circular references.
Page 71 of 252
2.2.14.2 Java
Java is an object-oriented language that is well suited to designing software for use over
the Internet. It was developed at Sun Microsystems in 1991. It is a cross-platform
language that can be used in many platforms, including Windows, Macintosh, Linux and
most versions of UNIX, including Solaris. The language is quite complex and includes
many disparate open source technologies. It is often run as an applet in Internet browsers,
which can also act as a SOAP Client.
2.2.14.3 C++
In the .NET platform, C++ is used to program .NET, which performs better than XML
Web Services. This is the language used in developing high performance ATL Server
Web Services. The ATL Server uses C++ code that is compiled at development time
before being deployed on the server. The ATL Server architecture runs on top of Internet
Information Server (I I S).
Page 72 of 252
3
3.1
METHODOLOGY
System Development Flow
First of all, the target users of the system had to be identified at the beginning phase of
the project. This is because these users will be able to determine what is required to be
developed. The first phase of this development involves the requirements specification,
which is the usual starting phase of all the software process models. These requirements
are described and defined in detail, serving as the system specification.
Since one of the objectives of this research is to build a framework and subsequently
develop a system on top of this framework, the next step is to plan the architecture of the
framework and follow by the architecture of the system. After spending some time in
researching and retrieving feedback from lecturer’s requirement, finally a complete
development flow is developed. Then, I researched some findings about the suitable
programming language, technology and application server that are essential in developing
the Intelligent Question Bank and Examination System. For the techniques, algorithms,
formulas and others reference systems can be founded through famous web portal as well
as journal. After analyzing the theories and algorithms, I try to filter, decompose and
derive it into computerize algorithm. The work is extremely complicated and takes a long
time to be finished. Finally, throughout continuous research and upgrading in computer
adaptive testing and guided problem solving knowledge, I had invented a few strategies
or techniques that can be used to come out with the Intelligent Question Bank and
Examination System. The outcome is encouraging, satisfying and fulfilling some
objectives in the study.
The further procedure is configuring Tomcat, MySQL, Hibernate as well as classpath and
path in environment variables. Then, the framework and system are being developed in
html, jsp, applet and core java.
During the development cycle, the system will be demonstrated to lecturers from time to
time. The reason is to gain feedback and further enhancement from them. Besides that, it
Page 73 of 252
is a mandatory procedure for incremental development model. Debug, enhance, testing
will continue many times until the system become stable.
Preparing some documents is a necessary task to ensure users familiar and understand on
how to use the system. Validations are done in many fields to ensure user key in data
correctly and the system is able to process the data in correct manner. Process of
verifying the accuracy and correctness of calculations, data structure, frond end
displaying and etc have taken time more than a month. In addition, comparison of
algorithm, assessment method, user friendly ability and others aspects had been verified
against the similar systems.
Finally, upon the system is stabilize and fulfill all mandatory functionalities, the system is
transferred for pilot run mainly for testing purposes. This is an important step to boost up
the system’s stability, robustness, user-friendly and discover new hidden bugs.
3.2
Software Methodology
Methodology covers the methods used to discover past, existing and forthcoming new
technologies in research and developing a new project. It is a collection of tools and
techniques used to aid system developers in the development of a new system, including
the proper procedures used to study existing systems with the aim of eliciting
improvements to current technology.
These tools and technology differ from discipline to discipline. There are qualitative and
quantitative approaches to arriving at a solution. Researchers have their own biases,
depending on their training and preferences. A quantitative approach provides a rich
resource for interpretation. Some prefer qualitative over quantitative approaches. An
integrated approach is the best option. This project tries to mix the best of both methods.
Research methodology comes loosely under the following approaches:
Page 74 of 252
Research Ethics
It is not ethical for research to copy wholesale from existing research findings,
although it is proper to base the research upon various research papers. However,
the research should always include new findings discovered by the researcher.
Secondary Data Sources
Conducting Interviews and surveys are time-consuming. Often it is impossible to
conduct such research in a local area. This is where the experience of other
researches can be drawn upon. The results of multiple surveys done by previous
researchers can be utilized effectively.
Qualitative methods
These methods include Action Research, Content Analysis, Ethnography,
Narrative Inquiry and many others. These are generally arts-based methods that
require descriptive explanations of various happenings and events. However, it
could include scientific qualitative explanations of phenomena or concepts in
technology.
Quantitative Methods
These methods refer to the use of statistical analysis of the results of surveys. The
idea is to draw conclusions or inferences about a certain pattern of behavior. Such
methods are useful for testing the functionality and performance of projects.
Textbook Research
Much information can be gathered from textbooks, which cover important
concepts related to an area of research. This provides a valuable source of
information that is reliable and dependable.
Internet research resources
This is a vast resource, and in recent years, the amount of information that can be
gathered has increased exponentially. A huge quantity of information can be
Page 75 of 252
obtained from various search engines, chief among recent ones are Google and
Yahoo.
Building computer software is an iterative learning process, and the outcome is an
embodiment of knowledge collected, distilled, and organized as the process is conducted
(Pressman, 2001). There is much to be learned, and the sum total of the knowledge aids
in the development process.
3.2.1
Techniques Used to Define Requirements
During the methodology stage, various techniques are used to define the requirements of
the system. These techniques helped in formulating the various requirements of the
system, as detailed in the next chapter. These techniques include the following:
•
Analysis of existing systems
•
Feedback from users of existing systems
•
Review of new technologies
•
Library Research
•
Purchase of new books on relevant topics
•
Internet Browsing
3.2.1.1
Analysis of existing systems
It is vital for the success of a new project to learn from the strengths and weaknesses of
existing systems. Studying the existing systems in fact resulted in the definition of the
bulk of the requirements of the system. From examining the code of the existing system,
we can trace how all the functions are designed. By using and testing these systems, it
was possible to determine what the requirements that must be included in the system.
Through this method, many of the bugs in the system were discovered, and from code
analysis, the source of these bugs was determined. However, this project did not attempt
Page 76 of 252
to rectify those bugs and weaknesses, as it was determined that the new system would
replace the old system.
3.2.1.2
Feedback from users of existing systems
From the comments received from users of the system, it was possible to determine the
performance and pitfalls the examination processing system. This is important because
through studying feedback from many types of users, whether people who are skilful in
the use of computers, or whether people who are novices in computers, it was necessary
to modify requirements to meet the usage pattern of a wide variety of people. It was also
possible to build a ‘wish-list’ of requirements based upon feedback from these users.
These would form the user-centred list of requirements.
The demography of the users consists of the following groups of users based upon their
job functions:
•
Education Institution/School Administrator
•
Department Head
•
Examinations Administrator
•
Clerical Staff
•
Course Lecturer
•
Student
Each of the above categories of users views the system in different ways. The education
institution/school administrators are primarily concerned with the final mark sheets and
analysis reports. The department heads view the overall analysis related only to their
respective departments. The examinations administrator is concerned with generating the
analysis reports and the mark sheets. The clerical staffs view only the mark sheets for
preparation of forecast results. The course lecturers are interested only in the performance
of their own courses/subjects.
Page 77 of 252
There are various sources of user feedback, which can form the basis of various system
requirements. These include verbal requirements requests, where lecturers and
administrators convey the types of reports they need, or would like to include into the
system. Minutes of staff meetings that record needs of examination requirements provide
some insight into some of the user requirements. One-to-one meetings with
institution/school administrators often bring out many requirements that help the
administrators view the overall trend of the examination results.
Questionnaires are generally difficult to administer in an education institution/school
environment. Lecturers generally are too busy with their teaching schedules to
conscientious write out their needs on paper. It is generally easier to have informal
interviews with them to extract their views on the system.
The role of questionnaires lies in getting their feedback on the user interface and the
implementation of the system. This was done with a preliminary questionnaire that
allows freely structured comments. This forms the basis of the final questionnaire that is
appended in Appendix B.
The methodology of the analysis of the questionnaire consists of the following steps:
•
Prepare the questionnaire
•
Explain the questionnaire to the users
•
Issue the questionnaire to the users
•
Collect the responses from the users
•
Populate result into Microsoft Excel
•
Insert charts to visualize the data
•
Summarize and analyze the results
Page 78 of 252
3.2.1.3
Review of new technologies
By reviewing and understanding the new features and capabilities of new technology, it
was found that there are requirements that could be included which otherwise would have
been deemed impossible. This is most often done by reading computer books, journals,
magazines and articles. Sources can be obtained from the Internet and book shops.
3.2.1.4
Library Research
The Library at the University of Malaya stocks many books of inestimable value to the
understanding of some of the technology used to develop this project. These books could
either be browsed or borrowed, resulting in a change in some of the requirements of the
system.
However, books in library on new technologies are often outdated, as library does not
generally purchase new books as fast as they are published.
3.2.1.5
Purchase of new books on relevant topics
The library does not stock many important titles that prove vital to kick-start this project.
These books had been purchased. As with all computer texts, many of these books
become outdated very quickly. Often a book would be bought, and it was found, through
the Internet, that a new edition had been published. This happened especially to the books
on the latest technologies, in particular, the J2EE Framework. So it was necessary to
check the latest edition of a particular title from the Internet, before embarking on the
purchase of a new book.
Some bookshops such as MPH have reading corners to browse through their collection of
books. This is the best source of the latest available books on new technologies. Some
books are too expensive to be purchased. With so many new titles being published, it is
not within budget to buy all available books in the market.
Page 79 of 252
3.2.1.6
Internet Browsing
This is a must, as much information in the form of reviews of Technology can be found
in the web sites of the vendors of the software or books. There are many tutorials, and
trial kits that are available for download from the Internet. White papers from various
vendors are valuable information that often can be downloaded for free. Just plain
Internet browsing and use of search engines such as from Yahoo and Google have proven
to result in vital information which could not be found in books. Through the search
engines, it was possible to find methodology for research on topics useful for the
development of this project.
It is also possible to purchase the latest publications on various topics of research through
the Internet. Amazon.com and InformIT.com are two good sources of books. So are the
web sites of various publications such as McGraw Hill and Wrox.com.
3.2.1.7
Subscription to e-magazines
There are many electronic magazines on various topics such as Search Database.com and
etc that deliver valuable information on a regular basis, up to on a daily basis. These
electronic magazines have links to the valuable Web Sites containing tutorials, tips which
have greatly assisted in determining technology features which helped in determining
requirements and the development process itself.
There are even some vendors which send their hardcopy magazines without charge upon
a free registration. These include IBM DeveloperWorks, Oracle and DB2 Magazine. One
such source was a free copy on an issue covering J2EE, which gave useful information
for literature review and the modification of some of the system requirements
(developerWorks Journal, Nov 2002). However, some of these magazines are available
only to residents of the United States of America.
Page 80 of 252
3.2.1.8
e-Books, that is, electronic books.
Many publishers have made some of their books available as e-books that could be
downloaded from the Internet. Some publishers such as Wrox Press even provide free
membership to some of their resource materials on a restricted basis. Many have made
some free chapter downloads, some of which were found to be extremely useful in the
development of this project. They even have paid membership that can access a larger
variety of e-books materials. e-books are also available on CD-ROMs which can be
purchased in many book shops and computer shops. Some of these CD-ROMS were
compiled through downloads from the Internet.
3.2.2
Software Process Model
There are various software process models that serve as an abstract representation of the
software process. The purpose of a software development process is to produce high
quality and timely results without imposing a large overhead on the project. The process
is depicted as a model that determines the order of stages involved in development and
evolution and establishes transition criteria for progressing between stages by specifying
the entrance and exit criteria. A software process is thus a method of developing or
producing software. These include:
The Waterfall Model
This involves projects that have well-defined phases of specification, software
design, development, implementation, validation and evolution. Each of these
phases follows one after another.
The Incremental Model
This model uses the waterfall model in an iteration fashion. It focuses on the
delivery of an operation product with each increment. It works well to address
technical risk management and ramp staffing with the complexity of the work.
Page 81 of 252
Evolutionary development
This approach is based on the idea of developing an initial implementation,
exposing a prototype to the user, obtain feedback, and revising the system until it
meets the requirements of the end-user. This is useful for projects where the enduser requirements are not clear at the beginning of the software process.
Formal systems development
A formal mathematical system specification is first produced, and then
transformed using mathematical methods. Verification of the various components
involves mathematical techniques.
Reuse-based development
The trend in recent software development is component technology, which fosters
the re-use of existing components to build newer and larger components.
Spiral Process Model
The Spiral Model uses a combination of the traditional life cycle models to best
serve a particular software system. The cycle is repeated for each stage with
cumulative cost to complete steps to date represented by radial distance, and
progress made in completing each cycle represented by angular distance. Each
spiral cycle begins by determining the objectives, alternatives, and constraints,
and then evaluates the alternatives to identify sources of risks in the project. Next,
it develops a strategy for resolving these risks using a combination of simulations,
models, benchmarks, prototypes, and user questionnaires. Planning for the next
step depends on the performance of the evaluation and user interface risks and
thereafter a combination of the previous models is applied. The cycle is
completed by a review of the progress made so far and commitments for the next
phase of work. Transition between stages is determined by a hypothesis set for it:
if hypothesis still holds, work progresses to the next stage, otherwise the cycle is
terminated.
Page 82 of 252
Object-oriented Software Process (OOSP) Model
The OOSP model framework provides an algorithm for identifying development
phases and baseline (work) products based on relationships among activities and
products. The model manages development progress using milestones established
at the end of each phase to check the baseline product, and establishes goals for
the next phase. Progress controlled by a number of clearly identifiable milestones
established for completed phases during the course of the project. The baseline
product is a work product checked at milestone and referred to by activities of
subsequent phases. The date on which a work product is completed is termed a
milestone. When a work product has been reviewed and agreed upon, it becomes
a baseline product.
3.2.3
Choice of Incremental Development Model as Methodology
After reviewing some of the popular methodologies, which are currently being
implemented widely, I decide to choose incremental development model as the
methodology for IQBAES. The incremental model is iteratively applying the waterfall
model. In this approach, development is organized into a series of short, fixed-length
mini-projects called iterations; the outcome of each is a tested, integrated, and executable
system. Each iteration includes its own requirements analysis, design, implementation,
and testing activities.
With the incremental development, it will partition a system into subsystems by
functionality. An early releases start with a small, functional subsystem, later releases
will add addition functionality. Each incremental is a partial product, not a prototype. It
will contain more functionality than the previous increment, but build on the previous
increment. Increments progresses with parallel, with the product delivery of one
increment feeding into the design phase of next. This model provides the customer with
early feedback, allowing alterations of project to occur with limited impact. Each
increment is functionally more complete version of the product.
Page 83 of 252
The incremental development model for object-oriented project comprises the following
phase:
Requirements workflow
The first phase of this development involves the Requirements specification, which is
the usual starting phase of all the software process models. We need to determine
what is required to be developed at the beginning phase of the project. Referring to
the users of the system sets out the system’s services, constraints and goals. These
requirements are described and defined in detail, serving as the system specification.
The full details of the requirements specification are described in Chapter 4. These
involve the functional requirements and the non-functional requirements. The
methodology used in gathering these requirements involves several channels of
requirements discovery. These sets of requirements are based upon extensive use and
development of examinations systems over the past decade. Previous development
was conducted in an ad hoc manner, where requirements were met as the need arose.
Some of the requirements were gathered through user feedback and user requests
conducted in an informal manner. Other requirements were the result of past
constraints on the database system capabilities. There are requirements that resulted
in evolving management goals and expectations, some from the education institution
management, while others are requirements imposed by the distant learning
management system. This project collects all these requirements together in a more
formal manner, in effect reusing these past requirements, and adding new
requirements as the system evolved.
The main goal of these requirements is to develop a system that enables lecturer to
author and store a bank of web-based questions. This system should be able to
assemble questions and generate exam based on lecturer’s specifications. The
examination module should allow user to conduct examination over the Internet and
Intranet. Questions from various topics with different level of complexity can be
Page 84 of 252
included and assigned to user as per their level of understanding. Upon examination
completed, the system should able to grade student based on the questions with
varying complexity they have answered.
In this phase, the information about the user’s requirements shall be gathered and
documented. This phase can last from a few days to few weeks and is usually carried
out on-site or through customer communication. Discussions shall be held to
understand the user’s requirements. The software requirements shall be documented.
The documentation shall use the terms familiar to the customer and shall encompass
the entire functionality of the product as foreseen by the customer. A review of the
user requirement document is to be done. User’s requirements will then be converted
to implementation specific statements. The user-interface shall also be detailed. The
user’s software requirements shall be thoroughly understood to convert to system
specifications. The software requirements specifications shall be documented using
Use Case diagrams. The documentation shall use the implementation specific terms
for the ease of understanding for the designers and developers. One user requirement
may be fulfilled by one or more software requirements specifications.
System Analysis and Design Workflow
The next stage involves identifying and prioritized user requirements. User
requirements are prioritized and the highest priority requirements are included in
the early increments. In this phase, the use cases are identified, the deployment
and component diagrams are arrived at and classes and the collaboration among
the classes are defined. The system design document shall be prepared. The class
diagrams, sequence diagrams, state transition diagrams and collaboration diagram
are included.
As we are developing web-base application framework and subsequently build the
IQBAES on top of this framework, thus in this phase we need to identify the
components that can be re-used and need to be developed and combine them into
Page 85 of 252
one single integrated framework. The framework components that need to be
developed are identified as follows:
•
Design of framework architecture
•
Design and develop of base classes, manager classes, utility classes, core
messages, validator, taglib and etc
•
Setting up application server
Once the base (framework) has been developed, we need to identify the
components required for setting up the IQBAES system. The components that
need to be developed are identified as follows:
•
Design and implementation of the new database management system
•
Design of the user client interfaces
•
Design of user manager module
•
Design of question bank module
•
Design of examination module
Implementation Workflow
In this phase, the actual coding shall be done according to the programming
standard. The code shall be unit tested. The programming standards to be used
shall be identified. Coding shall be done according to the programming standards
and class specifications.
This is one of the most critical phases as failure in coding will result in collapse of
the whole project. New training skills are required, which have to be learned or
developed. While system analysis and design workflow identifies the components
that need to be developed, this phase develops and implements all the design
requirements identified in the earlier phase.
Page 86 of 252
The development involves the implementation of IQBAES. This phase involves
the installation of java development kit, the MySQL Server, and the programming
environment, which is the Eclipse IDE etc. In order to implement this phase, new
skills have to be developed. Specifically, this means learning the java core
language, EJB, the J2EE Framework, hibernate and how it implements SQL
functionality.
Testing Workflow
In the system-testing phase, the product shall be tested module-wise and the
interdependencies among the modules shall be validated. The functionality of the
product shall be tested as a whole. The product needed to be tested for
conformance with the system requirements as well as assessed as to its suitability
for the task of processing examination results, that is, the overall goal of the
project. At this phase, user involvement is vital to the design, redesign and
validation of the user interfaces. During the user acceptance-testing phase, the
IQBAES system shall be validated against the user requirements, acceptance
criteria and acceptance data.
Page 87 of 252
Figure below illustrates the incremental development model:
Figure 16 – Incremental Development Model
The iteration and incrementation are used in conjunction with one another. There is no
single requirement phase or design phase. Instead, there are multiple instances of each
phase. The number of incremental will vary and all the workflows are performed over the
entire life cycle.
3.2.4
Advantages of Incremental Development Model
The advantages of incremental development model are as below:
Allow developer to take advantage of what was being learned during the
development of earlier, incremental and deliverable versions of the system.
Page 88 of 252
By keeping end user in the loop will prevent project to be killed or overhauled
before delivery
Provide customer/end user with early feedback, allowing alterations of project to
occur with limited impact
The complexity of project is never overwhelming.
Early feedback is generated, because implementation occurs rapidly for a small
subset of the system.
Page 89 of 252
4
4.1
SYSTEM ANALYSIS
Requirement Analysis
Requirement Analysis is the detailed documentation of the system and constraints. These
requirements should be precise and may serve as a contract between the system end user
and the software developer. It is the purpose of this chapter to develop a model of the
system’s desired behaviour. This is achieved by firstly looking at problem domain and
then forming a set of requirements using use case analysis. The use cases uncovered are
then elaborated upon to describe a ‘flow of events’, which takes place between the user
and the system. The use case analysis found herein is a product of much discussion
regarding system requirements with Mr. Ang Tan Fong (Project Supervisor) as acting
client. These are loosely divided into functional requirements and non-functional
requirements.
4.2
Functional Requirement
The propose project, IQBAES, will be developed following the web-base educational
system designed earlier. Three core functionalities will be incorporated into this system
in order to make it as an intelligent question bank and examination system. These three
functionalities are question difficulty assessment algorithm, automatic question generator
and intelligent questioning system.
Although there has been much development on web-based educational system, the
question difficulty assessment has not been systematically studied. Assessing the
difficulty of question in these systems is still a subjective process and is usually done by
instructors/teachers based on their experience. Thus, a systematic method is needed to
define question difficulty. IQBAES will define an assessment method for the level of
difficulty of exam questions and it will help to better allocate the credits/scores/marks to
exams. This assessment method is built based on intuitive that a question is more difficult
when more concepts are involved and more equations are used in solving a problem since
Page 90 of 252
it takes more time organize thoughts when more items are involved and more steps are
needed in solving a problem.
Over the past several years, considerable effort has been devoted to research in the area
of technology-enhanced education. Despite the effort, most of these systems are still not
judged to be effective. One of the major hurdles in building an effective system is it is
extremely difficult to build a robust student model. It is more possible and convenient to
improve this area by introducing a question model. This question model uses a
hierarchical knowledge map to direct the learning process based on student’s degree of
understanding of concepts. This question model is a formal representation of the question
database. After each question is answered, the question management system will update
the student’s progress through a knowledge map by taking into account the question
characteristics.
The following use cases and actors were found to exist in the problem situation. Each use
case is described in detail in their respective module section. These use case diagrams
model the desired behavior of the system.
Functional requirement of the IQBAES should be categorized into 5 main modules:
User Manager Module
Administrator Module
Lecturer Module
Student Module
Examination Module
Page 91 of 252
4.2.1
User Manager Module
User manager module mainly means for manage multiple user with multiple role for
accessing this system. With this module, one user can be assigned with more than one
role concurrently. Hence, a user with administrator role can also be assigned with a
lecturer role to the system. On top of that, this module also means to control user with
multiple role for accessing the system. With this functionality, user manager modules will
constraint user with specific role to access only certain module that they are given the
access right. In other words, a lecturer can only access the module that they are given the
access right and they will not have the permission to access the administrator module or
student module. The module also provides a means to enter user names and passwords
and the password will be encrypted in the database. Only the top user with the
administrative rights should be allowed to add, modify and delete users.
4.2.2
Administrator Module
Administrator module is the module that enable administrator to configure and maintain
various variable in the system. This module will allow administrator to configure general
information such as ‘Faculty Maintenance’, ‘Department Maintenance’, ‘Intake
Maintenance’, ‘Lecturer Maintenance’, ‘Program Maintenance’, ‘Course Maintenance’
and etc.
System user with administrator role is the user with super user role to the system. This
category of user will have the full administrative access rights to each module in the
system. The administrator is the "gatekeeper" of the IQBAES who creates user profiles
for the system and is responsible for restricting the access of item banks. The
administrator can add as many users as needed and he is the key person who will assign
user to different role of the system. For instance, administrator is responsible to assign a
new user with either lecturer role or student role of IQBAES.
Page 92 of 252
The User Manager Module and Administrator Module requirements analysis can be
transformed into the use case diagram as shown below:
Figure 17 – Administrator Module Use Case Diagram
Page 93 of 252
Table 1 – Administrator Module Actor Description
Actor
Description
Administrator
The system administrator who is super user
of the system and is responsible for
creating new user and assign user to
different group of the system.
Table 2 – Administrator Module Use Case Description
Use Case
Description
Add new user
When there is a new enrolment, new
students’ details are recorded into database.
Update user details
When user details are changed, old user
details is retrieved and updated with new
user details.
Delete user
When staff leaves the institution or student
is graduated, their details will be deleted
from database.
Assign individual user with role
Assign each new user with their respective
role. E.g. lecturer role or student role.
Assign group of user with role
Assign group of users with their respective
role. E.g. lecturer role or student role.
Update user with new role
When there is a change in user role, the
current user role is retrieved and updated
with new role.
Page 94 of 252
4.2.3
Lecturer Module
System user with lecturer role is those users with lecturer role as assigned by system
administrator. As the name given, this group of user will act as lecturer/instructor in the
system. Lecturer can:
•
Compile exams from an existing database of questions, case studies, images,
audio and video;
•
Add original questions, case studies, images, audio or video to the database;
•
Administer exams (create, update or delete an exam which he/she has been given
the access right);
•
View and compare student exam scores using an student exam tracking or student
exam summary reports;
•
Measure student performance in various competencies and/or difficulty levels.
Page 95 of 252
The requirement analysis for lecturer module can be transformed into the following use
case diagram:
Figure 18 – Lecturer Module Use Case Diagram
Table 3 – Lecturer Module Actor Description
Actor
Description
Lecturer
A person who is licensed to teach within
the institution. The lecturer works in faculty
and is mainly responsible for lecturing issue
for students including assign group of
Page 96 of 252
students to their course, create and prepare
examination question and answer, view
student performance history and etc.
Table 4 – Lecturer Module Use Case Description
Use Case
Description
Create Question
When a new question is created, the
question details will be recorded into
database.
Prepare Answer
Whenever a new question is created, a
corresponding answer for that question will
need to be prepared and added into
database.
Update Question
Occasionally a question might need to be
modified. The question is retrieved and
updated with the new details before
recorded into database.
Delete Question
When course content change, a need will
arise where non-related course content
might need to be removed from database.
The questions that tight to the old course
content will need to be removed from the
database.
Create Exam
At
the
end
of
each
semester,
an
examination will be created to evaluate
student understanding on the respective
course.
View student exam scores
When an examination completed, lecturers
Page 97 of 252
will keen to know how well their students
have been performed in the exam. This can
be done by viewing student performance
scores.
Assign student to course
When students enrolled in a course, the
respective
lecturer
register/assign
those
will
students
need
to
to
that
particular course.
Change password
For security purposes, lecturer can selfmaintained their password and constantly
changes their password.
4.2.4
Student Module
System user with student role is those users with student role as assigned by system
administrator. This group of user will act as student in the system. Student can:
•
Use IQBAES as a study tool by reviewing exams from the database of questions;
•
Take exam to test their own capabilities;
•
Keep track of their performance through the exam score log.
•
View own grade book.
The requirements analysis for student module can be reflected in the following use case
diagram:
Page 98 of 252
Figure 19 – Student Module Use Case Diagram
Table 5 – Student Module Actor Description
Actor
Description
Student
A person who is registered to take courses
and is under the supervision of lecturer.
They will sit for exams that assigned to
them by their respective lecturers.
Table 6 – Student Module Use Case Description
Use Case
Description
View own grade book
After an examination, students can view
their own performance by viewing their
own grade book.
Page 99 of 252
Take exam
Student takes exam that is assigned to them
in order to evaluate their understanding on
the particular course.
Change password
For security purposes, student can selfmaintained their password and constantly
changes their password.
4.2.5
Examination Module
Examination module is the module, which will interface with question bank to pick up
the questions and subsequently form a set of examination questions to test student
understanding. This module will perform auto-grading while student answering the
question and auto adjust the level of complexity for each question that assigned to
students. Thus, questions with varying complexity will be included in different exams
from a question bank.
Assessing the difficulty of an exam question is a subjective process and is usually done
by instructor based on experience. An accurate assessment of the difficulty of exam
questions is important and will help to better allocate credits to exams. IQBAES is
defining an approach to accessing questions difficulties. This approach makes use of
functional concept graph. Interested readers are referred to Li and Sambasivam (2003).
This is an extension of the Conceptual Graph Structure. A functional conceptual graph G
also refers to Directed Acyclic Graph (DAG) and it consists of a set of nodes V and a set
of directed edges E. The edges connect the nodes of V.
The functional concept graph is a hierarchical data structure: each node ni of the DAG is
associated with a level li. Node ni may have a set of incoming edges {ei,1, ei,2, . . .ei,k},
connecting lower level nodes. A node without any incoming edge is a source node.
Page 100 of 252
A node is also associated with a value. The value vi of a node ni is computed by a
function vi = ψi(vi,1, vi,2, . . . , vi,k), where vi,j is the value of the input node nj connected by
edge ei,j. The set of input nodes form the set of partners for node ni. Associated with each
node ni is also an interval (li,ui), which is empirically determined by the author of the
examination system. An inverse function φij is defined, when appropriate, for an input
edge ei,j such that vi,j = φij (ei,1, . . . , ei,j-1, ni, ei,j+1, . . . , ei,n). This value can be assigned to
the corresponding partner node nj. The existence of inverse functions may bring an extra
degree of flexibility and allow the system to generate more sophisticated questions.
I find, from the experience of using exercise questions from some textbooks, that students
tend to find a question more difficulty when more concepts are involved and more
equations are used in solving a problem. This is intuitively easy to understand since it
takes more time organize thoughts when more items are involves and more steps are
needed in solving a problem. Hence, we define the degree of difficulty as D = w1N +
w2P + w3M, where N is the number of conditions given in the questions (that is, the
number of terminal nodes), P is the number of downward edges in the paths traversed
during question generation, and M is the number of upward edges traversed during
question generation. Three weight factors w1, w2 and w3 is used to balance between the
path length and the number of conditions. Typically, w1<w2<w3 because upward edges
traversed represent more difficult concept association and downward edges, and the total
number of edges, which corresponds to the number of problem solving steps involved,
carries more information about the effort needed in problem solving. This definition of
degree of difficulty serves only as a rough guidance during automatic question
generation. It is by no means a perfect objective measure. However, it does help in
assessing the student’s understanding of a certain subject. If a student is able to solve a
problem involving most concepts and equations about a subject, he or she should have a
reasonably good understanding of the subject.
Page 101 of 252
Example of how weight is assign to each of the questions in question bank is given as
below:
Example A:
Figure 20 – Conceptual Graph
Weighted CPU time is defined as the sum of the weighted program time of the k
programs. The weighted program time γi of program i is the CPU time of executing the
program times the frequency of execution Fi of the program. The CPU time Ti is defined
as the product of cycle time Tc and (cycle count + stall cycles). We denote cycle count by
Nc and stall cycles by Ns. For programs executed on the same CPU, the cycle time is the
same.
Assume that three programs, P1, P2, and P3, under consideration. The total Weighted
CPU Time Φ is given. For program P1, P2 and P3, the student is given Ns1, Nc1, Ns2, Nc2,
Ns3, and Nc3. In addition, F1 and F2 are also given. The student is asked to find F3. This
Page 102 of 252
question has 9 given values (conditions). To compute F3, one must know γ3 and T3. Since
γ3=F3*T3, F3=γ3/T3. This requires one upward move to the node for γ3. Now, γ3 can be
computed from Φ, γ1 and γ2 as γ3=Φ-γ1- γ2. This requires another upward move to the
node for Φ. A total of two upward moves are needed. To compute each of T1, T2 and T3,
one must traverse three downward edges. To compute each of γ1 and γ2, two downward
edges are traversed. From Φ to γ1 and γ2, two downward edges are also traversed. This
results in a total of (3×3+2×2+2)=15 downward edges. We assign w1=1, w2=5 and
w3=25. The total complexity of the problem is thus 1×9 + 5×15 + 2×25 = 134.
Example B:
A sequence is defined recursively by Tn+1 = Tn + 2n − 3, T1 = 1 for all positive integers n.
Ti is defined as the product of T2 and T3. The student is given T1 and is asked to find T2 x
T3. This question has 1 given value (conditions). To compute T2, one must know T1. To
compute T3, one must know T2. Since T2=T1 + 2n -3, and T1 is given as 1, thus T2 is
having value 1 + 2 -3 = 0. To compute T3, one must traverse one downward edge. We
assign w1=1, w2=5 and w3=25. The total complexity of the problem is thus (1×1) +
(5×1) + (25×0) = 6.
Solution:
T2 = T1 + 2(1) – 3
=1+2–3
=0
T3 = T2 + 2(2) – 3
=0+4–3
=1
T2 x T3 = 0 x 1
=0
Page 103 of 252
Example C:
An analysis of access time to a computer disc system was made during the running of a
particular computer program, which utilized disc file handling facilities. The mode and
Pearson’s measure of skewness (Psk) are 47.5 and -0.388 respectively. The results of the
120 access times were as follows:
Access time in milliseconds
Frequency
32.5
17
37.5
24
42.5
19
47.5
28
52.5
19
57.5
13
Compute the standard deviation for this program.
Pearson’s measure of skewness (Psk) for computer program is defined as means minus
mode dived by standard deviation. The Psk, mode and total access times are given. In
addition, the access time x and frequency f are also given. The student is asked to find
standard deviation. This question has 16 given values (conditions). To compute standard
deviation (SD), one must know mean (ME), mode (MO) and Psk. Since Psk = (ME –
MO) / SD, SD = (ME - MO) / Psk. This required one upward move to the node for Psk.
To compute ME, one must traverse two downward edges. Hence, the total complexity of
the problem is thus (1x16) + (5x 2) + (25x1) = 51.
Solution:
Mean =
=
∑ fx
∑f
32.5(17) + 37.5(24) + 42.5(19) + 47.5(28) + 52.5(19) + 57.5(13)
120
Page 104 of 252
5335
120
= 44.46
=
Mode = 47.5
Psk =
mean − mod e
s tan dard deviation
mean − mod e
Psk
44.46 − 47.5
=
− 0.388
= - 7.835
s tan dard deviation =
Example D:
What attributes do all real world objects have?
Solution:
The question is to test students’ understanding of the definition of object. This question
will become the source node in Directed Acyclic Graph (DAG) because it does not
involve any concept but a basic understanding of the definition of object. Hence, it is a
source node without any incoming edge in the graph. It only involves 1 condition (N=1)
which is the definition of the object. No downward edges because no underlying
knowledge are required and thus P is 0. No upward edges involves because it does not
require more difficult concept association to solve this problem and hence M is 0. We
assign w1=1, w2=5 and w3=25. The total complexity of the problem is thus w1N + w2P
+
w3M=
(1×1)
+
(5×0)
+
(25×0)
=
1.
Page 105 of 252
Example E:
How to create an object?
Solution:
A class is a description of a kind of object. A programmer may define a class using Java,
or may use predefined classes that come in class libraries. A class does not by itself
create any objects. When a programmer wants to create an object the new operator is
used with the name of the class.
When it is executed (as the program is running), the line
str1 = new String("Random");
creates an object by following the description in class String. The class String is defined
in the class library java.lang that comes with the Java system. The computer system finds
a chunk of memory for the object and puts data and methods into it. The data in this
String object will be the characters "Random"
Before a student able to create an object, the student must know what is an object, how it
is related to a class and know the correct way of creating an object. This is the 3
prerequisites understanding/knowledge before a student can answer this question. Hence,
three downward edges are traversed (P=3) and no upward edge is required because it
does not require more difficult concept association and hence M is 0. It only involves 1
terminal node which makes N become 1. We assign w1=1, w2=5 and w3=25. The total
complexity of the problem is thus w1N + w2P + w3M = (1×1) + (5×3) + (25×0) = 16.
Page 106 of 252
Example F:
An example of java program given as below:
class ExampleF
{
public static void main ( String[] args )
{
String str;
str = new String("PUTRAJAYA");
System.out.println(str);
str = new String("CYBERJAYA");
System.out.println(str);
}
}
a) How many objects were created by the progam?
b) How many reference variables does the problem contain?
Solution:
a) How many objects were created by the progam?
Two objects were created. One for PUTRAJAYA and one for CYBERJAYA
b) How many reference variables does the problem contain?
One, which first refers to the PUTRAJAYA and then refers to CYBERJAYA.
The variable can refer to only one object at a time. It first refers to the PUTRAJAYA, and
then to the CYBERJAYA. Here are some details about how the reference variable and
the two objects relate:
Statement
Action
str = new String("PUTRAJAYA");
Create the first object.
Page 107 of 252
Put a reference to this object into str.
System.out.println(str);
Follow the reference in str to the first
object.
Get the data in it and print it.
str = new String("CYBERJAYA");
Create a second object.
Put a reference to the second object into str.
At this point, there is no reference to the
first object. It is now “garbage”
System.out.println(str);
Follow the reference in str to the second
object.
Get the data in it and print it.
Similar to the previous example, before student able to answers the questions, he/she
needs to know what an object is, how it is related to a class, the correct way of creating
an object and subsequently how to assign reference to the second object. These are the 4
basic concepts required in order to answer this question correctly. Thus, four downward
edges are traversed (P=4). These are the 4 basic straight forward concepts which do not
associate with any other more difficult concept and hence no upward edge is required
(M=0). It only involves 1 condition (N=1) which is how to assign reference to the second
object and hence N=1. We assign w1=1, w2=5 and w3=25. The total complexity of the
problem is thus w1N + w2P + w3M = (1×1) + (5×4) + (25×0) = 21.
Three weight factors w1, w2 and w3 is used to balance between the path length and the
number of conditions. Typically, w1<w2<w3 because upward edges traversed represent
more difficult concept association than the downward edges, and the total number of
edges, which corresponds to the number of problem solving steps involved, carries more
information about the effort needed in problem solving.
Page 108 of 252
In the example given above, the value given for w1, w2 and w3 only serve as a reference.
It can be varied and it can be changed while setting up a particular exam. The lectures
who set up the exam will need to aware that w1<w2<w3. The value is configurable based
on the respective exam. The reason being I choose w1=1, w2=5 and w3=25 is because
the degree of difficulty is defined as D = w1N + w2P + w3M, where P is the number of
downward edges in the paths and M is the number of upward edges. With the w2 as 5 and
w3 as 25, we can make the differences of the value for D become more obvious which
significantly indicating the difficulty level. In short, the higher the value for D, the more
difficult the question is as compare to the question with lower value of D.
Page 109 of 252
IQBAES system contains an automatic question generator, which is able to generate
questions based on the augmented concept graph. The questions to be generated typically
are the types of direct evaluation of a result from given conditions. The result can be
compared against a value or a range. This requires the construction of one concept
structure and the evaluation of a single result.
The algorithm is presented as below:
An integer named credit is what we use to represent complexity. The larger is this
number, the more complex the question is. We use a queue OPEN to store those nodes
that are not yet expanded. A node is expanded if its descendants are generated. We
assume that each node Ni stores an estimated maximum difficulty Di that is obtained by
downward expanding the concept graph rooted at this node to the maximum. In addition,
we assume that we have two basic types of nodes in a concept graph: the array nodes and
the regular nodes. An array node may have several descendants of the same class and a
result is computed from the descendent values. A regular node has descendants of
different classes.
4. Define a queue: Queue OPEN = new QueueList();
5. Select a node Nr from the entire concept graph and mark it as “closed”.
6. If Dr < credit, then
Traverse upward;
credit = credit–w3;
Push the parent of Nr to OPEN: OPEN.enqueue(parent of Nr);
else
Randomly decide traverse up or down to node N;
Push the descendants of N to OPEN: OPEN.enqueue(descendants of N);
credit = credit–w2* num_descendants;
5. Dequeue a node N from OPEN and mark it as closed: OPEN.dequeue();
6. If D >= credit, then
Page 110 of 252
Select a random value in the interval specified in the node for each node in
OPEN queue;
Finish question generation;
else
Randomly decide traverse up or down to node Nd;
Push the descendants of Nd to OPEN: OPEN.enqueue(descendants of Nd);
credit = credit-w2*num_descendants;
6. Go to step 3.
Remark: the credit value in fact denotes the desired question difficulty. The difficulty of
the automatically generated question may exceed the desired difficulty by a small
percentage.
Page 111 of 252
The requirements analysis for examination module can be reflected in the following use
case diagram:
Figure 21 – Examination Module Use Case Diagram
Table 7 – Examination Module Use Case Description
Use Case
Description
Adjust question complexity level
Auto adjusts question complexity level
while student answer the question.
Generate exam summary report
Generate an exam summary report upon
examination complete based on students’
level.
Page 112 of 252
4.2.6
Question Module
Question model will be a core component in the overall questioning system. Key aspects
of the question model include an intelligent questioning engine, a question/answer
interface tool, and connections to a knowledge database to monitor student progress. The
question model is a representation of the question database that is developed for each
module in a course for a program of study. Each question is characterized by type (true
false, multiple choice, numerical answers), weight, as determined by the difficulty level,
and a knowledge key that defines the scope of the question and its coverage of the
knowledge map. After each question is answered, the question management system then
updates the student’s progress through a knowledge map, again taking in account the
question characteristics.
The Intelligent Questioning System is comprised of three basic components: the User, the
Knowledge System and the Question System as shown in Figure 22 – Intelligent
Questioning System Concept. The Knowledge System presents the user with his or her
current level in the system. The Questioning System then identifies an appropriate set of
questions within the level. A question is then randomly selected and presented to the user.
When the user completes exam, the Questioning System provides an assessment measure
to the Knowledge System. The Knowledge System then updates the user’s status in the
system. Figure 23 – Questioning System Architecture below gives an overview of the
overall architecture.
Page 113 of 252
Figure 22 – Intelligent Questioning System Concept
Figure 23 – Questioning System Architecture
Page 114 of 252
To implement this system, several databases are required. The question database contains
the complete set of questions. The student system database encapsulates a model of the
student’s progress, maintaining information about which questions the student has
attempted and the assessment measure for each question. The knowledge system database
maintains the structure of knowledge flow.
The assessment system uses the weight of the question to provide the incremental score
contributed by the answer to this question. The “knowledge key” information about each
question is used by the adaptive questioning system, together with student input and the
current state of knowledge, to determine which question to pose next. After each question
is answered, the question management system updates the knowledge map.
Questions are grouped into a Question Set based on level of difficulty. The questioning
process involves: randomly selecting a question from the set, completing the question,
gathering assessment information, and providing a measure of the degree of
understanding back to the knowledge model for use in knowledge assessment. Besides
selecting and asking a question, the Question Set also provides a mechanism to quantify
degree of understanding. This is used to assist in assessing the student’s level of
competence in each topic.
Page 115 of 252
4.2.7
Flows of Events
The below ‘flow of events’ serves to elaborate on the use case descriptions given above
to describe the interaction that will need to take place between ‘the user’ and ‘the system’
in order to realize the use cases.
Add new user
This use case begins when administrator identifies the users they wish to add to
their list. The administrator then adds the users to the system and the system will
add the users to system’s database.
Update user details
The administrator identifies and selects from the list of users in the system they
wish to update their details. Then they proceed to update the user details and
system will capture the new details into database.
Delete user
The administrator identifies the users they wish to remove from their list. The
administrator selects the users they wish to remove from the list of users in the
system. The system then removes the given users and updates the database.
Assign individual user with role
The administrator identifies user with their respective role. They will then assign
the particular user with their respective role. System will record the user’s role
assignment into database.
Assign group of user with role
This use case begins when administrator wants to assign more than one user with
a role. Administrator will select the list of users and consequently their respective
role on which they will be assigned with.
Update user with new role
This use case happens only when there is a change to current user role
assignment. For instance, lecturers who undergo a higher course in the institution.
In such a case, this lecturer will also act as student on the course he/she registers
Page 116 of 252
with. Thus, administrator will update the user with their new role and this new
role should not conflict with their current role.
Create Question
This use case is initiated whenever there is a need arise from lecturer to add more
questions into question bank. Lecturer will need to prepare their questions.
System will capture the questions detail into question bank database.
Prepare Answer
This use case is tightly couple with create question use case. For every single
question that lecturers raise, there should come along with a set of answers. For
question of type multiple choice, more than one correct answer should be allowed.
Update Question
This use case is initiated when there is a need arise from lecturer to update the
questions in the question bank. The update might include adding more choice of
answer, make correction on existing question, upgrade level of difficulty etc.
Delete Question
This use case begins when there is a need to perform housekeeping of the current
questions in question bank. Those questions which are out-dated or out of scope
will be removed from question bank.
Create Exam
This use case begins when lecturers try to create exam to evaluate students’
performance.
View student exam scores
This use case begins only after students have been taken an exam. Their
competencies and performances will be analyzed and capture into system
database. With this, lecturer can view and evaluate their student performance.
Assign student to course
This use case normally begin at the beginning of a semester and after students
Page 117 of 252
have enrolled in a course. Lecturer will obtain their respective student course
enrolment list and proceed to assign those student into their course in the system.
Change password
This use case happens when there is a need arise from system user for instance,
administrator, lecturer or student who wish to change their password.
View own grade book
This use case begins only after students have been taken an exam. With this,
student can view their exam scores.
Take exam
This use case begins only after the respective exam has been created by lecturer in
the system and students are instructed to sit for the exam. Students will need to
login into the system and answer all the questions in the exam. If he/she manages
to answer the question correctly, the level of difficulty for the next question will
be increased. Else if the student fails to answer the question correctly, the level of
difficulty for the next question will be decreased. At the end of the exam, the
weight or difficulty level for all the questions that the student has been answered
either correctly or incorrectly will be summed up and percentage will be
calculated base number of question answered correctly against the total questions
answered. The student’s performance will be graded based on the value.
Page 118 of 252
4.3
Non-functional Requirement
Non-functional requirements do not directly govern the specific functions to be delivered
by the system. They relate to certain performance and reliability issues, as well as the
constraints under which the system must operate.
4.3.1
Product requirements
Platform
The system shall execute on the Microsoft Windows or Red Hat Linux
platform. This should be necessary for consistency of user interface in the
initial implementation of the system.
Reliability of database
The system should use a reliable database management system. All database
updates must be dependable. As such the database should be relational and
use the current form of database access methods.
Robustness
The system shall be built with robust error recovery routines to handle
system failures. The system should be able to guide the user out of a
situation where an error occurs. This is important to avoid human help
support, which is costly.
Recoverability
Should there be errors in accessing various parts of the routines, such as
database access, the system must provide a recovery routine. This routine
should guide the user through the recovery process.
Page 119 of 252
4.3.2
User Interface Requirements
The system must provide a user interface that is intuitive and does not cause confusion or
ambiguity to the user. It is also preferable that the users can expect what the system
should be able to provide, and provide those expectations. The users have used similar
systems before. Therefore, the users should be able to easily find the same features
without difficulty. In addition, the user interface should meet the following requirements:
Consistency of interface between various modules. Each module should be
designed with a similar look-and-feel, with similar type of buttons and
icons.
User Familiarity based upon previous software systems. The user should
know the capabilities of the system as a result of using previous systems,
and expect at least those features which were previously present. The system
can then provide newer features.
4.3.3
Organizational requirements
Reliability of access
The system should be reliably accessed over the institution intranet system or
World Wide Web. It can be assumed that the hardware linking the computers is
dependable.
Conformance
The database to be used shall closely conform to or can be easily integrated to
the current database system used by the institution.
Page 120 of 252
Scalability
The system shall use a database that is scalable. It is expected that MySQL will
be used in the future because it is open source and no budget concern is
required.
4.3.4
External requirements
Interoperability
There are many systems already in use in the institution. The system should
provide features to enable data to be transferable between these systems.
Satisfaction
The system should make the students, lecturers and administrators of the school
satisfied, which leads to increased productivity.
Modularity
The system should be able to operate on its own. It is independent of all other
software systems except the underlying operating system.
Affordability
The system should use technology that is fully open source. There is no need to
use high-end technology, as long as it performs the tasks well.
Page 121 of 252
4.4
System Tools Requirement
4.4.1
Hardware Requirements
The hardware requirement for the IQBAES system can be categorised into two
categories, which are:
Development Machine
The development of IQBAES is done on a machine with Intel® Pentium®
M processor 1400MHz 1.4GHz 512 MB of RAM. This is the propose
requirement for development pertaining to the various tools that will be used
during the development phase.
Web Hosting Machine
This machine will eventually be used to host the entire IQBAES system
with application server and database server installed on the same machine.
However, it is also feasible to install application server and database server
on two difference machines in order to increase the performance load on
each machine or to ease the task of load balancing or clustering in future.
The propose hardware requirement for IQBAES as follow:
Red Hat Linux Server with specification:
250GB of Server Space
Red Hat Linux Operating System Version 9.0
1 Dedicated IP Address
Support one Domain Name
Anti Virus for protection
Unlimited Bandwidth Usage
Page 122 of 252
FTP Access 24/7
MySQL Databases
Support TOMCAT / JBoss
Daily Backup
OR
Windows 2000 Advance Server with specification:
250GB of Server Space
Windows 2000 Advance Server
1 Dedicated IP Address
Support one Domain Name
Anti Virus Protection
Unlimited Bandwidth Usage
FTP Access 24/7
MySQL Databases
Support TOMCAT / JBoss
Daily Backup
Page 123 of 252
4.4.2
Software Requirements
The software requirement for the IQBAES system can also be categorised into
two categories, which are:
Development Machine
The development of IQBAES is done on this machine. Hence, the following
tools will be required during the development of IQBAES:
Java 2 SDK, SE v1.4.2_06
Java 2 Runtime Environment, SE v1.4.2_06
Internet Explorer
MySQL Servers and Clients 4.0.18
MySQL-Front 2.5
Tomcat Catalina 4.1
Log4j
Ant 1.5
JDBC Driver
JFreeChart
Eclipse 3.0
Page 124 of 252
Web Hosting Machine
This machine will be used to host the entire IQBAES system. Hence, the
following tools will be required for hosting IQBAES:
Java 2 SDK, SE v1.4.2_06
Java 2 Runtime Environment, SE v1.4.2_06
Internet Explorer
MySQL Servers and Clients 4.0.18
Tomcat Catalina 4.1
Log4j
JDBC Driver
JFreeChart
Page 125 of 252
4.4.3
Development Tools
The following tools will be used to develop the IQBAES:
Java 2 SDK, SE v1.4.2_06
Java 2 Runtime Environment, SE v1.4.2_06
Internet Explorer
MySQL Servers and Clients 4.0.18
MySQL-Front 2.5
Tomcat Catalina 4.1
Log4j
Ant 1.5
JDBC Driver
JFreeChart
Eclipse 3.0
Page 126 of 252
5
SYSTEM DESIGN AND DETAILED DESIGN
System design is a phase of transforming the system requirements to solutions or system
characteristic. Design of system takes place at two levels: system design and detailed
design. System design is concerned with the overall architecture of the system. Detailed
design is concerned with designing individual components to fit this architecture. In an
object-oriented system, the detailed design is mainly concerned with the design of objects
(class design). These two levels of design are described briefly in the following section.
5.1
System Architecture
IQBAES will be using a three-tier system architecture design. The conceptual
architecture of the three-tier application applies when we split an application across three
tiers. The system is split into three logical components: user interface, computational
logic and data storage. In reality, the three-tier web applications generally consist of a
web browser for the user interface, a web server connected to a middle tier application
and a persistent store that is frequently a relational database. The main purpose of having
three-tier architecture is to assign main functionality to each tier to ensure no function
overlapped. Different people could handle each tier using different languages. Therefore,
whenever there is error or system fault occurs, the problems can be detected and fixed
easily without interfering with other tier.
5.2
System Design
System design reflects the overall architecture of the system. Design of IQBAES system
takes place at two levels: IQBAES Framework Design and IQBAES System Design.
These levels are described as below. As IQBAES framework is designed based on Struts
framework, a brief explanation of Struts Framework is also described as below.
Page 127 of 252
5.2.1
Struts Framework
The Struts Framework is a standard for developing web applications. It is designed based
on the Model-View-Controller (MVC) design paradigm, distinctly separating all three
levels:
Model: application state
View: presentation of data (JSP, HTML)
Controller: routing of the application flow
It implements the JSP Model 2 Architecture and stores application routing information
and request mapping in a single core file, struts-config.xml. The Struts Framework itself
only fills in the View and Controller layers. The Model layer is left to the developer.
Figure 24 – Struts Framework Architecture
The Struts servlet controller will intercept all incoming requests. The Struts configuration
file struts-config.xml is used by the controller to determine the routing of the flow. This
flow consists of an alternation between two transitions:
From View to Action - A user clicks on a link or submits a form on an HTML or JSP
page. The controller receives the request, looks up the mapping for this request, and
forwards it to an action. The action in turn calls a Model layer (Business layer) service or
function.
Page 128 of 252
From Action to View - After the call to an underlying function or service returns to the
action class, the action forwards to a resource in the View layer and a page is displayed in
a web browser.
The diagram below describes the flow in more detail:
Figure 25 – Struts Sequence Diagram
1. User clicks on a link in an HTML page.
2. Servlet controller receives the request, looks up mapping information in strutsconfig.xml, and route to an action
3. Action makes a call to a Model layer service
4. Service makes a call to Data layer (database) and the requested data is returned.
5. Service returns to an action
6. Action forwards to a View resource (JSP page)
Page 129 of 252
7. Servlet looks up the mapping for the requested resource and forwards to an
appropriate JSP page.
8. JSP file is invoked and sent to the browser as HTML
9. User is presented with a new HTML page in a web browser.
5.2.2
IQBAES Framework Design
IQBAES framework is a web application framework builds from open source technology
which based on the Model-View-Controller Model 2 standard and design patterns. This
framework is a set of cooperating classes, servlets and JSP tags that make up a reusable
MVC Model 2 design. This implies that IQBAES framework is framework rather than a
library. It also contains an extensive tag library and utility classes that work
independently of the framework. It is developed mainly for building J2EE application.
This framework consists of a group of generic components which can either be self
sustain or plug-in together to become functional. With the self-sustainability of the
generic components, the IQBAES framework can evolve with on going advancing of
technology where the obsolete component can be replaced with new components. This
framework has been developed with the similar idea of Struts Framework.
The Model-View-Controller (MVC) design pattern is used to design IQBAES framework
because it helps to resolve some of the issues with the single module approach by
dividing the problem into 3 categories:
Model
The model contains the core of the application's functionality. The model
encapsulates the state of the application. It knows nothing about the view or
controller.
View
The view provides the presentation of the model. In addition, it knows nothing about
the controller. The view should be notified when changes to the model occur.
Page 130 of 252
Controller
The controller reacts to the user input. It creates and sets the model
Figure below illustrates the Model-View-Controller (MVC) design pattern:
Figure 26 – MVC design pattern
The MVC model 2 is used to design IQBAES framework instead of model 1. This is
because Model 1 and Model 2 differ essentially in the location at which the bulk of the
request processing is performed. In the Model 1 architecture, the java server page alone
is responsible for processing the incoming request and replying back to the client. There
is still separation of presentation from content, because all data access is performed using
beans. Model 1 architecture is more suitable for simple applications but it may not be
desirable for complex implementations. Improper use of this architecture usually leads to
a significant amount of scriptlets or Java code embedded within the JSP page.
Page 131 of 252
Figure below describe how the Model 1 works:
Figure 27 – Model 1 design pattern (from Sun Micro-system)
Model 2 is used in IQBAES framework development because model 2 is actually a
hybrid approach for serving dynamic content. It combines the use of both servlet and JSP.
Page 132 of 252
Figure below shows how the model 2 works:
Figure 28 – Model 2 design pattern(from Sun Micro-system)
The servlet will perform process-intensive tasks. It acts as the controller. It is in charge of
the request processing. It creates any beans or objects used by the JSP. It will decide
which JSP page to forward the request to depending on the user's actions. The JSP in turn
generates the presentation layer. It has no processing logic. It is mainly responsible for
retrieving any objects or beans that may have been previously created by the servlet. It
will extract the dynamic content from the servlet for insertion within static templates.
With this model 2 design pattern, it typically results in the cleanest separation of
presentation from content. It leads to clear delineation of the roles and responsibilities of
the developers and page designers.
Page 133 of 252
Figure 29 – IQBAES Framework Overview
The client browser will trigger a HTTP request. The Web container will respond with an
HTTP response. The controller will receive the request from the browser, and makes the
decision where to send the request. With IQBAES framework, the controller is
implemented as a servlet. The IQBAES.config.xml file will configure the controller. The
business logic updates the state of the model and helps control the flow of the application.
In IQBAES framework, this is done with an Action class as a thin wrapper to the actual
business logic. The actual business logic is encapsulated inside the IQBAESManager
class. The model represents the state of the application. The business objects update the
application state. The Action classes represent the Model state at a session or request
level. The hibernate classes represent the model state at the persistent level. The JSP file
reads information from the Action classes using JSP tags. The view is simply a JSP file
with no flow logic, no business logic, and no model information. The diagram below
describes the flow in sequence diagram format.
Page 134 of 252
Figure 30 – IQBAES Framework Flow
IQBAES framework was developed with the reuse and plug-in concept in mind. A few
reusable generic components were identified and these components have been built into
the kernel layer of this framework so that any vertical component build on top of
IQBAES’s base layer can utilize the component by calling its API. This framework will
serve as the baseline for any application development to avoid reinventing the wheel of
development for every new development work. Figure below illustrates the IQBAES
framework in graphical view.
Page 135 of 252
Figure 31 – IQBAES Framework
As depict from Figure 31 above, base classes and kernel classes serve as the base for this
framework. These two components will make up the core functionality of the framework.
For instance, database connection holder, session handler, property holder, message
template and message holder are java classes that reside in this layer, which are reusable
across the platform. Any application that builds on top of this framework can make use of
these classes for database connection, session management, message management and
etc.
User Management Class library is a module or layer that builds on top of this core
classes. By utilizing these user management classes as a reusable component, a system,
Page 136 of 252
which requires user management and / or user role management functionality, can call the
respective API to invoke and implement the user management module.
Other vertical components like Utility, CoreMsg, Validator and Menu can be customized
in accordance to the needs of the developed system. For instance, the validator classes
can be customized to provide the desire messages for fields’ validation on screen. Menu
can be customized to include only the necessary menu items. Utility classes can be
utilized by developer to customize the date format, the content of a message in plain text,
Html format and etc. All these reusable components can be reused for creating other web
application.
5.2.3
IQBAES System Design
Intelligent Question Bank and Examination System is developed on top of IQBAES
framework that we discussed in section 5.2.2. Hence, this system will make use the
client-server architecture in which the work of the system is divided between clients and
server. The IQBAES design is divided into multiple tiers: the client tier, the middle tier
and the backend tier. Partitioning the design into tiers allows IQBAES to choose the
appropriate technology for a given situation. Each of the three tiers plays a specific role
in the design.
With this, IQBAES make use of html and JSP pages in the clients tier as the view
component. The client tier is responsible for representing the data to the user, interacting
with the user, and communicating with the other tiers of the system. In this case, the
client tier is the only part of the system visible to the user. The user will be able to access
IQBAES’s client’s tier through their web browser.
The middle tier is responsible for processing. IQBAES’s controller servlet in this case
serves as the controller to control the traffic flow within the system. For every
HttpRequest and HttpResponse that controller servlet received, it will refer to the
config.xml to determine the action classes and manager classes that the request or
Page 137 of 252
response should be routed to. Action classes will then interact with the backend objects or
classes (hibernate generated classes) to access the MySQL database. Hibernate is an
object-relational mapping solution. It serves as a bridge between object and relational
database (acts as the interface between the middle tier and backend tier). With hibernate,
it allows this system to express queries using native SQL or java-based criteria queries.
Every relational table in database is mapped to a specific hibernate class or hibernate
object. Hence, every hibernate object will behave just like the table in the database.
IQBAES can query the object instead of query the database’s table with native SQL.
With this technology, it can reduce the connectivity to database and consequently reduce
the connection pooling related issue and tremendously reduce database deadlock problem
when too many connection was established to the database especially during student
sitting for the examination. This technology will give more advantages to the system that
have heavy loaded concurrent access.
The backend tier is the system information infrastructure. This tier includes relational
database management system. In such a case, IQBAES make use of MySQL as the
DBMS. This backend tier provides a layer of software that maps existing data and
application resources into the design of IQBAES. Graphical view of IQBAES system
architecture is shown in the Figure 32 – IQBAES System Architecture and the
implementation of IQBAES system is shown in Figure 33 – Implementation of IQBAES
system.
Page 138 of 252
Figure 32 – IQBAES System Architecture
Page 139 of 252
Figure 33 – Implementation of IQBAES system
5.3
IQBAES Security Design
On the whole lecturers are still wary of setting examinations online. When the students
are off-site the main concern is with student authentication. It is important to know that
who is exactly ‘pushing the buttons’ in the examination. Even though there is no way to
ensure the identity of a student at the remote assessment site the greater the concern to
many lecturer is how to ensure that students did not receive outside assistance to improve
their test score. Where examinations are conducted in a laboratory environment it needs
to be recognized that computer labs are rarely set up for testing. Wintington (1999)
argues that all computers to be used under exam conditions need to be audited. Computer
connected to the Internet are difficult to make secure and safety issues are particularly
important in longer exams.
Page 140 of 252
With the only use of password as a security protection is inadequate in examination
environment. Standard password protection schemes are used for access prevention but
not for authentication (no password - no access, but if you have one, you can be
anybody). It will rely on the password owner keeping it secret.
Hence, I would like to propose the following examination procedure as a security
measurement in IQBAES.
Before Examination Session Start:
Invigilator logs in to server, obtains session code
Invigilator enters session code to his/her workstation
Candidate list with authentication codes is produced
Invigilator logs into each workstation.
Log-in script starts browser with overlay windows (Figure 34 – Overlay Browser)
Short-cuts and external access is disabled. Browser displays initial dialog screen.
After Examination Session Start:
Candidates enter the room in any order when a workstation is free
Each candidate presents an ID and gets the authentication code.
He or she proceeds to a free workstation
Enters name and authentication code
Initial dialog screen communicates login data to server
Server checks and then invalidates authentication code
Server display the examination questions
Candidate enters his or her answers
Session terminates, candidate leaves browser displays initial dialog.
Page 141 of 252
Figure 34 – Overlay Browser
5.4
Detailed Design
Having investigated the behavior of our domain, it is now necessary to discover a mean
to give detail to IQBAES domain classes. After consultation with Mr. Ang Tan Fong
regarding the details of the class diagrams that should be covered in this report, it was
decided that this report would endeavour to give more emphasis on the detail design on
the aspect of user role management, question bank and examination system. The
following diagrams show the detail class design of user role management, question bank
and examination system in IQBAES.
Page 142 of 252
5.4.1
User Role Management Class Diagram
Having gathered some objects involved in different scenarios, classes are abstracted from
the objects that we have discovered. Classes are represented with a square box containing
three segments. All classes that are related in purpose are grouped and arranged into
packages. Figure 35 – Hierarchy diagram of BaseRole depicts the hierarchy of multiple
roles in IQBAES. Student, Lecturer, Admin and User roles in the system are extending
the base role from IQBAES framework.
Figure 35 – Hierarchy diagram of BaseRole
In Figure 36 – User Role Management Classes and Figure 37 – User Role Listing
Classes, it describes the user role management classes and user role listing classes
respectively in IQBAES. Figure 36 – User Role Management Classes illustrate the group
of classes that interact together to serve the purpose of role management in the system.
Whereas Figure 37 – User Role Listing Classes describe the classes that work together
for listing the entire roles in the system. These classes will list the entire roles available in
the system and subsequently enable System Administrator to maintain system users with
their respective roles.
Page 143 of 252
Figure 36 – User Role Management Classes
Page 144 of 252
Figure 37 – User Role Listing Classes
Page 145 of 252
5.4.2
Examination System Collaboration Diagram
Figure 38 – Collaboration diagram of Examination System
The collaboration diagram shown in Figure 38 – Collaboration diagram of Examination
System shows the situation where student sits for the examination and how IQBAES
picks up questions with varying complexity and includes those questions in the
examination and assigns to user as per level.
Page 146 of 252
5.4.3
Examination System Sequence Diagram
Figure 39 – Sequence diagram of Examination System
Sequence diagram are used to show scenarios from the use cases discovered during use
case modelling. A scenario represents an instance of a use case that user might navigate
along when performing a use case. It represents the duration in time that an activity takes
place and also the control relationship between the caller and the activation.
Page 147 of 252
5.4.4
Examination System Class Diagram
Figure 40 – Class diagram of Examination System
Figure above illustrates the association and dependency between classes that take place in
the examination system.
Page 148 of 252
5.4.5
Package Dependency Diagram
Figure 41 – Package Dependency of IQBAES
Figure above illustrates the packages dependency in IQBAES system.
Page 149 of 252
5.5
Entity Relationship Diagram
This section is an entity-relationship diagram, or “ERD,” for IQBAES. An ERD is a
model that identifies the concepts or entities that exist in a system and the relationships
between those entities. An ERD is often used as a way to visualize a relational database:
each entity represents a database table, and the relationship lines represent the keys in one
table that point to specific records in related tables. ERDs may also be more abstract, not
necessarily capturing every table needed within a database, but serving to diagram the
major concepts and relationships. This ERD is of the latter type, intended to present an
abstract, theoretical view of the major entities and relationships needed for IQBAES. The
Entity Relationship Diagram shows the data stores that have been identified during data
analysis of the system. These data stores constitute the main repository of data for the
system. They clearly reflect the functionality described above. Section 5.6 gives the
details for each table, regarding fields and keys. There are several different modelling
systems for entity relationship diagramming. This ERD is presented in the common ERD
notation.
5.5.1
Entities
Entities are concepts within the data model. Each entity is represented by a box within
the ERD. Entities are abstract concepts, each representing one or more instances of the
concept in question. An entity might be considered a container that holds all of the
instances of a particular thing in a system. Entities are equivalent to database tables in a
relational database, with each row of the table representing an instance of that entity.
5.5.2
Relationships
Relationships are represented by lines between entities. Relationship lines indicate that
each instance of an entity may have a relationship with instances of the connected entity,
and vice versa. If necessary, a relationship line may be labelled to define the relationship.
Page 150 of 252
5.5.3
Optionally and Cardinality
Symbols at the ends of the relationship lines indicate the optionally and the cardinality of
each relationship.
“Optionally” expresses whether the relationship is optional or
mandatory. “Cardinality” expresses the maximum number of relationships.
As a relationship line is followed from an entity to another, near the related entity two
symbols will appear. The first of those is the optionally indicator. A circle ( )
indicates that the relationship is optional—the minimum number of relationships between
each instance of the first entity and instances of the related entity is zero. One can think
of the circle as a zero, or a letter O for “optional.” A stroke ( | ) indicates that the
relationship is mandatory—the minimum number of relationships between each instance
of the first entity and instances of the related entity is one.
The second symbol indicates cardinality. A stroke ( | ) indicates that the maximum
number of relationships is one.
A “crows-foot”
(
) indicates that many such
relationships between instances of the related entities might exist.
The following diagram indicates all of the possible combinations:
A
B
Each instance of A is related to a minimum of
zero and a maximum of one instance of B
A
B
Each instance of B is related to a minimum of
one and a maximum of one instance of A
A
B
Each instance of A is related to a minimum of
one and a maximum of many instances of B
A
B
Each instance of B is related to a minimum of
zero and a maximum of many instances of A
Figure 42 - Crows-foot Notation
Page 151 of 252
5.5.4
Bridge Entities
When an instance of an entity may be related to multiple instances of another entity and
vice versa, that is called a “many-to-many relationship”. While this relationship model is
perfectly valid, it cannot be translated directly into a relational database design. In a
relational database, relationships are expressed by keys in a table column that point to the
correct instance in the related table. A many-to-many relationship does not allow this
relationship expression, because each record in each table might have to point to multiple
records in the other table.
In order to build a relational database that captures this relationship, it is necessary to
build a bridge between the two entities that uniquely expresses each relationship instance.
This can be modelled in an ERD with a “bridge entity,” an entity box containing a
diamond, which may replace the many-to-many relationship. (The diamond is used in
other ER modelling systems to indicate relationships, or may be viewed as the joining—
the bridge—of the many-to-many “crows-feet”).
In addition to explicitly depicting a relational database structure that can capture a manyto-many relationship, the bridge entity has an additional function in abstract entityrelationship modelling: A bridge entity may capture attributes that are specific to the
relationship between instances of the bridged entities. Any attributes that are dependent
on the relationship would be associated with the relationship’s bridge entity.
Page 152 of 252
5.5.5
Recursive Relationships
Instances of entities may have relationships with other instances of the same entity.
These relationships may be drawn with relationship lines that begin and end connected to
the same entity.
Common occurrences of these recursive relationships include
parent/child relationships.
5.5.6
Multiple Relationships Between Entities
An entity may have multiple relationships with another entity. These are depicted in the
ERD with multiple relationship lines connecting the two entities
5.5.7
Entity Subtypes
There are times when it is convenient to depict relationships that apply to an entire class
of things, as well as depict relationships that apply only to certain types of the larger
class. Entity subtypes accommodate these relationship depictions. In the ERD, entity
subtypes may be depicted by entity boxes shown within larger entity boxes. The large
entity represents the class, and the smaller boxes depict the subtypes.
5.5.8
Exclusive-Or Relationship
If an entity instance may have either one relationship or another, but not both, the
constraint may be modelled with an exclusive-or relationship, represented as a tree with a
solid dot where the tree branches.
5.5.9
User Role Management E-R Diagram
The following diagram is the complete Entity Relationship Diagram for Intelligent
Question Bank And Examination System. It presents an abstract, theoretical view of the
major entities and relationships needed for IQBAES. This entity-relationship diagram
depicts the major concepts and relationships needed for IQBAES. It is neither a complete
data model depicting every necessary relational database table, nor is it meant to be a
Page 153 of 252
proscriptive design for implementations of IQBAES. Figure 43 – User Role Management
E-R Diagram shows the E-R Diagram for user role management in IQBAES.
Figure 43 – User Role Management E-R Diagram
Page 154 of 252
5.5.10 Examination Management E-R Diagram
Figure 44 – Examination Management E-R Diagram shows the E-R Diagram for
examination management in IQBAES.
Figure 44 – Examination Management E-R Diagram
Page 155 of 252
5.5.11 Question Bank E-R Diagram
Figure 45 – Question Bank Management E-R Diagram shows the E-R Diagram for
question bank management in IQBAES
Figure 45 – Question Bank Management E-R Diagram
Page 156 of 252
5.5.12 Question System E-R Diagram
Figure 46 – Question System E-R Diagram shows the E-R Diagram for question
system in IQBAES.
Figure 46 – Question System E-R Diagram
Page 157 of 252
5.6
Data Dictionary
The data dictionary is the repository of all elements in the system. The following list
shows all the tables in the IQBAES database (database schema). This database schema
gives the details for each table, regarding fields and keys. Some table might not be used
now but exist in the current IQBAES database for future expansion.
Table 8 – chapter
Field
Id
Name
CourseId
CreatedDate
CreatedBy
ModifiedDate
ModifiedBy
Type
String
String
String
Date
String
Date
String
Comment
System generated unique id
Chapter name
Course unique id
Table 9 – classschedule
Field
Id
ClassId
StartDateTime
EndDateTime
CreatedDate
CreatedBy
ModifiedDate
ModifiedBy
Type
String
String
Date
Date
Date
String
Date
String
Comment
System generated unique id
Classroom unique id
Page 158 of 252
Table 10 – country
Field
Id
Name
CreatedDate
CreatedBy
ModifiedDate
ModifiedBy
Type
String
String
Date
String
Date
String
Comment
System generated unique id
Country name
Table 11 – course
Field
Id
Code
Name
ProgramId
SemesterId
GradeId
LecturerId
CreditHours
CreatedBy
ModifiedDate
ModifiedBy
Type
String
String
String
String
String
String
Date
String
String
Date
String
Comment
System generated unique id
Course code
Course name
Program unique id
Semester unique id
Grade unique id
Lecturer unique id
Course credit hours
Table 12 – courseenrolment
Field
Id
StudentId
CourseId
semesterDateId
CreatedDate
CreatedBy
ModifiedDate
ModifiedBy
Type
String
String
String
String
Date
String
Date
String
Comment
System generated unique id
Student unique id
Course unique id
Semester date unique id
Page 159 of 252
Table 13 – coursetype
Field
Id
name
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
Date
String
Date
String
Comment
System generated unique id
Course type name
Table 14 – department
Field
Id
name
facultyId
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
String
Date
String
Date
String
Comment
System generated unique id
Department name
Faculty unique id
Table 15 – exam
Field
examId
semesterDateId
programId
courseId
gradeId
description
noOfQuestion
duration
startDate
startTime
endTime
status
Type
String
String
String
String
String
String
Integer
String
Date
String
String
String
Comment
System generated unique id
Semester date unique id
Program unique id
Course unique id
Grade unique id
Page 160 of 252
createdDate
createdBy
modifiedDate
modifiedBy
Date
String
Date
String
Table 16 – exammanager
Field
id
examId
studentId
questionPass
questionFail
grade
questionId
level
isAnswer1
isAnswer2
isAnswer3
isAnswer4
isAnswer5
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
String
Integer
Integer
String
String
String
Integer
Integer
Integer
Integer
Integer
Date
String
Date
String
Comment
System generated unique id
Exam unique id
Student unique id
Question correct/wrong flag
Question correct/wrong flag
Question unique id
Table 17 – examsummary
Field
id
examId
studentId
level
status
QCounter
createdDate
Type
String
String
String
String
String
Integer
Date
Comment
System generated unique id
Exam unique id
Student unique id
Question Counter for each student exam
Page 161 of 252
createdBy
modifiedDate
modifiedBy
String
Date
String
Table 18 – faculty
Field
id
name
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
Date
String
Date
String
Comment
System generated unique id
Faculty name
Table 19 – gender
Field
id
name
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
Date
String
Date
String
Comment
System generated unique id
Gender name (E.g. Male/Female)
Table 20 – gradedetail
Field
id
masterId
fromMark
toMark
grade
cgpa
description
status
createdDate
createdBy
Type
String
String
Integer
Integer
String
String
String
String
Date
String
Comment
System generated unique id
Office address of the provider
Mark range from
Mark range to
Pass/Fail
Page 162 of 252
modifiedDate
modifiedBy
Date
String
Table 21 – grademaster
Field
id
code
name
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
String
Date
String
Date
String
Comment
System generated unique id
Grade code
Grade Name
Table 22 – intake
Field
id
intakeNo
startDate
endDate
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
Date
Date
Date
String
Date
String
Comment
System generated unique id
Intake No
Intake start date
Intake end date
Table 23 – lecturer
Field
id
code
name
qualification
address
email
telNo
Type
String
String
String
String
String
String
String
Comment
System generated unique id
Lecturer code
Lecturer name
Page 163 of 252
faxNo
createdDate
createdBy
modifiedDate
modifiedBy
String
Date
String
Date
String
Table 24 – levelofdifficulty
Field
id
questionId
weightFactor1
weightFactor2
weightFactor3
noOfCondition
noOfDownwardEdges
noOfUpwardEdges
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
Integer
Integer
Integer
Integer
Integer
Integer
Date
String
Date
String
Comment
System generated unique id
Table 25 – majordetail
Field
id
masterId
courseId
minMark
minGrade
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
String
String
String
Date
String
Date
String
Comment
System generated unique id
Major master table unique id
Course unique id
Page 164 of 252
Table 26 – majormaster
Field
id
name
code
programId
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
String
String
Date
String
Date
String
Comment
System generated unique id
Major name
Major code
Table 27 – maritalstatus
Field
id
name
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
Date
String
Date
String
Comment
System generated unique id
Single/Marriage
Table 28 – nationality
Field
id
name
countryId
createdDate
createdBy
modifiedDate
modified By
Type
String
String
String
Date
String
Date
String
Comment
System generated unique id
Nationality name
Country unique id
Page 165 of 252
Table 29 – placeofbirth
Field
id
name
stateId
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
String
Date
String
Date
String
Comment
System generated unique id
Place of birth name
State unique id
Table 30 – program
Field
id
name
code
facultyId
departmentId
programTypeId
creditHours
coordinatorId
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
String
String
String
String
String
String
Date
String
Date
String
Comment
System generated unique id
Program name
Program code
Faculty unique id
Department unique id
Program type unique id
Credit Hours
Lecturer unique id
Table 31 – programtype
Field
id
name
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
Date
String
Date
String
Comment
System generated unique id
Program type name
Page 166 of 252
Table 32 – questionbank
Field
id
programId
courseId
chapterId
topicId
problemType
parameter
answers
difficultyId
subQuestionListID
subQuestionParameters
isMultiple
question
option1
option2
option3
option4
option5
fileName
mark
isOption1
isOption2
isOption3
isOption4
isOption5
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
String
String
String
String
String
String
String
String
String
Integer
String
String
String
String
String
String
String
String
Integer
Integer
Integer
Integer
Integer
Date
String
Date
String
Comment
System generated unique id
Program unique id
Course unique id
Chapter unique id
Topic unique id in knowledge map
Question difficulty unique id
Question is of type multiple choice
Answer option 1
Answer option 2
Answer option 3
Answer option 4
Answer option 5
Page 167 of 252
Table 33 – race
Field
id
name
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
Date
String
Date
String
Comment
System generated unique id
Race name
Table 34 – religion
Field
id
name
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
Date
String
Date
String
Comment
System generated unique id
Religion name
Table 35 – resoccupancy
Field
id
resourceId
occupancyId
occupancyType
startDateTime
endDateTime
description
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
String
String
Date
Date
String
Date
String
Date
String
Comment
System generated unique id
Resource unique id
Occupancy unique id
Page 168 of 252
Table 36 – resource
Field
resourceId
name
resourceTypeId
location
remark
maxSeatNo
computerNo
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
String
String
String
Integer
Integer
Date
String
Date
String
Comment
System generated unique id
Lecturer name
Table 37 – resourcetype
Field
id
name
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
Date
String
Date
String
Comment
System generated unique id
Resource type name
Table 38 – salutation
Field
id
name
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
Date
String
Date
String
Comment
System generated unique id
Salutation name
Page 169 of 252
Table 39 – semester
Field
id
intakeId
semesterNo
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
Integer
Date
String
Date
String
Comment
System generated unique id
Inteke unique id
Semester no
Table 40 – semesterdate
Field
id
name
semesterNo
startDate
endDate
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
Integer
Date
Date
Date
String
Date
String
Comment
System generated unique id
Semester Date name
Table 41 – state
Field
id
name
countryId
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
String
Date
String
Date
String
Comment
System generated unique id
State name
Country unique id
Page 170 of 252
Table 42 – student
Field
studentId
status
dateEnrolled
name
salutationId
studentNo
isLocal
ICPassport
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
Date
String
String
String
Integer
String
Date
String
Date
String
Comment
System generated unique id
Enrolment date
Salutation unique id
Table 43 – studentcourse
Field
id
studentId
courseId
semesterId
status
semesterDateId
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
String
String
String
String
Date
String
Date
String
Comment
System generated unique id
Student unique id
Course unique id
Semester unique id
Semester date unique id
Table 44 – studentcoursehistory
Field
id
studentCourceId
status
description
Type
String
String
String
String
Comment
System generated unique id
Student course unique id
Page 171 of 252
actedBy
actedDate
String
Date
Table 45 – studentdetail
Field
studentId
age
maritalStatusId
genderCode
dateOfBirth
placeOfBirthId
religionId
raceId
nationalityId
contactAddress
contactPostcode
contactTown
contactState
contactCountry
contactTelNo
homeAddress
homePostCode
homeTown
homeState
homeCountry
homeTelNo
hpNo
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
Integer
String
String
Date
String
String
String
String
String
String
String
String
String
String
String
String
String
String
String
String
String
Date
String
Date
String
Comment
Studeny unique id
Marital status unique id
Place of birth unique id
Religion unique id
Race unique id
Nationality unique id
Page 172 of 252
Table 46 – studentprogram
Field
id
studentId
programId
intakeId
status
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
String
String
String
Date
String
Date
String
Comment
System generated unique id
Student unique id
Program unique id
Intake unique id
Table 47 – studentprogramhistory
Field
id
studentProgramId
status
description
actedBy
actedDate
Type
String
String
String
String
String
Date
Comment
System generated unique id
Student program unique id
Table 48 – studentsemester
Field
id
studentId
intakeid
semesterId
semesterDateId
semesterStatus
programId
examStatus
description
createdDate
createdBy
Type
String
String
String
String
String
String
String
String
String
Date
String
Comment
System generated unique id
Student unique id
Intake unique
Semester unique id
Semester date unique id
Semester status
Program unique id
Page 173 of 252
modifiedDate
modifiedBy
Date
String
Table 49 – t_grp_reg
Field
id
group_id
user_id
createdDate
createdBy
Type
String
String
String
Date
String
Comment
System generated unique id
Table 50 – t_property_holder
Field
id
code
value
Type
String
String
String
Comment
System generated unique id
Table 51 – t_role_reg
Field
id
role_id
entity_id
entity_type
created_date
created_by
Type
String
String
String
String
Date
String
Comment
System generated unique id
Table 52 – t_user
Field
id
user_id
first_name
last_name
Type
String
String
String
String
Comment
System generated unique id
Page 174 of 252
email
password
created_date
created_by
Modified_date
Modified_by
String
String
Date
String
Date
String
Table 53 – t_user_group
Field
group_id
name
description
created_date
created_by
modified_date
modified_by
Type
Comment
String
String
String
Date
String
Date
String
Table 54 – t_user_group_child
Field
group_id
child_id
Type
Comment
String
String
Table 55 – town
Field
id
name
stateId
createdDate
createdBy
modifiedDate
modifiedBy
Type
String
String
String
Date
String
Date
String
Comment
System generated unique id
Page 175 of 252
6
6.1
SYSTEM IMPLEMENTATION AND TESTING
Introduction
In the process of system implementation, the system requirements are converted to
program codes. This is the software development phase that is concerned with the
development environment.
6.2
Development Environment
During the early stages of development, various tools were tested to ensure the suitability
of the tools to the task. This is rather and important phase that precedes the actual
development itself. There is a need to discover what tools are available, and to test out
those tools that operate in the hardware and software development platform.
Admittedly, this phase took up a great deal of time in the search for the right
development tools. Tools and development environments need to fit into the available
resources. On the other hand, it may be necessary to acquire new equipment to properly
execute new tools.
The development and implementation were conducted in two locations:
•
Development Environment
The development activities were conducted at home and in this case home serves
as the main development environment.
•
Field Trial Environment
The developed system was then transported over to the actual environment
(university environment) for testing and implementation. Obviously, there were
problems that were encountered in transporting between the two environments.
Page 176 of 252
6.2.1
Hardware Environment
The following hardware specifications have been used to develop the IQBAES system.
There are two sets of development machines at home - 1 physical pc and 1 virtual pc.
Virtual pc was installed with Red Hat Linux 9.0 Operating System for testing IQBAES’s
compatibility on Linux platform. Each of which has the following configurations:
a. Intel Pentium III Processors running at 900 MHz
b. 512 MB RAM
c. 40GB Hard Disk
d. Standard hardware for IBM PC compatible machine
These two sets of machine also act as the client and server machine. Initially, the
physical machine, which was installed with Windows 2000 OS, was used as the server
whereas the Linux machine served as the client. This arrangement was then being
swapped, where Windows machine was interchanged with Linux machine to act as the
client and Linux machine served as the server. Again, the main purpose of swapping the
machine with different operating system was to make sure that IQBAES was fully
compatible with both of the platform.
6.2.2
Software Environment
A wide variety of software tools was used in the development of IQBAES. Many of these
tools could be obtained free from the Internet open source domain. Those tools used to
build this system were fully open source except the Windows operating system on which
the system running on was commercially available.
Page 177 of 252
The list of tools use in IQBAES is listed as below:
Table 56 – Software Technology Listing
Software Category
Operating System
Software Name
Microsoft
Features
Platform for IBAES system
Windows 2000,
Microsoft
Windows XP
Professional, Red
Hat Linux 9.0
Application Server
Web Server
Tomcat Catalina
Application server that host the IQBAES
4.1
system
Apache Web
Web server that used to host IQBAES web
Server
pages
Software Development Eclipse 3.0.1 IDE
Tool used for coding
Tools
Database Server
MySQL 4.0.14
IQBAES was hosted on this database
server
Database Client
MySQL-Front 2.5 Tools use to connect to MySQL database
server and browse the database schema
Web Browser
Internet Explorer
Client browser used to access the IQBAES
6.0, Netscape
web-site
UML
MagicDraw
For drawing UML diagram
Deployment Tools
Apache Ant
Build tools for compilation and deployment
to application server
Logging Tools
Log4j
Library for keep track of the system log file
Database Access
Hibernate
Object-relational persistence database layer
Layer
Page 178 of 252
6.2.3
Development with Provision for Disaster Recovery and Mirroring
It is to be noted that the development process has made provision for disaster recovery.
The impetus for this pattern of disaster recovery was driven by several disasters that had
happened before in the development process. The Reasons for the disaster recovery
Development Plan is to support continuous development so that development will
proceed with minimal down time in the case of a disaster. Development can be mirrored
in two locations. Sometimes it is necessary to port development from one location to
another, such as from home to office and back. The hardware and software configurations
in section 6.2.1 and 6.2.2 must be set up in both locations. The mirroring of the
development platform in this case need not be simultaneous or synchronous, as long as
the full development platform is available at two sites at different times. This is
asynchronous mirroring of the development platform.
Disasters in the development process can hinder the continuity of the project and can
arise as a result of many factors, including the following situations:
Virus attack. Since this development involves using the Internet, new viruses could
destroy all data and development in a network. There was a situation that happened
before where both the development servers and workstation was infected with viruses
requiring a reformat of all the affected hard disks.
Hard Disk Failure. A failure in the development hard disk can terminate
development, resulting in partial or total loss of the development software.
Theft. Burglary could cripple a development environment, with the loss of vital
equipment and software at the affected site.
These are real life development concerns that we have encountered, and provisions have
been made to recover from these situations with a minimal down time. The mirrored site
was able to support continued development while the damaged site was being restored.
Page 179 of 252
6.2.4
Implementation Environment
The system has been implemented in one of the computer laboratory of Inti College
Malaysia, Negeri Sembilan. The Layout of the computers is shown in Figure 47 – ICM
Computer Laboratory Layout below. There is an application server and a variety of
clients using only a Web Browser (IE6.0 for Windows client and Netscape for Linux
client) each.
Figure 47 – ICM Computer Laboratory Layout
Page 180 of 252
6.2.5
System Development
The development strategy is a top-down approach, where the most important components
were first developed. Specifically, the IQBAES framework was developed at an early
stage of the development. Then the IQBAES system (IQBAES system sub-modules) was
developed later. Overall, the development plan is built in layers from the top, as mapped
out in the development strategy shown in Figure 48 – Top-Down Development Strategy.
Figure 48 – Top-Down Development Strategy
6.2.6
Classes and Components
This section will discuss the different classes and components used in the development
and implementation of IQBAES. Since this research involves the development of a
framework and subsequently the development of IQBAES on top of this framework, we
will discuss the core component (base classes and kernel classes) in the framework and
how IQBAES making use of these classes/components from the base layer of this
framework.
Page 181 of 252
6.2.6.1
IQBAES Framework
As discussed in the earlier chapter, base classes and kernel classes serve as the base for
this framework. These two components will make up the core functionality of the
framework. For instance, database connection holder, session handler, property holder,
message template and message holder are java classes that reside in this layer, which are
reusable across the platform. Any application that builds on top of this framework can
make use of these classes for database connection, session management, message
management and etc. There are many more classes that build into this layer but only a
few important classes will be discussed in the following section.
User Management Class library is a module or layer that builds on top of this core
classes. By utilizing these user management classes as a reusable component, a system,
which requires user management and / or user role management functionality, can call the
respective API to invoke and implement the user management module.
Other vertical components like Utility, CoreMsg, Validator and Menu can be customized
in accordance to the needs of the developed system. For instance, the validator classes
can be customized to provide the desire messages for fields’ validation on screen. Menu
can be customized to include only the necessary menu items. Utility classes can be
utilized by developer to customize the date format, the content of a message in plain text,
Html format and etc. All these reusable components can be reused for creating other web
application.
6.2.6.1.1 Com.um.kernel.base.DBConnectionHolder
This is one of the kernel classes that extend java.lang.Object. This class contains the
hibernate database information.
Page 182 of 252
6.2.6.1.2 Com.um.kernel.base.SesionHandler
This class extends java.lang.Object and handles all the session related processing.
Currently, it handles only the HTTP session.
6.2.6.1.3 Com.um.kernel.base.PropertyHolder
This class extends java.lang.Object and java.io.Serializable. It is designed to hold the
code and value of the property. This object is updatable to database.
6.2.6.1.4 Com.um.kernel.base.MessageTemplate
This class extends java.lang.Object and it stores all the message template information
from the xml configuration file.
6.2.6.1.5 Com.um.kernel.base.MessageHolder
This class extends java.lang.Object and it is a container class for the message. It stores
the message code and the place holder parameter. The message code must be in this
format "Property Message Identifier":"Message Id". The Property Message Identifier is
defined in the XML configuration file. The Message Id is defined in the message
properties file. This class can then be used by the MessageManager to get the localised
message.
6.2.6.1.6 Com.um.kernel.base.ActionContext
This class extends java.lang.Object. The class contains all the information about the
action. The class allows developer to get the parameters from the HTTP request as well
as to handle the session information. In addition to that, developer gets the UserContext
object. If the current action does not use the default view, then developer can specify the
required view by using the setViewId().
Page 183 of 252
6.2.6.1.7 Com.um.kernel.base.BaseAction
This class extends java.lang.Object and java.io.Serializable. The BaseAction abstract
must be implemented by the action object.
6.2.6.1.8 Com.um.kernel.base.DefaultAction
This class extends BaseAction. The default action is used to redirect or dispatch a view
only. It does not do any business processing. Whenever a web resource requires default
security check by the system, it can define the web resource as an action in the
configuration file and then assign a appropriate role to the resource
6.2.6.1.9 Com.um.kernel.base.SystemException
This class extends com.um.kernel.base.BaseException. All the System Exception should
extends from this class.
6.2.6.2
IQBAES System
Intelligent Question Bank and Examination System is developed on top of IQBAES
framework that we discussed earlier. With this, IQBAES make use of html and JSP pages
in the clients tier as the view component. The client tier is responsible for representing
the data to the user, interacting with the user, and communicating with the other tiers of
the system.
The middle tier is responsible for processing. IQBAES’s controller servlet in this case
serves as the controller to control the traffic flow within the system. For every
HttpRequest and HttpResponse that controller servlet received, it will refer to the
config.xml to determine the action classes and manager classes that the request or
response should be routed to. Action classes will then interact with the backend objects or
classes (hibernate generated classes) to access the MySQL database.
Page 184 of 252
6.2.6.2.1 Com.um.kernel.servlet.ControllerServlet
This class extends javax.servlet.http.HttpServlet. It is designed to delegate the request to
the appropriate action and performs system wide validation. The delegation of the request
will be based on the configuration in the xml file.
The system wide validations are down in the following manner:
Check whether the action is defined in the xml file
Check whether the user can access to this action
The class also handles the exception.
6.2.6.2.2 Com.um.kernel.base.JSPView
JSP view handler is responsible for processing and invoking the appropriate jsp.
The class will encode whatever parameters that are added in the ActionContext class.
Page 185 of 252
6.2.7
Question Difficulty Assessment Implementation
Code snippet above shows how degree of difficulty for each question being defined in
IQBAES. Various weight factors which defined by lecturer were obtained in this action
class via jsp. Once the pre-defined values are obtained, these level of difficulty values are
computed in this class by calling the action class’s process() method. Once the level of
difficulty value for each question has been computed, it is then associated to the
respective question and subsequently stored in the question bank.
Page 186 of 252
6.2.8
Automatic Question Generator
Page 187 of 252
The code snippet above typically shows how it collects the conditions and how system
makes use of the node to associate with question. The credit value denotes the question
difficulty. The larger is this number, the more complex the question is. System makes use
of queue to store the node. After a question is generated, the system keeps the question
and its node in the OPEN queue. When the system initializes the question to student, the
node in the OPEN queue will be set to visible. If the student answers the question
correctly, the node is then eliminated from the OPEN queue. It will then set to invisible.
6.2.9
Intelligent Questioning System
Code snippet above shows how student’s current level being retrieved from database.
This method resides in IQBAESManager class and its main feature is used to retrieve
students’ current level based on their examination history.
Page 188 of 252
Page 189 of 252
Code snippet above shows how questions are being selected from the question bank
based on programId, courseId, chapterId and its level. Once the questions being selected,
the questions will be dumped into the list object or so call as question set. Subsequently, a
shuffle method will be called to shuffle the question in the collection. This operation is to
ensure that the student will get different set of question randomly.
When the user completes the exam, the questioning system will provide an assessment
measure. The system then updates the user’s status in the system via the code snippet as
shown above.
Page 190 of 252
6.2.10 Coding
While writing the programming codes for IQBAES, the developer has followed strict
coding standards to make it easier to trace requirements from analysis through design and
implementation. Other than that, coding and naming standards make a piece of coding
more readable and eases the process of debugging. Naming standards are for variables,
objects, classes and controls.
6.2.10.1 Comments
Besides this, comments are inserted into the coding to help other developers understand
and maintain the code. Comments begin with “//” or “/** …*/” sign. An example would
be as below:
(a)
/**
* <p> To initialize the ActionManager</p>
*
* @param HashMap - contains name of the action as key and the action object as value
* @return None
* @exception None
*
*/
public static void init(HashMap acHash) {
actionHash=acHash;
}
(b)
/**
* <p> To get the OutputStream of the uploaded file </p>
*
* @param
None
* @return OutputStream - the outputStream
* @exception ApplicationException
*
*/
Page 191 of 252
public OutputStream getOutputStream() throws ApplicationException{
try
{
return this.fileItem.getOutputStream();
}
catch (Exception e)
{
throw new ApplicationException("Base:error.upload.file.get.outputstream",e);
}
}
(c)
// default constructor
public UserGroup()
{
}
Page 192 of 252
6.2.10.2 Error Handling
The coding in IQBAES is supported by error-handling codes or segments. These
segments are inserted into IQBAES to intercept and handle all unexpected errors that will
happen. Other than that, all error handling segments can be self-programmed and written,
providing much flexibility to handle the errors according to the developer’s choice. With
all these error handling segments, IQBAES is much reliable and stable, because every
time when error happens, the error handling code segment will be executed to avoid
system crashes or unnecessary restarts. An example of error handling is shown in the
figure below:
try
{
String pattern = (String)properties.get("pattern");
String filename = (String)properties.get("filename");
// if the file name is not available, then it does not require auditing.
if (filename != null) {
requireAudit = true;
appAuditor.addAppender
(
new RollingFileAppender
(new PatternLayout(pattern),filename,true));
}
} catch (Exception e) {
e.printStackTrace();
System.out.println("Error initialising AuditManager ");
}
Error in this
code segment
would be
handled by the
error handler
Error
handling
codes
Figure 49 – Error Handling if IQBAES
Page 193 of 252
6.2.10.3 Object-oriented programming style
IQBAES was developed using java programming language and with the architecture of
J2EE. Hence, needless to say, its coding style is object-oriented as well. Compared to
structural programming, object-oriented programming enables the developer to reuse the
codes written, saving precious time and promotes efficient usage of the objects and
classes. In IQBAES, all reusable functions are written in standard modules and can be
invoked anywhere in the program. All the same classes are reused again and again in
different modules.
6.2.10.4 Complexity of programming IQBAES
In order to make IQBAES efficient and accurate, some complex programming techniques
were applied while developing the code for IQBAES. For instance, Queue and
Collections were used to enable the developer to control and manage the classes
efficiently and easily. Collections are used in Java to store zero or more classes into
single class. This enables the developer to access every single class in the collection by
just referring to one single collection class, providing the ease and flexibility in
programming. This advantage comes with a price, which is the coding gets more
complicated, because an object is used to refer to many other objects.
Page 194 of 252
6.3
Testing Approach
The goals of software testing are to establish the functionality of a system as well as to
detect the presence of errors or inaccuracies in the system. In software production
company, the testing activity is mean to minimize cost of quality and defects in software
production. However, the main objective of testing is to find defects, verify capability
and predict reliability. There are various approaches to testing; each type of testing
validates different aspects of the software. The selection of the type of testing depends on
the nature of the software system. The initial stages of testing should be the responsibility
of the developers to test the codes they have developed and coded. When this stage is
completed, the project is handed over to an independent team to perform integration
testing.
During the process of testing, the software requirements must be made available, and
tests are then designed based upon these requirements. However, testing should not rely
totally on the software specifications, as the development of software is never a static
process. There are various reasons for this phenomenon to happen:
•
Requirements may be modified without updating the specification.
•
As the development project progresses, new and better ways may have been found
since the initiation of the software process, that can solve the problem, usually is a
better way.
•
Time constraints in the release of the software could result in the use of ‘shortcuts’
that bypasses some routines, which were deemed superfluous or unnecessary.
•
Certain unforeseen constraints in the hardware or software may prevent the exact
implementation of the software specification. For example, certain functionality may
perform in a manner that may be too unresponsive owing to the hardware available.
Certain functionality may not be possible to be programmed in the selected
programming language.
Page 195 of 252
•
Differences in implementation environment may demand alterations in the software
functionality. A system may perform perfectly well in the development hardware and
software environment. However, when the system is implemented in the customer’s
site, severe errors may arise owing to certain factors that are not present in the
development environment. This could arise from myriad causes:
•
The customer site uses a different processor, that is, there is a hardware conflict.
•
At the customer site, they may have installed other types of software, which conflict
with the system being implemented.
•
The usage load could vary. Other systems running at the customer site may already
have a certain degree of usage pattern or usage load that cause performance problems
with the system that is implemented.
•
Switch in Build Configuration. During testing, the systems may have been built using
the Debug Configuration. Once the testing has been completed, it is changed to
Release Configuration. Certain routines perform well in the Debug Configuration, but
the performance deteriorates, or even fails when switched to the Release
Configuration. This switch could result in a deviation from the specification in order
to get the system functioning again.
All these factors have to be considered in the planning of the testing stages, failing which
the testing would be incomplete.
Page 196 of 252
6.4
Type of Testing
The types of testing that are to be executed depends on the stage in the development and
testing. The testing process of IQBAES follows a pattern that can be shown in the Figure
50 – Testing Process. The first stage in the testing process involve unit testing. There
could be multiple units to test. Hence the multiple boxes in the diagram. After this, there
is the module testing stage, which can involve multiple modules to be tested. If any of
these modules are defective, it is necessary to return to unit testing. However, this return
path is optional, and is performed only if the module testing failed. The dotted arrow
indicates this optional path. This is followed by Interface testing, to test the
communication between the various modules. When all these are done, the integration
phase begins, combining all the individual modules together. Finally the whole system is
tested.
Figure 50 – Testing Process
Page 197 of 252
6.4.1
Unit Testing
This stage aims at testing the smallest components of the code. The initial units in this
project are the individual modules, each of which may consist of several units. The tests
are done by using the JUnit. There are several approaches to unit testing, each of which
has their own strengths and weaknesses.
6.4.1.1
Black Box Testing
This is a functional testing procedure to establish whether the unit meets its
specifications. For example, in the screen that allows the entry of two inputs. The output
is then studied. The result is studied whether it produces a successful output which
indicating a successful test. In Black Box testing, we are only concerned about what the
inputs are and what outputs are generated as a result of the test. Black box testing is only
concerned about functionality. If the output is incorrect, or there is no data output, the test
has failed.
6.4.1.2
White Box Testing
If the result of a black box testing indicates a failure, there are other approaches to rectify
the problem. Here the structural testing approach is needed, where the code is analyzed.
Knowledge of the structure of the component is vital in white box testing.
6.4.1.3
Code Flow
It is very often that it is necessary to examine the flow, or path of a code segment. This
could lead to a deeper and deeper examination of the flow of the logic of the code. Path
testing is also a part of examining code flow, which traces the flow of the program from
one point to another.
Page 198 of 252
6.5
Module Testing
When each of the units have been tested, it is necessary to ensure that the modules work
together. Such is the case of the examination module as one unit. Then there is the
student module to initiate call to the examination module and consequently obtain the
results from the module. The examination module consists of several units that are
combined together (such as internal scheduler, stopwatch, grading and etc). Each of these
units must have been tested earlier, and combined together to form the examination
module. Each unit does not work alone, as each unit depends on another unit. For
example, an internal scheduler will activate an examination. Once the exam is activated,
the stopwatch unit will have the control over examination process. It will control over the
time and does not allow user to answer questions after the define time limit. The testing
must ensure that these sub-units work correctly in a module.
6.6
Interface Testing
Interface testing in this project is very important. Data is passed between Client and Web
Server. These data must conform to the same standards and must reach their destination
in the form that is expected. In Interface testing, we need to know whether these
messages actually are in the format that is expected by both ends.
6.7
Integration Testing
There are two types of integration testing which is incremental testing and nonincremental testing. Non-incremental testing will combine all the modules and test the
program. With incremental testing, it systematically combines and tests each module.
IQBAES integration testing makes use of the thread testing approach (one of the testing
approach in incremental testing) to carry out the integration test. The integration test
starts by integrating groups of module that implement function together. It then continues
integrating and testing by adding one function at a time. This approach provides better
visibility and tracking of integration testing activities. This testing will expose any
inconsistencies in each module, as well as inconsistencies in the integration itself. Since
IQBAES was developed on its own framework, there is also a need to test the consistency
Page 199 of 252
between IQBAES framework base module and IQBAES system module. Figure below
depicts the thread testing approach:
Figure 51 – Integration Thread Testing
6.8
System Testing
The final step in the test is system testing, where a series of tests are conducted in order
to exercise the whole system. The whole system includes both the interaction between the
modules, as well as the interaction with the hardware. It aims at finding discrepancies
between the system and its requirements. It concerns validating system’s functional
requirements. In this project, there is also the network factor, and the interaction between
different computers in the network.
If the system works on one machine, can it work on other machines with different sets of
hardware and software? Here it was necessary to test by hosting IQBAES on one
machine, and performing a test on how clients from other machines access the IQBAES
web pages.
There is also the need to test for system performance, stress test, security access rights,
usability, data integrity, error handling and recovery.
Page 200 of 252
7
EVALUATION AND RESULTS
7.1
Evaluation methods
IQBAES has been tested and evaluated by lecturers and students of Inti College
Malaysia, and it has been used fully to process the examination results. The evaluation
process was carried out upon the development and testing of IQBAES completed. A
Questionnaire was given to those users directly involved with using the system. The
evaluation was performed using an examination for a course with large number of
students. The students filled in the questionnaires directly after the grading was done for
the exam. The students had already received their grades when filled in the
questionnaires. It is believed that this approach provides a measure of objectivity on the
student side.
The evaluation process will be divided into three parts. The first part will involve the
evaluation of the proposed IQBAES system in term of its user interface. 60
questionnaires were sent out to users of the system in ICM, and 55 responses were
received as of the date of concluding this report. The second part will evaluate in term of
its implementation and the evaluation was done based on 52 subjects/courses. The actual
questionnaire is appended in Appendix B. The third part will evaluate in term of its
reliability based on the data captured in the database.
7.1.1
Evaluation in Term of User Interface
Table 57 – Results of tallying the Questionnaire responses
Questionnaire
Results
Section A
Job title
Sex
Age Group
A
B
C
D
E
F
G
A
Job Title Legend
Faculty Administrator
3
3
2
0
20
17
10
B
Department Head
M
F
C
Examinations Administrator
18
37
D
Head of Program
15
30
E
Senior Lecturer
40
50
Page 201 of 252
10
9
24
11
F
Lecturer
G
Student
Section B
Response
1
2
3
4
5
Question
1
0
0
0
8
47
2
0
0
0
6
49
1. The traditional manual system of conducting
examination is the most convenient.
2. The traditional manual system of conducting
examination is faster.
3
0
0
0
2
52
3. We should stick to the old traditional manual
system.
4
0
0
0
9
47
4. The old online examination system works
efficient enough.
5
0
0
0
9
47
6
0
0
0
5
51
5. The old online examination system looks more
familiar to the usual way of conducting
examination.
6. The old online examination system is good
enough.
7
49
6
0
0
0
8
49
6
0
0
0
7. The new online examination system is more
convenient to use.
8. The new online examination system saves more
time.
9. The new online examination system is more
user friendly.
9
48
7
0
0
0
10
46
9
0
0
0
11
48
7
0
0
0
12
50
5
0
0
0
13
47
7
0
0
0
14
50
3
0
0
0
15
51
3
0
0
0
16
52
2
0
0
0
17
48
5
1
0
0
1
2
3
0
0
0
0
0
0
0
0
0
11
8
2
42
45
52
4
0
0
0
5
49
Section C
10. It takes a shorter period of time to conduct
exams in the new online examination system.
11. The new online examination system is more
familiar to the Web surfing applications.
12. The new online examination system should be
implemented to replace the older system.
13. The new online examination system is fair and
efficient to use.
14. The new online examination system is more
colorful and pleasing to the eyes.
15. We must use the new online examination
system and implemented as soon as possible.
16. The new online examination system has more
useful features than the old system.
17. The new system should be used in all higher
education institutions.
(refer to questionnaire, Appendix B)
1. Colour scheme of the various pages
2. Layout of the entry boxes
3. Look and design of buttons
4. Design of the arrangement of menu of choices
of the main side box such as
Page 202 of 252
Enter Teacher Details, Class Info, Class-Subjects,
Student Info, Enter Marks
Section D
1
A
0
B
0
C
53
2
3
1
0
0
6
52
47
Implementation
(refer to questionnaire)
60 questionnaires were sent out to users of the system in ICM, and 55 responses were
received as of the date of concluding this report. The demography of the responses is
shown
in
the
Table
58
–
Job
Titles
to
Page 203 of 252
Table 60 – Age Groups of respondents. There appear to be discrepancies in the total
numbers in the tables because some chose not to reveal their gender and ages.
Table 58 – Job Titles
Job Title
Frequency
Faculty Administrator
Department Head
%
3
3
5.8%
5.8%
Examinations Administrator
Head of Program
Senior Lecturer
Lecturer
2
0
20
17
3.8%
0.0%
38.5%
32.7%
Student
10
19.2%
Total
55
Table 59 – Gender
Gender
Male
Female
Total
Frequency
18
37
%
32.7%
67.3%
55
Page 204 of 252
Table 60 – Age Groups of respondents
Age Group
15-29
30-39
40-49
50-59
Frequency
10
9
24
11
Total
%
18.5%
16.7%
44.4%
20.4%
54
Table 61 – Summary of Section B, questions 1 –3
Summary of responses to the manual
system1
response
strongly agree
agree
undecided
disagree
strongly disagree
total
freq.
0
0
0
16
148
percentage
0.0
0.0
0.0
9.8
90.2
164
Page 205 of 252
Table 62 – Summary of Section B, questions 4-6
Summary of responses to Old DOS
system2
response
strongly agree
Freq.
0
percentage
0.0
0
0
0.0
0.0
disagree
strongly disagree
23
145
13.7
86.3
total
168
agree
undecided
Table 63 – Summary of Section B, questions 7-17
Summary of responses to new Web
Services system
response
strongly agree
agree
undecided
disagree
strongly disagree
total
Freq.
percentage
555
55
4
0
90.4
8.9
0.7
0.0
0
0.0
614
Page 206 of 252
Table 64 – Section C, User Interface Colour Scheme
Section C
User Interface
This section deals with the design of the user interface as used in the new Web-based
computerized examinations system.
Question 1
response
1. Very bad,
2. Not
satisfactory
3. Satisfactory
Colour scheme of the various pages
freq
%
0
0.0%
0
0.0%
1
1.9%
4. Good
11
20.4%
5. Excellent
42
77.7%
Total
54
Page 207 of 252
Table 65 – Section C, User Interface Layout
Section C
User Interface
Question 2
response
Layout of the entry boxes
freq
%
1. Very bad,
0
0.0%
2. Not
satisfactory
0
0.0%
3. Satisfactory
0
0.0%
4. Good
8
15.1%
5. Excellent
45
84.9%
Total
53
Page 208 of 252
Table 66 – Section C, User Interface Buttons
Section C
User Interface
Question 3
Look and design of buttons
response
1. Very bad,
freq
%
0
0.0%
2. Not
satisfactory
0
0.0%
3. Satisfactory
0
0.0%
4. Good
2
3.7%
5. Excellent
52
96.3%
Total
54
Page 209 of 252
Table 67 – Section C, User Interface Menus
Section C
User Interface
Question 4
Design of the arrangement of menu
response
1. Very bad,
freq
%
0
0.0%
2. Not
satisfactory
0
0.0%
3. Satisfactory
0
0.0%
4. Good
5
9.3%
5. Excellent
49
90.7%
Total
54
Table 68 – Section D, Question 1
Section D
Implementation
Which of the following step is the most appropriate action to be taken by the
school administration concerning the new computerized system
1
A
Response
Wait for more testing to be done before making a decision
B
Wait until next year to implement the system
C
freq
%
0
0.0%
0
0.0%
The new system should be implemented immediately
53
100.0%
Total
53
Page 210 of 252
Table 69 – Section D, Question 2
Section D
Implementation
What is the most important change to be made before the new system should be
implemented
2
A
Response
Improve the colour scheme
B
The system needs a total overhaul
C
freq
%
1
1.9%
0
0.00%
No further change need to be made before implementing
52
98.1%
Total
53
Table 70 – Section D, Question3
Section D
Implementation
Do you need more training to use the new computerized
system
3
A
Response
Still cannot understand the new system
B
Need more explanation of the procedures
C
freq
%
1
1.2%
3
3.5%
The system is easy enough to understand, no further
training is needed
81
95.3%
Total
85
Page 211 of 252
7.1.2
Evaluation in Term of Implementation
We are interested in investigating the following questions:
Q1. Did students enjoy the IQBAES exam process?
Q2. Did students think they learned through the exam process?
Q3. Did students think the grading system was fair?
Q4. Would students rather take the traditional exam than the IQBAES exam?
Q5. Would students recommend using the IQBAES online exam in the future?
Q6. Do exam grades affect students’ perception of the exam process?
Q7. Does enjoyability of the exam process affect students’ perception of learning,
grading fairness and satisfaction?
Q8. Do students’ perception of learning, fairness in grading, and satisfaction affect their
recommendation of the process being used in the future?
Q9. Are there differences in students’ perception of exam enjoyability, learning,
satisfaction, and grading fairness between those taking the traditional in-class exam and
those taking the IQBAES exam?
Our hypotheses bases on the above questions are as below if this research fully meets its
objectives:
Students who participate in the IQBAES exam will report higher enjoyability of the
process than students who participate in the traditional in-class exam.
Students who participate in the IQBAES exam will report higher perceived learning
than students who participate in the traditional in-class exam.
Students who participate in the IQBAES exam will report higher satisfaction than
students who participate in the traditional in-class exam.
Students who participate in the IQBAES exam will report lower perceived fairness in
grading than students who participate in the traditional in-class exam.
Page 212 of 252
Students’ exam grade will positively correlate with students’ perceived enjoyability
of the exam process.
Students who perceive higher enjoyability will report higher perception of learning
through the exam process.
Students who perceive higher enjoyability will report higher satisfaction with the
exam process.
Students who perceive higher enjoyability will report higher perception of fairness in
grading.
Students who receive higher grades will report higher perception of learning through
the exam process.
Students who receive higher grades will report higher satisfaction with the exam
process.
Students who receive higher grades will report higher perception of fairness in
grading.
Students who report higher perceived learning will recommend the IQBAES exam
process more.
Students who are more satisfied will recommend the IQBAES exam process more.
Students who report higher perceived fairness in grading will recommend the
IQBAES exam process more.
7.1.3
Finding
In this section, we first present the evaluation results of the IQBAES exam process,
addressing questions 1-5. Next, we use factor analysis and correlation analysis to find the
underlying factors and the relationship among them, addressing questions 6 -8. Finally
we compare results between the IQBAES exam and the traditional in-class exam,
addressing question 9.
5-point scales were used throughout the survey and the results are scored on a basis of 15, where:
Page 213 of 252
SA: Strongly Agree = 5; A: Agree = 4; N: Neutral = 3; D: Disagree = 2; SD: Strongly
Disagree = 1
The total number of subjects/courses in the following tables is 52 unless noted otherwise.
7.1.3.1
Enjoyability of IQBAES Exam Process
Table below answers our first question: did students enjoy the collaborative online exam?
Results show that students found the IQBAES exam provides flexibility in having
multiple resources and a comfortable timeframe, is less stressful, and allows students to
demonstrate what they learned in class.
Table 71 – Enjoyability of IQBAES Exam Process
Item
SD
D
N
I enjoyed the flexibility
5.8%
7.7%
11.5%
11.5%
36.5%
25%
0%
13.5%
1.9%
7.7%
of
having
A
SA
Mean
S.D
3.90
1.14
1.9%
3.31
1.04
30.8%
40.4% 15.3%
3.58
0.91
5.8%
15.4%
53.8% 23.1%
3.90
0.89
13.5%
17.3%
46.2% 15.4%
3.48
1.15
40.4% 34.6%
multiple
reference available
I
felt
under
much
25%
pressure taking the exam
this way
The exam enabled me to
demonstrate
what
I
learned in class
The exam provided a
comfortable timeframe
I
enjoyed
the
exam
process
Page 214 of 252
7.1.3.2
Learning Effects
Table below addresses our second question: did students think they learned through the
exam process? The majority of students agreed that all phases of the exam process were
part of the learning process. In addition, students report that the IQBAES exam increased
critical thinking skills. Furthermore, students report higher motivation to learn, including
doing additional reading.
Table 72 – Perceived Learning in IQBAES Exam
Item
My
skill
in
critical
SD
D
N
1.9%
0%
42.3%
0%
1.9%
0%
A
SA
Mean
S.D
34.6% 21.2%
3.73
0.866
36.5%
40.4% 21.2%
3.81
0.793
11.5%
30.8%
34.6% 23.1%
3.69
0.961
5.8%
15.4%
36.5%
28.8% 13.5%
3.29
1.07
5.8%
7.7%
42.3%
38.5%
3.31
0.92
thinking was increased
My ability to integrate
facts improved
I was stimulated to do
additional reading
I learned from questions
in IQBAES
I mastered the course
5.8%
materials
7.1.3.3
Perceived Fairness in Grading
Table below addresses our third question: Did students think the grading system was fair?
Students’ responses fluctuated around neutral on these questions.
Table 73 – Perceived Fairness in Grading in IQBAES Exam
Item
SD
D
N
A
SA
Mean
S.D
I felt the grading process
11.5%
25%
19.2%
40.4%
3.8%
3.00
1.14
was fair
Page 215 of 252
7.1.3.4
Overall Satisfaction and Recommendation
Tables below address next two questions: would students prefer to take the traditional
exam than the IQBAES exam? Would students recommend using the IQBAES exam in
the future? Results indicate that students report subjective satisfaction with the IQBAES
in general, and are willing to recommend using this process in the future.
Table 74 – Overall Satisfaction with the IQBAES Exam Process
Item
SD
D
N
A
SA
Mean
S.D
I would rather take the
23.1%
21.2%
30.8%
15.4%
9.6%
3.33
1.26
traditional exam
Table 75 – Overall Recommendation of Using the IQBAES Exam Process
Item
Strongly Oppose Neutral Recommend
Oppose
I
would
3.8%
Strongly
Mean
S.D
3.67
1.06
Recommend
7.7%
30.8%
32.7%
25%
recommend
using
this
process in the
future
7.1.3.5
Factor Analysis and Correlation Analysis
Factor analysis was conducted to find the underlying factors. Three factors were found
through principle component factor analysis, each corresponding to one aspect of the
students’ responses. Reliability analysis shows all three factors are highly reliable.
Enjoyability of the Exam Process
Perceived Learning
Perceived Fairness in Grading
Page 216 of 252
Next we conducted correlation analysis to find the relationship between these factors.
First we are interested in the question (Q6): Do exam grades affect students’ perception
of the exam process? It is necessary to point out that students received their grades before
they answered the survey, and we hypothesize that students’ exam grade is positively
correlated with students’ perceived enjoyability of the exam process. In addition, we
hypothesize that students who receive higher grades would report higher perception of
learning through the exam process, higher satisfaction with the exam process, and higher
perception of fairness in grading.
Second in our correlation analysis, we are interested in the question (Q7): Does
enjoyability of the exam process affects students’ perception of learning, grading fairness
and satisfaction? We hypothesize those students who perceive higher enjoyability report
higher perception of learning through the exam process, higher satisfaction with the exam
process, and higher perception of fairness in grading. Given that all factors in this
analysis are normally distributed, we used parametric correlation analysis and find that
the enjoyability factor is significantly correlated with perceived learning (Pearson’s r =
0.486, p<0.01), satisfaction (Pearson’s r = 0.597, p<0.01), and perceived fairness in
grading (Pearson’s r = 0.376, p<0.01).
Third, we investigate in the question (Q8): Do students’ perception of learning, fairness
in grading, and satisfaction affect their recommendation of the process being used in the
future? We hypothesize that recommendation of the process is positively associated with
perceived learning, satisfaction, and perceived fairness in grading. Given all factors in
this analysis are normally distributed, we used parametric analysis, and find that
recommendation is significantly correlated with perceived learning (Pearson’s r = 0.431,
p<0.01), satisfaction (Pearson’s r= 0.71, p<0.01), and perceived fairness in grading
(Pearson’s r= 0.424, p<0.01).
Page 217 of 252
7.1.3.6
Comparing IQBAES Exam and Traditional Exam
Table below shows that students in the IQBAES exam as compares with the traditional
exam significantly enjoyed the exam process (mean at 3.11 and 3.46, p<0.05), and are
significantly more satisfied with the exam process (mean at 2.78 and 3.33, p<0.05).
Table 76 – Comparison between Traditional Exam and IQBAES Exam
Factor
Mean
S.D
41
3.11
0.70
IQBAES
52
3.46
0.81
Traditional
41
3.81
0.71
IQBAES
52
3.65
0.68
Traditional
41
3.62
0.79
IQBAES
52
3.40
0.97
Traditional
41
2.78
1.37
IQBAES
52
3.33
1.26
Exam
No Of
Type
student
Traditional
Enjoyability of the Exam Process
Perceived Learning
Perceived Fairness in Grading
Overall Satisfaction
7.1.4
Reliability Assessment
With the use of the students’ examination data which were captured in the IQBAES’s
database, we can evaluate the reliability of IQBAES examination against the traditional
in-class examination.
7.1.4.1
Testing Reliability
By definition, someone takes an exam or test measure achieves the “observed score.” The
observed score is thought to be composed of two different components: The true score –
an exact measure of the amount of skill assessed, offset by a random error score (Novak
Page 218 of 252
et al., 1999). The error may result from individual student responses to particular
questions (Manson & Branle, 1997) or external conditions such as fatigue and noise.
Thus,
Observed score = True score + Error score
The variance of the observed score is composed of the variance of true scores and the
variance of error.
Then,
σ 2o = σ 2t + σ 2e
where, σ 2 o = variance in observed scores, σ 2 t = variance in true scores, and σ 2 e =
variance of error.
The reliability of an exam/assessment measures its repeatability from random testing
errors. An assessment is considered reliable if it would give us the same result over and
over again. By definition, the reliability is the ratio of variance of the true skill divided by
the observed variance.
Reliability = [σ 2 t ] /[σ 2 o ] ] = [ σ 2 t ] / [ σ 2 t + σ 2 e ], so
Reliability = 1 - σ 2 e / σ 2 o
The reliability provides a direct guide to the reproducibility of the grade between two
similarly prepared tests. The second line shows that as the reliability => 1, the error => 0.
A standard method of determining the reliability of an assessment is to divide the test
questions into two subtests, for example the odd and even questions assuming they were
equal in coverage of topics and difficulty. If the resulting subtests determine skill with
perfect reliability, each student would receive the same score on both subtests
Page 219 of 252
Reliability = 1 - (1 + β ) 2 / 4 β * σ 2 ( a −b) / σ 2 o + (1 − β ) 2 / 4 β = 0.85
Where β = µ a / µ b and σ 2 o = σ 2 ( a +b ) . In this formula, the key determiner of the
reliability is the mean variation of the difference between the scores on the even and odd
collections of questions. Even if the mean on the “a” questions is 20% higher than that on
the “b” questions, the error σ 2 e equals σ 2 ( a −b) to within 1%.
Theoretically, if someone is highly skilful, they generally make fewer errors and take less
time to complete a task successfully than a less skilful person. Moreover, any individual
has the option to work more quickly at the expense of making more errors. Thus if we
include the number of wrong responses and the time taken by students, we get much
more insight into their skill level. With these considerations in mind, we use an
assessment method which mimics the lost point grade but depends on the time taken to
solve a question and student’s wrong answers. We make use of the data to plot a graph
from the even question and odd question for each student.
The curvature indicates a small excess of easier problems in the even group. The uneven
split reliability of the combined assessment is 0.9976. This is an improvement over the
traditional in-class examination. That means that 99.8 percent of the observed variance in
scores is due to true score variance. The IQBAES has 4 times less error per unit time than
the traditional in-class examination. Overall, it has 62 times less error than the traditional
in-class exam.
Table below shows the differences among traditional in-class exam versus IQBAES.
IQBAES gives 62 times less error than the traditional in-class examination.
Table 77 – Reliabilities’ Summary
Traditional
r2
Reliability
σ 2 e / σ 2t
σ e /σ t
0.41
0.85
0.17
0.42
Page 220 of 252
Exam
IQBAES
7.1.5
0.986
0.9776
0.0024
0.05
Fairness
The traditional exam contains error that 1/4 of the students assigned failing grades have a
passing skill and an equal number of students who should have failed are assigned a
passing grade. Thus, for each two students failed, one student was misdiagnosed. To see
how this arises, look at Figure 52 – Comparison of IQBAES and Traditional
Examination, which was captured from SPSS tool, where failure is set at one standard
deviation below average, which fails 16 percent of the students. The dashed line shows
the true skill distribution inferred from the measured error and the observed skill
distribution (heavy line), which is slightly broader. The number of students failed is
indicated by the small curve centred on skill = 40. (The width of this curve is very near to
the width of the error, that is 38% of σ true ). Below skill = 35 the curve of students failed
lies on top of the true skill distribution, indicating that all students whose skill is this low
fail. Between 30 and 46 (the selected true skill for passing) it drops well below the true
skill distribution, indicating that some students (lighter shaded area) whose ability is
below passing will pass. The distribution of failed students does not drop immediately to
zero above true skill = 46, indicating that some students (darker shaded area) whose true
ability is passing will fail the test.
Page 221 of 252
Figure 52 – Comparison of IQBAES and Traditional Examination
In both plots, the dashed curve is the distribution of true skill and the thick curve is the
distribution of observed skill (i.e. grade). The shaded area shows students who fail/pass
although their true skill level is passing/failing. a) In the case of the traditional exam, 1 in
4 students who fail actually deserve to pass b) If IQBAES were used instead of the
traditional exam, only 3 percent of the students failed would be misdiagnosed.
Page 222 of 252
8
8.1
CONCLUSION AND FUTURE WORK
Summary of Project
This thesis describes a research focusing on web-based examination system and question
bank. The work encompasses detailed investigation of the subject matter, development of
a question bank and examination system, proposal of an enhanced question bank
examination system, evaluation of both existing and the proposed intelligent question
bank and examination system through the developed system, and analysis of the results
from the system.
The thesis began by investigating the manual examination system and web-based
examination system, with emphasis on the examination process and the question bank
feature. This was to provide the foundation on which an intelligent question bank and
examination system is proposed. Next, the different approaches used in existing web-base
examination systems were presented, focusing on the examination and question bank
modules/features. Apart from this, the mechanisms, assessment approaches/procedures
that were used in these systems were also discussed.
Subsequently, by integrating the mechanisms and assessment approaches from the
discussed system, an enhancement approach for question bank and examination system is
suggested. Here, the functional concept graph/directed acyclic graph and the intelligence
questioning system approach method were discussed in detail.
In Chapter 3, the methodology used to develop the intelligent question bank and
examination system was presented. The benefits and justification of using the
Incremental Development Model to develop IQBAES was discussed.
In Chapter 4 and Chapter 5, the system analysis and design of IQBAES, a J2EE based
system was discussed. The flexibility of object-oriented design and the advantages of the
java programming language largely motivated the development of IQBAES. The
Page 223 of 252
IQBAES is an elegant, well-structured web-based framework and system that exploits
modern programming paradigm and GUI application design. The analysis and design
phase was modelled by Unified Modelling Language (UML) by using use cases, class
diagrams and packages to model the architecture of IQBAES.
The implementation and development of the IQBAES system was detailed in Chapter 6.
The framework serves as a platform for IQBAES to be developed on top of it. The
IQBAES provides a mechanism for evaluating the question difficulty, an automatic
question generator which automates the question generation process and intelligent
questioning system to direct the learning process based on student’s degree of
understanding of individual concept. The entire development process provided insight to
the detail working of the various open-source technologies and component of the java
programming language.
Chapter 7 described the evaluation methods used to obtain results regarding the proposed
system. Using the developed IQBAES system, the perceived learning, perceived fairness,
satisfaction, recommendation of the proposed system over existing system was
demonstrated through the obtained results, where improvements in precision were
observed.
8.2
Research Goals and Contributions
As mentioned above, the research goal of this thesis is to develop a framework and
subsequently to develop the question bank and examination system, IQBAES, on top of
this framework. The IQBAES system should be able to determine question’s difficulty
level, supports dynamic question creation from question bank and identify student’s level
of understanding with the assessment via the system. To achieve this research goal, four
main objectives were identified.
i.
To define a framework for web-based application development and develop a
question bank and examination system on top of the framework.
Page 224 of 252
This was addressed by developing a framework for web-based application. The
IQBAES system is then built with the architecture of this framework and with the
reusable components from the base class libraries. The same set of libraries
classes can be reused for other web-based application.
ii.
To develop a system that will define questions’ level of difficulty based on
functional concept graph (Conceptual Graph Structure).
This was addressed by developing the question difficulty assessment algorithm
with the use of functional concept graph. The assessment algorithm can
dynamically determine question’s difficulty level based on the input
parameters/conditions specified by lecturer.
iii.
To develop a system that serve as question bank for storing and creation of
examination questions dynamically.
This was addressed by creating databases for storing the examination questions. A
question generator was also developed. Primarily, the question generator will pull
the question from question management system and subsequently based on the
push and pull mechanism of queue concept to fetch and distribute the respective
question to student.
iv.
To develop a system which will provide fair assessment method for online
examination system.
This was addressed by developing the assessment module. The assessment
module uses the weight of the question to provide the incremental score
contributed by the answer to this question. The “knowledge key” information
about each question is used by the system to determine which question to pose
next. The fairness of the assessment was validated with the empirical studies as
discussed in the section 7.1.5.
Page 225 of 252
In achieving the research goal, the main contributions of this research can be summarized
as:
The development of a framework for web-base application and the development
of an educational assessment system on top of this framework.
Define a relatively objective approach to assessing question difficulties in an
online examination environment. This approach can be automated with other
computer software.
Improve traditional educational process by changing passive browsing homepage
situation or static module which could not be adjusted to the learner abilities into
active searching for selecting adequate question/content. With this, it can identify
the level of the learner ability and increase the depth and breadth of learning.
Monitor/measure student’s understanding on the web to assure the learning
effectiveness.
8.3
Problems Encountered and its Solutions
Many problems were encountered in the development of IQBAES. Some are technical
problems of hardware and software. There are also human relationship problems in
implementing a project of this magnitude.
8.3.1
Difficulties in configuring Jcrontab
Jcrontab was identified as one of the plug-in component to update the examination status
(Jcrontab acts as a cronjob to activate an examination with the ‘New’ status to ‘Active’
status or update an examination with ‘Active’ status to ‘End’ status). However, I
encountered the problem of configuring the Jcrontab in IQBAES system and integrating
Jcrontab into tomcat Catalina application server. After an extensive try with the Jcrontab
configuration and yet still fail to get the Jcrontab configured properly, I decided to switch
from using Jcrontab to Java scheduler. However, there is a drawback of using Java
scheduler, as this scheduler will lose its state once the application rebound. Hence, I will
take this as one of the future enhancement point.
Page 226 of 252
8.3.2
Problem with user access rights
Windows has some user access rights strategies, which is distinct from other components,
such as the database servers. There were some difficulties in finding compatibility
between these access privilege requirements.
8.3.3
Insufficient documentation for open-source plug-in component
Although there is vast amount of open source plug-in components available on the
Internet, most of them have limited documentation. Lacks of documentation on the
features, usage and configuration for these plug-in components have narrowed down the
scope of using these freely available features. On top of that, lot of effort was spending
on R&D for the proper usage of these plug-in components.
8.3.4
Problem in mirroring development environment
There were many problems related to porting the development platform from home to
office. Mirroring a development platform involves attempting to create the same
development environment in two places, in this case because of the need to develop
sequentially between home and office. Some work has been done at home. This work had
to be saved and ported back to the office environment in such a way that the development
work continues from where it was stopped at home. When the work was again stopped in
the office, it has to be saved, and transferred back to the home. This carries on in a
sequential cycle in order to speed development. Many difficulties were encountered when
this process was first carried out. These problems included the different structure of the
development platform between the home development machine and office development
machine. In such a case, the office development machine normally will have user access
right control. This privileges control has restricted me from configuring the database
server when setting up the development environment in office.
8.4
Evaluation by End Users
IQBAES has been tested and evaluated by lecturers and students of Inti College
Malaysia, and it has been used fully to process the examination results. A Questionnaire
Page 227 of 252
was given to those users directly involved with using the system to enter the examination
marks. The questionnaire was directed at just the user interface and implementation, as
they are not technically competent to comment on the architecture and design of the
system. The actual questionnaire is appended in Appendix B, together with the detailed
charts on each questionnaire item.
8.5
System Constraints and Future Enhancement
8.5.1
System constraints
The system was being implemented on the ICM Intranet. As there is no domain register
for IQBAES currently and there is no production server hosting the IQBAES system, it is
not possible for the information to be published over the Internet as a whole. However,
theoretically, the same system can work over the Internet once a domain is registered and
permanent ISP link can be established, and the application server configured for this
purpose is set up.
8.5.2
Future Enhancement
8.5.2.1
Integration with other existing system
Apart from Question Bank and Examination System, the system could be broadened to
include student registration module, student enrolment module, financial module,
timetabling module and etc. On top of that, if there is any knowledge base system
available in future, it can also be integrated into IQBAES to enhance its features.
8.5.2.2
Support multiple application server
Currently, IQBAES only integrated and tested with Tomcat Catalina application server.
This framework can be enhanced to support other application server like Jboss, JRunner,
Weblogic, Websphere and other application server. These can ultimately enhancement
the robustness of this framework.
Page 228 of 252
8.5.2.3
Support multiple databases
Apart from supporting MySQL database, IQBAES can be enhanced to support multiple
databases like Oracle, Informix, Microsoft SQL Server, Sybase, DB2 and etc.
8.5.2.4
Support multiple web browser
IQBAES can be improved to support multiple web browsers on top of Internet Explorer
and Netscape, which have been tested with. These browsers include Opera, Morziall
FireFox and miscellaneous web browser.
8.5.2.5
Support multiple device
IQBAES framework can be enhanced to support multiple devices like PDA, Handheld
terminal and Smart Phone.
8.6
Knowledge and Experience Gained
Working on this project has been a very rewarding experience. Even writing this report is
a joy, as it is wished that there were more time to write this report. Much of the
knowledge gain is in the area of the use of new technology. These technologies include:
8.6.1
Hibernate
Hibernate is a new database access technology that has improved very much over the
traditional database access technology. In software development, major portion of the
development of an application involves the creation and maintenance of the persistence
layer used to store and retrieve objects from the database. If changes are made to the
underlying database schema, it can be expensive to propagate those changes to the rest of
the application. Hibernate technology was invented to fill this gap by providing a
powerful object-relational persistence framework for Java applications. Hibernate
provides support for collections and object relations, as well as composite types. In
addition to persisting objects, Hibernate provides a rich query language to retrieve objects
Page 229 of 252
from the database, provides efficient caching layer and Java Management Extensions
(JMX) support.
8.6.2
Apache Ant
Apache Ant was invented to help developers in compiling and deploying their application
to the application server. In short, it is a tool for building java projects. It is similar to the
make command, but is Java-centric and has more modern features. Ant has become the
standard build tool for Java, and many commercial integrated development environments
provide some support for using ant build files.
8.6.3
JfreeChart
JFreeChart is a free Java chart library that supports pie charts (2D and 3D), bar charts
(horizontal and vertical, regular and stacked), line charts, scatter plots, time series charts,
high-low-open-close
charts,
candlestick
plots,
Gantt
charts,
combined
plots,
thermometers, dials and more. JFreeChart can be used in applications, applets, servlets
and JSP. In IQBAES, JFreeChart was used to generate the bar chart for examination
summary report.
8.6.4
Log4j
Log4j is technology that enables logging at runtime without modifying the application
binary. Logging behaviour can be controlled by editing a configuration file, without
touching the application binary. One of the features of log4j is the notion of inheritance in
loggers. With logger, it is possible to control which log statements are output at certain
granularity. This helps to reduce the volume of logged output and minimize the cost of
logging. The target of the log output can be a file, an OutputStream, a java.io.Writer, a
remote log4j server, a remote Unix Syslog daemon, or even a NT Event logger.
Page 230 of 252
8.7
Conclusion
This project is a real implementation in a campus environment. The project has lightened
the load of all lecturers. It has provided excellent, timely and useful functionality for its
users. The project has benefited from the vast experience gained in writing software to
handle these functionalities to solve one major assessment task in the university, that is,
the process of examination. Developing this system has increased my knowledge of J2EE
framework, open source technology and many aspects of programming in Java.
Page 231 of 252
9
APPENDIX A – LIST OF REFERENCES
Budi Kurniawan. 2002, Java for the Web with Servlets, JSP and EJB, New Riders,
Indianapolis.
Coad, R, and Yourdon, E 1991, Object-oriented Design, 2nd edn, Yourdon Press,
Englewood Cliffs, NJ.
Hoffer, Jeffrey A, George, Joey F, Valacich & Joseph S 2002, Modern Systems Analysis
& Design, 3rd edn, Prentice Hall, Upper Saddle River, N.J.
Marty Hall 2000, Core Servlets And Java Server Pages, Prentice Hall, United States of
America.
Patrick Naughton & Herbert Schildt 1999, Java 2: The Complete Reference, 3rd edn,
McGraw-Hill, United States of America.
Paul Tremblett 2000, Instant JavaServer Pages, McGraw-Hill, New York.
Simon Bennett, Steve McRobb & Ray Farmer 2002, Object-Oriented Systems Analysis
And Design Using UML, McGraw-Hill, British.
Cackowski, David Sep 2000, ‘Object analysis in organizational design: A solution for
matrix organizations’, Project Management Journal, Sylva, Vol. 31, Iss. 3; pg. 44, 8 pgs
Government of Malaysia. 1997, The Malaysian Smart School: A Conceptual Blueprint,
Kuala Lumpur: Ministry of Education.
Government of Malaysia. 1997, Concept Request for Proposal for Smart School
Management System, Kuala Lumpur: Ministry of Education.
MIND 2001 – 2002, MothersonSumi INfotech and Designs Ltd, Nioda - 201 301, India,
viewed 15 October 2004, <http://www.mind-infotech.com/case_studies/oes.htm>.
Page 232 of 252
Assessment Systems Corporation 2005, University Ave, MN, USA, viewed 16 October
2004, <http://www.assess.com/Software/Exam_Manager.htm>.
Inderjeet Singh, Beth Stearns, Mark Johnson, and the Enterprise Team 2002, Designing
Enterprise Applications with the J2EETM Platform, Second Edition, Sun Microsystems Inc,
California, United States, viewed 20 Oct 2004,
<http://java.sun.com/blueprints/guidelines/designing_enterprise_applications_2e/introduct
ion/introduction3.html#1041147>.
The Apache Software Foundation 2000 - 2005, The Apache Software Foundation, Munsey
Drive, Forest Hill, United States, viewed 23 Nov 2004,
<http://jakarta.apache.org/struts/>.
Fort Worth Java User's Group, Introduction to the Jakarta Struts Framework, IBM New
York, United States, viewed 15 Jan 2005,
<http://www.fwjug.org/fwjug/resources/topics/p200210.ppt>.
Page 233 of 252
10
APPENDIX B - QUESTIONAIRE
Questionnaires
Please answer the following questions according to the given scales.
Scales:
1 – Strongly Agree, 2 – Agree, 3 – Undecided, 4 – Disagree, 5 – Strongly Disagree
1. The traditional manual system of conducting
examination is the most convenient.
1
2
3
4
5
2. The traditional manual system of conducting
examination is faster.
1
2
3
4
5
3. We should stick to the old traditional manual
system.
1
2
3
4
5
4. The old online examination system works efficient
enough.
1
2
3
4
5
5. The old online examination system looks more
familiar to the usual way of conducting
examination.
1
2
3
4
5
6. The old online examination system is good enough.
1
2
3
4
5
7. The new online examination system is more
convenient to use.
1
2
3
4
5
8. The new online examination system saves more
time.
1
2
3
4
5
9. The new online examination system is more user
friendly.
1
2
3
4
5
10. It takes a shorter period of time to conduct exams
in the new online examination system.
1
2
3
4
5
Page 234 of 252
11. The new online examination system is more
familiar to the Web surfing applications.
1
2
3
4
5
12. The new online examination system should be
implemented to replace the older system.
1
2
3
4
5
13. The new online examination system is fair and
efficient to use.
14. The new online examination system is more
colorful and pleasing to the eyes.
1
2
3
4
5
1
2
3
4
5
15. We must use the new online examination system
and implemented as soon as possible.
1
2
3
4
5
16. The new online examination system has more
useful features than the old system.
1
2
3
4
5
17. The new system should be used in all higher
education institutions.
1
2
3
4
5
Page 235 of 252
11
APPENDIX C – IQBAES SCREEN SHOT
Figure 53 – IQBAES Login Page
Figure above illustrates common login page for System Administrator, Lecturer and
Student into IQBAES.
Figure 54 – System Administrator Main Page
Figure above illustrates the System Administrator main page after login to IQBAES.
Page 236 of 252
Figure 55 – User Role Listing Page
Figure above describes the list of users with their respective role in IQBAES.
Figure 56 – User Role Maintenance Page
Figure above describes the page that is used to maintain user role in IQBAES.
Page 237 of 252
Figure 57 – User Profile Listing Page
Figure above describes the list of users created in IQBAES.
Figure 58 – General Maintenance Page
Figure above describes the menu for accessing general maintenance pages in IQBAES.
Page 238 of 252
Figure 59 – Chapter Maintenance Page
Figure above illustrates the chapter maintenance page for courses registed in IQBAES.
Figure 60 – Examination Listing Page
Figure above illustrates the list of examination created by lecturer in IQBAES.
Page 239 of 252
Figure 61 – Examination Maintenance Page
Figure above describes the examination maintenance screen in IQBAES.
Figure 62 – Question Bank Listing By Chapter Page
Figure above describes the number of questions group by chapter for a particular course
currently available in question bank.
Page 240 of 252
Figure 63 – List of Examination Questions in Question Bank
Figure above describes the list of examination questions in IQBAES question bank.
Figure 64 – Examination Question Detail Page
Figure above illustrates the question detail for the selected examination question.
Page 241 of 252
Figure 65 – Examination Question Update Page
Figure above describes the update screen for the selected examination question.
Figure 66 – Student Course Enrolment Page
Figure above illustrates the list of students that enroll in particular course.
Page 242 of 252
Figure 67 – Student Action Bulletin Page
Figure above describes the active examination listing in student’s action bulletin.
Figure 68 – Examination Main Page
Figure above illustrates the examination main page.
Page 243 of 252
Figure 69 – Examination Page
Figure above describes the examination question page.
Figure 70 – Student Exam Summary Page
Figure above describes the examination summary page at the end of an examination.
Page 244 of 252
Figure 71 – Student Exam Summary Bar Chart
Figure above describes the number of questions answered correctly versus wrongly group
by level of difficulty at the end of student examination.
Figure 72 – Student Grade Book Page
Figure above describes the total attempts for questions in respective chapters that students
have answered during the examination.
Page 245 of 252
Figure 73 – Student Detail Grade Book Page
Figure above shows the number of students pass / fail in a particular question.
Figure 74 – Student’s Examination History Page
Figure above shows the student’s examination history page for the list of examinations
that they have taken.
Page 246 of 252
Figure 75 – Student’s Exam Question History Listing Page
Figure above shows the list of questions that student had answered in a particular
examination.
Figure 76 – Question Analysis Report Page
Figure above shows the percentage of student responses to each choice.
Page 247 of 252
Figure 77 – Student’s Exam Question History Detail Page
Figure above shows the student’s answer for a particular question in the examination.
Figure 78 – Examination Summary Report Page
Figure above shows the student’s performance for a particular course in the examination
Page 248 of 252
12
APPENDIX D – USER REGISTRATION FLOW
System
Administrator
Lecturer
IQBAES
Student
IQBAES user
Registration
Process
Receipt list of
IQBAES user
(lecturer and student)
Registration of User
System captures the
user details
System generate a
userId and password
base on criteria
provides by
administrator
Receive List of users
to be added to
IQBAES
Verify list of users
with their staff ID and
student ID
Accept users
conforming to term &
condition
Is Registration
complete?
Create User
(Lecturer/ Student)
No
Enter user details in
the system based on
the copy of user
details received
Notification Processing
System routes
incomplete
registration to
administrator
Create User Group / Role
(Lecturer / Student)
Yes
User Role Creation
Assign user to
respective user role
System captures the
user details
System generate a
user role
Is role creation
complete?
No
Yes
Send Notification
Send notification to
lecturer/student on
successful
registration
Page 249 of 252
13
APPENDIX E – QUESTION BANK FLOW
System
Administrator
Lecturer
IQBAES
Student
Question Bank
Creation
Process
Prepare list of
exam question
Prepare list of exam
questions to be
added to IQBAES
Prepare
miscellaneous
material if any
(graphic, chart,
graph, audio, video
and etc)
Chapter selection
Is chapter
exists?
No
select relevant
chapter on which
question will be
created
No
Chapter Creation
Create Chapter
System captures the
chapter details
System generate a
chapter base of
selected course
Enter chapter details
in the system based
on the program and
course details
No
Create Question
Enter question details
in the system based
on the copy of
questions details
prepared earlier
Is chapter creation
success?
Yes
Yes
Chapter Creation
System captures &
generate the question
System keeps
question into question
bank
Page 250 of 252
14
APPENDIX F – EXAMINATION CREATION FLOW
System
Administrator
Lecturer
IQBAES
Student
Exam Creation
Process
Exam Creation
System captures the
exam details
System generates
exam with status new
in IQBAES
Prepare exam
Prepare list of exam
to be created to
IQBAES
Is exam creation
success?
Create exam
Enter exam details in
the system based on
program, course and
semester.
Provide exam date
and time and no of
questions for the
exam
Update exam status
as new
Yes
Scheduler creation
No
System scheduler will
be created in order to
activate the exam
according to the time
and date captured in
system
Receive Notification
Student will receive
notification about
current active exam
on their action
bulletin board
Student can click on
the exam link and
proceed to take the
exam.
Is exam date reach?
No
Scheduler standby
Yes
System scheduler will
be running in
background
System scheduler will
update exam status
to active when date/
time reached.
Facilitate Update &
Acknowledgement
System scheduler will
facilitate exam status
update
System will
acknowledge student
about active exam on
their action bulletin
Page 251 of 252
15
APPENDIX G – EXAMINATION FLOW
System
Administrator
Lecturer
IQBAES
Student
Provide list of questions
System provides list
of random questions
base on initial level of
difficulty
Route the question to
student
Exam
Process
Receive Notification
Student receive
notification about
current active exam
on their action
bulletin board
Select exam and
proceed with
examination.
Grading
System performs
grading on the
questions
Receive question
Is answer correct?
Student receives
question
Student select
answer for the
question and submit
Yes
Increase Level Of
Difficulty
No
System will increase
level of difficulty
Level Of Difficulty
Adjustment
System will decrease
level of difficulty base
on current question
difficulty level
Is last question?
No
Provide new question
System provides new
question base on
new level of difficulty
Route question to
student
Last question grading
System performs
grading on last
question
System route exam
summary result to
student
Yes
Receive exam summary
Student receives
exam result
Student can view
their summary result
in bar chart
Page 252 of 252