Design a personalized e-learning system
based on item response theory and artificial neural
network approach
Ahmad Baylari, Gh.A. Montazer
IT Engineering Department, School of Engineering, Tarbiat Modares University,
Tehran, Iran
Expert Systems with Applications, Volume 36, Issue 4, May 2009, Pages 8013-8021
Reporter：Yu Chih Lin
Outline

Introduction

Item response theory

Test construction process

System architecture

System evaluation

Conclusion
Introduction

In web-based educational systems the structure of learning domain and
content are presented in the static way

Without taking into

account the learners’ goals

experiences

existing knowledge

ability

interactivity
Introduction

Personalization and interactivity will increase the quality of learning

This paper proposes a personalized multi-agent e-learning system

Item response theory (IRT)
•

Presents adaptive tests
Artificial neural network (ANN)
•
Personalized recommendations
Item response theory

Item response theory (IRT) was first introduced to provide a formal
approach to adaptive testing

Three common models for ICC

One parameter logistic mode(1PL)

Two parameter logistic model(2PL)

Three parameter logistic model (3PL)
Item response theory

1PL models

i : only one parameter

bi ：item difficulty

D：1.7

𝜃： ability scale
𝑃𝑖 ( 𝜃) =
1
1+e(−D(𝜃−𝑏𝑖 ))
Item response theory

2PL model
•
𝑎𝑖 : discrimination degree, added into the item characteristic function
•
bi : item difficulty
•
D : 1.7
•
𝜃 : ability scale
𝑃𝑖 ( 𝜃) =
1
1 + 𝑒 (−𝑎𝑖 𝐷(𝜃−𝑏𝑖 ))
Item response theory

3PL model

𝑐𝑖 : guess degree to the 2PL model

Potential guess behavior of examinees
𝑃𝑖 ( 𝜃) = 𝑐𝑖 + frac1 − 𝑐𝑖 1 + e(−𝑎𝑖D(𝜃−𝑏𝑖 ))
Item response theory

An estimation method called maximum likelihood estimator (MLE)
•
Effectively estimate item parameters and examinee’s abilities

𝑃𝑖(𝜃) : learner can answer the i th item correctly

Q𝑖(𝜃) : learner cannot answer the ith item correctly(1 - 𝑃𝑖(𝜃) )

ui : 1 (correct answer) or 0 (incorrect answer)
𝑛
𝐿 𝜃|𝑢1 , 𝑢2 , … . . 𝑢𝑛
𝑛
𝑃𝑖 (𝜃)𝑢𝑖 𝑄𝑖 (𝜃)1−𝑢𝑖
=
𝑖=1
Item response theory

Item information function (IIF) is the subject of the amount of information


Effectively distinguish between subjects potential ability to reduce the estimation
error
Test information function (TIF)
•
Sum of the amount of information the test results for each subject
Item response theory

IIF&TIF：
(IIF)
(TIF)

P’(θ) is the first derivative of Pi(θ) and Qi(θ) = 1 - Pi(θ)

I (θ) : amount of information for item,1~N
Test construction process

Three types of tests
•
pre-test
•
post-test
•
review tests(延後測)

All of these tests have 10 items

Use IRT-3PL model to test construction
•
appropriate post-test selection for learners
Test construction process

For posttest construction
System architecture

Proposed a personalized multi-agent e-learning system

Middle layer contains four agents

Activity agent :
records e-learning activities

Planning agent :
agent plans the learning process

Test agent :
based on the requests of planning agent , presents appropriate test
type to the learner

Remediation agent :
analyzes the results of review tests, and diagnoses learner’s
learning problems
System architecture

System architecture
System design and development

Collect stutdents’s responses and tests
•
•
Diagnose their learning problems
Recommend them appropriate learning materials

Maximum number of recommended LOs were five

Large number of responding states to a test
•
210 states for each test
System design and development

Experiment the remediation agent

Essentials of information technology management course
•
Divided into several Los
•
A few codes were allocated for all Los
System design and development

I1 to I10 columns are item codes

R1 to R10 are corresponding responses which code

1 : correct response

0 : incorrect response
System design and development

Use a back-propagation network(BPNN)
•
Learning Data

Use 20 input nodes

Use 5 output neurons
System design and development

Use items responses data as input data the neural network

Output neurons are recommended LOs
System design and development


Normalization of data within a uniform range 0–1

Prevent larger numbers from overriding smaller ones

Prevent premature saturation of hidden nodes
No one standard procedure

Input

Output
System design and development

Scale input and output variables (xi) in interval

𝑋𝑛 : normalized value of 𝑋𝑖 , 𝑋𝑖𝑚𝑖𝑛 , 𝑋𝑖𝑚𝑎𝑥

𝜆1 , 𝜆2
Normalized the input and output data , range 0.1 ~ 0.9
𝑋𝑛 = 𝜆1 + 𝜆2 − 𝜆1
𝑋𝑖 − 𝑋𝑖𝑚𝑖𝑛
𝑋𝑖𝑚𝑎𝑥 − 𝑋𝑖𝑚𝑖𝑛
System design and development

ANN requires partitioning of the parent database into three subsets

Training

Test

Validation

Training used 60% of all data

Validation 10% for data

Remaining data for testing the network
System design and development

Use one or two hidden layer

Trained with various neurons in each layer in MATLAB software

In hidden layer
•

Use sigmoid function as activation function
For output neurons

First use linear activation function

Second use sigmoid activation function
System design and development


Two different criteria used to stop training
•
Training error(MSE) ：10−4
•
Maximum epoch： 1000 epochs
Training back-propagation network
•
•
With a random set of connection weights (weight initialization)
Train with different architectures
System design and development

For example in training a network

15 neurons in one hidden layer
•

Output layer neurons
•

With sigmoid activation function
With linear activation function
MSE error 7.13
•
System design and development


For example in training a network
15 neurons in one hidden layer and 10 neurons in second hidden layer
•

Output layer neurons
•

With sigmoid activation function
With linear activation function
MSE error 0.0158
System design and development


Summarizes the results of trained networks with different architectures
Network configuration 20-20-5 (network No. 11)
System evaluation

Recommended LOs from the network compared with recommended LOs
from a human instructor

Output was exactly the same as the target output, 25 of 30 tests (83.3%)
Conclusion

Proposed a personalized multi-agent e-learning system

Estimate learner’s ability using item response theory

Diagnose learner’s learning problems


Recommend appropriate learning materials to the learner
Neural network approach to learning material recommendation