Application of Automatically
Constructed Concept Map of
Learning to Conceptual
Diagnosis of e-learning
Expert Systems with Application 36 (2009)
Chun-Hsiung Lee, Gwo-Guang Lee, Yungho Leu
Presenter : Liew Keng Hou
LOG
O
Outline
Introduction
Purpose in This Study
Research Approach
Experiment and Data Analysis
𝑇- test & Analysis
Conclusion and Discussion
2
What is Concept Map?
A
B
Epistemological order of concept map
3
Types Concept Map for Learning
Completely manual
Semi-automatic
Automatic
4
Outline
Introduction
Purpose in This Study
Research Approach
Experiment and Data Analysis
𝑇- test & Analysis
Conclusion and Discussion
5
Purpose in This Study
1. Develop the intelligent Concept Diagnostic
System(ICDS) of an automatically constructed
concept map of learning by the algorithm of Apriori
for Concept Map
2. Teachers were provided with the constructed concept
map of learners to diagnose the learning barriers
and misconception of learners.
3. Remedial-Instruction Path(RIP) was constructed
through the analyst of the concepts and weight in
the concept map to offer remedial learning.
4. Statistical methods were used to analyze whether
the learning performance of learners can be
significantly enhanced after they have been guided
by the RIP.
6
Flow Chart of Concept Diagnosis
7
Remedial-Instruction Path
A
Remedial-Instruction Path
B
C
D
E
Relationships of the epistemological order
8
Outline
Introduction
Purpose in This Study
Research Approach
Experiment and Data Analysis
𝑇- test & Analysis
Conclusion and Discussion
9
Presetting Conceptual Weight
Question
Concept
C1
C2
C3
C4
C5
Q1
1
0
0
0
0
Q2
0
1
0
0
0
Q3
0.5
0
0.5
0
0
Q4
0.3
0.4
0
0.3
0
Q5
0
0
0
0
1
‘0’: not relevant
‘1’: strongly relevant
10
Recording Test Portfolio of Testees
Question Testees
C1
C2
C3
C4
C5
Total
Q1
1
1
1
0
0
3
Q2
1
1
1
1
0
4
Q3
1
1
1
1
1
5
Q4
0
0
1
1
1
3
Q5
0
0
0
1
1
2
‘0’: Student answered correctly the test item
‘1’: Student failed to answer correctly the test item
11
Find Out All Large Item sets
Comparison Chart
Itemset
S1
Q1,Q2,Q3
Q1
3
S2
Q1,Q2,Q3
Q2
4
S3
Q1,Q2,Q3,Q4
Q3
5
S4
Q2, Q3, Q4, Q5
Q4
3
S5
Q3, Q4, Q5
Q5
2
12
No. of supports
Find Out All Large Item sets(Cont.)
Using association rules of data mining
Sets Min support(MS) = 0.4
(Depends on teacher)
Number of testees = 5
Questions with wrong answers given by
testees has to be ≥ MS x N
(0.4 x 5 = 2)
13
Find Out All Large Item sets(Cont.)
Itemset
Q1, Q2
Q1, Q3
Q1, Q4
Q2, Q3
Q2, Q4
Q3, Q4
Q4, Q5
No. of
support
Itemset
3
3
1
MS ≥ 0.4
0.4 x 5 ≥ 2
4
2
3
2
14
No. of
support
Q1, Q2
3
Q1, Q3
3
Q2, Q3
4
Q2, Q4
2
Q3, Q4
3
Q4, Q5
2
Ruling the Test Question Association
 The confidence level of the test question
association rule Q𝑖 → Q𝑗 is the concept of
conditional probability.
 It implies that a testee gives a wrong answer to
Question Q𝑖 , there is a probability for the testee
to give a wrong answer to Question Q𝑗, too
 The estimated confidence level formula is
15
Using Association Rules of Data Mining
Question Testees
C1
C2
C3
C4
C5
Total
Q1
1
1
1
0
0
3
Q2
1
1
1
1
0
4
Q3
3
1Confidence(Q1
1
1
13 = 100%
5
Confidence(Q1
→1Q2)=P(Q2|Q1)=
→ Q2)=P(Q1|Q2)=
= 75%
3
Q4
0
Q5
0
0
1 → Q2)=P(Q3|Q2)=
1
1
Confidence(Q3
0
0
1
1
Note: ‘0’: Student answered correctly the test item
‘1’: Student failed to answer correctly the test item
16
4
4
5
3
= 80%
2
Using Association Rules of Data Mining
Let the minimum confidence(MC) level be
below 70%
Rule 1. Confidence (Q1 → Q2) = 100%
Rule 2. Confidence (Q1 → Q3) = 100%
Rule 3. Confidence (Q2 → Q1) = 75%
Rule 4. Confidence (Q2 → Q3) = 100%
Rule 5. Confidence (Q3 → Q2) = 80%
Rule 6. Confidence (Q4 → Q3) = 100%
Rule 7. Confidence (Q5 → Q4) = 100%
17
Relationship Between Concept and Concept
 Conversion from “test question association rules”
to the effect of “relation between concept and
concept”
 Q𝑖: 𝑖th test question
C𝑥: 𝑥th concept
RQ𝑖C𝑥 : relavance degree between Q𝑖 and C𝑥
WC𝑥C𝑦: relevance degree between C𝑥 and X𝑦
18
Relationship
Between Concept and Concept
Question Concept
Question
1)
2)
3)
4)
5)
Concept
C1
C2
C3
C4
C1
C2
C3
C4
Q1 → Q2 → 𝑊𝐶1𝐶2 = C1 →C2 = Confidence
1 = 1  10  1 = 1 0
0
(Q1 → Q2) Q1RQ1C1  RQ2C2
Q1
1
0
0
0
1
0
0
Q1 → Q3Q2
→ Q2
𝑊𝐶1𝐶3 = 0
C1 0→C3 =1Confidence
0
0
(Q1 → Q3) Q3RQ1C1  RQ3C3
0.5 = 1  10  0.5 = 0.5
0.5
0
Q3
0.5
0
0.5
0
Q2 → Q1 → Q4
𝑊𝐶2𝐶1 = C20.3
→C1 = Confidence
0.4
0
0.3
Q4
0.3
0.4
0
0.3
Question
Concept
(Q2 → Q1)  RQ2C2  RQ1C1 = 0.75  1  1 = 0.75
Q5 Rule 2.
0 Confidence
0
0 Q1) = 100%
0
(Q3 →
Q5
0 C1
0 C2
0 C3
0 C4
C5
0
0
0
0
𝑊𝐶3𝐶2 = C3 →C2 = Confidence
0.4
0
(Q3 → Q2) Q4
 RQ3C3  R0.3
Q2C2 = 0.8  0.5  1 = 0.4
Q5
0
0
0
19
0
0
0
0
1
1 C5
Q2 → Q3 → 𝑊𝐶2𝐶1 = C2 →C1 = Confidence
(Q2 → Q3) Q1
 RQ2C2  RQ3C1
=10
1  0.5 = 0.5
1
0
0
Question Concept
(0.5 < 0.75(3))
Q2
0 →C3
1
0
0 100%
𝑊𝐶2𝐶3 Rule
= C2
= Confidence
Confidence
(Q2
=
C14.
C2
C3 → Q3) C4
(Q2 → Q3) Q3
 RQ2C2  R0.5
1  0.5 = 0.5
0.5
Q3C3 = 1  0
0
Q1
1
0
0
Q3 → Q2 →Q4
𝑊𝐶1𝐶2 = C1
0.3→C2 = Confidence
0.4
0
0
(Q3 → Q2) Q2
 RQ3C1  RQ2C2
= 0.8 1 0.5  1 =0 0.4
Q5
0
0
0
(0.4 < 1(1))
Q3
0.5
0
0.5
C5
0
0
0.3
0
0
0
0
C5
0
0
0
0
1
0
0.3
0
0
1
Relationship Between Concept and Concept
6)
Q4 → Q3 → 𝑊𝐶2𝐶3 = C2 →C3 = Confidence
(Q4 → Q3)  RQ4C2  RQ3C3 = 1  0.4  0.5 = 0.2
(0.2 < 0.5(4))
𝑊𝐶4𝐶3 = C4 →C3 = Confidence
(Q4 → Q3)  RQ4C4  RQ3C3 = 1  0.3  0.5 = 0.15
7)
Q5 → Q4 → 𝑊𝐶5𝐶1 = C5 →C1 = Confidence
(Q5 → Q4)  RQ5C5  RQ4C1 = 1  1  0.3 = 0.3
𝑊𝐶5𝐶2 = C5 →C2 = Confidence
(Q5 → Q4)  RQ5C5  RQ4C2 = 1  1  0.4 = 0.4
𝑊𝐶5𝐶4 = C5 →C1 = Confidence
(Q5 → Q4)  RQ5C5  RQ4C4 = 1  1  0.3 = 0.3
20
Preliminary Concept Maps(Stage 1)
C1
0.3
1
0.75
0.4
C2
C5
0.5
0.5
0.4
C3
0.3
0.2
C4
21
Preliminary Concept Maps (Cont.)
1)
Q1 → Q2 → 𝑾𝑪𝟏𝑪𝟐 = C1 →C2
= Confidence (Q1 → Q2)  RQ1C1  RQ2C2
=111=1
2)
Q1 → Q3 → 𝑾𝑪𝟏𝑪𝟑 = C1 →C3
= Confidence (Q1 → Q3)  RQ1C1  RQ3C3
= 1  1  0.5 = 0.5
3)
Q2 → Q1 → 𝑾𝑪𝟐𝑪𝟏 = C2 →C1
= Confidence (Q2 → Q1)  RQ2C2  RQ1C1
= 0.75  1  1 = 0.75
4)
Q2 → Q3 → 𝑾𝑪𝟐𝑪𝟏 = C2 →C1
= Confidence (Q2 → Q3)  RQ2C2  RQ3C1
= 1  1  0.5 = 0.5 (0.5 < 0.75(3))
Q2 → Q3 → 𝑾𝑪𝟐𝑪𝟑 = C2 →C3
= Confidence(Q2 → Q3)  RQ2C2  RQ3C3
= 1  1  0.5 = 0.5
5)
Q3 → Q2 → 𝑾𝑪𝟏𝑪𝟐 = C1 →C2
= Confidence (Q3 → Q2)  RQ3C1  RQ2C2
= 0.8  0.5  1 = 0.4 (0.4 < 1(1))
Q3 → Q2 → 𝑾𝑪𝟑𝑪𝟐 = C3 →C2 = Confidence
(Q3 → Q2)  RQ3C3  RQ2C2
= 0.8  0.5  1 = 0.4
6)
Q4 → Q3 → 𝑾𝑪𝟐𝑪𝟑 = C2 →C3
= Confidence (Q4 → Q3)  RQ4C2  RQ3C3
= 1  0.4  0.5 = 0.2 (0.2 < 0.5(4))
Q4 → Q3 → 𝑾𝑪𝟒𝑪𝟑 = C4 →C3
= Confidence (Q4 → Q3)  RQ4C4  RQ3C3
= 1  0.3  0.5 = 0.15
7)
Q5 → Q4 → 𝑾𝑪𝟓𝑪𝟏 = C5 →C1
= Confidence (Q5 → Q4)  RQ5C5  RQ4C1
= 1  1  0.3 = 0.3
Q5 → Q4 → 𝑾𝑪𝟓𝑪𝟐 = C5 →C2
= Confidence (Q5 → Q4)  RQ5C5  RQ4C2
= 1  1  0.4 = 0.4
Q5 → Q4 → 𝑾𝑪𝟓𝑪𝟒 = C5 →C1
= Confidence (Q5 → Q4)  RQ5C5  RQ4C4
= 1  1  0.3 = 0.3
22
Preliminary Concept Maps(Cont.)
C1
0.3
1
0.75
0.4
C2
C5
0.5
0.5
0.4
C3
0.3
0.2
C4
23
Adjusting Concept Map of Learning(Stage 2)
Child Concept
Parent concept
NP
Parent C𝑖
Child
NC
C1
C2
C3
C4
C5
C1
―
0
0
0
0.3
1
C2
1
―
0
0
0.4
2
C3
0.5
0.5
―
0.2
0
3
C4
0
0
0
―
0.3
1
C5
0
0
0
0
―
0
2
1
0
1
3
NP: Number of father concepts contained in the son concept
NC: Number of son concepts contained in the father concept
24
Complete Concept Map
C5
WC5C1 = 0.3
C1
WC1C2 = 1
WC1C3 = 0.5
WC5C4 = 0.3
WC5C2
= 0.4
C4
C2
WC2C3
= 0.5
WC4C3 = 0.2
C3
25
Determination of Learning barrier
Calculate the ratio of wrong answers given
in the test portfolio:
ER(C𝑗) =
𝑘 𝑒𝑘𝑗
𝑖 𝑒𝑖𝑗
 𝑒𝑘𝑗 : weight of the 𝑗th concept of the 𝑘th
test question which was wrongly
answer
𝑒𝑖𝑗 : weight of the 𝑗th concept in the whole
test paper
26
Table of Ratio of Wrong Answer (Failratio)
Question
Concept
C1
C2
C3
C4
C5
Q1
1
0
0
0
0
Q2
0
1
0
0
0
Q3
0.5
Q4
0.3
0.4
Q5
0
0
0
ER(C1)
(0.5+0.3)
0.5
0
=
0
1.8
0
0.3
(0+0.4)0
ER(C2)
=
=
0.44
1.4 1
0
0
= 0.29
𝑘
𝑖
𝑒𝑘𝑗
0.8
0.4
0.5
0.3
0
𝑒𝑖𝑗
1.8
1.4
0.5
0.3
1
0.44
0.29
1
1
0
ER(C𝑗)
27
Algorithm of Remedial-Instruction Path
010
020
030
040
050
060
070
080
090
100
110
120
130
140
150
160
170
180
190
Void main ()
Call Find_Remedial-Instruction_Path(k, Fault-Concept)
End
//Cj denotes the FaulConcept, and k denoted the index of a father
concept on Cj
Sub Find_Remedial-Instruction_Path(k,Cj)
//judge whether the failratio of Concept Cj is greater than the
tolerance for the ratio of the giving wrong answers.
If ER(Cj) failratio then
Push Cj
W = Max{WCiCjj1 5 i 5 n}
While (Cihi RootConcept)do //Not Find to Root-Concept
push Ci base on W
Wend
While Stack is not empty //Find to Root-Concept
//RIP: Remedial-Instruction_Path
RIP = Find_Remedial-Instruction_Path(i,Pop())
Wend
End if
End Sub
28
Intelligent concept diagnostic system(ICDS)
29
Intelligent concept diagnostic system(ICDS)
30
Outline
Introduction
Purpose in This Study
Research Approach
Experiment and Data Analysis
𝑇- test & Analysis
Conclusion and Discussion
31
Design of Experiment and Data Analysis
 Target of Study
 245 Grade 1 students of a senior high school
 Pre-test of “Visual Basic Programming Language”
 Table of discrimination index of Questions
Discrimination
Question
Q1
Q2
Q3
Q4
Q5
Q6
Q7
Q8
Q9
Q10
PH
0.93
0.93
0.93
0.93
0.93
0.71
0.86
0.64
0.14
0.86
PL
0.71
0.57
0.36
0.21
0.21
0.14
0.21
0.21
0
0.07
Discrimination Index
0.21
0.36
0.57
0.71
0.71
0.57
0.64
0.43
0.14
0.79
<0.2
32
Flow Chart of Data Analysis
33
Cluster
In order to understand the difference of
concept maps produced from the test
portfolio of students at different standards
Optimal ratio is 27% for the high-score
and low-score clusters
34
Sub-Cluster
1. Experimental group: The RIP in concept map
served as the learning guide
2. Control group: Traditional non-guided network
learning way was adopted
Group
Cluster
Cluster 1
(High-score
cluster)
Cluster 2
(Mediumscore cluster)
Cluster 3
(Low-score
cluster)
Experimental group
33
56
33
Control group
33
51
33
Number of students
66
113
66
35
Outline
Introduction
Purpose in This Study
Research Approach
Experiment and Data Analysis
𝑇- test & Analysis
Conclusion and Discussion
36
𝑇- test & Analysis
 : significant standard
H0 : There is a significant difference
between the mean of experimental
group and the mean of control group
If P-value < , then H0 is rejected.
If P-value ≥ , then H0 is not rejected
37
𝑇-test of Independent Samples of Experimental
Group and Control Group of Three Cluster
Cluster and group
High-score
cluster
Mediumscore cluster
Low-score
cluster
Item
Mean
Standard
deviation
Experimental group
72.57
15.83
Control group
67.86
12.97
Experimental group
52.25
11.52
Control group
40.13
10.23
Experimental group
37.29
9.59
Control group
19.29
8.62
*𝑝 < 0.1
*𝑝 < 0.01
38
𝑡-value
Significance
.610
.554
2409
.030*
3.695
.003**
Outline
Introduction
Purpose in This Study
Research Approach
Experiment and Data Analysis
𝑇- test & Analysis
Conclusion and Discussion
39
Conclusion and Discussion
1. Discrimination index of test questions
 If the test question is too simple or difficult?
2. Attribute of test questions
 Which type of test question?
3. Learning performance
 Which cluster(s) has better performance?
40
Thank You
for
Your Participating
LOG
O