Experimental Investigation of
Effectiveness of Logical
Expressions Using t-way Testing
RANJAN BHAMBROO
Introduction
 Experimental Evaluation of logical expressions.
 Same idea but continued on new path.
 T – way testing instead of pair-wise testing.
 New tool for generating logical expressions.
 New tool from NIST to construct t-way test sets.
T-way Testing
 T-way
 Pair – wise = 2-way
 T-way where t=2,3,4,5,6.
 T- way more effective than pairwise.
 But test cases not enough need more test cases.
T-way Testing
 Example:
 For 15 variables total number of combinations = 215
 215 = 32768 combinations
T-way values
Test Cases
2-way
10
3-way
24
4-way
58
5-way
131
6-way
296
Tools Used
 Boolean Expression Generator – ECU( Developed by
Ranjan Bhambroo)
 Advanced Combinatorial Testing System(ACTS) for
constructing t-way combinatorial test sets – NIST
 Fault Evaluator – ECU
Boolean Expression Generator(New Tool)
 Generates Logical Expressions Automatically.
 Simple and user friendly interface.
 Different Modes to generate Expressions:
a)Random Number of variables.
b)Fixed Number of variables.
c)Interval of variables
 Can Control complexity of expressions.
 Allows user to save expressions in a text file.
Experimental Results Of T-way Testing
Boolean
Expression
Generator
Fault Evaluator
Advanced
Combinatorial
Testing System

Effectiveness
Experimental Results Of T-way Testing
Note Using Pair Wise Test Cases
10
expressions/set
VNF
ORF
VRF
(ENF)
All Fault Types
(weighted avg.)
ASF
Set1
36.25 45.58 24.96
60.27
74.51
30.34
Set2
24.67 39.57
17.9
46.55
54.84
22.32
Set3
28.57 43.18 19.54
45.78
75
24.54
Set4
28.06 46.99 21.69
55.95
62.79
26.96
Set5
30.67 39.86 20.38
62.07
60.98
25.3
29.644 43.036 20.894 54.124
65.624
25.892
7.909
2.678
Average
Std. Deviation
3.823 2.974 2.376
6.801
Experimental Results Of T-way Testing
Note Using 3-Wise Test Cases
10
expressions/set
VNF
Set1
40.88 48.03 27.61 63.93
80
33.27
Set2
26.39 42.54 20.62 48.21 72.41
25.22
Set3
28.97 45.19 22.04 53.23 71.88
26.88
Set4
32.24
59.7 76.47
31.52
Set5
36.22 53.85 29.12 68.75 86.67
34.47
Average
32.94 48.344 25.082 58.764 77.486
30.272
Std. Deviation
5.157 4.201 3.248 7.339 5.457
3.611
ORF
VRF (ENF) ASF
52.11 26.02
All Fault Types
(weighted avg.)
Experimental Results Of T-way Testing
Note Using 4-Wise Test Cases
10
expressions/set
VNF
Set1
43.07 53.54
36.11
63.93
95
40.79
Set2
33.33 51.49
27.32
55.36
68.97
31.96
Set3
39.31 54.81
30.16
58.06
93.75
35.49
Set4
34.87 52.82
30.46
62.69
88.24
35.55
Set5
50.39 66.67
41.66
79.17
96.67
47.07
40.194 55.866 33.142 63.842 88.526
38.172
Average
Std. Deviation
ORF
6.14 5.508
VRF
5.125
(ENF)
8.266
ASF
10.179
All Fault Types
(weighted avg.)
5.267
Experimental Results Of T-way Testing
Note Using 5-Wise Test Cases
10
expressions/set
VNF
Set1
47.45 55.91
39.25
67.21
97.5
43.97
Set2
38.19
59.7
33.84
62.5
89.66
38.77
Set3
39.73 57.04
31.57
58.06
96.88
36.69
Set4
42.76 69.86
36.35
64.18
94.12
41.55
Set5
50.39 70.94
45.05
81.25
96.67
50.11
43.704 62.69 37.212
66.64
94.966
42.218
7.881
2.893
4.653
Average
Std. Deviation
ORF
4.601 6.423
VRF
4.68
(ENF)
ASF
All Fault Types
(weighted avg.)
Experimental Results Of T-way Testing
Note Using 6-Wise Test Cases
10
expressions/set
VNF
Set1
48.18 55.91
42.4
67.21
97.5
46.44
Set2
41.67 60.45 39.38
62.6
100
43.56
Set3
42.07 60.74 36.79
59.68
96.88
41.43
Set4
40.79 60.56 37.29
65.67
100
42.31
Set5
53.54 70.94 48.73
81.25
100
53.29
Average
45.25 61.72 40.918 67.282 98.876
45.406
Std. Deviation
4.902 4.953 4.377
ORF
VRF
(ENF)
7.449
ASF
1.391
All Fault Types
(weighted avg.)
4.29
Experimental Results Of T-way Testing
 2-way
 3-way
Experimental Results Of T-way Testing
 4-way
120
 5-way
120
100
100
80
80
60
60
40
20
0
40
20
0
Experimental Results Of T-way Testing
120
100
80
60
40
20
0
Set1
Set2
Set3
Set4
Set5
Average
Std.
Deviation
THANK YOU