International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org, editorijaiem@gmail.com
Volume 2, Issue 6, June 2013
ISSN 2319 - 4847
A Novel Approach of Calculating Affective Cost
in Cost Cognizant Regression Testing
Harsh Bhasin1, Satpal2 and Deepkiran Munjal3
1
Delhi Technological University, Delhi
2
BSAITM, Faridabad
3
BSAITM, Faridabad
ABSTRACT
The present methods of cost cognizant regression testing are based on the execution time of a test case. However, it has been
observed that in the testing of a professional enterprise resource planning system, a test case may take different time to run in
different testing cycles. This may be because of different number of concurrent processes running on the system or because of
the updates. The work proposes a method for the calculation of effective cost of a test suite in such situation. This would
facilitate the application of ‘cost cognizant regression testing’. Owing to the huge number of possible test cases, Genetic
Algorithms has been applied in order to achieve optimization.
Keywords: Regression testing, Cost Cognizant, APFD, Coefficient of Variation
1. INTRODUCTION
It is a known fact that software undergoes many changes in its life cycle. After every change the older test cases must be
re-executed to analyze the effect of changes on components developed in the older version. This exercise is referred as
regression testing and may consume a huge amount of time and resources. It is, therefore, desirable to minimize the test
suite or to order it in such a way that the important test cases are executed first. This mythology is referred to as
minimization. Minimization is one of the three techniques of regression testing. The other two being selection and
prioritization. Many techniques have been proposed in order to accomplish the above task. Cost Cognizant is one of
them. It takes into account the cost of each test case in order to determine the final ordering. However, the existing
techniques fail to handle the situation wherein many test cycle have been executed and a particular test case is taking
different time to execute in different cycles. This may happen due to different background processes running at that
time or the updates. The execution of the process and the amount of the RAM used also affect the running time of the
test case. This work presents a solution to handle such problems. The Novel and path-breaking technique will change
the way the cost factor is perceived as of now.
The work is a part of our endeavor to study various testing techniques [1-5] and test data generation techniques [69].The work is a continuation of earlier attempt in which time aware and cost cognizant regression testing techniques
were presented [11, 12]. The rest of the paper has been divided as follows: Section 2 of the paper examines genetic
algorithms. Section 3 throws light on applicability of genetic algorithms. Section 4 presents proposed work and section
5 presents results and conclusion.
2. GENETIC ALGORITHMS
Genetic algorithms are heuristic search processes which are based on the theory of natural selection. They are generally
applied when the population can be arranged on the basis of some value referred to as fitness value. It may be noted
that genetic algorithms are applied only when the search space is large. The steps of Simple Genetic Algorithm are as
follows.
Population Generation: A population of chromosomes is generated. A chromosome contains cells. The cells can be
binary or numeric. It may be noted that each cell represents some attribute.
Crossover: The crossover mingle the features of two chromosomes. There are many types of crossovers like one point
cross over, multi point cross over etc.
Mutation: The mutation operator breaks the local minima/maxima and helps to achieve optimization.
Selection: Selection is selecting the chromosome from amongst the population. This can be done by one of the many
techniques like roulette wheel selection.
It may be stated that in many experiments were carried out to find the crossover rate and the mutation rate at which
maximum optimization can be achieved [10, 13-19]. It was found the crossover rate of 2% and the mutation rate of
0.5% provides best results. However, it may also be stated that these rates just give an idea and not the exact value of
the rates. In the present work the effect of factors on the optimization was also analyzed.
Volume 2, Issue 6, June 2013
Page 202
International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org, editorijaiem@gmail.com
Volume 2, Issue 6, June 2013
ISSN 2319 - 4847
3. PROPOSED WORK
Since the cost of a particular test case is different in different cycles of testing, the affective cost of a test case needs to
be evaluated. In order to accomplish the task, the following steps must be carried out.
Step 1. Gather the cost of a test case in different cycles. The cost of the test cases can be gathered by the previous test
cycles carried out.
Step 2. Find the mean cost of each test case. The mean cost of text cases is given by Formula 1.
(1)
Step 3. Find the standard deviation of each test case. The formula of standard deviation is as follows.
(2)
Step 4. Divide the mean and the standard deviation to evaluate the coefficient of variation of each test case. The
coefficient of variation(CV) is given by the following formula.
(3)
Step 5. Evaluate the fitness function of a test suite. The fitness function of a test suite is the sum of finesses of each test
case.
Step 6. Arrange the test suites in order of their fitness function. The test suites can be created by using Genetic
Algorithms initial population.
Step 7. The new order of the test suite is the result, as per the technique.
The technique has been depicted in Figure 1.
Figure 1: Effective Cost Calculation
4. VERIFICATION
In order to verify the above technique, a Software develop by Sahib Soft has been taken. The Software is of around 3K
lines of code. The four cycles of Software testing have already been carried out. The following test suites are to be
prioritized using the proposed technique. Here, Ti represents ith test suite and TCj represents jth test case. The
following steps depict the implementation of the proposed technique.
First of all, an initial population is generated using the Genetic Tool. The number of test cases is 10.Therefore the no.
of cell in each chromosome of the genetic population should be 10.Table 1 depicts the initial Population generated by
the tool. Each Chromosome is then mapped against requisite test cases for example 1101000000 depicts test suite {T1,
T2, T3, T4}. Table 2 represents the running time of each test case in millisecond. On the basis of Step no. 3 effective
cost of each test case is calculated. It may be noted that at this point the cost is remains same in different cycles. Then
those values should be taken the final values. Otherwise the mean error and standard deviation should be calculated and
the statistically analysis should give the effective cost that can be used to calculate the fitness of each test case. On the
basis of data of the above step values of fitness of test suite are evaluated .These values are shown in Table 3. Then,
Volume 2, Issue 6, June 2013
Page 203
International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org, editorijaiem@gmail.com
Volume 2, Issue 6, June 2013
ISSN 2319 - 4847
Crossover is applied on the initial Population and new Population is generated. One point Crossover has been applied
with a crossover rate of 2.0. The new chromosome generated is then mapped with test cases and their fitness is again
calculated. This is followed by the application of mutation operator. The mutated population is shown in Table 3. The
coefficient of variation is then calculated for each test case. Table 4 shows the requisite data. The APFD of the final test
suite is coming out to be 0.94, which is much better then the random ordering.
Table1 Initial Population
0
0
1
0
0
0
1
1
0
1
1
1
0
1
0
1
1
0
0
0
1
0
1
0
0
1
0
1
0
0
0
0
0
1
1
0
0
1
1
0
0
1
1
1
1
0
0
0
1
0
0
0
0
0
1
1
0
1
0
0
1
1
0
1
1
1
0
1
0
0
0
1
0
0
0
0
1
0
1
0
1
1
0
0
0
0
1
0
1
1
0
1
0
0
1
1
0
0
0
0
1
1
0
0
0
1
0
1
1
0
1
0
1
0
0
0
0
0
1
0
0
0
1
1
0
0
0
1
1
1
1
0
0
1
0
0
0
1
0
1
0
0
1
1
0
0
1
0
1
0
0
0
1
1
0
1
0
0
1
0
1
1
0
1
0
1
0
1
0
0
0
0
0
0
1
1
0
1
0
1
1
1
1
0
0
1
1
1
0
0
0
0
0
1
1
0
0
0
1
0
0
0
0
1
0
0
1
1
1
0
Table 2: Running time in previous cycles.
67
451
129
343
196
348
496
7
413
76
415
134
141
336
320
86
324
305
25
432
348
14
130
168
301
76
253
144
277
208
400
364
167
87
306
356
307
223
173
454
Table 3: Mutation
TC1
TC2
TC3
TC4
TC5
TC6
TC7
TC8
TC9
TC10
T1
1
0
0
1
0
0
0
1
1
1
T2
1
0
1
1
0
0
0
1
0
1
T3
1
1
1
1
0
1
1
1
1
0
T4
1
0
0
1
1
1
0
1
1
0
T5
1
1
1
1
1
0
1
1
0
0
Volume 2, Issue 6, June 2013
Page 204
International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org, editorijaiem@gmail.com
Volume 2, Issue 6, June 2013
ISSN 2319 - 4847
T6
0
1
0
1
1
1
1
1
1
0
T7
0
1
0
0
0
1
1
0
1
1
T8
0
1
1
0
1
1
0
0
1
0
T9
1
0
1
0
0
1
0
1
0
1
T10
0
1
1
0
0
1
0
0
1
0
T11
0
1
1
0
1
1
0
1
1
1
T12
0
0
0
1
1
1
1
1
0
1
T13
0
0
1
0
1
1
1
0
1
1
T14
0
1
1
1
0
1
0
0
0
1
T15
1
0
1
1
0
0
0
1
1
0
T16
0
1
1
1
1
1
1
1
1
1
T17
1
0
1
0
0
1
1
1
0
0
T18
0
1
1
0
1
0
1
0
0
0
T19
1
1
1
1
0
1
0
1
0
1
T20
1
1
1
1
1
1
0
1
1
0
T21
1
0
1
0
0
0
0
1
1
1
T22
0
1
0
0
1
0
1
1
1
1
T23
1
0
1
0
0
1
0
1
1
1
T24
1
0
1
0
0
1
1
1
1
0
Table 4: CV of each Test Case
TC1
TC2
TC3
TC4
TC5
TC6
TC7
TC8
TC9
TC10
SD
CV
SD
CV
SD
CV
SD
CV
SD
CV
SD
CV
SD
CV
SD
CV
SD
CV
SD
CV
155.8453
0.629678
181.3345
0.692777
155.8533
0.600591
109.1407
0.494409
150.3305
0.553703
119.9166
0.726768
88.52824
0.45751
74.97458
0.240111
107.3103
0.468604
106.4856
0.368144
5. CONCLUSION
The cost of a test case can be perceived as the time it takes to execute the test case. The cost cognizant technique relies
on the cost of a test case. However, there is no definite procedure of calculating the effective cost when the value of cost
caries in different cycles of testing. The above work proposes a technique that evaluates the effective cost of testing. The
proposed technique is applied on professional software developed in C#, .NET framework. The results obtained are
encouraging and thus instills confidence in the technique. However, it is intended to amalgamate the concept of cost
and time in order to develop a technique which is more effective. The concept is being developed and being applied to a
professional Enterprise Resource Planning System.
Volume 2, Issue 6, June 2013
Page 205
International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org, editorijaiem@gmail.com
Volume 2, Issue 6, June 2013
ISSN 2319 - 4847
References
[1.] Harsh Bhasin et. al., “Regression testing using Fuzzy Logic”, International journal of Computer Science and
Information Technology, Vol. 4, No. 2, 2013.
[2.] Harsh Bhasin et. al., “Implementation of Regression testing Using Fuzzy Logic”, International Journal of
Application or Innovation in Engineering and Management, Vol. 2, No. 4, 2013.
[3.] Harsh Bhasin et. al., “Orthogonal Testing Using genetic Algorithms”, International Journal of Computer Science
and Information Technology, Vol. 4, No. 2, 2013.
[4.] Harsh Bhasin et. al., “Software Architecture Based Regression Testing”, IJCSE, Vol. 5, No. 04, 2013.
[5.] Harsh Bhasin et. al., “Software Architecture Based Regression Testing Implementation”, International Journal of
Computer Science and Engineering (IJCSE), Vol. 2, No. 3, 1-4, July 2013.
[6.] Harsh Bhasin et. al., “Test Data Generation using Artificial Life”, International Journal of Computer Applications,
Vol. 67, No. 12, 2013.
[7.] Harsh Bhasin et. al., “On The Application Of Artificial Life Based Test Data Generation”, International Journal of
Computer Science and Engineering, Vol. 5, 2013.
[8.] Harsh Bhasin, Neha Singla, “Test Data Generation Using Cellular Automata”, ACM Sigsoft Software Engineering
Notes, to be published in July 2013.
[9.] Harsh Bhasin, Neha Singla, “Cellular Genetic Test data Generation”, Communicated in ACM sigsoft Software
Engineering notes.
[10.] Harsh Bhasin, Surbhi Bhatia, “Use of Genetic Algorithms for Finding Roots of Algebraic Equations”, IJCSIT,
Volume 2, No. 4, Pages 1693-1696, 2011.
[11.] Harsh Bhasin et. al., “Novel Time Aware Regression Testing Technique”, International Journal on Computer
Science and Engineering (IJCSE), Vol. 5 No. 05, 2013.
[12.] Harsh Bhasin et al., “A Novel Approach to Cost Cognizant Regression Testing”, International Journal of
Computer Science and Management Research, Vol. 2, No. 5, 2013.
[13.] Harsh Bhasin, Neha Singla, “Genetic based Algorithm for N – Puzzle Problem”, International Journal of
Computer Applications, Vol. 51, No.22, 2012.
[14.] Harsh Bhasin, Rohan Mahajan, “Genetic Algorithms Based Solution To Maximum Clique Problem”, International
Journal on Computer Science and Engineering (IJCSE), Vol. 4, No. 08, 2012.
[15.] Harsh Bhasin and Neha Singla, “Harnessing Cellular Automata and Genetic Algorithms To Solve Travelling
Salesman Problem”, International Conference on Information, Computing and Telecommunications, (ICICT 2012), pp. 72 – 77, 2012.
[16.] Harsh Bhasin, Neha Singla, “Modified Genetic Algorithms Based Solution to Subset Sum Problem”, (IJARAI),
International Journal of Advanced Research in Artificial Intelligence, Vol. 1, No. 1, 2012.
[17.] Harsh Bhasin, Nishant Gupta, “Randomized algorithm approach for solving PCP”, International Journal on
Computer Science and Engineering (IJCSE), Vol. 4, No. 01, 2012.
[18.] Harsh Bhasin et. al., “Hybrid Genetic Algorithm for Maximum Clique Problem”, International Journal of
Application or Innovation in Engineering & Management, Volume 2, No. 4, April 2013
[19.] Harsh Bhasin, Gitanjali Ahuja, “Harnessing Genetic Algorithm for Vertex Cover Problem”, International Journal
of Computer Science and Engineering, Vol. 4, No. 2, 2012.
Volume 2, Issue 6, June 2013
Page 206