Exploring the Behaviour of Neural Network Based Object Oriented
Software Quality Models
S. KANMANI, V. RHYMEND UTHARIARAJ1 and R. NAKKEERAN2
Department of Computer Science and Engineering & Information Technology
Department of Electronics and Communication Engineering2
Pondicherry Engineering College
Pondicherry – 605 014
Ramanujam Computing Centre1
Anna University
Chennai – 605 025
INDIA
Abstract :- The application of statistical modeling techniques has been an intensely pursued area of research in
the field of Object-Oriented (OO) software engineering. The goal has been to model software quality and use
that information to better understand the development process. In this paper, we explore the application of
neural network approaches as a way of improving the predictive quality. We introduce two neural networks
namely Backpropagation Feedforward neural Networks (BPFN) and Probabilistic Neural Networks (PNN) for
the prediction of faultprone modules. We use size measures, OO measures and the combination of these two as
the independent variables with principal components to build three models. The dependent variable is the
membership in the class of faultprone modules. The results obtained by these systems are compared with that of
the two statistical approaches namely discriminant analysis and logistic regression. The results show that in
general the neural network systems do better and in particular PNN models outperform compared to other
models. In addition the measures combining size and OO measures do a better prediction.
Keyword : Object Oriented, metrics, faultproneness, neural network, discriminant analysis, logistic regression.
1 Introduction
Classifying the software modules as faulty or nonfaulty prior to coding alerts the developer. The
classification models can be used to forecast the
group membership of modules (fault or non-fault)
from a subsequent release or a similar system
developed in the same environment. This can help
the management to focus resources on those
modules that cause the bulk of problems. The need
for the quality prediction at early stage of the
software development phase made researchers to
work in two directions. Firstly to introduce new
design measures, which could predict the quality
attributes well. Secondly, to introduce new
approaches to build the prediction system. As the
software development environment moved to
Object-Oriented in 90’s, researchers focused on the
quality aspects in OO also. This made many
measures introduced for the various OO features of
the design - namely cohesion, coupling,
polymorphism, inheritance etc. Briand [4] explored
empirically the relationships between existing OO
coupling, cohesion and inheritance measures and
faultproneness using logistic regression model.
Briand et al [7] surveyed and analyzed the existing
quality models for OO systems, most of them are
statistical models. Quah [5] conducted a study in
the application of 15 OO metrics in predicting
number of defects and the number of lines changed
per module (class). They introduced Ward neural
network and General Regression Neural Network
(GRNN) in their analysis and found that GRNN
predicts more accurately.
We introduce in this paper two neural networks
– Back propagation Feed forward neural network
and Probabilistic neural network for predicting the
faultproneness of the OO module. To carry out this
case study we used the OO modules developed by
the students in our institution for the general library
maintenance process. We compared the results
obtained by these models with that of the statistical
models (discriminant analysis and logistic
regression) by computing misclassification rates.
The following sections discuss about the general
modeling method adopted, the four approaches used
for the development of prediction system, the case
study, results and discussion, and conclusion.
2 Modeling
Our goal is to develop a prediction model that will
classify the fault prone and non-fault prone
software modules. The following is a summary of
our methodology for building and validating the
quality models.
 Preprocess the metrics to improve the model.
We use principal component analysis to reduce
the dimensionality of the data and to produce
orthogonal independent variables.
 Select a model validation strategy. We used the
data splitting method
 Prepare fit (two third) and test (one third) data
sets. The fit data set is used to build a model.
The test data set is used to evaluate its
accuracy.
 Build the prediction model based on the fit data
set using any of the approaches. The OO
metrics are the independent variables and fault
proneness, the dependent variable.
 Evaluate the model using the validation data
sets, by comparing the prediction results with
the actual results.
2.1 Principal Component Analysis
Some of the OO metrics have high correlation (as
shown in Table 9) with each other. Principal
Component (PC) analysis is a standard technique
[4] and [5] to identify the orthogonal dimensions in
a data set. The components are linear combinations
of the standardized independent variables. The
varimax rotation method is adopted in this study. It
is an orthogonal rotation method that minimizes the
number of variable that have loadings on each
factor. We selected the PCs only whose eigen value
is larger than 1.0. The principal components
selected for the three category of prediction models
(with size measures, OO measures and both) are
given in the Tables 6, 7 and 8. The principal
components are identified for the data set using
SPSS V 7.2 package.
2.2 Evaluating the Accuracy of the model
To evaluate the model results the misclassification
rates in the prediction are used. If non-fault prone
modules are classified as fault prone (C1), it is
called as Type I error. If fault prone modules are
classified as non-fault prone (C2), it is called as
Type II error. Type I errors incur only modest cost
in terms of extra attention to those modules. Type II
errors risk unexpected problems late in the
development or even after release. The following
three parameters are used for the model evaluation
from the validation data set results.
Type I error rate 
C1
Total No. Faulty modules
(1)
Type 2 error rate 
C2
Total No. non faulty modules
(2)
Overall misclassif ication rate 
C1  C2
Total No. modules
(3)
3 Approaches
This section discusses about the various approaches
used in the case study to construct the prediction
model. The two statistical approaches (discriminant
analysis and logistic regression) are employed using
the SPSS V 7.2 package. The neural network
approaches (BPFN and PNN) are employed using
MATLAB V6.1 with neural network toolbox [3].
3.1 Discriminant Analysis
We use stepwise discriminant analysis [10] to
choose the domain metrics Dj that should be
included as independent variables in the
discriminant model. Variables are entered into the
model in an iterative manner, based on an F test.
Begin with no variables in the model. Add the
variable not already in the model with the best
significance as long as its significance is better than
the threshold (5%). Then remove the variable
already in the model with the worst significance
level, as long as it is worse than the threshold
(10%). Repeat these steps until no variable can be
added to the model. The final result is a subset of
Dj, j=1,…,p that are significantly related to
classification. We estimate the discriminant
function based on the fit data set and apply the
validate data set for evaluation.
3.2 Logistic Regression
Logistic regression is a standard technique based on
maximum likelihood estimation. A multivariate
logistic regression model is built based on the
following equation.
 ( X 1 , X 2 ,.... X n ) 
e (C0 C1 X1 ... Cn X n )
1  e (C0 C1 X1 ... Cn X n )
(4)
Where π is the probability that a fault is found in a
class and Xi’s are the design measures included as
the independent variables in the model. Logistic
regression is a form of regression, which is used
when the dependent is dichotomy and the
independents are of any type. We used stepwise
forward logistic regression model with 0.5 as the
classification cutoff. The threshold values for the
domain measures to enter and exit the model are
fixed at 0.5 and 0.10.
whose elements indicate how close to the input is to
a training input. The second (competitive) layer
sums these contributions for each class of inputs to
produce as its net output a vector of probabilities.
Finally a complete transfer function on the output
of the second layer picks the maximum of these
probabilities and produces a 1 for that class and a 0
for the other classes [1].
3.3 Backpropagation Feedforward Neural
Network
An artificial neural network is composed of
computational processing elements with weighted
connections. Each neuron has number of inputs, an
activation function and an output. The multilayer
perceptron model arranges these units in layers: one
layer for system input variables, another for system
output variables and between these layers one or
more hidden layers. We use a feedforward
multilayer
perceptron
network
and
the
backpropagation training algorithm [8].
The input layer has one neuron for each of the
input variable (domain measures). We use one
hidden layer. There are two units in the output
layer, one for each class (fault or non-fault). The
output unit with the greatest value indicates the
class selected by the network. The network learns
by finding a vector of connection weights and
minimizes the sum of squared errors on the training
data set. One pass through all of the training
observations is called an epoch. After training, the
network is simulated for the validation data set and
the classification outputs are obtained.
The activation function used is the ‘tansig’
function, the learning rate 0.1, the momentum
coefficient 0.4, the training algorithm ‘trainrp’ and
the number of epochs 1000. We trained the
network, where the weights are adjusted after each
observation. Various values for the number of
neurons are tested to find a preferred value [3].
3.4 Probabilistic Neural Network
Specht’s [9] claim that PNN can train faster than
backprobagation networks and also able to solve
any smooth classification problem, if, enough data
is given. The PNN architecture is given in Fig 1.
When an input is presented, the first (radial basis)
layer computes the distances from the input vector
to the training input vectors and produces a vector
Fig. 1 Architecture of Probabilistic neural
network
4 Case Study
4.1 System Description
The software system used for this study is
developed by the Undergraduate / Graduate
students of the Department of Computer Science
and Engineering at Pondicherry Engineering
College. All students have got experience with C
and C++ programming language. The students are
grouped into teams. Each team is asked to develop
a departmental library Management information
system that supports the issue and return process of
books to staff members and students. They develop
the system in Turbo C++ environment with
necessary libraries, using ASCII files for data
storage. A team of 3 experienced staff members of
the department carry out testing of the systems.
This group test the systems for the important test
cases derived for the problem. The total number of
classes (modules) in the resulted 200 software
systems is 1185. The size details of the modules
are given in Table 1.
Table 1 Size details of the modules developed in
the experiment
The Object Oriented Metric Calculator (OOMC) a measurement tool developed is used to collect the
values of the Object Oriented design measures from
the C++ source code [6].
In order to carry out this experiment, the OO
measures already used in the literature [4] are
considered. The tool collects 64 measures in total.
As recommended by Briand et al [4], the measures
with maximum number of zeroes are eliminated in
the first filtering process. Then the correlation
between the resulted measures are studied and the
measures with high correlation are removed.
Leaving the size measures, finally, resulted to 15
measures, which are explained in the Table 2
results of the model III gives the most accuracy in
prediction. The reason for good prediction by the
system developed with size measures is that
students developed the programs without
concentrating on User Interfaces. The software
modules are developed for the required
functionality of the problem, which is reflected in
the count of attributes, methods and statements
defined in them. Among the four approaches used
neural network based systems perform better than
their statistical equivalent. The newly introduced
PNN in general, outperforms in classification with
below 1% misclassification rate.
Table 3 Model I Results
Table 2 Object Oriented Metrics
Table 4 Model II Results
Table 5 Model III Results
4.2 Results and Discussion
In this section, we compare the prediction models
built from size measures only (Model I), from
coupling, cohesion and inheritance measures only
(Model II), and one allowing all measures (design
coupling, cohesion, inheritance and size - Model
III). The prediction systems are built only by the fit
data set in each category using four approaches.
The validation data set input values are fed into the
prediction systems and the classification results of
the systems are evaluated using the error rates
discussed in Section 2.2. The results obtained by
the four prediction systems for the three models are
given in Tables 3, 4 and 5. From the overall
performance of the prediction systems, it is found
that the model I based systems are able to predict
well than model II based systems however, the
5 Conclusion
In the development of prediction systems to classify
the faultprone modules, more reliable approaches
are expected by the software developers to reduce
the misclassifications. Type II misclassifications are
more serious, because they risk in releasing the
software with undetected faults which in turn may
cause operational failures. In this paper, we
introduced two neural network based approaches,
which perform better than the statistical approaches.
Among the two, PNN outperforms in prediction
with below 1% misclassification rate. Also from the
results of the three systems developed, it is found
that PNN performance is consistently better than all
other models and the Type II error rate is always
lesser than Type I error rate in each case.
References:
[1] S. Greenberg, I. Volchek and B. Kaysel,
Finger Print Classification using PNN,
WSEAS Trans. on Systems, Issue 3, Vol. 3,
2003, pp 566-571.
[2] T.S.Quah and M.M.T.Thewin, Prediction of
Software Development Faults In PL/SQL
Files using Genetic Nets, WSEAS Trans. on
Commn., Issue1, Vol. 3, Jan 2004, pp228233.
[3] Howard Demuth, Mark Beale, Neural
Network Toolbox: For use with MATLAB,
The Mathworks Inc. Natick Massachusets
USA, 1998.
[4] Briand L., J.Wust, J.W.Daly, D.V. Porter,
Exploring the Relationships Between Design
Measures and Software Quality, Journal of
Systems and Software, 51, 2000, pp. 245-273.
[5] T. S Quah, and M.M.T.Thewin, Application
of Neural Networks for Software Quality
Prediction using OO Metrics, Proc. of the
Int’l Conf. ICSM’03, Vol. 3, 2003.
[6] Kanmani S., V.Sankaranarayanan and
P.Thambidurai, A Measurement Model for
C++
Program
Complexity
Analysis,
Proceddings of the 9th International
Conference EPMESC, Macao, China, 2003,
pp. 575-580.
[7] Briand.L, Wust.J, Empirical Studies of
Quality Models in OO Systems, Advances in
computers, Academic Press, Vol. 56, 2002,
pp. 72-116
[8] Wasserman, P.D, Advanced Methods in
Neural Computing, new York, Van nostrand
Reinhold, 1993.
[9] Specht.D.F, Probabilistic Neural Networks,
Neural Networks, Vol. 3, No.1,1990, 109118.
[10] Tabachnick, Barbara G and Linda S.Fidell,
Using Multivariate Statistics, Fourth edition,
2001, Boston.
Table 6 Principal components for Model 1
Table 7 Principal components for Model 2
Table 8 Principal components for Model 3
Table 9 Pearson Correlation co efficient between
the measures