Comput. Lang. Vol. 17, No. 1, pp. 75-82, 1992
Printed in Great Britain. All rights reserved
0096-0551/92 $3.00+ 0.00
Copyright© 1991 PergamonPressplc
SPECIFICATION A N D TESTING OF ABSTRACT
DATA TYPES
PANKAJ JALOTE
Department of Computer Science and Engineering, Indian Institute of Technology, Kanpur-208016, India
(Received 21 January 1990; revision received 6 December 1990)
Abstract--Specifications are means to formally define the behavior of a software system or a module, and
form the basis for testing an implementation. Axiomatic specificationtechnique is one of the methods to
formally specify an abstract data type (ADT). In this paper we describe a system called SITE, that is
capable of automatically testing the completenessof an ADT specification.In addition, an implementation
of the ADT, can also be tested, with SITE providing the test oracle and a number of testcases. For
achieving these two goals, the system generates an implementation of the specifications and a set of
testcases from the given specifications. For testing the completeness of specifications, the generated
implementation is tested with the generated testcases. For testing a given implementation, the generated
and given implementations are executed with the generated testcases and the results compared. One way
to utilize the system effectivelyis to first test the specifications and modify them until they are complete,
and then use the specifications as a basis for prototyping and testing. The system has been implemented
on a Sun workstation.
Abstract data types
Axiomaticspecifications
Completenessof axioms
Softwaretesting
1. I N T R O D U C T I O N
Software specifications form a critical component in developing reliable software. Specifications
provide the means to properly communicate the functionality of the system to be developed, and
form the basis for testing the software. Correct, complete and unambiguous specifications are
essential for developing error-free software. Formal specifications have been proposed as a means
for making the communication about the functionality of the system precise, and less prone to
errors due to mis-interpretations [1].
An important use of specifications is during the testing of software developed to implement the
specifications. During testing, the consistency between the behavior of actual software and the
specification is checked for the given testcases. With formal specifications, a desirable goal is to
automate, as much as possible, the testing of software against its specifications.
However, formal specifications need not always be complete. That is, the specifications, though
formally stated, may not completely specify the behavior of the software. Incompleteness provides
r o o m for misinterpretations, and is likely to cause problems when the software is tested against
the specifications. Hence, another desirable goal with formal specifications is to test the completeness of specifications themselves. It has been argued that testing specifications early is quite
important of reducing the cost of software construction [2], as detecting problems in the
specifications later may result in considerable amount of wasted effort.
In this paper, we focus on abstract data types (ADTs). An A D T supports data abstraction, and
comprises of a group of related operations that act upon a particular class of objects, with the
constraint that the behavior of the objects can only be observed by applications of the operations
[1]. D a t a abstraction is extremely useful for information hiding and supporting high level
abstraction. M a n y languages like Simula [3], C L U [4], Euclid [5], and Ada [6] support data
abstraction.
Abstract data types have frequently been used for formal specifications, and various specification
techniques have evolved for specifying them [1]. One of the methods for specifying A D T s is the
axiomatic specification technique [7-11]. Axiomatic specification techniques employ axioms to
specify the behavior of the operations of the ADT.
In this paper we describe the Specification and Implementation TEsting (SITE) system. There
are two major goals of the system. First, to automatically test the completeness of a given set of
75
76
PANKA,1JALOTE
axions. This testing process is similar to testing programs for correctness, and does not prove
completeness. The second goal of SITE is to test an implementation of the data type whose
specifications are provided.
The SITE system consists of two major components: an implementation generator and a testcase
generator. The implementation generator translates the specifications into an excutable program,
such that if the specifications are not complete, the implementation is not complete and the
behavior of all the sequences of valid operations on the A D T is not defined. The testcase generator
produces a set of testcases which is used for testing completeness of specifications, as well as for
testing implementations. For testing the completeness of specifications, the generated implementation is executed with the generated testcases. For testing an implementation, the user implementation and the generated implementation are executed with the set of generated testcases and the
results compared.
In the next section we describe our specification language, and an example is given. We also
discuss the notion of completeness in more detail. The system and its components are described
in Section 3. In Section 4 we consider the testing of specifications, and in Section 5 testing of
implementations is described.
2. A X I O M A T I C
SPECIFICATION
The specification language we employ has two major components--syntactic specifications and
semantic specifications. The syntactic part defines the name of the data type, its parameters, the
type of the input parameters and the result for operations, and variables. The semantic part
specifies the axions for operations. Axioms attach meaning to operations by specifying the
relationship between operations. The construction of axioms for an ADT is beyond the scope of
this paper and the reader is referred to [9, 11]. As an example, the specifications of the abstract
type queue are shown in Fig. 1.
sl. queue [ Item ]
declare
s2.
newq ( ) -> queue ;
s3.
addq ( queue , Item ) -> queue ;
s4.
deleteq ( queue ) -> queue ;
s5.
emptyq ( queue ) -> boolean ;
s6.
appendq ( queue, queue ) -> queue ;
s7.
frontq ( queue ) -> Item U undefined ;
var
s8.
q , r : queue ;
s9.
,i : Item ;
forall
al.
emptyq ( newq ( ) ) = true ;
a2.
emptyq ( axldq ( q , i ) ) -- false ;
n3. frontq ( newq ( ) ) - undefined ;
a4.
frontq ( addq ( q , i ) ) = if emptyq ( q ) then i
else frontq ( q ) ;
aS.
deleteq ( newq ( ) ) = newq ( ) ;
a6.
deleteq ( addq ( q , i ) ) - if emptyq ( q ) then newq ( )
else axldq ( deleteq ( q ) , i ) ;
a7.
appendq ( q , newq ( ) ) = q ;
a8.
appendq ( r , addq ( q , i ) ) -- axidq ( appendq ( r , q ) , i ) ;
end
Fig. 1. Sp~ifications of the type queue.
Specification and testing of abstract data types
77
In SITE axioms are considered as rewrite rules [12, 13] and we require that the set of axioms
have the properties of finite-termination and unique termination [13]. That is, for any expression
no infinite rewriting is possible, and the rewriting always terminates in a unique expression. Finite
termination ensures that during execution the implementation generated by SITE will not get into
an infinite loop. With unique termination, the axioms can be applied in any order by the
implementation. No assumption is made about the structure of axioms, and the expression of
operations in the axioms need not be limited to, for example, the structure that results from the
axiom construction technique given in [9].
Two important issues for a set of axioms are completeness and consistency. In this paper we
restrict attention on the completeness issue. Consistency means that the semantics specified by the
different axioms are not contradictory. Determining consistency is a theoretically and undecidable
problem, but in practice it is often relatively simple to demonstrate the consistency of axioms [11].
The Knuth-Bendix algorithm [12] can also be used.
For defining completeness, the operations of a data abstraction are divided into two categories-those that return the ADT under consideration, and others which return some other type. The latter
type of operations are called behavior operations. We refer to the former as non-behavior
operations. In the example of the queue, the operations emptyq and frontq are the only behavior
operations.
In this paper, for completeness we use the notion of sufficient completeness [9, 11]. A set of
axioms specifying an abstract type is considered sufficiently complete if, and only if, for every
possible instance of the abstract type (i.e. one that can be created by some sequence of non-behavior
operations), the result of all the behavior operations of the type is defined by the specifications
[9, 11].
This notion of sufficient completeness is from the external viewpoint, i.e. the external behavior
of the ADT should always be specified. The specifications for the type queue given earlier can be
shown to be sufficiently complete. To show incompleteness we need to have an instance such that
the result of some behavior operation is not defined. This is the approach we take during testing.
However, the general problem of determining if a set of axioms is sufficiently complete is
undecidable [9].
The most obvious reason for incompleteness, is that some of the axioms are not provided.
Though completeness is defined from the point of view of externally observable behavior of the
type, incompleteness will result even if the missing axioms are for non-behavior operations as this
could lead to construction of an instance on which the result of some behavior operation is not
defined. In this paper, we only consider the incompleteness caused by missing axions.
3. THE SITE SYSTEM
The SITE system is implemented in C on a Sun Workstation and has two major components-the implementation generator and the testcase generator. The implementation generator is like a
compiler that takes in the specifications of the data type, and produces an implementation in C
for the specifications. As the axioms are tested during the completeness testing, the testcase
generator produces a set of testcases that depend largely on the syntactic part of the specifications.
This ensures that the testcases themselves are not affected by the incompleteness of axioms.
3.1. Implementation generator
In order to be able to generate an implementation of an arbitrary abstract type, we need to have
a general structure capable of representing instances of different data types. For this trees are
suitable. An instance of the data type is represented as a general tree, and the operations on the
ADT as tree manipulation functions. Each node in the tree contains, besides pointers to access the
parent node and different sub-trees, information about the operation that created that node,
including the parameters and name of the operation. This representation is quite general and also
simplifies the translation of operations into tree manipulation functions. Tree representation has
been used for representing expressions in interpretive systems [8, 10, 14]. The tree representation
in SITE is similar to others, though it is used to actually represent an instance of an ADT, and
is manipulated by operations on the ADT. In interpretive systems, the tree is used by the
78
PANl~J JALOTE
interpreter. In addition, in SITE, the input specifications need not be complete and the constructors (constructors are a subset of non-behavior operations using which any instance of the
ADT can be constructed.) of the ADT need not be known (as is the case in the representation
proposed in [10]).
Two sets of functions are generated for defining the operations on the ADT as tree manipulation
functions. First, functions are generated from the axioms. Generating these functions is relatively
straight forward. The function for an axiom is simply the right side of the axiom, with each variable
replaced by a reference string in the tree. The reference string for a variable is determined by the
left side of that axiom, and is the symbolic address in the trees representing the instances of the
data type on which the operation is performed.
The functions from the axioms of an operation are special case functions. These can be used to
implement the operation itself, if it can be decided which of these functions to invoke when the
operation is performed. For example, if emptyq is applied on an instance whose tree is such that
the last operation is newq, then the function corresponding to the axiom a l should be invoked.
To invoke the appropriate axiom function, a function for each operation is generated (which bears
the name of the operation).
These functions employ a "matching function" to decide which of the axioms, if any, is
applicable. During implementation generation, an encoding of the traversal strings of the parse
trees of the parameters in the left side of the axiom are also generated. The matching function
compares these strings with the traversal strings of the instances on which the operation is being
performed to determine which axiom is applicable. If an axiom is applicable, the matching function
returns the number of that axiom, which is used to invoke the function corresponding to that
axiom. If no axiom is applicable, the default condition is invoked.
A new node is created for the tree whenever a default condition is invoked for a non-behavior
operation. With the new node and the trees representing the instances on which the operation is
performed, a new tree is created. This tree represents the instance resulting from applying the
operation.
For the behavior operations the default condition has a different meaning. If the specifications
are complete, the axioms should specify the result of applying any behavior operation on any
instance of the data type. Hence, the default condition means that the set of axioms is incomplete.
This is the property that is used to test for incompleteness. The goal for detecting incompleteness
is to construct an instance of the abstract type on which the result of performing some behavior
operation is not defined. Further details about the implementation generation can be found in [15].
3.2. Testcase generator
The effectiveness of any testing process is highly dependent on the choice of testcases. The need
for proper testcases is even more important when testing for incompleteness. Completeness is
defined from the point of view of external behavior, and can only be detected by showing that
there is an instance of the data type for which the specifications do not define the value for some
behavior operation. In other words, incompleteness can only be detected by applying a behavior
operation. Furthermore, only the result of a behavior operation can be meaningfully displayed and
compared. For displaying or comparing the values of abstract data type instances, special functions
on that type will be needed. For these reasons, each testcase is considered to be an expression of
the form
behavior_operation (an ADT instance)
As our goal is to detect the incompleteness of the set of axioms, it is important that the testcase
generation strategy does not depend heavily on the axioms. The testcase generator in our system
generates the testcases largely from the syntactic part of the specifications. This approach is
different from the testcase generation strategy proposed in [16], where the testcase generation
depends on the structure of the axioms.
The basic scheme for generating testcases is simple. From the syntactic specifications, all the valid
expressions that produce an ADT instance, up to a given maximum nesting depth of operations,
are generated. Testcases are generated by applying the different behavior operations on these
instances.
79
Specification and testing of abstract data types
1. emptyq(newq0)
2. frontq(newq())
3. emptyq(addq(newq(),90))
4. frontq(addq(newq(),90))
5. emptyq(deleteq(newq0) )
6. front q(deleteq(ne'wq0) )
7. emptyq(appendq(newq(),q2))
8. front q(appendq(newq(),q2))
9. emptyq(addq(addq(q 1,75),84))
10. frontq(addq(addq(ql,75),84))
11. emptyq(deleteq(addq(ql,75)))
12. frontq(deleteq(addq(q 1,75)))
13. emptyq(appendq(addq(q 1,75),q2))
14. frontq(appendq(addq(q 1,75),q2))
15. emptyq(appendq(ql,addq(ql,75)))
16. frontq(appendq(ql,addq(ql,75)))
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
emptyq(deleteq(deleteq(ql)))
front q(deleteq(deleteq(ql)))
emptyq(addq(deleteq(ql),81))
frontq(addq(deleteq(ql),81))
emptyq(appendq(deleteq(ql),q2))
frontq(appendq(deleteq(ql),q2))
emptyq(appendq(ql,deleteq(ql)))
frontq(appendq(ql,deleteq(ql)))
emptyq(addq(appendq(ql,q2),74))
frontq(addq(appendq(ql,q2),74))
emptyq(deleteq(appendq(ql,q2)))
frontq(delctcq(appendq(ql ,q2)))
emptyq(appendq(appendq(ql,q2),q2))
frontq(appendq(appendq(ql,q2),q2))
emptyq(appendq(ql,appendq(ql,q2)))
frontq(appendq(ql,appendq(ql,q2)))
Fig. 2. Testcases generated for the type queue.
The immediate question is what should be the depth for which the expressions are generated.
In SITE the testcases up to a depth d are generated, where d is the maximum nesting depth of any
axiom. In the case of queue, the maximum nesting depth of the axioms is 2 (i.e. d = 2). Depth is
the only property of the axioms considered in testcase generation. Usually, many axioms may have
depth d, and omission of a few axioms will not affect d. For example, in the type q, all the axioms
have depth of 2. Some of the testcases generated for the type queue, are shown in Fig. 2.
The intuitive reason for selecting d as the depth up to which the expressions are generated is that
since expressions of up to depth d are considered in the axioms, then the expressions up to that
depth are most significant, and the behavior of different expressions up to depth d will depend on
the particular composition of operations. The depth of axioms for most data types is not less than
2, because the left side of the axioms usually contains some composition of operations. In fact,
if we follow the heuristics given in [9] for writing axioms, then the left side of all the axioms will
have depth 2. Depth of 2 or more ensures that the testcases generated by our scheme will exercise
different compositions of the operations. This is another reason for selecting our testcase generation
criteria. Furthermore, this strategy also produces testcases that exercise the "boundary conditions"
like frontq(newq()), deleteq(newq()) etc. Boundary condition testcases are generally regarded as
"high yield" testcases for program testing [17]. The usefulness of the generated testcases for testing
specifications and implementations is discussed in the next sections.
4. TESTING SPECIFICATIONS
For testing the completeness of specifications, the generated implementation and testcases are
compiled together and executed. The system configuration for testing completeness is shown in
Fig. 3.
Specs
SITE
Generated
Implementation
Testca.ses
Fig. 3. Testing completenessof specifications.
Testing
r results
80
PANKAJ JALOTE
In SITE, incompleteness can be detected only by applying a behavior operation on an instance
of the ADT. This is one reason why the testcases have the structure described above. For testing
incompleteness of specifications, the choice of testcases is very critical, and if proper testcases are
not provided, the incompleteness of axioms may go undetected. For example, suppose that the two
axioms for the deleteq operation are missing, and we give the following testcase for testing.
frontq (addq (addq (newq (), 5), 6))
For this testcase the generated implementation will return the correct answer (which is 5), and the
incompleteness will not be detected. This happens because the functions corresponding to the
missing axioms were not needed during the execution.
To evaluate the effectiveness of our system, we performed some experiments. In our experiments
we only consider incompleteness caused by missing axioms. For the type queue, we formed eight
different incomplete specifications by deleting one axiom at a time. These specifications were then
used to test if the system can detect incompleteness. Clearly, if the omission of any one axiom can
be detected, omission of a set of them can also be detected by the same set of testcases. The set
of testcases shown earlier were used [ql and q2 were set to newqO].The testcases were able to detect
the incompleteness in all the eight sets of axioms.
Similar experiments were also performed on many standard ADTs like stacks, trees, sets, strings
etc., and in all the cases incompleteness caused due to missing axioms was detected.
The information about which testcases detect the incompleteness can be quite useful in
determining what axioms are missing. Hence, the system not only detects the incompleteness, but
provides a good feedback about the source of incompleteness. This property of the system can be
used as an aid for writing complete specifications. Further details about testing of specifications
are given in [18].
5. TESTING IMPLEMENTATIONS
The SITE system can also be used for automating the testing of user implementations of data
types. The concept of using specifications for testing implementations of data abstraction is also
used in DAISTS [19, 20]. The DAISTS approach is to use the user implementation to evaluate the
left and right side of all axioms for some testcases, and then compare the results. This requires an
equality function for determining if two ADT instances are equal. DAISTS requires the user to
provide the equality function as well as the testcases.
The SITE approach for testing implementations is different from DAISTS. The generated
implementation and the implementation under testing are executed with the same set of testcases.
The output of the two implementations is then compared. Since the structure of testcases is such
that the outermost operation is a behavior operation, no equality function is needed to compare
instances of two ADTs. The system configuration for implementation testing is shown in Fig. 4.
The testcases generated by SITE can be used for testing implementations. The user can also
augment the set of testcases, if desired. However, the structure of the testcases must be as described
earlier.
Specs - ~
Generated
Implemlntati°n ]
SITE
'
>[
Testcases
l
User
I
Implementation I
Fig. 4. Testing an implementation.
Comparator
~
.
Testing
results
Specification and testing of abstract data types
81
Testing implementations is more complex than testing for incompleteness, in that the types of
errors in implementations is limited only by the ingenuity of the implementor, and cannot be
predicted. The goal of software testing, in general, is not to show that there are no errors (which
can only be done in very special circumstances), but try to uncover as many errors as can be
detected with a reasonable number of testcases.
The cost of testing software can be quite high and is often more than the cost of designing and
implementing the software. The high cost is largely due to the cost of generating testcases and the
cost of interpreting the result of the testcases. In order to decide if the results produced by an
implementation are correct, testing requires a "test oracle" that can tell the correct output for the
testcases. Frequently, the test oracle is a human being who works out by hand the correct result
of the testcases. This makes testing unreliable and expensive.
In SITE the testcases are automatically generated and a reliable test oracle is also produced,
thereby drastically reducing the effort for testing. To determine the coverage of the testcases
generated by SITE, we performed some experiments. Two data types were chosen--a priority queue
with some added operations to make it more interesting, and a binary tree. A set of 6 sophomore
students wers selected to implement each data type. Each of the students had knowledge of Pascal,
but had not taken the data structures course.
The variables in the testcases (like ql and q2) were initialized by the operation to create an
A D T instance (like newq). Each implementation was first tested with a set of testcases with depth
of 1. The author of the program had to correct any errors detected by these testcases, and then
repeat this process. All the errors detected were reported. Errors introduced during error-correction
were not included. Once an implementation successfully executed all the testcases for depth 1, it
was then tested with testcases of depth 2. Similar process for correcting and reporting error was
performed. Finally, an extensive testing was done with a mixture of testcases generated for depth
3 and some extra testcases. The variables were also assigned different initial values for extensive
testing.
We found that depth 1 testcases, though they detected on an average 0.8 errors, left some errors
undetected. With testcases of depth 2 an additional 2 errors were detected, on an average. And
extensive testing detected an additional 1 error. Further details of the experiments are given in [21].
Clearly, depth 1 testcases are insufficient. The depth 2 cases are also insufficient if the variables
are initialized with the operation that create an A D T instance. However, we found that if these
variables were given a more "interesting" initial value, more errors were detected. By assigning an
expression consisting of many operations to initialize the variables, we effectively increase the depth
of the testcases, without having the combinatoric explosion which occurs by merely increasing the
depth in the testcase generator. In most of the experiments, these testcases were able to detect all
the errors that were detected by extensive testing.
Though reducing the number of testcases is not a goal of SITE, as the testcases as well as the
test oracle are both automatically generated, we would still like to keep the number of testcase low,
without sacrificing the effectiveness of testing. From the experiments we believe that in most
situations testcases with depth 2, with carefully selected initial values, will detect most of the
implementation errors. However, if desired, the depth of the testcases can be increased and testcases
can also be added by the tester.
6. C O N C L U S I O N
Abstract data types are frequently used for formal specifications. One of the methods for
formally specifying ADTs is the axiomatic specification technique, which employs axioms to specify
the behavior of the operations of the ADT. An important issue with axiomatic specifications is if
the set of axioms is complete. Determining completeness of an arbitrary set of axioms is
undecidable, and frequently, formal verification on a case-by-case basis is done to determine if a
set of axioms is complete.
In this paper we describe the SITE system, which can automatically test the completeness of a
given set of axioms. The testing process is similar to testing programs for correctness, and does
not prove completeness. Another property of SITE is that it can test a given implementation of
the data type, from the specifications of the ADT.
82
PANK~ JALOTE
To provide these services the SITE system has an implementation generator and a testcase
generator. The implementation generator faithfully translates the specifications into an executable
program, such that if the specifications are not complete, the implementation is not complete and
the behavior of a sequence of valid operations on the ADT is not always defined. The testcase
generator produces a set of testcases which is used for testing completeness of specifications, as
well as for testing implementations.
The generated implementation is executed with the generated testcases to test the completeness
of specifications. In our experiments we found that incompleteness caused due to missing axioms
was detected for a variety of data type specifications. For testing an implementation, the given and
generated implementations are executed with the set of testcases and the results compared. We
found that in most cases the testcases generated by the testcase generator are capable of detecting
all the errors that we were able to detect by extensive testing, if the initial values assigned to the
variables used in the testcases are properly chosen by the tester.
We believe that a system like SITE can be very useful in software development, as it can be used
for writing formal specifications, rapid prototyping and software testing. Due to automated testing,
SITE can considerably reduce the cost for testing, and improve software reliability.
REFERENCES
1. Liskov, B. H. and Zilles, S. N. Specification techniques for data abstractions. IEEE Trans. Softw. Engng SE-I: 7-19;
1975.
2. Kemmerer, R. A. Testing formal specifications to detect design errors. IEEE Trans. Softw. Engng SE-II: 32-43; 1985.
3. Dahl, O. J., Myhrhaug, B. and Nygaard, K. The SIMULA 67 common base language. Norwegian Computing Center,
Oslo, Publication S-22; 1970.
4. Liskov, B. H., Snyder, A., Atkinson, R. and Schaffert, C. Abstraction mechanisms in CLU. Comman. ACM 20:
564-676; 1977.
5. Chang, E. and Zilles, S. N. Abstract data types in Euclid. In Notes on Euclid (Edited by Elliot, W. D. and Bamard,
D. T.), Technical Report 82. Computer Systems Research Group, Department of Computer Science, University of
Toronto; 1977.
6. U.S. Department of Defense. Reference Manual for ADA Programming Language. Bedim Springer; 1983.
7. Gehani, N. Specifications: formal and informal--a case study. Softw. Pract. Exper. 12: 433-444; 1982.
8. Goguen, J. and Tardo, J. J. An introduction to OBJ: a language for writing and testing formal algebraic program
specification. In Proceedings of Specification Reliable Software, pp. 170-189; 1979.
9. Guttag, J. V. and Homing, J. J. The algebraic specifications of abstract data types. Acta inform. 10: 27-62; 1978.
10. Guttag, J. V., Horowitz E. and Musser, D. R. Abstract data types and software validation. Commun. ACM 21:
1048-1063; 1978.
11. Guttag, J. Notes on type abstraction (version 2). IEEE Trans. Softw. Engng SE-6: 13-23; 1980.
12. Knuth, D. E. and Bendix, P. E. Simple word problems in universal algebra. In Computational Problems in Abstract
Algebra (Edited by Leech, J.), pp. 263-297. New York: Pergamon Press; 1970.
13. Musser, D. A. Abstract data type specification in the AFFIRM system. IEEE Trans. Softw. Engng SE-6: 24-32; 1980.
14. O'Donnel, M. J. Equational Logic as a Programming Language. The MIT Press; 1985.
15. Jalote, P. Synthesizing implementation of abstract date types from axiomatic specifications. Technical Report
UMIACS-TR-87-16, University of Maryland, College Park; 1987.
16. Bouge, L., Choouet, N., Epibourge, L. and Gaudel, M. C. Test sets generation from algebraic specifications using logic
programming. Universit6 de Paris-Sud, LRI, No. 240; 1985.
17. Myers, G. J. The Art of Software Testing. New York: Wiley; 1979.
18. Jalote, P. Testing the completeness of specification. IEEE Trans. Softw. Engng 15: 526-531; 1989.
19. Gannon, J., McMullin, P. and Hamlet, R. Data abstraction implementation specification and testing. ACM Trans.
Progr. Lang. Syst. 3: 211-223; 1981.
20. McMullin, P. R. and Gannon, J. D. Combining testing with formal specifications: a case study. IEEE Trans. Softw.
Engng SE-9: 328-335; 1983.
21. Cabalero, M. G. A case study in automatic test case generation for data abstraction. Project Report, Program Library,
Department of Computer Science, University of Maryland.
About the Author--Pankaj Jalote got his B.Tech. in Electrical Engineering from the Indian Institute of
Technology Kanpur in 1980, an M.S. in Computer Science from the Pennsylvania State University in 1982,
and a Ph.D in Computer Science from the University of Illinois at Urbana-Champaign in 1985. From
1985 to 1989 he was an Assistant Professor in the Department of Computer Science at the University
of Maryland at College Park. Since 1989 he has been an Assistant Professor in the Department of
Computer Science and Engineering at the Indian Institute of Technology Kanpur. His research Interests
are Software Engineering, Fault-tolerant Computing Systems and Distributed Systems.