Introduction
Definitions:
Contd…
Contd…
 In Software development, quality of design
encompasses requirements, specification and the
design of the system where as quality of conformance
is an issue focused primarily on implementation
 If the implementation follows the design and the
resulting system meets its requirements and
performance goals, conformance quality is high
Verification and Validation
 The program being developed must be checked to






ensure that it meets its specification and delivers the
functionality expected by the people paying for the
software.
Verification
Are you building the product right?
Software must conform to its specification
Validation
Are you building the right product?
Software should do what the user really requires
Test case
 A test case is a test related item which contains the
following information
1. A set of test inputs: These are data items received from
an external source by the code under test. The external
source can be hardware, software or human
2.Execution conditions: These are conditions required
for running the test for example a certain state of a
database or a configuration of a hardware device
3. Expected Outputs: These are the specified results to
be produced by the code under test.
Test oracle
 Test Oracle: A test oracle is a document or piece of
software that allows testers to determine whether a
test has been passed or failed.
 To check if a failure has occurred when executed with a
test case, we need to know the correct behavior
 That is we need a test oracle, which is often a human
 Human oracle makes each test case expensive as
someone has to check the correctness of its output
Role of Test cases
 Ideally would like the following for test cases
 No failure implies “no defects” or “high quality”
 If defects present, then some test case causes a failure
 Psychology of testing is important
 should be to ‘reveal’ defects(not to show that it works!)
 test cases must be “destructive”
 Role of test cases is clearly very critical
 Only if test cases are “good”, the confidence
increases after testing
Test case design
 During test planning, have to design a set of test
cases that will detect defects present
 Some criteria needed to guide test case selection
 Two approaches to design test cases
 functional or black box
 structural or white box
 Both are complimentary; we discuss a few
approaches/criteria for both
Black Box testing
 Software tested to be treated as a block box
 Specification for the black box is given
 The expected behavior of the system is used to design
test cases
 Test cases are determined solely from specification.
 Internal structure of code not used for test case design
Black box testing…
 Premise: Expected behavior is specified.
 Hence just test for specified expected behavior
 How it is implemented is not an issue.
 For modules:
 specification produced in design specify expected
behavior
 For system testing,
 SRS specifies expected behavior
Black Box Testing…
 Most thorough functional testing - exhaustive
testing
 Software is designed to work for an input space
 Test the software with all elements in the input space
 Infeasible - too high a cost
 Need better method for selecting test cases
 Different approaches have been proposed
Equivalence Class Partitioning
 If a tester is viewing the software-under-test as a black
box with well defined inputs and outputs, a good
approach to selecting test inputs is to use a method
called equivalence class partitioning
 equivalence class partitioning results in a partitioning
of the input domain of the software under test. The
technique can also be used to partition the output
domain, but this is not a common usage
Equivalence Class partitioning
 Divide the input space into equivalent classes
 If the software works for a test case from a class the it is
likely to work for all
 Can reduce the set of test cases if such equivalent classes
can be identified
 Getting ideal equivalent classes is impossible
 Approximate it by identifying classes for which different
behavior is specified
Equivalence Class Examples
In a computer store, the computer item can have a
quantity
between -500 to +500. What are the equivalence
classes?
Answer: Valid class: -500 <= QTY <= +500
Invalid class: QTY > +500
Invalid class: QTY < -500
Equivalence Class Examples
Account code can be 500 to 1000 or 0 to 499 or 2000 (the
field type is integer). What are the equivalence classes?
Answer:
Valid class: 0 <= account <= 499
Valid class: 500 <= account <= 1000
Valid class: 2000 <= account <= 2000
Invalid class: account < 0
Invalid class: 1000 < account < 2000
Invalid class: account > 2000
Equivalence class partitioning…
 Rationale: specification requires same behavior for
elements in a class
 Software likely to be constructed such that it either
fails for all or for none.
 E.g. if a function was not designed for negative
numbers then it will fail for all the negative numbers
 For robustness, should form equivalent classes for
invalid inputs also
Equivalent class partitioning
 Every condition specified as input is an equivalent
class
 Define invalid equivalent classes also
 E.g. range 0< value<Max specified
 one range is the valid class
 input < 0 is an invalid class
 input > max is an invalid class
 Whenever that entire range may not be treated
uniformly - split into classes
Equivalence class
 Once eq classes selected for each of the inputs, test
cases have to be selected
 Select each test case covering as many valid equivalence
classes as possible
 Or, have a test case that covers at most one valid class for
each input
 Plus a separate test case for each invalid class
Example
 Consider a program that takes 2 inputs – a string s and
an integer n
 Program determines n most frequent characters
 Tester believes that programmer may deal with diff
types of chars separately
Example
 Describe a valid and invalid equivalence classes
Example..
Example…
 Test cases (i.e. s , n) with first method
 s : str of len < N with lower case, upper case, numbers,
and special chars, and n=5
 Plus test cases for each of the invalid eq classes
 Total test cases: 1 valid+3 invalid= 4 total
 With the second approach
 A separate str for each type of char (i.e. a str of numbers,
one of lower case, …) + invalid cases
 Total test cases will be 6 + 3 = 9
Advantages of Equivalence Class Partitioning
White box testing
 Black box testing focuses only on functionality
 What the program does; not how it is implemented
 White box testing focuses on implementation
 Aim is to exercise different program structures with the
intent of uncovering errors
 Is also called structural testing
 Various criteria exist for test case design
 Test cases have to be selected to satisfy coverage
criteria
Types of structural testing
 Control flow based criteria
 looks at the coverage of the control flow graph
 Data flow based testing
 looks at the coverage in the definition-use graph
 Mutation testing
 looks at various mutants of the program
Testing
 Testing only reveals the presence of defects
 Does not identify nature and location of defects
 Identifying & removing the defect => role of debugging
and rework
 Preparing test cases, performing testing, defects
identification & removal all consume effort
 Overall testing becomes very expensive : 30-50%
development cost
Levels of Testing
 The code contains requirement defects, design defects,
and coding defects
 Nature of defects is different for different injection
stages
 One type of testing will be unable to detect the
different types of defects
 Different levels of testing are used to uncover these
defects
Acceptance testing
User needs
Requirement
specification
System testing
Design
Integration testing
code
Unit testing
Testing
Unit Testing
 Different modules tested separately
 Focus: defects injected during coding
 Essentially a code verification technique, covered in
previous chapter
 UT is closely associated with coding
 Frequently the programmer does UT; coding phase
sometimes called “coding and unit testing
Integration Testing
 Focuses on interaction of modules in a subsystem
 Unit tested modules combined to form subsystems
 Test cases to “exercise” the interaction of modules in
different ways
 May be skipped if the system is not too large
System Testing
 Entire software system is tested
 Focus: does the software implement the requirements?
 Validation exercise for the system with respect to the




requirements
Generally the final testing stage before the software is
delivered
May be done by independent people
Defects removed by developers
Most time consuming test phase
Acceptance Testing
 Focus: Does the software satisfy user needs?
 Generally done by end users/customer in customer
environment, with real data
 Only after successful AT software is deployed
 Any defects found,are removed by developers
 Acceptance test plan is based on the acceptance test criteria
in the SRS
Other forms of testing
 Performance testing
 tools needed to “measure” performance
 Stress testing
 load the system to peak, load generation tools needed
 Regression testing




test that previous functionality works alright
important when changes are made
Previous test records are needed for comparisons
Prioritization of testcases needed when complete test
suite cannot be executed for a change
Test Plan
 Testing usually starts with test plan and ends with
acceptance testing
 Test plan is a general document that defines the scope and
approach for testing for the whole project
 Inputs are SRS, project plan, design
 Test plan identifies what levels of testing will be done, what
units will be tested, etc in the project
Test Plan…
 Test plan usually contains
 Test unit specs: what units need to be tested separately
 Features to be tested: these may include functionality,
performance, usability,…
 Approach: criteria to be used, when to stop, how to
evaluate, etc
 Test deliverables
 Schedule and task allocation
Testing tools





i.
JUnit is a regression testing framework
Used by developers to implement unit tests in Java
Goal: Accelerate programming and increase the quality of code.
Part of XUnit family (HTTPUnit, Cactus), CppUnit
Why test? Why Junit?
Automated tests prove features
ii.
Tests retain their value over time and allows others to prove
the software still works (as tested).
iii.
Confidence, quality, sleep
iv.
v.
Effective, open source, integrated
Get to code sooner by writing tests
Selenium
Selenium is a robust set of tools that supports rapid
development of test automation for web-based
applications.
• Works anywhere JavaScript is supported
• Hooks for many other languages - Java, Ruby, Python
• Can simulate a user navigating through pages and then
assert for specific marks on the pages
• All you need to really know is HTML to start
using it right away
Maven Build tool
 A Java project management and integration build tool.
 Based on the concept of XML Project Object Model





(POM).
Originally developed for building Turbine.
A small core with numerous plugins (in Jelly).
One level above ant.
Make  ant  Maven
(Assembly  C  C++)