MIT Art Design and Technology University
MIT School of Computing, Pune
Software Engineering
Class - S.Y. (SEM-I)
Software Testing Principles
AY 2025-2026 - SEM-I
Unit IV- Software Testing Principles
Introduction to Software Testing, Principles of Testing, Testing Life Cycle,
Phases of Testing, Types of Testing, A Strategic Approach to Software
Testing, Strategic Issues, Test Strategies for Conventional Software, Test
Strategies for Object-Oriented Software, Test Strategies for WebApps,
Verification & Validation, Software Testing Fundamentals, White-Box
Testing , Basis Path Testing, Control Structure Testing, Black-Box Testing,
Model-Based Testing, Testing for Specialized Environments, Architectures,
Object-Oriented Testing Strategies, Object-Oriented Testing Methods,
Validation Testing
Test Strategies for Conventional Software
•
•
•
•
•
Top-down Testing
Bottom-up Testing
Big-bang Testing
Sandwich Testing
Example: Banking Application - test login, account summary, fund
transfer separately and then integrate.
Test Strategies for Object-Oriented Software
•
•
•
•
•
Testing Classes and Objects
State-based Testing
Cluster Testing
Integration through collaboration patterns
Example: Library Management System - test class behaviors (Book,
Member, Transaction) individually and in collaboration.
• Test Strategies for Object-Oriented Software
Initially, we were following the traditional method of testing which consists of procedures
operating on data. But now, we are following the object-oriented testing method that focuses
on the objects that are instances of classes. This change from traditional to object-oriented
made us reconsider old strategies and software testing methods. There are many advantages of
using the OO paradigm including reliability, interoperability, reusability, and extendibility.
5
Introduction to Software Testing
•
•
•
Definition: Process of evaluating software to determine if it meets
specified requirements.
Purpose: Identify defects, ensure quality, verify functionality,
performance, and security.
Importance: Saves cost and time, improves product quality and
customer satisfaction.
Principles of Software Testing
1. Testing shows the presence of defects.
2. Exhaustive testing is impossible.
3. Early testing saves time and money.
4. Defect clustering.
5. Pesticide paradox.
6. Testing is context dependent.
7. Absence-of-errors fallacy.
Software Testing Life Cycle (STLC)






Requirement Analysis
Test Planning
Test Case Development
Environment Setup
Test Execution
Test Cycle Closure
Phases of Testing
Types of Testing




Functional Testing (Smoke, Sanity, Regression)
Non-functional Testing (Performance, Usability, Compatibility)
Manual Testing
Automated Testing
A Strategic Approach to Software Testing
•
•
•
Testing as a process, not a phase.
Includes planning, test case design, execution, and result analysis.
Continuous testing through development.
Strategic Issues in Software Testing
1.
2.
3.
4.
5.
Who performs testing?
When to start testing?
How much testing is enough?
Balancing budget, time, and quality.
Use of tools and automation.
Strategic Issues in Software Testing
1. Use effective Use effective formal technical reviews formal
technical reviews as a filter prior to testing as a filter prior to testing
2. Develop a testing plan
3. Conduct Conduct formal technical reviews formal technical reviews
to assess the test strategy and to assess the test strategy and test
cases themselves.
13
Real-world Scenarios and Examples
Web application testing, a software testing technique exclusively adopted to test the applications
that are hosted on web in which the application interfaces and other functionalities are tested.
Web Application Testing – Techniques
1.
2.
3.
4.
5.
Functionality Testing
Usability testing
Interface testing
Performance testing
Security testing
Verification & Validation
Verification and Validation are the processes of investigating whether a software system satisfies
specifications and standards and fulfills the required purpose.
1.
Verification
Verification is the process of checking that software achieves its goal without any bugs. It is the
process to ensure whether the product that is developed is right or not. It verifies whether the
developed product fulfills the requirements that we have. Verification is static testing.
1.
Validation
Validation is the process of checking whether the Software Product is up to the mark, or in other
words product has high-level requirements. It is the process of checking the validation of the product
i.e., it checks whether what we are developing is the right product. It is a validation of the actual and
expected products. Validation is dynamic testing.
Verification & Validation
Verification refers to the set of activities that
ensure software correctly implements the
specific function
Validation refers to the set of activities that
ensure that the software that has been built is
traceable to customer requirements.
It includes checking documents, designs,
codes, and programs.
It includes testing and validating the actual
product
Verification is the Static testing.
Validation is Dynamic testing.
It does not include the execution of the code.
It includes the execution of the code.
Methods used in verification are reviews,
walkthroughs, inspections and desk-checking.
Methods used in validation are Black Box
Testing, White Box Testing and NonFunctional testing.
Software Testing Fundamentals
Fundamental Testing Processes:
Test Planning: Defining the scope, objectives, resources, and schedule for testing.
Test Design: Creating test cases and test scenarios based on requirements and specifications.
Test Execution: Running the tests and recording the results.
Defect Reporting: Documenting any defects found and tracking their resolution.
Test Automation: Utilizing tools to automate repetitive testing tasks for efficiency.
Continuous Integration: Integrating testing into the development process to identify issues early.
Review and Feedback: Analyzing test results and using them to improve the testing process.
White box testing
White box testing is a software testing method that involves examining the internal structure and workings of an
application, including its code, infrastructure, and integrations with external systems
Key Characteristics:
Internal Perspective:
White box testing focuses on the internal workings of the software, unlike black box testing which focuses on
functionality from the user's perspective.
Code Access:
Testers have access to the source code and design documents of the application.
Code Coverage:
White box testing aims to achieve thorough code coverage, ensuring that all or most of the code paths are tested.
Error Detection:
This method is effective in identifying errors, bugs, and vulnerabilities that might be hidden within the code.
White box testing
How it works:
Understanding the Code:
Testers analyze the application's source code, understanding its logic, control
flow, and data structures.
Test Case Design:
Based on the code analysis, testers design test cases that target specific code
paths, conditions, and data flows.
Execution and Analysis:
The test cases are executed, and the results are analyzed to verify the
correctness of the code.
White box testing
Benefits of White Box Testing:
Early Bug Detection:
Helps identify defects early in the development lifecycle, reducing the cost and effort of fixing them
later.
Improved Code Quality:
Ensures that the code is robust, reliable, and secure by testing its internal workings.
Thorough Test Coverage:
Ensures that all or most of the code is tested, leaving fewer areas for potential errors.
Increased Security:
Helps identify and fix vulnerabilities that might be exploited by malicious actors.
White box testing
Types of White Box Testing Techniques:
Statement Coverage: Ensures that each statement in the code is executed at
least once.
Branch Coverage: Ensures that each branch (e.g., if/else statements) in the
code is executed at least once.
Path Coverage: Ensures that all possible paths through the code are
executed.
Condition Coverage: Ensures that each condition in the code is evaluated to
both true and false.
Basis Path Testing
Path Testing is a white-box testing technique based on the control structure
of a program or a module. Using this structure, a control flow graph is
prepared, and the various possible paths present in the graph are executed as
a part of testing.
Path Testing Process
Path Testing is based on the control structure of the program, it requires
complete knowledge of the program's structure. To design test cases using
this technique, four steps are followed:
● Construct the Control Flow Graph.
● Compute the Cyclomatic Complexity of the Graph.
● Identify the Independent Paths.
● Design Test cases from Independent Paths
Control Structure Testing
Control structure testing is used to increase the coverage area by testing
various control structures present in the program. The different types of
testing performed under control structure testing are as follows1. Condition Testing
2. Data Flow Testing
3. Loop Testing
Control Structure Testing
Condition Testing:
Concept:
● Focuses on testing the individual components of complex conditions
within a program.
● This includes testing the true and false branches of each simple condition
within a compound condition (e.g., if (a > b && c < d) { ... }).
Example:
For the condition (x > 5 && y < 10), you would test cases where x is greater
than 5 and y is less than 10, as well as cases where one or both are not. This
testing helps uncover errors like incorrect Boolean operators or relational
expressions.
Control Structure Testing
Data Flow Testing:
Concept:
Focuses on the use of variables within the program, particularly where
variables are defined and used, including nested loops and if statements.
Example:
If a variable is defined inside a loop, the tests should verify that the variable
is used correctly after the loop is executed.
Control Structure Testing
Loop Testing:
Concept:
Focuses on testing the different aspects of loops (e.g., for, while, do-while).
Example:
For a while loop, tests should cover skipping the loop entirely, executing it
once, twice, and a number of times close to the boundary conditions (e.g., n1, n, n+1, where n is the maximum number of iterations).
Control Structure Testing
Benefits of Control Structure Testing:
Improved Code Coverage:
Ensures that more of the program's logic is tested, increasing confidence in the
software's correctness.
Early Error Detection:
Helps to identify potential errors related to control flow early in the development
cycle.
Reduced Bugs:
By thoroughly testing different paths, it reduces the likelihood of unexpected
behavior or errors during runtime.
Targeted Testing:
Allows testers to focus on critical parts of the code where control flow is more
complex.
Black Box Testing
Black box testing is a software testing technique where the tester is unaware
of the internal workings of the system being tested
Types Of Black Box Testing
The testing of an application without knowing the internal code or structure;
the following are the various types of black box testing:
1.Functional Testing
Functional testing is a type of software testing in which the system is tested
against the functional requirements and specifications.
Black Box Testing
Types Of Black Box Testing
2. Regression Testing
Regression Testing is like a Software Quality checkup after any changes are
made. It involves running tests to make sure that everything still works as it
should, even after updates or tweaks to the code.
3. Nonfunctional Testing
Non-functional Testing is a type of Software Testing that is performed to
verify the non-functional requirements of the application.
Black Box Testing
Advantages of Black Box Testing
● The tester does not need to have more functional knowledge or
programming skills to implement the Black Box Testing.
● It is efficient for implementing the tests in the larger system.
● Tests are executed from the user's or client's point of view.
Disadvantages of Black Box Testing
● There is a possibility of repeating the same tests while implementing the
testing process.
● Without clear functional specifications, test cases are difficult to
implement.
● It is difficult to execute the test cases because of complex inputs at
different stages of testing.
Model Based Testing
Model Based Testing – Overview
•
“Model-based testing is black box testing process
for
derivation of
test cases from a model that
describe functional aspects of the system under test
and executing those test cases.”
is a partial description of
Model
System
can be derived from
can be run against
Executable
Tests
Abstract
Tests
see: Practical Model Based Testing- A Tools
Approach, Mark Utting & Bruno Legeard
are abstract versions of
Model Based Testing Approach
Five Main Steps
1) Model the SUT
2) Generate abstract tests from the
model.
3) Concretize the abstract
tests to make them
executable
4) Execute the tests on
the SUT and assign
verdicts.
5) Analyze the test results.
see: Practical Model Based Testing- A Tools Approach, Mark Utting & Bruno Legeard
Why Does Model-Based Testing Work?
system under test
complexity
model
speed
“… I think that less than 10 percent of most programs’ code is specific
to the application. Furthermore, that 10 percent is often the easiest 10 percent.
Therefore, it is not unreasonable to build a model program to use as an oracle.”
–Boris Beizer, Black Box Testing, p.63
Benefits of Model-Based Testing
• Easy test case maintenance
• don’t reuse just test execution but also test generation!
• Reduced costs -> more tests
• Early bug detection and Increased bug count
• Time savings
• Start automated testing from Version 0.1
Obstacles to Model-Based Testing
•
Skill set
• Need testers who can build/use a model
•
Comfort factor
• Creating a model and understanding how it is traversed to generate tests can be
technical.
•
Expectations
• Models can be a significant upfront investment
• This contradicts an Agile approach
• MBT will never catch all the bugs
•
Metrics (to be discussed later)
• Bad metrics: bug counts, number of test cases
• Better metrics: spec coverage, code coverage
Object Oriented Testing
Objectives
To cover the strategies and tools associated with object oriented
testing





Analysis and Design Testing
Class Tests
Integration Tests
System Tests
Validation Tests
analysis
design
cod
e
tes
t
A Broader View of Testing
•
•
•
•
•
Nature of OO systems influence both testing strategy and methods
Will re-use mean less need for testing? NO
In Object Oriented systems the view of testing is broadened to
encompass Analysis and Design
“It can be argued that the review of OO analysis and design models
is especially useful because the same semantic constructs (e.g.,
classes, attributes, operations, messages) appear at the analysis,
design, and code level.”
Allows early circumvention of later problems
Object-Oriented Testing
Analysis and Design:
 Testing begins by evaluating the OOA and OOD models
 Cannot be executed, so conventional testing impossible
 Use formal technical reviews of correctness, completeness and consistency
Programming:
 OO Code testing differs from conventional methods:
1. The concept of the ‘unit’ broadens due to class encapsulation
2. Integration focuses on classes and their execution across a ‘thread’ or in the context of a usage scenario
3. Validation uses conventional black box methods
 Test case design draws on conventional methods, but also encompasses special features
Criteria for Completion of Testing
When are we done testing?
1.
Testing is never done, the burden simply shifts from you to the customer
2.
Testing is done when you run out of time or money
3.
Statistical Model:




Assume that errors decay logarithmically with testing time
Measure the number of errors in a unit period
Fit these measurements to a logarithmic curve
Can then say: “with our experimentally valid statistical model we have done sufficient testing to
say that with 95% confidence the probability of 1000 CPU hours of failure free operation is at
least 0.995”
More research needs to be done into how to answer this question
Strategic Issues
Issues to address for a successful software testing strategy:
 Specify product requirements in a quantifiable manner long before testing commences.






For example, portability, maintainability, usability
State testing objectives explicitly. For example, mean time to failure, test coverage, etc
Understand the users of the software and develop a profile for each user category. Use
cases do this
Develop a testing plan that emphasizes “rapid cycle testing”. Get quick feedback from a
series of small incremental tests
Build robust software that is designed to test itself. Exception handling and automated
testing
Conduct formal technical reviews to assess the test strategy and test cases themselves.
“Who watches the watchers”
Develop a continuous improvement approach to the testing process
Testing Analysis and Design
•
Syntactic correctness:
−
•
Semantic correctness:
−
−
•
Is UML notation used correctly?
Does the model reflect the real world problem?
Is UML used as intended by its designers?
Testing for consistency:
−
−
Are different views of the system in agreement?
An inconsistent model has representations in one part that are not
correctly reflected in other portions of the model
Testing the Class Model
Revisit the CRC model and the object-relationship model. Check
that all collaborations are properly represented in both
2. Inspect the description of each CRC index card to determine if a
delegated responsibility is part of the collaborator’s definition
1.
−
3.
Example: in a point of sale system. A read credit card responsibility of a credit sale class is
accomplished if satisfied by a credit card collaborator
Invert the connection to ensure that each collaborator that is asked
for a service is receiving requests from a reasonable source
−
Example: a credit card being asked for a purchase amount (a problem)
Final Steps in Testing the Class Model
4.
5.
Using the inverted connections examined in step 3, determine
whether other classes might be required or whether responsibilities
are properly grouped among the classes
Determine whether widely requested responsibilities might be
combined into a single responsibility
−
6.
Example: read credit card and get authorization could easily be grouped
into validate credit request
Steps 1 to 5 are applied iteratively and repeatedly
Testing OO Code
class tests
system
tests
integration
tests
validation
tests
[1] Class Testing




Smallest testable unit is the encapsulated class
A single operation needs to be tested as part of a class hierarchy because
its context of use may differ subtly
Class testing is the equivalent of unit testing in conventional software
Approach:
 Methods within the class are tested
 The state behavior of the class is examined


Unlike conventional unit testing which focuses on input-process-output,
class testing focuses on designing sequences of methods to exercise the
states of a class
But white-box methods can still be applied
Class Testing Process
class
to be
teste
d
results
software
engineer
test cases
Class Test Case Design
Each test case should be uniquely identified and should be
explicitly associated with the class to be tested
2. The purpose of the test should be stated
3. A list of testing steps should be developed for each test and should
contain:
1.
A list of specified states for the object that is to be tested
A list of messages and operations that will be exercised as a consequence of the test
A list of exceptions that may occur as the object is tested
A list of external conditions (i.e., changes in the environment external to the software that
must exist in order to properly conduct the test)
e. Supplementary information that will aid in understanding or implementing the test
a.
b.
c.
d.
Challenges of Class Testing
•
Encapsulation:
−
•
Inheritance:
−
−
•
Difficult to obtain a snapshot of a class without building extra methods which display
the classes’ state
Each new context of use (subclass) requires re-testing because a method may be
implemented differently (polymorphism).
Other unaltered methods within the subclass may use the redefined method and need
to be tested
White box tests:
−
Basis path, condition, data flow and loop tests can all be applied to individual methods
within a class but they don’t test interactions between methods
Random Class Testing
1.
2.
3.
4.
•
Identify methods applicable to a class
Define constraints on their use – e.g. the class must always be initialized first
Identify a minimum test sequence – an operation sequence that defines the
minimum life history of the class
Generate a variety of random (but valid) test sequences – this exercises more
complex class instance life histories
Example:
1.
2.
3.
4.
An account class in a banking application has open, setup, deposit, withdraw, balance, summarize and
close methods
The account must be opened first and closed on completion
Open – setup – deposit – withdraw – close
Open – setup – deposit –* [deposit | withdraw | balance | summarize] – withdraw – close. Generate
random test sequences using this template
Partition Class Testing
•
•
Reduces the number of test cases required (similar to equivalence
partitioning)
State-based partitioning
−
−
•
Attribute-based partitioning
−
−
•
Categorize and test methods separately based on their ability to change the state of a class
Example: deposit and withdraw change state but balance does not
Categorize and test operations based on the attributes that they use
Example: attributes balance and creditLimit can define partitions
Category-based partitioning
−
−
Categorize and test operations based on the generic function each performs
Example: initialization (open, setup), computation (deposit, withdraw), queries (balance,
summarize), termination (close)
[2] Integration Testing
•
•
OO does not have a hierarchical control structure so conventional
top-down and bottom-up integration tests have little meaning
Integration applied three different incremental strategies
−
−
−
Thread-based testing: integrates classes required to respond to one input or event
Use-based testing: integrates classes required by one use case
Cluster testing: integrates classes required to demonstrate one collaboration
Random Integration Testing
•
Multiple Class Random Testing
1.
2.
3.
4.
For each client class, use the list of class methods to generate a series of
random test sequences. The methods will send messages to other server
classes
For each message that is generated, determine the collaborating class and
the corresponding method in the server object
For each method in the server object (that has been invoked by messages
sent from the client object), determine the messages that it transmits
For each of the messages, determine the next level of methods that are
invoked and incorporate these into the test sequence
Behavioural Integration Testing
•
•
Derive tests from the object-behavioural analysis model
Each state in a State diagram should be visited in a “breadth-first”
fashion.
−
−
−
•
Each test case should exercise a single transition
When a new transition is being tested only previously tested transitions are used
Each test case is designed around causing a specific transition
Example:
−
−
A credit card can move between undefined, defined, submitted and approved states
The first test case must test the transition out of the start state undefined and not any of
the other later transitions
[3] Validation Testing
•
•
•
•
Are we building the right product? Validation succeeds when
software functions in a manner that can be reasonably expected by
the customer.
Focus on user-visible actions and user-recognizable outputs
Details of class connections disappear at this level
Apply:
−
−
−
Use-case scenarios from the software requirements spec
Black-box testing to create a deficiency list
Acceptance tests through alpha (at developer’s site) and beta (at
customer’s site) testing with actual customers
[4] System Testing
•
•
Software may be part of a larger system. This often leads to “finger
pointing” by other system dev teams
Finger pointing defence:
1.
2.
3.
•
Design error-handling paths that test external information
Conduct a series of tests that simulate bad data
Record the results of tests to use as evidence
Types of System Testing:
−
−
−
−
Recovery testing: how well and quickly does the system recover from faults
Security testing: verify that protection mechanisms built into the system will protect from
unauthorized access (hackers, disgruntled employees, fraudsters)
Stress testing: place abnormal load on the system
Performance testing: investigate the run-time performance within the context of an integrated
system
Automated Testing
•
•
CPPUnit on SourceForge.net
Differentiates between:
−
−
•
Basic unit of testing:
−
•
Errors (unanticipated problems usually caught by exceptions)
Failures (anticipated problems checked for with assertions)
CPPUNIT_ASSERT(Bool) examines an expression
CPPUnit has a variety of test classes (e.g. TestFixture). Approach is
to inherit from them and overload particular methods
Testing Summary
•
•
•
Testing is integrated with and affects all stages of the Software
Engineering lifecycle
Strategies: a bottom-up approach – class, integration, validation and
system level testing
Techniques:
−
−
−
white box (look into technical internal details)
black box (view the external behaviour)
debugging (a systematic cause elimination approach is best)
analysis
design
cod
e
tes
t
Real-world Scenarios and Examples
Login Page:
•
- Invalid credentials => Show error message
•
- Valid credentials => Redirect to dashboard
Online Shopping:
•
- Add item to cart => Cart updates instantly
•
- Checkout without address => Prompt to add address
Flight Booking App:
•
- Select source, destination => Show available flights
•
- Payment failure => Retry or choose alternative method
Banking App:
•
- Transfer funds => Confirmation message and balance update
•
- Enter incorrect IFSC => Show validation error