TEB3433 TFB3433
Software Testing and Quality
Assurance
Fundamentals of Software Testing
Objectives
01
• To explain the fundamental principles and axioms of software
testing and their importance in effective defect detection..
02
• To apply appropriate testing principles to select and prioritize
test cases for a given software scenario.
03
• To evaluate the strengths and limitations of different testing
approaches in achieving software quality.
Introduction
Fundamentals of Software Testing
• Software testing is a crucial, iterative process that ensures software applications
meet requirements (validation) and function correctly (verification), ultimately
enhancing quality and user satisfaction .
• It involves systematic, analysis, design, and execution of test cases to detect defects,
to prevent, identify, and resolve bugs .
• Software testing is governed by a set of fundamental principles and "axioms" —self evident truths —that guide the strategy, execution, and expectations of the QA
process .
• Understanding these ensures that testing remains efficient and realistic .
Principles and Axioms of Software Testing
• The Core Principles of Testing
• Software testing is constrained by Time, Cost, and Resources
• As E xhaustive Testing is IMPOSSIBLE, testing must be guided by well - established
principles (often called testing axioms).
• These principles help testers prioritize effort and maximize defect detection .
• Fundamental Testing Principles
• Principle 1 – Testing shows the Presence of Defects, Not their Absence
• Testing can reveal defects, but it cannot prove their absence.
• Even if no defects are found, the software may still contain undiscovered bugs.
• Testing reduces risk, not uncertainty.
• Principle 2 – Exhaustive Testing Is Impossible
• It is impossible to test: All inputs, All execution paths, All configurations
• Therefore, testers must:
• Select representative test cases
• Apply risk - based and requirement - based testing
Principles and Axioms of Software Testing
• Principle 3 – Early Testing Saves Time and Cost
• Defects found early are: Cheaper to fix, Easier to diagnose, Less disruptive
• This principle supports:
• Shift - left testing
• Early reviews and static testing
• Principle 4 – Defect Clustering
• A small number of modules usually contain the majority of defects
• This is known as the Pareto Principle (80/20 rule) in testing.
• This means that there should a focus testing effort on high - risk areas
• Principle 5 – Pesticide Parado x
• If the same tests are executed repeatedly, they stop finding new defects
• To remain effective, test cases must be:
• Reviewed
• Refined
• Extended
Principles and Axioms of Software Testing
• Principle 6 – Testing Is Context - Dependent
• Testing strategies depend on:
• Application domain (banking, healthcare, embedded systems)
• Risk level
• Regulatory requirements
• There is no one - size - fits - all testing approach.
• Principle 7 – Absence - of- Errors Fallacy
• A defect - free system is useless if It does not meet user needs or solves the wrong
problem.
• Correctness alone does not guarantee quality .
• Software must be fit for purpose
Principles and Axioms of Software Testing
• Software Testing Axioms - These are the "harsh realities" that testers must accept
to maintain a professional mindset.
• Axiom 1 - It is impossible to test a program completely .
• The number of inputs, combinations, and timing paths is effectively infinite (large) .
• The software specification is subjective
• Axiom 2 - Software testing is a risk - based exercise .
• Since we cannot test everything, we must decide what is most important to test .
• Focus on the features that are used most often or would cause the most damage if they failed .
Principles and Axioms of Software Testing
• Axiom 3 - Testing cannot show the absence of bugs .
• You can only find what you are looking for.
• Software testing is a process which can reveal the existence of a bug, but cannot reveal that there are no bugs in the software .
• Although, tests are performed to report and fix the bugs, it is not possible to guarantee bug free software .
• Axiom 4 - The more bugs you find, the more bugs there are .
• Bugs follow bugs
• If there is a high density of bugs in one area of the code, it is a signal that the logic is complex or the developer was having a "bad
day," implying more hidden issues likely exist there .
• Axiom 5 - Not all bugs found will be fixed .
• The reality about software testing is that, in spite of all the effort put in to find a bug, it might not be fixed .
• Hence, it is required that software testers to have good judgmental skills and make trade - offs, risk - based decisions for every bug
they find.
• WHY?
•
•
•
•
There is not enough time – Most projects are time - bound and the resources are usually limited
It is really not a bug - A bug found need not be a bug but could turn out to be its characteristic feature .
It is too risky to fix - Most often, it has been found that it is indeed very risky to fix bugs . Spaghetti codes .
It is just not worth it - Some of the bugs which affect the fringe features are dismissed
• Axiom 6 - It is difficult to say when a bug is indeed a bug .
• When a problem in the software is not discovered - is it a bug?
• Is it necessary for a bug to be observable?
Principles and Axioms of Software Testing
• Axiom 7 - Specifications are never final .
• The changing specifications make it difficult for complete testing to take place .
• Specifications can change due to:
•
•
•
Fierce Competition .
Rapid release cycles .
Change requirements .
• Axiom 8 - Software testers are not the most popular members of a project
• Software testers do have goals to:
•
•
Find bugs early and ensure that they are fixed as early as possible .
Ensure that they adhere to professional behavior without losing their temper
• Axiom 9 - Software testing is a disciplined and technical profession .
• Testing has now become a matured discipline and supports sophisticated techniques with good support of tools and also provides a
rewarding career for the testers .
Software Testing Strategies and Approaches
• Testing Strategies and Techniques
•
•
•
•
•
A testing strategy provides the
roadmap for the Quality Assurance (QA)
process.
It defines what will be tested , how it will be tested , and who will perform the testing .
A robust strategy ensures that the software is
reliable, scalable, and meets user requirements
It acts as a high - level plan guiding all testing activities.
That is, A Testing Strategy :
•
•
•
•
Overall testing objectives
Testing levels and types
Resource allocation
Risk mitigation approach
• Common Testing Approaches
•
Reactive Testing
• Testing is performed after development
• Defects drive future test design
• Common in traditional projects
• Limitation: Late defect discovery
•
Proactive Testing
• Testing begins early
• Focuses on defect prevention
• Emphasizes reviews and early test design
• Preferred in Agile and DevOps environments.
•
Risk - Based Testing - Testing effort is prioritized based on:
• Business risk
• Technical complexity
• Failure impact
• High - risk features receive more testing coverage.
.
Static vs Dynamic Testing
• Static Testing
•
•
•
•
Static testing involves examining software artifacts without executing the code
.
S tatic testing is carried out during the initial stage
and detects bugs at the earliest time.
The objective is to prevent defects.
Artifacts reviewed:
•
•
•
•
•
Static Testing Techniques
•
Informal Reviews
•
•
Walkthroughs
•
Inspections
•
•
Skilled people or the author of the product explains the product to the team and the scribe makes note of the review of comments .
A formal review process that follows a strict procedure to find defects . Reviewers have a checklist to review the work products . They record the defects and inform the participants to
rectify the errors .
Static code analysis
•
A systematic review of the source code of the project without executing the code . It checks the syntax of the code, coding standards, code optimization, etc .
Benefits of Static Testing
•
•
•
•
•
All the documents are presented to every team member, they just review the document and give informal comments on the documents .
No specific process is followed in this technique to find the errors in the document .
•
•
•
Requirement specification
Design documents
Source code
Test plans, test cases, test scripts
Detects defects early : Reviewing the documents and artifacts before execution, issues can be detected and resolved at an early stage, thus saving time and effort .
Reduces rework cost : Defects found during static testing are much cheaper to find and fix for the organization
Improves documentation quality : Increases development productivity due to quality and understandable documentation, and improved design .
Prevents defect propagation : identify coding errors and syntax issues resulting in cleaner and more maintainable code .
Limitation s of Static Testing
•
•
•
Detect some issues : Static testing may not uncover all issues that could arise during runtime . Some defects may appear only during dynamic testing when the software runs .
Depends on the reviewer's skills : The effectiveness of static testing depends on the reviewer's skills, experience, and knowledge .
Time - consuming : Static testing can be time - consuming when working on large and complex projects .
Static vs Dynamic Testing
• Dynam ic Testing
•
•
•
•
•
•
Dynamic testing involves executing the software and evaluating its dynamic behavior.
It includes the testing of the software for the input values and output values that are analyzed.
It is slightly complex as it requires the tester to have a deep knowledge of the system.
It provides more realistic results than static testing.
It is carried out once the entire software, a module, or unit is complete.
It focuses on:
•
•
•
Inputs and outputs
Runtime behavior
Performance and reliability
• Types of Dynamic Testing
•
•
•
Functional testing - verifies whether an application's features and functions work as expected according to the specified requirements
Non - functional testing - verifies the non- functional requirements of the application . It verifies whether the behavior of the system is as per the requirement or not.
Regression testing - re - executing a previously created test suite to verify that recent code changes haven't caused new issues
• Benefits of Dynamic Testing
•
•
•
Reveals runtime errors : It helps to reveal runtime errors, performance bottlenecks, memory leaks, and other issues that become visible only during the execution .
Verifies integration of modules : It helps to verify the integration of modules, databases, and APIs, ensuring that the system is working seamlessly .
Accurate reliability assessment : It helps to provide accurate quality and reliability assessment of the software thus verifying that the software meets the specified
requirements and functions as intended .
• Limitation s of Dynamic Testing
•
•
•
•
Time - consuming : Dynamic testing can be time - consuming in the case of complex systems and large test suites .
Requires effort : It requires significant effort in complex systems to debug and pinpoint the exact cause .
Challenging : In case of testing exceptional or rare conditions it can be challenging to conduct .
May not cover all scenarios : Dynamic testing may not cover all possible scenarios due to a large number of potential inputs and execution paths .
Static vs Dynamic Testing
Parameters
Static Testing
Dynamic Testing
Definition
Static testing is performed to check the defects in the software
without actually executing the code.
Dynamic testing is performed to analyze the dynamic behavior of the
code.
Objective
The objective is to prevent defects.
The objective is to find and fix defects.
Stage of execution
It is performed at the early stage of software development.
It is performed at the later stage of the software development.
Code Execution
In static testing, the whole code is not executed.
In dynamic testing, the whole code is executed.
Before/ After Code Deployment
Static testing is performed before code deployment.
Dynamic testing is performed after code deployment.
Cost
Static testing is less costly.
Dynamic testing is highly costly.
Documents Required
Static Testing involves a checklist for the testing process.
Dynamic Testing involves test cases for the testing process.
Time Required
It generally takes a shorter time.
It usually takes a longer time as it involves running several test
cases.
Bugs
It can discover a variety of bugs.
It exposes the bugs that are explorable through execution hence
discovering only a limited type of bugs.
Statement Coverage
Static testing may complete 100% statement coverage incomparably
in less time.
Dynamic testing only achieves less than 50% coverage.
Techniques
It includes Informal reviews, walkthroughs, technical reviews, code
reviews, and inspections.
It involves functional and non-functional testing.
Example
It is a verification process.
It is a validation process.
Manual vs Automated Testing
• Manual Testin g
• When the software is tested by people to find the bugs, it is called manual testing process.
• During this test, the tester acts as an end user and uses all the features of the software, and checks to ensure that they be
correctly.
• That is, testers design test cases, execute steps manually, and observe results.
• Advantages:
•
•
•
hav e
Flexible
Suitable for exploratory testing
Good for usability testing
• Limitation s :
•
•
•
Time - consuming
Error - prone
Not scalable for regression
• Automate d Testing
• In automated testing, a software program, commonly referred to as ‘
a testing tool’, runs the test software, provides proper inputs,
and checks the output against the expected output.
• A tester writes the test case, and the automated testing tools run the test software according to the test case without any h
uma n
intervention.
• At the end of the test, a detailed report is generated to indicate the result of any condition which is defined in the test c
ase .
• That is, a utomated testing uses scripts, tools, and frameworks.
• Advantages:
•
•
•
Faster execution
Repeatable
Reliable for large systems
• Limitations:
•
•
•
High initial cost
Requires maintenance
Not suitable for all test types
Note: Depending on the limitations of
time and cost factor, automation tests
are preferred over manual tests.
Manual vs Automated Testing
Parameters
Manual Testing
Automation Testing
Definition
In manual testing, the test cases are executed by the human tester.
In automated testing, the test cases are executed by the software tools.
Processing Time
Manual testing is time-consuming.
Automation testing is faster than manual testing.
Resources requirement
Manual testing takes up human resources.
Automation testing takes up automation tools and trained employees.
Exploratory testing
Exploratory testing is possible in manual testing.
Exploratory testing is not possible in automation testing.
Framework requirement
Manual testing doesn't use frameworks.
Automation testing uses frameworks like Data Driven, Keyword, etc.
Reliability
Manual testing is not reliable due to the possibility of manual errors.
Automated testing is more reliable due to the use of automated tools and
scripts.
Investment
In manual testing, investment is required for human resources.
In automated testing, investment is required for tools and automated
engineers.
Test results availability
In manual testing, the test results are recorded in an excel sheet so they are
not readily available.
In automated testing, the test results are readily available to all the
stakeholders in the dashboard of the automated tool.
Human Intervention
Manual testing allows human observation, thus it is useful in developing
user-friendly systems.
Automated testing is conducted by automated tools and scripts so it does not
involve assurance of user-friendliness.
Performance testing
Impractical for systematic performance testing (e.g., load, stress, spike
testing).
Supports performance testing (e.g., load, stress, spike testing with tools like
JMeter).
Batch Testing
In manual testing, batch testing is not possible.
You can batch multiple tests for fast execution.
Programming knowledge
There is no need for programming knowledge in manual testing.
Programming knowledge is a must in case of automation testing as using
tools requires trained staff.
Manual vs Automated Testing
• When to Perform Manual Testing?
• Manual testing is done when automation can't be used or isn't enough .
• K ey reasons and scenarios :
•
•
•
•
Exploratory Testing : Discovering issues in new or unclear features by exploring them .
Usability Testing : Checking if the UI is easy to use and looks good .
Ad- Hoc Testing : Doing quick, informal tests after updates or bug fixes .
Visual/GUI Testing : Checking the layout, colors, or how the site looks across different browsers .
• Manual testing should be conducted when :
• Flexibility is required : With manual tests, QA can quickly test and provide fast feedback .
• Short - term projects : It is not advisable to invest more money and effort to set up short - term projects that focus on
minor features because such setup will require huge effort and money that would be too high for such small projects .
• When testing end- user usability : Humans can use their sensibilities to understand the application behavior if the
application offers a satisfactory user journey . No machine can perform this task as humans can .
Manual vs Automated Testing
• When to Perform Automation Testing?
• When need to run repetitive tasks :
• Automated tests are the best option in scenarios where there is a requirement to run repetitive tests .
• For example, in the case of regression tests must be executed periodically to make sure that the newly added code
does not disrupt the existing functionality of the software .
• When human resources are scarce :
• Automated tests are viable and the best option to get tests executed within deadlines when there are only a limited
number of dedicated testers .
• Benefits of Automation Testing
• Finds more bugs : Automation testing helps to find more bugs and defects in the software .
• Reduce time for regression tests : Automated tests are suitable for regression tests as the tests can be executed in a
repetitive manner periodically .
• The process can be recorded : This is one of the benefits of using automation tests as these tests can be recorded and
thus allows to reuse of the tests .
• No fatigue : As automation, tests are executed using software tools so there is no fatigue or tiring factor as in manual
testing .
• Increased test coverage : Automation tests help to increase the test coverage as using the tool for testing helps to
make sure that not even the smallest unit is left for testing .
Levels of Software Testing
• Testing is performed at multiple levels, each targeting different objectives.
• Unit Testing
• This is the first step in testing your software .
• It focuses on checking individual components or functions of the application to make sure they work correctly on their
own.
• The goal here is to catch any issues early before those small components are integrated with the rest of the system .
• Unit tests help developers spot bugs early in the development process , making it easier and quicker to fix them .
• It's the first layer of defense to verify that each part of the application performs as expected .
• It ensure correctness of the smallest testable unit.
• Mainly features are :
• Testing individual functions or components .
• Ensuring correct behavior at the smallest level .
• Catching errors early .
• Integration Testing
•
•
•
•
•
After unit testing, Integration Testing takes over .
This level checks how different modules or components of the software work together .
It's important because even if individual parts work perfectly, they might face issues when interacting with one another .
Integration testing ensures that data flows correctly between modules and that they communicate seamlessly .
It helps catch problems that might arise when different parts of the system interact .
Levels of Software Testing
• System Testing
• System Testing is when you test the software as a system .
• This stage checks whether the entire system functions as expected in a real - world environment .
• It includes both functional and non- functional tests to ensure that the software meets customer needs .
• Full end- to- end testing of the software .
• Verifying both functional and non- functional requirements .
• Testing the software's behavior in real - world conditions .
• Acceptance Testing
• Acceptance Testing, also known as User Acceptance Testing (UAT), is the final test before releasing the software to the
end- users .
• In this phase, the customer or end- users verify if the software meets their needs and expectations .
• UAT is crucial because it verify the software is ready for production .
• It’s the last chance to catch any overlooked issues before deployment .
• Types include UAT, Alpha Testing, Beta Testing
• If the software passes this stage, the customer gives the green light for release .
• Validating the software against business requirements .
• Ensuring the software meets customer expectations .
• Getting final approval from the customer .
Levels of Software Testing
• Summary of Testing Levels
Level
Focus
Performed By
Unit
Individual components
Developers
Integration
Interfaces
Testers
System
Whole system
QA team
Acceptance
Business needs
End
Summary
• We have established the foundational concepts of software testing, emphasizing that:
• Testing is guided by principles, not intuition
• Early and proactive testing reduces risk
• Both static and dynamic testing are essential
• Manual and automated testing complement each other
• Testing must occur at multiple levels
• Effective software testing is systematic, risk - aware, and aligned with development goals .
Further Reading
▪ Book
▪ Foundations of Software Testing
– Rex Black, Erik van Veenendaal, Dorothy
principles, static vs dynamic testing, levels of testing, and testing strategies.
https://www.istqb.org/certifications/foundation - level
GrahamExcellent
coverage of testing
▪ Software Testing – Ron PattonHighly readable explanations of testing fundamentals, tester mindset, defect clustering, and
common testing myths. https://www.pearson.com/en - us/subject - catalog/p/software - testing/P200000006191
▪ Software Engineering – Ian SommervilleStrong theoretical grounding in verification, validation, and testing levels within
the SDLC. https://www.pearson.com/en - us/subject - catalog/p/software - engineering/P200000006193
▪ Web link s
• ISTQB Foundation Syllabus – Testing Principles
Clear and concise explanation of the seven fundamental testing principles discussed in Week 3.
https://www.istqb.org/certifications/foundation-level
• Cem Kaner – The Role of a Software Tester
Classic essays on tester mindset, exploratory testing, and defect discovery.
http://www.kaner.com
• Michael Bolton – Context-Driven Testing
Explains why testing strategies must adapt to context rather than follow rigid rules.
https://www.developsense.com
THANK YOU
© 2020 INSTITUTE OF TECHNOLOGY PETRONAS SDN BHD
All rights reserved.
No part of this document may be reproduced, stored in a retrieval system or transmitted in any form or by any means (electronic, mechanical, photocopying, recording or otherwise) without the permission of the copyright owner.