Dynamic Testing
Overview
•
•
•
•
•
•
•
•
•
Dynamic test
Test harness
Black box and white box test procedures
Black box techniques.
Class partition technique.
Test completeness criteria
Statement coverage.
Statement and branch coverage
Instrumentation
2
Dynamic testing
• Dynamic testing is a term used to describe the testing of the
dynamic behavior of code.
• Dynamic analysis refers to the examination of the physical
response from the system to variables that are not constant
and change with time.
• The software must actually be compiled and run;
• Dynamic Testing involves working with the software, giving
input values and checking if the output is as expected.
• These are the Validation activities. Unit Tests, Integration
Tests, System Tests and Acceptance Tests are few of the
Dynamic Testing methodologies. Dynamic testing means
testing based on specific test cases by execution of the test
object or running programs.
3
Static vs. Dynamic Testing
Sl.No Static Testing
Dynamic Testing
1
Static testing is a form of software testing In dynamic testing the software must
where the software isn’t actually used.
actually be compiled and run.
2
It is generally not detailed testing, but checks
mainly for the sanity of the code, algorithm,
or document. It is primarily syntax checking
of the code or and manually reading of the
code or document to find errors
Dynamic analysis refers to the examination
of the physical response from the system to
variables that are not constant and change
with time
3
This type of testing can be used by the
developer who wrote the code, in isolation.
Code reviews, inspections and walkthroughs
are also used.
This is the verification portion of
Verification and Validation
Some of dynamic testing methodologies
include unit testing, integration testing,
system testing and acceptance testing.
4
5
Dynamic testing is the validation portion of
Verification and Validation.
These are verification activites. Code These are the Validation activities. Unit
Reviews, inspection and walkthroughs are Tests, Integration Tests, System Tests and
few of the static testing methodologies.
Acceptance Tests are few of the Dynamic 4
Testing methodologies
Test Harness
• In software testing, a test harness or automated test framework is a
collection of software and test data configured to test a program unit by
running it under varying conditions and monitoring its behavior and
outputs.
• A basic test harness will contain at least the following elements:
application basics, test launching, and result reporting. It may also
include a graphical user interface, logging, and test case scripting.
• The typical objectives of a test harness are to:
• Automate the testing process.
• Execute test suites of test cases.
• Generate associated test reports.
• A test harness may provide some of the following benefits:
•
•
•
•
Increased productivity due to automation of the testing process.
Increased probability that regression testing will occur.
Increased quality of software components and application.
Ensure that subsequent test runs are exact duplicates of previous ones.
5
Black box testing
• Black box testing, also called functional testing and behavioral
testing, focuses on determining whether or not a program does
what it is supposed to do based on its functional requirements.
• Black box testing attempts to find errors in the external behavior of
the code in the following categories:
•
•
•
•
•
(1) incorrect or missing functionality;
(2) interface errors;
(3) errors in data structures used by interfaces;
(4) behavior or performance errors;
(5) initialization and termination errors.
• We can determine if the functions appear to work according to
specifications.
• No amount of testing can unequivocally demonstrate the absence of
errors and defects in your code.
6
White Box Testing
• White-box testing is a verification technique software engineers can
use to examine if the code works as expected
• White-box testing is testing that takes into account the internal
mechanism of a system or component (IEEE, 1990). White-box
testing is also known as structural testing, clear box testing, and
glass box testing
• Using the white-box testing techniques, a software engineer can
design test cases that
• (1) exercise independent paths within a module or unit;
• (2) exercise logical decisions on both their true and false side;
• (3) execute loops at their boundaries and within their operational
bounds;
• (4) exercise internal data structures to ensure their validity
• White-box testing is used for
• Unit testing
• Integration testing
• Regression testing
7
White Box vs. Black Box testing
1
2
Black Box
Focuses on the functionality of the system
Techniques used are :
 Equivalence partitioning
White Box
Focuses on the structure (Program) of the
system
Techniques used are:
 Basis Path Testing
 Boundary-value analysis
 Flow Graph Notation
 Error guessing
Control Structure Testing
 Race conditions
1. Condition Testing
 Cause-effect graphing
2. Data Flow testing
 Syntax testing
 Loop Testing
 State transition testing
1. Simple Loops
 Graph matrix
2. Nested Loops
3. Concatenated Loops
3
Tester can be non technical
4. Unstructured Loops
Tester should be technical
4
Helps to identify the vagueness and
contradiction in functional specifications
Helps to identify the logical and coding
issues
8
Testing Strategies
• Tests of Customer Requirements
• every single customer requirement has been tested at least once
• risky requirements are tested first (is devastating to find a critical flaw
right before delivery)
• Decision Table Testing
• are used to record complex business rules that must be implemented in
the program, and therefore tested
• Failure (“Dirty”) Test Cases
• Donald (father of computer science) sends checks to readers who find
errors in his books !!!
• Anticipating the unexpected
• Test Early and Often
• executing your test cases as soon as possible is an excellent way of getting
concrete feedback about your program.
• In order to run test cases early, programmers need to integrate the pieces
of their code into the code base often.
9
Testing Strategies
• Equivalence Partitioning
• reduce the number of test cases that need to be developed
• we don’t want to write several test cases that test the same aspect of
our program
• Equivalence partitioning divides the input domain of a program into
classes.
• For each of these equivalence classes, the set of data should be
treated the same by the module under test and should produce the
same answer.
• Test cases should be designed so the inputs lie within these
equivalence classes
• Boundary Value Analysis
• Boundary value is defined as a data value that corresponds to a
minimum or maximum input, internal, or output value specified for a
system or component
• Bugs lurk in corners and congregate at boundaries (Boris Beizer)
• Programmers often make mistakes on the boundaries of the
equivalence classes/input domain
10
Test Completeness
• How do we know when software testing is complete?
• How do you know when anything is complete?
• You measure it, and then compare that measurement against
completion objectives.
• How do we measure software testing ‘completeness’?
• Requirements Coverage
• Structural Coverage
• Architectural Coverage
• Requirements, structure and architecture are the three
software abstractions used to develop and build software.
• Together, they roughly describe the ‘size’ of the software.
• As such, they can be used as a measure of software testing.
• The completeness criteria are relatively easy to determine (e.g.
number of requirements
11
Testing Completeness
• Requirements Coverage
• Has the software been tested against all requirements for the normal
range of use?
• Has the software been tested against all requirements for abnormal or
unexpected usage?
•
•
•
•
Normal Range Coverage of Software Requirements
Normal Range Coverage of Software Design
Robustness Coverage of Software Requirements
Robustness Coverage of Software Design
• Structural Coverage
• Has each element of the software been exercised during testing?
• Code Statements
• Decisions
• Conditions and Decisions
• Architectural Coverage
• Have the actual control and data links been used during testing?
•
Data Paths
•
Control Links
12
Coverage
• Statement Coverage:
• In this the test case is executed in such a way that every statement of the code is
executed at least once.
• Branch/Decision Coverage:
• A test method which aims to ensure that each possible branch from each decision
point (e.g. "if" statement) is executed at least once, all reachable code is executed.
• A branch is the outcome of a decision, so branch coverage simply measures which
decision outcomes have been tested.
• It helps in validating all the branches making sure that no branch leads to abnormal
behavior of the application.
• Path Coverage:
• In this the test case is executed in such a way that every path is executed at least
once.
• All possible control paths taken, including all loop paths taken zero, once, and
multiple (ideally, maximum) items in path coverage technique, the test cases are
prepared based on the logical complexity measure of a procedural design.
• In this type of testing every statement in the program is guaranteed to be executed at
least one time.
13
Instrumentation
• The used tool inserts counters into the program initialized
with 0
• During program execution, counters are incremented when
they are passed
• At the end the counters are used to count the number of
passing to each point
• If a counter is 0, the according step has not been reached
14
•
•
•
•
http://www.softwaretestinggenius.com/download/HGCH3.pdf
http://www.testingreflections.com/node/view/3655
http://agile.csc.ncsu.edu/SEMaterials/BlackBox.pdf
http://agile.csc.ncsu.edu/SEMaterials/WhiteBox.pdf
15