Struktūrinis Testavimas
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 1
Testavimo procesas

Komponentų testavimas:
•
•
•

individualių programos komponentų testavimas,
įprastai atsakomybė už komponentų testavimą tenka
komponentų kūrėjams, išskyrus
kritines sistemas,
testai yra gaunami pagal kūrėjų patirtį.
Integravimo testavimas:
•
•
•
komponentų, sujungtų į atskiras grupes, testavimas, sukuriant
sistemas ar posistemes,
atsakomybė tenka nepriklausomoms testavimo komandoms,
testai remiasi sistemos specifikacija.
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 2
Testavimo fazės
Komponentų
testavimas
Integravimo
testavimas
Programinės įrangos kūrėjas
©Ian Sommerville 2000
Nepriklausoma testavimo
komanda
Software Engineering, 6th edition. Chapter 20
Slide 3
Introduction to Testing
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 4
First: a riddle about testing
by Brian Marick

A mathematician, a physicist, and an engineer are told: “All odd
numbers are prime.”
•
•
•
The mathematician says, “That’s silly, nine is a non-prime odd number.
The physicist says, “Let’s see, 3 is prime, 5 is prime, 7 is prime -- looks
like it’s true.”
The engineer says, “let’s see, 3 is prime, 5 is prime, 7 is prime, 9 is
prime, 11 is prime -- looks like it’s true.”
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 5
Software testing

Historically, was not popular:
•
•
•
•

with managers
with testers
with developers
with students
testing and many software innovations evolved out of
the “software crisis”
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 6
Failure Rate
Software Failure rate (ideal)
Time
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 7
Software Failure rate (real)
increased failure
rate due to side effects
Failure
rate
change
actual curve
idealized curve
Time
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 8
The Cost of Change
60-100x
1.5-6x
1x
Definition
©Ian Sommerville 2000
Development
After release
Software Engineering, 6th edition. Chapter 20
Slide 9
An error found after release costs
four times (W. Perry)




1st cost: developing program erroneously
2nd cost: system has to be tested to detect the error
3rd cost: wrong specs/code removed, correct
specs/code added
4th cost: system must be retested!
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 10
The “software crisis”





By the 1980’s, “quality” in software became a
goal; SEI was born
“software engineering” became popular
the life cycle was studied
software developers and testers began to work
together
by the 1990’s, testing tools became available
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 11
What is software testing



“The process of executing computer software in
order to determine whether the results it produces
are correct”, Glass ‘79
“The process of executing a program with the
intent of finding errors”, Myers ‘79
“Program testing can be used to show the presence
of bugs, but never their absence”, Dijkstra ‘72
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 12
What is software testing (cont)


“The aim is not to discover errors but to provide
convincing evidence that there are none, or to
show that particular classes of faults are not
present”, Hennell ‘84
“Testing is the measure of software quality”,
Hetzel ‘85
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 13
What is software testing (cont)

“The process of operating a system or component
under specified conditions, observing or
recording the results, and making an evaluation of
some aspect of the system or component.”
IEEE/ANSI, 1990
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 14
Testing is a state of mind


“If our goal is to show the absence of errors, we
will find very few of them”
“If our goal is to show the presence of errors, we
will discover a large number of them”
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 15
Time spent on testing


50%
Brooks/Myers, 1970s
80%
Arthur Andersons’ Director of testing
in North America, 1990s
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 16
Tester-to-developer ratios




1:5-10 Mainframes
i.e.,1 tester for every 5 to 10 developers
2:3 Microsoft, 1992
2:1 Lotus (for 1-2-3 Windows)
1:2 Average of 4 major companies,1992
Microsoft, Borland, WordPerfect,
Novell
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 17
Difficulties in testing software




poorly expressed requirements
informal design techniques
nothing executable until coding stage
Huge input set: consider testing software that
categorises an exam grade: 101 inputs
consider testing software that categorises two
exam grades: 101*101 inputs!
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 18
Difficulties in testing software (cont)




Exhaustive software testing is intractable
Even if all possible inputs could be identified, the
problem of identifying non-halting cases is
undecidable
Weyuker (1979) has shown that there is no algorithm
that can determine if a given statement, branch or path
will be exercised!
we’ll look at this difficulty in more detail after we
understand graphs
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 19
Struktūrinis testavimas



Dar vadinamas “baltos dėžės“ testu.
Testiniai atvejai gaunami iš programos
struktūros. Žinios apie programą
naudojamos nustatyti papildomus testinius
atvejus.
Tikslas yra išbandyti visus programos
operatorius (ne visas kelių kombinacijas).
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 20
“Baltos -dėžės” testavimas
Testavimo
duomenys
Testuoja
Gauna
Komponento
kodas
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Testavimo
rezultatai
Slide 21
Kelių testavimas



Kelių testavimo tikslas yra įsitikinti, ar testinių
atvejų rinkinys yra toks, kad kiekvienas kelias per
programą yra įvykdytas bent kartą.
Kelių tikrinime esminis yra programos
skaičiavimų (flow) grafas, kuris parodo
programos mazgus priimančius sprendimus ir
lankus, rodančius skaičiavimų valdymą.
Sąlyginiai operatoriai yra skaičiavimų grafo
mazgai.
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 22
control flow graph

Directed graph G(V, E)
•
•


V is set of vertices
E is set of edges, E = VXV
The granularity of the vertices can be an
operation, a statement or a basic block
The edges are directed; direction indicates flow
of control from one vertex to another
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 23
Programos skaičiavimų grafas


Aprašo programos skaičiavimų valdymą.
Kiekviena atšaka yra parodyta kaip atskiras kelias
ir ciklai parodyti kaip rodyklės grįžimo į ciklo
sąlygos mazgą.
Yra naudojamas kaip pagrindas ciklomatiniam
(cyclomatic ) sudėtingumui skaičiuoti:
ciklomatinis sudėtingumas = briaunų skaičius – mazgų skaičius + 2
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 24
Ciklomatinis sudėtingumas




Testų skaičius, kad patikrinti visas valdymo
instrukcijas prilygsta ciklomatiniam
sudėtingumui.
Ciklomatinis sudėtingumas lygus sąlygų skaičiui
programoje.
Naudingas, tačiau reikia naudoti atsargiai, nes
neadekvatus testavimui.
Skaičiavimų vykdymas visais keliais neatitinka
visų kelių kombinacijų vykdymo.
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 25
1
bottom > top
while bottom <= top
2
3
if (elemArray [mid] == key
4
8
5
(if (elemArray [mid]< key
6
9
7
Binary search flow graph
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 26
Nepriklausomi keliai






1,2,3,8,9;
1,2,3,4,6,7,2;
1,2,3,4,5,7,2;
1,2,3,4,6,7,2,8,9;
testiniai atvejai turi būti gauti taip, kad visi iš šių
kelių būtų įvykdyti;
dinaminis programos analizatorius gali būti
panaudotas patikrinti, ar tie keliai buvo įvykdyti.
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 27
Operatoriai, šakos, keliai

Example: Proc(x);
•
If x > 17 then
x := x - 17
If x = 13 then
(1)
(2)
(3)
(4)
x=0;
(5)
End;
(6)
Testiniai duomenys x = 30- visi operatoriai
Testiniai duomenys x = 30, x = 17- visos šakos
Testiniai duomenys x = 30, x = 17, x = 13, x = 21- visi keliai
{1,2,3,4,5,6}{ 1,2,4,6}{1,2,4,5,6}{1,2,3,4,6}
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 28
basic block (defn)

sequence of statements such that the only
entrance to the block is through the first statement
and the only exit from the block is through the
last statement
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 29
Let’s consider path testing


Construct test cases to exercise all paths through
a program.
Called “path coverage”.
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 30
Finding the square root of an
inputted value: an example
start
read number
root = square_root(number)
print root
end
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 31
Finding the square root
start
read number
if number > 0
root = square_root(number)
print root
else
print error message
endif
end
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
t
f
Slide 32
Finding the square root
start
read number
while number != 0
if number > 0
root = square_root(number)
print root
else
print error message
endif
read number
endwhile
end
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 33
How many paths?
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 34
Exam processing example



consider a program to process one exam result for
10 students
categorise the result as A, B, C, D, F
How many paths through the program?
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 35
Find the number of paths for 10 inputs
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 36
White Box Testing



Aim to test every “path” through the program
This should ensure 100% correct programs?
How many paths in the following program?
•
•
•

100 lines of C code, starts with var decls.
2 nested loops, executing between 1 & 20 times
Inside inner loop, four if-then-else statements
How long to test at 1ms per test?
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 37
Rationale for White Box Testing



Errors tend to occur in code written to handle
special cases
Our assumptions about which parts of our
programs are executed most often are frequently
wrong
Typos can occur in out-of-the-way parts of the
program as easily as in the main control flow.
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 38
Condition Testing

Branch Testing (Myers)
•

Write test cases so that the true and false branches of every
condition are executed at least once.
Domain Testing (White and Cohen)
•
•
•
For relational comparisons (e.g. x < y)
3 test cases: x < y, x = y, x > y
Choose neighbouring x and y
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 39
Loop Testing


Beizer proposed this approach to testing loops
Different types of loop:
•
•
•
•
simple
nested
concatenated
unstructured
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 40
Testing Simple Loops

Create a test case for the following situations:
•
•
•
•
•
•
•
the loop is never executed
the loop is executed once
the loop is executed twice
the loop is executed m times (m < n)
the loop is executed n - 1 times
the loop is executed n times
the loop is executed n + 1 times
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 41
Simple Loop Example
empRec := read(taxFile);
totalTax := 0;
while not eof(taxFile) do
begin
totalTax := totalTax + empRec.tax;
empRec := read(taxFile)
end;
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 42
Testing Nested Loops




Simple loop testing for inner-most loop, with outer loops
held at minimum iterations
Add other tests for excluded or out-of-range values
Work outwards, keeping outer loops at minimum
iterations and inner loops at “typical” numbers of
iterations
Continue until all loops have been tested
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 43
Testing Other Kinds of Loop

Concatenated loops
•
•

test as for simple loops if independent
test as for nested loops if dependent
Unstructured loops
•
Don’t start from here!!!
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 44
Test Coverage



Refers to the proportion of the potential paths
through a program that are covered by a given
test set.
Ideally, we want to maximise test coverage while
minimising the resources used during testing
McCabe proposed Basis Paths as solution
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 45
Basis Path Testing (1)

Depends upon view of a program
or design as a flow graph
initialise counter
read first record
while not eof do
process record
if okay then
increment counter
else
report error
endif
read next record
endwhile
1
2
3
F
11
T
4
T
F
5
6
7
8
9
10
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 46
Basis Path Testing (2)


A basis set of paths through a program executes
each instruction in that program at least once
An independent path in a basis set is one which
differs from other paths in the set in at least one
way
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 47
Independence of Paths
1

2
3
F
T
4
T
F
5
6
7
8
11
Example paths:
1,2,3,11
1,2,3,11
1,2,3,4,5,7,9,10,3,11
1,2,3,4,5,7,9,10,3,11
1,2,3,4,6,8,9,10,3,11
9
10
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 48
Cyclomatic Complexity


The number of independent paths through a graph
is called the cyclomatic complexity of the graph
v(G) = e(G) - n(G) + 1 (or sometimes 2)
•
•

e(G) = number of edges in G
n(G) = number of nodes in G
Gives an upper limit for number of test cases
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 49
Cyclomatic Complexity Example
1
e(G) = 12
n(G) = 11
2
3
F
11
T
4
T
F
5
6
7
8
v(G) = 12 - 11 + 1
=2
9
10
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 50
Basis Path Testing Cont.




Once you have determined the cyclomatic
complexity, v(G) test cases can be generated.
Determine basis set of independent paths
Prepare a test case that will cause execution of
each such path
Some basis paths can only be tested in
conjunction with others
©Ian Sommerville 2000
Software Engineering, 6th edition. Chapter 20
Slide 51