INFO5990 Professional Practice in IT
Software Testing
1. Software Quality: What makes a program ‘good’?
When we talk about a ‘good’ computer program we are referring to several desirable qualities that such a
program should display. Three essential qualities are correctness, reliability and robustness.
Correct
The program behaves according to the functional requirements specified and will produce the right
result for any given set of inputs.
Reliable
The program behaves as expected on every occasion and over any specified period of time no matter
how long. It is possible for a program to become unreliable over time even though it starts off
apparently correct.
Robust
The program behaves in a predictable and controllable fashion even if the input is not what the
developer anticipated and perhaps not even valid.
A program may be correct but not robust if, for example, some particular value of input causes it to
terminate prematurely (‘crash’), such as when division by zero is attempted or some non-numeric
value is input where numeric values are expected..
2. Other important qualities of programs
In addition to the three key qualities there are several others that developers aspire to incorporate in their
programs. Some of these are discussed below.
Useful
Does the program accomplish some useful task? Is the output something that the user can use? Can the
output be used as input to some other program? Is the functionality duplicating something available
elsewhere, i.e. is it re-inventing the wheel?
Usable
With most software, all we ever see is the user interface. It is the interface that determines how
‘usable’ the program is. Is it easy to figure out how to use the program without having to be shown,
and does it require the least amount of input from you as possible?
Some people use the term ‘user friendly’ in place of ‘usable’, but usable is a more meaningful term.
Programs can be useful even though they are not so easy to use. We can get used to them. Many
people prefer the unix operating system to Windows, even though the latter is much more ‘usable’ with
its ‘point and click’ user interface.
Maintainable
Can the developers easily modify the program to take into account new or changed user requirements
without having to seriously rework and test the program? This has much to do with the modular
architecture of the program.
Efficient
Does the program run to completion in the minimum possible time, using the least necessary amount
of resource? Does it implement the most efficient algorithm for the task?
Scalable
Will the program continue to behave in an acceptable manner if the volume of input is increased
without limit? If the size of numbers involved, for example distances to stars in the solar system, or
number of records involved, such as the number of pages on the internet indexed by Google, a
becomes very large, will the program continue to perform satisfactorily? The scalability of a program
can be significantly affected by its efficiency.
1
Software Testing Supplement
INFO5990 Professional Practice in IT
Part 1: Design of Test cases
The key to effective and efficient software testing is to have well designed ‘test cases’. In this exercise you
prepare test cases for two different software programs and in the second part you will make use of your test
cases to test the software. Since you are not expected to know the inner workings of these programs you will
have to employ a ‘black box’ testing approach.
3. Mass calculation for balsa
Since there is a mass constraint in your team project it may useful to be able to calculate the mass of a piece of
balsa, given its density and linear dimensions.
The density of balsa wood can vary quite widely from as little as 4 pounds/cubic foot to as much as 24
pounds/cubic foot. Quality model building material usually lies between 8 and 12 pounds/cubic foot.
You may have to do some experimental work to determine the density of your balsa wood. The conversion
factor between British and metric units is 1 pound/Cubic Foot = 0.0160187 g/cm3.
Determine as many test cases as you think are necessary to test a program which accepts the following inputs
and calculates the mass of balsa:



Density in pounds/cubic foot (lb/ft3)
Length and breadth in centimetres (cm)
Thickness in millimetres (mm)
The program should terminate if a value of 0 is entered for the density.
Complete the following test case table to test a program that can be used to calculate the mass of a piece of
balsa wood given this input. The first equivalence class is filled in for you. You can find a simple calculator
under your Start menu, Programs>Accessories.
Table 1: Test Cases for Mass of Balsa
ID
1.
2.
3.
4.
Density
(lb/ft3)
06 (zero)
.
Length
(cm)
-
Breadth
(cm)
-
Thickness
(mm)
-
Expected Output
Program terminates
7
.
5.
4. Calculating tax payable
The tax payable on a particular income can be calculated according to rules set down by The Australian Tax
Office. You can find the details for the tax year 2009-2010 at the following url:
http://www.ato.gov.au/individuals/content.asp?doc=/content/12333.htm
Table 2 sets out the rules for determining tax payable for individuals who:


are residents of Australia for tax purposes for the whole financial year, and
did not leave full-time education for the first time during the financial year.
Other circumstances may require more complex calculations. Note that the taxable income is taken as a whole
number, ignoring any number of cents. Note also that the tax payable does not include the Medicare levy of
1.5%.
2
Software Testing Supplement
INFO5990 Professional Practice in IT
Table 2: Tax rates 2009 - 10
Taxable income
Tax on this income
0 – $6,000
Nil
$6,001 – $35,000
15c for each $1 over $6,000
$35,001 – $80,000
$4,350 plus 30c for each $1 over $35,000
$80,001 – $180,000
$17,850 plus 38c for each $1 over $80,000
$180,001 and over
$55,850 plus 45c for each $1 over $180,000
1.
Determine suitable equivalence classes for the tax calculation rules specified above. You need consider
only valid input (that is, positive whole numbers) for taxable income up to and including $1,000,000.
Don’t forget to include boundary cases and any special cases. Draw up a table in your Lab Notebook.
Remember you only need to choose one value from each equivalence class. Be sure to choose a value
that makes calculations as simple as possible, for example an income of $35,100 means that the tax
payable will be $4,350 + 30 = $4,380. You don’t need a calculator.
2.
Complete Table 3 with test cases to test a program that is supposed to calculate the tax payable
according to the rules in Table 2.. The first two equivalence classes are filled in for you.
Table 3: Test Cases for Tax Payable
ID
1.
Equivalence Class Description
Termination
Input value
0
Expected Output
Program terminates
2.
Income less than $6000
5,000
Nil or 0
3.
4.
5.
6.
Part 2: Testing Programs
Having designed your test cases you are ready to test several computer programs purporting to implement the
functionality described above. You will have to decide whether or not they display the three desirable
properties of computer programs: correct, reliable and robust.
1. Have the right mind set
Before you start on software testing this week take some time to consider the following issues:
1.
When you find a program that gives the ‘correct’ result, check also whether it satisfies the stated
specifications?
2.
Is the program also useful? Is it useable? Is the interface ‘user friendly’?
3.
When you find a program that is ‘correct’ is it also ‘robust’? Is it ‘reliable’? How can you decide about
these two properties?
4.
When you find a program that is ‘correct’ is it also ‘bug free’? How confident can you be that there are
no further hidden bugs and/or defects present in such a program?
5.
List as many fundamentally different kinds of programs that you know about. Include computer games,
real time control systems, flight simulators, accounting systems, word processors, spread sheets for
starters. Consider what different kinds of inputs they each have and how such programs might be tested.
3
Software Testing Supplement
INFO5990 Professional Practice in IT
6.
Can the testing procedure which you use in this lab, using test cases, be applied to testing all kinds of
programs with various kinds of inputs? What limitations might there be?
2. Download files required for this exercise
Login using your SIT identifier and create a folder called Week08 under your U:\INFO5990 folder. From the
course web site Download tab find the files required for Week 8 and save these files to your Week08 folder.
Files downloaded for this week are seven executable files: BalsaMs1.exe, BalsaMs2.exe, and BalsaMs3.exe,
TaxCalc1.exe, TaxCalc2.exe, TaxCalc3.exe and TaxCalc4.exe.
3.
Mass calculation for balsa
Program Specification
The program will calculate and display the mass in grams of a piece of balsa wood given the Density in
pounds/cubic foot (lb/ft3), Length and Breadth in centimetres (cm), and Thickness in millimetres (mm).
If a value of 0 is entered for the density, the program terminates.
Test the programs BalsaMS1.exe, BalsaMS2.exe, and BalsaMS3.exe using your test cases from last week and
record your findings in the Table 1 below. In the Comments column indicate as far as you can what ‘bugs’ you
have uncovered. You will notice that the program files are .EXE files, which means that you can execute them
simply by double clicking on the object in Windows Explorer.
Table 1: Mass of Balsa Program Testing
Program
Correct
(Y/N)
Reliable
(Y/N)
Robust
(Y/N)
Comments
BalsaMs1.exe
BalsaMs2.exe
BalsaMs3.exe
4.
Calculating tax payable
Program Specification
The program will calculate and display the tax payable in dollars and cents given the Taxable Income as
a whole number of dollars for input values between 0 and $1,000,000 according to the tax rates
specified by the Australian Tax Office for the tax year 2007-2008.
If a value of 0 is entered for the taxable income, the program terminates.
Test the programs TaxCalc1.exe, TaxCalc2.exe, TaxCalc3.exe and TaxCalc4.exe using your test cases from
last week and record your findings in the Table 2 below. In the Comments column indicate as far as you can
what ‘bugs’ you have uncovered. Once again, double click on the objects in Windows Explore to execute them.
Table 2: Tax Payable Calculation Testing
Program
Correct
(Y/N)
Reliable
(Y/N)
Robust
(Y/N)
Comments
TaxCalc1.exe
TaxCalc2.exe
TaxCalc3.exe
TaxCalc4.exe
4
Software Testing Supplement
INFO5990 Professional Practice in IT
5.
Grade point average
Program Specification
Given the student Id of a student the program will calculate and display the grade point average (GPA)
achieved according to the rules given. Each student result, stored in a database, will include student Id,
unit of study code and mark out of 100. The number of credit points associated with each unit will also
be stored in the database.
The test programs will be implemented in the form of a Microsoft Access application. For the purposes of
testing, each program will be executed by means of a ‘test harness’, which also provides facilities to set up test
cases and store them in the database.
Using a ‘Test Harness’ to test programs
In cases where the program to be tested is not a ‘stand alone’ program and cannot be executed in the way you
did in the previous examples it is necessary to construct a ‘test harness’. This is a program that provides an
environment within which the test programs can be executed and tested. The application system
Lab10SWTesting.mdb that you downloaded this week is such a test harness.
Operating the test environment
1.
To begin testing locate the Access application Lab10SWTesting.mdb in Windows Explorer and double
click on it. The application will open with the ‘Main switch board’ as shown in Figure 1.
Should the application halt at any time you can restart it by double clicking on the form called
Switchboard.
Figure 1: Main switch board
2.
Figure 2: Set up Test Cases screen
On choosing the first option, there are two processes available (Figure 2). Select the first, “Enrol
Students in Units. The resulting screen is shown in Figure 3.
Select student from
drop down box
Select unit of study
from drop down
box
5
Software Testing Supplement
INFO5990 Professional Practice in IT
Figure 3: Enrol Students in Units screen
Select a student record from the drop down box. These have ID values 1, 2, 3, … 10. Then select as
many units of study as are required for the particular test case. The first three test cases have already
been entered. Repeat for each test case you have designed. Note that results are entered by unit, in the
next screen.
3.
On returning to the “Set up Test Cases” screen (Figure 2) select the second option. The resulting screen
is shown in Figure 4.
Figure 4: Record Results by Unit
In this form enter the marks for each student enrolled in the unit of study selected. Repeat for each unit
of study that you have included in your test cases.
4.
You are now ready to test five programs that are supposed to calculate the grade point average. Choose
each program in turn from the “Lab 10 Program Testing Screen” accessed from the main switch board.
The interface for the test harness is shown in Figure 5.
Select student from
drop down box
Figure 5: Appearance of the test harness
Select each test case (student Id) in turn from the drop down box and compare the value of the GPA
displayed with the value you calculated for that test case. If you press the “View Results” button you
can check on the details of the test case.
Testing the programs
Now test each of the programs GPACalc1, GPACalc2, GPACalc3, GPACalc4 and GPACalc5 using your test
cases from last week and record your findings in the Table 1 below. In the Comments column indicate as far as
you can what ‘bugs’ you have uncovered.
6
Software Testing Supplement
INFO5990 Professional Practice in IT
Table 3: Grade Point Average Calculation Testing
Program
Correct
(Y/N)
Reliable
(Y/N)
Robust
(Y/N)
Comments
GPACalc1.exe
GPACalc2.exe
GPACalc3.exe
GPACalc4.exe
GPACalc5.exe
6.
Food for thought
Take some time to consider the following issues in relation to your experience of software testing this week:
5.
When you found a program that gave the ‘correct’ result did it also satisfy the specifications? Was it
useful? Was it useable? Was it ‘user friendly’?
6.
When you found a program that was ‘correct’ was it ‘robust’? How can you test this?
7.
When you found a program that was ‘correct’ was it ‘bug free’? How confident are you that there were
no hidden bugs and/or defects present in such a program?
8.
Can this testing procedure using test cases be applied to all kinds of programs using all kinds of inputs?
What limitations might there be?
9.
List as many different kinds of programs as you can. Include computer games, real time control
systems, flight simulators, accounting systems, word processors for starters. Consider how they might
be tested.
10.
Could you consider any of the computer programs you tested ‘good’?
7
Software Testing Supplement