Software Testing
LAB COMPENDIUM FOR
SOFTWARE TESTING
TDDD04
LAB SERIES 2015
LAB 1
AUTHORS:
Ola Leifler
Usman Dastgeer
Per Böhlin
Mohamed Abu Baker
Mohsen Torabzadeh-Tari
Kristian Sandahl
DEPT. OF COMPUTER AND INFORMATION SCIENCE
LINKÖPING UNIVERSITY, SWEDEN
SPRING 2015
TDDD04 Software Testing
2015
1 Introduction (common for lab 1 & 3)
Read through the lab instructions before going to the lab. You will benefit from it. There are 12
hours scheduled in the labs (PC PULs). The total time expected to spend on this lab series,
including the reflection reports (see below), is approximated to about 50 hours. This mean that
you are expected to put in a lot of hours on your own. Coming prepared to the scheduled hours
will make things better for all of you. You will be able to ask intelligent questions and we (the
lab assistants) can spend our time giving relevant answers.
The lab should be conducted in pairs (two students). Pair programming1 should be practiced
during the lab and is considered part of the exercise. Studies suggest that pair programming is
most efficient when switching driver (the person typing) every 20 minutes. It is not necessary
for you to switch that often but try to switch every hour or at least after each completed task.
The lab series consists of two labs, each broken down into a number of exercises. The first lab
(exercises 1-4) is expected to take about 16 hours (4 of the scheduled hours), including writing
the reflective report. The second lab (exercises 1-4) is expected to take about 34 hours (8 of the
scheduled hours) including writing the reflective report. Each exercise is to be demonstrated to
a lab assistant. After completing all the exercises for a lab, code as well as a reflective report
should be submitted to your lab assistant for final review. Details on requirements and
submission are presented at the end of the labs’ descriptions.
Sharing working code solutions between groups, or directly copying complete solutions from
the Internet is not allowed and is considered cheating and can result in a report to the
disciplinary committee 2.
The programming exercises are fairly simple but you are expected to have at least basic
understanding of C++, Java and object oriented programming. Familiarity to Visual Studio and
Eclipse is also expected. If you feel uncertain about these areas you might have to spend some
extra time catching up. Please ask your lab assistant if you need help finding suitable resources.
Important! Due to the large number of students in this course, we do not have time to handle
multiple resubmissions. Therefore, please take great care to make the first version good. There
is only 1 resubmission opportunity and in case someone misses the resubmission deadline or
require a 2nd resubmission, he/she would have to wait until the possible retake period in
August.
1 Read more (not mandatory) about pair programming in Kent Beck’s book Extreme
Programming Explained – Embrace Change (2002 –2nd ed.); Williams’ and Kessler’s paper
“All I Really Need to Know about Pair Programming I Learned In Kindergarten” (1999)
http://collaboration.csc.ncsu.edu/laurie/Papers/Kindergarten.PDF; or on Wikipedia:
http://en.wikipedia.org/wiki/Pair_programming
2 The disciplinary committee decides on matters regarding cheating and other kinds of
misconduct. Cheating can result in a formal written warning or up to six months suspension
from all university activities –including lectures and exams.
1
TDDD04 Software Testing
2015
1.1 Deadlines
Deadlines for the two labs will be posted on the course web page.
http://www.ida.liu.se/~TDDD04/labs/index.en.shtml
1.2 Setting up the Lab Environment
How to setup the lab environment is described on the course web site. Please see
Labs Homepage: http://www.ida.liu.se/~TDDD04/labs/index.en.shtml
Helping material/hints: http://www.ida.liu.se/~TDDD04/labs/extra_lab_instructions.shtml
2
TDDD04 Software Testing
2015
Lab 1: Unit Testing, Coverage Measurements, and GUI
Testing.
1.3 Introduction and preparations
There are many testing strategies and testing tools and you will get acquainted with a few of
them in this lab. As part of the preparations you should read the included references to get
yourself familiar with the tools and practices.
The tools and practices used for lab 1 are:

CppUnit, unit testing framework for C++ (exercise 1 & 2)
Read: http://msdn.microsoft.com/en-us/library/hh694604.aspx
We will use the CppUnitTestFramework built in Visual Studio 2013. Follow the above link and
make yourself familiar with the syntax.

Test Driven Development/Design (TDD), a development practice (exercise 3)
Read: http://en.wikipedia.org/wiki/Test-driven_development 3 and
listen to: http://www.hanselminutes.com/default.aspx?showID=164 or
read the transcript at: http://ebookbrowse.com/hanselminutes-0146-pdf-d187778521

JUnit, unit testing framework for Java (exercise 4)
R e a d : http://junit.org/,
http://www.cavdar.net/2008/07/21/junit-4-in-60-seconds/,
http://junit.sourceforge.net/doc/faq/faq.htm
and
 EclEmma, code coverage measurement tool for Java (exercise 4)
A code coverage plugin for Eclipse that uses the JaCoCo coverage instrumentation framework.
Read: http://www.eclemma.org/
http://www.eclemma.org/jacoco/trunk/doc/counters.html

Abbot, GUI testing framework for Java (exercise 4)
Read: http://abbot.sourceforge.net/doc/overview.shtml
Specific versions of JUnit, djUnit and Abbot are included in the lab code skeleton that you will
use. (see Setting up the Lab Environment).
The tools included in this lab are not necessarily of the latest version. The versions used works
well together and are good enough to illustrate the higher principles and strategies that the labs
are trying to convey.
3 Students who want to learn more (not mandatory) should read Kent Beck’s book Test
Driven Development: by example (2003) and Roy Osherove’s book The Art of Unit Testing
(2009).
3
TDDD04 Software Testing
2015
Lab 1 consists of four exercises:
1. Black Box Testing Triangle Program: Use black box testing to test a program that shows
whether three entered side-lengths make a triangle and if so what kind of triangle.
2. White Box Unit Testing Triangle Class: Use white box testing to test a class that is used
to determine if a triangle is equilateral (all sides are of equal length), isosceles (two “legs”
are of equal length), or scalene (none of the sides are of equal length). The same class is
used in the first exercise. The exercise is done in C++.
3. Test Driven Development of a Text Formatter: Use Test Driven Development (TDD) to
develop a class that provides support for making text centered or shifted to the right or left.
The exercise is done in C++.
4. GUI Testing Temperature Converter Application: Try out GUI testing by testing a small
application in which temperatures can be converted between centigrade (Celsius) and
Fahrenheit.
OBS: In Lab 1, you can do all 4 exercises at your own and can demonstrate at the end.
4
TDDD04 Software Testing
Exercise 1.
2015
Black Box Testing Triangle Program
The purpose of this lab is to get a feel for the difficulty of achieving full test coverage when
doing black box testing. You will not be rated on the coverage achieved so there is no need to
cheat and look at the source code before hand. The important thing to bring from this exercise
is how the process differs from white box testing which will be done in the next exercise.
To become familiar with the program, run the command prompt (cmd) and go to
TDDD04_lab1_VS2013\bin. If the build process in Visual Studio was successful, you should
find a program called triangle.exe in the bin folder.
This program evaluates if three entered side lengths form a triangle and if so, what kind:
equilateral, isosceles or scalene. Run the program a couple of times with different values for
the sides to understand how the program works. The objective of the exercise is to write tests
for the triangle class (found in CUT4 folder) only based on the above program description
and observations.
Being familiar with the program, now it is time to write a few tests. In this exercise you will
learn to write unit tests in the Visual Studio CppUnitTestFramework. Each test should be
placed in its own method named appropriately. A test can contain several asserts but should
cohere to a single feature or aspect. The setup and tear down functions are called before and
after each test.
Tests are written in the test project (TDDD04_lab1A.tests). It has already been setup for use
with CppUnitTestFramework. Test code is written in the TriangleTest class file. A few tests
have been included to illustrate the syntax (for more information see Syntax and Example).
In CppUnitTestFramework, test project is compiled as a DLL (Dynamic Link Library). You can
run the tests inside Visual Studio by Test Explorer (Test -> Windows -> Test Explorer); see
running unit tests with test explorer.
Tasks
1. Compile the triangle program in Visual Studio.
2. Run the program with several sets of values to acquire knowledge about it.
3. Based on the information you learned in the previous step, add enough tests in the
TriangleTest class file.
4. Run the Visual Studio Test Explorer and study the results of the tests.
5. Analyze code coverage (Test -> Analyze Code Coverage -> All Tests) to find out the line
coverage of your test cases. See Code Coverage in Visual Studio.
Passing Requirements
You should be able to explain to your lab assistant:
4 CUT: Class Under Test. DOC: Depended On Class
5
TDDD04 Software Testing
2015

How does the Visual Studio Test Explorer and Code Coverage work (the general
technique used to collect measurement data)?

How can you interpret the results displayed in Visual Studio Test Explorer and Code
Coverage?

Were all statements executed by your tests?

If no, why not? What were the missing test cases?

Can all statements be reached? If not, why?
6
TDDD04 Software Testing
Exercise 2.
2015
White Box Unit Testing Triangle Class
The objective of this exercise is to write unit tests to obtain 100% condition/decision coverage
of the triangle class (found in CUT folder). The code coverage results of the previous
exercise in the Visual Studio provides you with some information about the code blocks which
were not reached. However, as Visual Studio only provides information about line coverage,
you may want to either modify the code to translate this measurement to condition/decision
coverage, or keep track of condition/decision coverage manually.
Tasks
1
Write test cases for the triangle class.
Passing Requirements

Condition/decision coverage of the triangle class should be 100%.

You should be able to explain the difference between line coverage, condition coverage,
decision coverage, and condition/decision coverage. Further, you should be able to discuss
how the information provided by Visual Studio can be used to speculate about different
coverage metrics.

You should be able to explain two different ways to test exceptional code in
CppUnitTestFramework (hint: see existing code in TriangleTest.cpp).
7
TDDD04 Software Testing
Exercise 3.
2015
Test Driven Development of a Text Formatter
In this exercise you will practice the art of unit testing and Test Driven Development (TDD) or
Test Driven Design as it is also referred to as. You are expected to follow the TDD pattern of
Red/Green/Refactor5.
In short it means that you:
1
Add a new test.
2
Run all tests and see if the new test fails.
3
If it does, write just enough code to make it pass. If the new test doesn’t fail, start over
with step 1.
4
Run all tests again to verify that the added code makes the test pass.
5
Clean up the added code and verify by running the tests again.
6
Start over with step 1.
You are expected to read the included references (see ) before starting the exercise.
TDD is like any other skill, it require practice; so don’t expect to be proficient if you haven’t
tried it before. Experience show that it usually takes a few months practice before a beginner is
as productive doing TDD as writing only production code.
The object of the exercise is to develop a simple class that does string formatting. It should be
able to center, right align and left align a string, given a text width.
The code skeleton includes a project ( TDDD04_lab1B) with an abstract class
IStringFormatter that defines the interface of the class you are expected to create. Create
your class in the same project and name it StringFormatter or something similar.
Since you are doing TDD, you are expected to write unit tests before writing implementation
code. The tests should reside in their own test project. We do not mix production code and test
code!
To create a test project, right-click on the Solution in the solution explorer and select Add ->
New Project. Select Visual C++ -> Test -> Native Unit Test Project;
name it TDDD04_lab1B.tests or similar. Before proceeding further, disable precompiled
header option for the newly created project. Do it for both debug and release configurations.
Now we are going to add include and build dependencies to TDDD04_lab1B project:

Right click on the newly created project, select Build Dependencies -> Project
Dependencies, check TDDD04_lab1B.
5 The Red/Green comes from the red and green bar that is usually displayed with graphical test
runners to indicate the status of the latest test run.
8
TDDD04 Software Testing

2015
Right click on the newly created project and select Properties. G o t o
Configuration Properties -> C/C++ -> General. Add Additional
Include Directories. Include directories tells the compiler where to look for header
files. We need to add the TDDD04_lab1B project directory so the header file to the
IStringFormatter interface and your implementation can be found by the compiler.
In order to make the setting independent of where the solution is placed we can use the
macro $(SolutionDir) to get hold of the directory of the solution. The Additional Include
Directories should look something like this:
$(SolutionDir)TDDD04_lab1B;$(VCInstallDir)UnitTest\include;
Make sure to add this for all configurations, not only the active one.
A test file (unittest1.cpp) is added by default in the test project; rename it appropriately
and write test cases for your string formatter implementation. Use the syntax learnt from
Exercise 2.. Also, add your string formatter’s cpp-file from TDDD04_lab1B as an “existing
item”6 to your test project alternatively include it directly using an #include directive in the
test’s file.
String Formatter Specification
The string formatter should support center, left and right alignment of strings through the
interface specified by the abstract class IStringFormatter.
If the specified text width is larger than the provided strings length, the empty space should be
padded with characters of the type specified in the function call. Left aligned text should have
padding to the right of the provided string.
leftString(“testing”, 10, ‘-‘) should return “testing---“
If the text width specified is shorter than the length of the string, the formatter should return a
substring with the specified length.
rightString(“Software”, 4, ‘*’) should return “ware”.
Zero specified text width should return an empty string.
In the case of central alignment, if no exact central character exists such in the case of
centralizing an even length string with an odd text width (or an odd length string with even text
width), centralization should be counted one off to the right.
Example:
centerString(“EVEN”, 3, ‘*’) should return “VEN”. The underlined E is considered
the central character.
centerString(“ODD”,
6,
‘*’) should return
“*ODD**”. The underlined D is
considered the central character.
centerString input, text width, and output examples:
6 To add an existing item, right click on the project and select: add -> Existing Item.
9
TDDD04 Software Testing
2015
Input
Text width
Output
Input
Text width
Output
Input
Text width
Output
even
1
e
even
6
*even*
odd
5
*odd*
even
2
ve
even
7
*even**
odd
6
*odd**
even
3
ven
odd
2
dd
even
4
even
odd
3
odd
even
5
even*
odd
4
odd*
Tasks
1
Create a test project and set it up to use CppUnitTestFramework.
2
Create a string formatter class implementing the IStringFormatter interface.
3
Implement the string formatter class to support the features specified using TDD and with
100% test coverage.
Passing Requirements

The implemented string formatter class should abide to the specification.

The code should be well structured, have reasonable variable and function names that
explain intent and purpose. Comment the code appropriately.

Unit tests should produce 100% line coverage. You should be able to reason about the
added requirements for 100% condition/decision coverage.

The string formatter should be implemented using TDD.

You should be able to explain the main benefits of TDD.

Your test cases must verify all aspects of the specification. Consider the strategies you
have learnt from the course book and lectures.
Hints

Look at the properties for TDDD04_lab1A.tests project.

Listen to the podcast or read the transcript referenced in the introduction.
10
TDDD04 Software Testing
Exercise 4.
2015
GUI Testing Temperature Converter Application
In this task you’ll use Eclipse IDE, and you’ll write your test cases using JUnit. You will also
perform GUI testing using Abbot, and get familiar with how coverage measurements can be
performed in Java using EclEmma.
Configuring your working environment

Start Eclipse using, Start menu -> Eclipse-EclEmma. The path to your workspace should
be z:\workspace.

Download the “TDDD04_lab1c.zip” from the course home page and save it into your
workspace. Unpack it, and copy the name of the directory. After downloading, go to your
eclipse workspace (as the previous instruction), and do the following:
o
Go to File -> New -> Java Project. Choose the name of the folder
you just copied, and Eclipse will reconigze automatically that there is an existing Java
project there with the given name (“TDDD04_lab1c”).
o (In case you run into any problem) Right click TDDD04_lab1c in Package
Explorer and select Build Path -> Configure Build Path…, select
the table Libraries and click on Add JARs…, browse to your downloaded
file, and select abbot.jar, do the same and add jdom-1.1.1.jar, and
junit-4.8.2.jar and then click OK.
Now you are ready to start your work…!
T h e lab1/src/converter package contains a graphical application that converts the
representation of temperature between centigrade’s (Celsius) and Fahrenheit. It is launched by
right-clicking CelsiusConverter.java in Package Explorer and selecting Run As ->
Java Application.
Enter a temperature, e.g., 20ºC, and press Convert, which results in 68ºF being displayed. Now,
try to enter 30ºC and press the Return key. The temperature representation is converted to 86ºF.
There are code skeletons available for the exercise and you will find them in
lab1/src/converter “ t h e o n e y o u a c c e s s e d b e f o r e , w h e n y o u r a n
CelsiusConverter.java” in Package Explorer.
It is recommended to separate test files from production code, which means that you need to
have a folder called test on the same level and with the same structure of packages as in the
already existing src folder. This is already done in the code skeleton.
N o w, i t ’s y o u r f i r s t t a s k t o e x e c u t e t h e t e s t c a s e s b y r i g h t c l i c k i n
CelsiusConverterTest.java i n Package Explorer and select Coverage As ->
JUnit test.
The class CelsiusConverterTest.java contains some test cases to test the application
using the GUI testing tool Abbot. You will notice that overall coverage is 85% (line coverage).
11
TDDD04 Software Testing
2015
Eclipse will display the results of your tests in a Junit test explorer view like so:
In addition, EclEmma will provide coverage metrics for the unit you just tested:
Tasks
1. There is a known bug that the converter accepts temperatures below absolute ZERO.
Add both test cases that diagnose the bug and the code that is needed to correct it.
Entering a temperature below -273.15ºC shall result in "NA" being displayed instead of
temperatures in Celsius and Fahrenheit.
2. Existing test cases only cover clicking the button Convert. Write test cases to check
that it also works when pressing the Return key after having entered a temperature.
Additional test cases are required to have a good test for the application and to satisfy
the following requirements.
Hints

A possible solution is to use the method actionActionMap (look for it in Abbot’s
API, http://abbot.sourceforge.net/doc/api/) and send "notify-field-accept" to the text field.
Passing Requirements
Good solutions for task 1 and 2, with:

94% overall line coverage, and

100% overall branch coverage.
In more detail: 100% coverage everywhere except line coverage for CelsiusConverter
(95%) and CelsiusConverterString (75%)
12
TDDD04 Software Testing
2015
1.4 Reporting Lab 1
To pass this lab you need to both demonstrate each of the exercises to your lab assistant as well
as hand in a written reflection report of what you have learnt together with printouts of the
source code.
You should submit:
o A copy of all source code written by you or changed by you (with changes clearly marked,
and properly commented), submitted via e-mail to your assistant. One hand-in per group!
o Exercise1: batch file with test cases or list of manually entered test cases.
o Exercise2: TriangleTest.cpp
o
Exercise3: StringFormatter.h, StringFormatter.cpp, StringFormatterTest.cpp of TestProject
o
Exercise4: CelsiusConverter.java, CelsiusConverterTest.java
o and any other files created or modified.
If the group's solution (source code) or the individual reflective lab report is not “Passed” and
you were asked to hand in an updated version of it then you must also attach the old version of
the source code/lab report and a list of changes.
Demonstration
During the demonstration you are expected to show your lab assistant a working solution in
accordance to the requirements specified for the individual exercises. Both group members
should be able to answer detailed questions regarding the solution implemented as well as the
topics specified in the requirements. Both group members are expected to participate equally as
this is an oral and practical examination.
13
TDDD04 Software Testing
2015
Reflective Report
Individually, you are required to submit (via email), a short discussion (2-3 pages):

Your own thoughts regarding pros and cons of the respective test strategies (unit test, GUI
test, and coverage measurement). Write at least two pros and cons of each test strategy. In
which situations are they useful and when are they not? (approximately 3/4 a page).

Short answers (~1-4 lines each) to the following topics about different exercises:
Exercise 1
 How does Visual Studio Test Explorer and Code Coverage work (the general technique used to
collect measurement data)?
 How can you interpret the results displayed in Visual Studio Test Explorer and EclEmma?
 Were all statements executed by your BlackBox tests? If not, why not? What were the missing
test cases?
 Is it even possible to reach all statements in the given triangle.cpp with BlackBox testing?
If not, why?
Exercise 2
 What is the difference between line coverage, condition coverage, decision coverage,
condition/decision coverage?
Exercise 3
 Explain the main benefits of TDD.

A short discussion about the contrast between the results of the black box testing in
Exercise 1 and the white box testing in Exercise 2.. In Exercise 4, is there a fair reason
why the initial test cases only reached 85% coverage? Is it necessary to reach 100%
coverage? What is your opinion regarding how to treat requirements of “full coverage” in
realistic settings?

Finally, your discussion must include a comparison between testing in C++ and Java.
From your own experience; explain the differences and similarities when thinking about
unit testing and coverage measurements. Which was easy/difficult to use? Why?
Note: List any references used, such as books or websites, must be used consistently and in
accordance to normal academic standards.
14