Unit Testing
What is the problem?
After a piece of code is written, usually a small class or function, it is tested independently. Then
it is added within the code base and tested in relation to other pieces of code. Finally, the requirements
are tested against the resulting system. Each level of testing aims at exposing errors as early as possible,
and is named respectively unit testing, integration testing, and acceptance testing.
Unit testing should be automated and assembled in a coverage testing framework to estimate
the quality of the code. In Java, jUnit provides unit testing (among other frameworks), djUnit or
EclEmma provides coverage testing.
How does it work?1
A unit test is an automated piece of code that invokes a unit of work in the system and then
checks a single assumption about the behavior of that unit of work.
A unit of work is a single logical functional use case in the system that can be invoked by some
public interface (in most cases). A unit of work can span a single method, a whole class or multiple
classes working together to achieve one single logical purpose that can be verified.
A good unit test is:
• Able to be fully automated
• Has full control over all the pieces running (Use mocks or stubs to achieve this isolation when
needed)
• Can be run in any order if part of many other tests
• Runs in memory (no DB or File access, for example)
• Consistently returns the same result (You always run the same test, so no random numbers, for
example. save those for integration or range tests)
• Runs fast
• Tests a single logical concept in the system
• Readable
• Maintainable
• Trustworthy (when you see its result, you don’t need to debug the code just to be sure)
• Anyone should be able to run it at the push of a button
•
•
•
•
Benefits2
Find problems early: test driven development requires unit tests to be written before the code.
Even if written after, unit tests save tremendous amount of time by finding quickly the location
of the problem.
Facilitates change: Unit testing allows the programmer to refactor code at a later date, and
make sure the module still works correctly (e.g., in regression testing). The procedure is to write
test cases for all functions and methods so that whenever a change causes a fault, it can be
quickly identified and fixed. Readily available unit tests make it easy for the programmer to
check whether a piece of code is still working properly.
Simplifies integration:
Documentation: Living documentation of the systems.
1
2
From http://artofunittesting.com/definition-of-a-unit-test/
From https://en.wikipedia.org/wiki/Unit_testing
•
•
Design: When software is developed using a test-driven approach, the unit test may take the
place of formal design. Each unit test can be seen as a design element specifying classes,
methods, and observable behaviour.
Separation of interface from implementation: Because some classes may have references to
other classes, testing a class can frequently spill over into testing another class. A common
example of this is classes that depend on a database: in order to test the class, the tester often
writes code that interacts with the database. This is a mistake, because a unit test should usually
not go outside of its own class boundary, and especially should not cross such process/network
boundaries because this can introduce unacceptable performance problems to the unit testsuite. Crossing such unit boundaries turns unit tests into integration tests, and when test cases
fail, makes it less clear which component is causing the failure.
Instead, the software developer should create an abstract interface around the database
queries, and then implement that interface with their own mock object. By abstracting this
necessary attachment from the code (temporarily reducing the net effective coupling), the
independent unit can be more thoroughly tested than may have been previously achieved. This
results in a higher quality unit that is also more maintainable.
Unit testing limitations
Testing cannot be expected to catch every error in the program: it is impossible to evaluate
every execution path in all but the most trivial programs. The same is true for unit testing.
Additionally, unit testing by definition only tests the functionality of the units themselves. Therefore,
it will not catch integration errors or broader system-level errors (such as functions performed
across multiple units, or non-functional test areas such as performance). Unit testing should be done
in conjunction with other software testing activities. Like all forms of software testing, unit tests can
only show the presence of errors; they cannot show the absence of errors.
Software testing is a combinatorial problem. For example, every boolean decision statement
requires at least two tests: one with an outcome of "true" and one with an outcome of "false". As a
result, for every line of code written, programmers often need 3 to 5 lines of test code. This
obviously takes time and its investment may not be worth the effort. There are also many problems
that cannot easily be tested at all – for example those that are nondeterministic or involve multiple
threads. In addition, writing code for a unit test is as likely to be at least as buggy as the code it is
testing. Fred Brooks in The Mythical Man-Month quotes: never take two chronometers to sea.
Always take one or three. Meaning, if two chronometers contradict, how do you know which one is
correct?
Another challenge related to writing the unit tests is the difficulty of setting up realistic and
useful tests. It is necessary to create relevant initial conditions so the part of the application being
tested behaves like part of the complete system. If these initial conditions are not set correctly, the
test will not be exercising the code in a realistic context, which diminishes the value and accuracy of
unit test results.
To obtain the intended benefits from unit testing, rigorous discipline is needed throughout
the software development process. It is essential to keep careful records not only of the tests that
have been performed, but also of all changes that have been made to the source code of this or any
other unit in the software. Use of a version control system is essential. If a later version of the unit
fails a particular test that it had previously passed, the version-control software can provide a list of
the source code changes (if any) that have been applied to the unit since that time.
It is also essential to implement a sustainable process for ensuring that test case failures are
reviewed daily and addressed immediately. If such a process is not implemented and ingrained into
the team's workflow, the application will evolve out of sync with the unit test suite, increasing false
positives and reducing the effectiveness of the test suite.
Example
Souce code: Math.java
public class Math {
static public int add(int a, int b) {
return a + b;
}
}
Test Code: MathTest.java
import static org.junit.jupiter.api.Assertions.*;
import org.junit.jupiter.api.Test;
class MathTest {
@Test
void testAddStatic() {
int num1 = 3;
int num2 = 2;
int total = 5;
int sum = 0;
sum = Math.add(num1, num2);
assertEquals(sum, total);
}
}
Note that you have to add JUnit 5 library to the build path. Eclipse proposes to do that for you if you rightclick and select new JUnit Test Case.
Now run the code as JUnit to obtain:
Now run the code as coverage testing by right-clicking the project, select Coverage As then select 2 JUnit
Test.
to obtain
i
You can see that the coverage is only 88.5%. The missing line is public class Math. That makes sense since
we called our class using the static method Math.add without implementing any object. Add a nonstatic
test as below and verify that you now reach 100% coverage.
Note that there is code duplication. Refactor your code to remove it by taking advantage of the
@BeforeEach annotation.
A list of Annotations is available at https://junit.org/junit5/docs/current/user-guide/#writing-tests
A very interesting annotation is @ParameterizedTest. It allows a very compact writing of tests, for
example:
References:i
JUnit Tutorial: http://www.vogella.com/articles/JUnit/article.html
The Art of Unit Testing: http://artofunittesting.com/
Wikipedia Unit Testing: https://en.wikipedia.org/wiki/Unit_testing
Wikipedia Mock Object: https://en.wikipedia.org/wiki/Mock_object