Automated Testing
Ted Driggs (tdriggs)
What
• Verify program behavior without human
interaction
• Programmatically load and run test code
on a wide array of systems
2
Why
3
Why
4
Why
1. Bugs are bad
2. Programs are big
3. Tests are boring
5
Types of automated testing
METHODS
6
Test Harnesses
• A test harness is a program which executes
test cases.
• Most unit-testing frameworks are test
harnesses.
7
tests. classSetup
Test Harnesses
The average test harness follows a
pattern like the one at right. There
is a single method which is called
when test execution starts: This
might be opening a connection to a
database.
For each test, setup and cleanup
methods are run. These methods
ensure that each test runs from a
consistent state and that test
failures are atomic.
∀ test ∈ tests :
tests. testSetup
try test()
tests. testCleanup
tests. classCleanup
Once the tests are all run, the class
cleanup method is run. This method
ensures the class does not leave the
system in an inconsistent state.
8
Data-Driven Tests
• Use configuration files to pass data to test
methods.
• Frequently XML, but not always
9
Data-Driven Tests
At left is the XML for a data-driven
test using TAEF (the Test Authoring
and Execution Framework). Each
test row is treated like a test from
the previous test harnesses.
<TestRow>
<Int32>4</Int32>
<String>This will be read</String>
<String>by a factory class</String>
<String>and transformed</String>
</TestRow>
<TestRow>
<Int32>5</Int32>
<String>into an object</String>
<String>and will höpefully</String>
<String>find lots of bugs.</String>
</TestRow>
10
Existing software
TOOLS
11
Java
The premiere unit testing framework for Java is jUnit. Supported by
a very good Eclipse plugin, jUnit
was one of the first successful test
harnesses.
12
C# and .NET
C# is a newer language, but the
similarity to Java means similar test
capabilities.
There are two options: nUnit, which
is a port of jUnit for .NET, and the
newer Microsoft Visual Studio
Unit Test Framework. The former
was widely used until recently, but
the expanded feature set of the
latter means it is likely to become
dominant.
13
Javascript
The most widely-used option for a
unit-tests is JsUnitTest, which is
based on Prototype.js.
14
Python
In keeping with Python’s “batteries
included” philosophy, the language
comes with the unittest module. It
behaves very similarly to the ones
shown previously.
15
Source Control Integration
RECOMMENDED USE
16
Source Control
• Tests are only good as long as they are
actually being used.
• Source control can verify that new code
passes existing automation.
17
Source Control
Write
Submit
Test
Commit
18
Source Control
Write
Submit
Test
Commit
19
Bad Ideas
• Test-driven development
• Opt-in testing (devs will not remember)
20
Final Remarks
SUMMARY
21
Benefits
•
•
•
•
Prevents feature regressions
Limits risk of refactoring
Increases tests’ code coverage
Makes life easier for developer and tester
22