Acceptance Test Driven Development
with SpecFlow and Friends
Christopher Bartling
Joel Levandoski
Contact information
Christopher Bartling
chris.bartling@gmail.com
Twitter: @cbartling
Joel Levandoski
joel.levandoski@gmail.com
Twitter: @joellevandoski
Topics
Acceptance Test Driven Development (ATDD)
SpecFlow for .NET
Behavior Driven Development (BDD) tool
WatiN and WebAii
Live demonstration
Questions
Acceptance Test Driven Development
ATDD
Acceptance tests are executable specifications
of desired behavior and functionality of the
system
Expressed in language of the problem domain
Automated
Regression suite
Development is driven from the outside-in
ATDD vs. TDD
TDD is extremely valuable, but you need more
Achieve great unit test coverage and still not
deliver value to the customer
ATDD focuses on complete features and
functionality
ATDD: macro view
TDD: micro view
SpecFlow for .NET
BDD testing framework
Integrates nicely with Visual Studio
Acceptance tests manifest themselves as
features and scenarios
SpecFlow generates automated test from
features
SpecFlow tests run as normal xUnit tests
Visual Studio test runner, build server
Features
Describe some piece of functionality of the
system
Maintained in a .feature file
Plain readable text
Understandable by all parties, including business
Gherkin format, popularized by Cucumber
Features contain one or more scenarios
Feature file example
Feature: Calculate Net Present Value
In order analyze the profitability of a project
As a project manager
I want to be able to calculate the project’s Net Present Value
Scenario: A project is rejected
Given a project to evaluate
When data is entered reflecting a failing project ROI scenario
Then the net present value calculator will reject the project
Scenarios
• A scenario describes a single acceptance test
for a feature
• Most features are composed of multiple
scenarios
• SpecFlow generates a test for each scenario
• The name of the test is generated from the
scenario title
Scenario example
Scenario: A project is rejected when its NPV is
negative
Given a project to evaluate
When data is entered reflecting a failing
ROI scenario
Then the net present value calculator will
reject the project
Scenario steps
• Used to compose a scenario
• Custom code to automate your application
• Describe preconditions, triggering action, and
verification of outputs for the acceptance test
• Given: preconditions
• When: triggering action
• Then: behavior and state verifications
Scenario step example
Given a project to evaluate
is matched to the following step definition binding…
[Given(@"a project to evaluate")]
public void GivenAProjectToEvaluate() {
...
}
Set up and tear down
• Attributes for before and after events
• TestRun
• Feature
• Scenario, ScenarioBlock
• Step
• Specificity achieved when used with tags
Tags
• Markers that can be applied to features and
scenarios
• Useful for selectively mixing in behavior
• Used to categorize scenarios
• Some unit test frameworks support this
categorization tagging
• Custom tags you define
Tag example
@WatiN
Scenario: Save valid sample size mid range
Given the user enters 10 as sample size
When the user selects save
Then the value is stored
Tag example used with BeforeScenario
[BeforeScenario("WatiN”)]
public void BeforeScenarioUsingWatiN()
{
...
}
Background
• Common preconditions for all scenarios in a
feature
• Contains one or more scenario steps
• Executed before any other steps in the
scenario
• SpecFlow generates a method from the
background element in the feature file in the
test class
• Invoked from each scenario test case in the test
class
Background example
Background:
Given that the welcome page is displayed
Scenario: Add a comment to a book being
reviewed
...
Scenario outlines
• Data-driven scenarios or scenario templates
• Consists of
• Scenario template specification with data
placeholders
• Set of examples providing values for placeholders
• SpecFlow generates parameterized test logic
for the scenario outline and individual test
method for each example set
Scenario outline example
Scenario Outline: Score calculation tables
Given a new bowling game
When I roll the following series:
<rolls>
Then my total score should be <total score>
Examples:
| game
| rolls
| total score |
| beginners game | 2,7,3,4,1,1,5,1,1,1,1,1,1,1,1,1,1,1,5,1 | 40
| one single spare | 2,8,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1 | 29
|
|
ScenarioContext
• ScenarioContext.Current.Pending()
• Causes the step definition to return pending
• Used to signal a stubbed step definition
• ScenarioContext.Current dictionary
• Allows you to carry context/state across step
definitions participating in scenario
WatiN
Open source browser automation tool for IE
and Firefox
Similar to Selenium, WebDriver, Watir
Browser abstraction
Find elements using CSS selectors
Interact with AJAX web elements
Handle popup dialogs (native and HTML)
WebAii
Browser automation tool from Telerik
Automates both web 2.0 and Silverlight
applications
HTML element wrappers
WaitForElement(s) support when using AJAX
Identifying elements using LINQ
Invoke JavaScript directly from test code
Best practices
• Write high-level specifications
• Specifications should remain stable over time
• Build a scripting interface for manipulating your
system under test (SUT)
• Focus specifications on isolated behaviors
• Think of specifications in Given-When-Then
format
• Use the Page Object pattern
Smells
• Specifications are constantly changing
• Specifications are composed of “sequential
command executions”
• Specifications have a lot of instrumentation or
fixture code
• Specification examples exhibit same structure
Net Present Value Calculator Demo
Used in capital budgeting
Measures the excess or shortfall of cash flows,
in present value terms, once financing terms
have been covered
Demo has two implementations
ASP.NET MVC
Silverlight
Questions?
Literature cited
http://www.concordion.org/Technique.html
http://www.telerik.com/automated-testingtools/webaii-framework-features.aspx
http://watin.org/
http://www.specflow.org/
http://code.google.com/p/selenium/wiki/PageO
bjects