Succeeding with Behavior Driven
Development (BDD) Testing and
Automation
Seattle Area Software Quality Assurance Group
Oct 18, 2012
Alan Myrvold
Google
Google Confidential and Proprietary
About me
Feb 2011 - now
Google
Test Engineer, Ads
2005 - 2011
Microsoft
SDET in Office Security + Outlook
1998 - 2005
Entrust
Test manager, development manager,
security assurance manager
1988 - 1998
Cognos
Tester, developer, test manager,
development manager
Google Confidential and Proprietary
More about me
Google Confidential and Proprietary
Buzzword bingo
ATDD
BDD
Cucumber
Cucumber - JVM
Gherkin
Jasmine
SpecFlow
Google Confidential and Proprietary
Buzzword bingo
ATDD
- acceptance test driven development
BDD
- behavior driven development
Cucumber
- a Ruby tool that supports BDD
Cucumber - JVM
- a Java tool that supports BDD
Gherkin
- the language used by Cucumber
Jasmine
- a javascript tool for BDD
SpecFlow
- a .NET tool for BDD
Google Confidential and Proprietary
Cucumber Example
Feature: Addition
In order to avoid silly mistakes
As a math idiot
I want to be told the sum of two numbers
Scenario: Add two numbers
Given I have entered 50 into the calculator
And I have entered 70 into the calculator
When I press add
Then the result should be 120 on the screen
Google Confidential and Proprietary
Given / When / Then
Given - precondition
When - action
Then - assertion on expected result
And - same action as before
Google Confidential and Proprietary
Cucumber step implementation
@When("I have entered (.*) into the calculator")
public void enterNumber(int number) {
...
}
Google Confidential and Proprietary
Scenario Outline Example
Scenario Outline: Add two numbers
Given I have entered <x> into the calculator
And I have entered <y> into the calculator
When I press add
Then the result should be <z> on the screen
Examples:
| x | y
| 2 | 3
| 0 | -1
| z
| 5
| -1
|
|
|
Google Confidential and Proprietary
Why BDD?
Clarifying requirements by example
Demystifying automated tests by using English
Demystifying repeated manual tests by emphasizing why
and what to verify.
Google Confidential and Proprietary
Some test types
Google Confidential and Proprietary
How I used BDD at Microsoft
Clarifying requirements in my test plan.
Google Confidential and Proprietary
How we are using BDD at Google
My group, DoubleClick Bid Manager, uses BDD for Java API-level
system tests and repeated manual tests.
Other groups use BDD tests for Java WebDriver tests.
We share the same framework, developed internally.
Google Confidential and Proprietary
Tools support + Books
Tools:
o
Cucumber - Ruby http://cukes.info
o Cucumber JVM
https://github.com/cucumber/cucumber-jvm
o SpecFlow - Binding business requirements to .NET code
http://specflow.org
Books
o
The Cucumber Book
o The RSpec Book
o Cucumber Recipies (beta, scheduled 3/7/2013)
All books from Pragmatic Programmers, http://pragprog.com
Google Confidential and Proprietary
When / Then elsewhere
Mockito … a Java unit testing framework
@Mock CalculationEngine engine;
Calculator calculator = new Calculator(engine);
when(engine.add(2, 2)).thenReturn(4);
calculator.parse("2 + 2 =");
assertEquals("4", caclulator.getResult());
Compare the syntax to EasyMock:
expect(engine.add(2, 2)).andReturn(4);
Google Confidential and Proprietary
describe / it
Rspec … a Ruby unit testing framework
describe Calculator, "#basics" do
it "return 4 for 2+2" do
calc = Caculator.new
calc.add(2, 2)
calc.result.should eq(4)
end
end
Jasmine … a Javascript unit testing framework
describe("calc", function() {
it("2+2 is 4", function() {
expect(calc(2, 2).toEqual(4);
});
});
Google Confidential and Proprietary
Failure mode #1 - Too implementation dependent
BAD
BETTER
Given I have entered 50
into the calculator
And I have entered 70 into
the calculator
When I press add
Then the result should be
120 on the screen
When I add 50 and 70
Then the result is 120
Google Confidential and Proprietary
Failure mode #2 - Programmatic Scripts
BAD
BETTER
When I set x to 1
And while x < 50
And set row x to "empty"
Then set the first 50 rows to
"empty"
Google Confidential and Proprietary
Failure mode #3 - Too low level
BAD
BETTER
When I go to the login page
And enter "bob" into the
username field
And enter "pass123" into
the password field
And click login
Then I am logged in as
"bob"
When I log in as "bob"
Google Confidential and Proprietary
Failure mode #4 - Not exploring interesting cases
BAD
BETTER
When I add 50 and 70
Then the result is 120
When I add 50 and 70
Then the result is 120
When I add 1e90, 0.1, and -1e90
Then the result is 0.1
Google Confidential and Proprietary
Failure mode #5 - Not using regex
BAD
BETTER
@When("Set the budget to
100")
public void setBudget()
{
...
}
@When("Set the budget to (.*)")
public void setBudget(int
amount)
{
...
}
Google Confidential and Proprietary
Failure mode #6 - Complex parsing logic
BAD
BETTER
@When("(.*) the (.*) to
(.*)")
public void doAction(String
action, String name,
String value)
{
if (action.equals("set")
&& name.equals("budget")
}
@When("Set the budget to (.*)")
public void setBudget(int
amount)
{
...
}
Google Confidential and Proprietary
Failure mode #7 - Leaking code details
BAD
BETTER
When I click the BTNREFRESH-ALL button
When I refresh all
Google Confidential and Proprietary
Failure mode #8 - Bad test architecture
BAD
BETTER
Calling entry points in
code that are fragile,
or disappear
Using supported public or test
API's
Google Confidential and Proprietary
Success tips from me
Use BDD for a small set of tests
Focus on human readability, as a domain expert using the feature
would describe a test
Google Confidential and Proprietary
Success tips from The Cucumber Book, by Matt
Wayne and Aslak Hellesoy
DAMP beats DRY
DAMP: Descriptive and meaningful phrases
DRY: Don’t repeat yourself
Declarative better than imperative
Declarative: Given I am logged in
Imperative: Log in as user “Bob”
Google Confidential and Proprietary
Questions?
http://testapprentice.com
amyrvold@google.com
Google Confidential and Proprietary