Ian Kelly
@kellizer
github.com/kellizer
Testing++
{
With Spock & Geb
{
Spock
The Logical Testing Framework
• Spock – A logical way to test
• Groovy based framework
• Follows Behaviour Driven Design (BDD) Paradigms
• (Tests are live documents)
• Works with IDEs/CI servers Out of the Box
• & live with existing tests
•
(And Bring Fun to Testing)
We are working
with: Spock 0.7
Geb 0.9
Groovy 2.0
Anatomy of a Spock Test
Fixture
Specification
Collaborator
Feature
System
Under
Specification
Collaborator
Collaborator
Our First Specification
/**
* System Under Specification
*/
public class Dog {
public String speak() {
return "WOOF";
}
}
All Specifications extend
spock.lang.Specification
class DogSpecification extends Specification {
def "Dogs should bark when told to speak"() {
setup:
def chip = new Dog()
expect:
chip.speak() == 'WOOF'
}
Test Execution
Feature
Specifications Hold 1 or more Feature(s)
class DogSpecification extends Specification {
def "Dogs should bark when told to speak"() {
setup:
def chip = new Dog()
expect:
chip.speak() == 'WOOF'
}
Feature Method
(The Test)
Lets Run The Test
Dogs Should Say bark when told to speak
Test Passed
Lets Run The Test
What happens if chip speaks like a duck?
Test Failed
Feature Methods
Feature Method Contain Blocks
class DogSpecification extends Specification {
def "Dogs should bark when told to speak"() {
setup:
def chip = new Dog()
expect:
chip.speak() == 'WOOF'
}
•
•
•
•
•
setup: or given: - to setup the test
expect: - to test when there are no side effects
when:/then: – apply the behaviour then validate the state
cleanup: – called at the end of the feature execution
where: – provides datasets to a feature
Spock Recognise Feature Methods and executes them
Feature Methods - Blocks
setup: (given:)
• Initialization of the feature
• Sits at the top of the method
• Can be termed as ‘given:’
• Optional & non-repeatable
Feature Methods - Blocks
when/then:
• when: stimulates the SUS
• then: validates the SUS
• Expressions are assertions
def "Check that Customer can have a zero balance"() {
setup:
def bankAccountWithOutOverdraft = new BankAccount()
when: "test a number of deposits and withdrawals"
bankAccountWithOutOverdraft.depositFunds(100)
bankAccountWithOutOverdraft.withdrawAmount(100)
then: "The account should be zero and no exception thrown"
bankAccount
bankAccount.isActiveAccount()
bankAccount.balance == 0
}
Feature Methods - Blocks
expect:
• Combines a stimulus and response
• More limited than a then: block
• Use with pure functions
def "2 should be larger than 1"() {
expect:
Math.max(1, 2) == 2
}
Stimulus
Expected Response
Feature Methods - Blocks
cleanup:
Use to free resources
• called Even if expectation is thrown
• Comparable to the java finally block
def "Test with a tmp file"() {
setup:
def file = new File("/new/file/tmp.out")
//Do stuff with the FS file
cleanup:
file.delete()
}
Feature Methods - Blocks
then: with old:
def bankAccount = new BankAccount()
def "Check that the bank account accepts a deposit"() {
when: "we deposit 100zl into the the account"
bankAccount.depositFunds(100)
then: "expect the old bal. to be 0 and new bal. to be 100"
old(bankAccount.balance) == 0
bankAccount.balance == 100
}
Captures the account balance
before applying stimulus
Feature Methods - Blocks
Descriptions on blocks:
def bankAccount = new BankAccount()
def "Check that the bank account accepts a deposit"() {
when: "we deposit 100zl into the the account"
bankAccount.depositFunds(100)
then: "expect the old bal. to be 0 and new bal. to be 100"
old(bankAccount.balance) == 0
bankAccount.balance == 100
}
Feature Methods - Blocks
then/and
def "Check that Authorised Overdrafts work as expected"() {
setup:
def bankAccountWithOverdraft = new BankAccount()
bankAccountWithOverdraft.setAllowedOverDraft(500)
when: "test a number of deposits and withdrawals"
bankAccountWithOverdraft.depositFunds(100)
and: "Note the and label"
bankAccountWithOverdraft.withdrawAmount(50)
and: "Balance should be 50 at this point"
bankAccountWithOverdraft.withdrawAmount(30)
and: "Balance should be 20 at this point"
bankAccountWithOverdraft.withdrawAmount(100)
then: "We should now be left with a balance of 80 overdrawn"
bankAccountWithOverdraft.isInOverdraft()
bankAccountWithOverdraft.balance == -80
}
Feature Methods
Thrown() within the then: block
def "Check that Customer cannot go over their allowed amount"() {
setup:
def bankAccountWithOutOverdraft = new BankAccount()
when: "test a number of deposits and withdrawals"
bankAccountWithOutOverdraft.depositFunds(100)
bankAccountWithOutOverdraft.withdrawAmount(110)
then:
thrown(InsufficientFundsException)
}
Expected outcome to be a thrown
InsufficientFundsException
Feature Methods
notThrown()
Expected outcome to be a thrown
InsufficientFundsException
Driving Tests with Data
Feature Methods - Blocks
where:
• Same Logic, Different Data
• Allows for Data Driven Feature Methods
• Last in Method – Cannot be repeated
• Data can be fed from external sources
• Support multiple formats
• | && || pipes to delimit values (Data Tables)
Feature Methods - Blocks
Where
def "Validate Age Calculation From DOB"() {
when:
Date date = new SimpleDateFormat("dd-mm-yyyy",
Locale.ENGLISH).parse(dob)
def calculatedAge = dateService.ageFromDOB(date)
calculatedAge == age
where:
Dob <<["01-02-1980”,"01-01-1990","12-12-1958"]
age << [33, 23, 55]
}
then:
Feature Methods - Blocks
Where – Data Tables
def "Validate Age Calculation From DOB"() {
when:
Date date = new SimpleDateFormat("dd-mm-yyyy", Locale.ENGLISH).parse(dob)
def calculatedAge = dateService.ageFromDOB(date)
then:
calculatedAge == age
where:
dob
|| age
"01-02-1980" || 33
"01-01-1990" || 23
"12-12-1958" || 55
}
Feature Methods - Blocks
Where – Database Driven
@Unroll //see all the tests.
def "maximum of two numbers, a=#a, b=#b so max would be #c"() {
expect:
Math.max(a, b) == c
where:
[a, b, c] << sql.rows("select a, b, c from maxdata")
}
Feature Methods - Blocks
where:
Wrong Stimulus
Test Output
@Unroll
• Reports Feature Executions independently
• Has no affect on the execution, just reporting
Mocking
Feature Methods - Mocking
Object being
mocked
Cardinality
Method
Argument
Return Object
Feature Methods - Mocking
Cardinality
Description
None
Optional
(n.._)
At least n times
n*
Exactly n times
(_..n)
Up to n times
Feature Methods - Mocking
Argument Constraint
Description
value
Argument Equals value
!value
Argument Not Equal
*_
Any number of arguments
_
Any argument
!null
Non-null Argument
Feature Methods - Mocking
Return Values
Description
>>
single return value, repeated indefinitely
>>>
multiple return values, last one repeated
indefinitely
>> { <<ACTION>> }
custom action
Feature Methods - Mocking
• Order of interaction not defined
• Extra then: guarantees ordering
def "example showing the ordered interaction"() {
given:
def subscriber1 = Mock(Subscriber) Subscriber subscriber2 = Mock()
Subscriber subscriber3 = Mock()
Publisher publisher = new StandardPublisher(subscribers: [subscriber1, subscriber2,subscriber3]);
Event event = Mock()
when:
publisher.send(event)
then:
1 * subscriber1.receive(event)
1 * subscriber2.receive(event)
// order of interaction within same then-block is not defined;
// hence, subscriber1 might be notified either before or after subscriber2
then:
// must come after all interactions in previous then-blocks;
// hence, subscriber3 must be notified after both subscriber1 and subscriber2
1 * subscriber3.receive(event)
}
Feature Methods - Mocking
def "load customer and apply airmiles but customer doesn't exist"() {
when:
airMilesProcessor.applyAirMiles(332211, 4000)
then:
1 * customerRepository.findById(332211) >> null
thrown(IllegalStateException)
}
Expected 2 invocations,
(only 1 registered)
Output
More Goodness
Fields
Created per feature method or shared
• New Instance Per Feature Method
@Shared Annotation
• Sharing Fields between Feature Methods
class DatabaseDrivenSpecification extends Specification {
@Shared sql = Sql.newInstance("jdbc:h2:mem:", "org.h2.Driver")
// normally an external database would be used,
// and the test data wouldn't have to be inserted here
def setupSpec() {
sql.execute("create table maxdata (id int primary key, a int, b int, c int)")
sql.execute("insert into maxdata values (1, 3, 7, 7), (2, 5, 4, 5), (3, 9, 9, 9)")
}
Fixture Methods
Invoke setupSpec()
Invoke setup()
Invoke feature method()
Invoke cleaup()
Invoke cleanupSpec()
Extensions
@IgnoreIf
• Ignores if a certain condition is true
@Requires
• Inverse of @IgnoreIf – only runs when condition
is true
Extensions
@Ignore
• Will ignore test(s)
• Can be place at feature or specification level
@IgnoreRest
• Ignores all other methods in the specification
@IgnoreIf
• Ignores if a certain condition is true
@Timeout
• Allows you to timeout your test if not completed in a predetermined
time
@Timeout(value = 5)
def "timeout feature method if not completed after 5 seconds"() {
def person = new Customer(name: "Fred", age: 22)
when:
person.age = 42
Thread.sleep(7000)
then:
person.age == 42
}
• Result
@AutoCleanup
• Will invoke close() on field (can be any method)
• Exception will be reported but not classed as a failure
• Preferred to cleanup()/cleanupSpec(
@AutoCleanup(quiet = true)
def input = new FileInputStream("myfile.txt")
def "some input stream tests"() {
// test uses file input stream
}
Feature Methods
Extended assert
def "Check that Customer can have a balance of total plus 1"() {
setup:
def bankAccountWithOutOverdraft = new BankAccount()
when:
bankAccountWithOutOverdraft.depositFunds(100)
then:
assert bankAccountWithOutOverdraft.balance == 100, “bank account total should be 100"
}
Test Output
More Gems - @Stepwise
• Run in Defined Order
• If 1 Test fails, the remainder get skipped
@Stepwise
class StepwiseSpecification extends Specification {
def "step 1"() {
println("step 1")
expect: true
}
def "step 2"() {
expect: true
println("step 2")
}
def "step 3"() {
expect: true
println("step 3")
}
}
More Gems - @Stepwise
@Stepwise
class StepwiseSpecification extends Specification {
def "step 1"() {
println("step 1")
expect: false
}
def "step 2"() {
expect: true
println("step 2")
}
def "step 3"() {
expect: true
println("step 3")
}
}
More Gems - @ Include/Exclude
class IncludeExcludeExtensionSpec extends Specification {
static {
//System.setProperty "spock.configuration", "IncludeFastConfig.groovy"
// Alternatively, try this:
System.setProperty "spock.configuration", "ExcludeSlowConfig.groovy"
}
@Fast
def "a fast method"() {
expect: true
}
@Slow
def "a slow method"() {
expect: true
}
def "a neither fast nor slow method"() {
expect: true
}
}
runner {
exclude Slow
}
ExcludeSlowConfig.groovy
Geb
Geb
Awesome Browser Automation
• Introduces the power of WebDriver to your tests
• Complements Spock but is Independent Project
• Uses Expressiveness of Groovy
• Has a jQuery like API (is not jQuery)
• Has Domain Modelling Support via Pages/Modules
Geb – Supported Browsers
WebDriver
• First Class Support
• Firefox
• Internet Explorer
• Google Chrome
• Mobile (Ipad/Iphone/Android)
• Remote Browsers
• Headless Browser
• PhantomJS
• W3C Standard
•
http://www.w3.org/TR/webdriver/
Geb abstracts WebDriver
Interactions
Spock & Geb
Spock
•Our Test
Specification
Geb/Spock
Adapter
• Geb
Support
for Spock
Geb DSL
•The Geb
DSL
Webdriver
•Browser
API support
Browser
• FF
• Chrome
• IE etc.
Web
Application
• What we
are
testing
Spock Test (Using Geb)
class LoginStorySpecification extends GebReportingSpecWithPause {
def "Login with an correct password"() {
given:
go "/login.html"
$("form").with {
username = 'admin'
password = 'password'
remember = true
}
when:
$("button").click()
then:
title == "Authenticated User"
}
Geb – Page Object Pattern
• Model web pages for re-useable and maintainable
code
• Reduces duplication
• Abstract the HTML implementation structure
• Changes happen
• Simply change in a single location
def "Login to The Secure Admin Server"() {
when:
to LoginPage
login("admin", "password")
then:
at AuthenticatedAdminPage
}
Geb – Page Object Pattern
class AuthenticatedAdminPage extends Page {
static content =
{
//optional content
administratorsName(required: false)
{
$("h2")
}
//stacked
users(wait: true) { $("li.span5.clearfix") }
user { i -> users[i] }
userName { i -> users[i].find("h4", 0) }
}
}
• Simply change elements in a single location
Geb – jQuery(ish) API
• Copying jQuery has many benefits
• CSS Content Lookup
• Selecting content on a page
• Traversing to/around content
• Methods for retrieving relative content
• Fluent API
• Geb (like jQuery) uses a “$” function
Geb != jQuery
Geb – jQuery(ish) API
Selecting content – DOM Searching
//match all ‘div’ elements on the page
$("div”)
//match all ‘div’ with a title attribute value of ‘section’
$("div", title: "section")
//match the first ‘div’ with the class ‘main’
$("div.main", 0)
//match all 'div' elements with the class ‘main’
$("div.main”)
Navigator Object
Geb – jQuery(ish) API
Selecting content – CSS Selector
$("div.some-class p:first[title='something']")
Geb – jQuery(ish) API
Selecting content – Index & Ranges
<p>a</p>
<p>b</p>
<p>c</p>
$("p", 0).text() == "a”
$("p", 2).text() == "c”
$("p", 0..1)*.text() = ["a", "b"]
$("p", 1..2)*.text() = ["b", "c"]
Geb – jQuery(ish) API
Index & Ranges
<p attr1="a" attr2="b">p1</p>
<p attr1="a" attr2="c">p2</p>
$("p", attr1: "a", attr2: "b").size() == 1
$("p", attr1: "a").size() == 2
$("p", text: "p1", attr1: "a").size() == 1
$("p", text: ~/p./).size() == 2
$("p", text: startsWith("p")).size() == 2
$("p", text: endsWith("2")).size() == 1
$("p", text: ~/p./).size() == 2
Geb – jQuery(ish) API
Pattern Matching
Keyword
Description
startsWith
Matches values that start with the given value
contains
Matches values that contain the given value anywhere
endsWith
Matches values that end with the given value
containsWord
Matches values that contain the given value surrounded by
either whitespace or the beginning or end of the value
notStartsWith
Matches values that DO NOT start with the given value
notContains
Matches values that DO NOT contain the given value
anywhere
notEndsWith
Matches values that DO NOT end with the given value
notContainsWord
Matches values that DO NOT contain the given value
surrounded by either whitespace or the beginning or end of
the value
Geb – jQuery(ish) API
Traversing content - Finding & Filtering
<div class="a">
<p class="b">geb</p>
</div>
<div class="b">
<input type="text"/>
</div>
$("div").find(".b") //p.b
$("div").filter(".b") //div.b
$(".b").not("p") //div.b
$("div").has("p") //div containing p
//div containing the input with a type attribute of “text”
$("div").has("input", type: "text")
Geb – jQuery(ish) API
Traversing content
$("p.d").previous() // 'p.c'
$("p.e").prevAll() // 'p.c' & 'p.d'
$("p.d").next() // 'p.e'
$("p.c").nextAll() // 'p.d' & 'p.e'
$("p.d").parent() // 'div.b'
$("p.c").siblings() // 'p.d' & 'p.e'
$("div.a").children() // 'div.b' & 'div.f'
<div class="a">
<div class="b">
<p class="c"></p>
<p class="d"></p>
<p class="e"></p>
</div>
<div class="f"></div>
</div>
Geb – Screenshots/HTML Reports
Automatically captures the State
• As a screenshot of the page
• Also captures the HTML content
• Optional (extend GebReportingSpec)
Geb – Configuration & Driver
Management
Looks for GebConfig class or
GebConfig.groovy file on classpath
Manages and optimises webdriver
instances & clearing cookie state
Geb – Direct Downloading
Browser.drive {
go "http://myapp.com/login"
// login
username = "me"
password = "secret"
login().click()
// now find the pdf download link
def downloadLink = $("a.pdf-download-link")
// now get the pdf bytes
def bytes = downloadBytes(downloadLink.@href)
Lots of work behind the scenes
Geb – CI Test Reports
Bring Gradle to
party
Ian Kelly
@kellizer
github.com/kellizer




Spock – http://docs.spockframework.org
Geb – http://www.gebish.org
Sauce Labs – http://www.saucelabs.com
Gradle – http://www.gradle.org/
Thank You