NoVaTAIG
April 7, 2010
Automated In-Browser Testing with
Cucumber and Watir
Rudy Regner
Mike Perz
Erik Scheirer
Rasik Pandey
Rudy Regner
• Introductory Notes
– We’re all on linkedin.com.
– Send us questions regarding any details presented
tonight
• Presentation Goals and Objectives
– Mike Perz: Test Automation Framework
– Erik Scheirer: End-to-End SDLC Architect
– Rus Pandey: BDD Tools Expert
Our Situation
• Writing regression tests for an app in production
– Only required to support IE
• Rails v1.2.3
– Precludes using Webrat latest version
– ?? Could possibly have used earlier version ??
• why regression tests for earmarks?
– Bug made it through manual testing
– Encapsulate critical functionality in automated
regression test scripts to decrease likelihood of this
failure occurring again.
Mike Perz
• How accomplished the automation framework
• Challenges encountered
Our Learning Curve
• Started with Watir
– Easy to get up and running
• Added Cucumber (RSpec)
• Replaced Watir with Webrat and Selenium
– Sought flexibility gain through re-use of Webrat to drive
Selenium, Mechanize or straight Rails
• Productivity dropped with Selenium
– Watir can manipulate the DOM as objects. (you can iterate
through the cells of an html table)
– With Selenium you would have to find each cell individually.
– Script debugging difficult without irb
• You can attach to a browser window* (with Watir)
• Converted existing Selenium code to Watir
Cucumber Feature snippet
associated Step Definition
Scenario Outline example
(similar to fitnesse tables)
Cucumber Test Summary Output
(sample)
Once we switched back to Watir
• At first our tests were dependent on each other
– They needed to be run in a particular order.
– We used Rake to accomplish this.
• Open 4 browsers with 4 different users (Watir::IE.new_process vs.
Watir::IE.new)
– Now we can attach Watir to whichever browser
session we need
• Watir::IE.attach
– We use SQLite to keep track of which user is
associated with which browser session
Interesting side benefits to writing
automated tests
• Writing automated tests helped/made us
understand the app better.
• Found application bugs that were missed
earlier
Other useful tools
•
•
•
•
Watircuke
IE Developer Toolbar
Firebug
FireWatir
recorder: www.itest2.com/downloads
Erik Scheirer
• Spreading the Developer Joy
• Benefits of a single executable Cucumber
artifact end-to-end
Status Your Organization
•
•
•
•
What Rules? Confusion? Sanity?
Even if its sane, it probably can be more so.
Some groups have managed a high degree
of sanity.
But what most organizations are
experiencing is ...
Chaos
•
•
•
•
little Trust between groups + infighting
bad communication
no ‘Process’ (or its limping, bleeding, out
the door)
no time to implement anything new,
constantly fighting fires instead
This approach
•
•
•
provides an end-to-end process via
architecture, convention, and tools
is easy to implement
is easy to customize to exacting technical
needs and group culture
This approach
•
•
•
builds better quality
builds more trust between technical and
non-technical stakeholders
utilizes resources more effectively
Ingredients
•
•
•
•
CTBE - catch that bug early!
Unit tests, feature and black-box testing,
via one single artifact
in this case an Agile approach (scrum)
artifact re-use creates self-reinforcing
system
One Artifact,
multiple uses
•
•
•
•
developer can create cucumbers as
part of T/BDD, and also to capture
‘drive-by requirements’
QA can create ‘cukes’ for regression
tests
Cukes can be fed from dev. to QA or
vice versa
Cukes can be used by Continuous
Integration, and in Production
monitoring
Next, the Process from multiple
perspectives:
The daily grind
Building Trust spreading the joy
Time
testing & daily
work
The overall lifecycle
Iteration
s
Stakeholder interaction,
design and architecture,
includes non-technical
Conclusion
•There is a way to achieve more success,
better utilization of existing resources, and
get more sleep; using a simple, adaptable
architecture coupled with process,
convention, and tools.
•Did we mention ‘metrics’ and how these
can be used to CTBE in this manner, too?
•Its also all open-source, so its
Rus Pandey
• How BDD would work
• Other possible tools
Ideal BDD Process:
•
•
•
•
•
•
1. Describe behaviour in plain text
2. Write a step definition in cucumber
3. Run and watch it fail
4. Write code to make the step pass
5. Run again and see the step pass
6. Repeat 2-5 for all your functionality
Performance:
• Since it takes 3 hours to run a test suite we
should investigate using spork and --drb and
testjour to distribute the test load if possible
to speed up test runs.
Test Web apps in any language:
Drive a full or headless browser using one of these
•
•
•
•
•
•
•
•
•
•
•
•
•
*Webrat – Ruby acceptance testing for web applications
*Capybara – Acceptance testing framework with a webrat-like API and support for multiple
backends, including RackTest, Selenium, Celerity and Culerity
*Steam – Drives a fast headless browser with Javascript support. Support for the normal
webrat step definitions, see Setting up Steam
*Testing PHP app using Webrat
*WebDriver – Drives IE, Firefox, Chrome
*Watir – Drives IE (Windows only), see: Setting up FireWatir
*FireWatir – Drives Firefox on Windows, OSX and GNU/Linux
*SafariWatir – Drives Safari (OS X only)
*ChromeWatir – Drives Google Chrome
*Celerity – Drives a fast headless browser with Javascript support. Examples here
*Culerity – For when you can’t run your app under jRuby
*Selenium – Runs any browser (any OS), see: Setting up Selenium
*Mechanize – Runs a headless browser (any OS)
Examples of Cucumber being used in the real world across
various languages/platforms to test all sorts of things
.
• http://wiki.github.com/aslakhellesoy/cucumb
er/tutorials-and-related-blog-posts