Test Driven Development
What Works & What Doesn’t
November 5, 2008
www.synerzip.com
Today’s Discussion
• Synerzip Introduction
• Agile Development Lifecycle
• Test Driven Development
–
–
–
–
–
–
Case for TDD & Continuous Integration
User Acceptance Testing
Functional Testing
Unit Testing
Regression Testing
Common Tools & Useful Resources
• Q&A
Confidential
Synerzip in a Nut-shell
1. Software development partner for small/mid-sized
technology companies
•
•
•
•
Focus: small/mid-sized technology companies
Deep experience in product development, testing, & deployment
Handles full software development life cycle
Technology and industry domain agnostic
2. Actually reduces risk of development/delivery
•
•
•
3.
Experienced software management team
Brings in appropriate level of engineering discipline
Practices Agile development – responsive & disciplined
Reduces cost – dual-shore team, 50% cost advantage
4. Offers long term flexibility – allows (facilitates) taking
offshore team captive
Confidential
Our Clients
Confidential
Today’s Discussion
• Synerzip Introduction
• Agile Development Lifecycle
• Test Driven Development
–
–
–
–
–
–
Case for TDD & Continuous Integration
User Acceptance Testing
Functional Testing
Unit Testing
Regression Testing
Common Tools & Useful Resources
• Q&A
Confidential
Agile Development
• Why agile development?
– Due to a dynamic , ever changing business environment;
requirements change.
– Innovation happens when requirements change.
– Agile not only allows but encourages requirements to change.
– Agile is an iterative show and build process. The customer gets to
see some new functionality at the end of each short iteration. Ideas
evolve and new requirements get generated when the customer
sees something working.
– Shorter the iteration closer the compliance with requirements.
• Challenges of agile development
– In a short iteration of 3 weeks if 1 week is required for integration,
testing and deployment then only 2 weeks are left for actual coding
or development work.
– There is an interdependence of TDD and automation.
Confidential
Common Misconception
Confidential
Agile Lifecycle – Iterations & QA
User Stories
Test Scenarios
Requirements
Customer Approval
Release
Planning
Confident
Estimates
Iteration Plan
Confidential
Feedback
Small
Release
Iteration
Unit and
Automated
F & R/ UA
Testing
Functional /
Regression
Testing
User
Acceptance
Testing
Testing Terminology
• Unit testing – testing a code component or class.
– Coverage is measured against lines of code
• Functional testing - testing against required
functionality by going through the same steps that
an end-user will execute.
– Coverage is measured against requirements
• Regression testing – when functional testing is
carried out again after fixing defects to ensure that
no new bugs get introduced.
• User acceptance testing – a subset of functional
test cases that form the basis on which the release
is accepted by the customer.
Confidential
Agile Lifecycle - Roles
Iteration zero
Iteration n
Final Iteration
Product
Owner
Requirements
analysis
Review of product
demo and changes
User acceptance
testing and sign
off.
Project
Manager
Estimation and
planning
Monitoring and reestimation
Monitoring and
release notes
Architect
Prototype or POC
design
Designing and code
review
Design documents
Developer
Estimation,
understanding
requirements and
coding POC
Writing unit tests and Fixing bugs and
code. Writing build
documentation.
automation script
QA
Writing test cases
Testing and some
automation
Confidential
Filing bugs and
regression testing
Today’s Discussion
• Synerzip Introduction
• Agile Development Lifecycle
• Test Driven Development
–
–
–
–
–
–
Case for TDD & Continuous Integration
User Acceptance Testing
Functional Testing
Unit Testing
Regression Testing
Common Tools & Useful Resources
• Q&A
Confidential
What is TDD?
• Test Driven Development (TDD) means
capturing the intent of requirements in the
form of a test case before starting to code
• Test case could be a manual test case or an
automated test script
• Test case could be a functional or a useracceptance or a unit test
Confidential
TDD Means More Code?
Use your judgement, e.g. don’t write tests for every getter and setter
Confidential
TDD in Practice
Value of TDD – especially in Agile
• Tests are more explicit expression of requirements
• Earlier defect detection by frequent testing
• Ensures better coverage - don’t end up having
superfluous code, without tests
Challenges – team resistance & reluctance?
• ROI is realized in the long term – in maint phase
• Extra coding effort seems unjustified in the
beginning
• Easier said than done- difficult to implement
• Developer resistance
Confidential
Role of Continuous Integration
• Agile development has its roots in lean
manufacturing. “Stop the line” principle at Toyota is
implemented in software development by making
automated tests a part of the build process.
• The build fails if even one test fails. This reduces
the time-lag between detection of defect and fixing
of defect. Ensures timely corrective action.
• This also ensures that tests are updated to stay in
sync with changing requirements.
Confidential
User Acceptance Testing (FIT)
• What Works
– Wiki based tables are used to express and share
requirements or user stories
– Test fixtures are written only for significant and complex
requirements. Ready made test fixtures are used for
simple requirements
– Subwikis are used to build test suites
• What Doesn’t
– Customer/ product owner doesn’t spend enough time
reviewing the test cases
– Developers spend too much time writing test fixtures
– FIT tables don’t cover unhappy path. A use case can go
wrong in many ways.
Confidential
Manual Functional Testing
• What Works
– Better expression of requirements by insisting on writing the test
cases up front (Note: 70% of defects arise due to faulty
requirements)
– Better understanding of the requirements by the QA staff
– Examples or test data
– QA should be a part of the Product Management/BA team
• What Doesn’t
– No time allocated by the customer and product owner to review the
test cases
– Test cases aren’t updated as new requirements evolve. Need to
make sure QA team and Product Mgmt/BAs are always in sync
– Test cases are at a high level – but difficult to be useful for
generating test data
Confidential
Unit Testing
• What Works
– Write tests first
– Write tests only for the complex methods/units
– No significant or complex code is written unless there is
a unit test already written for it.
• What Doesn’t
– Writing tests later after the code is written.
– Completing the formality of writing unit tests for each
and every method including accessor methods.
– Writing unit tests for legacy code
• Case Study: A customer started TDD practice by writing unit
tests for all the legacy code and ended up testing the tests with
the code.
Confidential
Mechanics of Unit TDD
Add some
code
Confidential
Functional/ Regression Testing
• What Works
– Automating only that functionality which has stable
manual test cases
– Test suites that run as a part of the build process
– Test reports automatically published on the wiki
• What Doesn’t
– Trying to automate everything. Coverage can’t go
beyond 70%
– Tests scripts that are never integrated into a test suite
– Tests are data specific and start breaking when the data
changes
Confidential
Continuous Integration
• What Works
– Separate build tasks for unit, regression and UAT
– Quick builds facilitate frequent (continuous) integration
– Reports are a part of the build
• What Doesn’t
– No distinction in dev, prod and test builds.
– Long and tedious builds. Developer’s attention gets
diverted if it takes any longer than 30 secs.
– Absence of teardowns to cancel all setup data
– Tasks are dropped if the build fails
– Untested dependencies
Confidential
Common Tools
Manual
Tools
Jira, Bugzilla, Test Director
Unit
Junit, Nunit, HttpUnit, DbUnit,
SpringUnit etc.
Functional and
regression
Watir, Selenium, QTP, AutoIt,
Winrunner, RSpec, Tkl/Tk
Performance
Loadrunner, OpenSta, jMeter,
User Acceptance
Fitnesse
Continuous
Integration
Maven,Cruise Control, Luntbuild, PMD,
FindBugs, Jalopy (automated code
review), Clover (code coverage)
Confidential
Useful Resources
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
http://www.agiledata.org/essays/tdd.html
http://www.methodsandtools.com/dynpoll/oldpoll.php?UnitTest2
http://www.testdriven.com/modules/news/
http://www.kevinwilliampang.com/post/Is-Code-Coverage-Really-AllThat-Useful.aspx
http://www.fitnesse.org
http://www.rubyforge.org
http://selenium.openqa.org
http://maven.apache.org/
http://cruisecontrol.sourceforge.net/
http://luntbuild.javaforge.com/
http://pmd.sourceforge.net/
http://jalopy.sourceforge.net/
http://findbugs.sourceforge.net/
http://www.atlassian.com/software/clover/
http://www.developer.com/design/article.php/3700651
Confidential
Today’s Discussion
• Synerzip Introduction
• Agile Development Lifecycle
• Test Driven Development
–
–
–
–
–
–
Case for TDD & Continuous Integration
User Acceptance Testing
Functional Testing
Unit Testing
Regression Testing
Common Tools & Useful Resources
• Q&A
Confidential
Contact Information
•
Hemant Elhence (Dallas based)
– hemant@synerzip.com
– Cell Phone: 214.762.4873
•
•
www.synerzip.com
HQ in Dallas, TX
– 14228 Midway Rd, #130, Dallas, TX 75244
– Office Tel: 469.322.0349
– Office Fax: 469.322.0490
Thanks!
Confidential