Hands On Open Source Development
Dr. Mark Pollack
Sr. Staff Engineer: SpringSource
© 2010 SpringSource, A division of VMware. All rights reserved
Introductions…




Who is this guy?
Who are you?
All about SpringSource and the Spring Framework
Project Goal: Build open source software
• Using ‘modern’ development processes and tools
 Why should you care?
2
Projects to develop
 Portable “Blob” storage API
 Data access library for the open source database CouchDB
3
Blob project
 “Cloud” data storage is where many individuals and companies
store data.
• Your personal family pictures
• Internal audit trail logs
 Several public providers of this fully managed service
• Amazon S3
• OpenStack
• Microsoft Azure
 Host your own service on your own infrastrucutre.
• Eucalyptus
• Riak
 10,000 ft project goals
• API ease of use for common cases
• API portability
4
Data Access library for CouchDB
 What is CouchDB
• Part of a new ‘wave’ of non-relational databases
• Open source Apache project
• Considered a ‘document’ database. Data is stored as a JSON document
• HTTP/REST requests to interact with the database
•
•
•
•
“GET” – get a document
“PUT - put a new document
“DELETE” - delete a document
…
• Query is done by creating ‘views’ on the document
• Specifically using “Map/Reduce” framework in JavaScript
 10,000 ft. project goals
• Provide full Java API for all HTTP requests
• Simplify the storage of Java objects in the database
• Simplify querying the database for common cases
5
Development Philosophy
 Follow agile development principals
• Adherence to core OO design principals
• Decoupling, encapsulation, orthogonality
• Unit/Integration tests
• Rapid feedback on the impact of my own changes
• Continuous integration
• Rapid feedback on the impact of everyone else’s changes
• Iterations/Sprints/Intervals…
• “Stakeholder” feedback on the impact of changes
6
Traditional and Agile software development approaches
 Traditional
• Assumes change is avoidable
• Manage change by sufficient pre-planning and design to avoid change
• Big design up front
 Agile
• Assumes change is inevitable and unavoidable
• Assumes impossible (or impractical) to plan-around change
• Ties to manage change by ensuring the software remainds flexible enough or
respond to change
• Ensures that sufficient tooling, process and methods are in place to allow
response to change within the context of an incredibly tight feedback loop
7
Key Practice : Version control
 Manages the source code across the team
• Mark known points in time for releases
 We will also integrate source control with bug tracking
 Version control is a requirement for Continuous Integration
 Will use subversion for version control
• Central repository model
• You may have to handle merges when there are simultaneous changes to the
same lines of code.
• Will be hosted at SpringSource
 Will use “Fisheye” to browse code on the web
• See https://fisheye.springsource.org/browse
8
Key Practice: Continuous Integration
 Automated way to enable us to rapidly respond to change while
continuing to have working software.
 Software that constantly checks out code and does the following
• Compiles code
• Run unit/integration tests
• Run code metrics (optional)
• Deploy (optional)
And notifies you on failure…
 We will use CI product “Bamboo”
• Setup for you already. See http://build.springsource.org/
• Just need to check email for notification
9
Key Practice: Unit/Integration Testing
 Developers write tests to ensure software behave as expected
 May test an individual class in isolation (unit testing) or larger parts
of the system collaborating together (integration testing)
• Both are important
 Regression harness
• Feedback on code changes
• Allows code to be changed/refactored quickly, easily, and safely in response to
change in requirement, architecture, etc.
• The quicker you find a bug, the cheaper it is to fix it
 To follow this practice need to select a testing framework
• We will use JUnit inside Eclipse
10
Key Practice: Bug, Issue, Project Tracking
 Manage project goals for components, versions, timelines
 “Issues” are entered into the system for new features
• Estimates made when to complete stories
 “Bugs” entered to report bugs
 In practice used to drive much of the reporting and status of the
project
 We will use JIRA
• See https://jira.springsource.org/secure/Dashboard.jspa
11
Product documentation
 Many ways to provide this, SpringSource has generally settled on
DocBook
• XML schema defines book content. e.g. chapter, paragraph, etc.
 XMLMind is a graphic semi WYSIWIG editor
12
General flow
 Discuss/teach tools/process at each meeting
 We will have a planning session at the beginning of each meeting
• We will make goals, decide what’s required to achieve those goals, and the
goals need to be measurable
• We will also talk about how well we did, what we could have done better, and
what to do for the next iteration- retrospective
• We will also need to figure out the feasibility and estimate the time it will take to
complete iterations
 During iterations, if we have questions contact Mark via the mailing
list.
 2 week iterations so iterations cycle=meeting cycle
13
Resources
 “Big Picture” agile overview
• Agile Samuari – some free chapters (on estimation)
• Code Leader: Using People, Tools, and Processes to Build Successful Software
• Agile Estimating and Planning – Mike Cohn
• Agile Software Development, Principle, Patterns, and Practices – Bob Martin
• http://www.agilefirestarter.com/ Free conference material (pdfs)
• https://github.com/agilefire/November2010 (power poit)
 Eclipse 3.6 or SpringSource Tool Suite 2.5.2
• http://eclipsetutorial.sourceforge.net/totalbeginnerlessons.html (videos!)
 JUnit
• www.junit.org (note: JUnit is already integrated into eclipse)
• http://www.vogella.de/articles/JUnit/article.html
• http://pragprog.com/titles/utj/pragmatic-unit-testing-in-java-with-junit (book)
14
Resources
 Subversion
• http://subversion.tigris.org/
• Tortoise SVN - http://tortoisesvn.tigris.org/
• http://subclipse.tigris.org/ eclipse add-in
• http://svnbook.red-bean.com (free book)
• Pragmatic Version Control using Subversion (book)
 DocBook
• http://www.docbook.org/
• http://www.xmlmind.com/xmleditor/
 JIRA
• See videos/docs on http://www.atlassian.com/software/jira/
 Continuous Integration (“Bamboo”)
• See videos on http://www.atlassian.com/software/bamboo/
• Continuous Delivery (book)
15
Resources
 CouchDB
• http://couchdb.apache.org/
• http://guide.couchdb.org/ (free book)
 Amazon S3
• http://aws.amazon.com/s3/
• http://docs.amazonwebservices.com/AmazonS3/latest/gsg/
• http://www.sitepoint.com/books/cloud1/ book
• Before, many out of date and difficult to read books on AWS/S3 out there…
16
Resources
 Spring Framework
• http://en.wikipedia.org/wiki/Spring_Framework
• http://www.springsource.org/documentation
• Spring in Action V3 (book)
 Spring RestTemplate
• http://blog.springsource.com/2009/03/27/rest-in-spring-3-resttemplate/
• Reference Manual
• RESTful Web Services (book)
17
Fun for the weekend
 Read the chapter on estimation in the Agile Samurai
• http://media.pragprog.com/titles/jtrap/estimate.pdf
 Take a look at CouchDB/Amazon S3
• With an eye to see which project you want to develop.
If you are not familiar with Java/Eclipse/JUnit already start a bit with the following
 Download latest Eclipse or SpringSource Tool Suite and get familiar
with the environment
 Follow the JUnit tutorial
18