Lecture 18 AGILE PROJECT MANAGEMENT METHODS: SCRUM AND TDD DR ELKE DUNCKER AND LINDSEY BRODIE Learning Outcomes Understand Scrum Understand Test-driven Development Discuss some of the challenges that Scrum and TDD present to organisations SCRUM SCRUM History 4 Developed by Jeff Sutherland and Ken Schwaber Based on "The New New Product Development Game" by Hirotaka Takeuchi and Ikujiro Nonaka published in the Harvard Business Review in 1986 First published in 1995 at OOPSLA ’95 (conference) Now used by 75% of agile teams worldwide (Sutherland 2011) Often in combination with other Agile PM methods such as pair programming and TDD SCRUM is …Lightweight …Easy to understand …Extremely difficult to master A process framework SCRUM is Empirical as opposed to a defined process Empirical = derived from or guided by experience or experiment s Normative = how it ought to be done Prince 2 is a collection of best practice, what is the difference to SCRUM? Focusing on process control to optimize predictability control risk Never compromise standards of quality SCRUM Principles Called ‘pillars’ in SCRUM terminology Transparency Significant aspects of the process must be visible to those responsible for the outcome Inspection frequent inspection of Scrum artifacts and progress toward a goal to detect undesirable variances. Must not get in the way of work Adaptation If one or several undesirable variances have been detected and the resulting product will be undesirable, the process and/or the material being processed have to be adjusted (Deemer et al. 2010) SCRUM Team Product Owner SCRUM Master Development Team Scrum Roles 10 Product Owner One person Owns product backlog Sets the priorities Says when things are “done” Scrum Master Service to team, product owner and organisation Coaches everybody involved on Scrum Finds the Product Owner Removes impediments to team progress Developers Development Team 3-9 Developers Self-organizing No project manager Cross-functional All skills present in the team as a whole Everybody does everything No special titles – all team members are called developers Some developers can focus on specialist areas but the responsibility for the work remains with the whole team No sub-teams within the development team SCRUM Events 12 Are time-boxed Sprint Between 1 week and 4 weeks All sprints of a project have the same duration Between 2 and 8 hours 2 hours for each week of the sprint First half: What’ will be delivered? Second half: How will it be achieved? 15 minutes at the start of each working day 4 hours At the end of the sprint Evaluating the prototype 3 hours Evaluating the team Sprint Planning Meeting Daily Scrum Sprint Review Sprint Retrospective (Sutherland and Schwaber, 2011) SPRINT Iteration of 1 to 4 weeks duration During the each sprint a product increment is developed, which has to be ‘Done’ Usable Coding, debugging and testing finished Usability testing Potentially releasable Not all end products of a sprint have to be released, but they all have to be fully functional and in good working order Definition of ‘Done’ Identified by the Product Owner Definition must be a shared by the Scrum team Typical definition of “done”: All planned sections of code Written Fully tested (from unit testing to application testing) Documented Turned into a deliverable No quality shortcuts as that leads to ‘technical debt’ building up future problems and delay (Sutherland and Schwaber 2011) Sprint Rules During the Sprint: No changes affecting the Sprint Goal Development Team composition remains constant Quality goals are not allowed to decrease Scope may be clarified and re-negotiated between the Product Owner and Development Team as more is learned (Sutherland and Schwaber 2011) Scrum Artifacts I Product Backlog Prioritized functionality for the whole product with estimated effort Sprint Backlog Set of tasks necessary to implement the functionality selected from the Product Backlog for a sprint The functionality of the product backlog needs to be broken down into tasks needed to achieve the functionality The tasks can then be assigned to individuals or pairs of developers Cancellation of a Sprint Only the product owner can cancel a sprint When is it advisable: Only, when the sprint goal has become obsolete Mostly due to external events Rare event because of short cycles What needs to be done Completed tasks have to be reviewed if they are still releasable Non-releasable tasks have to be put back on the product backlog and re-estimated Sprint cancellations are consuming resources and are traumatic to the team and have to be avoided Sprint Planning Meeting Team plans the work to be done during the sprint Product owner can be present SCRUM Master should be present Time-boxed 2 hours for each week of a sprint E.g. a four week sprint has an 8 hour planning meeting Two parts What will be done during this sprint? Defining the product increment to be delivered at the end of the sprint Product owner presents ordered and relevant items from the product backlog for this sprint Team forecasting product backlog items for this increment How will the work be achieved? Plan of delivery Work to be cut into daily units Re-negotiation of estimated effort By the end of the meeting team explains to SCRUM Master what has to be achieved and how they are planning on achieving the planned increment Daily Scrum To be held at the same time and place each day Each Developer explains: What s/he has accomplished since the last meeting? What s/he will do before the next meeting? What obstacles are in the way? Not a ‘state of the project’ meeting about work in progress to create the deliverable (Sutherland and Schwaber 2011) Sprint Review Focus on the product increment and the work done Product Owner identifies what has been “Done” and what has not been “Done” Development Team discusses what went well during the Sprint, what problems it ran into, and how those problems were solved Development Team demonstrates the work that it has “Done” and answers questions about the Increment Product Owner discusses the Product Backlog as it stands. He or she projects likely completion dates based on progress to date The entire group collaborates on what to do next, so that the Sprint Review provides valuable input to subsequent Sprint Planning Meetings (Sutherland and Schwaber 2011) Sprint Retrospective Focus on the team Inspect how the last Sprint went with regards to people, relationships, process, and tools Identify and order the major items that went well and potential improvements Create a plan for implementing improvements to the way the Scrum Team does its work (Sutherland and Schwaber 2011) (Deemer et al. 2010) Scrum Artifacts II Release Burndown Implemented functionality against total functionality estimated in points Must be based on previous performance Sprint Burndown Implemented functionality against functionality to be implemented during a sprint Also based on previous performance Benefits Scrum delivers in time and in budget Scrum aims at delivering high quality software Scrum reduces the risk to IT development projects Scrum is very agile Scrum puts the customer value at the centre of the project Drawbacks In practice, the time boxing of sprints leads to a neglect of system and application test cases Reduces the quality Increases risks that emerge when the software goes life Questions? Test-Driven Development Test-Driven Development A concept originally known as part of Extreme Programming Has become successful independently from XP Re-discoverery is attributed to Kent Beck (XP creator) Basic Concepts of TDD Extremely short development cycles First, the developer writes an initially failing automated test The test case for the automated test defines the desired outcomes of the function to be added or improvement to be made The developer has to make sure the test fails initially Then the developer writes the minimal amount of code that is necessary to pass the test He also runs all previous test to make sure his software still passes those Then the developer refactors the code, i.e. removes duplications and do other things so that the code adheres to accepted standards Principles of TDD Two basic principles: Never write a single line of code unless you have a failing automated test Eliminate duplication Other “principles”: “Fake it till you make it” (writing the code that passes the test) “KISS” – Keep it simple, stupid! “YAGNI” – You aren’t gonna need it Variations and Expansions ATDD = acceptance-test driven development Communication tool between customer/user, developer and tester to ensure well-defined and testable requirements Differences to TDD: Test automation not required in ADTT, but required in TDD Test cases in TDD can be derived from ADTT, but not the other way round Test cases in ADTT have to be comprehensible for customers, TDD test cases only have to be comprehensible for developers and testers If automated testing is used, the developer writes one test case Also called BDD=Behaviour-driven development Aim: Detailed, executable requirements on a just-in-time basis Links between BDD and TDD (Re-)write Acceptance test case Run acc test no Full TDD cycle Does the acc. test fail? yes Hand over to TDD with a change suggestion Run acc test yes no Does the acc. test fail? no Developm. complete? yes Benefits and Drawbacks of TDD Benefits: Increased productivity Less debugging required Very quickly functioning code The correlation with software quality is not quite as straightforward Unit testing and integration testing: very good System and application testing: less good Functional requirements testing: very good Non-function requirements testing: less good Questions? Organisational Challenges of Scrum and TDD Increases transparency Not everyone is in favour of transparency Some people like to use information as power base Some staff members may not want to share their timesheets or productivity logs Can reveal organisational problems That some organisations don’t want to fix Can lead to people starting to customise Scrum instead of fixing the organizational weaknesses e.g. not adapting the organizational structure to support Scrum teams Organisational challenges of Scrum and TDD Requires training effort to get cross-functional teams Requires training effort to get TDD going Requires letting-go and trusting the teams Transition from management to leadership Need to act to remove the impediments in order to increase the velocity Customer company and developer company have to agree with the use of agile methods Customer Company may not want to be involved in development process Scaling Scrum and TDD What about large projects? SAP and other organisations are using Scrum for large projects Scrum of Scrums TDD for complex systems: Use of test cases even more beneficial Potentially counteracted by an increasingly complex test case system Maintenance of test software becomes an issue of its own Criteria for a Disciplined Agile Team 39 Business value Produce a working solution on a regular basis which provides quantifiable value to stakeholders Validation Do continuous regression testing, and better yet take a Test-Driven Development (TDD) approach Stakeholder Collaboration Work closely with their stakeholders, or a stakeholder proxy, ideally on a daily basis Self organization Are self-organizing and work within an appropriate governance framework Improvement Regularly reflect on, and measure, how they work together and then act to improve on their findings in a timely manner © Scott Ambler + Associates Criteria for a Disciplined Agile Team 40 Teams Claiming to be Agile Teams Moving to Agile Business value 91% 83% Validation 88% 69% 99% 94% 72% 51% Improvement 92% 90% All Criteria 65% 39% Collaboration Self organization © Scott Ambler + Associates Summary Scrum Principles: Transparency, Inspection and Adaptation Teams: Scrum Team including product owner, Scrum Master and development team Events: Sprints, sprint planning meeting, daily scrum, sprint review and sprint retrospective Artefacts: Product backlog, sprint backlog, release burn down, sprint burn down High productivity, low risk, questionable quality TDD Principles: very small iterative steps; write test case first then write code to satisfy the test case; refactoring ATDD or BDD: include user acceptance testing and improve requirements spec High productivity, bug free code but quality still not 100%. Next Week Extreme Project management Lean project management Reference The slides of this slideshow are based on SCRUM: Schwaber, K. and J. Sutherland. 2011. “The SCRUM Guide”, available at http://www.SCRUM.org accessed 28/5/2013 TDD: Beck, K. Test-Driven Development by Example, Addison Wesley Vaseem, 2003 Scott W. Ambler, Introduction to Test-driven Development at http://www.agiledata.org/essays/tdd.html, accessed 2nd of June 2014 The statistics about the agility of disciplined development teams are taken from www.ScottAmbler.com References 44 Agile Manifesto. Available from: http://agilemanifesto.org/ [Accessed 1 November 2009]. Deemer, P., Benefield, G., Larman, C. and Vodde, B. (2010) The Scrum Primer [online]. Available from: http://www.scrumprimer.com/ [Accessed 23 May 2012]. Larman, C. (2004) Agile and iterative Development: A Manager’s Guide. Addison-Wesley. ISBN 0131111558. Østergaard, J. (2009) Why is Scrum so Hard? [online]. Available from: http://www.youtube.com/watch?v=q3t8twm3aUk&feature=related [Accessed 23 May 2012]. Sutherland, J. and Schwaber, K. (2011) Scrum Guide [online]. Available from: http://www.scrum.org [Accessed 1 November 2009]. Sutherland, J. (2011) Takeuchi and Nonaka: The Roots of Scrum [online]. Available from: http://scrum.jeffsutherland.com/2011/10/takeuchi-and-nonaka-roots-ofscrum.html [Accessed 23 May 2012]. Further Agile Reading/Viewing 45 Larman, C. & Basili, V. (2003), Iterative and Incremental Development: A Brief History, IEEE Computer, June 2003, pp 2-11. See http://www.craiglarman.com for a copy of this paper. Or via ‘Further Reading’ on Wikipedia information on ‘Agile Software Development’ Fowler, M. (2005) The New Methodology [online]. Available from: http://martinfowler.com/articles/newMethodology.html#XpextremeProgramming [Accessed 1 November 2009]. Gilb, T. http://www.Gilb.com under ‘Downloads’ then under ‘Gilb Papers’ Decomposition April 4 2008.pdf: Decomposition of Projects - How to design small incremental result steps CAI tomgilbinterview1.pdf: What are Evolutionary (EVO) Development Methods? CAI Interview on 21 February 2006 Evo Principles.pdf: Fundamental Principles of Evolutionary Project Management Abrahamsson, P., Salo, O., Ronkainen, J. and Warsta, J. (2002), Agile Software Development Methods. Review and Analysis, VTT Publications, Espoo, Finland, ISBN 9513860094. http://www.inf.vtt.fi/pdf/ Schwaber Scrum Video http://video.google.com/videoplay?docid=7230144396191025011&q=Google+Tech+Talks&hl=en# (61 minutes)