Agile Model Driven Development: Techniques for Scaling Agile Delivery Scott W. Ambler Senior Consulting Partner scott [at] scottwambler.com twitter.com/scottwambler Feel free to ask questions at any time © Scott W. Ambler + Associates 2 Agenda • • • What puzzles you? Traditional versus agile development A specification parable • • • What is modeling? Agile Modeling Agile models and documents • • Agile Model Driven Development (AMDD) Agile modeling and documentation practices • • • Modeling through the lifecycle AMDD and scaling agile Parting thoughts © Scott W. Ambler + Associates 3 Exercise: What Puzzles You? • Organize into teams of 4-6 people • Take a minute to introduce yourselves to one another • For 5 minutes, discuss: – What have you heard about modeling and documentation on agile projects? – What actual experiences do you have on agile project teams, if any? – What puzzles you about agile modeling and documentation? • A spokesperson from the team should be prepared to share one or two things that puzzle you about agile modeling with the larger group © Scott W. Ambler + Associates 4 Exercise: Traditional Versus Agile Greeting Cards Organize into teams of 4-6 people – Smaller is better, but you need at least 4 people If possible: – Work with people you have never met before – Work with people from different organizations – Have fun, but stay focused on the assignments The overall strategy: 1. Get organized 2. Create a greeting card in a traditional manner 3. Create a greeting card in an agile manner 4. Discuss what happened Step 1: Get Organized • Choose roles: – – – – 1 stakeholder 1 analyst 1 tester Everyone else is a developer • The stakeholder from each group must come to the front of the room for instructions. • Everyone else: – Read the next slide © Scott W. Ambler + Associates 6 Step 2: A Birthday Card for Grandma Model 3 min Code 3 min Test • Analyst interviews stakeholder, producing requirements document. Everyone else may listen but not speak. • Analyst hands over requirements document to development team. Analyst and Stakeholder now quiet. • Developers create card. Tester watches. • Developers provide card and original requirements document to the tester. • Tester inspects card and suggests improvements. 45 sec Fix 45 sec Rate • Developers “fix” card. • Developers provide card to the stakeholder. • Stakeholder rates the end product. Step 3: A Wedding Card for Friends Develop 60 sec Improve 30 sec • You are in the same roles as before • Organize the project into three short iterations • Each iteration: – Develop 60 sec Improve 30 sec Develop 60 sec Rate – Development (60 seconds): • Anyone can ask the stakeholder questions. • Anyone can work on the card • Anyone can suggest improvements Retrospective (30 seconds): • At the end of each iteration identify how you can together more effectively • Choose one or two ideas to help improve your approach next iteration Step 4: Review • What are your scores? – Instructor will record on a flip chart • As a team, take 5 minutes to discuss – – – – – – What worked best for you? Why? What are the risks with each approach? What are the trade-offs with each approach? Was there overhead between hand-offs? Why? How efficient was each approach? What can you apply to “the real world”? • A spokesperson from the team should be prepared to share some key learnings with the larger group A Specification Parable Two years ago I asked for a toy like the one my friends had. I didn’t know what it was called so I described it to Santa, hoping that he would know what I was talking about. My description: It is a toy for kids. You get on it and you bounce up and down. It is lots of fun. © Scott W. Ambler + Associates 10 This is what I found under the tree. It’s sort of fun, but it’s not what I wanted. © Scott W. Ambler + Associates 11 Last year I asked Santa Claus for a sports car. It should be bright red with some white detailing. It should have big racing tires so that it can hug the road on sharp turns. © Scott W. Ambler + Associates 12 This is what I found under the tree. © Scott W. Ambler + Associates 13 This year I really want a pony. So in my letter to Santa Claus this year I am going to be very clear that I want a real, live pony. Not a toy pony. The pony should be white and between 7 and 9 feet tall. I will even include this photo of the type of pony that I would like with my letter. There will be no mistakes… © Scott W. Ambler + Associates 14 Exercise: Specification • Get back into your teams • For 10 minutes, discuss within the team: • Why didn’t he get the trampoline? How does this relate to what you’ve seen in the workplace? • Why didn’t he get the car he wanted? How does this relate to what you’ve seen in the workplace? • Assume that he gets the pony that he asked for. How will this actually work out in practice? How does this relate to what you’ve seen in the workplace? • What approaches have you seen for eliciting requirements from stakeholders? What were the advantages and disadvantages in the long run? • Someone needs to be a spokesperson for your team to share a few key learnings with the rest of the group © Scott W. Ambler + Associates 15 Exercise: What is a Model? • Get back into your teams • Goal: • Define what a model is • Capture your definition on an index card • For 5 minutes, discuss within the team: • What are models used for? • Are sketches models? • Are text-based artifacts (e.g. user stories and use cases) models? • Are tests models? • Do you need tools to model? If so, what? • Why do you model? • A spokesperson should be prepared to share your definition with the larger group © Scott W. Ambler + Associates Ummm... not in our context 16 What is a Model? A model is an abstraction that describes one or more aspects of a problem or a potential solution addressing a problem • Commonly one or more diagrams plus any corresponding documentation • How the abstraction is captured, if at all, shouldn’t matter • Sketches should be considered models • Non-visual artifacts can also be models © Scott W. Ambler + Associates 17 Why Model? To communicate To think things through To specify © Scott W. Ambler + Associates 18 Primary Strategy for Modeling No Modeling Ad-Hoc Sketch to Think and Communicate Sketch and Capture Key Diagrams SBMT for Docs Traditional Iterative SBMT to Generate Code SBMT for Full Trip Engineering Agile 0% 20% 40% 60% 80% 100% Source: Dr Dobb’s 2008 Modeling and Documentation Survey © Scott W. Ambler + Associates 19 Exercise: Modeling in “non-Agile” Environments • Get back into your smaller teams • For 10 minutes, discuss: – – – – – – – What are your experiences with modeling on “non-agile” projects? When did you model during the project? How much modeling did you do? What types of models did you create? Who was involved with the modeling? How were the models used? What were the advantages and disadvantages of the approach? • A spokesperson should be prepared to share a few key learnings with the rest of the group © Scott W. Ambler + Associates 20 Exercise: Modeling in “Agile” Environments • Get back into your smaller teams • For 10 minutes, discuss: – – – – – – – What are your experiences with modeling on “agile” projects? When did you model during the project? How much modeling did you do? What types of models did you create? Who was involved with the modeling? How were the models used? What were the advantages and disadvantages of the approach? • A spokesperson should be prepared to share a few key learnings with the rest of the group © Scott W. Ambler + Associates 21 Agile Modeling (AM) • AM is a chaordic, practices-based process for modeling and documentation • AM is a collection of practices based on several values and proven software engineering principles • AM is a light-weight approach for enhancing modeling and documentation efforts for other software processes such as Extreme Programming (XP), Scrum, and Unified Process (UP) Principles Practices Values • AM is built right into Disciplined Agile Delivery (DAD) © Scott W. Ambler + Associates 22 What Are Agile Models? • Agile models: – – – – – – – As a student I want to enroll in a course so that I can complete my degree Fulfill their purpose Are understandable Are sufficiently accurate Are sufficiently consistent Are sufficiently detailed Provide positive value Are as simple as possible • Agile models are just barely sufficient! Student name address phoneNumber emailAddress studentNumber averageMark isEligible (name, studentNumber) getSeminarsTaken() 1 enrolled in 1..* EnrollmentRecord marksReceived 1..* getAverageToDate() getFinalMark() {ordered, FIFO} 0..* on waiting list Professor name address phoneNumber emailAddress salary getInformation() enrolled in 1 Seminar name seminarNumber fees 0..* waitingList addStudent(student) 0..* dropStudent(student) instructs 0..1 ?Some seminars may not have an instructor? © Scott W. Ambler + Associates 23 What are Agile Documents? • • • • • • • • • Focus on stable, not speculative concepts Are executable first, static only if you have to Maximize stakeholder ROI Are concise Fulfill a purpose Describe information that is less likely to change Describe “good things to know” Have a specific customer and facilitate the work efforts of that customer Are sufficiently accurate, consistent, and detailed Just Barely Good Enough Ideal Realistic Value Effort Copyright 2005 Scott W. Ambler Copyright 2012 Ambysoft Inc. 24 Agile Model Driven Development (AMDD): Project Level © Scott W. Ambler + Associates 25 Exercise: Agile Requirements Envisioning • Get back into your smaller teams • Goal: Create a project profile focused on initial requirements envisioning • Take 10 minutes to discuss: Identify the team member(s) who have been involved with an agile team where some initial requirements envisioning occurred. If there are several, quickly discuss each project and choose the one most interesting to you Describe how the team approached initial requirements envisioning – How much effort was it? What artifacts were created? Who was involved? How was the output used by the team? How was the architecture validated? Describe the project: How large was the team? Were you distributed? What organizational challenges did you face? What are the advantages and disadvantages of what the team did? Think short term and long term. • A spokesperson will share a few key learnings with the larger group © Scott W. Ambler + Associates 26 Exercise: Agile Architecture Envisioning • Get back into your smaller teams • Goal: Create a project profile focused on initial architecture envisioning • Take 10 minutes to discuss: Identify the team member(s) who have been involved with an agile team where some initial architecture envisioning occurred. If there are several, quickly discuss each project and choose the one most interesting to you Describe how the team approached initial architecture envisioning – How much effort was it? What artifacts were created? Who was involved? How was the output used by the team? How was the architecture validated? Describe the project: How large was the team? Were you distributed? What organizational challenges did you face? What are the advantages and disadvantages of what the team did? Think short term and long term. • A spokesperson will share a few key learnings with the larger group © Scott W. Ambler + Associates 27 Agile Modeling’s Best Practices © Scott W. Ambler + Associates 28 Test-Driven Development (TDD) Test-First Development (TFD) is a technique where you write a single test and then you write just enough production code to fulfill that test. Refactoring is a technique where you make a simple change to your code/schema to improve its quality without changing its semantics. TDD = TFD + refactoring © 2012 Scott W. Ambler + Associates 29 Acceptance and Developer TDD Together © 2012 Scott W. Ambler + Associates 30 Some Industry Stats • 2009 Agile Project Initiation Survey: – 89% of agile teams do some sort of up-front requirements modeling – 86% of agile teams do some sort of up-front architecture modeling – 56% create a project vision/chart document • 2008 Test-Driven Development Survey – When it came to requirements capture, 85% of respondents wrote some documentation, 53% did some modeling, and 45% wrote acceptance tests – When it came to design capture, 80% wrote documentation, 67% modeled, and 57% wrote developer tests • Survey details can be obtained free of charge at Ambysoft.com/surveys/ © Scott W. Ambler + Associates 31 Exercise: Adopting Test Driven Development (TDD) • Get back into your smaller teams • For 10 minutes, discuss: – – – – Has your team adopted acceptance TDD (ATDD)? Is it considering it? Has your team adopted developer TDD? Is it considering it? What are the challenges associated with adoption of each form of TDD? What are the benefits of each form of TDD? Think short term and long term. – What are the disadvantages? Think short term and long term. • Someone needs to be a spokesperson for your team • Be prepared to share with the larger group: – Your adoption stats of ATDD and developer TDD (for each of ATTD and developer TDD – Number doing it, thinking about it, Neither). – A few key learnings from your discussion © Scott W. Ambler + Associates 32 The Basic/Agile Lifecycle of Disciplined Agile Delivery (DAD) © Scott W. Ambler + Associates 33 DAD Lifecycle: Advanced/Lean © Scott W. Ambler + Associates 34 Exercise: Modeling and Documentation on an Agile Project • Get back into your teams • Choose a type of project (described on following slides) that you will take an disciplined agile approach with: – Data warehouse – Web site – Pacemaker monitoring software • For 10 minutes, discuss: – How you would go about initial modeling. What are the primary models that you would choose? – What documents would you create and maintain throughout the project? – How much detailed modeling you would need to do throughout this sort of project? • A spokesperson should be prepared to share key learnings with the class © Scott W. Ambler + Associates 35 Project Type: Data Warehouse • Scenario: – Your organization is in the process of developing a data warehouse (DW) to support your business intelligence (BI) efforts. – There are 17 sources of legacy data. • Challenges: – This is the third attempt at a DW. The previous two efforts took a traditional, data-driven approach. – The second project team was supported by a parallel Meta-Data Management (MDM) project which was declared a success. – The second version of the DW was deployed but few business stakeholders found it useful. – The data in the second DW was of good quality but didn’t support the actual reporting needs of the stakeholders. – The source data has the usual inconsistencies (semantic differences, time differences,…) © Scott W. Ambler + Associates 36 Project Type: Web Site • Scenario: – Your organization needs to redevelop its website. – You want to add social media features, such as the ability to add comments on some pages and the ability to vote on the usefulness of pages, to the site. – You want to allow customers to update their customer records via the site. These records are currently stored in an internal CRM system. • Challenges: – All but two of the original development team have left your organization to become agile coaches. – The original team followed Scrum and believed they didn’t need to create any sort of supporting technical documentation. – The existing user interface (UI) design looks nice but usability of the site is very poor due to poor flow between pages and poorly designed pages in general. – The existing solution uses technologies that aren’t on your enterprise architecture road map © Scott W. Ambler + Associates 37 Project Type: Pacemaker Monitoring Software • Scenario: – The latest generation of pacemakers are Bluetooth enabled. – The goal is for someone to put their phone close to their chest so that the pacemaker can either report its recent history to the person’s doctor or if the pacemaker detects a serious problem it can call for help. – You are building the system that responds to the call made by the pacemakers. – The data upload packet from the pacemaker is in a well defined, industry-standard format. • Challenges: – You are working in a life-critical environment. – Your main competitor in the marketplace is already running beta trials of a similar solution. © Scott W. Ambler + Associates 38 Architecture Throughout the Lifecycle Architectural vision guides development efforts Architecture owner facilitates architectural decisions throughout the project Architecture spikes to explore a technical issue Architecture handbook and models updated as required Initial architectural envisioning Reduce risk early by proving the architecture works © 2012 Scott W. Ambler + Associates 39 Analysis Throughout the Lifecycle Discuss requirements during iteration planning/modeling Active stakeholder participation throughout the project Look-ahead modeling for upcoming complex work items Initial requirements envisioning Pre-project problem exploration Identify new needs during demos © Scott W. Ambler + Associates Analyze incoming requests from production 40 Design Throughout Construction Discuss design implications during iteration planning/modeling Test-Driven Design (TDD) throughout Construction Look-ahead modeling for upcoming complex work items Consider design issues of incoming requests from production © Scott W. Ambler + Associates 41 DAD is Goal-Driven, Not Prescriptive • There are various goals that a team needs to address throughout the lifecycle • For each goal there are several issues to consider • For each issue there are different techniques to address it • Each technique has its advantages and disadvantages • Your choices depend on the context of the situation faced by a team • Some “modeling-oriented” goals: – – – – Explore the initial scope Identify the initial technical strategy Address changing stakeholder needs Regularly produce a potentially consumable solution © Scott W. Ambler + Associates 42 Goal: Explore the Initial Scope © Scott W. Ambler + Associates 43 Goal: Identify Initial Technical Strategy © Scott W. Ambler + Associates 44 Goal: Address Changing Stakeholder Needs © Scott W. Ambler + Associates 45 Goal: Produce a Potentially Consumable Solution © Scott W. Ambler + Associates 46 Goal: Move Closer to a Deployable Release © Scott W. Ambler + Associates 47 Scaling/Tailoring Complexity Factors Domain Complexity Straightforward Geographic Distribution Team Size Very complex Co-located 2 Technical Complexity Global 1000s motivates Straightforward Very complex motivates Team Culture Organizational Distribution Single Outsourcing varies by Compliance affects Agile None Rigid Life critical affects Organizational Culture Agile Rigid © Scott W. Ambler + Associates 48 Organizing Large Teams Organizational options: • Feature teams: A subteam works on a feature from end to end. • Component teams: A subteam does all the work for a specific component. • Internal open source: A component is developed via open source techniques. © Scott W. Ambler + Associates 49 Exercise: AMDD and Team Size • Get back into your teams • Instructions: – Take 15 minutes to discuss how you would approach modeling and documentation for an agile team of 10 co-located people, a team of 30 people on the same floor, and a team of 100 people in the same building – Focus on how it would affect goals such as Explore the Initial Scope, Identify Initial Technical Strategy, and Move Closer to a Deployable Release • A spokesperson will share with the larger group a few key learnings © Scott W. Ambler + Associates 50 Geographically Distributed/Dispersed Teams © Scott W. Ambler + Associates 51 Exercise: AMDD and Geographic Distribution • Get back into your teams • Instructions: – Take 15 minutes to discuss how you would approach modeling and documentation for an agile team of 10 co-located people, a team of 10 people fully dispersed, and three teams of 10 people on three different continents – Focus on how it would affect goals such as Explore the Initial Scope, Identify Initial Technical Strategy, and Address Changing Stakeholder Needs • A spokesperson will share with the larger group a few key learnings © Scott W. Ambler + Associates 52 Copyright 2012 Scott W. Ambler + Associates 53 Challenges Adopting AMDD © Scott W. Ambler + Associates 54 Modeling is an Important Aspect of Disciplined Agile Delivery • Disciplined agilists: – Invest some up-front time exploring the initial scope – Invest some up-front time identifying a viable technical strategy – Work closely with enterprise professionals, including enterprise architects and operations professionals, to ensure that what they’re producing is enterprise compliant – Tailor their approach to meet the context of the situation that they face – Model throughout construction in a just-in-time (JIT) manner – Document throughout construction because they realize that documentation is part of their overall deliverable – Work closely with their stakeholders whenever they can, not just with stakeholder proxies such as product owners © Scott W. Ambler + Associates 55 AMDD at the Enterprise/Program Level © Scott W. Ambler + Associates 56 The Value of Modeling © Scott W. Ambler + Associates 57 Exercise: What Have You Learned? • Individually, on a sheet of paper, answer the following questions: – What three new things have you learned about modeling in general? – What three new things have you learned about how modeling occurs on an agile project? – What improvements in the way you approach modeling and documentation can you make on your current project team? – What still puzzles you? © Scott W. Ambler + Associates 58 A Disciplined Ending…. Please… – Take the opportunity to thank your teammates – we all learned together – Fill out the workshop evaluation form(s) – Turn in the evaluation(s) to the instructor © Scott W. Ambler + Associates 59 Thank You! scott [at] scottwambler.com Twitter: scottwambler AgileModeling.com AgileData.org Ambysoft.com DisciplinedAgileDelivery.com EnterpriseUnifiedProcess.com ScottWAmbler.com © Scott W. Ambler + Associates 60 Recommended Resources 61 © Scott W. Ambler + Associates Backup Slides © Scott W. Ambler + Associates 62 AMDD and Large Teams • Larger teams are often a response to greater domain complexity, technical complexity, or cultural challenges • Inception: – You will likely need to invest a bit more time exploring the initial requirements – You may need to take an “API First” or “Contract Model” approach to the architecture where you define the interface to components in detail early in the project – There will likely be a bit more initial specification • Construction: – Product Owners will need to coordinate requirement dependencies – Architecture Owners will need to coordinate technical dependencies – TDD may need to be enhanced with parallel independent testing © Scott W. Ambler + Associates 63 AMDD and Geographic Distribution • Geographically distributed/dispersed teams are usually the result of large teams or organizational culture • Inception: – You will likely need to invest a bit more time exploring the initial requirements – You will likely need to take an “API First” or “Contract Model” approach to the architecture where you define the interface to components in detail early in the project – There will likely be a bit more initial specification – Get key players together physically for Inception • Construction: – – – – – Dispersed members will need to coordinate with their co-workers regularly Product Owners will need to coordinate requirement dependencies Architecture Owners will need to coordinate technical dependencies TDD will likely need to be enhanced with parallel independent testing Get key players together for critical milestones © Scott W. Ambler + Associates 64 AMDD and Compliance • Inception: – Invest time to understand the true implications of the regulations regarding specification, documentation, and traceability – The regulations MAY require more detailed requirements and architecture specification, some traceability, and some level of formality of validation of the specifications • Construction: – You may need to adopt more formal modeling and documentation tooling – You may need to keep all artifacts in sync throughout construction – You may need to invest in traceability activities, or better yet in activities and tooling that automatically result in sufficient traceability – TDD may need to be enhanced with parallel independent testing • Transition: – You may need to hold final reviews and sign-offs of key artifacts – You may need to generate final artifact manifests, traceability trees, and so on © Scott W. Ambler + Associates 65