Applying Lean Startup
Principles to Agile Projects
Mike Hall
Three Beacons
mike@threebeacons.com
214.783.3936
www.synerzip.com
Webinar Series 2015
0
Lean Startup
www.synerzip.com
Webinar Series 2015
1
Applicability
It’s for everyone: startups, new product development, new features, etc.
www.synerzip.com
Webinar Series 2015
2
Mantra
“What if we found ourselves building something that nobody wanted? In
that case, what did it matter if we did it on time and on budget?”
The goal of any project is to figure out the
right thing to build.
www.synerzip.com
Webinar Series 2015
3
Thinking
Question at the start of a typical project:
• Can we build a solution for that problem?
Lean Startup questions at the start:
• Do consumers recognize they have a
problem?
• If there was a solution, would they buy it?
• Would they buy it from us?
• Can we build a solution for that problem?
• Can we build a sustainable business
around this product/service?
Ask “Should it be built?” instead of “Can it be built?”
www.synerzip.com
Webinar Series 2015
4
Framework
• Adaptable framework
• Based on scientific methods
• A journey of “discovery”
-
Subject the vision to constant
hypothesis testing
-
React to customer feedback
-
Bypass work that does not lead to
learning
-
Adapt to what the data is telling you
“Successful entrepreneurs had the foresight, ability, and tools to discover
which part of their plans were working brilliantly and which were misguided,
and adapt their strategies accordingly.
www.synerzip.com
Webinar Series 2015
5
Lean Startup - Principles
www.synerzip.com
Webinar Series 2015
6
Assumptions as Hypotheses
• Identify your project/feature assumptions (continuously)
• Reword them as hypotheses
“The XYZ change will prove that customers want to ...” (value)
“The ABC feature will increase new customers by at least 15%.” (growth)
Avoid:
•
•
•
Acting as if assumptions are true and proceeding anyway! – Leaps of Faith
Taking statements for granted
Reports from anyone other than the customer
www.synerzip.com
Webinar Series 2015
7
A Tale of Two Teams
• What to build?
• Clear baseline metric
• Passionate debates
• Hypothesis on how to improve metric
• Suits decide
• Experiments to test hypothesis
• Implement several features at a time
• Empirical data from customer usage
• Celebrates any positive perception
• Celebrates learning
www.synerzip.com
Webinar Series 2015
8
Experiments
• Think of your project as a set of small experiments
• Break business plan down to its component parts – and test them
• Define experiments to test each hypothesis
• Results of the experiments guide decisions about product direction
Case Study: Zappos
Experiments allow us to transition from guesses to knowledge.
www.synerzip.com
Webinar Series 2015
9
Validated Learning
• The result of the experiments
• Can be positive or negative changes
• Empirical data from the customer
• “Learn lessons early” rather than “build
features and fix bugs”
• Faster and more accurate than market
forecasting and classical business planning
The measure of an effective team is how much validated learning did we
achieve (as opposed to how much did we build).
www.synerzip.com
Webinar Series 2015
10
Small Batch Size
• Allows us to identify quality problems sooner
• Pull – each step pulls the parts needed from
the previous step, Toyota JIT production
• As soon as we formulate a hypothesis, run the
experiment as quickly as possible using the
smallest batch size to get the job done!
“Large batch sizes can create a death spiral of re-doing work.”
www.synerzip.com
Webinar Series 2015
11
Build – Measure - Learn
Decision!
from the data
Minimize time
through loop
the smallest
batch possible
qualitative and quantitative
www.synerzip.com
Webinar Series 2015
12
Minimum Viable Product
www.synerzip.com
Webinar Series 2015
13
Minimum Viable Product
• The resultant output of successive Build – Measure – Learn loops
• Remove/Avoid any effort that does not lead to learning
• Goal of MVP – test your hypotheses, achieve validated learning
• Decision after learning: pivot/persevere/quit
• Iterate toward launchable product
“The only way to win is
to learn faster than
anyone else.”
www.synerzip.com
Webinar Series 2015
14
Case Study: Dropbox
• Very popular web-based file-sharing service
• Initial MVP: a YouTube video
• Targeted to early adopters
• Beta waiting list went from 5,000 to 75,000 overnight
• Company now worth over $1B
www.synerzip.com
Webinar Series 2015
15
MVP Patterns
• Concierge MVP – personalized service as a learning activity
• Wizard of Oz MVP – behind the scenes humans doing the work
• Case Study: Aardvark
• Low-quality MVP
• Case Study: Craigslist
• Case Study: IMVU avatar teleportation
• Smoke test - marketing materials
• UI mockups
• Etc.
www.synerzip.com
Webinar Series 2015
16
Pivot, Persevere, or Quit
• Based on the validated learnings of an MVP, decide!
• Pivot – structured course correction designed to test new hypotheses
• Persevere – continue on with next set of hypotheses
• Quit – cancel the project and move on to the next one
“There is no bigger destroyer of creative potential than the
misguided decision to persevere.”
www.synerzip.com
Webinar Series 2015
17
Case Study: Potbelly Sandwiches
• Started out as an antique store
• Began selling sandwiches to drive traffic to the stores in
the hopes of selling more antiques
• Lines formed out the door
• Pivoted to a sandwich store
• Today over 280 sandwich stores nationwide
www.synerzip.com
Webinar Series 2015
18
Pivot Types
• Zoom-in pivot – refocus product on what was previously considered one feature
• Zoom-out pivot – single feature is inadequate, so add features
• Customer segment pivot
• Customer need pivot (Potbelly)
• Platform pivot
• Business architecture pivot
• Value capture pivot
• Engine of growth pivot
• Channel pivot
• Technology pivot
Pivots take courage!
www.synerzip.com
Webinar Series 2015
19
Others
• Innovation Accounting
• Engine of Growth
• Adaptive Organization
www.synerzip.com
Webinar Series 2015
20
Application to
Agile Projects
www.synerzip.com
Webinar Series 2015
21
Case Study: NAV-API
• Navigation platform supports 3rd-party apps integration
• Goal is to accurately measure usage analytics
• Measurement messages are
- Start/Stop driving
- Route to destination
- Travel guide viewing, etc.
• Approach 1: downloadable SDK integrated into apps
- Caches measurements, sends every 5 minutes to server
- Re-issue app when SDK changes, re-certify
- SDK execution can crash 3rd-party app
• New product: web service to receive measurement messages
- Defined RESTful API called NAV-API
www.synerzip.com
Webinar Series 2015
22
Get Started: Hypotheses-Driven Vision
• Develop a classic vision board
• List assumptions
• Continue to identify assumptions as you go
• Translate implicit assumptions into explicit testable hypotheses
• List hypotheses
www.synerzip.com
Webinar Series 2015
23
NAV-API Vision Board
Vision Statement
For clients who have a need for capturing usage analytics on their connected applications, Navigation API (NAV-API) is a cloud-based
service that provides a simple easy-to-understand way of reporting measurements.
Unlike classic embedded SDK approaches, the NAV-API will provide a direct reporting experience based on web service calls.
Target Group
3rd-party app customers
• Yelp
• Trip Advisor
• TripIt
• Travefy
• AAA
• Tripomatic
• Hertz
• Avis
• TouristEye
• TripCase
• WorldMate
Needs
• Ease of measurement reporting
• One solution for all digital
• Increase revenues
• Use of familiar programmatic
• Cloud-based
• Satisfy pent-up
• Transparent evolution
approach
• Less software development
• No need to download/integrate
• Linear scaling as demand
Assumptions
• Customers will prefer NAV-API over the
• Increase digital
• 1-stop-shop
• Fault tolerant
SDK
demand
footprint
grows
Hypotheses
• > 80% of all customers will prefer NAV-API
• NAV-API can be self-certified by customers
embedded SDK
• NAV-API will make it easier to certify apps
• NAV-API can handle large amount of users
• NAV-API can handle 50K simultaneous
sessions
• NAV-API will need a super-fast DB
• Redis is the best DB for NAV-API
• An early release to friendly customers will
• An initial release can be built with limited (but
provide good feedback
www.synerzip.com
Value
Product
Webinar Series 2015
24
valuable) functionality for early adopters
Get Empathetic: Knowledge Broker Personas
• A special form of persona
• But emphasizes the knowledge sharing that each can bring
• Customer Archetype – humanizes the proposed target user
Knowledge Sharing
• Can explain why certain features are more important
• Can describe his previous SUVs and what he liked/disliked
• Will give an opinion on our new Sync+ system
• We can determine how likely he will upgrade to a full-size SUV in
the future
www.synerzip.com
Webinar Series 2015
25
Get Organized: Learning Maps
• Create a story map on a wall
• But organize and prioritize it by
Hypothesis from left to right
• Which will deliver the most learning?
• Which learnings are most crucial?
• Which learnings reduce risk?
• Which are most crucial in answering
“Are we building the right product?”
• For each hypothesis, name the user
stories and/or work items
• Prioritize the user stories top to bottom
www.synerzip.com
Webinar Series 2015
26
NAV-API Learning Map
Hypotheses
Experiments:
Stories, Work Items
www.synerzip.com
Webinar Series 2015
27
Tee It Up: Experiment Backlogs
• Similar to Scrum product backlog
• But is learning-based prioritization
• List of all experiments 1..n
• Stories, work items, research, etc.
• Tagged with Hypothesis name/description
www.synerzip.com
Webinar Series 2015
28
NAV-API Experiment Backlog
Epics + User Stories (Digital Engineering)
Displaying 86 issues at 14/Apr/15 1:15 PM.
Project
Key
Summary
Cloud API NMAPI-76 API Issues
Issue Type Status Resolution
Epic
Open Unresolved
Created
3/13/2015 19:09
NAV API
NAVAPI-4 Start session
Story
Open
Unresolved
3/13/2015 14:17
NAV API
NAVAPI-5 Start drive
Story
Open
Unresolved
3/13/2015 14:17
Updated
Description
3/19/2015 14:24 Hypothesis: NAV-API is resilient enough to appropriately handle session id and data value errors.
3/23/2015 9:00 As an application, I want to start a session, so that I can begin reporting metrics to the downsteam
systems.
• When I request a session, I expect that my session is started by NAV-API.
• When I request a session, I expect to receive a successful return code from NAV-API.
• When I request a session, I expect to receive a unique session id that I can use in subsequent NAVAPI calls. I also expect to receive an opt-out URL that I can display in my Privacy page.
• When I request a session and NAV-API is unable to start my session, I expect to receive an error
code that indicates the reason.
• When I request a session and my device/application is opted-out, I expect to receive an error code
that indicates opted-out as the reason.
• When I request a session, I expect NAV-API to read in my config file from the Config system. The
Config file contains variable name mappings that allow me to use my own defined variables instead of
the defaults.
3/19/2015 16:09 As an application, I want to signal start drive, so that I can begin collecting information about the trip.
Acceptance Criteria
• When I start driving. I expect to receive a successful return code from NAV-API
• When there is an error in the data transmission to NAV-API, I expect to receive an error response
code.
NAV API
NAVAPI-8 Stop drive
Story
Open
Unresolved
3/13/2015 14:18
3/23/2015 8:59 As an application, I want to report when I stop driving, so that I can complete information regarding a
trip.
• When I am stop driving, I expect NAV-API to accept the data as defined in the API such as event,
UTC time, and type.
• When I am stop driving, I expect NAV-API to accept the data as defined in the API such as event,
offset time, and type.
• When I stop driving, I expect an OK response code.
• When there is an error in the data transmission to NAV-API, I expect an error response code.
NAV API
NAVAPI83
Route to destination
Story
Open
Unresolved
3/16/2015 10:25
3/20/2015 12:14 As an application, I want to report when a route destination is in progress, so that I can complete
information regarding a routed trip.
• When I am routing, I expect NAV-API to accept the data as defined in the API such as event, UTC
time, and type.
• When I am routing, I expect NAV-API to accept the data as defined in the API such as event, offset
time, and type.
• When I start routing, I expect an OK response code.
• When there is an error in the data transmission to NAV-API, I expect an error response code.
NAV API
NAVAPI16
View travel guide
Story
Open
Unresolved
3/13/2015 14:20
3/23/2015 9:02 As an application, I want to report when a travel guide is viewed, so that I can complete information
regarding a view session.
• When I am view a travel guide, I expect NAV-API to accept the data as defined in the API such as
event, UTC time, and type.
• When I am view a travel guide, I expect NAV-API to accept the data as defined in the API such as
event, offset time, and type.
• When I start viewing, I expect an OK response code.
• When there is an error in the data transmission to NAV-API, I expect an error response code.
NAV API
NAVAPI15
Stop session
Story
Open
Unresolved
3/13/2015 14:19
3/23/2015 9:52 As an application, I want to stop a session, so that I can close out metrics to the downsteam
systems.
• When I stop a session, I expect that my session is stopped by NAV-API.
• When I stop a session, I expect to receive a successful return code from NAV-API.
• When I stop a session and NAV-API is unable to stp my session, I expect to receive an error code
that indicates the reason.
• When I stop a session and my device/application is opted-out, I expect to receive an error code that
indicates opted-out as the reason.
www.synerzip.com
Webinar Series 2015
29
Get Focused: Minimum Learning Product (MLP)
• Similar to MVP, but much smaller
• Learning-based, not viable productbased
• Smallest chunk of the Learning Map
that can be developed to learn
something important
• Real or mock form
• Goal is to get just enough learnings
- Then pivot, persevere, or quit
• Choosing the MLP replaces classic
Scrum sprint planning
- Break into tasks if it helps
www.synerzip.com
Webinar Series 2015
30
NAV-API Minimum Learning Product
MLP
www.synerzip.com
Webinar Series 2015
31
Experiment Test Iteration
• Experiment Test Iteration (ETI)
• Similar to Scrum sprint but variable time length
- Depends on size of experiment
- Get through Build/Measure/Learn as quickly as possible!
Scrum: fixed iteration length
ETI: variable iteration length
ETI 1 ETI 2
3 days 5 days
www.synerzip.com
ETI 3
9 days
ETI 4
17 days
Webinar Series 2015
ETI 5
6 days
ETI 6
7 days
32
Build It: ETI
• Build out the MLP
• Measure progress based on validated learning
• Use modified storyboard showing Validated column
Story
www.synerzip.com
To Do
In Work
Done
Webinar Series 2015
Validated
33
Get More Data: Learning Results Period
• Obtaining results from knowledge brokers
• Sometimes the validation takes longer than the ETI
• Run this in parallel with the next ETI
- Defer pivot/persevere/quit decision until this data is in
Pivot/Persevere/Quit
ETI3 Learning
Results Period
ETI 1 ETI 2
3 days 5 days
www.synerzip.com
ETI 3
9 days
ETI 4
17 days
Webinar Series 2015
ETI 5
6 days
ETI 6
7 days
34
Demo It: ETI Review
• Dev team demos their progress
• Discuss learnings obtained from the Learning Results Period
• Experiment findings are discussed with stakeholders
• Decision: pivot, persevere, or quit
www.synerzip.com
Webinar Series 2015
35
Think About It: ETI Retrospective
• Team discusses
- What went well, what did not go well
- How to get better
• A spirit of “continuous improvement”
• Plus:
- How is the team feeling about the assumptions?
- Are there any not identified previously?
www.synerzip.com
Webinar Series 2015
36
Rinse & Repeat: MLPs
• Build series of MLPs to reach final launchable product
• Use innovation accounting to “tune the growth engine”
• Be courageous in pivot/persevere/quit decisions
www.synerzip.com
Webinar Series 2015
37
Team Dynamics
• Scrum team becomes a small “innovation factory”
- Responsible for code and/or artifacts that prove/disprove a hypothesis
- Continuous innovation
• Practicing the art of “genchi genbutsu”
- “Go and see”
- The only way to truly understand the requirements is to get out of the
office and spend time with the customer
- Gemba – the real place
- Don’t rely on information from other sources
www.synerzip.com
Webinar Series 2015
38
Gemba Walk
• Gemba Walk
- Go see the actual process
- Purposeful attempt to learn what is really going on
- Direct customer interaction
- Ask questions
- Show respect
- Learn
www.synerzip.com
Webinar Series 2015
39
Team Dynamics (cont)
• ScrumMaster becomes “shusa”
• Chief engineer responsible for guiding the product to success
• Guides team on experiments and MLPs to product
www.synerzip.com
Webinar Series 2015
40
Conclusion
www.synerzip.com
Webinar Series 2015
41
Conclusion
• Lean Startup principles can and should be used in
Agile projects
- To help insure we build the right thing
• Approximately 10 techniques presented, but there
are probably even more
• This could be the next major evolution of Agile!
“If we stopped wasting people’s time, what
would they do with it?
We have no real concept of what is possible.”
www.synerzip.com
Webinar Series 2015
42
Introducing: Gemba
• Gemba: a validated learning Agile method
Gemba
Scrum
www.synerzip.com
Lean Startup
Webinar Series 2015
Lean
43
Gemba Manifesto
We value
• Validated learning over reasonable assumptions
• Data-driven decisions over plausible-sounding arguments
• Building minimum learning products over additional features
• The courage to build the right thing over something that might work
www.synerzip.com
Webinar Series 2015
44
Three Beacons
Agile Training
Scrum team training
Lean Startup training
Intro to Agile
User Stories
Extreme Programming
Agile Consulting
Onsite
Skype-based
© Three Beacons LLC, 2015
www.synerzip.com
Webinar Series 2015
45
THE END
Mike Hall
Three Beacons
mike@threebeacons.com
214.783.3936
www.synerzip.com
Webinar Series 2015
46
Backups
www.synerzip.com
Webinar Series 2015
47
Innovation Accounting
• Measure the progress of innovation
towards validated learning – instead of
burn rate or $
• Three steps
•
•
•
Use MVP to establish real data
Tune the engine from baseline towards ideal
Pivot, persevere, or quit
• Use actionable metrics – clear
cause&effect
•
•
•
Split-test of a feature caused 20% increase in
sales
Per-customer metrics
Cohort metrics – groups of customers
• Avoid vanity metrics
•
•
Number of hits to a website
Action to take is not obvious
www.synerzip.com
“If you are building the wrong thing,
optimizing the product or its marketing
will not yield significant results.”
Webinar Series 2015
48
Engine of Growth
• Use a small set of actionable metrics
•
•
•
•
•
Customer acquisition cost
Activations
Retention
Revenue
Referrals
• Consider viral coefficient
• How many friends will each customer bring?
• Case Study: Hotmail
• “Tune” the engine every time learning occurs
www.synerzip.com
Webinar Series 2015
49
Adaptive Organization
• Auto-adjust process and performance based on
current learnings
• Andon cord – anyone can stop the production line!
• Slow down – invest in preventing issues
• Ask “Why?” 5 times to get to root cause
Avoid
•
Handoffs and approvals
•
Making decisions on plausible-sounding arguments
•
Low quality products
•
Defects
www.synerzip.com
Webinar Series 2015
50
www.synerzip.com
Hemant Elhence
hemant@synerzip.com
469.374.0500
www.synerzip.com
Webinar Series 2015
51
•51
84
Synerzip in a Nutshell
 Software product development partner for small/mid-sized technology
companies
•
•
•

Dedicated team of high caliber software professionals for each client
•
•
•

Seamlessly extends client’s local team offering full transparency
Stable teams with very low turn-over
NOT just “staff augmentation, but provide full management support
Actually reduces risk of development/delivery
•
•


Exclusive focus on small/mid-sized technology companies, typically venturebacked companies in growth phase
By definition, all Synerzip work is the IP of its respective clients
Deep experience in full SDLC – design, dev, QA/testing, deployment
Experienced team – uses appropriate level of engineering discipline
Practices Agile development – responsive yet disciplined
Reduces cost – dual-site team, 50% cost advantage
Offers long-term flexibility – allows (facilitates) taking offshore team
captive – aka “BOT” option
www.synerzip.com
Webinar Series 2015
52
Synerzip Clients
www.synerzip.com
Webinar Series 2015
53
Join Us In Person
Agile Texas 2015 Tour
Presented by
Hemant Elhence & Vinayak Joglekar
www.synerzip.com
Webinar Series 2015
54
Next Webinar
Technical Topic: From Hadoop to Spark
Complimentary Webinar:
Wednesday, August 19, 2015 @ Noon CST
Presented by: Mark Kerzner
Co-Founder,
Elephant Scale
www.synerzip.com
Webinar Series 2015
55
Connect with Synerzip
@Synerzip_Agile
linkedin.com/company/synerzip
facebook.com/Synerzip
Hemant Elhence
hemant@synerzip.com
469.374.0500
www.synerzip.com
Webinar Series 2015
56