Lean Analytics
Use data to build a
better business faster.
@byosko | @acroll
www.leananalyticsbook.com
@leananalytics
Some background on Lean,
analytics, metrics, and
segmentation.
Most startups don’t know what they’ll be
when they grow up.
Paypal
first built for
Palmpilots
Hotmail
was a
database
company
Freshbooks
was invoicing
for a web
design firm
Flickr
was going to
be an MMO
Wikipedia
was to be
written by
experts only
Twitter
was a
podcasting
company
Mitel
was a
lawnmower
company
Autodesk
made
desktop
automation
Product/market
hypothesis
Trial startup
You
are
here
Product/mar
ket
hypothesis
Trial startup
Possible
problem
space
Trial startup
Product/market
hypothesis
Trial startup
Product/mar
ket
hypothesis
Kevin Costner is a lousy entrepreneur.
•Don’t sell what you can make.
Make what you can sell.
5 things you
need to know
about metrics
Qualitative or Quantitative
Exploratory or Reporting
Vanity or Actionable
Correlated or Causal
Leading or Lagging
Qualitative
Quantitative
Unstructured,
anecdotal,
revealing, hard to
aggregate.
Numbers and stats;
hard facts but less
insight.
Warm and fuzzy.
Cold and hard.
http://www.flickr.com/photos/zooboing/8388257248/
http://www.flickr.com/photos/x1brett/4665645157/
Exploratory
Reporting
Speculative, trying
to find unexpected
or interesting
insights.
Predictable, keeping
you abreast of
normal, managerial
operations.
http://www.flickr.com/photos/50755773@N06/5415295449/
http://www.flickr.com/photos/elwillo/4737933662/
Donald Rumsfeld on analytics
we know
Are facts which may be wrong and
should be checked against data.
know
we don’t
know
Are questions we can answer by
reporting, which we should
baseline & automate.
Things we
don’t
know
we know
we don’t
know
Are intuition which we should
quantify and teach to improve
effectiveness, efficiency.
Are exploration which is where
unfair advantage and interesting
epiphanies live.
(Or rather, Avinash Kaushik channeling Rumsfeld)
Vanity
Actionable
Picks a
direction.
Makes you feel
good, but doesn’t
change how you’ll
act.
http://www.flickr.com/photos/lostseouls/807253220/
http://www.flickr.com/photos/aussiegall/6382775153/
Hits
A metric from the early, foolish days of the Web.
Count people instead.
Page views
Marginally better than hits. Unless you’re displaying
ad inventory, count people.
Visits
Is this one person visiting a hundred times, or are a
hundred people visiting once? Fail.
Unique visitors
Followers/friends/l
ikes
Time on site, or
pages/visit
Emails collected
Number of
downloads
This tells you nothing about what they did, why
they stuck around, or if they left.
Count actions instead. Find out how many
followers will do your bidding.
Poor version of engagement. Lots of time spent on
support pages is actually a bad sign.
How many recipients will act on what’s in them?
Outside app stores, downloads alone don’t lead to
lifetime value. Measure activations/active accounts.
Correlated
Causal
Two variables that
change in similar
ways , perhaps
because they’re
linked to something
else.
An independent
factor that directly
impacts a
dependent one.
Summer
Ice cream
consumption
Correlated
Drowning
Causality is a superpower, because it lets you
change the future.
Correlation lets you
predict the future
Causality lets you
change the future
“I will have 420
engaged users and
75 paying customers
next month.”
“If I can make more
first-time visitors stay
on for 17 minutes I
will increase sales in
90 days.”
Find
correlation
Test causality
Optimize the
causal factor
Leading
Lagging
Number today that
shows metric
tomorrow—makes
the news.
Historical metric that
shows how you’re
doing—reports the
news.
A leading indicator for e-commerce
How many of
your customers
buy a second
time in 90 days?
1-15%
15-30%
>30%
Then you are in
this mode
Your customers
will buy from you
You are just
like
70%
Acquisition
Once
Hybrid
2-2.5
20%
per year
of retailers
Loyalty
of retailers
>2.5
10%
per year
of retailers
Focus on
Low CAC,
high
checkout
Increasing
returns
Loyalty,
inventory
expansion
(Thanks to Kevin Hilstrom for
this.)
Segments, cohorts, A/B, and multivariates
Cohort:
Comparison of
similar groups
along a timeline.
Segment:
Cross-sectional ☀
comparison of all
people divided by
☁
some attribute
(age, gender, etc.)
A/B test:
Changing one
thing (i.e. color)
and measuring
the result (i.e.
revenue.)
☀
☁
☀
☁
Multivariate
analysis
Changing several
things at once to see
which correlates
with a result.
Why use cohorts? Here’s an example.
Is this
company
growing or
stagnating?
How about
now?
January
February
March
April
May
Rev/customer
$5
$5
$4
$4
$5
Cohort
1
2
3
4
5
January
$5
$3
$2
$1
$1
$6
$4
$2
$1
$7
$6
$5
$8
$7
February
March
April
May
$9
Why use cohorts? Here’s an example.
Look at the
same data
in cohorts
Cohort
1
2
3
4
5
January
$5
$3
$2
$1
$1
February
$6
$4
$2
$1
March
$7
$6
$5
April
$8
$7
May
$9
Averages
$7
$5
$3
$1
$1
Frameworks we borrowed from.
Dave McClure’s Pirate metrics
Acquisition
AARRR
How do your users become aware of you?
SEO, SEM, widgets, email, PR, campaigns, blogs ...
Activation
Do drive-by visitors subscribe, use, etc?
Retention
Does a one-time user become engaged?
Revenue
Do you make money from user activity?
Referral
Features, design, tone, compensation, affirmation ...
Notifications, alerts, reminders, emails, updates...
Transactions, clicks, subscriptions, DLC, analytics...
Do users promote your product?
Email, widgets, campaigns, likes, RTs, affiliates...
Eric Ries’
Three engines
Approach
Math that
matters
Stickiness
Virality
Price
Keep people
coming back
Make people
invite friends
Spend revenue
getting customers
Get customers How many they Customers are
faster than you tell, how fast they worth more than
lose them
tell them
they cost to get
Ash Maurya’s
Lean canvas
Lean Canvas box
Some relevant metrics
Problem
Respondents who have this need; respondents who are aware of
having the need.
Solution
Respondents who try the MVP; engagement; churn; most-used/leastused features; people willing to pay.
Feedback scores; independent ratings; sentiment analysis; customerUnique Value Proposition worded descriptions; surveys; search and competitive analysis.
Customer Segments
How easy it is to find groups of prospects; unique keyword segments;
targeted funnel traffic from a particular source.
Channels
Leads/cust per channel; viral coefficient and cycle; net promoter score;
affiliate margins; open, click-through rate; PageRank; message reach.
Unfair Advantage
Respondents’ understanding of the USP; patents; brand equity;
barriers to entry; number of new entrants; exclusivity of relationships.
Revenue Streams
Lifetime customer value; Average revenue per user; Conversion rate;
Shopping cart size; Click-through rate.
Cost Structure
Fixed costs; cost of customer acquisition; cost of servicing the nth
customer; support costs; keyword costs.
Sean Ellis’
Startup growth pyramid
Scale
growth
Step on the gas in
new markets,
products, channels.
Stack the odds
Find a defensible
unfair advantage
and tweak it.
Product/market fit
Decide what you
sell to whom, then
prove it.
The business model flipbook.
Acquisition
channel
Selling
tactic
Revenue
model
Product
type
Delivery
model
How the visitor,
customer, or user finds
out about the startup.
Paid advertising
Search Engine Mgmt.
Social media outreach
Inherent virality
Artificial virality
Affiliate marketing
Public relations
App/ecosystem mkt.
Banner on Informationweek.com
High pagerank for ELC in kid’s toys
Active on Twitter i.e. Kissmetrics
Inviting team member to Asana
Rewarding Dropbox user for others’ signups
Sharing a % of sales with a referring blogger
Speaker submission to SXSW
Placement in the Android market
What the startup does
to convince the visitor
or user to become a
paying customer.
Simple purchase
Discounts & incentives
Free trial
Freemium
Pay for privacy
Free-to-play
Buying a PC on Dell.com
Black Friday discount, loss leader, free ship
Time-limited trial such as fitbit Premium
Free tier, relying on upgrades, like Evernote
Free account content is public, like Slideshare
Monetize in-app purchases, like Airmech
How the startup
extracts money from its
visitors, users, or
customers.
One-time transaction
Recurring subscription
Consumption charges
Advertising clicks
Re-sale of user data
Donation
Single purchase from Fab
Monthly charge from Freshbooks
Compute cycles from Rackspace
PPC revenue on CNET.com
Twitter’s firehose license
Wikipedia’s annual campaign
What the startup does
in return. May be a
product or service; may
be hardware or
software; may be a
mixture.
Software
Platform
Merchandising
User-generated content
Marketplace
Media/content
Service
Oracle’s accounting suite
Amazon’s EC2 cloud
Thinkgeek’s retail store
Facebook’s status update
AirBnB’s list of house rentals
CNN’s news page
A hairstylist
Hosted service
Digital delivery
Physical delivery
Salesforce.com’s CRM
Valve purchase of desktop game
Knife shipped from Sur La Table
How the product gets to
the customer.
Business
aspect
Flipbook
page(s)
Dropbox example
Inherent virality.
Artificial virality.
Sharing files with others.
Free storage when others sign up.
Freemium.
Limited-capacity accounts are free;
subscribe when you need more.
Recurring
subscription.
$99/year, monthly fees, enterprise
tiers.
Product
type
Platform.
Storage-as-a-service with APIs,
collaboration, synchronization tools.
Delivery
model
Hosted service.
Digital delivery.
Cloud storage, web interface.
Desktop client software.
Acquisition
channel
Selling
tactic
Revenue
model
The six business models.
E-commerce
TL;DR:
•Are you focused on loyalty or acquisition?
•Pricing matters more than you think
•Don’t overlook logistics, delays, and ratings
•Old “average conversion rates”
e-Commerce: metrics that matter
• Conversion rate (the number of visitors who buy something.)
• Purchases/year (the number of purchases made by each customer per year)
• Average shopping cart size (the amount of money spent in a purchase.)
• Abandonment (the percentage of people that begin to make a purchase, and
then don’t.)
• Cost of customer acquisition (the money spent to get someone to buy
something.)
• Top keywords driving traffic to the site.
• Top search terms that lead to revenue; top search terms that don’t have any
Business
aspect
Flipbook
page(s)
E-commerce example
Acquisition-focused
(Virality, SEO/SEM,
paid advertising).
Word of mouth, sharing purchases,
affiliates. Less emphasis on loyalty
programs or return purchases.
Selling
tactic
Simple purchase, with
discounts.
Seasonal specials, free shipping.
Revenue
model
One-time transaction.
Price of purchase.
Merchandising.
Bulk-breaking, assortment, and
distribution of physical goods.
Physical delivery.
Ground shipping of purchases via
courier service.
Acquisition
channel
Product
type
Delivery
model
Software-as-a-Service
TL;DR:
•Eventually, focus on Customer Acquisition Payback
•Engagement varies by intended use of the app (i.e. CRM versus travel
booking)
•Credit cards up front have a huge effect
•Freemium is a sales tactic, not a business model
•Churn, acquisition cost, and lifetime value
•Subscriptions may be a bad thing (per-transaction pricing is an option too.)
SaaS: metrics that matter
• Attention (how effectively can you get people to visit?)
• Enrollment (how many visitors become users?)
• Stickiness (how much do customers use the product?)
• Conversion (how many users become paying customers?)
• Revenue per Customer (how much money do you make from a customer per
period?)
• Customer Acquisition Cost (how much does it cost to get a paying user?)
• Virality (how likely are customers to invite others and spread the word?)
Business
aspect
Acquisition
channel
Selling
tactic
Revenue
model
Product
type
Delivery
model
Flipbook
page(s)
SaaS example
Inherent virality.
Invite others to a collaborative
project as part of the natural use of
the tool.
Free trial.
Credit card up front, no charge for 30
days.
Recurring
subscription.
$20 a month, per seat.
Software.
Collaborative project management
tool.
Hosted service.
Run on a cloud platform, delivered
via a web browser.
Mobile
TL;DR:
•App stores make or break you
•In-app revenue is the only way to make a living
Mobile: metrics that matter
• Downloads: how many people have downloaded the application
• Related metrics: app store placement, downloads and ratings
• Customer Acquisition Cost: How much does it cost to get a paying use?
• Launch rate: What percentage launch the app and create an account?
• Percent Active Users/Players: What percent use it on a daily/monthly basis?
• This is your daily active users (DAU), and monthly active users (MAU)
• Percentage of users who pay
• Time to first purchase
Business
aspect
Acquisition
channel
Selling
tactic
Revenue
model
Product
type
Delivery
model
Flipbook
page(s)
Mobile app example
App/ecosystem
market, paid ads in
other games.
Lobbying Apple & Android; paid
downloads.
Free to play.
No charge for basic game.
One-time
transactions.
Re-selling user data.
Purchase of in-game components.
Licensing of usage data to 3rd party.
Software.
Collaborative project management
tool.
Digital delivery.
Game and extra content delivered to
the mobile device.
Media
TL;DR:
•Advertising is a complicated, many-faced beast
Media: metrics that matter
• Audience and churn: How many people visit the site, and do they keep
coming back?
• Ad inventory: How many impressions do we have that we can monetize?
• Ad rates: What might we be paid for those impressions, based on the content
we cover the people who visit us?
• Click-through rates: How many of those impressions actually turn into
money?
• The content/advertising balance: Are we properly balancing money, ad
inventory, and content to maximize overall performance?
Business
aspect
Flipbook
page(s)
Media example
SEO/SEM, social
media outreach,
app/ecosystem
market.
Publishing to social and RSS feeds;
promoting tablet/mobile app.
N/A (no transaction).
Have good content; make it relevant
to desirable ad placement.
Ad clicks.
Banner, PPC, and affiliate links
generate revenue.
Product
type
Media/content.
News site with editorial content.
Delivery
model
Hosted service, digital
delivery.
Media is online but also available
through downloadable app.
Acquisition
channel
Selling
tactic
Revenue
model
User-generated content
TL;DR:
•Content virality and user virality
•Trying to move users up the engagement funnel
•More time on fraud than you expect
•Notifications and email are the real user interface
•Passive content creation is on the horizon
UGC: metrics that matter
• Number of engaged visitors: How often do people come back, and how long
do they stick around?
• Content creation: What percentage of visitors interacts with content in some
way, from creating it to voting on it?
• Engagement funnel changes: How well is it moving people to more engaged
levels of content over time?
• Value of created content: What is the business benefit of content?
• Content sharing and virality: How does content get shared, and how does
this drive growth?
• Notification effectiveness: When we tell a user something, do they act on it?
UGC engagement funnels
Business
aspect
Acquisition
channel
Selling
tactic
Revenue
model
Product
type
Delivery
model
Flipbook
page(s)
UGC example
SEO/SEM, social
media outreach,
natural/artificial
virality.
Publishing to social and RSS feeds;
encouraging inviting & sharing with
friends.
N/A (no transaction).
Have good content; make it relevant
to desirable ad placement.
Ad clicks, re-sale of
user data.
Banner, PPC, and affiliate links;
licensing user data feed.
User-generated
content.
Visitors submit and vote on content.
Hosted service.
Available via web and mobile
browsers.
2-sided market
TL;DR:
•Focus on the money side
•Breaking the chicken-and-egg
2-sided market: metrics that matter
• Buyer and seller growth: The rate at which you’re adding new buyers and
sellers, as measured by return visitors.
• Inventory growth: The rate at which sellers are adding inventory (such as
new listings,) as well as completeness of those listings.
• Search effectiveness: What buyers are searching for, and whether it
matches the inventory you’re building.
• Conversion funnels: The conversion rates for items sold, and any
segmentation that reveals what helps sell items (such as the professional
photographs of a property mentioned in the AirBnB example earlier in the
book.)
• Ratings and signs of fraud: The ratings for buyers and sellers, signs of
fraud, and tone of the comments.
Business
aspect
Acquisition
channel
Selling
tactic
Revenue
model
Product
type
Delivery
model
Flipbook
page(s)
2-sided market example
Loyalty-focused
(Virality, email,
engagement).
More emphasis on returning users
(stickiness) than acquiring new ones.
Simple purchase.
Emphasis on search results,
accuracy of buyer/seller matches.
One-time transaction;
subscription
Percentage of each transaction as a
fee; premium vendor
services/prominent listing are extra.
Marketplace.
Bringing together buyers and sellers
for a price.
Physical delivery.
Labelling and logistics services for
sellers run through the site.
The five stages of Lean Analytics
Where is the risk?
Real need?
Key: Empathy
Good product?
Sustainable biz?
Healthy market?
Successful exit?
Key: Growth rate
Right solution?
Key: Stickiness
Key: Virality
Key: Revenue
Key: Scale
Combining stage and model to find your OMTM.
What’s your OMTM?
Ecommerce
Empathy
Stickiness
Virality
2-sided
market
Scale
User-gen
content
Media
Interviews; qualitative results; quantitative scoring; surveys
Loyalty,
conversion
Inventory,
listings
CAC, shares,
SEM, sharing
reactivation
(Money from transactions)
Revenue
SaaS
Mobile
app
Engagement, Downloads,
churn
churn, virality
Content,
spam
Traffic, visits,
returns
Inherent
virality, CAC
Invites,
sharing
Content
virality, SEM
WoM, app
ratings, CAC
(Money from active users)
Transaction,
CLV
Transactions,
commission
Upselling,
CAC, CLV
CLV,
ARPDAU
Affiliates,
white-label
Other
verticals
API, magic #, Spinoffs,
mktplace
publishers
(Money from ad clicks)
Ads,
donations
CPE, affiliate
%, eyeballs
Analytics,
user data
Syndication,
licenses
Example: a restaurant
• Empathy: Before opening, the owner first learns about the diners in its area,
their desires, what foods aren’t available, and trends in eating.
• Stickiness: Then he develops a menu and tests it out with consumers,
making frequent adjustments until tables are full and patrons return regularly.
He’s giving things away, testing things, asking diners what they think. Costs
are high because of variance and uncertain inventory.
• Virality: He starts loyalty programs to bring frequent diners back, or to
encourage people to share with their friends. He engages on Yelp and
Foursquare.
• Revenue: With virality kicked off, he works on margins—fewer free meals,
tighter controls on costs, more standardization.
• Scale: Finally, knowing he can run a profitable business, he pours some of the
revenues into marketing and promotion. He reaches out to food reviewers,
travel magazines, and radio stations. He launches a second restaurant, or a
franchise based on the initial one.
Example: a software company
• Empathy: The founder finds an unmet need, often because she has a background in a
particular industry or has worked with existing solutions that are being disrupted.
• Stickiness: She meets with an initial group of prospects, and signs contracts that look more
like consulting agreements, which she uses to build an initial product. She’s careful not to
commit to exclusivity, and tries to steer customers towards standardized solutions, charging
heavily for custom features. She supports the customers directly from the engineering team
until the product is stable and usable.
• Virality: Product in hand, she asks for references from satisfied customers, and uses them
as testimonials. She starts direct sales, and grows the customer base. She launches a user
group, and starts to automate support. She releases an API, encouraging third-party
development and scaling potential market size without direct development.
• Revenue: She focuses on growing the pipeline, sales margins, and revenues while
controlling costs. Tasks are automated, outsourced, or offshored. Feature enhancements are
scored based on anticipated payoff and development cost. Recurring license and support
revenue becomes an increasingly large component of overall revenues.
• Scale: She signs deals with large distributors, and works with global consulting firms to have
them deploy and integrate her tool. She attends trade shows to collect leads, carefully
measuring cost of acquisition against close rate and lead value.
Lean Analytics lifecycle
for an enterprise-focused startup
Stage
Do this
Fear this
Consulting to test ideas and
bootstrap the business
Lock-in, IP
control, overfitting
Stickiness
Standardization and integration;
shift from custom to generic
Ability to
integrate; support
Virality
Word of mouth, references, case
studies
Bad vibes;
exclusivity
Revenue
Growing direct sales, professional
services, support
Pipeline, revenue
recognition, comp
Scale
Channels, analysts, ecosystems,
APIs, vertically targeted products
Crossing the
chasm; Gorillas
Empathy
The Lean Analytics lifecycle
for an Intrapreneur
Stage
Do this
Fear this
Get buy-in
Political fallout
Entitled, aggrieved
customers
Stickiness
Find problems; don’t test demand.
Skip the business case, do
analytics
Know your real minimum based on
expectations, regulations
Virality
Build inherent virality in from the
start; attention is the new currency
Luddites who don’t
understand sharing
Revenue
Consider the ecosystem, channels,
and established agreements
Channel conflict,
resistance, contracts
Hand the baton to others gracefully
Hating what happens
to your baby
Beforehand
Empathy
Scale
Hidden “must haves”,
feature creep
Tool: scoring problem interviews.
How to score problem interviews
• Did the interviewee successfully rank the problems you presented?
• Are they actively trying to solve the problems (or have they done so in the
past)?
• Were they engaged and focused throughout the interview?
• Did they agree to a follow-up meeting or interview (to present your solution)?
• Did they offer to refer others to you for interviews?
• Did they offer to pay you immediately for the solution?
Did the interviewee successfully rank the
problems you presented?
YES
Sort of
NO
10 points
5 points
0 points
• If the interviewees ranked the problems with strong interest (irrespective of the
ranking) that’s a good sign. Score 10 points.
• If they couldn’t decide which problem was really painful, but they were still
really interested in the problems, that’s OK but you’d rather see more definitive
clarity. Score 5 points.
• If they struggled with this, or they spent more time talking about other
problems they have, that’s a bad sign. Score 0 points.
Are they actively trying to solve the problems (or
have they done so in the past)?
YES
Sort of
NO
10 points
5 points
0 points
• If they’re trying to solve the problem with Excel and fax machines, you may
have just hit on the Holy Grail. Score 10 points.
• If they spend a bit of time fixing the problem, but just consider it the price of
doing their job, they’re not trying to fix it. Score 5 points.
• If they don’t really spend time tackling the problem, and are okay with the
status quo, it’s not a big problem. Score 0 points.
Were they engaged and focused throughout the
interview?
YES
Sort of
NO
8 points
4 points
0 points
• If they were hanging on your every word, finishing your sentences, and
ignoring their smartphone, score 8 points.
• If they were interested, but showed distraction or didn’t contribute comments
unless you actively solicited them, score 4 points.
• If they tuned out, looked at their phone, cut the meeting short, or generally
seemed entirely detached—like they were doing you a favor by meeting with
you—score 0 points.
Did they agree to a follow-up meeting or interview
(to present your solution)?
Yes, without being asked
Yes, when you asked them to
No
4 points
2 points
0 points
• If they are demanding the solution “yesterday”, score 4 points.
• If they’re OK with scheduling another meeting, but suddenly their calendar is
booked for the next month or so, score 2 points.
• If you both realize there’s no point showing them anything in terms of a
solution, score 0 points.
Did they offer to refer others to you for
interviews?
Yes, without being asked
Yes, when you asked them to
No
4 points
2 points
0 points
• If they actively suggested people you should talk to without being asked, score
4 points.
• If they suggested others at the end, in response to your question, score 2
points.
• If they couldn’t recommend people you should speak with, score 0 points (and
ask yourself some hard questions about whether you can reach the market at
scale.)
Did they offer to pay you immediately for the
solution?
Yes, without being asked
Yes, when you asked them to
No
3 points
1 point
0 points
• If they offered to pay you for the product without being asked, and named a
price, score 3 points.
• If they offered to pay you for the product, score 1 point.
• If they didn’t offer to buy and use it, score 0 points.
How to score problem interviews
• A score of 31 or higher is a good score. Anything under is not.
• Try scoring all the interviews, and see how many have a good score.
• This is a decent indication of whether you’re onto something or not with the
problems you want to solve.
• Then ask yourself what makes the good score interviews different from the
bad score ones.
• Maybe you’ve identified a market segment; maybe you have better results
when you dress well; maybe you shouldn’t do interviews in a coffee
shop. Everything is an experiment you can learn from.
How to score problem interviews
• You can also sum up the rankings for the problems that you presented. If you
presented three problems, which one had the most first place rankings? That’s
where you’ll want to dig in further and start proposing solutions (during
Solution Interviews.)
• The best-case scenario is very high interview scores within a subsection of
interviewees where those interviewees all had the same (or very similar)
rankings of the problems. That should give you more confidence that you’ve
found the right problem and the right market.
Tool: scoring problem interviews.
Metrics in practice:
The Lean Analytics Cycle
Success!
Pick OMTM
Pivot or
give up
Draw a new line
Try again
Did we move
the needle?
Measure
the results
Design a test
Draw a line
in the sand
Find a
potential
improvement
Without
data: make
a good
guess
With data:
find a
commonality
Hypothesis
Make changes
in production
Tool: scoring problem interviews.
Three threes
Three
assumptions
What big bets are you making?
•“People will answer questions”
•“Organizers are frustrated with how to run conferences”
•“We'll make money from parents”
•“Amazon is reliable enough for our users.”
Three actions
to take
What are you doing to make these assumptions happen
(or identify they’re wrong and change course?)
•Product enhancements
•Marketing strategies
Three experiments
to run
• Feature tests
• Continuous deployment
• A/B testing
• Customer survey
Three threes
Three
assumptions
Monthly
Board, investors,
founders
Strategy
Three actions
to take
Weekly
Executive team
Tactics
Three
experiments
to run
Daily
Employees
Execution
Three threes
Three
assumptions
Three actions
to take
Three
experiments
to run
Many people will
answer questions
Get more
people
Change
the UI
Increase
answer %
Test
timings
Test better
questions
Questions
from peers
Tool: the problem/solution canvas