New Ways to Understand the
Impact of Stock-outs
(Julie Holland Mortimer and Christopher T. Conlon)
Presented by:
Julie Holland Mortimer
Associate Professor
Economics Department
Harvard University
Introduction:
Economists have devoted a lot of energy to
understanding demand in different industries.
We can tell you all kinds of things about aggregate
demand, especially for manufacturing.
For example: we were very good at predicting which
cars people would switch to when GM discontinued
the Oldsmobile brand, using only data on national
market shares.
But believe it or not, we almost never pay attention to
availability.
Why has availability been ignored?
• Previous work focused on manufacturing, not retail or
distribution.
• Data on availability was pretty scarce until relatively recently.
• Many people thought of availability as a small "friction" but
not a main focus of firms.
When does "full availability" fail?
•
•
•
•
Stock-outs
Capacity constraints
Search costs
Awareness
What goes wrong by ignoring availability?
An example:
• We stock 5 units of product A and 10 units of product B each week.
• Product A sells out immediately.
• Consumers that arrive looking for product A choose B instead, and B sells
10 units by the end of the week.
• When we look at aggregate sales we get the story wrong. B appears to be
more popular, but the product that truly sells is A.
So sales data don't reflect true demand patterns.
This gives two kinds of errors:
• Censoring: We only observe demand up to capacity. This understates
demand for the stocked-out product.
• Forced Substitution: We observe sales for products that represent
consumers' second choices as they substitute away from missing goods.
This overstates demand for remaining products.
Bad demand estimates lead to bad
predictions about profitability.
• Some consumers may walk away when a product is
stocked-out.
• Many others may simply purchase a different
product.
• Effect on profitability depends on this behavior, and
on the margins of the different products.
• The effects might be different in the short-run than in
the long-run.
Research goals: Incorporate availability in demand estimation, and
estimate its impact for firms.
•
•
•
•
Test the impact of stock-outs in the field.
Quantify the impact of stock-outs on profitability.
Develop a method to handle stock-outs, even with incomplete data reporting.
Quantify the size of the error from ignoring the problem.
Future goal: How can firms use wireless data to optimize restocking
visits? (We need to understand the impact of stock-outs first.)
Why vending? (I started in video rentals....)
•
•
•
Great data for identifying stock-out events.
Some aspects of supply are relatively straightforward (e.g., pricing).
Feasible laboratory for field experiments.
Other settings: retail, perishable and seasonal goods,
sporting/entertainment events, airlines, etc.
Outline:
•
•
•
•
•
•
Describe field experiments on availability
Results from field experiments
Impact of stock outs on profitability in experiments
Preliminary model of consumer decisions
Results from the model
Comparison of the model results to the field
experiments
• Implications for vending operators and lessons for
economists
Description of
Field Experiments
Field experiments implemented by Mark Vend
Company in Chicago, Illinois.
• A total of 62 machines in office buildings in downtown
Chicago
• Spread across 5 sites/locations
• “White collar” customer base
• Fairly stable demand patterns over time at these sites/machines
Experimental design:
• 8 experiments were run (2 were repeated for accuracy)
• 6 experiments stocked out a single product
• 2 experiments stocked out two products simultaneously
The 8 experiments:
1.
2.
3.
4.
5.
6.
7.
8.
Snickers
Zoo Animal Crackers
Dorito Nacho
Cheetos
Chocolate Chip Famous Amos
M&M Peanut
Dorito Nacho and Cheetos
Snickers and M&M Peanut
More on the experimental design:
• For each run, we removed the focal product(s) for about 2.5
weeks
• Each machine is visited about 3 times during the experiment
• Data were collected from January, 2006 – February, 2009
• Experimental dates range from June 2007 to September 2008
• Experiments were run during the months of May – October
(one in February, 2008)
Data detail:
• DEX data collected at each service visit for each
product/machine
• Few stock-outs occur outside of the experiments
Small products are consolidated into “generic”
category products for reporting. Example:
3-Musketeers (Reg., Dk Chc Mint)
Heavenly Dark (Plain, Almond)
Reeses (PB Cup, Pieces)
Hershey (Almond, Take Five)
Raisinets
Butterfinger (Reg., Crisp)
Heath Bar
M&M (PB, Crunchy M-Azing)
Cadbury (Milk, Caramello)
Milky Way (Reg., Midnight)
Nestle Crunch
Rolo
Peanut Butter Twix
“Generic
Candy
Chocolate”
Finding a baseline for comparison:
• We compare each visit during the experimental periods to several “control
visits.”
• The control visits come from the same set of machines at other times.
• We “match” each experimental visit to four control visits of similar length
(adjusting for weekends).
• In order to find a good “match” we use control visits with similar rates of
sales for “non-substitutes.”
Example of how we make a match:
• The focal product is Snickers, so all Snickers bars are removed during the
experiment.
• To make a “match” we look at the rate of sales of salty snack products.
• We find visits during the control period in which sales of Doritos, Ruffles,
and Cheetos are the same as the sales of Doritos, Ruffles, and Cheetos
during the experimental period.
Calculating the effect of a stockout:
• A stockout changes the rate of sales of substitute products.
• Compare rates during the experimental periods to rates during
control periods.
There is one thing we can’t observe directly (at least
without a video camera and approval from the
Government): consumers “walking away”
• But we can estimate this from the change in rate of total vends.
• Economists call this the “outside good.”
An example of the data, from Experiment 1:
Product
Category
Generic Candy Chocolate
M&M Milk Chocolate
M&M Peanut
Snickers
Twix Caramel
Candy Chocolate Total
Generic Candy Non Chocolate
Candy Non Chocolate Total
Choc Chip Famous Amos
Choc SandFamous Amos
Generic Cookie
Grandmas Choc Chip
Rasbry Knotts
Zoo Animal Cracker Austin
Cookie Total
Generic Other
Kar Sweet & Salty Mix
Other Total
Generic Pastry
Strwbry Pop-Tarts
Pastry Total
Cheeto LSS
Dorito Nacho LSS
Frito LSS
Generic Salty Snack
Hot Stuff Jays
Original LSS Ruffles
Smartfood LSS
Sun Chip LSS
Salty Snack Total
Total
Candy Chocolate
Candy Chocolate
Candy Chocolate
Candy Chocolate
Candy Chocolate
Candy Chocolate
Candy Non Chocolate
Candy Non Chocolate
Cookie
Cookie
Cookie
Cookie
Cookie
Cookie
Cookie
Other
Other
Other
Pastry
Pastry
Pastry
Salty Snack
Salty Snack
Salty Snack
Salty Snack
Salty Snack
Salty Snack
Salty Snack
Salty Snack
Salty Snack
Experimental Period
Control Period
Visits
Sales
S.D.
Visits
Sales
S.D. ExpRate
Rate
Cntl
223
1.20
1.45
954
0.97 1.17
0.23
107
0.27
0.87
579
0.26 0.63
0.01
268
1.44
1.17
1076
0.92 0.84
0.52
270
0.00
0.00
1069
0.83 0.77
-0.83
218
0.75
0.78
866
0.50 0.69
0.24
270
3.66
2.61
1080
3.48 2.57
0.18
244
0.48
0.84
1009
0.63 0.97
-0.14
244
0.48
0.84
1009
0.63 0.97
-0.14
270
0.53
0.60
1070
0.45 0.56
0.08
91
0.12
0.55
452
0.09 0.39
0.02
253
0.48
0.65
1050
0.45 0.63
0.02
208
0.23
0.37
803
0.22 0.43
0.00
204
0.15
0.36
842
0.15 0.33
0.00
270
0.79
0.94
1078
0.52 0.62
0.28
270
2.29
1.57
1080
1.89 1.56
0.40
270
1.35
1.31
1080
1.45 1.38
-0.10
170
0.33
0.63
674
0.26 0.56
0.06
270
1.68
1.54
1080
1.71 1.58
-0.04
68
0.06
0.43
471
0.10 0.58
-0.04
244
0.36
0.48
960
0.36 0.59
0.00
257
0.42
0.52
1013
0.46 0.68
-0.04
264
0.73
0.81
1059
0.72 0.66
0.01
270
0.54
0.58
1079
0.55 0.52
-0.01
217
0.41
0.68
806
0.39 0.61
0.02
270
4.37
3.42
1080
4.19 2.68
0.19
235
0.22
0.38
879
0.21 0.38
0.01
244
0.61
0.72
933
0.57 0.66
0.03
131
0.19
0.48
655
0.19 0.43
0.01
247
0.49
0.53
967
0.44 0.49
0.06
270
7.57
5.17
1080
7.25 4.21
0.31
270 16.09
9.40
1080 15.41 9.03
0.68
Sig. (1%)
**
**
**
**
**
**
Identifying the best substitutes:
• There is always noise in sales rates, so we look at products
with a statistically significant increase in sales.
• Call these products “substitutes.”
Identifying “walkers”:
• The experiments show no change in total vends, implying no
“walkers”.
• We calculate the impact of the stock-out under two scenarios:
– Assume there really are no walkers (May be OK in the short-run).
– Assume that some percentage of people walk away when their favorite
product is stocked-out.
– Estimate the percentage that walk away for each product from a model
of consumer choice, which examines how total vends fall when a
product is not carried in a machine.
Modeling “walkers”:
•
•
•
•
Use the maximum rate of sales at any machine for a visit.
This allows for slower sales rates at Christmas, for example.
Assumes that no machine beats your busiest machine.
Under this assumption, model consumer choice during control periods to
see how total sales respond when various products are not stocked. (This
includes responses from machines that have different facings, in addition to
stock-out events.)
Results (for these machines/locations):
• Consumers of salty snacks rarely walk away (1%).
• Consumers of cookies walk away more often (11-12%).
• Consumers of chocolate bars walk away most often (20%).
Alternatives:
• See how your own machines respond and use that to deflate.
• More data collection (use pressure pads or video cameras).
• Best method may vary based on machine location (public, office, school).
Where do Snickers consumers go,
under the two scenarios?
What is the impact on the sales of
Snickers substitutes?
The impact on profitability depends on:
• The number of “walkers”
• The margins on Snickers and all its substitutes
For the two scenarios, we have the following impact:
• Scenario 1 (no walkers):
– We lose 827 sales of Snickers @ $0.24 margin (-$202.43).
– We gain 827 sales of substitutes @ $0.30 avg. margin ($242.57).
– The net effect is $40.14.
• Scenario 2 (20% walkers):
– We lose 827 sales of Snickers @ $0.24 margin (-$202.43).
– We gain 664 sales of substitutes @ $0.30 avg. margin ($194.67).
– The net effect is $-7.76.
In this case:
• The higher margin on substitutes (esp. cookie
products) means that the stock-out may actually be
profitable (depending on “walkers”).
• At least, in the short run.
• Longer-run effects, if clients get upset over time,
aren’t captured here.
Results from All Experiments,
No Walkers
Effects on Sales of Substitutes,
No Walkers
Effects on Profitability
Snickers:
For the two scenarios, we have the following impact:
• Scenario 1 (no walkers):
– We lose 827 sales of Snickers @ $0.24 margin (-$202.43).
– We gain 827 sales of substitutes @ $0.30 avg. margin ($242.57).
– The net effect is $40.14.
• Scenario 2 (20% walkers):
– We lose 827 sales of Snickers @ $0.24 margin (-$202.43).
– We gain 664 sales of substitutes @ $0.30 avg. margin ($194.67).
– The net effect is $-7.76.
Animal Crackers:
For the two scenarios, we have the following impact:
• Scenario 1 (no walkers):
– We lose 383 sales of Animal Crackers @ $0.43 margin (-$166.79).
– We gain 383 sales of substitutes @ $0.34 avg. margin ($129.36).
– The net effect is -$37.43.
• Scenario 2 (11.5% walkers):
– We lose 383 sales of Animal Crackers @ $0.43 margin (-$166.79).
– We gain 339 sales of substitutes @ $0.34 avg. margin ($114.46).
– The net effect is -$52.33.
Dorito Nachos:
For the two scenarios, we have the following impact:
• Scenario 1 (no walkers):
– We lose 451 sales of Doritos @ $0.44 margin (-$197.02).
– We gain 451 sales of substitutes @ $0.40 avg. margin ($179.05).
– The net effect is $-17.97.
• Scenario 2 (1% walkers):
– We lose 451 sales of Doritos @ $0.44 margin (-$197.02).
– We gain 447 sales of substitutes @ $0.40 avg. margin ($177.53).
– The net effect is $-19.50.
Cheetos:
For the two scenarios, we have the following impact:
• Scenario 1 (no walkers):
– We lose 568 sales of Cheetos @ $0.44 margin (-$252.30).
– We gain 568 sales of substitutes @ $0.43 avg. margin ($244.29).
– The net effect is $-8.01.
• Scenario 2 (1% walkers):
– We lose 568 sales of Cheetos @ $0.44 margin (-$252.30).
– We gain 563 sales of substitutes @ $0.43 avg. margin ($242.20).
– The net effect is $-10.11.
Chocolate Chip Famous Amos:
For the two scenarios, we have the following impact:
• Scenario 1 (no walkers):
– We lose 363 sales of Famous Amos @ $0.47 margin (-$170.84).
– We gain 363 sales of substitutes @ $0.46 avg. margin ($167.07).
– The net effect is $-3.76.
• Scenario 2 (11% walkers):
– We lose 363 sales of Famous Amos @ $0.47 margin (-$170.84).
– We gain 322 sales of substitutes @ $0.46 avg. margin ($148.07).
– The net effect is $-22.77.
M&M Peanut:
For the two scenarios, we have the following impact:
• Scenario 1 (no walkers):
– We lose 517 sales of M&M Peanut @ $0.24 margin (-$124.52).
– We gain 517 sales of substitutes @ $0.34 avg. margin ($174.11).
– The net effect is $49.59.
• Scenario 2 (20% walkers):
– We lose 517 sales of M&M Peanut @ $0.24 margin (-$124.52).
– We gain 414 sales of substitutes @ $0.34 avg. margin ($139.61).
– The net effect is $15.10.
Dorito Nachos and Cheetos:
For the two scenarios, we have the following impact:
• Scenario 1 (no walkers):
– We lose 1019 sales of Doritos and Cheetos @ $0.44 margin (-$444.48).
– We gain 1019 sales of substitutes @ $0.44 avg. margin ($444.48).
– The net effect is $0.
• Scenario 2 (1% walkers):
– We lose 1019 sales of Doritos and Cheetos @ $0.24 margin (-$444.48).
– We gain 1010 sales of substitutes @ $0.44 avg. margin ($440.49).
– The net effect is $-3.99.
M&M Peanut and Snickers:
For the two scenarios, we have the following impact:
• Scenario 1 (no walkers):
– We lose 1344 sales of both products @ $0.24 margin (-$322.56).
– We gain 1344 sales of substitutes @ $0.29 avg. margin ($388.80).
– The net effect is $66.24
• Scenario 2 (21% walkers):
– We lose 1344 sales of both products @ $0.24 margin (-$322.56).
– We gain 1057 sales of substitutes @ $0.29 avg. margin ($305.89).
– The net effect is $-16.67.
Model of Consumer Decisions
Model of Consumer Decisions
• We generally don’t get the chance to run experiments,
so we do our best with “observational” data.
• We think of a consumer buying 1 unit.
• She chooses the product that makes her the happiest,
given her choice set.
• Each product has an “average quality,” and people
have “idiosyncratic” tastes for different products.
• There are a few common models; a simple one
assumes that a consumer’s tastes for products are
similar within a category.
Consumer Decision Tree
No
Purchase
Pastry
Chocolate
Purchase
Salty Snack
Strawberry
Pop-Tart
Snickers
Dorito
Nacho LSS
Generic Pastry
M&M Peanut
Candy
Cookie
Other
Choc Chip
Famous Amos
Kar Sweet
& Salty Mix
Cheeto LSS
Zoo Animal
Cracker Austin
Generic
Other
Twix Caramel
Sun Chip LSS
Grandmas
Choc Chip
M&M Milk
Chocolate
Original
LSS Ruffles
Raspberry
Knotts
Generic Candy
Chocolate
Hot Stuff
Jays
Choc Sand
Famous Amos
Frito LSS
Generic
Cookie
Smartfood
LSS
Generic
Salty Snack
Generic Candy
Non Chocolate
How does the model interpret sales data?
• Products with more sales overall get higher “average quality”
(more people choose these products).
• When different products are stocked at different machines or
time periods, we observe that people’s choices are different.
Stock-out events and price changes give similar information.
• The difference in these choices tells us about consumers’ tastes
for different products.
• We need to assume that consumer populations are similar
across machines/time (both in tastes, and rates of arrival).
• The model predicts “walkers” from choices of the “outside
good.” (When a product isn’t stocked, total sales are lower.)
Differences in the Baseline Data
• In these results, we show substitution for different products
from a single version of the model.
• This version uses the entire set of potential “match” visits
(January 2006 – February 2009).
• Thus, the results don’t account for differences in the baseline
data across experiments due to matching.
• Example: suppose one experiment is run in August, when
many people are on vacation, and vend rates are lower. The
experimental results will compare outcomes to other “low”
periods, but the baseline model will not.
• Next step: estimate the model on the same “matched” data for
direct comparison to the experiments.
Results from the Model,
(Only “Inside Goods”)
Comparison of Experiments and the Model
• Compare results from the experiments and the model
• Assume that “walkers” are as predicted in the model
for both experimental results and model predictions
• Examine which products people move to, and the
effects on substitutes
• Current results not yet “apples to apples” because of
differences in baseline data
Comparison of Which
Products People Move To
Experiment / Model
Experiment / Model
Experiment / Model
Experiment / Model
Experiment / Model
Experiment / Model
Experiment / Model
Experiment / Model
Comparison of the
Effects on Substitutes
Experiment / Model
Experiment / Model
Experiment / Model
Experiment / Model
Experiment / Model
Experiment / Model
Experiment / Model
Experiment / Model
Implications for Vending Operators
and Lessons for Economists
What is the Effect of a Stock Out?
• In the experiments, most people buy something else.
• The model predicts some “walkers”. This may be
more likely in the long-run.
• The impact on profits depends on how many people
walk away, and on the margins of the focal products
and its substitutes. A stock-out may actually be
profitable in some cases.
• Of course, consumers are a little less happy, and this
may be an issue in the long-run.
What is NOT the Effect of a Stock Out?
• When a product stocks out, the effect on profitability
is not the same as losing that product’s rate of sales.
• Most consumers buy something else, and firms make
a margin on those sales.
• The nature of this substitution likely depends on:
which other products are stocked in the machine, the
type/location of the machine (office, school, public
area, etc.), and even the time of day, or day of week.
Implications for Operational Decisions
• Servicing decisions should depend on the impact of
stock outs on profitability.
• Restocking machines frequently is costly, so if the
impact of stock outs on revenues is not large, it might
make sense to restock less frequently.
• (Again: short run vs. long run implications!)
• Capacity decisions affect stock out probabilities.
• Acquiring more information about your own
machines depends on data collection. (Wireless?)
What might be the role of wireless data?
• From an information point of view, wireless data are
helpful for understanding the impact of stock outs
across different machines, locations, or time periods.
• Wireless data can help avoid over-servicing machines
that are still relatively full.
• They may also help streamline “binning” procedures
in the warehouse so that drivers are more efficient.
• Overall impact depends on the cost of the service, and
the nature of demand at different machines.
Lessons for Economists:
Do Stock Outs Matter for Demand Estimates?
• In related work, we’ve found that ignoring stock-outs
can seriously impact demand estimates.
• We find: products that frequently stock out are much
better sellers than our “standard models” predict.
• Allowing for stock-out events corrects this problem.
The “sell-out” products are estimated to be more
popular, and the “second choice” products less so.
More Lessons for Economists:
The Effect of Stock Outs on Other Calculations
• When demand estimates change, we estimate
different effects of many other policies.
• Examples: the value of new products, the impact of
mergers, and firms’ choices of optimal capacity.
• Also, the impact of stock outs for firms and
consumers.
• This is a broad phenomenon. It’s important in
vending, but also in retail, sports events, concert
performances, airline pricing, etc.