A Parsimonious Model of Stock-keeping-Unit
(SKU) Choice
Teck H. Ho
Haas School of Business
UC, Berkeley
Joint work with Juin-Kuan Chong, NUS
The Goal
Search for best-fitting model in SKU choice
January 2004
Teck H. Ho
Berkeley
1
Purchase History of Panelist 14110874
Purchase
SKU
Occasion
Bought
Brand
COKE
PEPSI
ROYAL
Size
TAB
CROWN
PRIVATE
9.00
7.50
4.50
3.75
3.00
2.25
1.06
LABEL
1
73
0
1*
0
0
0
0
0
1*
0
0
0
0
2
73
0
2*
0
0
0
0
0
2*
0
0
0
0
3
73
0
3*
0
0
0
0
0
3*
0
0
0
0
4
69
1*
3
0
0
0
0
0
4*
0
0
0
0
5
102
2*
3
0
0
0
0
0
4
0
1*
0
0
6
99
2
4*
0
0
0
0
0
4
0
2*
0
0
7
73
2
5*
0
0
0
0
0
5*
0
2
0
0
8
73
2
6*
0
0
0
0
0
6*
0
2
0
0
9
99
3*
6
0
0
0
0
0
6
0
3*
0
0
10
99
4*
6
0
0
0
0
0
6
0
4*
0
0
11
14
4
7*
0
0
0
1*
0
6
0
4
0
0
12
73
4
8*
0
0
0
1
0
7*
0
4
0
0
13
3
5*
8
0
0
0
2*
0
7
0
4
0
0
14
73
5
9*
0
0
0
2
0
8*
0
4
0
0
15
61
6
9
0
0
0
3*
0
8
0
4
0
0
CLASSIC
DIET
CAFFEINE
CAFFEINE FR
CRISP
FREE
CLASSIC
Purchase
SKU
Occasion
Bought
Flavor
1
73
0
0
1*
0
0
2
73
0
0
2*
0
0
3
73
0
0
3*
0
0
4
69
0
0
3
1*
0
5
102
0
0
3
2*
0
6
99
0
0
4*
2
0
7
73
0
0
5*
2
0
8
73
0
0
6*
2
0
9
99
0
0
6
3*
0
10
99
0
0
6
4*
0
11
14
0
1*
6
4
0
12
73
0
1
7*
4
0
13
3
1*
1
7
4
0
14
73
1
1
8*
4
0
15
61
1
1
8
5*
January 2004
0
Teck H. Ho
Berkeley
2
The consumer choice setting

Household i chooses a product (or stock-keeping-unit
(SKU)) from a choice menu on a series of purchase
incidences indexed by t

Before each purchase incidence, each SKU j is
characterized by a set of marketing-mix activities:
price (P), display (D), and feature advertisement (AD)

The modeler also observes:
 Household i’s SKU choices on purchase incidences 1, 2, …, t-1
January 2004
Teck H. Ho
Berkeley
3
Research question

To develop a good descriptive model of SKU choice to
predict the probability of household i choosing SKU j on
purchase incidence t
Pij (t )
January 2004
Teck H. Ho
Berkeley
4
Criteria of a “good” model - Specification
Simple (i.e, small number of parameters)
Model complexity does not increase with number of
items/feature levels in the choice menu
Increasing number of items
Satisfies plausible principles of human behavior
 Incorporate psychological findings into model
Fits and predicts choice behaviors well (e.g., Guadagni
and Little, 1983; Fader and Hardie, 1996)
January 2004
Teck H. Ho
Berkeley
5
Complex menus &
increasing number of items
Category
Bacon
Number of
SKUs
Year 1 Year 2 New
Number of
Brands
Year 1 Year 2 New
Number of
Sizes
Year 1 Year 2 New
Number of
Flavors
Year 1 Year 2 New
49
56
13
21
26
5
7
7
0
11
10
1
Cola
118
134
23
16
15
1
14
13
2
6
8
2
Eggs
35
35
3
10
10
2
3
3
0
5
5
0
Frozen Pizza
298
290
39
32
40
8
138
135
7
63
65
8
Hotdogs
112
121
16
32
37
6
9
11
2
15
15
0
IceCream
376
384
45
35
37
2
8
9
1
131
135
14
Potato Chips
207
235
78
24
26
5
30
32
4
28
29
4
Regular Cereal
197
234
45
29
34
6
63
70
10
43
44
2
Spaghetti Sauce
166
174
28
37
37
4
26
30
4
29
28
2
Yogurt
241
254
47
14
12
1
7
7
0
64
67
10
87
87
19
21
19
0
9
11
2
8
8
0
250
264
71
38
40
3
48
54
14
17
20
4
84
82
24
25
23
2
7
9
2
14
17
3
Soap
207
206
36
46
44
1
39
41
3
16
17
2
Toothpaste
219
237
40
22
24
2
40
40
4
44
50
7
176
186
35
27
28
3
30
31
4
33
35
4
Bathroom Tissue
Detergent
Paper Towel
AVERAGE
January 2004
Teck H. Ho
Berkeley
6
Criteria of a “good” model – Estimation

Does not aggregate choice (i.e., at the SKU level)
 Heterogeneity across products (biased estimates); avoid
“average” variables; inventory planning

Does not throw away observations
 Choice-based sampling (biased estimates) (Ben-Akiva and
Lerman, 1985)
January 2004
Teck H. Ho
Berkeley
7
Criteria “violations”

Model specification
 Model complexity
 Many models have complexity increases with number of items
 Plausible principles of behavior
 Few attempts to incorporate findings from psychological research in consumer
behavior

Model estimation
 Aggregate choice
 Violation examples: Brand-size combination; “other” product
 Throwing away observations
 Violation examples: Top n SKUs; ignore SKUs that have few purchases
January 2004
Teck H. Ho
Berkeley
8
Notations
Examples





Household i
SKU j
Purchase Occasion t
Attribute k
Attribute level l
Panelist 14110874
UPC 11200000847
July 17, 97
Brand
COKE
For estimation, every product category is assumed to have three
attributes (Brand, Size, Flavor)
January 2004
Teck H. Ho
Berkeley
9
Utility specification
U ij (t )  Vij (t )  M ij (t )   ij (t )




Utility = intrinsic value + value associated with
marketing-mix activities
Error structure captures serial correlations in
attribute-level and product-specific utilities
Uses latent class to capture heterogeneity
No product or attribute-level specific intercept terms!
January 2004
Teck H. Ho
Berkeley
10
Intrinsic Value
K
Lk
Vij (t )   Aikl (t )  I jkl  Aij (t )
k 1 l 1


Intrinsic value consists of both product-specific and attribute-level
experiences
Example: SKU 14 = {PEPSI, 9.0, DIET}, Panelist = Grace
VGrace,14 (t )  AGrace, PEPSI (t )  AGrace,9.0 (t )  AGrace, DIET (t )  AGrace,14(t )
January 2004
Teck H. Ho
Berkeley
11
Marketing-mix response
M j (t )   P Pj (t )   jD D j (t )   jAD ADij (t )



Control for price, display, and feature advertisement
on local newspapers
Display and feature advertisement are dummy
variables
Actual price paid (incorporating coupons and etc.)
January 2004
Teck H. Ho
Berkeley
12
An overview of the model
Productspecific
Experience
Previous
Cumulative
Reinforcement
Incremental
Reinforcement
Intrinsic Value
Size
Utility
Attributelevel
Experience
Marketing-mix
Response
Consumption
Shopping
Previous
Cumulative
Reinforcement
Consumption
Brand
Flavor
Incremental
Reinforcement
Shopping
•Intrinsic value consists of both product-specific and attribute-level experiences
• Consumption and shopping experiences depend on product and attribute-level familiarity
January 2004
Teck H. Ho
Berkeley
13
Cumulative attribute-level reinforcement, Aikl(t)
Aikl (t )  k  Aikl (t  1)  Rikl (t )
Cikl (t  1)  Sikl (t ), if (k,l) was chosen in t - 1
Rikl (t )  
Sikl (t ), otherwise


Cumulative attribute-level reinforcement = Decayed previous
reinforcement + immediate incremental reinforcement
Incremental reinforcement consists of consumption as well as
“shopping” experience for chosen level and “shopping”
experience only for unchosen levels
January 2004
Teck H. Ho
Berkeley
14
An overview of the model
Productspecific
Experience
Previous
Cumulative
Reinforcement
Incremental
Reinforcement
Intrinsic Value
Size
Utility
Attributelevel
Experience
Marketing-mix
Response
Consumption
Brand
Aikl (t )
Flavor
Shopping
Previous
Cumulative
Reinforcement
Aikl (t  1)
Consumption
Cikl (t  1)
Incremental
Reinforcement
Rikl (t )
Shopping
Sikl (t )
January 2004
Teck H. Ho
Berkeley
15
Consumption (Cikl(t-1)) &
shopping (Sikl(t)) experiences
Cikl (t  1) Ck 0  Ck 1  Fikl (t  2)
S ikl (t )  S k 0  S k 1  Fikl (t  1)


Consumption & shopping experiences depend on consumer’s
familiarity with the level
“Shopping” experience because people care about foregone
utilities from actions/products that they could have chosen
(Camerer and Ho, 1999)

Ck1 < 0 captures “law of diminishing marginal utility”

Sk1 > 0 captures “memory-based decision making”
(Alba, Hucthinson, and Lynch, 1991)
January 2004
Teck H. Ho
Berkeley
16
Variety-seeking behavior
Cikl (t ) Ck 0  Ck 1  Fikl (t  1)
Sikl (t )  S k 0  S k 1  Fikl (t )

Modeled as negative reinforcement (e.g., Lattin, 1987)

Under our model setup, it is driven by Ck1 < 0 (satiation) (Erdem,
1992; McAlister, 1982) or Sk1 > 0 (“grass is greener” effect) (Kahn, 1998)
January 2004
Teck H. Ho
Berkeley
17
Product and attribute-level familiarities
Fikl (t )  ln( 1   a  Tikl (t ))
Fij (t )  ln ( 1  θ p  Tij(t))



Product and attribute-level familiarity is concave in
number of times the product and attribute levels are
consumed (Tikl(t) & Tij(t))
Also tried linear and step functions
Log function fits best and is also the most appealing
conceptually
January 2004
Teck H. Ho
Berkeley
18
Main ideas





Utility consists of intrinsic value and value associated
with marketing-mix response
Intrinsic value has two components: product-specific
and attribute level experiences
Incremental reinforcement has both consumption and
shopping experience, which depends on product and
attribute-level familiarity
Each unchosen attribute level receives a different
“shopping” reinforcement
The model has 5  6  ( K  1) parameters for a Kattribute product category
 Example: K=3 (brand, size, flavor), the model has 29 parameters
January 2004
Teck H. Ho
Berkeley
19
Data Set




Panel-level market basket data set
124,000 product purchases across 15 product
categories (10 food + 5 non-food)
Purchases made by 513 households at 5 stores located
within the same neighborhood over a 2-year period
+ Data from Fader and Hardie (1996)
January 2004
Teck H. Ho
Berkeley
20
Data Set
Category Summary
Total Sample Size
Number of Households
Number of SKUs
Total Number of Levels
in All Salient Attributes
1
Number of Parameters
Our Model
Fader and Hardie
Guadagni and Little
Flavor/Ingredient
Total Number
Example
January 2004
Large Product Categories
Egg
9903
9781
14590
14705
3698
12218
4111
7022
12594
4226
5214
2993
7171
12978
482
38
594
59
528
106
429
141
314
62
495
108
334
128
382
285
356
288
320
194
384
243
306
259
471
321
480
242
20
22
40
41
45
53
64
95
96
102
107
119
124
153
59
75
91
73
79
133
59
115
227
59
117
297
59
125
139
59
141
231
59
163
271
59
225
585
59
227
591
59
239
403
59
249
501
59
273
533
59
283
657
59
341
499
10
Downy
Snuggle
Bounce
21
Scottissue
Northern
Charmin
17
Coca Cola
Pepsi
Ryl Crown
26
Oscar Mayer
W.CornKing
Lazy Maple
29
Jays
Lays
Ruffles
15
Dannon
Yoplait
Kemps
41
Ragu
Prego
Hunts
47
Dial
Dove
Ivory
24
Crest
Colgate
Arm&Hammer
41
Tide
Wisk
All
35
Kellogg
Gnrl Mills
Post
4
Small
Medium
Large
11
4 rl.
1 rl.
12 rl.
16
67.6 oz.
288 oz.
144 oz.
7
16 oz.
12 oz.
24 oz.
34
6.5 oz.
7 oz.
6 oz.
7
6 oz.
8 oz.
32 oz.
30
30 oz.
26 oz.
14 oz.
42
15 oz.
14 oz.
9.5 oz.
44
6.4 oz.
4.6 oz.
6 oz.
62
64 oz.
128 oz.
42 oz.
73
12 oz.
18 oz.
15 oz.
Salient Attribute Description
Brand
Total Number
12
Example
Crystal Fm.
Prv. Label
W.R.Valley
Package Size
Total Number
Example
Small Product Categories
Bathroom
Tissue
Cola
Bacon
Fabric
Softener
3
12 ct.
18 ct.
6 ct.
5
Large
Extra Large
Jumbo
Paper
Towel
Hotdog
Potato
Chip
27
38
Bounty
Oscar Mayer
Scottowels
Hygrade
Versatile
W.CornKing
9
1 rl.
3 rl.
2 rl.
11
16 oz.
12 oz.
40 oz.
Yogurt
Spaghetti
Sauce
Soap
Toothpaste
Detergent
Regular
Cereal
4
8
8
12
17
15
32
74
31
18
Regular
Unscented
Regular
Regular
White Paper
Beef
Regular
Plain
Plain
Regular
Staingard
Regular
Diet
Smoked
Print
Chckn&Pork
BBQ
Strawberry Itln. Garden Original
Light
Soft Scented Caffn. Free Hkry Smoked As.Colors Pork&Turkey SC & Onion Raspberry Tmt. & Herb Unscented
Teck H. Ho
51
Tartar Ctrl
Bk. Soda
Regular
21
45
Reg. Liquid
Corn
Con. Pwd Wheat Bran
Reg. Pwd
Rice
Berkeley
21
Estimation
Max LL       I ij (t )  Pij (t )
k ; Ck 0 , Ck1 ; S k1 ;  p ; C p 0 , C p1 ; S p1
i
j
t
 A ,  p ;  X , X  P, D, AD



Maximize the likelihood of observing the data
The first 13 weeks of data for initialization; the next
65 weeks for calibration and the last 26 weeks for
model validation
Benchmark against Fader and Hardie (1996)’s model
January 2004
Teck H. Ho
Berkeley
22
FH Model
K
Lk
Vij (t )  [vkl  Aikl (t )]  I jkl
k 1 l 1


Has attribute-level specific terms
Does not capture familiarity-based consumption as
well as shopping experience
S k 1  Ck 1  0
January 2004
Teck H. Ho
Berkeley
23
Key Results (Small Categories)

Number of parameters
Our model = 59 (two-segment models); FH model = 75-163
Comparison was made on small product categories (less than
200 parameters)

Calibration
The hit probability is 7% better than F&H model
Better in every single product category

Validation
The hit probability is 8% better than F&H model
Better in every single product category
January 2004
Teck H. Ho
Berkeley
24
Key Results (Small Categories) - Calibration
Table 2: Calibration and Validation Results for the Small Product Categories
Egg
Fabric
Bathroom
Softener
Tissue
Cola
Bacon
Paper
Towel
Hotdog
Calibration
Sample Size
6252
4417
9303
9241
2383
7768
2577
Log-likelihood
Our Model
Fader and Hardie
Guadagni and Little
Empirical Frequency
-5414
-5699
-5978
-8691
-2600
-3074
-3039
-15504
-11287
-13196
-30384
-10592
-11911
-34861
-3272
-3523
-3892
-6000
-8845
-9407
-25104
-3635
-3927
-8502
Average Hit Probability
Our Model
Fader and Hardie
Guadagni and Little
Empirical Frequency
0.55
0.53
0.53
0.33
0.83
0.82
0.81
0.03
0.51
0.45
0.06
0.60
0.55
0.03
0.37
0.32
0.27
0.12
0.56
0.52
0.05
0.47
0.44
0.06
Adjusted  2
Our Model
Fader and Hardie
Guadagni and Little
0.37
0.34
0.30
0.83
0.80
0.80
0.63
0.56
-
0.69
0.65
-
0.44
0.39
0.33
0.65
0.62
-
0.57
0.52
-
January 2004
Teck H. Ho
Berkeley
25
Key Results (Small Categories) - Validation
Validation
Sample Size
2494
2137
3510
3495
842
2889
927
Log-likelihood
Our Model
Fader and Hardie
Guadagni and Little
Empirical Frequency
-2262
-2486
-2518
-3781
-1484
-1814
-1650
-7867
-4357
-5346
-12108
-3910
-4527
-12463
-1201
-1283
-1521
-2461
-3194
-3467
-11781
-1445
-1556
-3089
Average Hit Probability
Our Model
Fader and Hardie
Guadagni and Little
Empirical Frequency
0.56
0.53
0.55
0.30
0.81
0.80
0.79
0.03
0.50
0.42
0.05
0.58
0.53
0.03
0.39
0.33
0.26
0.11
0.57
0.54
0.04
0.46
0.42
0.06
Adjusted 2
Our Model
Fader and Hardie
Guadagni and Little
0.39
0.32
0.31
0.80
0.76
0.77
0.64
0.55
-
0.68
0.63
-
0.49
0.43
0.33
0.72
0.69
-
0.51
0.44
-
Note 1: FH's fabric softener has 4 attributes, hence the number of parameters are adjusted accordingly.
In addition, they initialized with 52 weeks of data which we also adhere to here.
January 2004
Teck H. Ho
Berkeley
26
Key Results (Large Categories)
Table 3: Calibration and Validation Results for the Large Product Categories
Potato
Chip
Yogurt
Spaghetti
Sauce
Soap
Regular
Cereal
Ice Cream
Frozen
Pizza
4395
7949
2701
3197
1892
4596
8262
4351
3396
-5485
-17868
-6930
-36341
-3076
-10601
-3605
-13007
-1762
-7636
-4287
-20008
-8998
-36301
-2854
-19318
-2496
-14915
Average Hit Probability
Our Model
Empirical Frequency
0.57
0.03
0.74
0.01
0.60
0.03
0.65
0.02
0.70
0.02
0.71
0.02
0.68
0.02
0.77
0.02
0.76
0.01
Adjusted 2
Our Model
0.69
0.81
0.70
0.72
0.76
0.78
0.75
0.85
0.83
Sample Size
1698
3189
1085
1268
792
1677
3040
1623
1412
Log-likelihood
Our Model
Empirical Frequency
-1309
-7210
-2755
-15202
-1147
-4304
-1269
-5598
-693
-4067
-1036
-10015
-2686
-13265
-880
-7960
-894
-7017
Average Hit Probability
Our Model
Empirical Frequency
0.65
0.04
0.73
0.01
0.60
0.03
0.67
0.02
0.72
0.02
0.79
0.02
0.71
0.02
0.80
0.02
0.77
0.01
Adjusted 2
Our Model
0.81
0.81
0.72
0.76
0.82
0.89
0.79
0.88
0.86
Toothpaste Detergent
Calibration
Sample Size
Log-likelihood
Our Model
Empirical Frequency
Validation
January 2004
Teck H. Ho
Berkeley
27
Tests of Key Behavioral Premises
Table 4: Tests of Behavioral Premises: Log-likelihood Ratios of Nested Models
Egg
Small Product Categories
Fabric Bathroom
Softener Tissue
Cola
Bacon
Large Product Categories
Paper
Towel
Hotdog
Potato
Chip
Spaghetti
Yogurt Sauce
Soap
Regular
Frozen
Toothpaste Detergent Cereal Ice Cream Pizza
The Full Model (Log-likelihood)
-5414
-2600
-11287
-10592
-3272
-8845
-3635
-5485
-6930
-3076
-3605
-1762
-4287
-8998
-2854
-2496
Behavioral Premise 1
No Attribute-level Reinforcement
246*
722*
2370*
936*
568*
1281*
405*
3014*
414*
2110*
1479*
1932*
695*
581*
728*
2598*
No Product-level Reinforcement
1080*
176*
3868*
3385*
568*
3699*
1013*
4523*
694*
3400*
846*
2196*
618*
2215*
293*
2748*
Behavioral Premise 2
No Consumption Experience
306*
70*
737*
326*
237*
1487*
370*
3226*
1299*
1646*
234*
2202*
309*
230*
44*
533*
No Shopping Experience
775*
1027*
2566*
1725*
556*
1751*
449*
5134*
724*
3288*
1801*
2168*
2093*
8038*
1680*
1621*
Behavioral Premise 3
No Familiarity
765*
549*
3636*
1498*
1623*
3075*
719*
1763*
696*
2259*
3469*
2074*
955*
4813*
293*
645*
Note 1: * indicates significance at 1%.
January 2004
Teck H. Ho
Berkeley
28
Conclusion

Our model
Simple but fits and predicts better
 Neither aggregates choice nor discards data
Shows both product and attribute-level experiences matter
 Shows consumers accumulate both shopping and
consumption experiences

IRI has implemented this model at a leading consumer
packaged goods firm
January 2004
Teck H. Ho
Berkeley
29