Bayesian Analysis
Slide 1
Previous lectures indicated that it was important for managers to assess the value of conducting
research. If the cost exceeds the value, then the manager shouldn’t order research. Typically,
managers rely on an informal process for assessing net worth; instead, I suggest that you think
in terms of a more formal procedure. I assure you that in all the consulting research I did for the
hospitality industry, not once did a client ask me to show that the value of my proposed study
exceeded the price being charged.
I recommend that you think in terms of a Bayesian approach, which is a more formal approach
to assessing worth. The point of this lecture is to alert you to what that analysis would entail.
Slide 2
As this cartoon suggests, Bayesian analysis is not a panacea for assessing the value for
worthiness of marketing research. The trick is to assess the numbers that must be plugged into
the analysis. However, despite its difficulties, I recommend a formal procedure as opposed to an
informal seat of the pants procedure.
Slide 3
The next five slides contain a basic reading on the use of Bayesian analysis for assessing the
value of information. I urge you to read these slides carefully before you proceed to the
examples.
Slides 4 to 8 (No Audio)
Slide 9
Here’s example #1, which is an example of using Bayesian analysis and research to make a
simple pricing decision. Let’s pretend we’re a marketing manager who must decide about a
pricing strategy for a new product. This slide indicates the payoff table for that pricing decision.
You’ll notice that this table contains alternatives, in terms of the marketing choices for a pricing
strategy. The states of nature, which are possible demand levels for the product, are assumed
as one of three possibilities: light demand, moderate demand, and heavy demand. The
consequences are the intersections between those strategies or alternatives and possible states
of nature. You’ll notice that the three possible levels of demand differ. The probability of light
demand, all else being equal, is 0.6 (60%); the probability of moderate demand is 0.3 (30%);
and the probability of heavy demand is 0.1 (10%). These are prior probabilities, in the sense
that no research has been done to improve these estimates and the assumption is that they are
based on historical data.
Looking at this combination of pricing strategies and possible levels of demand, we can perform
the expected value calculation that is shown at the bottom of the slide. You’ll see the expected
value, in terms of return to the company—we’ll assume millions of dollars—of choosing a
skimming pricing strategy for this new product. There’s a 60% chance that demand will be light,
in which case the company will earn $100 million; there’s a 30% chance that demand will be
moderate, in which case the company will earn $50 million; and there’s a 10% chance that
demand will be heavy, in which case the company will lose $50 million. If you add these
weighted profits and loss together, you’ll find an expected $70 million profit. In doing the same
calculation for the intermediate pricing strategy and the penetration pricing strategy, you’ll find
Page | 1
values less than the expected $70 million profit. Thus, our optimal choice before conducting any
research is the skimming pricing strategy because, on average, it will yield the highest expected
profit of $70 million dollars.
Slide 10
Now let’s assume we can purchase some marketing research prior to making a decision about a
skimming, intermediate, or penetration pricing strategy for this new product. The table indicates
the likelihood of each test market result, given that ultimately what will occur is light demand,
moderate demand, or heavy demand. You’ll notice that the numbers in each column sum to 1.0,
but they don’t sum to 1.0 across rows. That’s because once light demand has occurred, then
the probability is 1.0 (100%) that the test market result—assuming a test market has been
conducted—was either disappointing or moderate or highly successful. If demand is moderate,
there’s a 100% chance that the test market result would have been one of these three ways. If
demand is ultimately heavy, there’s also a 100% chance that the test market result would have
been one of these three ways.
Assuming demand ultimately will be light, there’s a 70% chance that a test market to predict
demand will be disappointing, a 20% chance that a test market to predict demand will be
moderately successful, and a 10% chance that a test market to predict demand will be highly
successful. In this sense, you can see that the third result—the highly successful test market—is
a very erroneous prediction of ultimate demand because demand ultimately will be light, and yet
the test market suggested that demand will be heavy. You can read down the remaining
columns and get the same sense. For another example, consider the last column. If you ran a
test market to predict what demand will be and demand ultimately will be heavy, then there’s
only a 10% chance that the test market will be disappointing; a 30% chance that the test market
will be moderately successful, and a 60% chance that the test market will be highly successful.
Slide 11
Although the previous slide contains useful information—in this case, the historical accuracy of
a test market for predicting ultimate demand—that’s just historical data and an intermediate
step. We want to know the probability of a certain level of demand occurring, given a certain test
market result. In other words, given a certain level of demand will occur, what is the probability
of each test market prediction. This slide shows how to calculate these probabilities.
Given that we commissioned a test market and it produced a certain result, what are our revised
probabilities for the ultimate level of demand: light, moderate, or heavy? This slide indicates how
you would perform that calculation. Column #1 under each test market result lists the three
different levels of demand. Column #2 and #3 list the prior probabilities that appeared in the
previous slide. You may recall, based strictly on managerial intuition and historical results from
products of this type, there’s a 0.6 or 60% chance that demand will be light, a 0.3 or 30%
chance that demand will be moderate, and a 0.1 or 10% chance that demand will be heavy.
Column #4 also reflects the previous slide about the relative accuracy of such test markets to
predict ultimate levels of demand. In this case, reading across the rows from the previous slide,
there’s a 70% chance the test market will be disappointing, a 20% chance demand will be
moderate, and a 10% chance that demand will be heavy. Notice that the three probabilities for
each type of test market result in Column #4 need not sum to 1.0; they do by coincidence for
disappointing test market result, but they do not for moderate or highly successful result. As in
the previous slide, the probabilities sum to 1.0 down a column but not across a row.
Page | 2
To calculate the joint probability, think about the odds associated with flipping a coin. The odds
of a coin coming up heads on one flip are 50%, and the odds of it coming up heads on two
consecutive flips are 25%. The way we calculated that probability was by multiplying 50% by
50%. We assumed a fair coin and a fair flipper—the heads-versus-tails odds are even and the
coin flips are independent—so we multiplied the probabilities of both outcomes to determine the
probability of those outcomes occurring jointly. In essence, Column #5 provides that calculation
for Columns #3 and #4. If the prior probability of light demand is 0.6 and the probability of a
disappointing test market result given eventual light demand is 0.7, then their joint probability is
0.6 x 0.7, or 0.42. We can perform the same calculation for the remaining eight joint
probabilities; those probabilities are 0.06 and 0.01 for the remaining disappointing test market
result, 0.12, 0.18, and 0.03 for the moderate test market result, and 0.06 for each highly
successful test market result.
Although an intermediate step, notice the probability of each test market result. The probabilities
with an asterisk to the right are 0.49, 0.33, and 0.18, which sum to 1.0 because the probability of
some test market result, assuming a test market is run, is 1.0.
The probability we really want is in Column #6; specifically, we want to know the probability of a
certain level of demand (state of nature) given that we receive a certain test market result
(rather than the probability of a certain test result given an eventual level of demand, as in
Column #4). To calculate this last probability, we need to standardize the numbers in Column
#5; in other words, force the probabilities associated with all possible levels of demand under
each test market result to sum to 1.0. We make this calculation by taking the three probabilities
grouped under each test market result in Column #5 and dividing them by the sum of those
three probabilities. For example, there’s a 49% chance of the disappointing test market result
(Z1). So take 0.42, 0.06, and 0.01 and divide each one by 0.49 to calculate those first three
probabilities in Column #6. Note that those probabilities—0.858, 0.122, and 0.020—sum to 1.0.
The results of the same calculations for a moderately successful test market result and a highly
successful test market result produced the remaining probabilities listed in Column #6.
From a managerial perspective, it’s important to recognize the degree to which each research
result revises initial estimates about probabilities for the different levels of demand. Before we
commissioned any research, we were 60% confident that demand would be light, 30% confident
that demand would be moderate, and 10% confident that demand would be heavy. Those
probabilities appear in Column #3. Now combine those initial probability assessments with the
test market results. If the result is disappointing, then all we’ve done is reinforce our initial
assessment: we’re now 85.8% rather than 60% certain that demand will be light, 12.2% rather
than 30% certain demand will be moderate, and 2% rather than 10% certain that demand will be
heavy. So, if we commission a test market given our prior beliefs and the result is disappointing,
then all we’ve done is reinforce our initial beliefs.
However, if we commission a test market and it’s highly successful—which is summarized at the
bottom right of this slide—our assessment changes markedly. We’re now clueless about the
likely level of demand; we’ve gone from 0.6—0.3—0.1 probabilities to equal (1/3rd) probabilities
for light, moderate, and heavy demand. So, if the test market is disappointing, given our prior
beliefs, then we’d probably be comfortable assuming light demand and choosing a skimming
pricing strategy. If the test market is highly successful, then we’re totally uncertain about the
likely level of demand and optimal course of action. In this latter case, we’d probably want to
conduct additional research to forecast the level of demand.
Page | 3
Slide 12
Here are those expected value calculations, with revised probabilities for the ultimate level of
demand, based on managerial intuition plus the results of the research. We can see that if the
test market results are disappointing, then we recalculate the expected value accordingly. The
expected value is even higher—only $70 million on Slide #9—at roughly $90 million for the
skimming pricing strategy. If the test market is disappointing, then a skimming pricing strategy
remains the optimal decision. However, the expected value calculation indicates that the optimal
strategy shifts if the test market results is either moderately or highly successful; instead, the
optimal decision is the intermediate pricing strategy. We can see that the research doesn’t
contribute to a change in the decision if the test market result is disappointing, but if the test
market result is either moderately or highly successful, then the manager is advised to modify
the chosen pricing strategy; rather than selecting a skimming pricing strategy, an intermediate
pricing strategy should be selected.
Slide 13
My point with the first example was merely to demonstrate how one would use expected values
and research to revise marketing decisions. I never mentioned the value of the research. That
particular dimension is added to this second Williams Company example. Before I begin, please
note the following. First, Williams Company manufactures soft drinks and is considering running
a special promotion that would cost $100,000, but it hasn’t had any meaningful experience with
running such promotions and has no idea whether or not that $100,000 will be wasted or will be
spent successfully. The Williams Company manager has no clear idea about the likely
consumer response to this promotion. The manager’s confusion is reflected in the probabilities
of S1, S2, and S3 in the middle of the slide; he or she is 30% confident that the promotion will
cause a 10% or more increase in market share, is 40% confident that the promotion will cause a
5-10% increase in market share, and is 30% confident that the promotion will represent totally
wasted money because it won’t increase sales. This is about as uncertain as you can get about
the efficacy of a $100,000 promotional campaign. So what is this manager to do given this high
level of uncertainty? As stated in the last paragraph on the slide, Williams Company could run a
marketing research study that would cost $25,000 and include some copy testing. So, the
manager’s decision seems straightforward: either (1) proceed to spend $100,000 on the
promotion, which has a 30% chance of gaining no customers and costing $100,000, or (2)
spend $25,000 on research first—a seemingly reasonable amount—to gain a better sense
about the likelihood of a $100,000 loss. As a first step in deciding whether (2) is the best
approach, the manager needs a sense of these copy tests’ historical accuracy in forecasting
consumers’ response to such promotions.
Slide 14
This table is very similar to the one from the previous example, which related the different states
of nature to the likelihood of different research outcomes. Again, you’ll notice that the numbers
in each column sum to 1.0 but the numbers across rows do not. For example, an extremely
favorable consumer reaction to the research company’s tests means there’s a 70% chance that
market share will increase by 10% or more, a 30% chance that market share will increase by 510%, and a 0% chance that market share won’t increase. In contrast, the last column shows
that if the research company runs its copy test and the result is unfavorable, then there’s a 0%
that market share will increase by 10% or more (so that big $400,000 pay day will not occur), a
20% chance that market share will increase by 5-10% (thus increasing profits a net $100,000),
and an 80% chance that market share won’t increase (thus costing the $100,000 associated
with the promotion).
Page | 4
Slide 15
The first step in solving this problem formally is setting up the payoff table. That table appears at
the bottom of this slide. At the top you can see the alternatives (A1 and A2) and states of nature
(S1, S2, and S3) in this case. Here, A1 stands for deciding not to run the $100,000 promotion,
and A2 stands for deciding to run the $100,000 promotion. S1, S2, and S3 correspond with a
market share increase of 10% or more (S1), a market share increase of 5-10% (S2), and a
market share increase of 0% (S3). Given that you now understand what these symbols mean,
you can see that the one trick in setting up this payoff table is setting up a not running the
promotion row. In other words, you must recognize that the expected value of doing nothing is
nothing ($0); hence, the three $0 in that row. In contrast, if the manager decides to run the
promotion, as indicated on the first page of the problem, there’s a 30% chance—based on
intuition or minimal prior experiences—Williams Company will net $400,000 additional profit, a
40% chance Williams Company will net $100,000 additional profit, and a 30% chance Williams
Company will lose $100,000 by running a promotion that fails to increase market share.
Slide 16
Similar to Slide #11 from the first example, here are the calculations for the revised probabilities
if Williams Company receives certain copy testing results from Surveys Unlimited. The Z1
probabilities in Column #2 indicate a relative uncertain manager who believes in a 30% chance
market share will increase more than 10%, a 40% chance market share will increase 5-10%,
and a 30% chance market share won’t increase at all. If Williams Company receives a strongly
positive copy testing result from Surveys Unlimited, that 30%—40%—30% distribution is
replaced by a 72.4% chance market share will increase by more than 10%, a 27.6% chance
market share will increase between 5-10%, and a 0% chance market share will not increase,
which means there’s a 0% chance that Williams Company would lose its $100,000 investment.
Now look at the range of probabilities for the moderately positive and slightly positive copy
testing results. Let’s start with the slightly positive ones. If Surveys Unlimited’s results are only
slightly positive, then there’s a 0% chance market share will increase by more than 10%, a 25%
chance the market shares will increase 5-10%, and a 75% chance market share will not
increase. Consider the general accuracy of Surveys Unlimited’s copy testing studies for
predicting changes of demand. Column #3 shows that there’s a 0% chance Surveys Unlimited
research will predict a 0% growth in market share if its copy tests are strongly positive. Looking
further down Column #3, we can see that there’s a 0% chance market share increases by more
than 10% if Surveys Unlimited’s test results are only slightly positive. These accuracy numbers
indicate a 0% chance of extremely inaccuracy in forecasts based on Surveys Unlimited’s copy
testing results. Even the likelihood of a moderately inaccurate result is minimal; for example,
there’s only a 20% chance of either a strong or slightly positive test result and subsequent a
market share increase of 5-10%. So, Surveys Unlimited’s research seems highly predictive and
cost only $25,000. Furthermore, this slide suggests that when the test results are either highly
positive or slightly positive, the manager goes from high uncertainty to relatively high certainty.
For example, when the test result is only slightly positive, the manager probabilities shift from
30%—40%—30% to 0%—25%—75% for the three levels of market share increases. This
increased certainty strongly suggests that the Surveys Unlimited research is worth purchasing.
However, the remaining analysis suggests otherwise.
Slide 17
Here are the expected value calculations. The optimal decision for this marketing manager, in
the absence of conducting any research, is to run the promotion. The expected value of doing
Page | 5
nothing is nothing ($0), but the expected value of running the promotion is $400,000 x 0.3 +
$100,000 x 0.4 +(-$100,000) x 0.3. All these numbers come from the payoff table. As the sum of
those three sets of numbers is $130,000, the expected value of running the promotion is
positive, which suggests that the manager should run it.
The next thing to determine is what the manager would do if he or she knew, with absolute
certainty, the result to running the promotion. Again, this is the tricky part. Knowing what will
happen doesn’t mean a specific outcome is guaranteed to happen; it merely means certainty
about what the future will bear. The expected value under certainty indicates a 30% chance that
market share will increase more than 10% and Williams Company will gain $400,000. There’s a
40% chance that market share will increase between 5-10% and Williams Company will gain
$100,000, and there’s a 30% chance market share won’t increase. If the manager knew in
advance that the promotion wouldn’t boost market share, then he or she wouldn’t run it. The
expected value of the optimal decision for this manager, knowing with certainty about an
increase or lack of increase in market share, is $160,000. The value of perfect information,
EV(PI), is the difference between the expected value under certainty—knowing the future
without doubt—and the expected value with additional information. The difference between
$160,000 and $130,000 is the maximum value of Surveys Unlimited’s research. Because
Surveys Unlimited is charging $25,000 for research that could be worth as much as $30,000, it’s
possible the manager should buy it, but that possibility depends on how close Surveys
Unlimited’s research findings are to perfect information.
Slide 18
To make that final calculation for how close to $30,000 the Surveys Unlimited research is worth,
the following calculation is needed. Let’s assume that the Surveys Unlimited result for its copy
test was extremely positive. Again, the expected value of doing nothing is nothing ($0). The
expected value of doing the promotion with an extremely positive Surveys Unlimited result is
$317,200; as that is far greater than $0, running the promotion is the optimal course of action. If
the result of Surveys Unlimited’s copy test is only moderately positive, then the expected value
of running the promotion is $138,500; again, far more than $0, so the optimal decision is to run
the promotion. Finally, if Surveys Unlimited runs its copy test and the result is only slightly
positive, the expected value of running the promotion is -$50,000, and that’s far less than $0. In
this case, the optimal decision is to avoid the promotion altogether.
Slide 19
Putting together the last four slides, here’s the question that requires an answer: How likely is
each research result and associated optimal alternative, and therefore in the gain from Surveys
Unlimited’s research? The expected value of the research is the probability of a certain research
result times the expected value of the optimal act, given that you got that result. That’s the
calculation you see in the center of the page. There’s a 29% chance that Surveys Unlimited will
get a highly positive result, a 39% chance that Surveys Unlimited will get a moderately positive
result, and a 32% chance that Surveys Unlimited will get a slightly positive result. The expected
value for the optimal act in each those cases is $317,200, $138,500, and $0, respectively. All
these numbers come from the previous slides. If you calculate this weighted sum, you get
$146,000. From an earlier slide, we discovered that the expected value of doing no research
was $130,000. The difference is only $16,000. At most, Surveys Unlimited research is worth
$16,000 to this marketing manager and to Williams Company and yet, Surveys Unlimited is
charging $25,000. Therefore, it’s in the best interest of Williams Company not to retain Surveys
Unlimited for copy testing.
Page | 6
Slide 20
Finally, here’s a way to test that you understood everything to this point. Here’s a third example:
the Toys for All case. I suggest you try to answer the four questions on the last slide, which are
related to this case. You’ll find that you need to perform an analysis similar to the one I just
showed for Williams Company; the only difference is the numbers that you plug into the analysis
will change. If you’re interested, try this analysis and try to answer the four questions on the last
slide; if you do, send me your answers and I’ll give you some feedback as to whether or not
those answers are correct.
Slide 21 to 22 (No Audio)
Page | 7