A research and education initiative at the MIT
Sloan School of Management
Does Providing Competitive Information to Your Own
Customers Increase Sales?
Paper 272
April 2010
Guilherme Liberali
Glen L. Urban
John Hauser
For more information,
please visit our website at http://digital.mit.edu
or contact the Center directly at digital@mit.edu
or 617-253-7054
Does Providing Competitive Information to Your Own Customers
Increase Sales?
by
Guilherme Liberali
Glen L. Urban
and
John R. Hauser
April 15, 2010
Guilherme (Gui) Liberali is a Visiting Scholar at MIT Sloan School of Management, and Assistant Professor of Marketing, Erasmus School of Economics, Erasmus University Rotterdam,
3000 DR Rotterdam, The Netherlands, (1.781) 632-7674, fax (+31)10-408-9169, liberali@mit.edu.
Glen L. Urban is the David Austin Professor of Marketing, MIT Sloan School of Management,
Massachusetts Institute of Technology, E40-159, 1 Amherst Street, Cambridge, MA 02142,
(617) 253-6615, glurban@mit.edu.
John R. Hauser is the Kirin Professor of Marketing, MIT Sloan School of Management, Massachusetts Institute of Technology, E40-179, 1 Amherst Street, Cambridge, MA 02142, (617) 2532929, hauser@mit.edu.
This research was supported by the MIT Sloan School of Management, the Center for Digital
Business at MIT (ebusiness.mit.edu), and General Motors, Inc. We gratefully acknowledge the
contributions of our industrial collaborators, research assistants, and faculty colleagues: Eric
Bradlow (Wharton), Michael Braun (MIT), Luis Felipe Camargo (UNISINOS), Daria Dryabura
(MIT), Patricia Hawkins (GM), Daniel Roesch (GM), Dmitriy A Rogozhnikov (IBM), Catherine
Tucker (MIT), JuanJuan Zhang (MIT).
Does Providing Competitive Information to Your Own Customers
Increase Sales?
Abstract
We examine data from a field experiment and a quasi-experiment in which a major US
automaker provided competitive information to its own customers. The automaker believed that
it was offering excellent vehicles but that consumers would not consider its vehicles because of
past experiences. Competitive information took many forms. An Auto Show in Motion made it
easy for consumers to test drive competitive vehicles from Chrysler, Ford, General Motors, Honda, Mercedes, Toyota and other manufacturers. eBrochures provided brochures on over 100
competitive vehicles. Auto Choice Advisor provided unbiased purchase recommendations. And a
community forum encouraged consumers to discuss buying and owning experiences.
In year 1 the automaker provided four treatments to consumers in a 2 x 2 x 2 x 2 field experiment. In year 2, to simulate a national launch, the automaker used an opt-in quasi-experiment
with a control cell, a forced-exposure cell, and a pure opt-in cell. In both years data were collected monthly for six months. We model consumer transitions to consideration and purchase using a (hidden) continuous-time Markov process which accounts for potential misclassifications
of consumers’ behavioral states. The analyses suggest that both competitive test drives and competitive brochures are effective strategies for increasing consideration and purchase. The treatments are mediated through trust. Neither the online advisor nor the community forum were effective in generating trust, consideration, or sales.
Keywords:
Automobile Applications, Competitive Information, Continuous-time Markov
Processes, Communications, Electronic Marketing, Hidden States, Information
Search, Misclassification, Quasi-experiments, Trust
1
Competitive Information to Your Own Customers
INTRODUCTION AND MOTIVATION
Information is everywhere on the Internet. If a firm does not provide information to its
customers, someone else will. This is particularly true in the automotive market. Websites such
as Autotrader.com, Cars.com, ConsumerReports.org, Edmunds.com, Kelly Blue Book
(kbb.com), and TheAutoChannel.com compete to provide specifications, reviews, prices, and
availabilities for most makes and models. Automakers often do not control this information. On
the other hand, one of the most comprehensive sources of consumer information, test drives, are
offered only by manufacturer-affiliated dealers.
Suppose an automaker believes that its vehicles are more reliable and satisfy consumer
needs significantly better than key competitors and has evidence that consumers do not share this
belief. Suppose that by drawing on experiences with this automaker in the last 10-20 years consumers reject the automaker’s vehicles before searching for information on potential purchases.
If consumers never search for information and never test drive an automaker’s vehicles, then
those consumers will never buy the automaker’s vehicles. Its sales will suffer. Unfortunately, this
scenario described key US automakers in the first decade of the 21st century and was a contributing factor in their bankruptcies. For example, despite the facts that Buick was tied with Lexus
for the top spot in J. D. Power’s 2007 vehicle dependability ranking, was the top US brand in
2008 Consumer Reports, and was the number one brand in China, roughly half of all US consumers (and almost 2/3rds in California) would not even consider a Buick.
In this paper we analyze an experiment and a quasi-experiment by a US automaker
(USAM) to provide competitive information to consumers. USAM provided consumers with the
ability to test drive 90 competitive vehicles, get unbiased competitive eBrochures, have access to
an unbiased web-based advisor that often recommended competitive vehicles, and join an online
2
Competitive Information to Your Own Customers
community forum that discussed both USAM vehicles and competitive vehicles. USAM wanted
to test whether such competitive information would increase consideration and purchase of
USAM’s vehicles.
In the first year USAM randomly assigned treatments in a 2 x 2 x 2 x 2 field experiment.
The year-1 results identify which strategies have the most potential. In the second year, USAM
used a business-driven quasi-experiment to determine whether or not the marketing actions could
be implemented nationwide. To mimic a national launch USAM encouraged consumers to opt-in
to treatments. To address opt-in self-selection, USAM used quasi-controls: a control group in
which consumers were not given the opportunities for the test drives, brochures, advisors, or
communities, and a forced-exposure group in which consumers were given strong incentives to
visit the website from which they could opt-in to such competitive information.
USAM realized that the impact of competitive information might be indirect. Competitive information might encourage consumers to consider USAM but not affect purchase conditioned on consideration. (This would still increase net purchases because the pool of consumers
who consider USAM is now larger.) Theoretically, competitive information should enhance consideration because, according to the evaluation-cost theory of consideration, consumers are more
likely to consider a product if a firm lowers the cost of evaluating that product relative to the
brands already in the consideration set (Hauser and Wernerfelt 1990). Alternatively, competitive
information might increase consideration and purchase by increasing trust in USAM even if
there is no direct effect on consideration and purchase. Trust is a common mediating variable
(Bart, et al. 2005; Urban 2004).
Modeling these indirect effects requires non-conventional analyses. First, to model indirect effects through consideration the model must account for consideration as a latent construct;
3
Competitive Information to Your Own Customers
empirical measures might have misclassified “consideration” as “not considered,” or vice versa.
Second, because automotive purchases occur over many months, USAM used a panel in which
subjects report consideration and purchase monthly for six months both year 1 and in year 2.
Third, observed “flows” from “not consider” to “consider” and to purchase (or not) may occur
faster than the interval of observation. Our model uses discrete observations from a continuous
system to infer the net effect of multiple unobserved paths to purchase.
Our primary method of analysis is a continuous-time Markov process (CTMP) with discrete-time observation and subject to misclassifications of behavioral states. We begin by describing USAM’s year-1 experimental treatments and the measurements. We present one- and
two-stage CTMP models and address the impact of misclassification analysis. We next describe
the year-2 quasi-experiment, examine potential self-selection tests, and compare analyses of the
year-2 quasi-experiment to analyses of the year-1 experiment. The results suggest that competitive test drives and competitive brochures enhance consideration and sales, although the effect is
indirectly through trust. We close with a discussion of managerial implications.
YEAR-1 EXPERIMENTS: COMPETITIVE-INFORMATION STRATEGIES
The year-1 panel ran from October 2003 to April 2004. (This was five years prior to the
bankruptcies of two US automakers.) Members of Harris Interactive’s panel were screened to be
in the market for a new vehicle in the next year, on average within the next 6.6 months, and invited to participate and complete six monthly questionnaires. In total, Harris Interactive enrolled
615 Los Angeles consumers of whom 317 completed all six questionnaires for an average completion/retention rate of 51.5%. USAM did not retain recruitment rate statistics for year 1, but,
based on year 2, we estimate an initial recruitment rate of about 40%. Consumers were assigned
randomly to experimental cells in the 2 x 2 x 2 x 2 field experiment.
4
Competitive Information to Your Own Customers
Competitive Test Drives: Auto Show in Motion
Consumers were invited to an event at a California test track to drive vehicles from
BMW, Chrysler, Dodge, Ford, General Motors, Honda, Lexus, Mercedes, and Toyota without
sales pressure. In returned they received coupons good at Amazon.com and a chance to win
$10,000. Figure 1a reproduces information about the test drive, called Auto Show in Motion
(ASIM). ASIM had substantial fixed costs to set up the test track, procure and maintain over 100
vehicles, assure safety, and provide staff assistance. Per consumer costs, including recruiting, incentives, and lunch, were in the $50-100 range. ASIM was available in period 4 to 39.1% of the
consumers.
Insert Figure 1 about here.
Customized Brochures
In year 1 this experimental treatment provided information about USAM rather than
competitive vehicles. (In year 2 USAM used competitive brochures.) Specifically, year-1 consumers received brochures that were targeted to their specific needs as determined by measures
prior to the experiments (Figure 1b). The brochures were mailed in either period 2 or 3 to 51.7%
of the consumers.
Competitive Online Advisor: Auto Choice Advisor
Consumers were invited to use a web-based advisor that recommended vehicles based on
a series of questions that revealed the consumers’ wants and needs. The web-based advisor,
known as the Auto Choice Advisor (ACA), was similar to the advisor described in Urban and
Hauser (2004). See Figure 1c. ACA was available in periods 2 through 6 to 49.2% of the consumers.
5
Competitive Information to Your Own Customers
Competitive Community Forum
Consumers participated in an CommuniSpaceTM online forum that discussed both USAM
and competitive vehicles. See Figure 1d. Consumers were free to participate in any of the over
30 dialogues which averaged over 30 comments in each. Consumers could discuss experiences
with their buying or ownership experience for any competitor and do so in an unbiased manner.
The community was available in periods 2 through 6 to 47.6% of the consumers.
Trust Covariate
Consumers have varied experiences in the auto market and varied perceptions of USAM.
USAM believed that it was possible that competitive information (and brochures) would affect
consideration and purchase by increasing consumers’ trust in USAM. Trust measures also enable
us to control for past history and to test for indirect effects. USAM measured consumers’ trust
using a five-item scale with items such as “I believe that this company is willing to assist and
support me.” or “Overall, this company has the ability to meet customer needs.” The items exhibited high reliability: Cronbach’s α = 0.95.
Measures Used in the Analyses
Dependent measures. In each period consumers reported the vehicles that they were
“considering for your next purchase or lease.” They indicated the make-model combinations
from a drop-down menu of 348 make-model combinations. USAM was interested in whether or
not one of these vehicles was a USAM vehicle. With 348 make-model combinations on a dropdown menu, misclassification was a real concern. The purchase dependent variable was based on
purchasing records maintained by USAM. For modeling purposes, we assume that misclassification of the purchase observation is small compared to misclassification of consideration.
Covariates. Many automotive consumers are brand loyal. The first of six surveys col-
6
Competitive Information to Your Own Customers
lected data on the vehicles consumers owned prior to the experiment. We dummy-code these data as “own USAM,” “own other American vehicle,” and “own Japanese.” The dummy variables
are relative to “own European.” USAM measured age with 11 categories, which we dummycode with 10 categorical variables. The covariates do not vary by observation period.
Experimentally-assigned treatments versus treatments that were received. Some
consumers could not access the treatment(s) to which they were assigned. For example, some
consumers experienced technical difficulties with ACA and some could not come to ASIM. Fortunately, the data contain self-reported treatment effects which we use to improve precision on
which consumers actually received treatments. The self-reports match up well with the assigned
treatments and seem to capture phenomena where consumers were not able to access the treatment. We ran analyses with dummy-coded treatment variables and with dummy-coded selfreports. Because they were similar and provided the same strategic interpretations, we report only the analyses based on the dummy-coded self-reported treatment effects. We now describe the
model and use it to analyze the year-1 experiment and the year-2 quasi-experiment.
LATENT BEHAVIORAL STATES AND CONTINUOUS TRANSITIONS
USAM is interested in whether or not competitive information encourages consumers to
consider and/or purchase USAM vehicles. We represent this focus with the Markov diagram in
Figure 2. The diagram is Markov because flows among behavioral states depend only on the current behavioral state (and the explanatory variables), not the entire past history of flows.
Insert Figure 2 about here.
We index continuous time with t and we index behavioral states with i where i = 1, 2, and
3 index “do not consider USAM,” “consider USAM,” and “purchase USAM,” respectively. Let
1 if the consumer is in state i at time t, and let
0 otherwise. (To simplify notation
7
Competitive Information to Your Own Customers
we suppress the subscript for consumer.) While consumers flow continuously among states and
may make multiple transitions in a given month, we only observe behavioral states at monthly
intervals. Let
be the observation time for the
monthly observation. Because we only ob-
serve the result of continuous transitions, we derive an expression for the probability,
,
that the consumer was in state i at
be
and in state j at
, where
. Let
’s.
the transition matrix of the
Competitive information affects the rate at which consumers flow among states. Let
be the instantaneous flow rate during the
the
’s. Mathematically for j ≠ i,
observation period and let An be the flow matrix of
Δ is the probability that the consumer flows from state i
to state j in the time period between t and t + Δ for very small Δ during the
observation pe-
riod. For small Δt the only way to be in state j at time t + Δt is to be there at time t or move there
from another state in time Δ . This property gives the following differential flow equation:
(1)
Δ
1
Δ
Δ
Following Cox and Miller (1965) and Hauser and Wisniewski (1982) we let Δt → 0 to obtain a
differential equation for the transition matrix.
(2)
The solution to the differential equation requires matrix exponentiation which is a difficult numerical problem. Equation 3 provides the solution to the differential equation and provides two
ways by which the solution can be computed. (
exp Λ
is the matrix of Eigenvectors of
and
is the matrix with the exponentiation of the Eigenvalues on the diagonal.)
8
Competitive Information to Your Own Customers
(3)
exp Λ
!
To capture the effect of competitive information, trust, and other covariates we let the
flows be a function of these variables. By definition, the off-diagonal elements of
tive, so we use logarithms. Let
servation period n. Otherwise
are posi-
1 if the consumer received experimental treatment k in ob0. Let
be the trust as measured at the beginning obser1 period. Let
vation period n, that is, as measured at the end of the
ℓ
be the ℓ covariate.
We represent flows by Equation 4 for the feasible behavioral-state transitions in Figure 2. We
seek to estimate the unknown parameters:
(4)
,
,
, and
ℓ
log
.
ℓ ℓ
ℓ
We code our dependent-variable observations such that
served in behavioral state i at
and in behavioral state j at
1 if the consumer is ob. Using Equations 3 and 4 we set
up the data likelihood. Although medical researchers have developed reversible jump Monte
Carlo Markov Chain estimation to obtain estimates of the parameters for moderately-sized
CTMP models (e.g., Suchard, Weiss and Sinsheimer 2001), matrix exponentiation presents practical problems for many empirical applications, especially when the number of observed transitions are small compared to the sample size as is the case in our data with USAM purchases. 1
We found maximum likelihood methods to be more stable for the USAM experiments and quasiexperiments (and more common in the CTMP literature). See Kulkarni (1995) for a review of
1
Pn is a stochastic matrix (rows sum to 1) which implies that the rows of An sum to zero. Thus the first Eigenvalue
of Pn is 1.0 and describes steady-state behavior. The remaining Eigenvalues are less than 1.0 and describe the dynamic behavior. If tn is too large or too small relative to the dynamic behavior of the system there will be only one
non-zero Eigenvalue. The remaining Eigenvalues will be close to zero. Slight numerical errors could make them
negative, which would imply imaginary flows when we use the logarithmic representation in Equation 3. This causes problems with a Bayesian sampler.
9
Competitive Information to Your Own Customers
computational methods to deal with matrix exponentiation. (Our one-stage models have a running time of 1-5 hours with maximum-likelihood estimation. Two-stage models are quicker.)
MODELING POTENTIAL MISCLASSIFICATION OF DEPENDENT MEASURES
Consideration is both a transient construct and one that is measured with potential error.
For example suppose a consumer is not quite sure whether he or she is seriously considering a
USAM vehicle. The consumer who is unsure might say he/she is considering a USAM vehicle in
period 2 but not say so in period 3. Even if the consumer is sure about consideration, he/she
might miss a vehicle in a drop-down menu of over 348 vehicles.2
Survey noise might cause us to infer phantom flows from “not consider USAM” to “consider USAM” and back. These phantom flows might disguise the effects of the experimental
1 if we
treatments, trust, or the covariates. To account for misclassification we define
observe that the consumer says that he or she is in state i at the end of the
observation period.
The true state,
and
, is not observed. To simplify notation let
We seek to infer the probability of correct classification, Pr
Pr
|
|
.
, and of misclassifications,
for j ≠ i.
To model misclassification we adopt Jackson, et al.’s (2003) hidden Markov model. To
model misclassification we recognize that the likelihood of a series of observed states,
,
,…,
set of true states,
, is equal to the probability that we made those observations conditioned on a
,
,…,
. If there were no restrictions, then for any observed state the
consumer could have been in any true state. For any sequence of observed states, if all transitions
were allowed, there is a non-zero probability for any of the 3x3x3x3x3x3 = 729 paths through
the true states at times
,
,…,
. In our model “purchase” is a trapping state and assumed to
2
Toward the end of the decade USAM developed improved methods to measure consideration that mimicked the
manner by which consumers choose consideration sets. See Hauser, et al. 2010.
10
Competitive Information to Your Own Customers
have negligible observation error, so the number of paths is far less, but not trivial. To form the
likelihood, we sum over all feasible true paths. For example, the likelihood that we observe “not
consider” at the end of the first observation period and “consider” at the end of the second observation period is given by Equation 5 where Pr (5)
Pr
,
,
is a prior probability.
Pr
|
Pr
Pr
|
Pr
|
Pr
|
Pr
|
Pr
|
Pr
Pr
|
Pr
|
Pr
|
Pr
|
As we expand Equation 5 to all six periods the likelihood gets complicated, but is handled easily by computer. Indeed, Jackson, et al. (2003, p. 197) provide a compact matrix notation
to sum the likelihood over all true paths. Based on the data likelihood we estimate the parameters
of the CTMP (the β’s) and the misclassification probabilities simultaneously.
TESTING THE DIRECT EFFECT OF COMPETITIVE INFORMATION
Table 1 summarizes the direct estimation. To simplify Table 1, we do not report ’s for
the covariates – none were significant at the 0.10 level. Because of the logarithmic specification,
a negative coefficient indicates that a flow rate decreased, not that the flow is negative. Misclassification was moderate; approximately 12% of the consumers were estimated to be misclassified
as “consider USAM” when they did not yet consider USAM and approximately 6% were estimated to be misclassified as “do not consider USAM” when they considered USAM.
Insert Table 1 about here.
Each of the four sets of two columns represents one of the four allowable flows in Figure
2. Table 1 indicates that there is no identifiable direct effect due to the competitive information
(the experimental treatments). However, lagged trust significantly increases flows from consideration to purchase and significantly decreases flows from “consider” to “do not consider.” To
11
Competitive Information to Your Own Customers
anticipate a two-stage model, we estimated a reduced-form CTMP model with only lagged trust
as a variable. The results, shown in the lower portion of Table 1, have similar implications: trust
is a key driver of purchase and of maintaining consideration.
COMPETITIVE INFORMATION ACTS THROUGH TRUST
To examine the indirect effects of competitive information we develop a two-stage model. For the first stage we regress trust on competitive test drives, customized brochures competitive advisors, and competitive community forums. We include lagged trust because the experimental treatments are likely to increase or decrease trust and we include covariates to account for
unobserved propensity for trusting USAM. We include dummy variables for observation periods
to account for unobserved advertising and other actions by USAM and to account for unobserved
environmental shocks. (Period 1 is a pre-measure and the period-2 dummy variable is set to zero
for identification.) The period dummy variables also account for any measurement artifact that
might boost trust (“Hawthorne” effect). The trust regression is summarized in Table 2. (To simplify exposition we suppress the categorical age variables; none were significant.)
Insert Table 2 about here.
Table 2 suggests that both competitive test drives (ASIM) and customized brochures increase consumers’ trust in USAM. Given the importance of trust in the first stage (CTMP) this is
an important finding. The impact of customized brochures is consistent with other published studies of customization (e.g., Ansari and Mela 2003; Hauser, et al. 2009). The significance of
ASIM demonstrates that competitive information is likely to maintain consideration and increase
purchases of USAM’s vehicles. We believe that such large-scale field-experiment evidence of
the positive effect of competitive information is relatively novel.
To examine the second stage of the CTMP analysis, we use estimated trust ( ̂
) from
12
Competitive Information to Your Own Customers
the first stage rather than measured trust (
) in the CTMP conditional likelihood. These two-
stage estimates are limited-information maximum-likelihood (LIML) estimates. LIML estimates
are consistent but we need to use bootstrap methods to check the standard errors for the ’s
(Berndt, et al. 1974; Efron and Tibshirani 1994; Wooldridge 2002, p. 354, 414). Table 3 reports
both the LIML estimates and the means from 1,000 bootstrap replicates.
Insert Table 3 about here.
The estimates in Table 3, both LIML and bootstrap, are similar to the reduced model in
Table 1 confirming the impact of trust on consideration and purchase. 3 The two-stage model
confirms that some competitive information (competitive test drives) increases consideration and
purchase, but that the effect is through increased trust. This important finding suggests that
USAM explore further strategies to increase trust. We discuss such strategies later in the paper.
We did not find an effect for the competitive advisor and the competitive forum. This is
intuitive for the 2004-2005 time period. Although USAM’s vehicles had improved relative to the
early 2000s, ACA was recommending other vehicles (particularly the Japanese vehicles) and
USAM did not always get positive reviews in community forums. Before we examine the impact
of competitive information in the year-2 quasi-experiments, we detour to examine the relative
value of modeling misclassification.
THE IMPACT OF MODELING MISCLASSIFICATION IN A CTMP MODEL
Many applications in various literatures have modeled misclassification including the di-
3
The LIML and reduced-form estimates are almost identical. The LIML and bootstrap coefficients of lagged trust
are not significantly different. The constants representing flows between “not consider USAM” and “consider
USAM” do differ, but offset. This difference is likely due to misclassification analysis interacting with the bootstrap
procedure. Specifically, because bootstrap randomly selects observations, many observations are repeated in a given
replicate causing that replicate to underestimate misclassification. The means of the 1,000 replicates were almost
identical to the medians suggesting that outliers were not a problem. The running time is about 250 hours for 1,000
two-stage replicates.
13
Competitive Information to Your Own Customers
agnosis of disease progression, the diagnosis of toxoplasmosis infection, the spread of HIV, the
probability of workers changing jobs, volcanic activity, and Monte Carlo studies of forecasting
error (Aspinall, et al. 2005; Bessec and Bouabdallah 2005; Chen and Sen 2007; Hausman, Abrevaya and Scott-Morton 1998; Jackson, et al. 2003; Rahme, Joseph and Gyorkos 2000; Tzavidis
and Lin 2006). We test the impact of modeling misclassification in USAM’s experiment by using 5-fold cross-validation to examine predictive ability. For each of five subsamples we estimate the two-stage CTMP model with 80% of the data and predict behavior for the remaining
20%. Accounting for misclassification improves hit rates significantly from 66.1% to 77.1% (p <
0.001). The coefficients themselves also vary. When we ignore misclassification, the constants
and lagged (estimated) trust remain significant in the same pattern as in Table 3, but we underestimate the impact of trust for maintaining consideration.
YEAR 2 QUASI-EXPERIMENTS ON COMPETITIVE INFORMATION
Buoyed with the success of the year-1 experiment USAM sought to test further the impact of competitive information. The year-2 quasi-experiment further analyzed competitive test
drives (ASIM) but expanded the competitive information treatments to include competitive brochures. USAM maintained both the competitive advisor and the community forum even though
they had no identifiable effect in year 1.
The big difference in year 2 was that USAM sought to test the ability to launch competitive-information strategies on a national basis by allowing consumers to opt-in to the strategies.
By its very nature, opt-in makes the test a quasi-experiment requiring USAM to test for potential
self-selection. USAM included two quasi-controls. Consumers were assigned randomly to one of
three cells. Consumers in the control cell received no treatments. Consumers in the forcedexposure cell were invited to an “Internet study” that included a visit to USAM’s “My Auto Ad14
Competitive Information to Your Own Customers
vocate” website at which they could opt-in to competitive information treatments. See Figure 3a.
Consumers in the pure opt-in cell received an advertisement inviting them visit the “My Auto
Advocate” website.
Insert Figure 3 about here.
The year-2 panel ran from January to June, 2005. Members of Harris Interactive’s panel
were again screened to be in the market for a new vehicle, on average within the next 2.2 years,
and invited to participate and complete six monthly questionnaires. (The year-1 sample was restricted to 12-month intenders, the year-2 sample was not.) Incentives were similar to year 1. In
total, Harris Interactive invited 6,092 Los Angeles consumers of which 1,720 completed all six
questionnaires for an average response/completion/retention rate of 28.2%. This rate was not
significantly different across the three conditions.
Competitive Test Drives: Auto Show in Motion
Consumers were invited to an event at one of three test tracks. Otherwise ASIM was similar to year 1. Opt-in respondents received 20 reward certificates for participating.
Competitive Brochures and USAM Booklets
In year 2 USAM continued to offer brochures (called eBooklets), but this time on an optin basis. To test competitive information, consumers could also download or obtain a CD with
competitive brochures. Although many competitive brochures were available on manufacturers’
websites, USAM’s single-source webpage made it more convenient for consumers to compare
vehicles. Opt-in consumers received 5 reward certificates for downloading a USAM brochure.
See Figure 3b.
15
Competitive Information to Your Own Customers
Competitive Online Advisor: My Product Advisor
USAM updated ACA to “My Product Advisor (MPA)” and made it available directly
from the “My Auto Advocate” website. Besides an improved interface, MPA had a “garage” at
which consumers could store vehicle descriptions. Like ACA, MPA was unbiased. Opt-in consumers received 5 reward certificates for using MPA. See Figure 3c.
Competitive Community Forum
The community forum was updated and integrated with the “My Auto Advocate” website. Opt-in consumers received at least 5 reward certificates for participating, but could earn up
to 20 reward certificates for active participation. See Figure 1d.
Trust, Dependent Measures, and Other Covariates
Trust, consideration, and purchase were measured as in year 1, as were prior ownership
of USAM, other American, and Japanese vehicles. Age was measured directly rather than by
categories and sex of the respondent was recorded.
EXAMINING ISSUES OF SELF-SELECTION TO TREATMENTS
It is possible that consumers who take advantage of competitive information are more
likely to be interested in purchasing an automobile in the near future, but this self-selection may
or may not translate into greater consideration for or purchase of USAM vehicles.
The Effect of Cell Assignment for Non-Participating Consumers
Many consumers in the opt-in (35.9%) and forced-exposure (45.0%) cells and all consumers (100%) in the control cell did not participate in the opt-in treatments. If self-selection impacts the dependent measures, then we should have a non-random removal of consumers from
the opt-in and forced-exposure cells. If self-selection compromises the quasi-experiment these
consumers should be less likely to consider and/or purchase USAM vehicles. To analyze this ef16
Competitive Information to Your Own Customers
fect we created dummy variables for experimental cell and estimated a CTMP model with misclassification. There are no significant effects due to cell assignment (all p’s > 0.14 for all dummies for all flows; often much higher). We obtain similar results controlling for trust.
Exposure to My Auto Advocate
We have already established that the there is no significant difference in net response rate
between those consumers who (a) were required to visit “My Auto Advocate” and completed the
six questionnaires (forced exposure) and (b) those who could opt-in to “My Auto Advocate” and
completed the six questionnaires (p > 0.17). To examine whether “My Auto Advocate” itself had
a significant effect on consideration and purchase we repeat the dummy-variable CTMP model
with misclassification, but for all consumers. The cell-assignment dummies remain insignificant
(all p’s > 0.23) with and without controlling for trust.
While these tests do not unequivocally rule out self-selection, they suggest it is reasonable to examine the potential impact of providing consumers with competitive information and to
compare the effects to the year-1 experiment.
YEAR-2 ANALYSIS OF COMPETITIVE INFORMATION
We repeated all year-1 analyses with the year-2 data. As in year 1, none of the quasiexperimental treatments had direct significant effects in the CTMP model and the constants and
coefficients of lagged trust in the reduced model were similar to those in the full model. Misclassification was slightly lower than in year 1; approximately 9% of the consumers were estimated
to be misclassified as “consider USAM” when they did not yet consider USAM and approximately 3% were estimated to be misclassified as “do not consider USAM” when they actually
considered USAM. Accounting for misclassification increased the predictive ability in a fivefold cross-validation from 74.7% to 87.1% (p < 0.001). Because the two-stage LIML and boot17
Competitive Information to Your Own Customers
strap estimates are similar to the reduced model we simplify exposition and report only the twostage estimates in Tables 4 and 5. 4
Insert Tables 4 and 5 about here.
In year 1 both lagged trust and estimated lagged trust had significant effects on consideration and purchase (e.g., p ≤ 0.03, bootstrap estimates). In year 2 both lagged trust and estimated
lagged trust also have significant effects on consideration and purchase (p ≤ 0.05, bootstrap estimates). (The bootstrap standard errors are more accurate than the LIML standard errors.) Increasing trust is clearly beneficial for USAM in year 2 as it was in year 1.
Competitive test drives still increase trust in year 2, although the effect is now only marginally significant (p = 0.10). The interesting new implication is that competitive brochures increased trust in USAM significantly in year 2 (p = 0.04). Sex and age are significant in year 2
possibly reflecting unobserved changes in USAM’s product mix or advertising. Overall, the
year-2 two-stage model is remarkably similar to the year-1 model despite changes in the experimental treatments, unobserved changes in the environment, the change from an experiment to a
quasi-experiment, and a sample that is not limited to consumers who plan to purchase in 12
months. Tables 1 through 5 suggest strongly that competitive information is an effective strategy
to increase trust (in USAM) and, through trust, to increase consideration and purchase of USAM
vehicles.
These quantitative conclusions are consistent with qualitative comments by consumers
who participated in the quasi-experiment:
4
All reduced-model and LIML estimates were not significantly different (all p’s > 0.32). Differences in the LIML
and bootstrap constants are explained in footnote 3. The mean and median bootstrap estimates are almost identical.
One coefficient of lagged trust was significant for the bootstrap estimates, but not for the LIML estimates, however,
the value of the coefficient itself is almost identical (0.468 vs. 0.474).
18
Competitive Information to Your Own Customers
My Auto Advocate: “I've learned a lot more about USAM. I didn't realize how many dif-
ferent models of cars they own. It was eye opening. I enjoyed it and I have a more positive view
of USAM than previously.”
Competitive test drives: “Please don't stop doing these events, USAM. This was the one
and only reason we purchased a USAM car over a Mustang GT or a Dodge Magnum R/T. There
was no way we would have test-driven a USAM car had it not been for ASIM. It's the best experience I can imagine for overcoming people's prejudices against USAM and selling them on your
many terrific products (like the …). I doubt I'd make the trip to different dealers (…) to drive
those models.”
Competitive Brochures: “The CD was downright entertaining! I liked how it got a lot of
info across in a concise, easy-to-follow fashion, and made the specifics I was seeking easy to
find.”
MANAGERIAL IMPLICATIONS
US automobile manufacturers face challenges as they try to improve sales and profit post
bankruptcy. Our analyses have two implications. First, trust is key. Those consumers who trust
USAM are more likely to consider and purchase USAM vehicles. Second, competitive test
drives and competitive product brochures influence consumers to avoid rejecting USAM vehicles before they gather information. The effect is predominately through increased trust in
USAM. USAM’s managers felt the results were consistent with qualitative data and sufficiently
compelling to investigate further.
Competitive test drives and competitive brochures were not rolled out nationally in large
part because of other distractions during the automotive and financial crises at the end of the first
decade of the 21st century. Competitive test drives are extremely expensive adding substantially
to the cost of selling a vehicle. Competitive brochures are more likely to be cost effective and,
with the growth of computer power and Internet use, are becoming more relevant. At minimum
USAM’s experiments and quasi-experiments provide evidence that honest competitive compari19
Competitive Information to Your Own Customers
sons are effective for firms that are disadvantaged by low consideration despite having good
products.
As the new USAM seeks to regain profitability they have implemented many strategies to
encourage consumers to actively compare USAM vehicles to competitors. In 2009 a series of
advertisements starring a well-known sports announcer made explicit comparisons to competitors featuring “surprises” such as good fuel economy. Another campaign was based on “May the
Best Car Win.” In the fall of 2009 USAM implemented a policy by which consumers could try
vehicles for 60 days and return them if they were not satisfied. USAM also ran successful miniexperiments in which USAM dealers invited customers to test drive competitive vehicles. All of
these marketing tactics sought to encourage competitive comparisons and/or increase trust in
USAM.
SUMMARY
If a firm has products that are much better than consumers perceive them to be, our results suggest that consideration and purchase increase if the firm makes competitive information
available. (If a firm has inferior products then providing competitive information to consumers is
unlikely to be profitable.) Competitive information reduces search and evaluation costs making it
more likely for consumers to add another good product to their consideration sets. The firm realizes added benefits if it is transparent that the firm is providing competitive information. “Credit” for providing information likely leads consumers to place more trust in the firm. This appears to be what happened in USAM’s experiments and quasi-experiments. Competitive information increased demand through consideration and purchase and did so by increasing trust in
USAM.
Managerial, the open question is whether providing competitive information is cost effec20
Competitive Information to Your Own Customers
tive. Paying for competitive test drives does not appear to be cost effective, but encouraging consumers to test drive both competitive and USAM vehicles does appear to be cost effective. Providing eBrochures appears to be a cost effective strategy.
To analyze the quasi-experiment we used CTMP models to address the data constraint
that transitions occur continuously while we only observe the results of these transitions at
monthly intervals. While CTMP and related models, including hidden Markov models, have
been used successfully to analyze other marketing issues, we are unaware of other applications in
marketing that address potential misclassification errors in CTMPs (Ding and Eliashberg 2008;
Eliashberg, et al. 2000; Hauser and Wisniewski 1982; Netzer, Lattin and Srinivasan 2008; Roberts, Morrison and Nelson 2004, 2005; Srinivasan and Kim 2009; Weerahandi and Moitra
1995). The analysis of misclassification proved to be a key methodological component of our
analyses.
CTMP, accounting for misclassification, is a powerful tool. Because flows were sparse in
the USAM quasi-experiment, stability issues made maximum likelihood estimation the best numeric technique. When flows are less sparse, matrix exponentiation is more stable numerically
and MCMC analysis will be computationally feasible. When there is more data per consumer
than in the USAM quasi-experiment, CTMP analyses could be extended to include heterogeneity
in flow rates among consumers. Perhaps new data will become available to develop these extensions for CTMP models which account for misclassification.
21
Competitive Information to Your Own Customers
REFERENCES
Ansari, Asim and Carl F. Mela (2003), “E-Customization,” Journal of Marketing Research, 40,
(May), 131-145.
Aspinall, W. P., R. Carniel, O. Jaquet, G. Woo, and T. Hincks (2006), “Using Hidden MultiState Markov Models with Multi-parameter Volcanic Data to Provide Empirical Evidence for Alert Level Decision-Support,” Journal of Volcanology and Geothermal Research, 153, 112-124.
Bart, Yakov, Venkatesh Shankar, Fareena Sultan, and Glen L. Urban (2005), “Are the Drivers
and Role of Online Trust the Same for All Web Sites and Consumers? A Large-Scale
Exploratory Empirical Study,” Journal of Marketing, 69, (October), 133-152.
Berndt, Ernie K., B. H. Hall, R. E. Hall, and Jerry Hausman (1974), “Estimation and Inference in
Nonlinear Structural Models,” Annals of Social Measurement, 3, 653-665.
Bessec, Marie and Othman Bouabdallah (2005), “What Causes The Forecasting Failure of Markov-Switching Models? A Monte Carlo Study,” Studies in Nonlinear Dynamics & Econometrics, 9, 2, Article 6.
Chen, Pai-Lien and Pranab K. Sen (2007), “Markov Chain Model Selection by Misclassified
Model Probabilities,” Communications in Statistics—Theory and Methods, 36, 143–153.
Cox, David R. and Hilton D. Miller (1965), The Theory of Stochastic Processes, (London, UK:
Chapman & Hall).
Ding, Min and Jehoshua Eliashberg (2008), “A Dynamic Competitive Forecasting Model In‐
corporating Dyadic Decision Making,” Management Science, 54, 4, 820‐834. Efron, Bradley and Robert J. Tibshirani (1994), An Introduction to the Bootstrap, (New York,
NY: Chapman & Hall/CRC).
22
Competitive Information to Your Own Customers
Eliashberg, Jehoshua, Jedid‐Jah Jonker, Mohanbir S. Sawhney, and Berend Wierenga (2000), Marketing Science, 19, 3, 226‐243.
Hausman, Jerry A., Jason Abrevaya and Fiona M. Scott-Morton (1998), “Misclassification of the
Dependent Variable in a Discrete-response Setting,” Journal of Econometrics, 87, 239269.
Hauser, John R., Olivier Toubia, Theodoros Evgeniou, Daria Dzyabura, and Rene Befurt (2010),
“Cognitive Simplicity and Consideration Sets,” Forthcoming Journal of Marketing Research, 47, (June).
------, Glen L. Urban, Guilherme Liberali, and Michael Braun (2009), “Website Morphing,”
Marketing Science, 28, 2, (March-April), 202-224.
------ and Birger Wernerfelt (1990), "An Evaluation Cost Model of Consideration Sets," Journal
of Consumer Research, Vol. 16, (March), 393-408.
------ and Wisniewski, Kenneth J. (1982), “Dynamic Analysis of Consumer Response to Marketing Strategies,” Management Science, 28, 5, 455-484.
Jackson, Christopher H., Linda D. Sharples, Simon G. Thompson, Stephen W. Duffy and Elisabeth Couto (2003), “Multistate Markov Models for Disease Progression with Classification Error,” The Statistician, 52, 193-209.
Kulkarni, V. (1995), Modeling and Analysis of Stochastic Systems, (London, UK: Chapman &
Hall/CRC).
Netzer, Oded, James M. Lattin and V. Srinivasan (2008),” A Hidden Markov Model of Customer
Relationship Dynamics,” Marketing Science, 27, 2, 185-204.
Rahme, Elham, Lawrence Joseph, and Theresa W. Gyorkos (2000), Applied Statistics, 49, 119128.
23
Competitive Information to Your Own Customers
Roberts, John H., Pamela D. Morrison and Charles J. Nelson (2004), “Implementing A Pre‐
launch Diffusion Model: Measurement And Management Challenges Of The Telstra Switching Study,” Marketing Science, 23, 2, 186‐191.
-----, -----, and ----- (2005), “A Prelaunch Diffusion Model for Evaluating Market Defense Strategies,” Marketing Science, 24, 1, 150-164.
Srinivasan, V. and Sang-Hoon Kim (2009), “A Conjoint‐Hazard Model of the Timing of Buy‐
ers' Upgrading to Improved Versions of High‐Technology Products,” Journal of Product Innovation Management, 26, 3, 278‐290.
Suchard, Marc A., Robert E. Weiss and Janet S. Sinsheimer (2001), “Bayesian Selection of Continuous-Time Markov Chain Evolutionary Models,” Molecular Biology Evol., 18, 6,
1001-1013.
Tzavidis, Nikos and Yan-Xia Lin (2006), “Estimating from Cross-Sectional Categorical Data
Subject to Misclassification and Double-Sampling: Moment-based, Maximum Likelihood
and Quasi-Likelihood Approaches,” Journal of Applied Mathematics and Decision
Sciences, 2006, 1-13.
Urban, Glen L. (2004), “The Emerging Era of Customer Advocacy,” MIT Sloan Management
Review, (Winter), 45, 2, 77-82.
------ and John R. Hauser (2004), “’Listening-In’ to Find and Explore New Combinations of Customer Needs,” Journal of Marketing, 68, (April), 72-87.
Weerahandi, Samaradasu and Soumyo Moitra (1995), “Using Survey Data to Predict Adoption
and Switching for Services,” Journal of Marketing Research, 32, 1, 85-96.
Wooldridge, Jeffrey M. (2002), “Econometric Analysis of Cross Section and Panel Data,” (Cambridge, MA: MIT Press).
24
TABLE 1
ONE-STAGE ANALYSIS OF COMPETITIVE INFORMATION
CTMP with Misclassification Modeled
State at Tn-1:
From “Do Not Consider USAM”
From “Consider USAM”
State at Tn:
to Consider USAM
to Do Not Consider USAM
Full Model
p value
to Purchase USAM
p value
to Purchase USAM
p value
p value
Constant
0.005 b
0.10
0.022 b
0.10
0.013 b
0.09
0.084 a
0.01
Lagged Trust
-0.043 c
0.84
0.607 c
0.42
-0.470 b
0.10
0.278 a
0.04
Competitive
Test Drives
-0.071 c
0.97
-2.138 c
0.86
-0.058 c
0.94
-0.359 c
0.50
Customized
Brochures
0.747 c
0.28
1.639 c
0.25
-4.832 c
0.60
0.189 c
0.53
Competitive
Online Advisor
-1.130 c
0.21
6.075 c
0.55
-0.639 c
0.47
-0.138 c
0.64
Competitive
Forum
0.365 c
0.57
0.288 c
0.85
0.767 c
0.35
0.185 c
0.53
Constant
0.043 a
<0.01
0.001 b
0.10
0.066 a
<0.01
0.121 a
<0.01
Lagged Trust
0.06 c
0.75
0.556 c
0.63
-0.502 a
0.01
0.255 a
0.04
Reduced Model
a
Significant at the 0.05 level (shown in bold).
b
Significant at the 0.10 level, but not 0.05 level (shown in bold italics)
TABLE 2
TRUST AS A FUNCTION OF COMPETITIVE INFORMATION (YEAR 1)
Key Variables
Effect
p value
Intercept
0.640 b
0.06
Lagged Trust
0.860 a
Competitive Test Drives
Covariates
Effect
p value
Own USAM
0.000 c
0.84
<0.01
Own American
0.021 c
0.59
0.380 a
<0.01
Own Japanese
-0.019 c
0.62
Customized Brochures
0.171 a
<0.01
Period 3
-0.220 a
<0.01
Competitive Advisor
-0.057 c
0.39
Period 4
-0.295 a
<0.01
Competitive Forum
0.045 c
0.22
Period 5
-0.127 a
<0.01
Adjusted R2
0.749 c
Period 6
-0.251 a
<0.01
a
b
Significant at the 0.05 level (shown in bold).
Significant at the 0.10 level, but not 0.05 level (shown in bold italics)
TABLE 3
SECOND-STAGE ANALYSIS BASED ON ESTIMATED LAGGED-TRUST (YEAR 1)
CTMP with Misclassification Modeled
State at Tn-1:
From “Do Not Consider USAM”
From “Consider USAM”
State at Tn:
to Consider USAM
to Do Not Consider USAM
to Purchase USAM
p value
LIML
p value
to Purchase USAM
p value
p value
Constant
0.042 a
<0.01
0.001 b
0.08
0.066 a
<0.01
0.120 a
<0.01
Lagged Trust
(estimated)
0.123 c
0.60
-0.380 c
0.65
-0.471 b
0.06
0.275 b
0.06
Bootstrap (1,000 replicates)
Constant
0.100 a
<0.01
0.001 c
0.36
0.156 a
<0.01
0.129 a
<0.01
Lagged Trust
(estimated)
0.209 c
0.13
-0.130 c
0.69
-0.325 a
0.03
0.275 a
0.01
a
Significant at the 0.05 level (shown in bold).
b
Significant at the 0.10 level, but not 0.05 level (shown in bold italics)
TABLE 4
TRUST AS A FUNCTION OF COMPETITIVE INFORMATION (YEAR 2)
Key Variables
Effect
p value
Intercept
0.902 a
<0.01
Lagged Trust
0.824 a
Competitive Test Drives
Covariates
Effect
p value
Own USAM
0.132 c
<0.01
<0.01
Own American
0.023 c
0.19
0.089 b
0.10
Own Japanese
-0.059
Competitive Brochures
0.050 b
0.04
Sex
0.037 a
0.02
USAM Booklets
0.017 c
0.52
Age
-0.088 a
0.02
Competitive Advisor
-0.020 c
0.39
Age-Sq/100
0.009 a
0.01
Competitive Forum
0.003 c
0.91
Period 3
0.038 c
0.15
Period 4
0.040 c
0.13
Period 5
0.028 c
0.30
Period 6
0.050 b
0.06
Adjusted R2
a
0.700 c
b
Significant at the 0.05 level (shown in bold).
c
<0.01
Significant at the 0.10 level, but not 0.05 level (shown in bold italics)
TABLE 5
SECOND-STAGE ANALYSIS BASED ON ESTIMATED LAGGED-TRUST (YEAR 2)
CTMP with Misclassification Modeled
State at Tn-1:
From “Do Not Consider USAM”
From “Consider USAM”
State at Tn:
to Consider USAM
to Do Not Consider USAM
to Purchase USAM
p value
LIML
p value
to Purchase USAM
p value
p value
Constant
0.025 a
<0.01
0.001 a
<0.01
0.099 a
<0.01
0.006 a
<0.01
Lagged Trust
(estimated)
0.989 a
<0.01
0.468 c
0.36
-0.425 a
<0.01
-0.018 c
0.96
Bootstrap (1,000 replicates)
Constant
0.072 a
<0.01
0.001 a
0.01
0.188 a
<0.01
0.006 a
<0.01
Lagged Trust
(estimated)
0.614 a
<0.01
0.474 a
0.05
-0.362 a
<0.01
0.060 c
0.87
a
Significant at the 0.05 level (shown in bold).
b
Significant at the 0.10 level, but not 0.05 level (shown in bold italics)
FIGURE 1
YEAR-1 COMPETITIVE-INFORMATION EXPERIMENTAL TREATMENTS
(a) Competitive Test Drive
(b) Customized Brochures
Site contained
over 60
dialogues
averaging over
60 comments.
(c) Competitive Online Advisor
(d) Competitive Community Forum
FIGURE 2
CONTINUOUS-TIIME FLOW
WS AMONG
G BEHAVIO
ORAL STAT
TES
FIGURE 3
YEAR
R-2 COMPE
ETITIVE-INFORMATIO
ON QUASI-EXPERIM
MENTAL TR
REATMENT
TS
(a
a) My Auto Advocate
A
Ho
omepage (Op
pt-in)
(b) Competitive E-Brrochures
(cc) Competitiv
ve New-Veh
hicle Advisorr
(d) Competitive Com
mmunity Foru
um