AMA ServSig Doctoral Consortium Taipei 2013
National Taiwan University, Taipei, Taiwan — July 3-4, 2013
Research Journey
Our Investigation of the
Net Promoter Score
Lerzan Aksoy
Timothy Keiningham
Associate Professor of Marketing
Global Chief Strategy Officer
FORDHAM UNIVERSITY
IPSOS LOYALTY
Lerzan Aksoy & Timothy Keiningham
The Team
Tor Wallin Andreassen
Timothy Keiningham
Professor of
Marketing
Norwegian School of Management
Global Chief Strategy
Officer
Ipsos Loyalty
Lerzan Aksoy
Bruce Cooil
Associate Professor of
Marketing
Fordham University
The Dean Samuel B. and Evelyn R.
Richmond Professor of Management
Vanderbilt University
“A Longitudinal Examination of Net Promoter and Firm
Revenue Growth,” Journal of Marketing, July 2007
2007 MSI H. Paul Root Award
“The Value of Different Customer Satisfaction and Loyalty Metrics in
Predicting Customer Retention, Recommendation, and Share-of-Wallet,”
Managing Service Quality, 17 (4), 2007
2007 Outstanding Paper (Best Paper) Award
“Linking Customer Loyalty to Growth,”
MIT Sloan Management Review, Summer, 2008
REALLY
Know the Topic
Timothy Keiningham
Net Promoter Score Featured in the
Harvard Business Review, MIT Sloan Management
Review, and the book The Ultimate Question
How to Calculate a Net Promoter Score
 Net Promoter asks one question, “Would you
recommend us to a friend or colleague?” that
customers answer on a scale of 0 to 10.
 Responses are grouped into three categories:
Detractors, Passives and Promoters.
 Subtract Detractors from Promoters and you have a
Net Promoter Score.
Likelihood to Recommend
Not At All
Likely
0
Extremely
Likely
Neutral
1
2
3
4
5
6
Detractors
Promoters
Detractors
7
8
Passives
9
10
Promoters
Net Promoter
Net Promoter Is Sold As “The Single Most Reliable
Indicator” Of Firm Growth
http://www.netpromoter.com/
Reichheld’s Analysis
Reichheld, in partnership with
Satmetrix and Bain & Company,
conducted both a micro (customerlevel) analysis and a macro (firmlevel) analysis as part of his Net
Promoter research.
Key Findings from Reichheld’s Macro-level (Firm-Level)
Analysis
“Net Promoter is the “single most reliable indicator of a
company's ability to grow.”
• Net Promoter links to firms’ relative growth rates
within their respective industries
•
•
Net Promoter leaders outgrow their competitors in most
industries by an average of 2.5 times.
A twelve-point increase in Net Promoter leads to a
doubling in a company’s rate of growth on average.
• The key finding of this research is that firms need
only track Net Promoter if the goal is firm growth.
Reichheld, Frederick F. (2003), “The One Number You Need to Grow,” Harvard Business Review, vol. 81, no. 12
(December), 46-54.
Satmetrix (2004), “The Power Behind a Single Number: Growing Your Business with Net Promoter,” Satmetrix Systems
white paper <http://www.satmetrix.com/pdfs/netpromoterWPfinal.pdf>
Two Key Findings from Reichheld’s Micro-level
(Customer-Level) Analysis
“The likelihood to recommend question proved to be
the top correlate to actual customer behavior 80% of
the time” (Satmetrix 2004).
“More explicitly, if customers reported that they were likely to
recommend a particular company to a friend or colleague, then
these same customers were also likely to actually repurchase from
the company, as well as generate new business by referring the
company via word-of-mouth.”
Recommend intention is the only variable needed to
predict customers’ loyalty behaviors.
Reichheld states, “Interestingly, creating a weighted index – based
on the responses to multiple questions and taking into account the
relative effectiveness of those questions – provided insignificant
predictive advantage.” (Reichheld 2003).
Reichheld, Frederick F. (2003), “The One Number You Need to Grow,” Harvard Business Review, vol. 81, no. 12
(December), 46-54.
Satmetrix (2004), “The Power Behind a Single Number: Growing Your Business with Net Promoter,” Satmetrix Systems
white paper <http://www.satmetrix.com/pdfs/netpromoterWPfinal.pdf>
Data Is King!!!
We Obtained Data to Replicate the Micro-Level Analysis
from a Large Market Research Firm
•
We examined data from a two-year study of
over 8,000 customers of firms in one of three
industries: retail banking, mass-merchant retail,
and Internet service providers (ISPs).
•
Individual customer ratings of common
satisfaction and loyalty metrics were monitored
over two years.
•
In the second year of the study, customers’
purchasing (retention and share-of-wallet) and
recommendation behaviors were also tracked.
Our Findings from the Micro-Level Analysis
Customers’ future loyalty behaviors are
distinct (retention, share of wallet, and
word of mouth). As a result,
no single measure adequately
predicted customers' future loyalty
behaviors
Finding Valid Firm-Level Multi-Industry Longitudinal Data
for the Macro-Level Analysis Proved Difficult
•
The American Customer Satisfaction Index
(ACSI) does not ask a recommend intention
question
•
Fortunately, we were able to obtain equivalent
data to that used by Reichheld from the
Norwegian Customer Satisfaction Barometer
(NCSB) .
•
We used 15,500+ interviews from the NCSB to
replicate the macro-level analysis
Our Findings from the Macro-Level Analysis
No single measure did an
adequate job of predicting firm
growth.
For example, Net Promoter was
only best 2 out of 19 times!
Possible Research Bias?
We examined potential factors that
could have caused our results to
differ, however, none appear
plausible.
Based upon our analysis, it is
difficult to imagine a scenario
whereby Net Promoter would be
classified as the superior metric.
Go the Extra Mile
Timothy Keiningham
How Can We Make and Apples-to-Apples Comparison?
•
Without access to the raw data used by
Reichheld, it is impossible to know with
certainty if the variables under
consideration were examined fairly.
•
There is, however, one opportunity to
compare Net Promoter to one metric that
Reichheld examined and criticizes: the
American Customer Satisfaction Index
(ACSI).
Reichheld Specifically Targets the American Customer
Satisfaction Index as Inferior in His HBR Article
Our research indicates that
satisfaction lacks a consistently
demonstrable connection to
actual customer behavior and
growth. This finding is borne out
by the short shrift that investors
give to such reports as the
American Customer Satisfaction
Index. The ACSI, published
quarterly in the Wall Street
Journal, reflects customer
satisfaction ratings of some 200
U.S. companies. In general, it is
difficult to discern a strong
correlation between high
customer satisfaction scores and
outstanding sales growth.
Reichheld Specifically Targets the American Customer
Satisfaction Index as Inferior in His Book
“Some recent evidence that there is little connection between satisfaction
scores and economic results comes from the ACSI itself, whose data used to
be published each quarter in the Wall Street Journal (under the heading of
marketing, not investing)” (The Ultimate Question, p. 85).
Reichheld States that Bain Research Shows a ZERO
Correlation between the ACSI and Growth
“A Bain team looked at the correlation between growth and
satisfaction, and found there is none.” Reichheld 2004
An R-square of 0.00
indicates absolutely no
correlation whatsoever!
The ACSI
Frederick F. Reichheld (2004), “Net Promoters,” Bain Audio Presentation, (February 24), <http://resultsbrief.bain.com/videos/0402/main.html>
An Opportunity to Compare Net Promoter and the
American Customer Satisfaction Index


Reichheld presents charts for three industries
that are also tracked by the ACSI: airlines, life
insurance, and computers.
Data showing the relationship between Net
Promoter scores and growth were
reconstructed based upon scatter-plots of the
data featured in the appendix to The Ultimate
Question (Reichheld 2006, pp. 192-194).
•
•

To insure accuracy, the tables were scanned, and the
corresponding graphics imported into a charting software
package where they were used as background images.
As a final check of the data, the R-square of the recreated
data was compared to the R-square reported by Reichheld -all were identical.
As these data sets were specifically selected
by Reichheld to demonstrate the linkage
between Net Promoter and growth, they
clearly should reveal relationships where Net
Promoter is a superior predictor of growth to
other metrics.
Comparison of Net Promoter and ACSI:
Personal Computers
American Customer Satisfaction Index (ACSI)
40%
Three year shipment growth
Three year shipment growth
Net Promoter
30%
Dell
20%
10%
0%
IBM
H-P
-10%
-20%
Compaq
0%
10%
20%
Gateway
30%
40%
50%
Net Promoters Q1 2001 - Q4 2002
R2:
60%
40%
30%
Dell
20%
10%
H-P
0%
-10%
-20%
Compaq
67
69
71
Gateway
73
75
77
ACSI 2001 - 2002
.68
R2 (ACSI only companies*): .70
* The ACSI does not track IBM
R2 (ACSI only companies*): .76
79
Comparison of Net Promoter and ACSI:
Life Insurance
American Customer Satisfaction Index (ACSI)
15%
State Farm
Northwestern
10%
Dell
5%
Primerica
Met Life
0%
-5%
-20%
NY Life
Prudential
-10%
0%
10%
20%
30%
40%
Premium Growth (1999-2003)
Premium Growth (1999-2003)
Net Promoter
15%
Northwestern
10%
Dell
5%
Met Life
0%
-5%
NY Life
Prudential
73
Net Promoters Q1 2001 - Q4 2002
R2:
75
77
79
81
ACSI 2001 - 2002
.86
R2 (ACSI only companies): .83
* The ACSI does not track Primerica and State Farm
R2 (ACSI only companies): .58
83
Comparison of Net Promoter and ACSI:
Airlines
Net Promoter
American Customer Satisfaction Index (ACSI)
10%
Southwest
5%
Am. West
Three year growth
Three year growth
10%
AMR
Alaska
Northwest
0%
Continental
-5%
Delta
TWA
-10%
-10%
US Air
5%
AMR
Northwest
0%
Continental
-5%
Delta
US Air
United
0% 10% 20% 30% 40% 50%
Net Promoters Q1 2000 - Q4 2002
R2:
Southwest
60%
-10%
58
60
United
62
64
66
68
70
ACSI 2000 - 2002
.68
R2 (ACSI only companies): .57
R2 (ACSI only companies): .70
* The ACSI does not track Alaska, Am. West, and TWA
72
74
Slow = Dead
Someone Has to Drive
the Process
Keeping the Process Moving Quickly
Will Not Make You Popular
Getting There Is Not Half the Fun?
Why do you
keep this job?
It’s killing you.
I thought
“publish or
perish” was
rhetorical.
Actual Quote from Co-Author
Regarding the Relentless Pressure to
Complete This Paper Quickly
“If our friendship survives
this paper, we will be
friends forever.”
The Reality:
If Your Topic Is Important, Someone Else Is Working On It Too!
 Leading scientific
researchers teamed up
with J.D. Power to gather
the necessary data to
investigate the ability of
Net Promoter to predict
financial performance.
 The results of this
research were published
in Marketing Research in
the Summer of 2007.
•
Our Journal of Marketing
paper was published in
July 2007.
You Cannot Please
Everyone
Important Works Often
Have Difficult Reviews
Reviewer Reaction Is Split!
The reviews of 06-263-IR, “A Longitudinal
Examination of ‘Net Promoter’ … ,” are now
in. This version was reviewed by a panel of
four knowledgeable reviewers. In addition I
read the paper carefully myself.
The reviewer reaction is split, with
Reviewers 1 and 4 recommending
acceptance, Reviewer 3 recommending
revision, and Reviewer 2 recommending
rejection.
The Entire Review from One
Reviewer
Reviewer Number: 2
This paper is interesting and clearly written.
Unfortunately it is not particularly scholarly.
Almost no literature is cited, no theories drawn upon, or
built upon. The introduction is high in fluff quotient.
Difficult to assess whether data differences (i.e.,
whether the Norwegian data are comparable to the
findings in Sweden and the U.S.) are attributable to
cultural differences or mere measurement issues.
Methodological techniques are far too narrow and
simple.
The Importance of a Strong
Editor Who Knows the Topic
I know enough about this industry to know the potential
importance of this paper in the business world, given
the surprising popularity of Reichheld’s measure
among business leaders. This paper gives proper
scrutiny to a current management fad.
…
Overall I believe that the positives of the paper, noted
clearly by the reviewers, far outweigh the negatives. …
Because of this, based on the reviewers’ evaluations
and my own reading, I am pleased to conditionally
accept your paper for publication in JM.
PROMOTE,
PROMOTE,
PROMOTE!
Getting the Word Out
We researched every article and
blog post we could find that had
written on Net Promoter.
We personally contacted every
one of these authors to let them
know the findings of our research.
Net Promoter Score Under Attack,
Research Magazine, July 2007
The Net Promoter Debate, Ad Map, May 2007
Reichheld Under Attack,
Colloquy, Summer 2007
The Only Number You Need to Know Does Not
Add Up to Much, Marketing Week, March 6 2008
One Question and Plenty of Debate,
Wall Street Journal, December 4, 2007
If Your Research
Challenges Something
Popular, Expect a
Backlash
Nobody Likes to Be Told the Emperor Has No Clothes,
Particularly Those Who Have Chosen the Emperor’s Tailor
No Regrets
Despite the backlash
(and career threats),
we would do it again…
in a heartbeat!
The Keiningham-Aksoy
Guiding Principles for
Writing & Publishing
The Keiningham-Aksoy Guiding
Principles for Writing & Publishing
1.
2.
You never know what will get accepted, so work on lots of
things simultaneously.
You cannot be an expert at everything, so
a) build great teams, and
b) do everything you can to make your teams successful.

3.
4.
Work with people you like, and who are committed to working as
a team.
Faster, faster, faster! A slow paper is an idea that will be
published first by someone else.
Get your paper published! Our personal addition is “even if
it is on a cereal box!”

Everything you write cannot be a Tier 1 paper, but if your
research was important enough for you to conduct and report,
then it needs to be in print.
Discussion & Questions
Timothy Keiningham