www.kdd.uncc.edu
CCI, UNC-Charlotte
Hierarchical Recommender System for Improving NPS.
presented by
Zbigniew W. Ras
Research sponsored by:
CONSULTING COMPANY in Charlotte
Client 1
Client 2
Client 3
shops
shops
shops
What we have to do
Client 4
Build
Recommender System
for each client
(34 clients) helping to
increase its revenue
Build Personalized
Recommender System
for each shop helping to
increase its revenue
Services
Parts
(heavy equipment repair)
CUSTOMERS
NPS of a company is correlated with its revenue growth
Net Promoter Score (NPS) –
today’s standard for measuring customer loyalty
Promoter - {9,10}
Passive – {8,7}
Detractor – {1,2,3…,6}
NPS rating for all shops
What is our goal?
Build recommender system giving possibly the simplest
and the best advice to every client/shop on the changes
they need to make in order to improve their NPS ratings.
Initial Dataset – provided by Consulting Company
About 100,000 records representing answers to a questionnaire
collected from over 35,000 randomly chosen customers in 2010 -2013.
Customers use services provided by 34 clients
A questionnaire consists of:
Information about the customer
Customer’s name, location, phone number…
Information about the service
Client’s name, invoice amount, service type…
Information on customers’ feeling about the service
was the job completed correctly
are you satisfied with the job
likelihood to refer to friends
…
Personal
Benchmarks
BenchmarkAllOverallSatisfaction
35
BenchmarkAllLikelihoodtobeRepeatCustomer
34
BenchmarkAllDealerCommunication
33
BenchmarkServiceRepairCompletedCorrectly
32
BenchmarkReferralBehavior
31
BenchmarkServiceFinalInvoiceMatchedExpectations
31
BenchmarkEaseofContact
30
BenchmarkAllDoesCustomerhaveFutureNeeds
28
BenchmarkServiceTechPromisedinExpectedTimeframe
26
BenchmarkServiceRepairCompletedWhenPromised
26
BenchmarkServiceTimelinessofInvoice
25
BenchmarkServiceAppointmentAvailability
24
BenchmarkServiceTechEquippedtodoJob
23
BenchmarkAllContactStatusofFutureNeeds
22
BenchmarkServiceTechArrivedWhenPromised
21
BenchmarkAllHasIssueBeenResolved
19
BenchmarkAllContactStatusofIssue
17
BenchmarkServiceTechnicianCommunication
6
BenchmarkServiceContactPreference
3
BenchmarkCBCallansweredpromptly
1
BenchmarkServiceReceivedQuoteforRepair
1
BenchmarkCBAutoattendantansweredbycorrectdepartment
1
BenchmarkServiceCallAnsweredQuickly
1
BenchmarkAllMarketingPermission
1
Randomly chosen customers are
asked to complete Questionnaire –
It has questions concerning
personal data + 30 benchmarks
To compute NPS we calculate average score
of all benchmarks for all customers. Knowing the
number of promoters and detractors we know NPS.
Classical Approach
Data
Data Preprocessing (including reduction of benchmarks)
Knowledge Extraction
(Classical Tools - WEKA +
Our Software for Extracting Action Rules & Their Triggers)
Recommender Systems
Classification
Selection of Best Classification Algorithm
0.4
0.405
0.41
0.415
0.42
0.425
0.43
0.435
Time Taken(secs)
Accuracy
PART
RBF
BayesNet
NaiveBayes
KNN
RandomForest
J48
0
10
20
30
40
50
60
70
80
PART
RBF
BayesNet
NaiveBayes
KNN
RandomForest
J48
Accuracy
2
3
4
1
5
7
6
Time Taken
2
4
6
7
3
1
5
Total score
4
7
10
8
8
8
11
9
Two Options
Initial Clients Datasets
2
1
Recommender Systems
local approach
global approach
Using Information about 34 Clients to Enlarge these Datasets
2
global approach
More Powerful Recommender Systems
FIRST APPROACH for Dataset Enlargement
Semantic Distance between Clients
J48 classifier extracted from
dataset D1 of Client-1
J48 classifier extracted from
Dataset for Client-2
dataset D2 of Client-2
Dataset for Client-1
attr1
attr2
Vector describing customer 1
Vector describing customer 1
attr2
attr3
attr4
attr3
attr2
Vector describing customer 2
Vector describe ing customer 3
attr1
attr3
attr4
attr3
attr1
Vector describing customer 2
Vector describe ing customer 3
More similar these two classification trees, more close semantically the clients are
Semantic distance based dendrogram
for Service
Recommender System Engine based on Semantic Similarity of Clients
Start with n1.
If NPS(n1) has to be improved, “move to n3”
If NPS(n2) > NPS(n1), then
Table3 := Table1  Table 2 is assigned to n3.
Classifier from Table3 is extracted and its
F-score is computed.
If F-score3 > F-score1,
then “move to n5”, otherwise STOP
n6
Table 5, Classifier5
n5
n7
n3
Table3
Classifier3
F-score3
n1
n2
\
NPS(n1)
Table1
Classifier1
F-score1
NPS(n2)
Table2
Classifier2
F-score2
For each client (node) we need to identify the best ancestor to be used for action rules discovery
QUESTIONS?